Digital Image Processing - CREATIS

7 downloads 446 Views 3MB Size Report
Electrical Engineering department - DIP - Olivier Bernard. 3. Image analysis. Gonzales : Digital Image Processing, Prentice Hall. (3th Ed.) ▫ Books ...
[email protected]

Digital Image Processing Image Analysis: Part 1

IMESI - 2010

Summary I. II. III. IV.

Introduction Digital image fundamentals Discrete 2D processing Image improvement

V. Image analysis

(6 houres)

Electrical Engineering department - DIP - Olivier Bernard

2

Image analysis  Books Gonzales : Digital Image Processing, Prentice Hall (3th Ed.)

Electrical Engineering department - DIP - Olivier Bernard

3

Image analysis  Image analysis: what for ?

Electrical Engineering department - DIP - Olivier Bernard

4

Image analysis  Image analysis: what for ? Recognition Tracking Detection Prediction Decision …

Electrical Engineering department - DIP - Olivier Bernard

5

Image analysis  Image analysis: what for ? Recognition

Electrical Engineering department - DIP - Olivier Bernard

6

Image analysis  Image analysis: what for ? Recognition

Electrical Engineering department - DIP - Olivier Bernard

7

Image analysis  Image analysis: what for ? Recognition

Electrical Engineering department - DIP - Olivier Bernard

8

Image analysis  Image analysis: what for ? Recognition

link Electrical Engineering department - DIP - Olivier Bernard

9

Image analysis  Image analysis: what for ? Tracking

link

Electrical Engineering department - DIP - Olivier Bernard

10

Image analysis  Image analysis: what for ? Tracking

link Electrical Engineering department - DIP - Olivier Bernard

11

Image analysis  Image analysis: what for ? Tracking

link Electrical Engineering department - DIP - Olivier Bernard

12

Image analysis  Image analysis: in which applications ?

Electrical Engineering department - DIP - Olivier Bernard

13

Image analysis  Image analysis: in which applications ? Industry (industrial vision) Medical imaging Multimedia domain - Internet (video, google map like applications) - Cell phones - Digital camera

- GPS, … Electrical Engineering department - DIP - Olivier Bernard

14

Image analysis  Image analysis: in which applications ? Medical imaging

link Electrical Engineering department - DIP - Olivier Bernard

link 15

Image analysis  Image analysis: in which applications ? Multimedia domain: digital camera

link Electrical Engineering department - DIP - Olivier Bernard

16

Image analysis Each of the presented result has been obtained from Matlab code. This code is accessible from the following web address: www.creatis.insa-lyon.fr/~bernard/AnalyseImage

Electrical Engineering department - DIP - Olivier Bernard

17

Summary V. Image Analysis 

Part 1  



Mathematical Morphology Contour detection and analysis

Part 2 



Region-based segmentation

Part 3  

Shape analysis Shape recognition Electrical Engineering department - DIP - Olivier Bernard

18

Summary V. Image Analysis 

Part 1  



Mathematical Morphology Contour detection and analysis

Part 2 



Region-based segmentation

Part 3  

Shape analysis Shape recognition Electrical Engineering department - DIP - Olivier Bernard

19

Mathematical Morphology  Context 

Definition  Principle  Properties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

20

Mathematical Morphology  Context 

Definition  Principle  Properties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

21

Mathematical Morphology Context - Definition In linear image processing, the filtering process corresponds to the removal of some frequency components of an image filtering = convolution

Initial image

Mean filter

Electrical Engineering department - DIP - Olivier Bernard

22

Mathematical Morphology Context - Definition In mathematical morphology processing, the filtering process corresponds to an image simplification (removal of some geometric structures)

Initial image

Morphological filter

Electrical Engineering department - DIP - Olivier Bernard

23

Mathematical Morphology Context - Definition Objective: put the emphasis on the morphology (shape) of an object Mathematical tools : Set theory, Closed topology space, probability Use of classical set operators (union, intersection, inclusion …)

Electrical Engineering department - DIP - Olivier Bernard

24

Mathematical Morphology  Context 

Definition  Principle  Properties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

25

Mathematical Morphology  Context - Principle Let’s define an image A to process We define a particular mask called structuring element B B will move on A such as a convolution operator

At each position of B, we compute some logical operations (essentially non linear) between A and B Electrical Engineering department - DIP - Olivier Bernard

26

Mathematical Morphology  Context - Principle Diagram illustrating mathematical morphology principle

Electrical Engineering department - DIP - Olivier Bernard

27

Mathematical Morphology  Context - Principle Some example of structuring elements

Morphological results directly depend on the size and shape of the structuring elements involved in the process Electrical Engineering department - DIP - Olivier Bernard

28

Mathematical Morphology  Context 

Definition  Principle  Properties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

29

Mathematical Morphology  Context - Properties Morphological filters simplify the image by preserving structures but, in the same time, by loosing information Growing properties

X ,Y ,

X

Y

(X )

(Y )

Morphological filters are stable and have an known invariance class

Electrical Engineering department - DIP - Olivier Bernard

30

Mathematical Morphology  Context – some operators that will be used Alternative Sequential

Original

Opening

Closing

Derivative

Electrical Engineering department - DIP - Olivier Bernard

31

Mathematical Morphology  Context 

Definition  Principle  Propreties

 Classical operators   

 

Dilation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

32

Mathematical Morphology  Binary dilation Definition Set of pixels belonging to the structuring element, when scanning the object of interest (its center staying inside the object)

Illustration Structuring element Initial object

Dilated object

Electrical Engineering department - DIP - Olivier Bernard

33

Mathematical Morphology  Corresponding matlab code % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in); % Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ]; % Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end

Electrical Engineering department - DIP - Olivier Bernard

34

Mathematical Morphology  Corresponding matlab code Load image % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in); % Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ]; % Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end

Electrical Engineering department - DIP - Olivier Bernard

in 35

Mathematical Morphology  Corresponding matlab code Structuring Element % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in);

j (or x) (-1,0)

% Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ];

(0,0)

% Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end

i (or y) (0,1) (0,-1)

Electrical Engineering department - DIP - Olivier Bernard

(1,0)

se 36

Mathematical Morphology  Corresponding matlab code Dilation % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in); % Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ];

j

i

% Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end

out (after 5 dilations)

Electrical Engineering department - DIP - Olivier Bernard

37

Mathematical Morphology  Corresponding matlab code

% Load image in = imread('leafBin.png'); in = double(in); % Structuring Element se = strel('disk',1); % Dilation out = imdilate(in,se);

in

out (After 5 dilations)

Electrical Engineering department - DIP - Olivier Bernard

38

Mathematical Morphology  Binary dilation Expressions B

(X )

B

(X )

X

B

z/ X

Bz

where Bz is a structuring element centered at z

Properties Allow to add closed pixels to an object, in the directions defined by the structuring element

Electrical Engineering department - DIP - Olivier Bernard

39

Mathematical Morphology

Initial image

Electrical Engineering department - DIP - Olivier Bernard

40

Mathematical Morphology  Gray-scale dilation Expression B

( f ) sup f ( y ) / y

Bx

where f is an image

Initiale image Electrical Engineering department - DIP - Olivier Bernard

Dilated image 41

Mathematical Morphology  Binary erosion Definition Complementary operation (dilatation) Set of pixels that do not belong to the strucuting element, when scanning the object of interest (its center staying outside the object)

Illustration Structuring element

Initial object

Erode object

Electrical Engineering department - DIP - Olivier Bernard

42

Mathematical Morphology  Binary erosion Expressions B

(X )

B

(X )

X

B

z / Bz

X

where Bz is a structuring element centered at z

Properties Allow to put off pixels that are closed to the boundaries of an object , in the directions defined by the structuring element

Electrical Engineering department - DIP - Olivier Bernard

43

Mathematical Morphology

Initial image

2 steps (erosion)

6 steps (erosion)

12 steps (erosion)

Electrical Engineering department - DIP - Olivier Bernard

44

Mathematical Morphology  Gray-scale erosion Expression B

( f ) inf f ( y ) / y

Bx

where f is an image

Initial image Electrical Engineering department - DIP - Olivier Bernard

Erode image 45

Mathematical Morphology  Dilatation, erosion Each morphological operators are derived from the combinaison of dilatations and erosions

Example: GB ( f )

B

(f)

(f)

B

(f)

B

(f)

B

(

B

( f ))

B

(f)

B

(

B

( f ))

B

(f)

B B

(f) 2f

Electrical Engineering department - DIP - Olivier Bernard

46

Mathematical Morphology  Context 

Definition  Principle  Propreties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

47

Mathematical Morphology  Gray-scale gradient GB ( f )

B

(f)

B

(f)

B

(f)

f

GB ( f ) B

B

(f)

B

(f)

(f)

Electrical Engineering department - DIP - Olivier Bernard

48

Mathematical Morphology  Gray-scale Laplacien B

(f)

B

(f)

B

(f) 2f

B

(f)

f B

B

(f)

B

(f)

B

(f) 2f

(f)

Electrical Engineering department - DIP - Olivier Bernard

49

Mathematical Morphology  Context 

Definition  Principle  Propreties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

50

Mathematical Morphology  Opening Expressions B

(f)

B

(f)

f B B

(

B

( f ))

(f

B)

B

Properties Smooth contours, delete small or irregular structures and preserve in the same time the original size of the object

Electrical Engineering department - DIP - Olivier Bernard

51

Mathematical Morphology  Closing Expressions B

(f)

B

(f)

f

B B

(

B

( f ))

(f

B)

B

Properties Delete holes, attenuate shape irregularities, connect closed object

Electrical Engineering department - DIP - Olivier Bernard

52

Mathematical Morphology Example: binary opening and closing

Electrical Engineering department - DIP - Olivier Bernard

53

Mathematical Morphology Example: gray-scale opening and closing

Electrical Engineering department - DIP - Olivier Bernard

54

Mathematical Morphology  Context 

Definition  Principle  Propreties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

55

Mathematical Morphology  Alternative sequential filters Definition Succession of opening and closing operators using structure elements with increasing size

Expressions 1

(f)

n

(f)

1

( 1 ( f ))

n

(

n

(

n 1

( f )))

1

(f)

n

(f)

1

( 1 ( f ))

n

( n(

n 1

( f )))

Properties Noise removal tool (progressively remove structures of different size: from small to big) Electrical Engineering department - DIP - Olivier Bernard

56

Mathematical Morphology

Initial image

1

2

2

5

Electrical Engineering department - DIP - Olivier Bernard

2

8 57

Mathematical Morphology 1D

Sum

Noise

Direct application of

Electrical Engineering department - DIP - Olivier Bernard

58

Mathematical Morphology  Context 

Definition  Principle  Propreties

 Classical operators   

 

Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard

59

Mathematical Morphology  Others Connected filters Remove structures by preserving contours

Initial image

Reconstructive opening

Electrical Engineering department - DIP - Olivier Bernard

60

Mathematical Morphology  Others Geodesic filters Application: labeling

Initial image

Labeling image

Electrical Engineering department - DIP - Olivier Bernard

61

Mathematical Morphology  Others Geodesic filters Application: fill holes

Initial image

Filled image

Electrical Engineering department - DIP - Olivier Bernard

62

Mathematical Morphology  Others Opening region filters Example: Text suppression (object whose size is lower than a threshold value)

Electrical Engineering department - DIP - Olivier Bernard

63

Matlab code – text suppression % Load image in = imread(‘image.jpg'); in = rgb2gray(in); % Threshold level = graythresh(in); BW = im2bw(in,level); % Regions suppression inf. to 200 pixels BW2 = bwareaopen(BW,200);

Input image

Output image

Electrical Engineering department - DIP - Olivier Bernard

64

Summary V. Image Analysis 

Part 1  



Mathematical Morphology Contour detection and analysis

Part 2 



Region segmentation

Part 3  

Shape analysis Shape recognition Electrical Engineering department - DIP - Olivier Bernard

65

Contour detection and analysis Classical methods

« snake » methods

Geodesic methods

Electrical Engineering department - DIP - Olivier Bernard

66

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

67

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

68

Contour detection and analysis Context - definition Contour detecters are techniques that reduce the amount of information in the processed image

The aim is to transform the image on a set of curves, not necessary closed, that delineate the boundaries of the object of interest

Electrical Engineering department - DIP - Olivier Bernard

69

Contour detection and analysis  Context – continuous model Let defined a continuous image

I ( x, y )

A contour is defined as a curve that hold high variations of I ( x , y )

Electrical Engineering department - DIP - Olivier Bernard

70

Contour detection and analysis  Context – continuous model In 1D, a contour corresponds to a maximum of the first derivative, that is the zero-crossing of the second derivative

Electrical Engineering department - DIP - Olivier Bernard

71

Contour detection and analysis  Context – continuous model In order to keep these properties in 2D, we have to work in the gradient direction Gradient definition I x

I 2

I

2

I

I

x

y

Electrical Engineering department - DIP - Olivier Bernard

I y

2

1/ 2

T

72

Contour detection and analysis  Context – continuous model By expressing the image and its derivatives in the local gradient reference system (g,t) (Jacobian) I g

I cos x

I sin y

I t

I sin x

I cos y

where

arctan

Electrical Engineering department - DIP - Olivier Bernard

I I

x y 73

Contour detection and analysis  Context – continuous model By replacing in the two equations by its expression, we obtain:

I g I t

I 2

0

Electrical Engineering department - DIP - Olivier Bernard

74

Contour detection and analysis  Context – continuous model Conclusion In order to keep these properties in 2D, we have to work in the gradient direction In this context, the contour is defined as the place of the maxima of the first derivative values of the image in the gradient direction Electrical Engineering department - DIP - Olivier Bernard

75

Contour detection and analysis Context – family of contour analyser Contour analyser

Classical approaches

Active contours

Snake models Electrical Engineering department - DIP - Olivier Bernard

Geodesic models 76

Contour detection and analysis Context – family of contour analyser Contour analyser

Classical approaches

Active contours

Snake models Electrical Engineering department - DIP - Olivier Bernard

Geodesic models 77

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

78

Contour detection – classical approaches  General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest

Pre-filtering

Contour detection

Contour extraction

Electrical Engineering department - DIP - Olivier Bernard

Contour closing

79

Contour detection – classical approaches  General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest

Pre-filtering

Contour detection

Contour extraction

Electrical Engineering department - DIP - Olivier Bernard

Contour closing

80

Contour detection – classical approaches  Pre-filtering The goal is to decrease the amount of noise and to keep the contour information in the same time Smoothing filter, median filter, mean-shift method,...

Initial image

Image processed by a median filtering

Electrical Engineering department - DIP - Olivier Bernard

81

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

82

Contour detection – classical approaches  General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest

Pre-filtering

Contour detection

Contour extraction

Electrical Engineering department - DIP - Olivier Bernard

Contour closing

83

Contour detection – classical approaches  Contour detection There exists three main groups Contour detection using filtering

Contour detection using mask Contour detection using zero-crossing

Electrical Engineering department - DIP - Olivier Bernard

84

Contour detection – classical approaches  Contour detection using filtering The signal filtering theory can be applied to image

Example: high-pass filter of an image using a Butterworth filter of order n

Electrical Engineering department - DIP - Olivier Bernard

85

Contour detection – classical approaches  Contour detection using filtering Example: high-pass filter of an image using a Butterworth filter of order n H PH ( f )

1

2

1

fC f

H PH ( f x , f y )

2n

1

2

2n

fC

1 fx

Transfert function in 1D

f cx

2

fy

f cy

2

Transfert function in 2D centered on (fcx,fcy)

Electrical Engineering department - DIP - Olivier Bernard

86

Contour detection – classical approaches Example: high-pass filter of an image using a Butterworth filter of order n

DFT

f[i,j]

F[u,v]

DFT-1

x multiplication in Fourier space

F’[u,v]

f’[i,j]

HPH[u,v] Electrical Engineering department - DIP - Olivier Bernard

87

Contour detection – classical approaches  Contour detection There exists three main groups Contour detection using filtering

Contour detection using mask Contour detection using zero-crossing

Electrical Engineering department - DIP - Olivier Bernard

88

Contour detection – classical approaches  Contour detection using mask Derive from local derivator estimator of discrete images Correspond to the convolution of an image with 2D filter having finite impulse response

f new [i , j ]

f [ k , l ] h[i k , j l ] k

l

Electrical Engineering department - DIP - Olivier Bernard

89

Contour detection – classical approaches Example: Sobel filter 1

0

-1

2

0

-2

1

0

-1

dx ( dx ) 2

dy 1

2

1

0

0

0

-1

-2

-1

( dy ) 2

Norm of gradient

Images of directional derivatives Electrical Engineering department - DIP - Olivier Bernard

90

Contour detection – classical approaches  Contour detection There exists three main groups Contour detection using filtering

Contour detection using mask Contour detection using zero-crossing

Electrical Engineering department - DIP - Olivier Bernard

91

Contour detection – classical approaches  Contour detection using zero crossing Use of Laplacian operator

Initial Image

Laplcaian image

Electrical Engineering department - DIP - Olivier Bernard

92

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

93

Contour detection – classical approaches  General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest

Pre-filtering

Contour detection

Contour extraction

Electrical Engineering department - DIP - Olivier Bernard

Contour closing

94

Contour detection – classical approaches  Contour extraction Two different approches will be studied Contour extraction using thresholding

Contour extraction using multiscale contour analysis

Electrical Engineering department - DIP - Olivier Bernard

95

Contour detection – classical approaches  Contour extraction from thresholding Usually the detection step gives too much information on the contour to extract The goal is to keep only the principal components of the contour detected from the previous step Used of thresholding methods

Electrical Engineering department - DIP - Olivier Bernard

96

Contour detection – classical approaches  Contour extraction from thresholding Example: norm of gradient threshold

Initial image

Norm of gradient

Norm of gradient thresholded at pixel value of 25

Norm of gradient thresholded at pixel value of 70

The result of the method highly depends on the threshold parameter Electrical Engineering department - DIP - Olivier Bernard

97

Contour detection – classical approaches  Contour extraction from thresholding Example: threshold detector with hysteresis The goal is the decrease the influence of the threshold value

Electrical Engineering department - DIP - Olivier Bernard

98

Contour detection – classical approaches Example: threshold detector with hysteresis Two threshold : one with high value th, one with low value tb We first choose pixels whose value is higher than th We then apply the threshold tb by keeping only the connected components having at least one pixel value higher than th

Electrical Engineering department - DIP - Olivier Bernard

99

Contour detection – classical approaches Example: threshold detector with hysteresis

Threshold = 25

hysteresis

Threshold = 70

Electrical Engineering department - DIP - Olivier Bernard

100

Contour detection – classical approaches  Contour extraction Two different approches will be studied Contour extraction using thresholding

Contour extraction using multiscale contour analysis

Electrical Engineering department - DIP - Olivier Bernard

101

Contour detection – classical approaches  Contour extraction using multiscale contour analysis Image

I Smoothed image

Gaussian

G

image of contours

convolution

Laplacian 2

2

Ic

convolution

Electrical Engineering department - DIP - Olivier Bernard

102

Contour detection – classical approaches  Contour extraction using multiscale contour analysis Properties used

Ic

G

n

I G

I

n

G

G x Electrical Engineering department - DIP - Olivier Bernard

2

G x2 103

Contour detection – classical approaches

scale

 Contour extraction using multiscale contour analysis

I

Ix

I

I xx

Electrical Engineering department - DIP - Olivier Bernard

I yy

I 104

Contour detection – classical approaches  Contour extraction using multiscale contour analysis

Initial image

Laplacian image

Electrical Engineering department - DIP - Olivier Bernard

Laplacian image thresholded (low value)

105

Contour detection – classical approaches  Contour extraction using multiscale contour analysis

1.0

1.5

2.0

2.5

3.5

5.0

Electrical Engineering department - DIP - Olivier Bernard

106

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

107

Contour detection – classical approaches  General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest

Pre-filtering

Contour detection

Contour extraction

Electrical Engineering department - DIP - Olivier Bernard

Contour closing

108

Contour detection – classical approaches  Contour enclosing Usually, the previous step is not sufficient to detect closed contours

Need to use an other step in order to close the detected contours

Electrical Engineering department - DIP - Olivier Bernard

109

Contour detection – classical approaches  Contour enclosing Methods based on graph search Digital images can be assimilated to a mesh or a graph where each node corresponds to a site si or a pixel (voxel) s0

Site or pixel Mesh or graph sf

Electrical Engineering department - DIP - Olivier Bernard

110

Contour detection – classical approaches  Methods based on graph search The closure step can be viewed as finding a path going through the graph nodes starting from the site s0 to the site sf (which correspond to the extremities of the curve to close) s0

Contour Solution path sf

Electrical Engineering department - DIP - Olivier Bernard

111

Contour detection – classical approaches  Methods based on graph search How can a path be selected ? Definition of an energy function that could depends on:

a data attached energy associated to each graph node the length of the contour (minimal length) the curvature of the contour (minimal curvature) The optimal path corresponds to the less expensive path according to the designed energy function Electrical Engineering department - DIP - Olivier Bernard

112

Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient Extraction of contours with one pixel width and the norm of the image gradient Extraction of « safe » contours by fixing a high level of threshold (to avoid noise detection)

From each extremities of the previous detected curve, build a path that follows the high values of the norm of the image gradient Electrical Engineering department - DIP - Olivier Bernard

113

Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient 1st step : detection of « safe » contours

Initial image

Image obtained with multi-scale contours

Electrical Engineering department - DIP - Olivier Bernard

114

Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient 2nd step: detection of extreme points

Electrical Engineering department - DIP - Olivier Bernard

115

Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient Norm of gradient

closing

Norm of gradient threshoded with value 70 Electrical Engineering department - DIP - Olivier Bernard

116

Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient

link

Electrical Engineering department - DIP - Olivier Bernard

117

Contour detection and analysis Context – family of contour analyser Contour analyser

Classical approaches

Active contours

Snake models Electrical Engineering department - DIP - Olivier Bernard

Geodesic models 118

Contour detection and analysis  Context  Definition  Continuous contour representation  Family of contour analyser

 Classical approaches    

Pre-filtering Contour detection Contour extraction Contour closing

 Snake method Electrical Engineering department - DIP - Olivier Bernard

119

Contour detection and analysis Classical methods

« snake » methods

Geodesic methods

Electrical Engineering department - DIP - Olivier Bernard

120

Contour detection and analysis Classical methods

« snake » methods

Geodesic methods

Electrical Engineering department - DIP - Olivier Bernard

121

Contour detection – Snake approach  Principle Curve evolution inside an image where the final state defined the boundaries of the object of interest

initialization

evolution

link

Electrical Engineering department - DIP - Olivier Bernard

at convergence 122

Contour detection – Snake approach  How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the acitve contour 2 - design of an energy functional whose minimum

corresponds to the boundaries of the object of interest

3 - evolution of the active contour governed by the energy minimization process

Electrical Engineering department - DIP - Olivier Bernard

123

Contour detection – Snake approach  How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the active contour 2 - design of an energy functional whose minimum

corresponds to the boundaries of the object of interest

3 - evolution of the active contour governed by the energy minimization process

Electrical Engineering department - DIP - Olivier Bernard

124

Contour detection – Snake approach Choice of the representation of the active contour Continuous representation

( s, ) :[0,1] [0, [ in

,

out

2 ( s, )

inside and outside regions of

out

in

Discrete representation The contour is sampled t 0

( )

( ),

t 1

( ),

t n 1

,

( )

t i

0 n 1

where

i

( )

xi ( ), yi ( )

t

Electrical Engineering department - DIP - Olivier Bernard

125

Contour detection – Snake approach  How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the active contour 2 - design of an energy functional whose minimum

corresponds to the boundaries of the object of interest

3 - evolution of the active contour governed by the energy minimization process

Electrical Engineering department - DIP - Olivier Bernard

126

Contour detection – Snake approach Design of an energy criterion What is an energy criterion ? Positive function (called for example E) Derivative function

Function whose minimum corresponds to the boundaries of the object of interest The minimum usually corresponds to E = 0

Electrical Engineering department - DIP - Olivier Bernard

127

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity

E (P ) ?

 u

E (P ) : energy criterion depending on position P

 u P

Electrical Engineering department - DIP - Olivier Bernard

128

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity G I (P )

 u

E (P ) : energy criterion depending on position P

 u P

Electrical Engineering department - DIP - Olivier Bernard

129

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity  (G I ( P ))

 u

E (P ) : energy criterion depending on position P

 u P

Electrical Engineering department - DIP - Olivier Bernard

130

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity 

E (P) 1

1 (G I ( P ))

 u

E (P ) : energy criterion depending on position P

 u P

Electrical Engineering department - DIP - Olivier Bernard

131

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity 

E (P) 1

1 (G I ( P ))

 u

E (P ) : energy criterion depending on position P

 u P

Electrical Engineering department - DIP - Olivier Bernard

132

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity 

E (P) 1

 u

E (P ) : energy criterion depending on position P

1 (G I ( P ))

E P

 u P

Electrical Engineering department - DIP - Olivier Bernard

133

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity 

E (P) 1

 u

E (P ) : energy criterion depending on position P

Electrical Engineering department - DIP - Olivier Bernard

1 (G I ( P ))

E P

 u P 134

Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity 

E (P) 1

 u

E (P ) : energy criterion depending on position P

Electrical Engineering department - DIP - Olivier Bernard

E P

1 (G I ( P ))

0

 u P 135

Contour detection – Snake approach Design of an energy criterion 2D classical energy function

E full

Eint ernal

Edata

Eexternal

Eint ernal

: mechanical energies applied to the contour (rigidity and elasticity of the contour)

Edata

: potential energy derived from the image which aims to stick the evolving contour to the boundaries of the object of interest

Eexternal

: energy used to deal with specific problems (ex: shape priors) Electrical Engineering department - DIP - Olivier Bernard

136

Contour detection – Snake approach Design of an energy criterion 2D classical energy function – internal part 1

Eint erne

0

(s)

(s) s

2

2

(s)

(s) s2

2

ds

- acts on the length of

- acts on the curvature of

- linked to rigidity

- linked to elasticity

Electrical Engineering department - DIP - Olivier Bernard

137

Contour detection – Snake approach Design of an energy criterion 2D classical energy function – data attached part

Eimage E

image

1 0

1 ds I ( s ) ds I ( s )) 0(G 1

1

Attract the active contour to image regions with high gradient intensity values contour detection

Electrical Engineering department - DIP - Olivier Bernard

138

Contour detection – Snake approach Design of an energy criterion 2D classical energy function – external part Introduction of high level constrains by the user For example, to place the contour in a desired region to exploit control points to define a part the contour which has already detected a region of interest

Electrical Engineering department - DIP - Olivier Bernard

139

Contour detection – Snake approach  How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the active contour 2 - design of an energy functional whose minimum

corresponds to the boundaries of the object of interest

3 - evolution of the active contour governed by the energy minimization process

Electrical Engineering department - DIP - Olivier Bernard

140

Contour detection – Snake approach Evolution of the active contour governed by the energy minimization process This step explicitly depends on the choice of the contour representation with continuous representation, use of adapted mathematical tools, such as Euler-Lagrange equation, which allow to derive an explicit contour evolution equation

with discrete representation, use of a graph model to find the next position for each contour point which would decrease the energy function Electrical Engineering department - DIP - Olivier Bernard

141

Contour detection – Snake approach Algorithm example with a discrete contour representation Compute the energy value for each point Create a point list from the lowest energy to the highest

Move the point with highest energy to a position where the corresponding energy is lower Attach this energy to the moving point and rebuild the list If the distance between two consecutive points is too high, add a point between them Electrical Engineering department - DIP - Olivier Bernard

142

Contour detection – Snake approach Example of an algorithm with a discrete contour representation Example on a simulated image

Electrical Engineering department - DIP - Olivier Bernard

143

Contour detection – Snake approach Example of an algorithm with a discrete contour representation

Difficulties in locating corners Difficulties in creating new points The number of hyper-parameters can be high

Electrical Engineering department - DIP - Olivier Bernard

144

Contour detection – Snake approach Example of an algorithm with a discrete contour representation

link

Electrical Engineering department - DIP - Olivier Bernard

145

Image analysis

End of first part

Electrical Engineering department - DIP - Olivier Bernard

146

ANNEXE 

Example of matlab code

Electrical Engineering department - DIP - Olivier Bernard

147

Image analysis - Annexe You can download all the matlab code used to generate all the presented results at the following web-link adress www.creatis.insa-lyon.fr/~bernard/AnalyseImage

Electrical Engineering department - DIP - Olivier Bernard

148

Image analysis - Annexe List of morphological functions that can be used under matlab 7 (type « morphology » in the documentation)

Electrical Engineering department - DIP - Olivier Bernard

149

Annexe – Contour detection % Reading image img = imread(‘ellipse.gif'); img = double(img(:,:,1)); % Compute the norm of the image gradient [FX,FY] = gradient(img); NG = sqrt( FX.*FX + FY.*FY ); % Display the result figure; colormap(gray); imagesc(NG); axis image;

Initial image

Norm of the image gradient

Matlab code

Electrical Engineering department - DIP - Olivier Bernard

150

Contour detection Example: gradient

% Reading image img = imread(‘ellipse.gif'); img = double(img(:,:,1)); % Compute the gradient [FX,FY] = gradient(img); xSpace=(1:2:size(img,1)); ySpace=(1:2:size(img,2)); qx=interp2(FX,xSpace, ... ySpace'); qy=interp2(FY,xSpace, ... ySpace');

Initial image

% Display image figure; quiver(xSpace, … ySpace,qx,qy); axis image;

Matlab code

Image gradient (vector)

Electrical Engineering department - DIP - Olivier Bernard

151

Annexe – Classical approaches Example: Fourier

% Reading image img = imread(‘ellipse.png'); img = double(img(:,:,1)); % Parameters F_cut = 50; cx = 128; cy = 128; ButtOrd = 2; [dimx,dimy] = size(img); % Compute the fft imf=fftshift(fft2(img)); H=zeros(dimx,dimy); for i=1:dimx for j=1:dimy d=sqrt((i-cx)^2+(j-cy)^2); H(i,j)=sqrt(1/(1+… ((F_cut/d)^(2*ButtOrd)))); end end

Initial image

% Extract contour outf = imf .* H; % Compute the inverse fft out = abs(ifft2(outf));

Electrical Engineering department - DIP - Olivier Bernard

Image reconstructed from high frequencies

152

Annexe – Classical approaches Example: Sobel filter

% Reading image img1 = imread(‘ellipse.png'); img1 = double(img1(:,:,1)); % Create sobel filter S = [ 1 0 -1; 2 0 -2; 1 0 -1 ];

% Compute derivatives img2 = conv2(img1,S,'same'); img3 = conv2(img1,S’,'same'); % Extract contour img4 = sqrt( img2.^2 + ... img3.^2 + 1e-6);

Initial image

Image of high gradient values Code matlab Electrical Engineering department - DIP - Olivier Bernard

153

Annexes – hysteresis thresholding Matlab code of the hysteresis function (4 connexities) function out = hysteresis(in1,in2)

out = in2; count = 1; k=0; MAXITERATION = 200; while ( ( count ~= 0 ) && ( k < MAXITERATION ) ) count = 0; for i=2:(size(out,1)-1) for j=2:(size(out,2)-1) if ( out(i,j) > 0 ) if ( in1(i-1,j) > 0 ) out(i-1,j) = 255; count = count + 1; end if ( in1(i+1,j) > 0 ) out(i+1,j) = 255; count = count + 1; end if ( in1(i,j-1) > 0 ) out(i,j-1) = 255; count = count + 1; end if ( in1(i,j+1) > 0 ) out(i,j+1) = 255; count = count + 1; end end end end k = k + 1; end Electrical Engineering department - DIP - Olivier Bernard

154