Electrical Engineering department - DIP - Olivier Bernard. 3. Image analysis.
Gonzales : Digital Image Processing, Prentice Hall. (3th Ed.) ▫ Books ...
[email protected]
Digital Image Processing Image Analysis: Part 1
IMESI - 2010
Summary I. II. III. IV.
Introduction Digital image fundamentals Discrete 2D processing Image improvement
V. Image analysis
(6 houres)
Electrical Engineering department - DIP - Olivier Bernard
2
Image analysis Books Gonzales : Digital Image Processing, Prentice Hall (3th Ed.)
Electrical Engineering department - DIP - Olivier Bernard
3
Image analysis Image analysis: what for ?
Electrical Engineering department - DIP - Olivier Bernard
4
Image analysis Image analysis: what for ? Recognition Tracking Detection Prediction Decision …
Electrical Engineering department - DIP - Olivier Bernard
5
Image analysis Image analysis: what for ? Recognition
Electrical Engineering department - DIP - Olivier Bernard
6
Image analysis Image analysis: what for ? Recognition
Electrical Engineering department - DIP - Olivier Bernard
7
Image analysis Image analysis: what for ? Recognition
Electrical Engineering department - DIP - Olivier Bernard
8
Image analysis Image analysis: what for ? Recognition
link Electrical Engineering department - DIP - Olivier Bernard
9
Image analysis Image analysis: what for ? Tracking
link
Electrical Engineering department - DIP - Olivier Bernard
10
Image analysis Image analysis: what for ? Tracking
link Electrical Engineering department - DIP - Olivier Bernard
11
Image analysis Image analysis: what for ? Tracking
link Electrical Engineering department - DIP - Olivier Bernard
12
Image analysis Image analysis: in which applications ?
Electrical Engineering department - DIP - Olivier Bernard
13
Image analysis Image analysis: in which applications ? Industry (industrial vision) Medical imaging Multimedia domain - Internet (video, google map like applications) - Cell phones - Digital camera
- GPS, … Electrical Engineering department - DIP - Olivier Bernard
14
Image analysis Image analysis: in which applications ? Medical imaging
link Electrical Engineering department - DIP - Olivier Bernard
link 15
Image analysis Image analysis: in which applications ? Multimedia domain: digital camera
link Electrical Engineering department - DIP - Olivier Bernard
16
Image analysis Each of the presented result has been obtained from Matlab code. This code is accessible from the following web address: www.creatis.insa-lyon.fr/~bernard/AnalyseImage
Electrical Engineering department - DIP - Olivier Bernard
17
Summary V. Image Analysis
Part 1
Mathematical Morphology Contour detection and analysis
Part 2
Region-based segmentation
Part 3
Shape analysis Shape recognition Electrical Engineering department - DIP - Olivier Bernard
18
Summary V. Image Analysis
Part 1
Mathematical Morphology Contour detection and analysis
Part 2
Region-based segmentation
Part 3
Shape analysis Shape recognition Electrical Engineering department - DIP - Olivier Bernard
19
Mathematical Morphology Context
Definition Principle Properties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
20
Mathematical Morphology Context
Definition Principle Properties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
21
Mathematical Morphology Context - Definition In linear image processing, the filtering process corresponds to the removal of some frequency components of an image filtering = convolution
Initial image
Mean filter
Electrical Engineering department - DIP - Olivier Bernard
22
Mathematical Morphology Context - Definition In mathematical morphology processing, the filtering process corresponds to an image simplification (removal of some geometric structures)
Initial image
Morphological filter
Electrical Engineering department - DIP - Olivier Bernard
23
Mathematical Morphology Context - Definition Objective: put the emphasis on the morphology (shape) of an object Mathematical tools : Set theory, Closed topology space, probability Use of classical set operators (union, intersection, inclusion …)
Electrical Engineering department - DIP - Olivier Bernard
24
Mathematical Morphology Context
Definition Principle Properties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
25
Mathematical Morphology Context - Principle Let’s define an image A to process We define a particular mask called structuring element B B will move on A such as a convolution operator
At each position of B, we compute some logical operations (essentially non linear) between A and B Electrical Engineering department - DIP - Olivier Bernard
26
Mathematical Morphology Context - Principle Diagram illustrating mathematical morphology principle
Electrical Engineering department - DIP - Olivier Bernard
27
Mathematical Morphology Context - Principle Some example of structuring elements
Morphological results directly depend on the size and shape of the structuring elements involved in the process Electrical Engineering department - DIP - Olivier Bernard
28
Mathematical Morphology Context
Definition Principle Properties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
29
Mathematical Morphology Context - Properties Morphological filters simplify the image by preserving structures but, in the same time, by loosing information Growing properties
X ,Y ,
X
Y
(X )
(Y )
Morphological filters are stable and have an known invariance class
Electrical Engineering department - DIP - Olivier Bernard
30
Mathematical Morphology Context – some operators that will be used Alternative Sequential
Original
Opening
Closing
Derivative
Electrical Engineering department - DIP - Olivier Bernard
31
Mathematical Morphology Context
Definition Principle Propreties
Classical operators
Dilation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
32
Mathematical Morphology Binary dilation Definition Set of pixels belonging to the structuring element, when scanning the object of interest (its center staying inside the object)
Illustration Structuring element Initial object
Dilated object
Electrical Engineering department - DIP - Olivier Bernard
33
Mathematical Morphology Corresponding matlab code % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in); % Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ]; % Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end
Electrical Engineering department - DIP - Olivier Bernard
34
Mathematical Morphology Corresponding matlab code Load image % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in); % Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ]; % Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end
Electrical Engineering department - DIP - Olivier Bernard
in 35
Mathematical Morphology Corresponding matlab code Structuring Element % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in);
j (or x) (-1,0)
% Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ];
(0,0)
% Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end
i (or y) (0,1) (0,-1)
Electrical Engineering department - DIP - Olivier Bernard
(1,0)
se 36
Mathematical Morphology Corresponding matlab code Dilation % Load image in = imread('leafBin.png'); in = double(in); [dimI,dimJ] = size(in); % Structuring Element se=[-1 0 0 0 1; 0 -1 0 1 0 ];
j
i
% Dilation out=zeros(dimI,dimJ); for i=2:(dimI-1) for j=2:(dimJ-1) if (in(i,j)>0) for k=1:size(se,2) out(i+se(1,k),j+se(2,k))=1; end end end end
out (after 5 dilations)
Electrical Engineering department - DIP - Olivier Bernard
37
Mathematical Morphology Corresponding matlab code
% Load image in = imread('leafBin.png'); in = double(in); % Structuring Element se = strel('disk',1); % Dilation out = imdilate(in,se);
in
out (After 5 dilations)
Electrical Engineering department - DIP - Olivier Bernard
38
Mathematical Morphology Binary dilation Expressions B
(X )
B
(X )
X
B
z/ X
Bz
where Bz is a structuring element centered at z
Properties Allow to add closed pixels to an object, in the directions defined by the structuring element
Electrical Engineering department - DIP - Olivier Bernard
39
Mathematical Morphology
Initial image
Electrical Engineering department - DIP - Olivier Bernard
40
Mathematical Morphology Gray-scale dilation Expression B
( f ) sup f ( y ) / y
Bx
where f is an image
Initiale image Electrical Engineering department - DIP - Olivier Bernard
Dilated image 41
Mathematical Morphology Binary erosion Definition Complementary operation (dilatation) Set of pixels that do not belong to the strucuting element, when scanning the object of interest (its center staying outside the object)
Illustration Structuring element
Initial object
Erode object
Electrical Engineering department - DIP - Olivier Bernard
42
Mathematical Morphology Binary erosion Expressions B
(X )
B
(X )
X
B
z / Bz
X
where Bz is a structuring element centered at z
Properties Allow to put off pixels that are closed to the boundaries of an object , in the directions defined by the structuring element
Electrical Engineering department - DIP - Olivier Bernard
43
Mathematical Morphology
Initial image
2 steps (erosion)
6 steps (erosion)
12 steps (erosion)
Electrical Engineering department - DIP - Olivier Bernard
44
Mathematical Morphology Gray-scale erosion Expression B
( f ) inf f ( y ) / y
Bx
where f is an image
Initial image Electrical Engineering department - DIP - Olivier Bernard
Erode image 45
Mathematical Morphology Dilatation, erosion Each morphological operators are derived from the combinaison of dilatations and erosions
Example: GB ( f )
B
(f)
(f)
B
(f)
B
(f)
B
(
B
( f ))
B
(f)
B
(
B
( f ))
B
(f)
B B
(f) 2f
Electrical Engineering department - DIP - Olivier Bernard
46
Mathematical Morphology Context
Definition Principle Propreties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
47
Mathematical Morphology Gray-scale gradient GB ( f )
B
(f)
B
(f)
B
(f)
f
GB ( f ) B
B
(f)
B
(f)
(f)
Electrical Engineering department - DIP - Olivier Bernard
48
Mathematical Morphology Gray-scale Laplacien B
(f)
B
(f)
B
(f) 2f
B
(f)
f B
B
(f)
B
(f)
B
(f) 2f
(f)
Electrical Engineering department - DIP - Olivier Bernard
49
Mathematical Morphology Context
Definition Principle Propreties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
50
Mathematical Morphology Opening Expressions B
(f)
B
(f)
f B B
(
B
( f ))
(f
B)
B
Properties Smooth contours, delete small or irregular structures and preserve in the same time the original size of the object
Electrical Engineering department - DIP - Olivier Bernard
51
Mathematical Morphology Closing Expressions B
(f)
B
(f)
f
B B
(
B
( f ))
(f
B)
B
Properties Delete holes, attenuate shape irregularities, connect closed object
Electrical Engineering department - DIP - Olivier Bernard
52
Mathematical Morphology Example: binary opening and closing
Electrical Engineering department - DIP - Olivier Bernard
53
Mathematical Morphology Example: gray-scale opening and closing
Electrical Engineering department - DIP - Olivier Bernard
54
Mathematical Morphology Context
Definition Principle Propreties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
55
Mathematical Morphology Alternative sequential filters Definition Succession of opening and closing operators using structure elements with increasing size
Expressions 1
(f)
n
(f)
1
( 1 ( f ))
n
(
n
(
n 1
( f )))
1
(f)
n
(f)
1
( 1 ( f ))
n
( n(
n 1
( f )))
Properties Noise removal tool (progressively remove structures of different size: from small to big) Electrical Engineering department - DIP - Olivier Bernard
56
Mathematical Morphology
Initial image
1
2
2
5
Electrical Engineering department - DIP - Olivier Bernard
2
8 57
Mathematical Morphology 1D
Sum
Noise
Direct application of
Electrical Engineering department - DIP - Olivier Bernard
58
Mathematical Morphology Context
Definition Principle Propreties
Classical operators
Dilatation / Erosion Gradient / Laplacian Opening / closing Alternative Sequential filters others … Electrical Engineering department - DIP - Olivier Bernard
59
Mathematical Morphology Others Connected filters Remove structures by preserving contours
Initial image
Reconstructive opening
Electrical Engineering department - DIP - Olivier Bernard
60
Mathematical Morphology Others Geodesic filters Application: labeling
Initial image
Labeling image
Electrical Engineering department - DIP - Olivier Bernard
61
Mathematical Morphology Others Geodesic filters Application: fill holes
Initial image
Filled image
Electrical Engineering department - DIP - Olivier Bernard
62
Mathematical Morphology Others Opening region filters Example: Text suppression (object whose size is lower than a threshold value)
Electrical Engineering department - DIP - Olivier Bernard
63
Matlab code – text suppression % Load image in = imread(‘image.jpg'); in = rgb2gray(in); % Threshold level = graythresh(in); BW = im2bw(in,level); % Regions suppression inf. to 200 pixels BW2 = bwareaopen(BW,200);
Input image
Output image
Electrical Engineering department - DIP - Olivier Bernard
64
Summary V. Image Analysis
Part 1
Mathematical Morphology Contour detection and analysis
Part 2
Region segmentation
Part 3
Shape analysis Shape recognition Electrical Engineering department - DIP - Olivier Bernard
65
Contour detection and analysis Classical methods
« snake » methods
Geodesic methods
Electrical Engineering department - DIP - Olivier Bernard
66
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
67
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
68
Contour detection and analysis Context - definition Contour detecters are techniques that reduce the amount of information in the processed image
The aim is to transform the image on a set of curves, not necessary closed, that delineate the boundaries of the object of interest
Electrical Engineering department - DIP - Olivier Bernard
69
Contour detection and analysis Context – continuous model Let defined a continuous image
I ( x, y )
A contour is defined as a curve that hold high variations of I ( x , y )
Electrical Engineering department - DIP - Olivier Bernard
70
Contour detection and analysis Context – continuous model In 1D, a contour corresponds to a maximum of the first derivative, that is the zero-crossing of the second derivative
Electrical Engineering department - DIP - Olivier Bernard
71
Contour detection and analysis Context – continuous model In order to keep these properties in 2D, we have to work in the gradient direction Gradient definition I x
I 2
I
2
I
I
x
y
Electrical Engineering department - DIP - Olivier Bernard
I y
2
1/ 2
T
72
Contour detection and analysis Context – continuous model By expressing the image and its derivatives in the local gradient reference system (g,t) (Jacobian) I g
I cos x
I sin y
I t
I sin x
I cos y
where
arctan
Electrical Engineering department - DIP - Olivier Bernard
I I
x y 73
Contour detection and analysis Context – continuous model By replacing in the two equations by its expression, we obtain:
I g I t
I 2
0
Electrical Engineering department - DIP - Olivier Bernard
74
Contour detection and analysis Context – continuous model Conclusion In order to keep these properties in 2D, we have to work in the gradient direction In this context, the contour is defined as the place of the maxima of the first derivative values of the image in the gradient direction Electrical Engineering department - DIP - Olivier Bernard
75
Contour detection and analysis Context – family of contour analyser Contour analyser
Classical approaches
Active contours
Snake models Electrical Engineering department - DIP - Olivier Bernard
Geodesic models 76
Contour detection and analysis Context – family of contour analyser Contour analyser
Classical approaches
Active contours
Snake models Electrical Engineering department - DIP - Olivier Bernard
Geodesic models 77
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
78
Contour detection – classical approaches General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest
Pre-filtering
Contour detection
Contour extraction
Electrical Engineering department - DIP - Olivier Bernard
Contour closing
79
Contour detection – classical approaches General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest
Pre-filtering
Contour detection
Contour extraction
Electrical Engineering department - DIP - Olivier Bernard
Contour closing
80
Contour detection – classical approaches Pre-filtering The goal is to decrease the amount of noise and to keep the contour information in the same time Smoothing filter, median filter, mean-shift method,...
Initial image
Image processed by a median filtering
Electrical Engineering department - DIP - Olivier Bernard
81
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
82
Contour detection – classical approaches General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest
Pre-filtering
Contour detection
Contour extraction
Electrical Engineering department - DIP - Olivier Bernard
Contour closing
83
Contour detection – classical approaches Contour detection There exists three main groups Contour detection using filtering
Contour detection using mask Contour detection using zero-crossing
Electrical Engineering department - DIP - Olivier Bernard
84
Contour detection – classical approaches Contour detection using filtering The signal filtering theory can be applied to image
Example: high-pass filter of an image using a Butterworth filter of order n
Electrical Engineering department - DIP - Olivier Bernard
85
Contour detection – classical approaches Contour detection using filtering Example: high-pass filter of an image using a Butterworth filter of order n H PH ( f )
1
2
1
fC f
H PH ( f x , f y )
2n
1
2
2n
fC
1 fx
Transfert function in 1D
f cx
2
fy
f cy
2
Transfert function in 2D centered on (fcx,fcy)
Electrical Engineering department - DIP - Olivier Bernard
86
Contour detection – classical approaches Example: high-pass filter of an image using a Butterworth filter of order n
DFT
f[i,j]
F[u,v]
DFT-1
x multiplication in Fourier space
F’[u,v]
f’[i,j]
HPH[u,v] Electrical Engineering department - DIP - Olivier Bernard
87
Contour detection – classical approaches Contour detection There exists three main groups Contour detection using filtering
Contour detection using mask Contour detection using zero-crossing
Electrical Engineering department - DIP - Olivier Bernard
88
Contour detection – classical approaches Contour detection using mask Derive from local derivator estimator of discrete images Correspond to the convolution of an image with 2D filter having finite impulse response
f new [i , j ]
f [ k , l ] h[i k , j l ] k
l
Electrical Engineering department - DIP - Olivier Bernard
89
Contour detection – classical approaches Example: Sobel filter 1
0
-1
2
0
-2
1
0
-1
dx ( dx ) 2
dy 1
2
1
0
0
0
-1
-2
-1
( dy ) 2
Norm of gradient
Images of directional derivatives Electrical Engineering department - DIP - Olivier Bernard
90
Contour detection – classical approaches Contour detection There exists three main groups Contour detection using filtering
Contour detection using mask Contour detection using zero-crossing
Electrical Engineering department - DIP - Olivier Bernard
91
Contour detection – classical approaches Contour detection using zero crossing Use of Laplacian operator
Initial Image
Laplcaian image
Electrical Engineering department - DIP - Olivier Bernard
92
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
93
Contour detection – classical approaches General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest
Pre-filtering
Contour detection
Contour extraction
Electrical Engineering department - DIP - Olivier Bernard
Contour closing
94
Contour detection – classical approaches Contour extraction Two different approches will be studied Contour extraction using thresholding
Contour extraction using multiscale contour analysis
Electrical Engineering department - DIP - Olivier Bernard
95
Contour detection – classical approaches Contour extraction from thresholding Usually the detection step gives too much information on the contour to extract The goal is to keep only the principal components of the contour detected from the previous step Used of thresholding methods
Electrical Engineering department - DIP - Olivier Bernard
96
Contour detection – classical approaches Contour extraction from thresholding Example: norm of gradient threshold
Initial image
Norm of gradient
Norm of gradient thresholded at pixel value of 25
Norm of gradient thresholded at pixel value of 70
The result of the method highly depends on the threshold parameter Electrical Engineering department - DIP - Olivier Bernard
97
Contour detection – classical approaches Contour extraction from thresholding Example: threshold detector with hysteresis The goal is the decrease the influence of the threshold value
Electrical Engineering department - DIP - Olivier Bernard
98
Contour detection – classical approaches Example: threshold detector with hysteresis Two threshold : one with high value th, one with low value tb We first choose pixels whose value is higher than th We then apply the threshold tb by keeping only the connected components having at least one pixel value higher than th
Electrical Engineering department - DIP - Olivier Bernard
99
Contour detection – classical approaches Example: threshold detector with hysteresis
Threshold = 25
hysteresis
Threshold = 70
Electrical Engineering department - DIP - Olivier Bernard
100
Contour detection – classical approaches Contour extraction Two different approches will be studied Contour extraction using thresholding
Contour extraction using multiscale contour analysis
Electrical Engineering department - DIP - Olivier Bernard
101
Contour detection – classical approaches Contour extraction using multiscale contour analysis Image
I Smoothed image
Gaussian
G
image of contours
convolution
Laplacian 2
2
Ic
convolution
Electrical Engineering department - DIP - Olivier Bernard
102
Contour detection – classical approaches Contour extraction using multiscale contour analysis Properties used
Ic
G
n
I G
I
n
G
G x Electrical Engineering department - DIP - Olivier Bernard
2
G x2 103
Contour detection – classical approaches
scale
Contour extraction using multiscale contour analysis
I
Ix
I
I xx
Electrical Engineering department - DIP - Olivier Bernard
I yy
I 104
Contour detection – classical approaches Contour extraction using multiscale contour analysis
Initial image
Laplacian image
Electrical Engineering department - DIP - Olivier Bernard
Laplacian image thresholded (low value)
105
Contour detection – classical approaches Contour extraction using multiscale contour analysis
1.0
1.5
2.0
2.5
3.5
5.0
Electrical Engineering department - DIP - Olivier Bernard
106
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
107
Contour detection – classical approaches General scheme Use a chain of image processing in order to define a set of connected points that characterize the contour of interest
Pre-filtering
Contour detection
Contour extraction
Electrical Engineering department - DIP - Olivier Bernard
Contour closing
108
Contour detection – classical approaches Contour enclosing Usually, the previous step is not sufficient to detect closed contours
Need to use an other step in order to close the detected contours
Electrical Engineering department - DIP - Olivier Bernard
109
Contour detection – classical approaches Contour enclosing Methods based on graph search Digital images can be assimilated to a mesh or a graph where each node corresponds to a site si or a pixel (voxel) s0
Site or pixel Mesh or graph sf
Electrical Engineering department - DIP - Olivier Bernard
110
Contour detection – classical approaches Methods based on graph search The closure step can be viewed as finding a path going through the graph nodes starting from the site s0 to the site sf (which correspond to the extremities of the curve to close) s0
Contour Solution path sf
Electrical Engineering department - DIP - Olivier Bernard
111
Contour detection – classical approaches Methods based on graph search How can a path be selected ? Definition of an energy function that could depends on:
a data attached energy associated to each graph node the length of the contour (minimal length) the curvature of the contour (minimal curvature) The optimal path corresponds to the less expensive path according to the designed energy function Electrical Engineering department - DIP - Olivier Bernard
112
Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient Extraction of contours with one pixel width and the norm of the image gradient Extraction of « safe » contours by fixing a high level of threshold (to avoid noise detection)
From each extremities of the previous detected curve, build a path that follows the high values of the norm of the image gradient Electrical Engineering department - DIP - Olivier Bernard
113
Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient 1st step : detection of « safe » contours
Initial image
Image obtained with multi-scale contours
Electrical Engineering department - DIP - Olivier Bernard
114
Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient 2nd step: detection of extreme points
Electrical Engineering department - DIP - Olivier Bernard
115
Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient Norm of gradient
closing
Norm of gradient threshoded with value 70 Electrical Engineering department - DIP - Olivier Bernard
116
Contour detection – classical approaches Example: contour enclosing using high values of the norm of image gradient
link
Electrical Engineering department - DIP - Olivier Bernard
117
Contour detection and analysis Context – family of contour analyser Contour analyser
Classical approaches
Active contours
Snake models Electrical Engineering department - DIP - Olivier Bernard
Geodesic models 118
Contour detection and analysis Context Definition Continuous contour representation Family of contour analyser
Classical approaches
Pre-filtering Contour detection Contour extraction Contour closing
Snake method Electrical Engineering department - DIP - Olivier Bernard
119
Contour detection and analysis Classical methods
« snake » methods
Geodesic methods
Electrical Engineering department - DIP - Olivier Bernard
120
Contour detection and analysis Classical methods
« snake » methods
Geodesic methods
Electrical Engineering department - DIP - Olivier Bernard
121
Contour detection – Snake approach Principle Curve evolution inside an image where the final state defined the boundaries of the object of interest
initialization
evolution
link
Electrical Engineering department - DIP - Olivier Bernard
at convergence 122
Contour detection – Snake approach How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the acitve contour 2 - design of an energy functional whose minimum
corresponds to the boundaries of the object of interest
3 - evolution of the active contour governed by the energy minimization process
Electrical Engineering department - DIP - Olivier Bernard
123
Contour detection – Snake approach How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the active contour 2 - design of an energy functional whose minimum
corresponds to the boundaries of the object of interest
3 - evolution of the active contour governed by the energy minimization process
Electrical Engineering department - DIP - Olivier Bernard
124
Contour detection – Snake approach Choice of the representation of the active contour Continuous representation
( s, ) :[0,1] [0, [ in
,
out
2 ( s, )
inside and outside regions of
out
in
Discrete representation The contour is sampled t 0
( )
( ),
t 1
( ),
t n 1
,
( )
t i
0 n 1
where
i
( )
xi ( ), yi ( )
t
Electrical Engineering department - DIP - Olivier Bernard
125
Contour detection – Snake approach How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the active contour 2 - design of an energy functional whose minimum
corresponds to the boundaries of the object of interest
3 - evolution of the active contour governed by the energy minimization process
Electrical Engineering department - DIP - Olivier Bernard
126
Contour detection – Snake approach Design of an energy criterion What is an energy criterion ? Positive function (called for example E) Derivative function
Function whose minimum corresponds to the boundaries of the object of interest The minimum usually corresponds to E = 0
Electrical Engineering department - DIP - Olivier Bernard
127
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity
E (P ) ?
u
E (P ) : energy criterion depending on position P
u P
Electrical Engineering department - DIP - Olivier Bernard
128
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity G I (P )
u
E (P ) : energy criterion depending on position P
u P
Electrical Engineering department - DIP - Olivier Bernard
129
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity (G I ( P ))
u
E (P ) : energy criterion depending on position P
u P
Electrical Engineering department - DIP - Olivier Bernard
130
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity
E (P) 1
1 (G I ( P ))
u
E (P ) : energy criterion depending on position P
u P
Electrical Engineering department - DIP - Olivier Bernard
131
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity
E (P) 1
1 (G I ( P ))
u
E (P ) : energy criterion depending on position P
u P
Electrical Engineering department - DIP - Olivier Bernard
132
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity
E (P) 1
u
E (P ) : energy criterion depending on position P
1 (G I ( P ))
E P
u P
Electrical Engineering department - DIP - Olivier Bernard
133
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity
E (P) 1
u
E (P ) : energy criterion depending on position P
Electrical Engineering department - DIP - Olivier Bernard
1 (G I ( P ))
E P
u P 134
Contour detection – Snake approach Design of an energy criterion Example: 1D case Pixel intensity
E (P) 1
u
E (P ) : energy criterion depending on position P
Electrical Engineering department - DIP - Olivier Bernard
E P
1 (G I ( P ))
0
u P 135
Contour detection – Snake approach Design of an energy criterion 2D classical energy function
E full
Eint ernal
Edata
Eexternal
Eint ernal
: mechanical energies applied to the contour (rigidity and elasticity of the contour)
Edata
: potential energy derived from the image which aims to stick the evolving contour to the boundaries of the object of interest
Eexternal
: energy used to deal with specific problems (ex: shape priors) Electrical Engineering department - DIP - Olivier Bernard
136
Contour detection – Snake approach Design of an energy criterion 2D classical energy function – internal part 1
Eint erne
0
(s)
(s) s
2
2
(s)
(s) s2
2
ds
- acts on the length of
- acts on the curvature of
- linked to rigidity
- linked to elasticity
Electrical Engineering department - DIP - Olivier Bernard
137
Contour detection – Snake approach Design of an energy criterion 2D classical energy function – data attached part
Eimage E
image
1 0
1 ds I ( s ) ds I ( s )) 0(G 1
1
Attract the active contour to image regions with high gradient intensity values contour detection
Electrical Engineering department - DIP - Olivier Bernard
138
Contour detection – Snake approach Design of an energy criterion 2D classical energy function – external part Introduction of high level constrains by the user For example, to place the contour in a desired region to exploit control points to define a part the contour which has already detected a region of interest
Electrical Engineering department - DIP - Olivier Bernard
139
Contour detection – Snake approach How makes the active contour evolve ? Mathematical modeling 1 - choice of the representation of the active contour 2 - design of an energy functional whose minimum
corresponds to the boundaries of the object of interest
3 - evolution of the active contour governed by the energy minimization process
Electrical Engineering department - DIP - Olivier Bernard
140
Contour detection – Snake approach Evolution of the active contour governed by the energy minimization process This step explicitly depends on the choice of the contour representation with continuous representation, use of adapted mathematical tools, such as Euler-Lagrange equation, which allow to derive an explicit contour evolution equation
with discrete representation, use of a graph model to find the next position for each contour point which would decrease the energy function Electrical Engineering department - DIP - Olivier Bernard
141
Contour detection – Snake approach Algorithm example with a discrete contour representation Compute the energy value for each point Create a point list from the lowest energy to the highest
Move the point with highest energy to a position where the corresponding energy is lower Attach this energy to the moving point and rebuild the list If the distance between two consecutive points is too high, add a point between them Electrical Engineering department - DIP - Olivier Bernard
142
Contour detection – Snake approach Example of an algorithm with a discrete contour representation Example on a simulated image
Electrical Engineering department - DIP - Olivier Bernard
143
Contour detection – Snake approach Example of an algorithm with a discrete contour representation
Difficulties in locating corners Difficulties in creating new points The number of hyper-parameters can be high
Electrical Engineering department - DIP - Olivier Bernard
144
Contour detection – Snake approach Example of an algorithm with a discrete contour representation
link
Electrical Engineering department - DIP - Olivier Bernard
145
Image analysis
End of first part
Electrical Engineering department - DIP - Olivier Bernard
146
ANNEXE
Example of matlab code
Electrical Engineering department - DIP - Olivier Bernard
147
Image analysis - Annexe You can download all the matlab code used to generate all the presented results at the following web-link adress www.creatis.insa-lyon.fr/~bernard/AnalyseImage
Electrical Engineering department - DIP - Olivier Bernard
148
Image analysis - Annexe List of morphological functions that can be used under matlab 7 (type « morphology » in the documentation)
Electrical Engineering department - DIP - Olivier Bernard
149
Annexe – Contour detection % Reading image img = imread(‘ellipse.gif'); img = double(img(:,:,1)); % Compute the norm of the image gradient [FX,FY] = gradient(img); NG = sqrt( FX.*FX + FY.*FY ); % Display the result figure; colormap(gray); imagesc(NG); axis image;
Initial image
Norm of the image gradient
Matlab code
Electrical Engineering department - DIP - Olivier Bernard
150
Contour detection Example: gradient
% Reading image img = imread(‘ellipse.gif'); img = double(img(:,:,1)); % Compute the gradient [FX,FY] = gradient(img); xSpace=(1:2:size(img,1)); ySpace=(1:2:size(img,2)); qx=interp2(FX,xSpace, ... ySpace'); qy=interp2(FY,xSpace, ... ySpace');
Initial image
% Display image figure; quiver(xSpace, … ySpace,qx,qy); axis image;
Matlab code
Image gradient (vector)
Electrical Engineering department - DIP - Olivier Bernard
151
Annexe – Classical approaches Example: Fourier
% Reading image img = imread(‘ellipse.png'); img = double(img(:,:,1)); % Parameters F_cut = 50; cx = 128; cy = 128; ButtOrd = 2; [dimx,dimy] = size(img); % Compute the fft imf=fftshift(fft2(img)); H=zeros(dimx,dimy); for i=1:dimx for j=1:dimy d=sqrt((i-cx)^2+(j-cy)^2); H(i,j)=sqrt(1/(1+… ((F_cut/d)^(2*ButtOrd)))); end end
Initial image
% Extract contour outf = imf .* H; % Compute the inverse fft out = abs(ifft2(outf));
Electrical Engineering department - DIP - Olivier Bernard
Image reconstructed from high frequencies
152
Annexe – Classical approaches Example: Sobel filter
% Reading image img1 = imread(‘ellipse.png'); img1 = double(img1(:,:,1)); % Create sobel filter S = [ 1 0 -1; 2 0 -2; 1 0 -1 ];
% Compute derivatives img2 = conv2(img1,S,'same'); img3 = conv2(img1,S’,'same'); % Extract contour img4 = sqrt( img2.^2 + ... img3.^2 + 1e-6);
Initial image
Image of high gradient values Code matlab Electrical Engineering department - DIP - Olivier Bernard
153
Annexes – hysteresis thresholding Matlab code of the hysteresis function (4 connexities) function out = hysteresis(in1,in2)
out = in2; count = 1; k=0; MAXITERATION = 200; while ( ( count ~= 0 ) && ( k < MAXITERATION ) ) count = 0; for i=2:(size(out,1)-1) for j=2:(size(out,2)-1) if ( out(i,j) > 0 ) if ( in1(i-1,j) > 0 ) out(i-1,j) = 255; count = count + 1; end if ( in1(i+1,j) > 0 ) out(i+1,j) = 255; count = count + 1; end if ( in1(i,j-1) > 0 ) out(i,j-1) = 255; count = count + 1; end if ( in1(i,j+1) > 0 ) out(i,j+1) = 255; count = count + 1; end end end end k = k + 1; end Electrical Engineering department - DIP - Olivier Bernard
154