Reliability-based regularization for image registration. c TANG ..... Measure in noiseâfree condition. 20 ..... Labels: discretize solution domain (R2 in 2D) â ti â L.
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-Driven, Spatially-Adaptive Regularization for Deformable Registration Lisa Tang1 , Ghassan Hamarneh1, and Rafeef Abugharbieh2
1 Medical Image Analysis Lab. Simon Fraser University, B.C., Canada
2
Biomedical Signal and Image Computing Lab., University of British Columbia, B.C., Canada
MIAL
M e d i c a l I ma g e A n a l y s i s L a b
July 2010 c Reliability-based regularization for image registration. TANG
1 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Image Registration and Regularization •
Find transform T that maps points in F to points in M arg min D(F , T ◦ M) + λR(T ) T | {z } | {z } Dissimilarity metric
(1)
Regularization
c Reliability-based regularization for image registration. TANG
2 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Image Registration and Regularization •
Find transform T that maps points in F to points in M arg min D(F , T ◦ M) + λR(T ) T | {z } | {z } Dissimilarity metric
•
(1)
Regularization
Scalar weight λ balances these two terms
c Reliability-based regularization for image registration. TANG
2 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Image Registration and Regularization •
Find transform T that maps points in F to points in M arg min D(F , T ◦ M) + λR(T ) T | {z } | {z } Dissimilarity metric
•
(1)
Regularization
Scalar weight λ balances these two terms
From Incorporating Rigid Structures in Non-rigid Registration using Triangular B-splines [Wang et al. VLSM 2005]
c Reliability-based regularization for image registration. TANG
2 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Adaptive Regularization
Anatomy-based Lester et al.
Fluid-based
Deformation is allowed or restricted
Kabus et al.
Diffusion-based
Constraints are relaxed near region boundaries
Rexilius et al.
Fluid-based
Inhomogeneous elasticity parameters varied
Nagel & Enkelmann
Optical-flow
Apply anisotropic diffusion smoothing on deformation field
Suàrez et al.
Block-matching
Regularization based on local structure
Stefanescu et al.
Optical-flow
Regularization based on local confidence
Data-driven
c Reliability-based regularization for image registration. TANG
3 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Adaptive Regularization
Anatomy-based Lester et al.
Fluid-based
Deformation is allowed or restricted
Kabus et al.
Diffusion-based
Constraints are relaxed near region boundaries
Rexilius et al.
Fluid-based
Inhomogeneous elasticity parameters varied
Nagel & Enkelmann
Optical-flow
Apply anisotropic diffusion smoothing on deformation field
Suàrez et al.
Block-matching
Regularization based on local structure
Stefanescu et al.
Optical-flow
Regularization based on local confidence
Data-driven
c Reliability-based regularization for image registration. TANG
3 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Adaptive Regularization
Anatomy-based Lester et al.
Fluid-based
Deformation is allowed or restricted
Kabus et al.
Diffusion-based
Constraints are relaxed near region boundaries
Rexilius et al.
Fluid-based
Inhomogeneous elasticity parameters varied
Nagel & Enkelmann
Optical-flow
Apply anisotropic diffusion smoothing on deformation field
Suàrez et al.
Block-matching
Regularization based on local structure
Stefanescu et al.
Optical-flow
Regularization based on local confidence
Data-driven
c Reliability-based regularization for image registration. TANG
3 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data-Driven Adaptive Regularization 1) Suàrez et al. Nonrigid registration using regularized matching weighted by local structure, MICCAI 2003
• •
Block-matching algorithm Regularization controlled by local structure measure
50
50
100
100
150
150 50
100
150
200
c Reliability-based regularization for image registration. TANG
50
100
150
200
4 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data-Driven Adaptive Regularization 2) Stefanescu et al. Grid powered nonlinear image registration with locally adaptive regularization, MIA 2004
• •
Optical-flow approach Part of regularization depends on local confidence measure
50
50
100
100
150
150 50
100
150
200
c Reliability-based regularization for image registration. TANG
50
100
150
200
5 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data-Driven Adaptive Regularization Local structure [Suàrez et al.] under 0.5% of white-noise corruption 2
structure(x) =
det T (x) D trace T (x)
c Reliability-based regularization for image registration. TANG
(2)
6 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data-Driven Adaptive Regularization Local structure [Suàrez et al.] under 2% of white-noise corruption 2
structure(x) =
det T (x) D trace T (x)
c Reliability-based regularization for image registration. TANG
(2)
6 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data-Driven Adaptive Regularization Local confidence [Stefanescu et al.] under 0.5% of white-noise corruption −c
confidence(x) = e I(x)γ −2 +1e−5
c Reliability-based regularization for image registration. TANG
(2)
6 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data-Driven Adaptive Regularization Local confidence [Stefanescu et al.] under 2% of white-noise corruption −c
confidence(x) = e I(x)γ −2 +1e−5
c Reliability-based regularization for image registration. TANG
(2)
6 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Our Proposed Method • Adapt regularization based on a measure that describes reliability of each pixel... • Examine local curvature and edge cues
c Reliability-based regularization for image registration. TANG
7 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Our Proposed Method • Adapt regularization based on a measure that describes reliability of each pixel... • Examine local curvature and edge cues • Adapt regularization accordingly:
Regions containing these cues Regions that lack information
→ Low regularization → High regularization
c Reliability-based regularization for image registration. TANG
7 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Our Proposed Method • Adapt regularization based on a measure that describes reliability of each pixel... • Examine local curvature and edge cues • Adapt regularization accordingly:
Regions containing these cues Regions that lack information
→ Low regularization → High regularization
• Account for local noise levels
• Encode reliability measure for regularization: arg min D(F , T ◦ M) + λR(T ) | {z } T | {z } Dissimilarity metric
(3)
Regularization
c Reliability-based regularization for image registration. TANG
7 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Outline
• Introduction • Proposed Method • Reliability Measure • Reliability-Based Adaptive Regularization • Implementation • Results • Synthetic Experiments • Segmentation-Based Evaluation • Discussions & Conclusions
c Reliability-based regularization for image registration. TANG
8 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Proposed Reliability Measure Goal: Extract local edge and local curvature cues in a noise-insensitive manner Local edge cues: G(x, y ) = |∇I(x, y )| Local curvature cues [Lindeberg 1993]: κn (x, y ) 1. Compute local curvature in scale space and normalize: κ ˜ n (x, y ; σ) 2. For scale-selection, select σ at which κ ˜ n assumes maximum value: κn (x, y ) = maxσ σ3 κ ˜ n (x, y ; σ) Local noise levels: N(x, y )
•
Extend Spectral Flatness Measure [Johnston 1988] for calculation on images
exp N(x, y ) =
•
“
1 4π 2
Rπ Rπ
ln S(ωx , ωy )∂ωx ∂ωy −π −π Rπ Rπ 1 S(ωx , ωy )∂ωx ∂ωy 4π 2 −π −π
”
(4)
Yields high response when signals occupy a wide and flat spectrum
Reliability measure: R(x, y ) = G(x, y )(1 − N(x, y )) κn (x, y )(1 − N(x, y )) | {z }| {z } noise−gated edge cues
noise−gated curvatures
c Reliability-based regularization for image registration. TANG
9 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Proposed Reliability Measure Goal: Extract local edge and local curvature cues in a noise-insensitive manner Local edge cues: G(x, y ) = |∇I(x, y )| Local curvature cues [Lindeberg 1993]: κn (x, y ) 1. Compute local curvature in scale space and normalize: κ ˜ n (x, y ; σ) 2. For scale-selection, select σ at which κ ˜ n assumes maximum value: κn (x, y ) = maxσ σ3 κ ˜ n (x, y ; σ) Local noise levels: N(x, y )
•
Extend Spectral Flatness Measure [Johnston 1988] for calculation on images
exp N(x, y ) =
•
“
1 4π 2
Rπ Rπ
ln S(ωx , ωy )∂ωx ∂ωy −π −π Rπ Rπ 1 S(ωx , ωy )∂ωx ∂ωy 4π 2 −π −π
”
(4)
Yields high response when signals occupy a wide and flat spectrum
Reliability measure: R(x, y ) = G(x, y )(1 − N(x, y )) κn (x, y )(1 − N(x, y )) | {z }| {z } noise−gated edge cues
noise−gated curvatures
c Reliability-based regularization for image registration. TANG
9 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Proposed Reliability Measure Goal: Extract local edge and local curvature cues in a noise-insensitive manner Local edge cues: G(x, y ) = |∇I(x, y )| Local curvature cues [Lindeberg 1993]: κn (x, y ) 1. Compute local curvature in scale space and normalize: κ ˜ n (x, y ; σ) 2. For scale-selection, select σ at which κ ˜ n assumes maximum value: κn (x, y ) = maxσ σ3 κ ˜ n (x, y ; σ) Local noise levels: N(x, y )
•
Extend Spectral Flatness Measure [Johnston 1988] for calculation on images
exp N(x, y ) =
•
“
1 4π 2
Rπ Rπ
ln S(ωx , ωy )∂ωx ∂ωy −π −π Rπ Rπ 1 S(ωx , ωy )∂ωx ∂ωy 4π 2 −π −π
”
(4)
Yields high response when signals occupy a wide and flat spectrum
Reliability measure: R(x, y ) = G(x, y )(1 − N(x, y )) κn (x, y )(1 − N(x, y )) | {z }| {z } noise−gated edge cues
noise−gated curvatures
c Reliability-based regularization for image registration. TANG
9 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Proposed Reliability Measure Goal: Extract local edge and local curvature cues in a noise-insensitive manner Local edge cues: G(x, y ) = |∇I(x, y )| Local curvature cues [Lindeberg 1993]: κn (x, y ) 1. Compute local curvature in scale space and normalize: κ ˜ n (x, y ; σ) 2. For scale-selection, select σ at which κ ˜ n assumes maximum value: κn (x, y ) = maxσ σ3 κ ˜ n (x, y ; σ) Local noise levels: N(x, y )
•
Extend Spectral Flatness Measure [Johnston 1988] for calculation on images
exp N(x, y ) =
•
“
1 4π 2
Rπ Rπ
ln S(ωx , ωy )∂ωx ∂ωy −π −π Rπ Rπ 1 S(ωx , ωy )∂ωx ∂ωy 4π 2 −π −π
”
(4)
Yields high response when signals occupy a wide and flat spectrum
Reliability measure: R(x, y ) = G(x, y )(1 − N(x, y )) κn (x, y )(1 − N(x, y )) | {z }| {z } noise−gated edge cues
noise−gated curvatures
c Reliability-based regularization for image registration. TANG
9 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Proposed Reliability Measure Goal: Extract local edge and local curvature cues in a noise-insensitive manner Local edge cues: G(x, y ) = |∇I(x, y )| Local curvature cues [Lindeberg 1993]: κn (x, y ) 1. Compute local curvature in scale space and normalize: κ ˜ n (x, y ; σ) 2. For scale-selection, select σ at which κ ˜ n assumes maximum value: κn (x, y ) = maxσ σ3 κ ˜ n (x, y ; σ) Local noise levels: N(x, y )
•
Extend Spectral Flatness Measure [Johnston 1988] for calculation on images
exp N(x, y ) =
•
“
1 4π 2
Rπ Rπ
ln S(ωx , ωy )∂ωx ∂ωy −π −π Rπ Rπ 1 S(ωx , ωy )∂ωx ∂ωy 4π 2 −π −π
”
(4)
Yields high response when signals occupy a wide and flat spectrum
Reliability measure: R(x, y ) = G(x, y )(1 − N(x, y )) κn (x, y )(1 − N(x, y )) | {z }| {z } noise−gated edge cues
noise−gated curvatures
c Reliability-based regularization for image registration. TANG
9 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Proposed Reliability Measure Reliability measure: R(x, y ) = G(x, y )(1 − N(x, y ))κn (x, y )(1 − N(x, y )) Corrupted Image
Local noise level
Local edge cues
Local curvature cues
Reliability 1 0.8 0.6 0.4 0.2
c Reliability-based regularization for image registration. TANG
10 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Noise-corrupted image
c Reliability-based regularization for image registration. TANG
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Proposed measure
Measure in noise−free condition
20 40 60 80 100 120 20
40
60
80
100
c Reliability-based regularization for image registration. TANG
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Proposed measure
Measure in noise−free condition
20 40 60 80 100 120 20
40
60
80
100
Measure under noise corruption
20 40 60 80 100 120 20
40
60
80
100
c Reliability-based regularization for image registration. TANG
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Proposed measure
Local confidence [Stefanescu et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
40
40
60
60
80
80
100
100 120
120 20
40
60
80
100
20
40
60
80
100
Measure under noise corruption
20 40 60 80 100 120 20
40
60
80
100
c Reliability-based regularization for image registration. TANG
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Proposed measure
Local confidence [Stefanescu et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
40
40
60
60
80
80
100
100 120
120 20
40
60
80
20
100
Measure under noise corruption
60
80
100
Measure under noise corruption
20
20
40
40
60
60
80
80
100
100
120
40
120 20
40
60
80
100
20
40
60
80
100
c Reliability-based regularization for image registration. TANG
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Proposed measure
Measure in noise−free condition
Local confidence
Local structure
[Stefanescu et al.]
[Suàrez et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
20
40
40
40
60
60
60
80
80
80
100
100
100
120
120 20
40
60
80
Measure under noise corruption
40
60
80
100
20
40
60
80
100
Measure under noise corruption
20
20
40
40
60
60
80
80
100
100
120
120 20
100
120 20
40
60
80
100
20
40
60
80
100
c Reliability-based regularization for image registration. TANG
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Gaussian Noise Proposed measure
Measure in noise−free condition
Local confidence
Local structure
[Stefanescu et al.]
[Suàrez et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
20
40
40
40
60
60
60
80
80
80
100
100
100
120
120 20
40
60
80
Measure under noise corruption
120 20
100
40
60
80
100
Measure under noise corruption
20
20
20
40
40
40
60
60
60
80
80
80
100
100
100
120 20
40
60
80
100
60
80
100
Measure under noise corruption
20
120
40
120 20
40
60
80
100
c Reliability-based regularization for image registration. TANG
20
40
60
80
100
11 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Noise-corrupted image
c Reliability-based regularization for image registration. TANG
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Proposed measure
Measure in noise−free condition 20 40 60 80 100 120 140 50
100
150
c Reliability-based regularization for image registration. TANG
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Proposed measure
Measure in noise−free condition 20 40 60 80 100 120 140 50
100
150
Measure under noise corruption 20 40 60 80 100 120 140 50
100
150
c Reliability-based regularization for image registration. TANG
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Proposed measure
Local confidence [Stefanescu et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
40
40
60
60
80
80
100
100
120
120
140
140 50
100
150
50
100
150
Measure under noise corruption 20 40 60 80 100 120 140 50
100
150
c Reliability-based regularization for image registration. TANG
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Proposed measure
Local confidence [Stefanescu et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
40
40
60
60
80
80
100
100
120
120
140
140 50
100
150
Measure under noise corruption
50
150
Measure under noise corruption
20
20
40
40
60
100
60
80
80
100
100
120
120
140
140 50
100
150
50
100
150
c Reliability-based regularization for image registration. TANG
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Proposed measure
Measure in noise−free condition
Local confidence
Local structure
[Stefanescu et al.]
[Suàrez et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
20
40
40
40
60
60
60
80
80
80
100
100
100
120
120
120
140
140 50
100
150
Measure under noise corruption
100
150
50
100
150
Measure under noise corruption
20
20
40
40
60
140 50
60
80
80
100
100
120
120
140
140 50
100
150
50
100
150
c Reliability-based regularization for image registration. TANG
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Robustness to Spatially-Varying Noise Proposed measure
Measure in noise−free condition
Local confidence
Local structure
[Stefanescu et al.]
[Suàrez et al.]
Measure in noise−free condition
Measure in noise−free condition
20
20
20
40
40
40
60
60
60
80
80
80
100
100
100
120
120
120
140
140 50
100
150
Measure under noise corruption
140 50
100
150
Measure under noise corruption
50
20
20
40
40
40
60
60
80
80
80
100
100
100
120
120
120
140
140 100
150
150
Measure under noise corruption
20
50
100
60
140 50
100
150
c Reliability-based regularization for image registration. TANG
50
100
150
12 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Quantitative Comparison with Other Measures Sensitivity of confidence measure [Suàrez et al.] 1 Gaussian Salt+Pepper Speckle SVG
0.8
0.6
0.4
0.2
0 0
0.05 0.1 0.15 0.2 0.25 0.3 0.35 Variance of noise (or density of Salt+Pepper noise)
0.4
Sensitivity = correlation coefficient between measure computed before and after noise-corruption
c Reliability-based regularization for image registration. TANG
13 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Quantitative Comparison with Other Measures Sensitivity of local structure [Stefanescu et al.] 1 Gaussian Salt+Pepper Speckle SVG
0.8
0.6
0.4
0.2
0 0
0.05 0.1 0.15 0.2 0.25 0.3 0.35 Variance of noise (or density of Salt+Pepper noise)
0.4
Sensitivity = correlation coefficient between measure computed before and after noise-corruption
c Reliability-based regularization for image registration. TANG
13 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Quantitative Comparison with Other Measures Sensitivity of proposed reliability measure 1 Gaussian Salt+Pepper Speckle SVG
0.8
0.6
0.4
0.2
0 0
0.05 0.1 0.15 0.2 0.25 0.3 0.35 Variance of noise (or density of Salt+Pepper noise)
0.4
Sensitivity = correlation coefficient between measure computed before and after noise-corruption
c Reliability-based regularization for image registration. TANG
13 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Graph-Based Registration and Adaptive Regularization Registration energy: Tˆ = arg min D(F , T ◦ M) +λ T {z } | Image similarity metric
R(T ) | {z }
(5)
Regularization
Graph-based approach - assignment of ti ∈ L on G(V, E):
c Reliability-based regularization for image registration. TANG
14 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Graph-Based Registration and Adaptive Regularization Registration energy: Tˆ = arg min D(F , T ◦ M) +λ T {z } | Image similarity metric
R(T ) | {z }
Regularization
Graph-based approach - assignment of ti ∈ L on G(V, E): X X ψij (p, q, ti , tj ) E(T ) = ψi (p, ti ) + λ p∈V
(5)
(6)
(p,q)∈E;i,j≤L
c Reliability-based regularization for image registration. TANG
14 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Graph-Based Registration and Adaptive Regularization Registration energy: Tˆ = arg min D(F , T ◦ M) +λ T {z } | Image similarity metric
R(T ) | {z }
Regularization
Graph-based approach - assignment of ti ∈ L on G(V, E): X X ψij (p, q, ti , tj ) E(T ) = ψi (p, ti ) + λ p∈V
(6)
(p,q)∈E;i,j≤L
Regularization based on reliability: X X E(T ) = ψi (p, ti ) + p∈V
(5)
λ(p, q)ψij (p, q, ti , tj )
(7)
(p,q)∈E;i,j≤L
c Reliability-based regularization for image registration. TANG
14 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-Based Regularization Encoding reliability-based regularization: 1. Continuous: λ(p, q) = e −R(p)R(q) 2. Discrete: Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows:
1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
15 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-Based Regularization Encoding reliability-based regularization: 1. Continuous: λ(p, q) = e −R(p)R(q) 2. Discrete: Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows:
1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
15 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-Based Regularization Encoding reliability-based regularization: 1. Continuous: λ(p, q) = e −R(p)R(q) 2. Discrete: Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows:
1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
15 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Graph-Based Image Registration with Reliability-Based Regularization Energy minimization for deformable registration with reliability-based regularization: X X λ(p, q)ψij (p, q, ti , tj ) (8) E(T ) = ψi (p, ti ) + p∈V
(p,q)∈E;i,j≤L
1. Labels: discretize solution domain (R2 in 2D) → ti ∈ L 2. Image similarity metric: absolute difference 3. Regularization: Distance-based regularization term 4. Transforms: • DENSE: per-pixel-assignment • SPARSE: assignment on control point of B-Spline grid
c Reliability-based regularization for image registration. TANG
16 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Graph-Based Image Registration with Reliability-Based Regularization Energy minimization for deformable registration with reliability-based regularization: X X λ(p, q)ψij (p, q, ti , tj ) (8) E(T ) = ψi (p, ti ) + p∈V
(p,q)∈E;i,j≤L
1. Labels: discretize solution domain (R2 in 2D) → ti ∈ L 2. Image similarity metric: absolute difference 3. Regularization: Distance-based regularization term 4. Transforms: • DENSE: per-pixel-assignment • SPARSE: assignment on control point of B-Spline grid
c Reliability-based regularization for image registration. TANG
16 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data Term Modification To improve fidelity of data term ψi ...
M(x)
F(x)
ti (xp) xp
ti
xp
If M(xp ) and F (xp + ti (xp )) are unreliable → ψi (ti , p) = η where: η = mean of all reliable unary costs ψi reliability determined by threshold τrely (25th percentile of R) c Reliability-based regularization for image registration. TANG
17 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data Term Modification To improve fidelity of data term ψi ...
M(x)
F(x)
ti (xp) xp
ti
xp
If M(xp ) and F (xp + ti (xp )) are unreliable → ψi (ti , p) = η where: η = mean of all reliable unary costs ψi reliability determined by threshold τrely (25th percentile of R) c Reliability-based regularization for image registration. TANG
17 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Evaluation and Results
c Reliability-based regularization for image registration. TANG
18 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations Procedure:
• Apply known deformations TGT to image F to generate M • Register F and M to obtain Tˆ • Evaluate quality of registration by computing the mean of Euclidean distance (MED) between TGT and Tˆ
Evaluation tasks:
• Registration accuracy under different types of noise and different noise levels
• Compared each deformable registration against registration with uniform regularization (with optimally tuned λ) Dataset:
• A pair of magnetic resonance imaging brain slices from BrainWeb
c Reliability-based regularization for image registration. TANG
19 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations Procedure:
• Apply known deformations TGT to image F to generate M • Register F and M to obtain Tˆ • Evaluate quality of registration by computing the mean of Euclidean distance (MED) between TGT and Tˆ
Evaluation tasks:
• Registration accuracy under different types of noise and different noise levels
• Compared each deformable registration against registration with uniform regularization (with optimally tuned λ) Dataset:
• A pair of magnetic resonance imaging brain slices from BrainWeb
c Reliability-based regularization for image registration. TANG
19 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations Procedure:
• Apply known deformations TGT to image F to generate M • Register F and M to obtain Tˆ • Evaluate quality of registration by computing the mean of Euclidean distance (MED) between TGT and Tˆ
Evaluation tasks:
• Registration accuracy under different types of noise and different noise levels
• Compared each deformable registration against registration with uniform regularization (with optimally tuned λ) Dataset:
• A pair of magnetic resonance imaging brain slices from BrainWeb
c Reliability-based regularization for image registration. TANG
19 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations Results - Effects of Data Term Modification Under different amounts of Gaussian noise
Scheme vs. registration error (MED) 5
6% noise level 18% 30%
4
MED
3
2
1
0
Uniform
Uniform + Data Term Mod.
Discrete
Discrete + Data Term Mod.
c Reliability-based regularization for image registration. TANG
20 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations Results - Effects of Data Term Modification Under different amounts of Gaussian noise
Scheme vs. registration error (MED) 5
6% noise level 18% 30%
4
MED
3
2
1
0
Uniform
Uniform + Data Term Mod.
Discrete
Discrete + Data Term Mod.
c Reliability-based regularization for image registration. TANG
20 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations Results - Effects of Data Term Modification Under different amounts of Gaussian noise
Under different amounts of speckle noise
Scheme vs. registration error (MED) 5
Scheme vs. registration error (MED)
6% noise level 18% 30%
4
3
5
MED
MED
3
2
2
1
1
0
6% noise level 18% 30%
4
Uniform
Uniform + Data Term Mod.
Discrete
Discrete + Data Term Mod.
0
Uniform
c Reliability-based regularization for image registration. TANG
Uniform + Data Term Mod.
Discrete
Discrete + Data Term Mod.
20 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations DENSE to recover thin-plate-spline warps (dmax = 8mm)
Average MED under DENSE
5
MED
10% 15% 20%
3
0.5
Uniform
CONT
CLUST Scheme
DISCR (0.8) DISCR (1.5)
c Reliability-based regularization for image registration. TANG
21 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 1 - Synthetic Deformations BSPLINE to recover B-Spline warps (dmax = 12mm)
Average MED under BSPLINE
3.5
15% 20% 25% 30%
3 2.5 2 1.5 1
Uniform
CONT
CLUST Scheme
DISCR (0.8)
c Reliability-based regularization for image registration. TANG
DISCR (1.5)
21 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 2 - Evaluation on Real Data Dataset: • 18 sagittal brain slices from ternet Brain Segmentation Repository (IBSR) • Each contain segmentation labels Procedure: • Perform registration of F and M • Evaluate quality of registration through target overlap and distance error of their corresponding segmenation labels • 153 registrations were done per registration scheme
c Reliability-based regularization for image registration. TANG
22 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 2 - Evaluation on Real Data Dataset: • 18 sagittal brain slices from ternet Brain Segmentation Repository (IBSR) • Each contain segmentation labels Procedure: • Perform registration of F and M • Evaluate quality of registration through target overlap and distance error of their corresponding segmenation labels • 153 registrations were done per registration scheme
c Reliability-based regularization for image registration. TANG
22 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Experiment 2 - Performance of different regularization schemes as evaluated on real data
8
76
7.5
74
7
Target overlap Distance error
72
70
6.5
Distance error
Target overlap
Segmentation−based evaluation measures 78
6
Uniform
Continuous
Discrete
Discrete-2
Regularization schemes
c Reliability-based regularization for image registration. TANG
23 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
An Example of a DENSE Registration Trial Reliability Measure
Moving Image
1.0 0.75
0.5
0.25 0 BSPLINE
Uniform
Continuous
Discrete
8 6 4 2 0
c Reliability-based regularization for image registration. TANG
24 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Discussions and Conclusions
c Reliability-based regularization for image registration. TANG
25 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Summary of Results Synthetic experiments:
• Adaptive regularization recovers thin-plate-spline and B-Spline warps more accurately than those recovered with uniform regularization
• Improvements in registration error: • BSPLINE: 2.45 pixels • DENSE: 0.90 pixels Segmentation-based evaluation:
• Bring improvement in target overlap by 6% Overall:
• Data term modification improves registration accuracy in all schemes • Discrete encoding scheme yielded highest registration accuracy in most cases
c Reliability-based regularization for image registration. TANG
26 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Summary of Results Synthetic experiments:
• Adaptive regularization recovers thin-plate-spline and B-Spline warps more accurately than those recovered with uniform regularization
• Improvements in registration error: • BSPLINE: 2.45 pixels • DENSE: 0.90 pixels Segmentation-based evaluation:
• Bring improvement in target overlap by 6% Overall:
• Data term modification improves registration accuracy in all schemes • Discrete encoding scheme yielded highest registration accuracy in most cases
c Reliability-based regularization for image registration. TANG
26 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Summary of Results Synthetic experiments:
• Adaptive regularization recovers thin-plate-spline and B-Spline warps more accurately than those recovered with uniform regularization
• Improvements in registration error: • BSPLINE: 2.45 pixels • DENSE: 0.90 pixels Segmentation-based evaluation:
• Bring improvement in target overlap by 6% Overall:
• Data term modification improves registration accuracy in all schemes • Discrete encoding scheme yielded highest registration accuracy in most cases
c Reliability-based regularization for image registration. TANG
26 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Conclusions
• We proposed a data-driven approach for adaptive regularization • Based on a reliability measure that: 1. Examines local edge and curvature cues 2. Accounts for local noise levels • Different schemes of encoding reliability-based regularization were proposed and evaluated
• Tested under two graph-based deformable registration frameworks • Future work: extension to 3D, conduct more validation
c Reliability-based regularization for image registration. TANG
27 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Conclusions
• We proposed a data-driven approach for adaptive regularization • Based on a reliability measure that: 1. Examines local edge and curvature cues 2. Accounts for local noise levels • Different schemes of encoding reliability-based regularization were proposed and evaluated
• Tested under two graph-based deformable registration frameworks • Future work: extension to 3D, conduct more validation
c Reliability-based regularization for image registration. TANG
27 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Conclusions
• We proposed a data-driven approach for adaptive regularization • Based on a reliability measure that: 1. Examines local edge and curvature cues 2. Accounts for local noise levels • Different schemes of encoding reliability-based regularization were proposed and evaluated
• Tested under two graph-based deformable registration frameworks • Future work: extension to 3D, conduct more validation
c Reliability-based regularization for image registration. TANG
27 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Questions?
c Reliability-based regularization for image registration. TANG
28 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Supplements
c Reliability-based regularization for image registration. TANG
29 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-based regularization Encoding reliability-based regularization: 1. Continuous (CONT): λ(p, q) = e−R(p)R(q)
•
If both have high reliability, regularization between their displacements is decreased, etc.
2. Clustered scheme (CLUST)
• •
Follows quantization scheme in [Zhang 2007] Computes λ(p, q) as in CONT and clusters the set of weights into K values using K-means
3. Discrete scheme (DISCR): Assumes change in intensity values indicates presence of boundary between different tissues types (aka “intensity cues” [Boykov et al. 2001] )
•
Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows: 1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths
•
Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
30 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-based regularization Encoding reliability-based regularization: 1. Continuous (CONT): λ(p, q) = e−R(p)R(q)
•
If both have high reliability, regularization between their displacements is decreased, etc.
2. Clustered scheme (CLUST)
• •
Follows quantization scheme in [Zhang 2007] Computes λ(p, q) as in CONT and clusters the set of weights into K values using K-means
3. Discrete scheme (DISCR): Assumes change in intensity values indicates presence of boundary between different tissues types (aka “intensity cues” [Boykov et al. 2001] )
•
Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows: 1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths
•
Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
30 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-based regularization Encoding reliability-based regularization: 1. Continuous (CONT): λ(p, q) = e−R(p)R(q)
•
If both have high reliability, regularization between their displacements is decreased, etc.
2. Clustered scheme (CLUST)
• •
Follows quantization scheme in [Zhang 2007] Computes λ(p, q) as in CONT and clusters the set of weights into K values using K-means
3. Discrete scheme (DISCR): Assumes change in intensity values indicates presence of boundary between different tissues types (aka “intensity cues” [Boykov et al. 2001] )
•
Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows: 1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths
•
Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
30 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-based regularization Encoding reliability-based regularization: 1. Continuous (CONT): λ(p, q) = e−R(p)R(q)
•
If both have high reliability, regularization between their displacements is decreased, etc.
2. Clustered scheme (CLUST)
• •
Follows quantization scheme in [Zhang 2007] Computes λ(p, q) as in CONT and clusters the set of weights into K values using K-means
3. Discrete scheme (DISCR): Assumes change in intensity values indicates presence of boundary between different tissues types (aka “intensity cues” [Boykov et al. 2001] )
•
Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows: 1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths
•
Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
30 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-based regularization Encoding reliability-based regularization: 1. Continuous (CONT): λ(p, q) = e−R(p)R(q)
•
If both have high reliability, regularization between their displacements is decreased, etc.
2. Clustered scheme (CLUST)
• •
Follows quantization scheme in [Zhang 2007] Computes λ(p, q) as in CONT and clusters the set of weights into K values using K-means
3. Discrete scheme (DISCR): Assumes change in intensity values indicates presence of boundary between different tissues types (aka “intensity cues” [Boykov et al. 2001] )
•
Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows: 1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths
•
Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
30 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Reliability-based regularization Encoding reliability-based regularization: 1. Continuous (CONT): λ(p, q) = e−R(p)R(q)
•
If both have high reliability, regularization between their displacements is decreased, etc.
2. Clustered scheme (CLUST)
• •
Follows quantization scheme in [Zhang 2007] Computes λ(p, q) as in CONT and clusters the set of weights into K values using K-means
3. Discrete scheme (DISCR): Assumes change in intensity values indicates presence of boundary between different tissues types (aka “intensity cues” [Boykov et al. 2001] )
•
Assigns λ(p, q) = {w1 , w2 , w3 , w4 }, w1 < w2 < w3 < w4 , as follows: 1. Both reliable, high noise-gated edge strengths → λ(p, q) = w1 2. Both reliable, but low noise-gated edge strengths → λ(p, q) = w4 3. Otherwise, assign intermediate weights depending on their noise-gated local edge strengths
•
Weights parameterized by µ: λ(p, q) = {w1 = µ, w2 = 2µ, w3 = 3µ, w4 = 4µ}
c Reliability-based regularization for image registration. TANG
30 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Summary of proposed methods • Reliability measure • αC • αG • Reliability-based regularization encoded in graph-based
deformable registration • DISCR: τrely , τedge , µ • CLUST: K • Registration parameters (discretization level)
• Data term modification • τrely • η (mean of all ψ)
c Reliability-based regularization for image registration. TANG
31 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data Term Modification To improve fidelity of data term ψi ...
M(x)
F(x)
ti (xp) xp
ti
xp
If M(xp ) and F (xp + ti (xp )) are unreliable → ψi (ti , p) = η where: η = mean of all reliable unary costs ψi reliability determined by threshold τrely (25th percentile of R) c Reliability-based regularization for image registration. TANG
32 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data Term Modification To improve fidelity of data term ψi ...
M(x)
F(x)
ti (xp) xp
ti
xp
If M(xp ) and F (xp + ti (xp )) are unreliable → ψi (ti , p) = η where: η = mean of all reliable unary costs ψi reliability determined by threshold τrely (25th percentile of R) c Reliability-based regularization for image registration. TANG
32 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data Term Modification To improve fidelity of data term ψi ...
M(x)
F(x)
ti (xp) xp
ti
xp
If M(xp ) and F (xp + ti (xp )) are unreliable → ψi (ti , p) = η where: η = mean of all reliable unary costs ψi reliability determined by threshold τrely (25th percentile of R) c Reliability-based regularization for image registration. TANG
33 / 28
Introduction
Methods
Results
Discussions and conclusions
Misc.
Data Term Modification To improve fidelity of data term ψi ...
M(x)
F(x)
ti (xp) xp
ti
xp
If M(xp ) and F (xp + ti (xp )) are unreliable → ψi (ti , p) = η where: η = mean of all reliable unary costs ψi reliability determined by threshold τrely (25th percentile of R) c Reliability-based regularization for image registration. TANG
33 / 28
