Jun 14, 2008 - Improving a registration accuracy by an order of magnitude .... Directly estimation in Fourier domain to identify the sub- pixel shift. ⢠Using linear ...
A projection-based extension to phase correlation image alignment Signal processing vol. 87, no. 1, Jan. 2007 Y. Keller and A. Averbuch Presented by Ho-Gun Ha
School of Electrical Engineering and Computer Science Kyungpook National Univ.
Abstract Proposed
method
– Masking operator • Projection of the estimated correlation function on the space • Rejection of components that are unrelated to estimated motion
– Advantage of the method • Improving a registration accuracy by an order of magnitude – Noisy images and volumes
• Improving a registration of rotated images
2 /28
Introduction Image
and volume registration
– Vital role in many applications • Video compression • Video enhancement • Medical data processing • Range image registration
– Various computational techniques • Gradient methods • Fourier transform based scheme – Phase correlation
3 /28
Fourier
transform based scheme
– Phase shift property
r ur • Fourier transform of I (t + Δ )
r ur ur j (ωur⋅Δur ) F {I (t + Δ )} = Iˆ(ω )e where
(1)
ur ur Δ = (Δ x , Δ y ) and Δ = (Δx , Δy , Δz ) for the 2-D and 3-D
cases, respectively.
– Related by input images / volumes ( I1 and I2 ) • Spatial domain
r ur r I1 (t + Δ ) = I 2 (t )
• Frequency domain ur j (ωur⋅Δur ) ur ˆI (ω )e ˆ = I 2 (ω ) 1
(2)
(3) 4 /28
– Phase difference matrix ur ur ur ˆI (ω ) j (ω ⋅Δ ) 2 ˆ C (ω ) = ur = e Iˆ (ω ) ur
(4)
1
where
r Δ is a translation of the image.
ur – Recovering of the translation Δ • Spatial domain – Spatial PC function r r ur ) ur −1 C (t ) F {C (ω )} = δ (t + Δ )
(5)
– Estimation of the translation » Recovering of the translation by locating the max value of C(t) ur r (6) Δ = arg max{ C (t )} r (t )
5 /28
– Spatial PC function • Integer pixel translation – Delta-like function for integer translations
• Non integer pixel translation – Dirichlet kernel » Not a smooth function – Spread peak across neighboring pixels – Degrading the accuracy of the translation estimate
– Reformulation of phase difference matrix • Compensating for intensity changes ur
Cˆ N (ω )
ur ur ˆI * (ω ) Iˆ (ω ) 1 ur 2 ur Iˆ (ω ) Iˆ (ω ) 1
where
(8)
2
Iˆ* is the complex conjugate of Iˆ . 6 /28
– Fourier domain approaches • Directly estimation in Fourier domain to identify the subpixel shift • Using linear regression to fit the phase values to a function – 2-D linear function
ur ur ur ˆ ω ⋅ Δ = arg C (ω )
(7)
• Inaccuracy estimate of a linear regression – Noisy phase data » Aliasing » Phase warpping » Aliasing effect
7 /28
The correlation masking operator Correlation
function analysis
– 1-D discrete phase correlation function e jk ΔωΔx , 0 ≤ k ≤ N
C (k ) where and
(9)
N is the length of the signal. Δx is the shift. Δω = 2π / N is the spacing of the phase axis.
– Inverse DFT of C (k ) • Dirichlet kernel C ( n ) = F − 1 {C ( k )} sin(π ( Δ x + n )) sin(π ( Δ x + n ) / N ) sin(π ( Δ x% + Δ x + n )) = sin(π ( Δ x% + Δ x + n ) / N ) = D (n − Δx) =
where Δx = Δx% + Δ x , Δ x is the integer part of Δx
(10)
8 /28
Concentration
of Dirchlet kernel’s energy
– Limited spatial support around the integer part of the translation R sin(π (Δx% + n)) sin(π (Δx% + n)) ≈∑ ∑ % % + n) / N sin( π ( x n ) / N ) Δ + n =− R n =− R π ( Δx R
=N
sin(π (Δx% + n)) ∑ π (Δx% + n) n =− R R
(11)
Denote
sin(π ( Δx% + n)) N∑ π (Δx% + n) P ( R ) = n =−∞ R sin(π (Δx% + n)) N∑ π (Δx% + n) n =−∞ R
where
(12)
P ( R ) is the energy concentration of C (n) in an interval [− R, R] relative to the total energy.
9 /28
– Analysis of Dirichlet kernel’s energy • Invariant to the length of input signal N • Computation of support R – Certain percentage of the Dirichlet kernel’s energy – Utilizing the fact that P(R) is a monotonic increasing function – Having the largest value in Δx = 0.5
– Separability of the DFT in high dimensions • Decomposing along different axes ur
Cˆ (ω ) = e
ur ur j (ω ⋅Δ )
N
= ∏ e jωk Δxk
(13)
k =1
N ur ur r −1 j (ω ⋅Δ ) C (t ) = F {e } = ∏ D ( xk )
(14)
k =1
10 /28
Fig. 1. The energy concentration property of the phase correlation function. In both the 1-D (a) and 2-D (b) cases, the energy of the Dirichlet kernel is concentrated in a small support around the integral part of the translation. (c) and (d) show the energy concentration as a function of the support in the 1-D and 2-D cases, respectively. 11 /28
The correlation function masking operator Aims
of the masking operator
– Improvement of the registration of noisy images • Assuming a noise to be AWGN – Spreading evenly over the entire spatial domain
• Restricting a spatial support – Limiting the support of PC function to the region where its energy is concentrated
Consideration
of the range of possible shifts D
D = {(dx, dy ), dxmin ≤ dx ≤ dxmax , dymin ≤ dy ≤ dymax } and (δ x , δ y ) = (dxmax − dxmin , dymax − dymin )
12 /28
r – Support of C (t ) R R R R⎫ ⎧ Dˆ = ⎨(dx, dy ), dxmin − ≤ dx ≤ dxmax + , dymin − ≤ dy ≤ dymax + ⎬ 2 2 2 2⎭ ⎩
Fig. 2. The proposed phase-correlation masking operator. The masking operator restricts the support of the spatial correlation function. The inner rectangle [−δ x / 2, − δ x / 2] × [−δ y / 2, − δ y / 2] is the interval of uncertainty of the translations and the phase correlation peak might be anywhere within this rectangle. The dotted line marks the extension of the uncertainty support by R. The masking operator zeros the correlation function beyond this support.
13 /28
– Attenuation of the noisy energy • Ration of the areas ΔWGN ≈ where
N ⋅M ( R + δ x)( R + δ y )
(15)
N and M are the initial size of the image.
– Overall improvement ΔSNR ≈ where
N ⋅M ⋅P ( R + δ x)( R + δ y )
(16)
N is the percentage of the energy of the phase correlation r function C (t ) , which is retained.
14 /28
– Example of improvement • Typical effective size used in our experimental – For N = M = 512, R = 7, (δ x, δ y ) = (10,10) 5122 ⋅ 0.9 ΔSNR ≈ = 816 = 29dB (7 + 10)(7 + 10)
(17)
– For a similar 3-D scenario
5123 ⋅ 0.9 4 ΔSNR ≈ = 2.45 ⋅ 10 = 43dB (7 + 10)3
(18)
– Property of a masking operator • The smaller the translation uncertainties δ x and δ y , the better the SNR improvement • SNR improvement that is unrelated to the spectral content of the registered images 15 /28
Fig. 3. An example of the phase denoising achieved by the masking operator. The operator is applied to the phase-correlation function of the Grapefruit MRI images (a) and (b) (used by courtesy of W.S. Hoge). The images are related by a translation of .(δ x, δ y) = (10,10) , and are of size 2562 . By masking the phase correlation function 16 /28 (depicted in (c)) using R=11 the phase in (d) is denoised.
– Phase of the masked PC function • Phase shift of the original images dx dy ⎞ ⎛ (Δϕ x , Δϕ y ) = ⎜ 2π , 2π ⎟ M N⎠ ⎝ where
(19)
(dx, dy ) is the pixelwise shift in the input images.
– Projecting PC function • Phase shift of the masked PC function ⎛ dx dy ⎞ (Δϕ x , Δϕ y ) = ⎜ 2π , 2π ⎟ R +δ x R +δ y ⎠ ⎝
(20)
17 /28
Application of the phase masking operator to motion estimation – Algorithm 1 1) Input : Two input image or volumes I1 and I 2 . 2) Apply Eq. (8) to compute the normalized phase correlation r function Cˆ N (ω ) , while masking the input images as in [13]. r ) ur −1 3) Compute the spatial correlation function C (t ) F {C (ω )}. r 4) Apply the masking operator to Cˆ N (t ) andr compute the projected phase correlation function CNP (t ) ⎧ r ⎪ r ⎪C (t ) CNP (t ) = ⎨ N ⎪ ⎪ ⎩ 0
R R ≤ dx ≤ dxmax + 2 2 R R dymin − ≤ dy ≤ dymax + 2 2 otherwise dxmin −
(21) 18 /28
In the 2-D case, or
⎧ ⎪ ⎪ r ⎪⎪C (t ) r CNP (t ) = ⎨ N ⎪ ⎪ ⎪ ⎪⎩ 0
R R ≤ dx ≤ dxmax + 2 2 R R dymin − ≤ dy ≤ dymax + 2 2 R R dzmin − ≤ dz ≤ dzmax + 2 2 otherwise dxmin −
(22)
for the 3-D case.
5) Compute theurFourier transform of the masked correlation r function CNP (ω ) = F −1{CNP (t )}. 6) Apply a motion estimation scheme such as the PCSVD ur P [13-15] to the project correlation function CN (ω ). 19 /28
Experimental Results Verifying
proposed scheme
– Sub-pixel shifted images – Sub-pixel shifted volumes – Rotation estimate Used
algorithm of estimating shift
– Sinc function interpolation – Copping a high frequency components
20 /28
Preparing the ground truth images and volumes – Algorithm 2 1) Given a pair of input images or volume I1 and I 2 . 2) Let the sub-pixel motion one would like to induce
r r r : Δ = (k / m) where (Δx, Δy) = (k1 / m1, k2 / m2 ) or (Δx, Δy, Δz) = (k1 / m1, k2 / m2 , k3 / m3 ) ,for the 2-D and 3-D cases, or
respectively. 3) I1 and I 2 are smoothed using a Gaussian kernel of size
r max ( m) .
r r r 4) The overlapping supports of I1 (t ) and I 2 (t + k ) are
extracted.
r m 5) The extracted images are downsampled by factors of to generate the sub-pixel shifted images. 21 /28
– Sub-pixel shifted replica of images
Fig. 4. The images used for verification: (a) Grapefruit MRI image (courtesy of W.S. Hoge). (b) CT slice image. (c) MRI slice image.
22 /28
– Comparing three method of image registration • Correlation function shape analysis – Sinc function interpolation
• PCSVD – SVD decomposition of phase correlation
• PPCSVD – Masking operator with the SVD decomposition
– performance of registration on various image • Best accuracy method – PPCSVD » More accuracy than PCSVD by a factor of 2-3
23 /28
Table. 1. Registration results for noise-free medical images.
– Noisy filtering of the PPSVD
Fig. 5. An example of the noise attenuation achieved by the masking operator when applied to noisy images. (a) The input noisy image (σ = 40) was registered to a noisy replica translated by (t x , t y ) = (0.5,0.5). (b) The phase of the noisy phase correlation function. (c) The phase of the projected phase correlation function. The lines of 24 /28 constant phase correspond to frequencies where ωxtx + ωy ty = C.
Fig. 6. Average accuracy of the 2-D registrations with added noise levels. The denoising properties of the masking operator are demonstrated by the improvement in the registration accuracy for high noise levels. The combined error ε was computed by the Euclidean norm of the errors ε = ε x 2 + ε y 2 . 25 /28
– 3-D volume registration
Fig. 7. Average accuracy of 3-D registration with varying added noise levels. (a) MRI scan of a human head. (b) CT scan of a human skull. (c) Applying the masking operator to the HOSVD-based scheme significantly improved the registration accuracy of the noisy volumes. The combined error ε was computed by the Euclidean norm of the errors ε = ε x2 + ε y 2 + ε z 2 .
26 /28
– Rotation estimation
Fig. 8. The images used for applying the proposed approach to rotation estimation: (a) SAR image. (b) Airfield image. (c) Pentagon. Table. 2. Rotation estimation results.
27 /28
Summary and conclusions
Proposed method – Improvement of the precise registration • Masking operator that is applied to the phase correlation function – Projecting the PC function to the space of function related to a particular range of translations – Attenuation of other phase components that unrelated to the motion estimate
– Complementary method to other sub-pixel registration • Significant accuracy improvement – PCSVD (rank one PC scheme) – Sinc interpolation
28 /28
