Increasing the information content in the final image by ... Propagating the outlier pixels into final HR image .... Modifying the step depending on pixel location.
Adaptive Outlier Rejection in Image Super-resolution EURASIP Journal on Applied Signal Processing vol. 2006 Mejdi Trimeche, Radu Ciprian Bilcu, and Jukka Yrjänäinen Presented by Ho-Gun Ha
School of Electrical Engineering and Computer Science Kyungpook National Univ.
Abstract Proposed
method
– Integrated adaptive filtering method • Isolating the outlier image region by decreasing the corresponding coefficient • Enhancing robustness in subpixel registration
– Advantage of proposed method • Performing well in the presence of motion outliers • Relatively simple and fast mechanism • Effective to Gaussian noise
2 /36
Introduction Super
resolution
– Definition • Increasing the information content in the final image by exploiting an additional spatio-temporal information – Using each of the LR images
• Combing a set of LR to reconstruct a high-resolution image
– Necessity of super resolution in mobile devices • Overcoming the limitations due to optics and sensor resolution – Pricing constraints – Computational and memory resources
3 /36
– Quality of the super-resolved image • Depending on heavily on the accuracy of the motion estimation – subpixel precision
– Major problem with global registration • Limited to the assumed parametric model • Completely failed in the presence of outliers – Moving objects inside the scene – Repetitive textures – Localized noisy areas Robustness towards registration error is a critical requirement in super resolution
• Suppressing useful high frequency information – Leading to smooth result
• Not properly handled in this model – Propagating the outlier pixels into final HR image
– Mixed noise model • Handling through the minimization of the L p norm • Used noise model – Laplacian distribution rather than Gaussian distribution
• Using a pixelwise median 5 /36
– Disadvantage of median • Not optimal in Gaussian noise • Delicate trade-off between outlier rejection performance and the capability to reconstruct aliased high frequency
– Proposed method • Efficiently handing local outlier – Scanning progresses over the image grid – LMS estimator
6 /36
Imaging model Forward
synthesis model
– Relating the HR image to the LR observations • Obtaining different views of a single continuous HR image • Involving consecutively geometric transformation, sensor blurring, subsampling, additive noise gi ( x, y ) = S ↓ (hi (u , v) ∗ f (ξi ( x, y ))) + ηi ( x, y ) where
and
(1)
g i is the th observed LR image. i f is the HR reference image. hi is the point spread function (psf). ξi is the geometric warping. S ↓ is the downsampling operator. ηi is the additive noise term. * denotes the convolution operator. 7 /36
– Matrix form • Discretization of the forward synthesis model
g i = A i f + ηi where
and
(2)
A i combines successively, the geometric transformation ξi , the convolution operation with the blurring parameters of hi , and the downsampling operator S ↓ . gi , f , and ηi are lexicographically ordered.
8 /36
Fig. 1. An illustration of the image degradation process following the model in (2).
9 /36
Iterative
super resolution
– Super resolution reconstruction • Estimating the best HR image • Generating the closest estimate of LR image
– Cost function • Minimizing the error function ε i = gˆ i − gi where
2
= Ai f + gi
2
(3)
gˆ i is the simulated LR image through the forward imaging model.
10 /36
– Minimizing the error function • Iterative gradient descent method • Optimization technique for seeking converge ε i f n +1 = f n + μin ri n where and
(4)
μin is the step size at iteration n. ri n is the residual gradient at iteration n .
– Residual gradient ri n = Wi ( g i − A i f n )
(5)
where Wi combines successively the upsampling. −1 and ξi is the inverse geometric warp.
11 /36
– Step size μi
n
• Given by steepest decent method
μ = n i
gi − Ai f A i ri
n 2
n 2
(6)
• Scaled gradient term – Data that point to the values from each gradient images at the pixel position
z k = { pi ( k ), i = 1,K , N }
• Resulting updated value on the HR image grid yk = Φ ( z k )
(7)
where Φ is a generic filtering operator that performs the fusing of the pixel from all available gradient images. 12 /36
– Regularization of solution • Ill-proposed inverse problem of super resolution • Obtaining stable solution – Neglecting the regularization factor in simulation » Focusing efficient implementation
f k n +1 = f k n + yk + μ nα sk where
α
(8)
is the regularization parameter that controls the condition of the solution.
n is the number of iteration. and
sk is the contribution that is due to the regularization process at pixel k.
13 /36
Fig. 2. Genetic block diagram of the iterative super-resolution process. The gradient images are combined using a filtering operator Φ that can be modulated depending on the application. 14 /36
Fusing
the gradient images
– Filtering operator • Retaining the novel information from each LR frame • Filtering out the noise • Rejecting motion outlier
– Using median filter • Effective against impulse error – Noise of Laplacian distribution
• Improving the robustness against motion outlier
– Adaptive fusing strategy • Automatic isolation of localized outlier • Not modeling noise of a stationary distribution 15 /36
Our approach Outlier
rejection by adaptive FIR filtering
– Output of an FIR filter • Related contribution that each LR image brings into fuse image • Modulating the weights associated with each input image – Taking into account the presence of outlier regions N
yk = ∑ ai pi (k ) = aT z k
(9)
i =1
where
a is the FIR coefficient vector.
16 /36
Coefficient adaptation – Spatially adapting the FIR coefficients • LMS coefficient adaptation (1) initialization : a0 = [1 / N ,K1 / N ] (2) for k = 1K1
T (2.1) filtering : yk = a k −1 z k
(2.2) error computation : ek = d k − yk = median (z k ) − yk (2.3) coefficient update : a k = a k −1 + λ ek z k (2.4) move to next pixel location k + 1 where λ is the step-size parameter. and d k is the desired response of the LMS estimator.
17 /36
– LMS estimator • Using median estimator – Tuning the algorithm for robust against local outlier
• Modulating the estimator – Depending on specific object » Speed » performance
18 /36
Fig. 3. Block diagram of the proposed fusing method. The gradient images are combined with a spatially varying FIR filter. The coefficients of the FIR are chosen with an LMS estimator that is tuned to reject outliers. 19 /36
Stability of LMS adaptation – Initialization of the step size • Trade off between speed of convergence and quality of adaptation
