Computational Methods for Image Deconvolution - Emory University

Introduction and Applications Mathematical Model Computational Approaches

Computational Methods for Image Deconvolution Mathematics and Computer Science Emory University

James Nagy Julianne Chung, Daniel Fan, Piotr Wendykier

Mathematics and Computer Science Emory University

Computational Methods for Image Deconvolution


Outline

1

Introduction and Applications

2

Mathematical Model

3

Computational Approaches




Image Deconvolution: Simple Example

Given blurred image, and some information about the blurring. Goal: Compute approximation of true image.




Image Deconvolution: Simple Example

Given blurred image, and some information about the blurring. Goal: Compute approximation of true image.




Cosmology Measuring Cosmic Microwave Background Radiation Observed data


Deconvolved data



Astronomy Viewing distant star fields using ground based telescopes. Observed data


Deconvolved data



Space Observations Viewing space vehicles, satellites and other space junk. Observed data


Deconvolved data



Medical Imaging

Observed data


Deconvolved data



Microscopy Observed data


Deconvolved data



Applications: Super Resolution for Iris Recognition Basic idea: Given set of low resolution images,




Applications: Super Resolution for Iris Recognition Basic idea: Given set of low resolution images,




Applications: Super Resolution for Iris Recognition Basic idea: Given set of low resolution images, combine to get a high resolution image




Applications: Super Resolution for Iris Recognition Basic idea: Given set of low resolution images, combine to get a high resolution image without distortions.




Applications: Super Resolution for Medical Imaging Similar to iris recognition, but Registration (alignment) of images more difficult. Deconvolution is a subproblem of a nonlinear optimization method.




Mathematical Model of Image Formation

t

v

s

u




Mathematical Model of Image Formation x(s, t)

t

v

s

u




Mathematical Model of Image Formation ZZ a(u, s, v , t)x(s, t)ds dt

t

v

s

u




Mathematical Model of Image Formation ZZ b(u, v ) =

a(u, s, v , t)x(s, t)ds dt + e(u, v )

t

v

s

u




”Convolution” implies shift invariance ZZ b(u, v ) =

a(u − s, v − t)x(s, t)ds dt + e(u, v )

t

v

s

u




Some remarks Convolution: Given functions x(s, t) and a(u − s, v − t), form the integration: ZZ b(u, v ) = a(u − s, v − t)x(s, t)ds dt

Deconvolution: Given functions b(u, v ) and a(u − s, v − t), ”undo” the integration to compute x(s, t). The inverse problem of deconvolution is very sensitive to noise: ZZ b(u, v ) = a(u − s, v − t)x(s, t)ds dt + e(u, v ) That is, small changes in e can produce large changes in x. Mathematics and Computer Science Emory University



Discrete Model Images are usually given by discrete pixel values, not functions! The theoretical mathematical model: ZZ b(u, v ) = a(u − s, v − t)x(s, t)ds dt + e(u, v ) can be approximated by matrix-vector equation: b = Ax + e A is defined by the ”point spread function” a(u − s, v − t). If the PSF is not known, it can be estimated from the image of a ”point source” object. Mathematics and Computer Science Emory University



Properties of the Discrete Model From the matrix-vector equation b = Ax + e Given b and A (or the PSF), compute an approximation of x Regarding the noise, e: It is usually not known. However, some statistical information may be known. It is usually small, but it cannot be ignored! That is, solving the linear algebra problem: Ax = b

⇒

x = A−1 b

usually does not work. Mathematics and Computer Science Emory University



Regularization Basic Idea: Modify the inversion process to compute: xreg = A−1 reg b so that ˆx = A−1 reg b = A−1 reg (Ax + e) −1 = A−1 reg Ax + Areg e

where A−1 reg Ax ≈ x

and

A−1 reg e is not too large




SVD Analysis An important linear algebra tool: Singular Value Decomposition Let

A = UΣVT

where

Σ =diag(σ1 , σ2 , . . . , σn ) , UT U = I ,

σ1 ≥ σ2 ≥ · · · ≥ σn ≥ 0

VT V = I




SVD Analysis An important linear algebra tool: Singular Value Decomposition Let

A = UΣVT

where σ1 ≥ σ2 ≥ · · · ≥ σn ≥ 0

Σ =diag(σ1 , σ2 , . . . , σn ) , UT U = I ,

VT V = I

Columns of U and V are orthonormal basis vectors U=

u1

u2

···

un

V=

v1

v2

···

vn




SVD Analysis The na¨ıve inverse solution can then be represented as: x

=

A−1 b

=

VΣ−1 UT b

=

n X uT b i

i=1

σi


vi



SVD Analysis The na¨ıve inverse solution can then be represented as: ˆx

=

A−1 (b + e)

=

VΣ−1 UT (b + e)

=

n X uT (b + e) i

i=1

σi


vi



SVD Analysis The na¨ıve inverse solution can then be represented as: ˆx

=

A−1 (b + e)

=

VΣ−1 UT (b + e)

=

n X uT (b + e) i

σi

i=1

=

n X uT b i

i=1

=

σi

vi +

vi

n X uT e i

i=1

σi

vi

x + error




Filtering Algorithms Basic Idea: Filter out effects of small singular values.

xreg =

n X i=1

φi

uT i b vi σi

where the ”filter factors” satisfy 1 if σi is large φi ≈ 0 if σi is small




Some Filtering Methods 1

Truncated SVD, or Pseudo Inverse Filter: xtsvd =

k X uT b i

i=1

2

vi

Tikhonov, or Wiener Filter: xtik =

n X i=1

3

σi

σi2 uT i b vi 2 2 σi + λ σi

Iterative Filters: (more later)




Some Filtering Methods 1

Truncated SVD, or Pseudo Inverse Filter: xtsvd =

k X uT b i

i=1

σi

0.8

filter factor

1

vi

0.6

n = 64, k = 36

0.4

0.2

0

−5

−4

10

−3

10

−2

10

−1

10

0

10

1

10

10

singular values

Tikhonov, or Wiener Filter: xtik =

n X i=1

1

0.8

σi2 uT i b vi 2 2 σi + λ σi

filter factor

2

0.6

α = 0.001

0.4

0.2

0

3

Iterative Filters: (more later)


−5

10

−4

10

−3

10

−2

10

−1

10

0

10

1

10

singular values



Remarks on Computational Methods

SVD filtering can be computationally expensive. Further simplifying approximations are often used to obtain more efficient algorithms: Fast Fourier transforms (FFT) based methods are most popular. Other fast transforms can be used. Exploiting structure in A can provide additional options.




Iterative Algorithms Get an initial estimate: x0 ≈ x Generate a sequence of solutions: x1 , x2 , . . . The computational effort to compute each xk is much less than directly computing A−1 b. Theoretically, xk converges to A−1 b Hopefully convergence is fast!




Iterative Algorithms xk converges to A−1 b For well-posed problems this is good! For ill-posed problems this is bad!





Want xk to converge to A−1 reg b





Want xk to converge to A−1 reg b Observation: Appropriate stopping iteration can give a good approximation of x. How to know when to stop? What if stopping rule is imprecise? If convergence is too slow, how to accelerate?




Current Research Projects Encouraging iterative method to converge to A−1 reg b. (Julianne Chung)

Determining when to stop the iteration. (Julianne Chung) Accelerating convergence (preconditioning). Applications in biometrics, astronomy, medical imaging, microscopy. Incorporating physical constraints in the problem. High performance computing/software issues. (Piotr Wendykier) Using other bases (e.g., wavelets, fractals). (Daniel Fan) Inpainting applications. (Daniel Fan) Mathematics and Computer Science Emory University



Software and more can be found at: http://www.mathcs.emory.edu/∼nagy/RestoreTools • MATLAB • Java (using ImageJ) • Python




Software and more can be found at: http://www.mathcs.emory.edu/∼nagy/RestoreTools • MATLAB • Java (using ImageJ) • Python

THANKS!



Computational Methods for Image Deconvolution - Emory University

Computational Methods for Image Deconvolution - Emory University

Suggest Documents

Computational methods for nonlinear image registration

ARTICLES - Emory Genetics - Emory University

Iterative deconvolution and semiblind deconvolution methods ... - CMAP

PDF - Emory Physics - Emory University

2mm Blind Image Deconvolution

Practical Computational Methods for Regulatory ... - Brown University

BLIND IMAGE DECONVOLUTION USING ... - CiteSeerX

Image patching Sparse deconvolution Background

INVERSE SCALE SPACE METHODS FOR BLIND DECONVOLUTION

Records Management Archivist - Emory Libraries - Emory University

Chemical Hygiene Plan - Emory EHSO - Emory University

Emory Chemical Hygiene Plan - Emory EHSO - Emory University

Patent filing_H_1 - Emory Economics - Emory University

Patent filing_H_1 - Emory Economics - Emory University

Soft jammed materials - Emory Physics - Emory University

Squishy Materials - Emory Physics - Emory University

europhysics letters - Emory Physics - Emory University

Rare Book Cataloger - Emory Libraries - Emory University

Keeping The Beats - Emory Libraries - Emory University

Spring 2018 - Emory Libraries - Emory University

Emory Libraries, Fall 2017 - Emory University

2mm Blind Image Deconvolution [PDF]

Expanding Exposure - Emory Economics - Emory University

Alcohol Alternatives - Emory Psychiatry - Emory University