regularized blur-assisted displacement field ... - Semantic Scholar

1 downloads 0 Views 360KB Size Report
spread field, a motion field and a restored image in an ... of the blur in order to restore the scene. ... train in a direction dictated by the displacement vector.
REGULARIZED

BLUR-ASSISTED Damon

DISPLACEMENT

L. Tull and Aggelos Northwestern

Department

of Electrical Evanston,

ABSTRACT

ESTIMATION

K. Katsaggelos

University and Computer

Illinois

damonQece.nwu.edu,

FIELD

Engineering

60208-3118 [email protected]

a regularized

solution

approach.

tion required some knowledge

The problem

formula-

of the point spread sup-

Due to the finite acquisition time of practical cameras, objects can move during image acquisition, therefore introducing motion blur degradations. Traditionally, these degradations are treated as undesirable artifacts that should be removed before fllrther processing. In this work, we consider the use of motion blur as an indication of scene motion. JVe present two robust regularized motion estimation algorithms that consider the use of (motion) blur in their formulation. The first algorithm uses motion blur as prior knowledge for the estimation of the motion field. The second algorithm

complete knowledge of the blur point spread function in order to restore the scene. The blur was derived from

considers the joint estimation of the motion and motion blur. Each approach results in a motion blur point

an estimate of the image motion of that frame using a constant velocity assumption. The (motion) blur point

spread field, a motion field and a restored image in an approach that is different from previous work. Prelim-

spread field was modeled

inary results are presented.

field (DVF)

port.

In their most recent work, You and Kaveh in [2]

estimated the support of the blur function using a pruning algorithm which gave promising results in practical image scenes. In [1], we considered fication

and restoration

the spatially

algorithm

train in a direction

identi-

of blurs caused by object

tion during image acquisition. age restoration

variant

A robust regularized

was proposed

impulse

by the displacement

at each pixel.

im-

which required

as a two dimensional

dictated

mo-

The support

vector

of the motion

blur PSF at each pixel was assumed to be proportional 1.

to the extent of the motion at each site. This constant velocity (CV) assumption proved to be useful in a wide

INTRODUCTION

It is common to observe blur due to motion in the frames of an image sequence. If any point on an ob,ject

variety of degraded

passes over two or more photoreceptors

tivation.

acquisition, a blur will occur. single class in a host of possible

during

image

}Iotion blur is only a image distortions. The

In both between

the work of You image sequences

treated

as an undesirable

artifact

moved before further processing. a spatially

varying

degradation

blur has been

that should Motion

be re-

blur is often

as in the case of fast

was the mo-

restoratio~) and propose the use of blur in the estimate of image motion. We propose an extension of

ple degradations.

motion

[1, 3].

[2], image restoration

In this work, we consider the relationship motion and motion blur (identification and

ability to classify degradations is an important step towards the restoration of image sequences with multiTraditionally,

image sequences

[1] and

and Kaveh

to the case of dynamic

where motion

is defined.

CV motion assumption developed DVF to a parameterized (motion) the scene.

A bilinearly

interpolated

Using the

in [1], we relate the blur field (PBF) for motion blur model

moving objects [1]. Most image restoration algorithms require knowledge of the blur in order to restore the

is used to provide a discrete approximation tinuous PBF. Two regularized displacement

scene. The identification varying case is a difficult

of the blur in the spatially problem addressed by only a

algorithms are proposed. Each algorithm is based on a robust regularized formulation that allows disconti-

few researchers. In recent work by You and Kaveh [2], blur identification is considered in the context of a blind deconvolution approach where a restored image and the blur

nuities in both the DVF and the PBF. In the first algorithm, motion information, obtained from the PBF, is incorporated as “cue” or a priori knowledge of the motion

field using a recently

point spread function

robust

convex

are simultaneously

estimated

in

stabilizers.

proposed

of the conestimation

entropic class of

The entropic

class of stabi-

lizers allows

the incorporation

of prior

knowledge

of the

structure of the DVF into the estimation procedure, in a non-linear manner similar to the maximum informa[4]. The second algorithm

tion principle

combines

mo-

tion estimation, blur identification and image restoration in an approach that represents a joint estimation of image motion. In the following section, we discuss a robust regularized motion estimation formulation and an iterative

solution

approach.

Section

3 describes

a

data fidelity energy while ~D measures the smoothness of the DVF estimate. Selecting dD and #D to be convex measures allows the optimal regularized estimate of the DVF to be obtained

v~+l(m,

n) =

from the iteration, ~QD(ub,

v~(m, n) – v

vb,

~)

dv~(m, n)

where v~+l (m, n) is the updated

estimate



(3)

of the hori-

simultaneous blur identification and image restoration algorithm following [2]. Section 4 present the first of

zontal component of the motion field at iteration i and q is the relaxation parameter to ensure proper conver-

two algorithms

which utilize

gence of the iteration.

tion estimation

criterion.

algorithm

in Section

the DVF.

Section

blur in the mo-

the joint

image restoration

mation problem.

Our proposed

by some preliminary

from the

3 is used as prior knowledge

5 describes

blur identification.

tion 6. Section

motion

The PBF obtained

formulation

and motion

approach

esti-

in motivated

results which are presented

7 concludes

of of

The

solution

to obtain

motion

fields in the presence of motion

in Sec-

our discussion.

3.

A REGULARIZED MOTION ESTIMATION ALGORITHM

The goal of most DVF estimation

DFDk(m.

(coding

problem

based)

A REGULARIZED

In [2], a solution

formulations

of the

is to minimize,

BLUR ALGORITHM

to the identification

problem

the restored

n) = - UD(Tmn), n

- VD(m,

in the

case of spatially variant blur is proposed. The formulation incorporates piecewise smoothness of the PBF and image in the identification

anisotropic diffusion. The minimization ing functional is proposed,

fk(m, n) - f~-l(?n

field

discontinuities via a robust convex selection of $. We adopt this approach for the subsequent analysis.

IDENTIFICATION 2.

for the vertical

component of the motion is obtained by replacing v with u in Eq. (3). This iteration was used in [5, 6, 7]

process

using

of the follow-

n))(l) MN

where f~ (m, n) is the pixel intensity at discrete location (rn,n) at image frame k. components

of the DVF

respectively.

Finding

The vertical

and horizontal

are uD (m, n.) and ~D(m, n),

the motion

field that minimizes

the DFD at each pixel location is an ill-posed problem in that there are many motion fields that minimize the DFD.

To make the minimization

posed problem, smoothness,

prior knowledge,

is imposed

of Eq.

(1) a well-

often a desired level of

on the solution.

For an LI x N

image, a well-posed or regularized estimate of the DVF can be obtained through the minimization of the cost function, h[ QD(UD,

IID, A) =

where.

AF, ~}, Afi are the regularization

parameters

and uE, uE are the vertical and horizontal components of the PBF. The functions @F, OF, QB are the identification

fidelity, restored

smoothnem

image smoothness

measures respectively,

bust and convex (i.e., log(cosh(x) of the current estimate

denoted

and PBF

all selected to be ro) ). The residual error

by e(m, n), is given by,

N

~

~

rn=l

71=1

?bD(DFD~(m.

e(m, n) = g(m, n) – h(m, n) * f(m, n)

n))

where g(m, n) is the observed

MN

(5)

image, * denotes convo-

lution and h(m, n) is the response of the degradation filter at (m,n). In this work, h(m, n) is chosen to model where V~D(m,

n) and V~D(m.

dient of the horizontal

7L) are the Spatial gra-

and vertical

components

of the

motion field, respectively. The constant A E [0, m) is the regularization parameter which controls the tradeoff between original

smoothness

(A = m)

data (~ = O). The function

and fidelity

CV motion blur such that the pixel location tains the value given by,

to the

@D measures

the

h(m, n) * f(m, n) =

(m, n) ob-

where

~1 = l/L and L is the

mined

by

length

of the

blur

cleter-

This criterion is convex for fixed prior knowledge, allowing for efficient optimization. The optimal solution of this criterion

L = /UB(~.~)2

+ uB(~..

n)2.

(7)

The robust selection of the measures @B in Eq. (4) allows for edge preserving regularization similar to the anisotropic diffusion mechanism described in [8]. The vertical and horizontal components of the PBF (UB, ~B )

the DVF

structure

cedure is detailed

MOTION

in [2].

A COOPERATIVE

Incorporating

AS A MOTION

prior knowledge

ill-posed problem

is common

forming

the problem

ditional

regularization

a way of making for trans-

explicitly

through

problem.

incorporates

the addition

Tra-

smooth-

of a regular-

taneous

spatially

constraint.

varying

The result is a simul-

motion

and blur estimation image.

By selecting

@B = @D to be a robust, convex functional, and using the CV constraint to express the PBF in terms of the DVF, we obtain t,he robust PBF-DVF criterion, QBD(UD, hf

can introduce prior knowledge about the solution in a non-linear manner. It is called the robust minimum (R.MIAI) measure

ALGORITHM

scheme that results in a restored

ization term like 4D (.) in Equation (2). In [4], Zervakis et al. proposed a robust, convex functional that

absolute-information},

of is,

of the DVF and the PBF may be combined

using the CV motion

about the solution in an

into a well-posed

ness of the solution

PRIOR

That

BLUR-ASSISTED

ESTIMATION

Estimation

BLUR

the PBF.

proach to incorporating prior knowledge has been recently shown to be effective in image restoration [4, 3]. 5.

MOTION

by prior knowledge

from

features such as discontinuities and edges in the PBF will be imposed on the solution of the DVF. This ap-

are obtained from the alternating minimization of Eq. (4) between the restored image j and the PBF. This pro-

4.

is guided obtained

~JD,f,~)

=

N

~~~(e(m,n))

~

+ @D(DFD(m,n)) +

m=l n=l

defined by,

+~-1)

~B(l

[@D (l~UD(m,n)/)

+ @D (l~vD(m,~)l)]

(11)

+~F#F([~f(m,n)l)

Since @ and d are convex where c is a small constant zero and T represents

used to avoid division

prior or target knowledge

by

of the

estimate Z. The exponential constant e is used to scale the functional so that the optimal solution would be Z. This functional was shown to be a powerful means of introducing

prior knowledge.

rate the PBF as a motion the DVF by invoking straint [l],that

that this criterion

is also a function

6.

velocity

(CV)

RESULTS

JVe are concerned ~’UD

=

of a, which relates

~uD,

AND

DISCUSSION

contion.

UB

(9)

with degradations

The assumptions

motion

be of constant

proper

idefitification,

to this point velocity

time.

Physically,

represents

related to the camera

a is a global

the fraction

image

of the DVF

by an intensity point during practical cameras a ~ [0, 1).

image

Replacing ~ with of in Eq. (2)gives prior criterion,

exposure

parameter

distance

that

traveled

acquisition.

For

the robust PBF-

of blur is a common quences.

occurance

Figure 1 illustrates

related impose

to mothat the

and translational.

via motion

estimation,

must persist for at least two frames. where a is a constant

of

and ~. Note

In order to incorpo-

the constant

=

the DVF,

the motion to the PBF. This constant can be found iteratively, solving the motion blur identification problem in the process.

prior, we relate the PBF to

is. ‘UB

in Eq. (11) the iteration

Eq. (3) can be used to obtain

For

the blur

The persistence

in practical

image se-

the degradation

caused by

a moving block over a uniform background. The length of the DVF is 10 pixels and the length of the PBF is 5, which corresponds to a = 0.5. Figure 2 shows spatially variant restoration this object

using the algorithm

using the DVF

of Section

3 of

and using the CV assump-

tion. k~ QPRIOR(UD,

VD, ~,~)

=

+~D@I(l~uD(m,

+ADdI(luvD(m,

N

~ ~ 4(DFD~(m,n)) ln=l rz=l n)l, ~-ll~UB(m,

n)l, a-11vVB(m,7L)[).

7. ~)l)

(lo)

CLOSING

REMARKS

In this work, we consider the use of blur information in the regularized formulation of the motion estimation

Figure 2: Spatially a by product

variant restoration

of using the algorithm

In ICASSP,

variant blurs. 2610. IEEE,

of motion

blur.

of Section 3

volume

4. pages 2607–

May 1995.

[3] D .L. Tull and A.K. Katsaggelos. Iterative restoration of fast, moving objects in dynamic image sequences. Figure

1:

Synthetic

example

moving left to right.

of motion

blur.

Optical

Engineering,

To appear.

1997.

Block

DVF length = 10, PBF length =

5. effective a = 0.5

[4] LM.E. Zervakis, A.K. Katsaggelos. and T.K. Kwon. A class of robust functional for image restoration.

IEEE Trans. on Image Process i?t,q,3(14] :752-773, June 1995.

problem.

We proposed

t~vo algorithms

for this purpose.

The first approach models the role of motion blur as being simply a cue or a “stimulus”. providing prior structural information for motion estimation. Our approach differs from previous work for example, Chen et al. [9], in that we consider tion approach explicitly second

accounts

proposed

or cooperative counts

a robust regularized

to the motion

estimation

for the source of motion algorithm

estimate

for (motion)

represents

that

blur.

Our

a simultaneous

of the image motion

along with additional

fields.

Conference on Image ProIn International volume 2, pages 270–27’4. November 1994.

cessing,

[6] D .L. Tull and A.K. tion estimation In International

Conference

[7] D .L. Tull and A.K.

Results

In IEEE

International

Systems,

Atlanta,

Regularized

entropic on

volume 3, pages 212–215,

timation

results will

Kat,saggelos,

using robust

that ac-

blur in the formulation.

for both approaches

solu-

problem

[5] B. Rouchouze, P. Mathieu, T. Gaidon, and \I. Barlaud. Motion estimation based on markov random

Image

October

Regularized

displacement Conference

Proce,ssi~ig,

1995.

Katsaggelos.

of occluded

mo-

functional.

vector on

es-

fields.

Circuits

and

GA, May 1996.

be shown during our presentation. [8] .J. Malik and P. Perona. tection using anisotropic 8.

REFERENCES

actions gence,

[1] D .L. Tull and A .1