The design of a markovian image-matching technique ...

The design of a markovian image-matching technique and its comparison to a variational technique in the context of mammogram registration. Frédéric J.P. Richard MAP5, FRE CNRS 2428, University Paris 5 - René Descartes, Dept of Mathematics and Computer Science, 45, rue des Saints Pères, 75270 Paris Cedex 06, FRANCE. [email protected] Tel. : 33 (0)1 44 55 35 41, Fax. : 33 (0)1 44 55 35 35.

Abstract In this paper, we deal with the image-matching problem. Our goal is on the one hand to design an image-matching technique based on the theory of MRF (Markov Random Fields) and on the other hand to compare experimentally this stochastic approach to variational approaches. For that, we present a deterministic and discrete image-matching framework which is equivalent to an usual variational framework. This discrete framework is further described from the stochastic point of view of MRF. In the markovian framework, we design a multigrid image-matching technique using principles of the ICM (Iterated Conditional Mode). In the mammogram registration context, we compare performances of this technique to those of an usual multigrid variational technique. Results suggest that the relaxed ICM and the variational algorithm have same registration performances. Keywords. image registration, image matching, markov random fields, Iterated Conditional Modes, Partial Derivative Equations, variational equations, multigrid, finite elements, mammography.

1. Introduction There are two famous mathematical ways to tackle image-processing problems. The first one consists of using the theory of MRF (Markov Random Fields) and the second one is based on the calculus of variations. These mathematical approaches were first championed by Stuart and Donald Geman [6] and David Mumford and Jayant Shah [10] in their respective works about the image segmentation and restoration problems. These pioneer works have inspired a lot of researches on other image-processing problems. For instance, concerning the optical flow problem, some authors have investigated techniques which enable to recover the optical flow while

preserving flow discontinuities using variational principles [5] and markovian approaches based on line processes introduced Geman [9]. Markovian and variational approaches have been sometimes compared. For instance, in [10], Mumford and Shah gave some comments about links between their problem formulations and those of Geman. However, experimental and theoretical comparisons of both approaches remain very few. Besides, although it is related to the optical flow problem, the image-matching problem has been mainly tackled in continuous settings using variational approaches [1, 2, 4, 8, 11, 16]. To our knowledge, stochastic image-matching techniques which have been proposed are not based on MRF [4, 7, 8]. In this paper, we design an image-matching technique following principles of the MRF theory. Our motivation is to compare performances of markovian-like techniques to usual variational techniques. We present such a comparison in the mammogram registration context which we have already investigated with several variational approaches [13, 14, 16, 17]. In section 2, we remind an usual variational imagematching model and propose an equivalent one which is discrete and can be interpreted in stochastic and markovian terms. In section 3, a new resolution technique is designed following optimization principles of the ICM (Iterated Conditional Mode) [3]. In this section, the multigrid variational resolution technique we proposed in our previous works is also briefly described. In section 4, some comparisons of the markovian and variational techniques are given in the mammogram registration context.

2. Models In this section, we present successively two equivalent image-matching models. The first one is continuous and suitable for a variational approach. The second one is discrete and suitable for a markovian approach. This model is described from two different points of view. The first point of view is deterministic and the second one is

(a)

(b)

(c)

(d)

Figure 1. Application of algorithms to a mammogram pair (bilateral pair mdb077/078 of the MIAS database): (a) the left mammogram, (b) the left mammogram deformed onto the right one by the ICM algorithm 1, (c) the left mammogram deformed onto the right one by the gradient descent (algorithm 2), (d) the right mammogram.

the images    and   are, the lower this term is. It is an intensity-based matching constraint which will be called the similarity term. The first term is a regularization term which ensures that the problem is well-posed and that solutions are non-degenerate solutions. Its design is usually based on a strain energy of the continuum media mechanics. Inspired by the theory of linearized Elasticity, we define the strain energy for all % and L in 0  as in [16] by

stochastic.

2.1. Continuous model The classical variational framework for imagematching is the following [1, 2, 4, 16]. Let be a connected and open subset of
 and  and  be two images defined on using interpolation. For the sake of presentation simplicity, we assume that the image domain is the square unit   . Let us denote by the set which is the closure of (with respect to the euclidean norm of
 ) and contains the set and its boundary. Let  be a image coordinate change, ie a smooth functions mapping onto itself. We denote by   the geometric deformation of  induced by  :    !"$#$   . We denote by % the displacement field associated to  . The variable % is equal to '&)(+* , where (+* is the identity map (', -(+*  ./ ). We assume that the variable % belongs to a space 0  composed of smooth function mapping onto itself. In this framework, matching  and 1 consists of finding an element % of 0  which is such that the deformed image 24 3657 is “similar” to   . This is expressed in terms of an inverse problem [1, 2, 4, 16]: Model 1 Find an element of 0 ergy 8 of the following form

; =  %>-LMONPFM%>-LQ = /R

(2)

 IH
? and

where N+DUTDVQ = is the usual scalar product on FG F is the following operator1

:Gf a FM%WX&Y*ZU[\6]I^`_ Pa %?-b(+*dce@ %hgi

(3)

:kjl:

In this definition, (+*dc is the identical matrix of size : and a %? is the linearized strain tensor nm Po %?pq@ o %? . f The coefficients ] and are two positive values called the Lame coefficients.The elastic smoothing term is suitable for the registration of images which do not have large geometric disparities. In the application to mammograms, it ensures that problem solutions are homeomorphisms. For the minimization problem to have a solution, it is assumed j that 0 is the Sobolev space r,  
 rs  
 .

 which minimizes an en-

8  %9 < : ;$=  %>-%?@/A    &,  A = 

F9%  SDTL  KE =



(1)





2.2. Discrete model

with homogeneous Dirichlet boundary conditions, ie under the constraint that %  BC for all pixels  on the boundary of . In equation (1), ADEIH A = denotes the  usual quadratic norm on the space G F 
 ( A A = J       +   K  ). =

2.2.1 Deterministic view point As in the previous section, we consider two images   and  and assume they are defined on the continuous domain 1 If t is a uGvIu -matrix, then wyx-zt|{ is equal to tq}b}d~t€b . If  is a smooth function mapping ‚ into the uv9u -matrix set, then the value of ƒ6„ …‡†ˆŠ‰ at a point ‹ of ‚ is a bi-dimensional vector having the ŒŽ component equal to ‘T’h!zU‹{”“ } ~T‘h•h'z–‹{Ž“  .

The energy in equation (1) is composed of two terms. The second term depends on images. The more similar 2

. We define a discrete sub-grid






of

O       G,\ TDDTDh





 $  @ : /%     &s%    "  @ %    &,%  "      i   &%  f

as follows

 g   

(4)



where is in  . We denote by    the point      of the grid . In a discrete approach, the image-matching goal is to map coordinates of image  which are located exclusively on onto those of   . The set of possible displacement fields   is defined as follows. Let us denote by  a finite set of real values and by  the set of   @/ 6 j   @B 6 -matrices which have components in  . Then,

2.2.2 Stochastic view point The minimization problem in model 2 can be interpreted in a stochastic framework using a Bayesian approach. Let us define a lattice 0 as follows.

0

\%W  %  ˆ%  -%  ˆ%      (5)     %    ˆZ    _ g     where the vector %  $  %?    -%    denotes the respective components of the ith row and jth column of matrices %  %   is a displacement and % . In this definition, vector   . the on the point    of In this framework, the inverse problem is stated as follows.

 which minimizes an en-

  %?@!  %?hi

8  %?

conditional to

#" $         @ %   S&,      -  i   %M :   &    &% 

(10) where 8  %6ˆ%  is defined by equations (6), (7) and (8). In this context, the problem in model 2 is equivalent to the following statistical problem.

(7) The definition of the term is obtained from the discretization of the strain energy of the linearized Elasticity (equation 2) using the finite difference method: the discretization of a %? (equation (3)) at a points    of is a : be the neighborhood system > defined on the lattice 0 by the relation: two sites     and @? &Ay of 0 are neighbors if and only if :

CBED A  ? &  A  @ AFA& A HGJI . Let us associate to >  the set K of cliques of order below L (a clique is a single element of 0 or a set of elements of 0 which are neighbors); see the illustration in figure 2. It can be shown that = is a Gibbs random field with respect to the clique set K [12]: the probability law of = (equation (10)) has an exponential form and the energy 8 can be written as a sum of potential defined on cliques of K . Hence, according to the Hammersley-Clifford theorem [6], we can deduce that the random field = is markovian with respect to the neighborhood system. In other

&,%?  "   h  &,%    " h

  ˆ, '   &)   %    &,% "   - i   Replacing a %? by a  %? in the expression of ; =  % ˆ %

 *)    +)    +) a &,%    "

 %     %    %   E@

(equation (2)), we obtain the following regularization term

 %M f



$  %     &,%  "      @  %    &,%    "      &%   $  ] @ : .%    &,%  "    @ %    &,%    "   

 &%  .-

 `1T is

5 -  1  (1  M  %  ˆ%   A    ˆ  -M 7:9 ;  &I8  %  -%  -

   6-('     -( '      -( '    - %?   

(9)

5 - 1  ,1  M  %  -%  -

68 7:9 ;  &<  %  -%  - 6 where is the partition function. Using the Bayes formula, it can be shown that the a posteriori law of  l 1 6,Š 1 

(6)

For this minimization problem to be equivalent to the one in model 1, the energy 8 is defined using a discretization of 8 (equation 1). More precisely, the definition of the term H  results from a discretization of the norm of F9 
 :

a % a % a %

 g j \6 DTDTDT  g

Let us assume that images and displacement components are observations of two random fields denoted  and 1 respectively. Observations of the field  belong to the probability space 2 = (where 2 is a gray level set). Observations of the field 1 belong to the probability space  q *3 . Let us also assume that there exists a law which describes the stochastic relations between image pairs  ˆ and which is of the following form      (+* @  1!n,1I -@.4 , where is a normalized zero-means Gaussian field and  1'n,1I  are displacement components. Let us also express an a priori law on displacements  1!n,1I  as follows

 

Model 2 Find an element of  ergy 8 of the following form:

\6 TDDTD

(8)

3

and decompose the lattice 0 into non-overlapping square : j|: blocks which are of size and of the form Figure 2. Cliques of order below L associated to the neighborhood system > . On the : first raw, the cliques of orders and . On the second raw, the cliques of order L .




5   A     ? +AP  =  k  'A    

  ˆ   @? &  AyG



 

 K    



  %?-hi (12)







/





 %

 5@ 4 A

1

 4

%



 





5 1

 6 7

@ L  q m

ˆ  ˆ   ’

(15)

8

; this set among the displacements of a finite set  will be defined in the next paragraph. 5 Step B.3. If the probability is increased, go back to Step B.2. and start scanning all blocks again. If not, go to step C.

, Step C. Update displacements % with %)@ L . If ? change the scale of the approximation set by updating ? with ? &€ and go back to step B. If not, take displacements % as an estimation of the MAP.

:9

The application of this algorithm to a mammogram pair is illustrated in figure 1.





0

3

The sites of the lattice 0 are scanned successively. On each site , the current displacement % is updated with the one which is a maximum of the probability of the displacement field 1 on the site conditional to displacement fields on other sites. After a scan of all the sites, the 5 variation of the a posteriori probability is evaluated. A 5 new scan is started if increased and the algorithm is stopped if not.

Let be an integer and assume that the dimension : : a power of and above . Let us fix ? in \ TDTDDh

.



3.1.1 Principles of the ICM

3.1.2 Multigrid implementation

 10




3.1. Multigrid ICM



(14)

Algorithm 1 Step A. Initialization: ? , %) . Step B. Iteration k: Displacements % and variable ? are fixed. Using the ICM on blocks, seek a maximum of 5 the probability - 1'n(1ITq % @/L|A  ˆ- for L varying in  :  Step B.1. Ll/ . : jk: Step B.2. Scan successively all the blocks of size (equation (13)). On each block , update displacements L with the ones which maximize the following conditional probability

The ICM algorithm of Besag [3] and the simulated annealing [6] are two techniques which enable estimations of MAP of a posteriori laws such as the one defined by equation (10). In this section, we present a multigrid algorithm which is based on principles of ICM (Iterated Conditional Mode) and enable the numerical resolution of problem in model 3. In addition, we briefly present a usual multigrid variational algorithm which was used in [14, 15, 16, 17] for the resolution of problems such as the one in model 1 . Both multigrid algorithms will be compared in section 4.



(*)+-,

2

3. Optimization Techniques



  %"$• # %  "'• & % g 

!

\ %|  b    G.0<!% 





$

(13)

where is the euclidean division of by . The sets   are embedded (ie   DDTD     ). In order to approximate the MAP 5 (equation 5 of the probability (10)), the probability is maximized progressively on the successive sets  from the coarsest set 5

). Maximization of (? ) to the finest one ( ? on each set   is done using ICM. The algorithm can be described as follows.

where     is the set of neighbors of the site     . In   addition, for all site set , conditional probabilities are of the following form

5  = !