Indirect Image Registration with Large Diffeomorphic ... - arXiv

Indirect Image Registration with Large Diffeomorphic Deformations∗ ‡ ¨ Chong Chen† and Ozan Oktem

arXiv:1706.04048v2 [math.NA] 15 Jun 2017

Abstract. We introduce a variational framework for indirect image registration where a template is registered against a target that is known only through indirect noisy observations, such as in tomographic imaging. The registration uses diffeomorphisms that transform the template through a (group) action. These diffeomorphisms are generated using the large deformation diffeomorphic metric mapping framework, i.e., they are given as solutions of a flow equation that is defined by a velocity field. We prove existence of solutions to this indirect image registration procedure and provide examples of its performance on 2D tomography with very sparse and/or highly noisy data. Key words. indirect image registration, shape theory, large deformations, diffeomorphisms, shape regularization, image reconstruction, tomography AMS subject classifications. 65F22, 65R32, 65R30, 94A12, 94A08, 92C55, 54C56, 57N25, 47A52

1. Introduction. Image registration (matching) is the task of transforming a template image so that it matches a target image. The need for determining such correspondence between images arises in many fields, such as in quality control in industrial manufacturing [13], various applications in remote sensing [12], face recognition [35, 25], robotic navigation [6], and medical imaging [27, 32]. This paper considers indirect image registration, i.e., the case when the template image is registered against a target that is known only through indirect noisy observations, such as in tomographic imaging. It makes use of the large deformation diffeomorphic metric mapping (LDDMM) framework for diffeomorphic image registration and thereby extends the indirect image registration scheme in [22], which uses (small) linearized deformations. Indirect image registration can be seen as part of an ongoing development of regularization theory where reconstruction and feature extraction steps are pursued simultaneously, such as in joint segmentation and reconstruction. In this case, the feature in question is the “shape” of the objects in the image and for spatiotemporal imaging, their temporal variability. See [22] for a survey and use cases of indirect image registration. There is an extensive literature on image registration where a template is registered by means of a diffeomorphism against a target so that their “shapes” agree. Image registration is then recast as the problem of finding a suitable diffeomorphism that deforms the template into the target image [10]. The underlying assumption is that the target image is contained in the orbit of the template under the group action of diffeomorphisms. This principle can ∗

Submitted to the editors DATE. ¨ Funding: The work by Chong Chen and Ozan Oktem has been supported by the Swedish Foundation for Strategic Research grant AM13-0049. Chen was also supported in part by the National Natural Science Foundation of China (NSFC) under the grant 11301520. † Department of Mathematics, KTH–Royal Institute of Technology, 100 44 Stockholm, Sweden; LSEC, ICMSEC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, People’s Republic of China ([email protected]). ‡ Department of Mathematics, KTH–Royal Institute of Technology, 100 44 Stockholm, Sweden ([email protected]). 1

¨ C. CHEN AND O. OKTEM

2

be stated in a very general setting where diffeomorphism acts on various image features, like landmark points, curves, surfaces, scalar images, or even vector/tensor valued images [34]. This extends readily to indirect image registration and here the approach is worked out for the case when one seeks to register scalar images. 2. Overview of paper and specific contributions. The main contribution is to develop a variational framework for indirect image registration (section 6) where a template is registered against a target that is known only through indirect noisy observations (data). The template is deformed by diffeomorphisms obtained from the LDDMM framework and these act on images through some group action as explained in section 5. An important theoretical topic is existence and uniqueness. Existence of solutions holds, but as with most other variational schemes for image registration, one cannot expect to have uniqueness due to lack of convexity (section 7). This theoretical investigation is followed by explicit calculations of the derivative of the objective functional in the variational formulation of the indirect registration problem (section 8). Numerical implementation, which relies on theory of reproducing kernel Hilbert space (section 4) is outlined in section 9. Section 10 contains some numerical experiments from tomography that show performance of indirect image registration. The results support the claim that shape information contained in the template has a strong regularizing effect, and especially so for noisy highly undersampled inverse problems. Furthermore, the experiments also suggest that the prior shape information does not have to be that accurate, which is important for the use cases mentioned in [22]. None of these claims are however proved formally by mathematical theorems. Finally, section 11 discusses various extensions, like spatiotemporal imaging. 3. Inverse problems, ill-posedness and variational regularization. The purpose of this section is to set some notations and concepts used throughout the paper. Image reconstruction. The goal in image reconstruction is to estimate some spatially distributed quantity (image) from indirect noisy observations (measured data). Stated mathematically, the aim is to reconstruct an image ftrue ∈ X from data g ∈ Y where (1)

g = T (ftrue ) + 4g.

Here, X (reconstruction space) is the vector space of all possible images, so it is a suitable Hilbert space of functions defined on a fixed domain Ω ⊂ Rn . Next, Y (data space) is the vector space of all possible data, which for digitized data is a subset of Rm . Furthermore, T : X → Y (forward operator ) models how a given image gives rise to data in absence of noise and measurement errors, and 4g ∈ Y (data noise component) is a sample of a Y –valued random element whose probability distribution is (data noise model ) assumed to be explicitly expressible in terms of T (ftrue ). Ill-posedness. A naive approach at reconstructing the true (unknown) image ftrue is to try to solve the equation T (f ) = g. Often there are no solutions to this equation since measured data is not in the range of T for a variety of reasons (noise, modeling errors, etc.). This is commonly addressed by relaxing the notion of a solution by considering (2)


min D T (f ), g .

f ∈X

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

3

The mapping D : Y × Y → R (data discrepancy functional ) quantifies the data misfit and a natural candidate is to choose it as a suitable affine transformation of the negative log likelihood of data. In such case, solving (2) amounts to finding maximum likelihood (ML) solutions, which works well when (2) has a unique solution (uniqueness) that depends continuously on data (stability). This is however not the case when (1) is ill-posed, ion which case one needs to use regularization that introduces stability, and preferably uniqueness, by making use of prior knowledge about ftrue . Variational regularization. The idea here is to add a penalization term to the objective functional in (2) resulting in a variational problem of the type h
i (3) min µR(f ) + D T (f ), g for some given µ ≥ 0. f ∈X

Such regularization methods have gained much attention lately, and especially so in imaging [26]. The functional R : X → R introduces stability, and perhaps uniqueness. Often it does so by encoding some a priori known regularity property of ftrue , e.g., assuming X ⊂ L 2 (Ω, R) and taking the L 2 -norm of the gradient magnitude (Dirichlet energy) is known to produce smooth solutions whereas taking the L 1 -norm of the gradient magnitude (total variation) yields solutions that preserve edges while smooth variations may be suppressed [8]. 4. Reproducing kernel Hilbert spaces. The diffeomorphic deformations constructed in the LDDMM framework (subsection 5.2.2) will make use of velocity fields that at each time point are contained in a RKHS. The gradient computations in section 8 also rely on this assumption. The short introduction to the theory of RKHS provided here gives the necessary background needed in subsequent sections. The theory of reproducing kernel Hilbert spaces was initialized in the 1940s for spaces of scalar-valued functions [1], it was later extended to spaces of functions with values in locally convex topological spaces [29]. It has lately gained significant interest due to applications in machine learning [28, 4, 20]. The starting point is an abstract Hilbert space V whose elements are functions defined on a fixed domain Ω ⊂ Rn that take values in a real Hilbert space X. Such a space is a RKHS if evaluation functionals δxa : V → R

 δxa (ν) := ν(x), a X for ν ∈ V are bounded for every x ∈ Ω and a ∈ X. One way to construct a RKHS is to specify a reproducing kernel function, which is an operator K : Ω × Ω → L (X, X) such that (a) K( · , x)(a) ∈ V for all x ∈ Ω and a ∈ X. (b) The reproducing property holds for K, i.e., if x ∈ Ω then



 ν(x), a X = ν, K( · , x)(a) V for any ν ∈ V and a ∈ X. L (X, X) above is the Banach space of bounded linear operators on X. A central result is that a Hilbert space V of X-valued functions is a RKHS if and only if it has a continuous reproducing kernel K : Ω × Ω → L (X, X). Consider vector fields in Rn , so X = Rn and V contains Rn -valued functions. It is a RKHS if it has a continuous positive definite reproducing kernel K : Ω × Ω → L (Rn , Rn ), which in

¨ C. CHEN AND O. OKTEM

4

turn can be represented by a continuous positive definite function K : Ω × Ω → Mn×n + . Here, Mn,m denotes the vector space of all (n × m) matrices and Mn,m denotes those matrices that + 2 n are positive definite. Furthermore, if V ⊂ L (Ω, R ) then one can relate the inner product on V to the L 2 -inner product by a straightforward calculation:  Z 
(4) hη, νiL 2 (Ω,Rn ) = ν, K( · , x) η(x) dx . Ω

V

Note also that k · kV is different from k · kL 2 (Ω,Rn ) . 5. Shape theory. Shape theory seeks to develop quantitative tools for studying shapes and their variability. The approach considered here is based on deformable templates and it can be traced back to work by D’Arcy Thompson in beginning of 1900’s [11] and it is now an active field of research [34, 30]. The starting point is to specify a shape space where the elements are image features whose shape one seeks to analyze. There is no formal definition of what constitutes an image feature, but intuitively it is a representation of the image that retains information relevant for its interpretation, so the choice of shape space is highly task dependent. Next, is to define a set of deformations whose elements map a shape space into itself, so deformations model shape variability. Clearly, the identity mapping preserves all aspects of shape and it is therefore the “smallest” deformation. A key step is to define a metric on the set of deformations, which can then be used to define a shape similarity measure between two elements in shape space. 5.1. Shape space. As already mentioned, the shape space defines which image feature to consider and there are many to choose from [2]. Image features that are natural for visual perception need to couple image scales in a structured way and these can be characterized axiomatically [18, 19]. For (indirect) image matching, it is important that image feature is deformable, i.e., it should be possible to act on it by means of a deformation. Examples of deformable image features are landmark points, curves, surfaces, grey-scale images, or even tensor valued images. Common for all these is that they are supported in a fixed open and bounded set Ω ⊂ Rn (image domain). This paper focuses on grey-scale images, i.e., our shape space X will be some suitable vector space of real-valued functions defined on Ω. It will also be equipped with an inner product, e.g., when X ⊂ L 2 (Ω, R). 5.2. Deformations. Deformations are operations that transform elements in a shape space. In our setting, deformations will be represented by mappings from the image domain Ω ⊂ Rn into Rn along with an action describing how they deform deformable objects. The set G of deformations should be rich enough to capture the shape variability arising in the application, yet restricted enough to allow for a coherent mathematical theory. 5.2.1. Basic requirements. Many applications in imaging require deformations that are not necessarily rigid, so clearly rigid body motions are too limited. Next, it should be possible to form new deformations by composing existing ones, so G should be closed under composition and the identity mapping becomes the natural ‘zero’ deformation. Furthermore, it should also be possible to reverse a deformation, i.e., G is closed under inversion. Taken together, this implies that G forms a group under the group law given by composition of functions. The

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

5

group structure also implies that a deformation transforms a deformable object by means of a group action. Finally, often it also makes sense to assume that deformable objects do not tear, fold or collapse, i.e., the deformation should preserve the topology. One class of non-rigid transformations that satisfy the above requirements is Diff(Ω), the group of diffeomorphisms supported in Ω (invertible functions that are continuously differentiable with a continuously differentiable inverse from the image domain Ω to itself). There are different group actions that can be used to define how a diffeomorphism deforms a deformable object, see, e.g., subsection 5.4 for the case when the deformable object is a scalar function representing a grey-scale image. For a computationally feasible theory, the diffeomorphisms need to be further restricted, i.e., one needs to consider suitable sub-groups of Diff(Ω). 5.2.2. Diffeomorphic deformations generated by flows. A natural starting point for constructing deformations is to perturb the identity map by a smooth displacement field [22]. The resulting map, which is referred to as a linearized deformation, is however invertible only when the displacement field is small. On the other hand, large displacements may yield mappings that are not invertible. Hence, the set of linearized deformations becomes a semigroup since it is closed under composition but its elements are not necessarily diffeomorphisms. The LDDMM framework addresses the issue of invertibility also for large deformations by considering transformations given as a composition of infinitesimally small linearized deformations. This adds a temporal dimension by replacing the fixed displacement field with a velocity field and the idea in LDDMM is to construct diffeomorphisms from a flow equation driven by such a velocity field. More precisely, start by considering a fixed Banach space V ⊂ C01 (Ω, Rn ) of vector fields that vanish on ∂Ω. Next, for some fixed p ≥ 0, define the set of velocity fields n o (5) VVp (Ω) := ν : [0, 1] × Ω → Rn : ν(t, ·) ∈ V and kνkV p (Ω) < ∞ V

with the associated norm Z kνkV p (Ω) := V

0

1/p

ν(t, ·) p dt . V

1

VVp (Ω)

becomes a normed space for p ≥ 1 (we will mainly consider the case p = 2). Clearly, Now, for a given velocity field ν ∈ VVp (Ω), consider the flow equation 
  d ψ(t, x) = ν t, ψ(t, x) dt (6) for x ∈ Ω and 0 ≤ s, t ≤ 1.  ψ(s, x) = x One can show that (6) has a unique solution if the velocity field is Lipschitz with respect to the Banach space variable [34, Theorem C.3]. For such velocity fields, define φνs,t : Ω → Rn as (7)

φνs,t := ψ(t, · )

where ψ is the unique solution to (6).

Then, [34, Proposition C.6] yields that (8)

φνs,t = φντ,t ◦ φνs,τ

and

(φνs,t )−1 = φνt,s

holds for 0 ≤ s, τ, t ≤ 1.

¨ C. CHEN AND O. OKTEM

6

Furthermore, by [34, Theorem C.7] the mapping φνs,t is a homeomorphism of Ω for all 0 ≤ t ≤ 1. One may then, for fixed p ≥ 1, define the corresponding set of deformations as n o (9) GV := φν0,1 : Ω → Rn : φν0,1 given by (7) with ν ∈ VVp (Ω) . An important question is when one can ensure that GV ⊂ Diff(Ω). Elements in GV are given by the flow equation in (6) and if these are to be diffeomorphisms, then φν0,1 ∈ Diff(Ω) for any ν ∈ VVp (Ω). This only holds if the velocity field ν(t, · ) ∈ V has further regularity, like Sobolev regularity of sufficiently high-order. The next definition gives the precise regularity notion that is needed. Definition 5.1. A Banach space V ⊂ C01 (Ω, Rn ) is admissible if it is (canonically) embedded in C01 (Ω, Rn ) with the k · k1,∞ norm, i.e., there exists a constant C > 0 such that kνk1,∞ ≤ CkνkV

for all ν ∈ V .

The k · k1,∞ –norm on C01 (Ω, Rn ) is defined as kνk1,∞ := kνk1 + kDνk∞ . Remark 5.2. An important case is when V is a RKHS with a symmetric and positive definite reproducing kernel. Then V is an admissible Hilbert space (section 4). Admissibility is an essential assumption within the LDDMM theory since it guarantees that the flow equation in (6) does generate diffeomorphisms. As shown in [34, Theorem 8.14], if V is admissible and p = 2, then the flow equation in (6) defines a unique isotopy (a flow of diffeomorphisms) for any ν ∈ VV2 (Ω), i.e., GV in (9) is a sub-group of Diff(Ω). 5.2.3. Metric on the diffeomorphisms. Our next task is to define a metric on GV by making use of the norm on VV1 (Ω) and admissibility will again play a key role. Begin by defining dG : GV × GV → R+ as (10)

dG (φ, ψ) :=

inf

ν∈VV1 (Ω) ψ=φ◦φν 0,1

kνkV 1 (Ω) V

for φ, ψ ∈ GV .

If V is admissible, then (10) defines a metric on GV which becomes a complete metric space under this metric [34, Theorem 8.15]. The associated norm is defined as (11)

kφkGV := dG (Id, φ)

for φ ∈ GV .

Furthermore, it turns out that there exists a minimizing velocity field in VV2 (Ω) realizing the distance between two given two arbitrary elements in GV , i.e., the infimum in (10) over VV1 (Ω) can be replaced by a minimum over the Hilbert space VV2 (Ω). The formal statement that follows from [34, Theorems 8.18 and 8.20] is that if V is an admissible Hilbert space, then for any φ, ψ ∈ GV there exists a velocity field ν ∈ VV2 (Ω) such that (12)

dG (φ, ψ) = kνkV 2 (Ω) V

and ψ = φ ◦ φν0,1 .

5.3. The shape functional. The action of the group GV on the shape space X allows us to use the metric in (10) to quantify shape similarity between objects in X. For this purpose, given a fixed template I ∈ X, we introduce the shape functional S( · ; I) : X → R+ as (13)

S(f ; I) := inf dG (φ, Id)2 = inf kφk2GV φ∈GV φ.f =I

φ∈GV φ.f =I

for f ∈ X.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

7

5.4. Group actions. Elements in the group GV of diffeomorphisms in (9) may act on images X ⊂ L 2 (Ω, R) by means of a group action that defines the operator W : GV × X → X.

(14)

The following two group actions are natural in the context of imaging. Geometric deformation. Deforms images without changing grey-scale values. This choice is suitable for inverse problems where shape, not texture, is the main image feature. (15)

W(φ, I) := I ◦ φ−1

for I ∈ X and φ ∈ GV .

Mass-preserving deformation. Deformation changes intensities but ensures mass is preserved. This choice is suitable for inverse problems where intensities are allowed to change while preserving mass. (16) W(φ, I) := D(φ−1 ) (I ◦ φ−1 ) for I ∈ X and φ ∈ GV . In the above, D(φ−1 ) denotes the Jacobian determinant of φ−1 . 6. Indirect image registration. Consider the inverse problem in (1) where the shape space X has elements representing grey-scale images defined over some fixed image domain Ω ⊂ Rn . Next, assume the true (unknown) target image ftrue ∈ X in (1) can be written as an admissible deformation of a given shape template I : Ω → R, i.e., (17)

ftrue = W(φ∗ , I)

on Ω for some φ∗ ∈ GV .

One can then register I onto ftrue by solving the following variational problem:  h
i   inf γS(f ; I) + µR(f ) + D T (f ), g f ∈X (18)  f = W(φ, I) on Ω for some φ ∈ G . V In the above, D : Y × Y → R is the data discrepancy functional introduced in (2), R : X → R is the regularity functional that encodes regularity properties of the target ftrue that are known before hand, and S( · ; I) : X → R+ is the shape functional defined in (13). Finally, γ, µ ≥ 0 are regularization parameters in which γ regulates the influence of the a priori shape information and µ regulates the a priori regularity information. The constraint in (18) simply states that the solution must be contained in the orbit of I under the group action. Furthermore, (19)

S( · ; I) ◦ W( · , I) = k · k2GV

on GV .

Hence, (18) can be reformulated as (20)

inf

φ∈GV

h i γkφk2GV + L ◦ W(φ, I)

where L : X → R is given by (21)

L(f ) := µR(f ) + D T (f ), g




for f ∈ X.

¨ C. CHEN AND O. OKTEM

8

6.1. Reformulating the variational problem. If GV is a Riemannian manifold, then one could solve (20) using a Riemannian gradient method [24]. It is however often more convenient to work with operators defined on vector spaces and GV lacks a natural vector space structure, e.g., the point-wise sum of two diffeomorphisms is not necessarily a diffeomorphism. This poses both mathematical and numerical difficulties. If GV can be equipped with a manifold structure, then one approach to tackle these difficulties is to reformulate (20) as an optimization over some suitable vector space, say the tangent space at the identity mapping. Such a reformulation is in our case based on first observing that diffeomorphisms in GV are generated by velocity fields in VV2 (Ω) through (6). There is therefore a natural vector space associated to GV , namely VV2 (Ω). More precisely, if V is admissible, as shown in [34, Theorem 11.2 and Lemma 11.3], then (20) is equivalent to i h 2 (22) inf γkνkV 2 (Ω) + L ◦ V(ν, I) ν∈VV2 (Ω)

V

where V : VV2 (Ω) × X → X is the deformation operator that is defined as (23)

V(ν, I) := W(φν0,1 , I)

where φν0,1 ∈ GV is given by (7).

To summarize, for admissible V we have that (24)

ν ∗ ∈ VV2 (Ω) solves (22) ⇐⇒ f ∗ := V(ν ∗ , I) solves (18).

Note also that V is often infinite dimensional Hilbert space, in which case (22) is a minimization over the infinite dimensional Hilbert space VV2 (Ω). 6.1.1. Extending to abstract setting. The above approach for indirect image registration is a straightforward extension of the LDDMM based framework for image registration. This can now be formulated in a much more abstract setting. Then, the group GV of diffeomorphisms in (9) is replaced by an abstract Lie group G of transformations that acts on the shape space X and the vector space V is the corresponding Lie algebra of G. This abstract formulation is straightforward in the finite dimensional setting as shown in [7], but it becomes more intricate if G is not finite dimensional as in our case in (9). The main culprit is to introduce a manifold structure on GV , which in the infinite dimensional setting is highly non-trivial. As an example, the GV given in (9) is not a Lie group [31, p. 7]. It is however a Fr´echet manifold that is regular, so it carries a smooth Riemannian structure and the associated Lie exponential map exp : V → GV is given as the flow at time t = 1 of the (constant) velocity field ν(t, · ) = ν. The Hilbert space V then plays the role of the tangent space of GV at the identity map, i.e., V is the corresponding “Lie algebra”. 6.1.2. PDE constrained formulation. Note that evaluating ν 7→ V(ν, I) requires solving the ordinary differential equation (ODE) in (6), so the variational problem in (22) is an ODE constrained optimization problem:   
 2  min γkνkV 2 (Ω) + L ◦ W φ(1, · ), I   V  ν∈VV2 (Ω)
d (25)  φ(t, · ) = ν t, φ(t, · ) on Ω and t ∈ [0, 1],   dt    φ(0, · ) = Id on Ω.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

9

For image registration the above can also be formulated as solving a PDE constrained optimization problem with a time dependent image, see, e.g., [16, eq. (1)]. As we show next, such a reformulation easily extends to the indirect image registration setting. To formulate (25) as a PDE constrained minimization, start by considering the time derivative of
f (t, x) := W φ(t, · ), I (x)

(26)

for x ∈ Ω and t ∈ [0, 1].

Since φ(t, · ) is given by ν ∈ VV2 (Ω) through (6), it should be possible to express the time derivative of the right-hand-side of (26) entirely in terms of f and ν. Furthermore, f (0, · ) = I

and f (1, · ) = W φ(1, · ), I




on Ω.

Hence, (25) can be re-stated as a PDE constrained minimization with objective functional
ν 7→ γkνk2V 2 (Ω) + L f (1, · ) .

(27)

V

Note that f (1, · ) above depends on ν since it depends on φ(1, · ) through (26) and φ(1, · ) depends on ν by the ODE constraint in (25). The precise form for the PDE depends on the choice of group action, see subsection 5.4 for a list of some natural group actions. Geometric group action. Let f : [0, 1] × Ω → R be given by (26) and consider the group action is given by (15). Then,
f t, φ(t, · ) = I

(28)

on Ω for t ∈ [0, 1] where φ solves (6).

Differentiating (28) w.r.t. time t and using (6) yields 


dφ(t, · ) ∂t f t, φ(t, · ) + ∇f t, φ(t, · ) , dt


 Rn


D

E = 0. = ∂t f t, φ(t, · ) + ∇f t, φ(t, · ) , ν t, φ(t, · ) n R

Since the above holds on Ω, it is equivalent to (29)

D E ∂t f (t, · ) + ∇f (t, · ), ν(t, · ) n= 0

on Ω for t ∈ [0, 1].

R

Hence, using the geometric group action in (22) yields the following PDE constrained formulation: h 
i 2  γkνk + L f (1, · ) min  VV2 (Ω)  2  ν∈VV (Ω) D E (30) ∂ f (t, · ) + ∇f (t, · ), ν(t, · ) = 0 on Ω and t ∈ [0, 1], t   Rn    f (0, · ) = I on Ω.

10

¨ C. CHEN AND O. OKTEM

Mass-preserving group action. Let f : [0, 1] × Ω → R be given by (26) and consider the group action is given by (16). Then,

(31) D φ(t, · ) f t, φ(t, · ) = I on Ω and t ∈ [0, 1] with φ solving (6). Differentiating (31) w.r.t. time t yields   


dφ(t, · ) = ∂t f t, φ(t, · ) + div f (t, · ) ν t, φ(t, · ) = 0. ∂t f t, φ(t, · ) + ∇ · f (t, · ) dt The last equality above makes use of (6) and the definition of the divergence operator. Since the above holds on Ω, it is equivalent to
(32) ∂t f (t, · ) + div f (t, · ) ν(t, · ) = 0 on Ω for t ∈ [0, 1]. Hence, using the mass-preserving group action in (22) yields the following PDE constrained formulation:  h
i 2  min + L f (1, · ) γkνk  2 VV (Ω)   ν∈VV2 (Ω)
(33) ∂t f (t, · ) + div f (t, · ) ν(t, · ) = 0 on Ω and t ∈ [0, 1],     f (0, · ) = I on Ω. Remarks and observations. The PDE constraint encodes both the generative model for the diffeomorphisms and their action on images, so changing any of these will result in a different PDE. An advantage with the ODE constrained formulation is that these two components, the generative model for the diffeomorphisms and their action on images, are explicit whereas in the PDE constrained formulation they are “hidden” in the PDE. Another potential advantage relates to computational feasibility. Within the LDDMM theory, there are several numerical methods based on various characterizations of minimizers to (22), see e.g., [34]. An advantage of using the PDE constrained formulation is that it may be easier to extend to other more complex models for deforming images or relax the smoothness assumptions on the velocity fields. Furthermore, the PDE constrained formulation may also be better suited in applications where one is interested in the actual image trajectory and not only the final deformed image. 7. Existence and uniqueness. A basic property that a reconstruction scheme should fulfill is existence of solutions, since otherwise it is of limited usefulness for reconstruction. In our case, this translates into proving existence of solutions for (18). By (24), existence of a minimizer to (18) over X follows from existence of a minimizer to (20) with L : X → R given as in (21). Next, by [34, Lemma 11.3], (20) is equivalent to (22), so it is enough to prove the existence of a minimizer to (22). T Theorem 7.1. Assume that V ⊂ C01 (Ω, Rn ) L 2 (Ω, Rn ) is an admissible Hilbert space
and f 7→ D T (f ), g and f 7→ R(f ) are lower semi-continuous on X. Then, the variational problem in (22) has a solution expressible as V(ν, I) ∈ X for some ν ∈ VV2 (Ω).

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

11

Proof. Assume first that we have proved that the objective functional in (22) is lower semicontinuous in VV2 (Ω) w.r.t. the weak topology. Then, V is separable so VV2 (Ω) is separable, which in turn implies that bounded balls are weakly compact. Hence, if {ν n }n ⊂ VV2 (Ω) is a minimizing sequence, then there exists a subsequence that weakly converges to an element ν ∈ VV2 (Ω). Now, by the lower semi-continuity of the objective functional, ν also minimizes the objective functional, which proves existence of solutions for (22). Hence, it remains to prove that the objective functional in (22) is lower semi-continuous on VV2 (Ω) w.r.t. the weak topology. Since V is admissible, by [34, Theorem 8.11] we have that ν 7→ φν0,1 is continuous from VV2 (Ω) into C0 (Ω, Rn ) in the weak topology on VV2 (Ω). Hence, ν 7→ V(ν, I) is continuous in
the weak topology on VV2 (Ω) for he group actions listed in subsection 5.4 and f 7→ D T (f ), g is by assumption lower semi-continuous on X, so ν 7→ L ◦ V(ν, I) is lower semi-continuous in the weak topology of VV2 (Ω), i.e., the objective functional in (22) lower semi-continuous. This concludes the proof. Another desirable property of a reconstruction scheme is uniqueness, i.e., the scheme in (22) renders a unique solution for given data. Unfortunately, ν 7→ V(ν, I) is non-convex, so there are no
guarantees that (22) has a unique solution even when both f → R(f ) and f → D T (f ), g are strictly convex. As with all reconstruction methods that involve solving non-convex optimization problems, there is always the issue of getting stuck in local extrema. One option to address this is to further restrict the set V of velocity fields, but it is highly non-trivial to work out conditions on V that would guarantee uniqueness. On a final note, formal uniqueness is actually not required from a regularization scheme even though it is a desirable property. More importantly, a variational reconstruction scheme (22) is a regularization if existence holds along with proof of stability and convergence [26, Chapter 3], see also the notion of “well-defined regularization method” in [14]. Preferably, one can also provide convergence rates and stability estimates. This is however yet to be done for the scheme in (22). 8. Derivative and gradient computations. To goal here is to compute the derivative and gradient of the objective functional in (22). To simplify the notation, E : VV2 (Ω) → R will henceforth denote this objective functional, i.e., (34)

E(ν) := γkνk2V 2 (Ω) + L ◦ V(ν, I) V

for ν ∈ VV2 (Ω)

where L : X → R given by (21) and V : VV2 (Ω) × X → X given by (23). The chain rule can be used to compute the derivative of E and its gradient. Since this functional is defined on the Hilbert space VV2 (Ω), the gradient associated with its Gˆateaux derivative will be based on the Hilbert structure of VV2 (Ω), which is the natural inner-product space for the optimization problem (22). Some of the explicit expressions assume V is an RKHS with a continuous positive definite reproducing kernel K : Ω × Ω → L (Rn , Rn ) represented by a function K : Ω × Ω → Mn×n (see section 4). +

¨ C. CHEN AND O. OKTEM

12

8.1. The deformation operator. Here the derivative of the deformation operator V : VV2 (Ω) × X → X in (23) is computed. For natural reasons, the expressions will depend on the choice of group action and here we consider the geometric and mass-preserving actions given in subsection 5.4. The starting point however is to consider the derivative of diffeomorphisms in GV w.r.t. the underlying velocity field. 8.1.1. Derivative of diffeomorphisms w.r.t. the velocity field. Each diffeomorphism in GV depends on an underlying velocity field in VV2 (Ω). The derivative of such a diffeomorphism w.r.t. the underlying velocity field can be explicitly computed and is given by the following theorem. Proposition 8.1. Let ν, η ∈ VV2 (Ω) and let φνs,t ∈ GV be given as in (7). Then, (35) (36)

Z t


 d ν+η
φs,t D φντ,t φνs,τ (x) η τ, φνs,τ (x) dτ (x) = d =0 s Z th


i−1 
 d ν+η −1 (φs,t ) (x) = − D φνs,τ φνt,s (x) η τ, φνt,τ (x) dτ d =0 s

for x ∈ Ω and 0 ≤ s, t ≤ 1. Proof. The first equality (35) follows directly from [15, Lemma 11.5] (see also [34, Theorem 8.10]) and the second, (36), follows directly from [15, Lemma 12.8] (see also [34, eq. (10.16)]) and (8). 8.1.2. Geometric group action. For the (left) geometric group action in (15), the deformation operator becomes (37)

V(ν, I) = I ◦ (φν0,1 )−1 = I ◦ φν1,0

where the last equality follows from (8). Its Gˆateaux derivative is given by the next result. Theorem 8.2. If I ∈ X is differentiable, then the deformation operator V( · , I) : VV2 (Ω) → X in (37) is Gˆ ateaux differentiable at ν ∈ VV2 (Ω) and its Gˆ ateaux derivative is Z 1 
ν ν ν (38) ∂V(ν, I)(η)(x) = − ∇(I ◦ φt,0 ) φ1,t (x) , η t, φ1,t (x) dt 0

Rn

for x ∈ Ω and η ∈ VV2 (Ω). Proof. The Gˆ ateaux derivative is a linear mapping ∂V(ν, I) : VV2 (Ω) → X given as  d d ν+η −1 ∂V(ν, I)(η)(x) := V(ν + η, I)(x) = I ◦ (φ0,1 ) (x) d d =0 =0 for η ∈ VV2 (Ω) and x ∈ Ω. Since I ∈ X ⊂ L 2 (Ω, R) is differentiable, the chain rule yields  
d ν+η −1
ν (39) ∂V(ν, I)(η)(x) = ∇I φ1,0 (x) , (φ0,1 ) (x) for x ∈ Ω. d =0 Rn

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

13

The second term in the scalar product on the right hand side of (39) is the derivative of a flow with respect to variations in the associated field. The following equation now follows from (36) in Proposition 8.1.  Z 1
h
ν
i−1 
 ν ν ν (40) ∂V(ν, I)(η)(x) = − ∇I φ1,0 (x) , D φ0,t φ1,0 (x) η t, φ1,t (x) dt. 0

Rn

To prove (38), consider first the chain rule:
D(φν0,t ◦ φν1,0 )(x) = D(φν0,t ) φν1,0 (x) ◦ D(φν1,0 )(x). Next, φν0,t ◦ φν1,0 = φν1,t , so
 −1 D(φν0,t ) φν1,0 (x) = D(φν1,t )(x)◦ D(φν1,0 )(x) , which in turn implies that, h h i−1
i−1 (41) D(φν0,t ) φν1,0 (x) = D(φν1,0 )(x) ◦ D(φν1,t )(x) . Inserting (41) into (40) yields  Z 1 h i−1 

 dt ∂V(ν, I)(η)(x) = − ∇I φν1,0 (x) , D(φν1,0 )(x) ◦ D(φν1,t )(x) η t, φν1,t (x) 0 Rn  Z 1 i−1 
  ∗ 
 h dt η t, φν1,t (x) =− D(φν1,0 )(x) ∇I φν1,0 (x) , D(φν1,t )(x) 0 Rn  Z 1  −1 
 ν ν ν dt =− ∇(I ◦ φ1,0 )(x), D(φ1,t )(x) η t, φ1,t (x) 0 Rn Z 1 
ν ν ν =− ∇(I ◦ φt,0 ) φ1,t (x) , η t, φ1,t (x) dt. 0

Rn

The last equality above follows from  ∗ 
 ∇(I ◦ φν1,0 )(x) = ∇(I ◦ φνt,0 ◦ φν1,t )(x) = D(φν1,t )(x) ∇(I ◦ φνt,0 ) φν1,t (x) . This concludes the proof of Theorem 8.2. 8.1.3. Mass-preserving group action. Here we consider the (left) mass-preserving group action in (16), so the deformation operator becomes


(42) V(ν, I) = D (φν1,0 )−1 I ◦ (φν0,1 )−1 = D(φν1,0 ) I ◦ φν1,0 where the last equality follows from (8). Its Gˆateaux derivative is given in the next result. Theorem 8.3. If I ∈ X is differentiable, then V( · , I) : VV2 (Ω) → X in (42) is Gˆ ateaux differentiable at ν ∈ VV2 (Ω) and its Gˆ ateaux derivative is  Z 1 h
ν

 i ν ν ν (43) ∂V(ν, I)(η)(x) = − div D φt,0 φ0,t η t, φ0,t I ◦ φ1,0 (x) D(φν1,0 )(x) dt 0

for x ∈ Ω and η ∈ VV2 (Ω).

¨ C. CHEN AND O. OKTEM

14

Proof. The Gˆ ateaux derivative is the linear mapping ∂V(ν, I) : VV2 (Ω) → X given as ∂V(ν, I)(η)(x) :=

d d ν+η ν+η
(x) (x) I ◦ φ1,0 V(ν + η, I)(x) = D φ1,0 d d =0 =0

for η ∈ VV2 (Ω) and x ∈ Ω. By (35), we have Z 0


 d ν+η D φνt,0 φν1,t (x) η t, φν1,t (x) dt φ1,0 (x) = d =0 1 Z 1


 =− D φνt,0 φν0,t ◦ φν1,0 (x) η t, φν0,t ◦ φν1,0 (x) dt 0   Z 1
ν

 ν ν D φt,0 φ0,t η t, φ0,t dt ◦ φν1,0 (x) = h ◦ φν1,0 (x) = − 0

where Z h(x) := −

1




 D φνt,0 (x) φν0,t (x) η t, φν0,t (x) dt.

0 ν+η (x) at  = 0 gives Taking a Taylor expansion of x 7→ φ1,0 ν ν φν+η 1,0 (x) = φ1,0 (x) +  (h ◦ φ1,0 )(x) + o()

(44)

which in turn implies


ν+η
D φ1,0 (x) = D φν1,0 (x) +  D φν1,0 (x) div(h) ◦ φν1,0 (x) + o(). Taking the derivative w.r.t.  and evaluating it at  = 0 yields

d ν+η
(45) D φ1,0 (x) = D φν1,0 (x) div(h) ◦ φν1,0 (x), d =0 which in turn implies that  D E
d ν+η I ◦ φ1,0 (x) = ∇I φν1,0 (x) , (h ◦ φν1,0 )(x) n . d R =0

(46) Hence



ν ∂V(ν, I)(η)(x) = D φ1,0 (x) div(h) ◦ φν1,0 ( · )I( · ) ◦ φν1,0 (x) E
D
+ D φν1,0 (x) ∇I φν1,0 (x) , (h ◦ φν1,0 )(x) n R 

ν ν = D φ1,0 (x) div I( · )h( · ) ◦ φ1,0 (x). This proves (43).

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

15

8.2. Matching functionals. The aim is to compute the derivative and gradient of a registration functional JI : VV2 (Ω) → R of the form in (22), i.e., (47)

JI (ν) := L ◦ V(ν, I)

for ν ∈ VV2 (Ω)

where L : X → R is a sufficiently regular, e.g., Gˆateaux differentiable, and V is given by (23). First, consider the abstract setting. As to be expected, in order to obtain more explicit expressions, one needs to choose the group action and the Hilbert space structures of X and V (for gradient calculations). This is done in the sections that follow. 8.3. The abstract setting. If the matching functional L : X → R in (21) and the deformation operator V( · , I) : VV2 (Ω) → X in (23) are both Gˆateaux differentiable, then by the chain rule one immediately obtains an expression for the Gˆateaux derivative of the registration functional JI in (47):

(48) ∂JI (ν)(η) = ∂L V(ν, I) ∂V(ν, I)(η) for ν, η ∈ VV2 (Ω). Deriving an expression for its gradient requires introducing Hilbert space structures. Assume VV2 (Ω) is a Hilbert space, e.g., one inherited from a Hilbert space structure on V . One can by Riesz representation theorem define the gradient of JI as the mapping ∇JI : VV2 (Ω) → VV2 (Ω) that is implicitly given by the relation

 (49) ∂JI (ν)(η) = ∇JI (ν), η V 2 (Ω) for ν, η ∈ VV2 (Ω). V

Likewise, if X is a Hilbert space, then the gradient of L is the mapping ∇L : X → X given implicitly by the relation

 (50) ∂L(f )(h) = ∇L(f ), h X for f, h ∈ X. Combining (48) and (49) gives



(51) ∇JI (ν), η V 2 (Ω) = ∂L V(ν, I) ∂V(ν, I)(η) . V

Next, using (50) to rewrite the right-hand-side of (51) gives D E


(52) ∇JI (ν), η V 2 (Ω) = ∇L V(ν, I) , ∂V(ν, I)(η) . V

X

In the above, ∂V(ν, I) : VV2 (Ω) → X is a linear map that has a VV2 (Ω)–adjoint ∂V(ν, I)∗ : X → VV2 (Ω), so D

   E (53) ∇JI (ν), η V 2 (Ω) = ∂V(ν, I)∗ ∇L V(ν, I) , η 2 . V

VV (Ω)

Hence, the VV2 (Ω)–gradient of JI in (47) is given by   (54) ∇JI (ν) = ∂V(ν, I)∗ ∇L V(ν, I) . More explicit expressions requires choosing a specific group action and Hilbert space structures on X and V (for gradient expressions). This is done in the following sections.

¨ C. CHEN AND O. OKTEM

16

8.3.1. Geometric group action. The deformation operator is here given by the geometric group action in (37). Furthermore, consider the case when X has the L 2 –Hilbert space structure and V is a RKHS. Under these conditions, it is possible to provide explicit expressions for the derivative and gradient of JI . Theorem 8.4. Let the registration functional JI : VV2 (Ω) → R be given as in (47) where I ∈ X is differentiable and L : X → R is Gˆ ateaux differentiable on X. Furthermore, assume GV acts on X by means of the (left) geometric group action (15). Then, the Gˆ ateaux derivative of JI is given as (48), where ∂V(ν, I)(η) : Ω → R is given as (38). Furthermore, if X ⊂ L 2 (Ω, R) and V is a RKHS with a reproducing kernel represented n×n by a symmetric and positive definite function K : Ω × Ω → M+ (see section 4), then the 2 VV (Ω)–gradient is Z

(55) ∇JI (ν)(t, x) = − D(φνt,1 )(y) ∇L V(ν, I) φνt,1 (y) K(x, y) · ∇(I ◦ φνt,0 )(y) dy Ω

for x ∈ Ω and 0 ≤ t ≤ 1. Proof. In the geometric group action, the deformation operator V( · , I) is given by (37) and its derivative is given by (38) in Theorem 8.2. Using chain rule, the first statement is readily proved. To derive the expression in (55) for the gradient, one needs to use the assumption that X ⊂ L 2 (Ω, R). An expression for ∂V(ν, I)(η) is given by (38), which then can be inserted into (52):   Z 1D

E
∇(I ◦ φνt,0 ) φν1,t ( · ) , η t, φν1,t ( · ) dt ∂JI (ν)(η) = ∇L V(ν, I) ( · ), − n =−

L 2 (Ω,R)

R

0

Z 1




∇L V(ν, I) ( · )∇(I ◦ φνt,0 ) φν1,t ( · ) , η t, φν1,t ( · )

0

 dt. L 2 (Ω,Rn )

The last equality makes use of the fact that the inner product in L 2 (Ω, Rn ) (square integrable Rn -valued functions) is expressible as the inner product in Rn followed by the inner product in L 2 (Ω, R) (square integrable real valued functions). Note also that

x 7→ ∇L V(ν, I) (x)∇(I ◦ φνt,0 ) φν1,t (x) ,
x 7→ η t, φν1,t (x) are both mappings from Ω into Rn , so the integrand in the last expression, which is given by the L 2 (Ω, Rn ) inner product, is well defined. Now, introduce the variable y := φν1,t (x), so x = φνt,1 (y) by (8), and use the fact that the inner product is symmetric. This gives Z 1D E

η(t, · ), D(φνt,1 )( · ) ∇L V(ν, I) φνt,1 ( · ) ∇(I ◦ φνt,0 )( · ) 2 dt. ∂JI (ν)(η) = − n L (Ω,R )

0

L (Rn , Rn ),

Finally, V is a RKHS with reproducing kernel K : Ω × Ω → so by (4) we get   Z 1 Z
˜ (t, x) dx dt ∂JI (ν)(η) = − η(t, · ), K( · , x) ν 0

Ω

V

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

17

˜ (t, · ) : Ω → Rn is defined as where ν

˜ (t, x) := D(φνt,1 )(x) ∇L V(ν, I) φνt,1 (x) ∇(I ◦ φνt,0 )(x). ν It is now possible to read off the expression for the VV2 (Ω)–gradient of JI : Z ∇JI (ν)(t, x) = −


˜ (t, y) dy, K(x, y) ν

Ω n×n and inserting the matrix valued function K : Ω × Ω → M+ representing the reproducing kernel yields (55). This concludes the proof of Theorem 8.4.

8.3.2. Mass-preserving group action. The deformation operator is here given by the mass-preserving group action in (42). Furthermore, consider the case when X has the L 2 – Hilbert space structure and V is a RKHS. Under these conditions, it is possible to provide explicit expressions for the derivative and gradient of JI . Theorem 8.5. Let the assumptions in Theorem 8.4 hold with the (left) mass-preserving group action (16) instead of the geometric one. Then, the Gˆ ateaux derivative of JI is given as in (48) with ∂V(ν, I)(η) : Ω → R is given as (43). Furthermore, the corresponding VV2 (Ω)– gradient is Z  
(56) ∇JI (ν)(t, x) = D(φνt,0 )(y) (I ◦ φνt,0 )(y)K(x, y) · ∇ ∇L V(ν, I) ◦ φνt,1 (y) dy Ω

for x ∈ Ω and 0 ≤ t ≤ 1. Proof. Under the mass-preserving group action, the deformation operator V( · , I) is given by (42) and its derivative is calculated by (43) in Theorem 8.3. By chain rule, the first statement is immediately proved. To prove (56), start by inserting the expression for ∂V(ν, I)(η) into (52): Z 1 ∂JI (ν)(η) =

  

ν


ν ν ν ν ∇L V(ν, I) , − D φ1,0 div D φt,0 φ0,t η t, φ0,t I ◦ φ1,0

dt

L 2 (Ω,R)

0

Z =

1

 


D φνt,0 I ◦ φνt,0 ∇ ∇L V(ν, I) ◦ φνt,1 dt, η t, ·

0

 . L 2 (Ω,Rn )

Now, (56) follows from combining the above with (4) and reading off the gradient term. 8.4. Indirect image registration using the L 2 -norm. Consider (1) where X ⊂ L 2 (Ω, R) and Y ⊂ L 2 (M, R) are Hilbert spaces endowed with the L 2 –Hilbert space structure. Here, M denotes a smooth manifold providing coordinates for data. Finally, assume that T is Gˆateaux differentiable and 4g is independent of T (ftrue ) and Gaussian. Given these assumptions, the natural registration functional is Z

2
2

(57) L(f ) := T (f ) − g Y = T (f )(y) − g(y) dy for f ∈ X, M

¨ C. CHEN AND O. OKTEM

18

for given data g ∈ Y . The corresponding registration functional JI : VV2 (Ω) → R is then

2


JI (ν) := T V(ν, I) − g

(58)

L 2 (M,R)

.

Note also that the Gˆ ateaux derivative of L and its corresponding gradient are given by D
E ∂L(f )(h) = 2 ∂T (f )∗ T (f ) − g , h X
∇L(f ) = 2∂T (f )∗ T (f ) − g

(59)

for f, h ∈ X.

Here, “∗” denotes the Hilbert space adjoint and ∂T is the Gˆateaux derivative of T . When T is linear, then ∂T (f ) = T in (59). Next, compute the Gˆateaux derivative of JI and its corresponding gradient under the two group actions. 8.4.1. Geometric group action. Here, the registration functional is given by (58) with the deformation operator given as in (37), i.e., JI (ν) =

(60)

Z 

2
T I ◦ φν1,0 (x) − g(x) dx.

M

Corollary 8.6. Let the assumptions in Theorem 8.4 hold and let JI : VV2 (Ω) → R be given as in (60) with T : X → Y Gˆ ateaux differentiable. Then (61)

D E
∗
∂JI (ν)(η) = 2 ∂T V(ν, I) T V(ν, I) − g , ∂V(ν, I)(η)

L 2 (Ω,R)

for ν, η ∈ VV2 (Ω), where ∂V(ν, I)(η) : Ω → R is given as (38). Furthermore, if X ⊂ L 2 (Ω, R) and V is a RKHS with a reproducing kernel represented by the symmetric and positive definite function K : Ω × Ω → Mn×n (see section 4), then the + 2 corresponding VV (Ω)–gradient is (62) ∇JI (ν)(t, x) Z
∗

= − D(φνt,1 )(y) ∂T V(ν, I) T V(ν, I) − g φνt,1 (y) K(x, y) · ∇(I ◦ φνt,0 )(y) dy Ω

for x ∈ Ω and 0 ≤ t ≤ 1. Proof. The proof of (61) follows directly from inserting the expression for ∂L in (59) into (48). Similarly, (62) follows directly from inserting the expression for ∇L in (59) into (55). 8.4.2. Mass-preserving group action. The registration functional is here given by (58) with the deformation operator given as in (42), i.e., (63)

Z    2 ν ν JI (ν) = T D(φ1,0 ) I ◦ φ1,0 (x) − g(x) dx. M

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

19

Corollary 8.7. Let the assumptions in Corollary 8.6 hold but with mass-preserving group action instead of the geometric one. Then, η 7→ ∂JI (ν)(η) is given as in (61) but with ∂V(ν, I)(η) : Ω → R is given as (43). Furthermore, the corresponding VV2 (Ω)–gradient is Z
(64) ∇JI (ν)(t, x) = 2 D φνt,0 (y) (I ◦ φνt,0 )(y)K(x, y)· Ω h i 
∗ 
ν ∇ ∂T V(ν, I) T V(ν, I) − g ◦ φt,1 (y) dy for x ∈ Ω and 0 ≤ t ≤ 1. The proof is analogous to the proof of Corollary 8.6. 8.5. The shape functional. Define Λ : VV2 (Ω) → R+ as Λ(ν) := γ dG (φν0,1 , Id)2 . Then, (13), (19), and (23), yields (65)

Λ(ν) = γkφν0,1 k2GV = γkνk2V 2 (Ω) V

for ν ∈ VV2 (Ω).

The Gˆateaux derivative and associated VV2 (Ω)–gradient of Λ is then (66)

∂Λ(ν)(η) = 2γhν, ηiV 2 (Ω) V

∇Λ(ν) = 2γν

for ν, η ∈ VV2 (Ω).

8.6. Gradient of the objective functional. The goal here is to provide expressions for the gradient of objective functional in (34) for the two group actions, geometric and masspreserving. 8.6.1. Geometric group action. The following generalization of [15, Theorem 16.2] provides an explicit expression for the gradient of the objective functional in (34). Corollary 8.8. Let the assumptions in Theorem 8.4 hold. Then, the VV2 (Ω)–gradient of ν 7→ E(ν) in (34) is Z (67) ∇E(ν)(t, x) = 2γν(t, x) −



D(φνt,1 )(y) ∇L V(ν, I) φνt,1 (y) K(x, y) · ∇(I ◦ φνt,0 )(y) dy

Ω

for x ∈ Ω and 0 ≤ t ≤ 1. Proof. We know that E = Λ + JI with Λ given as in (65) and JI given as in (47), so by linearity ∇E(ν) = ∇Λ(ν) + ∇JI (ν)

for ν ∈ VV2 (Ω).

V is, by assumption, a RKHS so we can use (55) in Theorem 8.4 to obtain an explicit expression for the gradient ∇JI : VV2 (Ω) → VV2 (Ω). Likewise, (66) gives an explicit expression for the gradient ∇Λ : VV2 (Ω) → VV2 (Ω). Inserting these expressions into the above equality yields the equality in (67).

¨ C. CHEN AND O. OKTEM

20

8.6.2. Mass-preserving group action. The following result is the version of Corollary 8.8 under the mass-preserving group action where the deformation operator is given as in (42). Corollary 8.9. Let the assumptions in Theorem 8.5 hold. Then, the VV2 (Ω)–gradient of ν 7→ E(ν) in (34) is Z  

(68) ∇E(ν)(t, x) = 2γν(t, x)+ D φνt,0 (y) (I ◦φνt,0 )(y)K(x, y)·∇ ∇L V(ν, I) ◦φνt,1 (y) dy Ω

for x ∈ Ω and 0 ≤ t ≤ 1. Proof. The VV2 (Ω)–gradient of ν 7→ E(ν) in (34) is given as ∇E(ν)(t, x) = 2γν(t, x) + ∇JI (ν)(t, x) for ν ∈ VV2 (Ω) and x ∈ Ω, 0 ≤ t ≤ 1, where ∇JI (ν) is the VV2 (Ω)–gradient of ν 7→ JI (ν). The claim in (68) now follows directly from Theorem 8.5. 9. Numerical implementation. The focus here is the numerical methods for solving (22). 9.1. Optimization strategies. Section 8 gives explicit expressions for the VV2 (Ω)–gradient of the objective functional in (22). Hence, one can use any optimization algorithm that makes use of gradient information, like gradient descent. The numerical implementation of such a gradient descent scheme is outlined in subsection 9.2. An alternative approach is to consider the gradient in the Hamiltonian form, which directly incorporates the dimension reduction that arises from the projection onto a finite dimensional subset of the RKHS V , see [34, section 11.6.2] (in particular algorithm 4 on [34, p. 275]) for further details. Yet another approach is the shooting method outlined in [34, section 11.6.4]. Here, one makes use of the characterization of a minimizer based on the momentum geodesic equations. The numerical implementation can make use of the natural finite dimensional counterpart obtained by projecting onto a finite dimensional subset of the RKHS V . Even though the above approaches may require less iterates than gradient descent, they have an important drawback. They rely on discretizing the vector fields in the RKHS V by control points that do not remain stationary over iterates, so the control points do not necessarily remain on a fixed regular grid. Thus, evaluating the velocity field at some point in time by convolving against the RKHS cannot easily make use of efficient Fast Fourier transform (FFT) based schemes. This becomes in particular troublesome when the shape space is scalar valued functions defined in 2D/3D. In contrast, the gradient descent scheme can be formulated so that the RKHS kernel used to evaluate the vector fields is given by a fixed set of control points. For this reason, the gradient descent scheme competes favorably regarding computational complexity against the shooting method. 9.2. Gradient descent. The gradient descent scheme for solving (22) is a first-order iterative optimization algorithm given as (69)

ν k+1 = ν k − α∇E(ν k ).

Here, α > 0 is the step-size and ∇E is the VV2 (Ω)–gradient of the objective functional E in (34).

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

21

Corollary 8.8 gives an expression for ∇E in (67) where the reproducing kernel function K : Ω × Ω → Mn×n is evaluated on points that do not move as iterates proceed. By choosing + a translation invariant kernel and points on a regular grid in Ω, we can use computationally efficient FFT-based schemes for computing the velocity field at each iterate (see [34, section 11.7.2]). This is necessary for numerically evaluating ∇E(x) for x ∈ Ω in the aforementioned grid and it is computationally more feasible than letting the kernel depend on points that move in time as in the shooting method. Computing diffeomorphic deformations. Evaluating ν 7→ ∇E(ν) in (69) requires computing the diffeomorphic deformations φνt,0 and φνt,1 . Since this is done repeatedly within an iterative scheme, one needs to have an efficient implementation for computing these diffeomorphisms. Note first that φν0,t solves the ODE in (70) forward in time: (
∂t ϕ(t, x) = ν t, ϕ(t, x) (70) for x ∈ Ω and 0 ≤ t ≤ 1. ϕ(0, x) = x Next, recall that φν1,t = (φνt,1 )−1 and φν1,t = φν0,t ◦ φν1,0 , so it can be written as the solution of the ODE in (71) backwards in time: (
∂t ϕ(t, y) = ν t, ϕ(t, y) for y ∈ Ω and 0 ≤ t ≤ 1. (71) ϕ(1, y) = y Integrating (70) and (71) w.r.t. time t gives φν0,t (x)

(72)

Z

t


ν τ, φν0,τ (x) dτ,

= Id + 0

φν1,t (x) = Id −

(73)

Z

1


ν τ, φν1,τ (x dτ

t

for 0 ≤ t ≤ 1 and x ∈ Ω. A numerical implementation needs to discretize time, which can be done by sub-dividing the time interval [0, 1] uniformly into N parts, i.e., ti = i/N for i = 0, 1, . . . , N . Then, from (72) we get φν0,t0 = φν0,0 = Id and (74)

φν0,ti ≈ Id +

i−1   1 X 1 ν(tj , · ) ◦ φν0,tj ≈ Id + ν(ti−1 , · ) ◦ φν0,ti−1 N N j=0

for i = 1, . . . , N . Furthermore, from (8) we know that φν0,ti = (φνti ,0 )−1 , which combined with (74), yields the following approximation:   1 for i = 1, . . . , N . (75) φνti ,0 ≈ φνti−1 ,0 ◦ Id − ν(ti , · ) N Similarly, from (73) we get φν1,tN = φν1,1 = Id and (76)

φν1,ti ≈ Id −

N   1 X 1 ν(tj , · ) ◦ φν1,tj ≈ Id − ν(ti+1 , · ) ◦ φν1,ti+1 N N j=i+1

¨ C. CHEN AND O. OKTEM

22

for i = N − 1, . . . , 0. Furthermore, from (8) we know that φνti ,1 = (φν1,ti )−1 , which combined with (76), yields the following approximation:   1 φνti ,1 ≈ φνti+1 ,1 ◦ Id + ν(ti , · ) for i = N − 1, . . . , 0. N

(77)

Remark 9.1. The approximation used when deriving (75) and (77) from (74) and (76), respectively is (78)



Id +

−1 1 1 ν(ti , · ) ≈ Id − ν(ti+1 , · ) N N

for i = 0, . . . , N − 1.

Gradient descent algorithm. The approximations in the aforementioned paragraph play a central role in the gradient descent algorithm that implements (69) for minimizing for finding a local extrema to (22). The numerical implementation is outlined in Algorithm 1 for the case with geometric group action. The version for mass-preserving group action can be obtained from Algorithm 1 after modifying two steps. The first is to replace step 8 with: k Update the Jacobian determinant Dφνti ,0 based on (75):    νk  Dφt ,0 ← 1 − 1 div ν k (ti , · ) Dφνt k ,0 ◦ Id − 1 ν k (ti , · ) i i−1 N N k k for i = 1, . . . , N , where Dφνt0 ,0 = Dφν0,0 = 1. The second is to replace step 10 with: Compute ∇E(ν k ) using (68) (use FFT based techniques for computing the kernel): ∇E(ν k )(ti , , · ) ← 2γν k (ti , , · ) Z  
k
k k
+ D φνti ,0 (y) I ◦ φνti ,0 (y)K( · , y) · ∇ ∇L V(ν k , I) ◦ φνti ,1 (y) dy Ω

for i = N, N − 1, . . . , 0. 10. Application to 2D tomography. Indirect registration with geometric group action is here applied to 2D parallel beam tomography with very sparse and highly noisy data. This is not a full evaluation, still it does illustrate the performance of indirect registration. 10.1. The indirect registration problem. Consider the shape space X = Lc2 (Ω, R) of real valued function defined on a domain Ω ⊂ R2 and diffeomorphisms act through a geometric group action. The goal is to register a template I ∈ X against a target that observed indirectly as in (1) where g ∈ Y = L 2 (M, R) with M denoting a fixed manifold of parallel lines in R2 (parallel beam data). The forward operator T : X → Y is here the 2D ray/Radon transform, see [21] for further details. The indirect registration scheme is given as the solution to (22) where L : X → R in (21) is given as

2 L(f ) := T (f ) − g L 2 (M,R) for f ∈ X.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

23

Algorithm 1 Gradient descent scheme for minimizing E in (34) with geometric group action 1: 2: 3: 4: 5: 6: 7:

Initialize: k ← 0. ti ← i/N for i = 0, 1, . . . , N . ν k (ti , · ) ← ν 0 (ti , · ) where ν 0 (ti , · ) is a given initial vector field. Error tolerance  > 0, step size α > 0, and maximum iterations K > 0. while ∇E(ν k ) >  and k < K do k Compute I ◦ φνti ,0 using (75) by 
 1 k k I ◦ φνti ,0 ← I ◦ φνti−1 ,0 ◦ Id − ν k (ti , · ) N k

k

8:

for i = 1, . . . , N , where I ◦ φνt0 ,0 = I ◦ φν0,0 = I. k Update the Jacobian determinant Dφνti ,1 based on (77):

9:

    νk Dφt ,1 ← 1 + 1 div ν k (ti , · ) Dφνt k ,1 ◦ Id + 1 ν k (ti , · ) i+1 i N N νk νk for i = N − 1, . . . , 0, where DφtN ,1 = Dφ1,1 = 1.
k Compute ∇L V(ν k , I) ◦ φνti ,1 using (77) by    

1 k k ∇L V(ν k , I) ◦ φνti ,1 ← ∇L V(ν k , I) ◦ φνti+1 ,1 ◦ Id + ν k (ti , · ) N for i = N − 1, . . . , 0, where


k k ∇L V(ν k , I) ◦ φνtN ,1 = ∇L V(ν k , I) ◦ φν1,1 = ∇L V(ν k , I) .

10:

Compute ∇E(ν k ) using (67) (use FFT based techniques for computing the kernel): ∇E(ν k )(ti , , · ) ← 2γν k (ti , , · ) Z

k k k − ∇L V(ν k , I) φνti ,1 (y) K( · , y) · ∇(I ◦ φνti ,0 )(y) D(φνti ,1 )(y) dy Ω

11:

for i = N, N − 1, . . . , 0. Update ν k : ν k (ti , · ) ← ν k (ti , · ) − α∇E(ν k )(ti , · )

for i = N, N − 1, . . . , 0. k ← k + 1. 13: end while k 14: return I ◦ φν ti ,0 for i = 1, . . . , N . 12:

¨ C. CHEN AND O. OKTEM

24

Hence, µ = 0 in (21), i.e., there is no additional regularization and (22) reduces to (79)



2


min γkνk2V 2 (Ω) + T I ◦ φν0,1 − g

ν∈VV2 (Ω)

L 2 (M,R)

V

 .

In the above, γ > 0 is a fixed regularization parameter that weights the need for minimal deformation against the need to register against the indirectly observed target. Next, consider a set V of vector fields that is a RKHS with a reproducing kernel represented by symmetric and positive definite Gaussian function K : Ω × Ω → M2×2 given as + (80)

 1  1 0 K(x, y) := exp − 2 kx − yk2 0 1 2σ

for x, y ∈ R2 .

The parameter σ > 0 above acts as a regularization parameter. 10.2. Phantoms and data acquisition protocol. All images in the shape space X, such as the target and template, are supported in a fixed rectangular image domain Ω = [−16, 16]× [−16, 16]. Data is obtained by first evaluating the 2D parallel beam ray transform on M , which consists of parallel lines whose directions are equally distributed within [0◦ , 180◦ ]. Next, we add Gaussian white noise at varying noise levels. The noise level in data is quantified in terms of the signal-to-noise ratio (SNR) expressed using the logarithmic decibel (dB) scale: (81)

"

#

gideal − µideal 2 SNR(g) = 10 log10 for g = gideal + 4g.

Y

4g − µnoise 2 Y

In the above, gideal := T (ftrue ) is the noise-free component of data and 4g ∈ Y is the noise component. Furthermore, µideal is the average of g ideal and µnoise is the average of 4g. Note here that computing the data noise level requires access to the noise-free component of data, which is the case for simulated data. 10.3. Reconstruction protocol. The implementation follows Algorithm 1. The choice of the regularization parameters γ, σ > 0 vary depending on the noise level in data. In general, we choose values that give the best visual agreement. Test suites 1 and 2 involve comparisons against direct reconstruction of the target from tomographic data. Here, we have used filtered back projection (FBP) and TV methods and values of the corresponding regularization parameters are again choose as to give the best visual agreement with the target. The reason for comparing against direct reconstruction is to assess the influence of a priori information contained in (17). This simply states that the unknown target can be obtained by a diffeomorphic deformation of a given template, so to some extent the indirect registration problem can be seen as a shape reconstruction problem. All reconstruction methods, including the implementation of Algorithm 1, are available in the Operator Discretization Library (http://github.com/odlgroup/odl). • FBP reconstruction is obtained using linear interpolation for the back projection and the Hamming filter at varying frequency scaling.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

25

• TV reconstruction is obtained by solving (3) with R as the total variation functional: Z R(f ) := ∇f (x) dx. Ω

The non-smooth minimization (3) is solved using the primal-dual method in [9]. 10.4. Test suites and results. The test suites seek to assess robustness and performance against different data sets, sensitivity of registration against choice of regularization parameters γ, σ, and impact of using a template with different topology from target. The targets include both single- and multi-object images. Test suite 1: Single-object indirect registration. The aim here is to investigate the performance if indirect registration against highly noisy sparse tomographic data from a single object target. The choice of template and target are similar to the direct registration test in [3, section 4.2]. Images in shape space are digitized using 64 × 64 pixels. The parallel beam tomographic data consists of 92 parallel projections along 10 uniformly distributed directions in [0◦ , 180◦ ]. Hence, in a fully digitized setting, the inverse problem in (1) amounts to solving for 4 096 unknowns from 920 linear equations. Hence, data is highly under-determined and highly noisy, noise level is SNR of 4.87 dB. When solving the indirect registration problem, the time interval [0, 1] for the flow of diffeomorphisms is sampled uniformly at N = 20 time points. The gradient step size is set to α = 0.02 and the regularization parameters are γ = 10−7 and σ = 6.0. The gradient descent was stopped after 200 iterations. Finally, comparison is against direct reconstructions from FBP (Hamming filter with frequency scaling 0.4) and TV (1000 iterations, regularization parameter µ = 3.0). The results, which are shown in Figure 1, show that indirect registration performs fine against a single-object target as long as the template has the topology (e.g., no holes). Test suite 2: Multi-object indirect registration. The previous test used a target that only contains one single object. Here we consider a multi-object target, namely a phantom consisting of six separately star-like objects with grey-values in [0, 1], which is digitized using 438 × 438 pixels. The choice of target is similar to the direct registration test in [5]. The set of data is taken as parallel beam tomographic projections along only 6 directions that are uniformly sparse distributed in [0, 180◦ ]. At each sampled angle, a set of uniformly spacing 620 parallel projections is produced. Hence, the inverse problem in (1) amounts to solving for 191 844 unknowns from 3 720 linear equations. Besides being highly underdetermined, data is also highly noisy with a noise level of SNR of 4.75 dB, resulting in a severely ill-posed inverse problem. As in test suite 1, the time interval [0, 1] for the flow of diffeomorphisms is sampled uniformly at N = 20 time points, the gradient step size is set to α = 0.04, and the regularization parameter is γ = 10−7 . On the other hand, the other regularization parameter regulating the width of the RKHS kernel is set to σ = 2.0 and the gradient descent was stopped after 200 iterations. Finally, comparison is against direct reconstructions from FBP (Hamming filter with frequency scaling 0.4) and TV (1000 iterations, regularization parameter µ = 1.0). The results, which are shown in Figure 2, show that indirect registration performs fine also against a multi-object target as long as the template has the same number of objects.

¨ C. CHEN AND O. OKTEM

26

γ σ 1.0 2.0 2.5 3.0 4.0 8.0

10−7

10−5

10−3

10−1

10

0.8958 26.69 0.9039 26.96 0.9018 26.90 0.9007 26.90 0.8992 26.88 0.8305 24.15

0.8958 26.69 0.9039 26.96 0.9018 26.90 0.9007 26.90 0.8992 26.88 0.8305 24.15

0.8957 26.69 0.9038 26.96 0.9017 26.89 0.9006 26.89 0.8992 26.87 0.8307 24.16

0.8820 26.18 0.8958 26.72 0.8964 26.74 0.8968 26.77 0.8979 26.77 0.8438 24.84

0.6800 17.82 0.6967 18.45 0.7057 18.71 0.7164 19.03 0.7426 19.79 0.7960 21.27

Table 1: Test suite 3: SSIM and PSNR values of indirectly registered images as compared to target for varying values of the regularization parameters γ and σ, see Figure 3 for selected images. Each table entry has two values, the upper is the SSIM and the bottom is the PSNR.

Test suite 3: Sensitivity w.r.t. choice of regularization parameters. Choosing the regularization parameters is a well-known issue in most regularization schemes. Even though there is some theory for how to do this, in practice it is often chosen using heuristic methods and especially so when data is highly noisy and/or under-sampled. A natural question is therefore to empirically investigate the sensitivity of the indirect matching against variations in the regularization parameters γ and σ that enter in (79) and (80), respectively. Here we consider the well-known Shepp-Logan phantom consists of 10 ellipsoids with greyvalues in [0, 1]. Images in shape space are digitized using 256 × 256 pixels. The parallel beam tomographic data consists of 362 parallel projections along 10 uniformly distributed directions in [0◦ , 180◦ ]. Hence, in a fully digitized setting, the inverse problem in (1) amounts to solving for 65 536 unknowns from 3 620 linear equations. Besides being highly under-determined, data is also highly noisy with a noise level of SNR of 7.06 dB, resulting in a severely ill-posed inverse problem. As in test suite 1, the time interval [0, 1] for the flow of diffeomorphisms is sampled uniformly at N = 20 time points, the gradient step size is set to α = 0.02, and the regularization parameter is γ = 10−7 . The regularization parameters (γ, σ) are varied and each choice results in a different registered image. Each of these are then matched against the target and the matching is quantitatively assessed using structural similarity (SSIM) [33] and peak signal-to-noise ratio (PSNR) [17] as figure of merits. The SSIM and PSNR values are tabulated in Table 1, see also Figure 3 for some selected reconstructions. The results show that SSIM and PSNR values are quite similar for γ = 10−7 , 10−5 , 10−3 and σ = 2.0, 3.0.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

27

Test suite 4: Topology of the template. This test investigates the influence of a template with a topology that differs from the target. The test suite involves two tests, one where the the template lacks an object when compared to the target and the other where there is an additional object in the template that does not exist in the target. Since registration is through diffeomorphisms, we do not expect the final registered image to have the same number of objects as the target. In both cases, the template and the target are modifications of the Shepp-Logan phantom used in test suite 3 and tomographic data is generated following the same protocol as in test suite 3, albeit with slightly differing noise levels (7.06 dB and 6.46 dB, respectively). Finally, apart from the number of iterations which here is 1000, indirect registration in both cases was performed using the same parameter setting as in test suite 3. The first test has a template with one less object than the target, the second test considers the reverse situation, i.e., a template with one extra object. Results are shown in Figure 4. It is clear that the final indirectly registered template retains the deficiency, which is to be expected since diffeomorphic registration using geometric group action can never introduce or remove an object that is not in the template. This also points to the importance of having a template with the correct topology when using diffeomorphic (indirect) registration with a geometric group action. 11. Extensions. Our main motivation is to use indirect registration as part of spatiotemporal image reconstruction. There are many ways to do this, below is one formulation and for further discussion the reader may consult [22, section 12.3]. "Z  Z  # 1 t

2
ν

ν(s, ·) ds + L ◦ W φ0,t , I dt . γ (82) inf V ν∈VV2 (Ω) I∈X

0

0

The diffeomorphism φν0,t : Ω → Ω is given by ν through (7), W : GV × X → X is given by the group action, and L : X → R is as in (21). 11.1. Shape based reconstruction. It is possible to view indirect registration as a variational reconstruction scheme that makes use of a priori shape information encoded by the template. The tests in section 10 indicate that if the template has the correct topology, then indirect registration performs fairly well. Furthermore, it is also robust against choice of template and regularization parameters. When using geometric group action, indirect registration is mostly useful as a reconstruction scheme in imaging problems where the aim is to recover the shape, so intensity variations are of little, or no, importance. One example of such an imaging problem in applications is when electron tomography (ET) is used to image the internal 3D structures at nano-scale of a specimen [5]. This is however a rather limited category of inverse problems and many of the medical imaging applications do not fall under this category. One approach to address this is to consider mass-preserving group action instead of the geometric one. Another is to keep the geometric group action, but let it act on the intensity map. This leads to the metamorphosis that we briefly consider in subsection 11.2. 11.2. Metamorphosis. The LDDMM approach with geometric deformations only moves intensities, it does not change them. Metamorphosis extends LDDMM by allowing the diffeo-

¨ C. CHEN AND O. OKTEM

28

morphisms to also act on intensities. This is achieved by coupling the flow equation in (6) to a similar flow equation for the intensities. More precisely, since one needs to consider time dependent images, we introduce X as the set of time dependent images that are square integrable in time and contained in X at all time points. Hence, if ζ ∈ X then ζ(t, · ) ∈ X and s Z 1

ζ(t, · ) 2 dt < ∞. kζkX := X 0

The Hilbert space structure of X induces a Hilbert space structures on X . Next, introduce I : [0, 1] × Ω → R as the solution to   d I(t, x) = ζ t, φ(t, x)
(t, x) ∈ [0, 1] × Ω, dt (83)  I(0, x) = I(x) x ∈ Ω. In the above, φ is the diffeomorphism that solves (6), I ∈ X is the template, and ζ ∈ X is a “source” term that is contained in the space X of time dependent images introduced above. Bearing in mind the above, metamorphosis based image registration in [23] easily extends to the indirect setting, which in turn extends the LDDMM based indirect registration in (22): h 
i 2 2  inf γkνk + τ kζk + L ◦ V ν, I(1, · )  X VV2 (Ω) ν∈V 2 (Ω)  V    ζ∈X (84)
d   I(t, x) = ζ t, φν0,t (x) (t, x) ∈ [0, 1] × Ω,   dt    I(0, x) = I(x) x ∈ Ω. Here, L : X → R is given by (21) and V : VV2 (Ω) × X → X is given by (37) (geometric group action). Next, note that the velocity field ν ∈ VV2 (Ω) in the objective also generates, through (7), the diffeomorphism φν0,t : Ω → Ω in the ODE constraint. Furthermore, I(1, · ) ∈ X in the objective functional depends on the optimization variable ζ ∈ X through the ODE constraint. Finally, as in subsection 6.1.2 for LDDMM, one can re-phrased (84) as a PDE constrained problem:  h
i 2 2  inf γkνk + τ kζk + L f (1, · )  X VV2 (Ω)  2   V (Ω) ν∈V ζ∈X D E (85) ∂ f (t, x) + ∇ f (t, x), ν(t, x) = ζ(t, x) (t, x) ∈ [0, 1] × Ω,  t x   Rn   f (0, x) = I(x) x ∈ Ω. REFERENCES [1] N. Aronszajn, Theory of reproducing kernels, Transactions of the American Mathematical Society, 68 (1950), pp. 337–404.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

29

[2] A. I. Awad and M. Hassaballah, eds., Image Feature Detectors and Descriptors: Foundations and Applications, vol. 630 of Studies in Computational Intelligence, Springer-Verlag, 2016. [3] M. Bauer, S. Joshi, and K. Modin, Diffeomorphic density matching by optimal information transport, SIAM Journal on Imaging Sciences, 8 (2015), pp. 1718–1751. [4] A. Berlinet and C. Thomas-Agnan, Reproducing Kernel Hilbert Spaees in Probability and Statisties, Springer-Verlag, 2004. [5] E. Bladt, D. M. Pelt, S. Bals, and K. J. Batenburg, Electron tomography based on highly limited data using a neural network reconstruction technique, Ultramicroscopy, 158 (2015), pp. 81–88. [6] F. Bonin-Font, A. Ortiz, and G. Oliver, Visual navigation for mobile robots: A survey, Journal of Intelligent and Robotic Systems, 53 (2008), pp. 263–296. [7] M. Bruveris and D. D. Holm, Geometry of image registration: The diffeomorphism group and momentum maps, in Geometry, Mechanics, and Dynamics: The Legacy of Jerry Marsden, C. D. E., D. D. Holm, G. Patrick, and T. Ratiu, eds., vol. 73 of Fields Institute Communications, Springer-Verlag, 2015, pp. 19–56. [8] M. Burger and S. Osher, A guide to the TV zoo, in Level Set and PDE Based Reconstruction Methods in Imaging, M. Burger and S. Osher, eds., vol. 2090 of Lecture Notes in Mathematics, Springer-Verlag, 2013, pp. 1–70. [9] A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2011), pp. 120–145. [10] G. E. Christensen, R. D. Rabbitt, and M. I. Miller, Deformable template model using large deformation kinematics, IEEE Transactions on Image Processing, 5 (1996), pp. 1435–1447. [11] T. D’Arcy, On Growth and Form, Cambridge University Press, New York, 1945. [12] S. Dawn, V. Saxena, and B. Sharma, Remote sensing image registration techniques: A survey, in Image and Signal Processing. Proceedings of the 4th International Conference, ICISP 2010, TroisRivi`eres, QC, Canada, June 30-July 2, 2010, A. Elmoataz, O. Lezoray, F. Nouboud, D. Mammass, and J. Meunier, eds., vol. 6134 of Lecture Notes in Computer Science, 2010, pp. 103–112. [13] C. Demant, B. Streicher-Abel, and C. Garnica, Industrial Image Processing: Visual Quality Control in Manufacturing, Springer-Verlag, 2013. [14] M. Grasmair, Generalized Bregman distances and convergence rates for non-convex regularization methods, Inverse Problems, 26 (2010), p. 115014. [15] U. Grenander and M. Miller, Pattern Theory. From Representation to Inference, Oxford University Press, 2007. [16] Y. Hong, S. Joshi, M. Sanchez, M. Styner, and M. Niethammer, Metamorphic geodesic regression, in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012 15th International Conference, Nice, France, October 1-5, 2012, Proceedings, Part II, N. Ayache, P. Delingette, H. Golland, and K. Mori, eds., vol. 7512 of Lecture Notes in Computer Science, Springer-Verlag, 2012, pp. 197–205. [17] Q. Huynh-Thu and M. Ghanbari, Scope of validity of PSNR in image/video quality assessment, Electronics Letters, 44 (2008), pp. 800–801. [18] T. Lindeberg, Generalized axiomatic scale-space theory, Advances in Imaging and Electron Physics, 178 (2013), pp. 1–96. [19] T. Lindeberg, Image matching using generalized scale-space interest points, Journal of Mathematical Imaging and Vision, 52 (2015), pp. 3–36. [20] K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT Press, Cambridge, MA, USA, 2012. ¨ bbeling, Mathematical Methods in Image Reconstruction, Mathematical Mod[21] F. Natterer and F. Wu eling and Computation, Society for Industrial and Applied Mathematics, 2001, http://epubs.siam. org/doi/book/10.1137/1.9780898718324. ¨ [22] O. Oktem, C. Chen, N. O. Domanic ¸ , P. Ravikumar, and C. Bajaj, Shape based image reconstruction using linearised deformations, Inverse Problems, 33 (2017), p. 035004. [23] C. L. Richardson and L. Younes, Metamorphosis of images in reproducing kernel Hilbert spaces, Advances in Computational Mathematics, 42 (2016), pp. 573—603. [24] W. Ring and B. Wirth, Optimization methods on Riemannian manifolds and their application to shape space, SIAM Journal of Optimization, 22 (2012), pp. 596–627. ¨ nes¸, and A. Cavallaro, Automatic analysis of facial affect: A survey of regis[25] E. Sariyanidi, H. Gu

30

[26] [27]

[28] [29] [30] [31]

[32] [33] [34] [35]

¨ C. CHEN AND O. OKTEM tration, representation, and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 37 (2015), pp. 1113–1133. O. Scherzer, M. Grasmair, H. Grossauer, M. Haltmeier, and F. Lenzen, Variational Methods in Imaging, vol. 167 of Applied Mathematical Sciences, Springer-Verlag, New York, 2009. J. A. Schnabel, M. P. Heinrichb, B. W. Papiez˙ c, and M. J. Brady, Advances and challenges in deformable image registration: From image fusion to complex motion modelling, Medical Image Analysis, 33 (2016), pp. 145–148. B. Scholkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, Cambridge, MA, USA, 2001. L. Schwartz, Sous-espaces Hilbertiens d’espaces vectoriels topologiques et noyaux associ´es (noyaux reproduisants), Journal d’Analyse Math´ematique, 13 (1964), pp. 115–256. ´ and L. Younes, Shape spaces, in Handbook of Mathematical Methods in Imaging, S. Otmar, A. Trouve ed., Springer-Verlag, 2011, ch. 30, pp. 1309–1362. F.-X. Vialard, Computational anatomy from a geometric point of view. Presentation at the Geometry, Mechanics and Control, 8th International Young Researchers workshop 2013 held in Barcelona, December 2013. M. A. Viergever, J. B. Maintz, S. Klein, K. Murphy, M. Staring, and J. P. Pluim, A survey of medical image registration, Medical Image Analysis, 33 (2016), pp. 140–144. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Transactions on Image Processing, 13 (2004), pp. 600–612. L. Younes, Shapes and Diffeomorphisms, vol. 171 of Applied Mathematical Sciences, Springer-Verlag, 2010. ´ and J. Flusser, Image registration methods: a survey, Image and Vision Computing, 21 B. Zitova (2003), pp. 977–1000.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

Template

t = 0.25

t = 0.5

t = 0.75

t = 1.0

Target

Data at 0◦

FBP

TV

31

Figure 1: Test suite 1: Single-object indirect registration. The template (top left) is deformed by a diffeomorphism to match the target (rightmost, second row). Images labeled with time t show the final flow of diffeomorphic deformations obtained from indirect registration with t = 1.0 denoting the final (indirectly) registered template. Data is highly noisy and undersampled, bottom leftmost image shows data at angle 0◦ (blue smooth curve is noise-free data, red jagged curve is noisy data). The FBP and TV reconstructions are for comparison.

¨ C. CHEN AND O. OKTEM

32

Template

t = 0.25

t = 0.5

t = 0.75

t = 1.0

Target

Data at 0◦

FBP

TV

Figure 2: Test suite 2: Multi-object indirect registration. The template (top left) is a phantom consisting of 6 separately triangle-like objects. Images labeled with time t show the final flow of diffeomorphic deformations obtained from indirect registration with t = 1.0 denoting the final (indirectly) registered template. Data is highly noisy and under-sampled, bottom leftmost image shows data at angle 0◦ (blue smooth curve is noise-free data, red jagged curve is noisy data). For comparison we also show the FBP and TV reconstructions.

INDIRECT IMAGE REGISTRATION WITH LARGE DIFFEOMORPHIC DEFORMATIONS

γ = 2.0, σ = 10−7

γ = 2.0, σ = 10−5

γ = 2.0, σ = 10−3

γ = 3.0, σ = 10−7

γ = 3.0, σ = 10−5

γ = 3.0, σ = 10−3

γ = 8.0, σ = 10−7

γ = 3.0, σ = 10

γ = 8.0, σ = 10

Template

33

Target

Figure 3: Test suite 3: Sensitivity w.r.t. choice of regularization parameters γ and σ. Images show final registered result (i.e., t = 1) for different values of these parameters. See Table 1 for a more quantitative comparison that includes a wider range of parameter values.

¨ C. CHEN AND O. OKTEM

34

Template

t = 0.25

t = 0.50

t = 0.75

t = 1.0

Target

Template

t = 0.25

t = 0.50

t = 0.75

t = 1.0

Target

Figure 4: Test suite 4: Topology of the template. First two rows is the case when the template lacks one object as compared to target, the two following rows is the case when template has one extra object. Images labeled with time t show the final flow of diffeomorphic deformations obtained from indirect registration with t = 1.0 denoting the final (indirectly) registered template.