Generalized Rigid and Generalized Affine Image Registration and Interpolation by Geometric Multigrid

Transcription

1 Generaized Rigid and Generaized Affine Image Registration and Interpoation by Geometric Mutigrid Stephen L. Keeing * Abstract. Generaized rigid and generaized affine registration and interpoation obtained by finite dispacements and by optica fow are here deveoped variationay and numericay as we as with respect to a geometric mutigrid soution process. For high order optimaity systems under natura boundary conditions, it is shown that the convergence criteria of [10] are met. Specificay, the Gaerkin formaism is used together with a muti-coored ordering of unknowns to permit vectorization of a symmetric successive over-reaxation on image processing systems. The geometric mutigrid procedure is situated as an inner iteration within an outer Newton or agged diffusivity iteration, which in turn is embedded within a pyramida scheme that initiaizes each outer iteration from predictions obtained on coarser eves. Differences between resuts obtainabe by finite dispacements and by optica fows are eucidated. Specificay, independence of image order can be shown for optica fow but in genera not for finite dispacements. Aso, whie autonomous optica fows are used in practice, it is shown expicity that finite dispacements generate a broader cass of registrations. This work is motivated by appications in histoogica reconstruction and in dynamic medica imaging, and resuts are shown for such reaistic exampes. 1 Introduction Separate images of reated objects are compared or aigned by at east impicity conceiving a correspondence between ike points. When an expicit coordinate transformation connecting ike points is constructed, images are said to be registered. When a parameterized transformation permits images to be morphed one to the other, images are said to be interpoated. In this work, a registration or interpoation method is said to be generaized rigid or generaized affine if it seects a rigid or an affine transformation respectivey when one fits the given images, and otherwise the degree to which the method produces a rigid or an affine transformation can be reguated. These concepts are here formuated variationay by penaizing the departure of a transformation from being rigid or from being affine. The motivation for considering generaizations of rigid or affine transformations ies in their appicabiity in two important categories of biomedica imaging. First, because of the ubiquity of rigid objects in the human body, generaized rigid registration and interpoation are of particuar interest, for instance, to faciitate medica examination of dynamic imaging data particuary by increasing the tempora resoution. Secondy, generaized affine registration and interpoation are of particuar interest, for instance, for object reconstruction from histoogica data since histoogica sections may be affine deformed in the process of sicing. Both of these appications are considered together in this work with respect to both finite dispacements and optica fow since these concepts are so interreated and they share so much in common in the variationa and numerica formuation as we as in the geometric mutigrid soution process deveoped here. Since both appications invove the processing of sets as opposed to pairs of images, the proposed concepts are aso formuated for image sequences. Generaized rigid registration was first considered in [1] where it was shown that that this concept is effectivey imited to optica fow and that it is a more natura formuation of eastic reguarization than is reguarization of finite dispacements by inearized eastic potentia energy [8] [6]. Generaized affine registration is impemented for both optica fow and for finite dispacements using second order penaties variationay [7] [7] but utimatey reaized * Institut für Mathematik und Wissenschaftiches Rechnen, Kar-Franzens-Universität Graz, Heinrichstraße 36, 8010 Graz, Austria; emai: stephen.keeing@uni-graz.at; te: ; fax:

2 by empoying natura boundary conditions. The combination of second order penaties together with a ocay penaized departure from rigidity for finite dispacements has been impemented recenty in [4]. A generaized affine method is aso reaizabe in the spine approach of [7] as we as with the diffeomorphic approach of [9], which aso incudes reated fows as we as a muti-scae framework. In the present work, as we as in [1], it is desired to aow optica fows to be ess reguar and their associated registrations to be not necessariy diffeomorphic, for instance, to aow for object excision. The consistent use of natura boundary conditions in a formuations of the present work creates a chaenge for the numerica soution of discretized optimaity systems as is evident from the geometric mutigrid formuation deveoped in [0] and adapted here for image registration and interpoation probems. As pointed out in [0], agebraic mutigrid is generay imited to discretizations resuting in M-matrices [8], which is not possibe for the high order probems considered here. On the other hand, the geometric mutigrid foundation of [10] is we suited to the probems at hand. Specificay, the Gaerkin formaism is used together with a muticoored ordering of unknowns to permit vectorization of a symmetric successive over-reaxation in systems such as MATLAB [] or particuary in the image processing system IDL [16]. Other mutigrid formuations for other registration methods have been impemented successfuy [13] [1] as have fast Fourier methods [8]. It is aso a priority in the present work that the image registration or interpoation resut be independent of the order of two given images; see aso [5], [11], [1]. Here it is seen that this goa can be achieved for optica fow but in genera not for finite dispacements. Another important distinction between finite dispacements and optica fow is the cass of registrations which can be achieved by one in reation to the other. On the basis of reaistic appications it is proposed in [1] that optica fow can practicay be constrained to be autonomous. However, an expicit counterexampe is given here to show that the set of registrations achieved by finite dispacements is indeed arger than that of those achieved by autonomous optica fows. Nevertheess, ony autonomous optica fows are considered here numericay. The paper can now be summarized as foows. In Section, a variationa framework used throughout the paper is presented. Specificay, image simiarity measures and transformation reguarity measures are defined for finite dispacements and for optica fow formuations. An expicit counterexampe is aso given here to show that the set of registrations achieved by finite dispacements is indeed arger than that of those achieved by autonomous optica fows. In Section 3, optimaity systems are derived for the cost functions introduced in Section. Aso, the inear(ized) systems, which are ater discretized, are shown to be we posed. At the end of Section 3, the use of these optimaity systems to sove their corresponding minimization probems is summarized agorithmicay. In Section 4, modifications of the resuts of Sections and 3 are summarized for the case that sequences as opposed to pairs of images are to be registered or interpoated; see aso [5]. In Section 5, the optimaity systems of Section 3 are discretized numericay, incuding the finite eement discretization whose anaysis is summarized. In Section 6, a geometric mutigrid formuation is presented aong with the demonstration that the convergence criteria of [10] are met. Specificay, the approximation property is fufied through the Gaerkin formaism, and the smoothing property is fufied by a symmetric successive over-reaxation. Section 7 is concerned with detais of computationa impementation and with aternative formuations. For instance, umping of zero-order terms is beneficia in the imit of high data fideity [19]. Aso, a muti-coored ordering of unknowns is used to permit that the reaxation scheme is vectorizabe in image processing systems. Chaenges associated with noninear mutigrid in the present context are iustrated in order to expain the use of inear mutigrid as an inner iteration with respect to an outer quasi-newton or agged diffusivity iteration. Aso a pyramida scheme is described in which each outer iteration on a finer eve is started by predictions obtained from a coarser eve. Finay, in Section 8, the numerica methods deveoped and anayzed in previous sections are first appied to simpe test cases, and the mutigrid convergence guaranteed in Section 6 is exhibited for each registration formuation.

3 Then the methods are appied to reaistic biomedica images to demonstrate generaized affine transformations in terms of histoogica reconstruction and generaized rigid transformations in terms of dynamic medica imaging. The Variationa Framework Foowing the iustration in Fig. 1 for D images, et two given images I 0 and I 1 be situated ζ I 1 at z = 1 ξ ξ 1 z x x 1 I 0 at z = 0 Figure 1: The domain Q with D images I 0 and I 1 on the front and back faces Ω 0 and Ω 1, respectivey. Curviinear coordinates are defined to be constant on trajectories connecting ike points in I 0 and I 1. respectivey on the front and back faces, Ω 0,Ω 1 R N, of a box, Q R N+1, i.e., Q = {(x 1,...,x N,z) = (x,z) : 0 < x 1,...,x N,z < 1} (.1) I 0 on Ω 0 = {(x,z) Q : z = 0} (.) I 1 on Ω 1 = {(x,z) Q : z = 1} (.3) and et Ω R N denote an otherwise generic cross section of Q. Then define curviinear coordinates (ξ 1,...,ξ N,ζ) = (ξ,ζ) so that ξ is constant aong trajectories through Q that connect ike points in I 0 and I 1, and ζ = z. Aso, suppose that x = ξ hods in Ω 0 and therefore the dispacement vector within Q is d = x ξ. Simiary define η and y so that η is constant aong trajectories together with ξ, and y = η hods in Ω 1. Further, a trajectory tangent is given by (u 1,...,u N,1) in terms of the optica fow defined as: u = (u 1,...,u N ) = x ζ. (.4) As mentioned in Section 1, for computationa exampes it wi be assumed that the optica fow is autonomous, u z = 0. Note that it is not assumed that every point in Ω 0 finds a ike point in Ω 1, i.e., trajectories are aowed to move out of the box Q. Let the subsets of Ω 0 and Ω 1 with respect to which trajectories extend competey through the fu depth of Q be denoted respectivey by Ω c 0 = {ξ Ω 0 : x(ξ,ζ) Q,0 < ζ < 1} and Ω c 1 = {η Ω 1 : y(η,ζ) Q,0 < ζ < 1}. For those trajectories extending incompetey through Q define Ω i 0 = Ω 0\Ω c 0 and Ωi 1 = Ω 0\Ω c 1. Then an image simiarity measure, which does not favor a given image order, is given for finite 3

4 dispacements d(ξ,1) = x(ξ,1) ξ by the foowing sum of squared intensity differences: S fd (x) = + Ω c 0 Ω c 1 [I 0 (ξ) I 1 (x(ξ))] dξ + [I 0 (y(η)) I 1 (η)] dη + Ω i 0 Ω i 1 [I 0 (ξ) I 1 ] dξ (x(ξ) = x(ξ,1)) [I 0 I 1 (η)] dη (y(η) = y(η,0)) (.5) where I 0 and I 1 are the background intensities respectivey of I 0 and I 1, understood as those intensities for which no active signa is measured. These background intensities are used as though the side of the box Q, Γ = Q\{Ω 0 Ω 1 } (.6) were not present and the trajectories impinging upon Γ from Ω 1 or Ω 0 woud respectivey be connected with I 0 and I 1 continued in R N by their background intensities. The measure (.5) is equivaent to the foowing, Ω c 0 1 under the constraints [1]: and 0 [ ] di dζdξ + dζ Ω i 0 ζ(ξ) 0 [ ] di 1 dζdξ + dζ Ω i 1 `ζ(η) [ ] di dζdη (.7) dζ I(ξ,0) = I 0 (ξ), I(x(ξ,1),1) = I 1 (x(ξ,1)), ξ Ω c 0 (.8) I(x(ξ, ζ), ζ) = I 1, x(ξ, ζ) Γ, I(y(η, `ζ), `ζ) = I 0, y(η, `ζ) Γ (.9) where ζ and `ζ, as iustrated in Fig., denote the ζ coordinates at which trajectories emanating η I 1 on Ω 1 ζ(ξ) `ζ(η) I 0 on Ω 0 ξ I 0 I I 1 (η) I 0 (ξ) I I 1 Figure : ζ(ξ) and `ζ(η) denote the ζ coordinates at which trajectories emanating respectivey from ξ Ω i 0 and η Ω i 1 meet Γ. respectivey from Ω i 0 and Ωi 1 meet Γ. Thus, ζ = ζ(ξ) and `ζ = `ζ(η) are respectivey defined impicity as functions of ξ Ω i 0 and η Ωi 1 for (.7). A simiarity measure which invoves infinitesima instead of finite dispacements is obtained by appying the optica fow equation [15]: in (.7) to obtain: di dζ (x(ξ,ζ),ζ) = xi x ζ + I ζ = x I u + I z, (.10) S of (I,u) = Q [ x I u + I z ] dxdz, (.11) which has an integrand invoving purey oca information throughout Q. It aso differs from (.7) by not incuding the transformation Jacobian 1/det[ ξ x]. In other words, (.11) gives a 4

5 convenient Euerian (oca) counterpart to the Lagrangian (trajectory foowing) form appearing in (.7). Furthermore, the counterpart to (.8) in the Euerian context is given by: By defining the outfow and infow portions respectivey of Γ, I = I 0 on Ω 0, I = I 1 on Ω 1. (.1) Γ + = {(x,z) Γ : u n > 0}, Γ = {(x,z) Γ : u n < 0} (.13) the counterpart to (.9) in the Euerian context is given by: I = I 1 on Γ +, I = I 0 on Γ. (.14) Thus, an image simiarity measure is given for optica fow by (.17) under the constraints (.1) and (.14). Image registration is achieved by means of finite dispacements by minimizing: J fd (x) = S fd (x) + R fd (x) (.15) where the simiarity measure S fd is given in (.5) with the notations x(ξ) and y(η) used in the context of finite dispacements since the aternative notations x(ξ, 1) and y(η, 0) refer to trajectories. Aso, the reguarity measure R fd is defined by: R fd (x) = µ α =! α! Ω 0 α ξ x dξ (.16) where!/α! is the mutinomia coefficient for a muti-index α. The second-order reguarity measure R fd under natura boundary conditions provides generaized affine registration since J fd vanishes for images reated by an affine transformation. As shown in [1], generaized rigid registration is effectivey rued out for finite dispacements. Note that in order for J rd to be defined with the same symmetry as S fd, an additiona term R fd (y) penaizing y = x 1 over Ω 1 exacty as x is penaized over Ω 0 in R fd (x) may be added to J fd with considerabe cost in compexity of the formuation. Image registration and interpoation are achieved by means of optica fow by minimizing [1]: J of,i (I,u) = S of (I,u) + R of,i (u), i = 1, (.17) subject to (.1) and (.14), where the simiarity measure S of is given in (.11). Under natura boundary conditions the reguarity measure R of,i can be defined as: [ R of,1 (u) = φ( u T + u ) + γ u z ] dxdz. (.18) for generaized rigid registration and as: [ R of, (u) = µ Q Q α = α N+1 =0 ]! α! α x u + γ u z dxdz (.19) for generaized affine registration, since J of,1 vanishes for images reated by a rigid transformation and J of, vanishes for images reated by an affine transformation. In (.18), u = u : u and : denotes a componentwise matrix scaar product. Aso φ(s) = s for tota variation reguarization which is appropriate for object excision, and φ(s) = s for Gaussian reguarization which is appropriate for smoother reguarizations [1]. Trajectories through the domain Q are defined by integrating the optica fow under boundary conditions, i.e., by soving: ζ x(ξ,ζ) = ξ + u(x(ξ,ρ),ρ)dρ, ξ Ω 0, ζ [0,1] (.0) 0 5

6 and 1 y(η,ζ) = η + u(y(η,ρ),ρ)dρ, η Ω 1, ζ [0,1]. (.1) ζ A registration is given by the coordinate transformation x(ξ,1) and by the inverse transformation y(η,0). The given images I 0 and I 1 are interpoated by the intensity I. The fact that the reguarity measure (.18) provides generaized rigid registration and interpoation is estabished in [1]. That (.19) provides generaized affine registration and interpoation can be estabished as foows. Note that with x ζ (ξ,ζ) = u(x(ξ,ζ),ζ) and R of, (u) = 0 it foows that: ζ α ξ x = α ξ u = x u α ξ x, α =. (.) Since x(ξ,0) = ξ impies ξ α x = 0, α =, for ζ = 0, it foows with (.) that the transformation ξ x(ξ,ζ) is affine for a ζ [0,1]. As expained in [1], computationa experiments indicate that in practice the optica fow can as we be chosen to be autonomous. On the other hand, the foowing exampe shows now that there is indeed a theoretica difference between the autonomous and nonautonomous cases of γ = and γ < respectivey; see [4] for a reated exampe. The nonautonomous fow, x 1 (ξ 1,ξ,ζ) = 1 + (ξ 1 1 )cos[(π + ε(ξ 1 ) )ζ] + (ξ 1 )sin[(π + ε(ξ 1 ) )ζ] x (ξ 1,ξ,ζ) = 1 (ξ 1 1 )sin[(π + ε(ξ 1 ) )ζ] + (ξ 1 )cos[(π + ε(ξ 1 ) )ζ] (.3) is diffeomorphic for ε sufficienty sma, det[ x/ ξ] = 1 εζπ i (ξ i 1 ) > 0, and preserves circes, i(x 1 1 ) = i(ξ 1 1 ). Aso, the fow manifests the foowing very imited periodicity with respect to the axis, Ξ 1 = {ξ : ξ = 1 }, as can be visuaized in Fig. 3. The discrete map Figure 3: The two given images I 0 and I 1 are shown on the eft and on the right. The sequence of images, read from eft to right, has been determined by minimizing S of (u, I) + R of,1 (u). ξ x(ξ,1) returns Ξ 1 when twice iterated, x(ξ 1,) x(x(ξ 1,1),1) = x(ξ 1,1) = ξ 1, ξ 1 Ξ 1, but otherwise it hods that x(ξ,) ξ, ξ Ξ 1. If any associated optica fow, u(x) = x ζ, were autonomous, u z = 0, then for ξ 1 Ξ 1 it woud foow from x(ξ 1,) = x(ξ 1,0) that x(ξ 1, + ζ) = x(ξ 1,ζ) [4]. When this equaity is combined with x(x(ξ 1,ζ),) = x(ξ 1, + ζ), a consequence of the uniqueness of soutions to x ζ = u(x) [4], the resut with ξ 0 = x(ξ 1,ζ) is x(ξ 0,) = ξ 0. However, the ast equation cannot hod when ζ is chosen so that ξ 0 Ξ 1. Since x(ξ,1) is essentiay determined by I 0 (ξ) = I 1 (x(ξ,1)) for the strategicay constructed images I 0 and I 1 shown respectivey on the eft and on the right of Fig. 3, these images cannot be interpoated or registered by an autonomous optica fow. The sequence of images shown in Fig. 3 has been determined by minimizing J of,1 for γ < whie no such resut is possibe with γ =. Nevertheess, as mentioned above, it is found in practice that the optica fow can as we be chosen to be autonomous, and autonomous fows are the focus for the computationa exampes of Section 8. 3 Optimaity Conditions In this section, the necessary optimaity conditions for the variabes of J fd and J of,i are derived. As discussed in detai in Section 5, the intent is to sove cycicay for one variabe with the others hed fixed. Here, and throughout the paper, for a sufficienty reguar domain 6

7 D R m, et C ν (D,R n ) denote the Banach space of functions mapping D into R n with continuous derivatives up to order ν. Aso L p (D,R n ) denotes the Banach space of Lebesgue measurabe functions from D into R n with integrabe pth power whie essentiay bounded Lebesgue measurabe functions are denoted by L (D,R n ). Then W p,ν (D,R n ) denotes the Soboev space of functions with derivatives up to order ν in L p (D,R n ) whie H ν (D,R n ) = W,ν (D,R n ). The usua norms, semi-norms and inner products on these spaces are denoted by W p,ν (D,R n ), W p,ν (D,R n ) and (, ) H ν (D,R n ). See [1] for further detais of these function spaces. The cost J fd is stationary in the dispacement when x satisfies: where B fd and F fd are defined by: 0 = 1 δj fd δx (x, x) = B fd(x, x) F fd (x, x), x H (Ω,R N ) (3.1) B fd (x, x) = µ F fd (x, x) = + α = Ω c 0 Ω c 1! [ ξ α x] [ ξ α x]dξ (3.) α! Ω 0 [I 0 (ξ) I 1 (x(ξ))] x I 1 (x(ξ)) T x(ξ)dξ (3.3) [I 0 (y(η)) I 1 (η)] y I 0 (y(η)) T η y(η) x(y(η))dη. The derivation of B fd = 1 δr fd/δx in (3.) is standard, and F fd = 1 δs fd/δx in (3.3) is derived by extending I 0 and I 1 outside Ω 0 and Ω 1 by their background intensities and assuming (based, e.g., on mutiinear interpoation of pixe vaues) that the extended images are in W 1, (R N,R). Without the integras in S fd which invove background intensities, the extension of images described here coud not be used, and the variationa derivative of S fd woud necessariy refect the dependence of the domains Ω i 0 and Ωi 1 on the dispacement. Now, the variationa derivative of the first ine in (.5) is computed as foows: [ 1 δ δx Ω c 0 [ ] 1 δ δx [I 0(ξ) I 1 (x(ξ))] dξ R N ] [I 0 (ξ) I 1 (x(ξ))] dξ + [I 0 (ξ) I Ω i 1 ] dξ 0 (x; x) (x; x) = [I 0(ξ) I 1 (x(ξ))] x I 1 (x(ξ)) T x(ξ)dξ R N = [I 0 (ξ) I 1 (x(ξ))] x I 1 (x(ξ)) T x(ξ)dξ. Ω c 0 (3.4) The second ine in (.5) is differentiated as foows. For x ε = x + ε x, et y ε = x 1 ε variationa derivative of y with respect to x is computed according to: so that the 0 = ξ ξ = y ε(x(ξ) + ε x(ξ)) y(x(ξ)) ε y = im ε (x(ξ) + ε x(ξ)) y ε (x(ξ)) y + im ε (x(ξ)) y(x(ξ)) ε 0 ε ε 0 ε (3.5) = x y(x(ξ)) x(ξ) + δy (x(ξ); x(ξ)). δx 7

8 Thus, the variationa derivative of the second ine of (.5) is computed as foows: [ ] 1 δ [I 0 (y(η)) I 1 (η)] dη + [I δx Ω c 1 Ω i 0 I 1(η)] dη 1 = 1 [ ] δ δx R N[I 0(y(η)) I 1 (η)] dη = R N[I 0(y(η)) I 1 (η)] y I 0 (y(η)) T (y=x 1 ; x) [ ] δy (η; x) dη δx = ηy(η) x = [I 0 (y(η)) I 1 (η)] y I 0 (y(η)) T η y(η) x(y(η))dη Ω c 1 (y=x 1 ; x) which eads to (3.3). Now, (3.1) is soved by the foowing quasi-newton iteration: { Dfd (dx k,x k, x) = [B fd (x k, x) F fd (x k, x)], x H (Ω 0,R N ) x k+1 = x k + θdx k k = 0,1,,... where: D fd (dx k,x k, x) = B fd (dx k, x) + Ω c 0 (3.6) (3.7) [ x I 1 (x k (ξ)) dx k (ξ)][ x I 1 (x k (ξ)) x(ξ)]dξ (3.8) and θ is chosen by a ine search to minimize S fd [13]. Note that no additiona boundary conditions are imposed by restricting the domain of the forms B fd, F fd and D fd, and thus natura boundary conditions hod. The sovabiity of (3.7) for fixed k is estabished in Theorem 1 beow, which together with the other existence theorems beow, depends upon the foowing emma. Lemma 1 Suppose X and Y are Hibert spaces with Y compacty embedded in X. Further suppose that X is equipped with the norm X and Y with the norm Y = [ X + Y ]1/ where Y is a semi-norm on Y. Suppose biinear forms B 0 and B 1 are given satisfying: and c 1 y Y B 1 (y,y), y Y (3.9) B 0 (y,y) > 0, y 0 with B 1 (y,y) = 0. (3.10) Then there exists a constant c such that B = B 1 + B 0 satisfies: Proof: Assume there exists a sequence {y n } Y such that c y Y B(y,y), y Y. (3.11) y n Y = 1 whie B(y n,y n ) 0. (3.1) Since Y is compacty embedded in X, there is a subsequence {y n } which converges in X. Because the seminorm Y satisfies the inequaity: both terms on the right side of the foowing vanish: c 1 y n Y B 1(y n,y n ) B(y n,y n ) (3.13) y n y nk Y = y n y nk X + y n y nk Y. (3.14) It foows that {y n } is a Cauchy sequence in Y with some imit y Y which satisfies: c 1 y Y B(y,y ) = im n 0 B(y n,y n ) = 0. (3.15) Then from B(y,y ) = 0 it foows that B 0 in (3.10) vanishes; thus, y = 0 foows and contradicts the assumption that y n Y = 1. Therefore, B is coercive on Y. 8

9 Theorem 1 For x k H (Ω 0,R N ) and y k H (Ω 1,R N ) suppose x k = y 1 k on Ω c 0 = Ω 0 y k (Ω 1 ) and y k = x 1 k on Ω c 1 = Ω 1 x k (Ω 0 ). Suppose that the image I 1 W 1, (Ω 1,R) satisfies: x I 1 (x k (ξ)) (c + Wξ) dξ > 0, for a nonvanishing c R N,W R N N. (3.16) Ω c 0 Then there exists a unique dx k H (Ω 0,R N ) such that (3.7) hods. Proof: The caim foows from the Lax-Migram Lemma [6] once it is shown that D(dx, x) = D fd (dx,x k, x) is bounded and coercive on H (Ω 0,R N ) and that F( x) = B fd (x k, x)+ F fd (x k, x) is bounded on H (Ω 0,R N ). The boundedness of F is estabished by estimating (3.) and (3.3): F( x) µ x k H (Ω 0,R N ) x H (Ω 0,R N ) + [ I 0 L (Ω 0,R) + I 1 L (Ω 1,R)] I 1 W 1, (Ω 1,R) x L 1 (Ω 0,R N ) + [ I 0 L (Ω 0,R) + I 1 L (Ω 1,R)] I 0 W 1, (Ω 0,R) y k H 1 (Ω 1,R N ) x L (Ω 0,R N ) {µ x k H (Ω 0,R N ) + [ I 0 L (Ω 0,R) + I 1 L (Ω 1,R)] [ I 1 W 1, (Ω 1,R)µ(Ω 0 ) + I 0 W 1, (Ω 0,R) y k H 1 (Ω 1,R N ) ]} x H (Ω 0,R N ). The boundedness of D is readiy estabished, (3.17) D(dx, x) [µ + I 1 W 1, (Ω 1,R) ] dx H (Ω 0,R N ) x H (Ω 0,R N ) (3.18) and the coercivity of D on H (Ω 0,R N ) foows using Lemma 1. The simiarity measure S of is stationary in the intensity I for fixed u when I satisfies the foowing in terms of quantities defined beow in a Lagrangian frame [1]: I(x(ξ, ζ), ζ) = { I0 (ξ)[1 U(ξ,ζ,1)] + I 1 (x(ξ,1))u(ξ,ζ,1), ξ Ω c 0 I 0 (ξ)[1 U(ξ,ζ, ζ)] + I 1 U(ξ,ζ, ζ), x(ξ, ζ) Γ, ξ Ω i 0 (3.19) I(y(η, ζ), ζ) = { I1 (η)[1 V (η,0,ζ)] + I 0 (y(η,0))v (η,0,ζ), η Ω c 1 I 1 (η)[1 V (η, `ζ,ζ)] + I 0 V (η, `ζ,ζ), y(η, `ζ) Γ, η Ω i 1. (3.0) As iustrated in Fig., the parameters ζ and `ζ denote the ζ coordinates at which trajectories emanating respectivey from Ω i 0 and Ωi 1 meet Γ. Then, define U and V by: U(ξ,ζ, ζ) Ũ(ξ,ζ) Ũ(ξ,0) ζ [ ] = Ũ(ξ, ζ) Ũ(ξ,0), Ũ(ξ,ζ) = exp u(x(ξ,ρ),ρ)dρ d, (3.1) ζ 0 ζ 0 for ξ Ω 0, ζ [0, ζ], and arbitrary ζ 0 [0, ζ], and: V (η, `ζ,ζ) Ṽ (η,1) Ṽ (η,ζ) =, Ṽ (η,ζ) = Ṽ (η,1) Ṽ (η, `ζ) ζ ζ 0 exp [ ] u(y(η,ρ),ρ)dρ d, (3.) ζ 0 for η Ω 1, ζ [`ζ,1], and arbitrary ζ 0 [`ζ,1]. The cost J of,1 is stationary in the optica fow u for fixed I when u satisfies: 0 = 1 δj of δu (u,ū) = B of,1(u,u,ū) F of (ū), ū H 1 (Q,R N ), (3.3) where B of,1 and F of are defined by: B of,1 (u,v,ū) = [( I u)( I ū) + γ (u z ū z )] dxdz + Q F of (ū) = Q ( v φ T + v ) ) ( u T + u : Q ( ) ū T + ū dxdz (3.4) I z I ūdxdz. (3.5) 9

10 Now, (3.3) is soved by the agged diffusivity iteration [9]: B of,1 (u k,u k 1,ū) = F of (ū), ū H 1 (Q,R N ), k = 1,,3,... (3.6) Note that no additiona boundary conditions are imposed by restricting the domain of the form B of,1, and thus natura boundary conditions hod. The sovabiity of (3.6) is stated as foows and estabished in [1], with trivia modifications of the kind appearing in the proof of Theorem 3 for the autonomous case that γ =. Theorem Suppose that the intensity I W 1, (Q,R) satisfies: Q I (c + Wx) dxdz > 0, for every nonvanishing c R N and for every nonvanishing skew symmetric matrix W R N N. (3.7) Then with φ(s) = µ(x,z)s, 0 < µ 0 µ(x,z) µ 1 <, there exists a unique u k H 1 (Q,R N ) such that (3.6) hods. If γ = so that the arguments of B of,1 in (3.4) are z-independent, then (3.6) hods with Q repaced by Ω. It is furthermore assumed that u H 1 (Q,R N ) has sufficient reguarity so that (.0) and (.1) are we defined; see the discussion and references in [1]. The cost J of, is stationary in the optica fow u for fixed I when u satisfies: where B of, is defined by: 0 = 1 δj of δu (u,ū) = B of,(u,ū) F of (ū), ū H (Q,R N ), (3.8) B of, (u,ū) = [( I u)( I ū) + γ (u z ū z )]dxdz Q + µ! ( α u) ( α ū)dxdz (3.9) α! α = α N+1 =0 F of is again defined by (3.5), and the Hibert space and norm are defined according to: Q H (Q,R N ) = {u H 1 (Q,R N ) : α u L (Q,R N ), α =, α N+1 = 0} u H (Q,R N ) = u H 1 (Q,R N ) + α = α N+1 =0 Q α u dxdz. (3.30) Note that no additiona boundary conditions are imposed by restricting the domain of the form B of,, and thus natura boundary conditions hod. The sovabiity of (3.8) is estabished as foows. Theorem 3 Suppose that the intensity I W 1, (Q,R) satisfies: Q I (c + Wx) dxdz > 0, for a nonvanishing c R N,W R N N. (3.31) Then there exists a unique u H (Q,R N ) such that (3.8) hods. If γ = so that the arguments of B of, in (3.9) are z-independent, then (3.8) hods with Q repaced by Ω. 10

11 Proof: Note that H (Q,R N ) is estabished as a Hibert space with the same methods as the usua Soboev spaces are. If γ =, H (Q,R N ) is repaced by H (Ω,R N ) and a norms of u or ū over Q are repaced by corresponding norms over Ω. The caim foows from the Lax-Migram Lemma [6] once it is shown that B of, is bounded and coercive on H (Q,R N ) and that F of is bounded on H (Q,R N ). The boundedness of B of, and F of is readiy estabished: B of, (u,ū) I W 1, (Q,R) u L (Q,R N ) ū L (Q,R N ) + γ u z L (Q,R N ) ū z L (Q,R N ) + µ u H (Q,R N ) ū H (Q,R N ) [ I W 1, (Q,R N ) + γ + µ] u H (Q,R N ) ū H (Q,R N ) (3.3) F of (ū) I W 1, (Q,R) ū L (Q,R N ) I W 1, (Q,R) ū H (Q,R N ). (3.33) The coercivity of B of, on H (Q,R N ) foows using Lemma 1. It is furthermore assumed that u H (Q,R N ) has sufficient reguarity so that (.0) and (.1) are we defined; see the discussion and references in [1]. Athough (.5) and (.11) have been constructed with the priority that registration be independent of image order, this order independence can ony be shown in genera for optica fow and not for finite dispacements. Specificay, suppose two one-dimensiona images satisfy the foowing one-dimensiona formuation of (3.1): µd 4 ξx(ξ) + [I 1 (x(ξ)) I 0 (ξ)][i 1(x(ξ)) + I 0(ξ)] = 0 (3.34) where Dξ 4 is defined with the natura boundary conditions that third and second order norma derivatives vanish on Ω. Then the foowing does not necessariy hod: µd 4 ηy(η) + [I 0 (y(η)) I 1 (η)][i 0(y(η)) + I 1(η)] = 0 (3.35) uness y = x 1 satisfies Dηy(η) 4 = Dξ 4 x(ξ), which is the case for instance when the registration is affine. However, as indicated in the discussion of (.16), when J fd contains a term R fd (y) penaizing y over Ω 1 exacty as x is penaized over Ω 0 in R fd (x), the asymmetry in (3.34) and (3.35) disappears, abeit with considerabe cost in compexity of the formuation. On the other hand, order independence for optica fow can be seen readiy by exchanging images and by repacing I(x, z) and u(x) with I(x, 1 z) and u(x) respectivey and observing equaity in (3.19), (3.0), (3.3) and (3.8) after the transformation. The registration schemes for finite dispacements and for optica fow can now be summarized agorithmicay as foows. For finite dispacements: set x(ξ) = ξ and continue the foowing unti changes in x meet a convergence criterion: sove one step of (3.7) for the soution of (3.1), and in the process, perform a ine search to determine the step size in (3.7) so that (.5) is minimized. For optica fow: set u = 0 and continue the foowing unti changes in u meet a convergence criterion: perform the integrations (.0), (.1), (3.1) and (3.), compute I from (3.19) and (3.0), sove (3.8) for the optica fow, or sove one step of (3.6) for the soution of (3.3), whie (3.6) is equivaent to (3.3) when (3.3) is inear. The discretization and numerica soution of these equations is detaied in Sections 5 and 6. 11

12 4 Processing Image Sequences An image sequence may be registered or interpoated of course by processing the images ony pairwise and concatenating the resuts. On the other hand, a couping among images may be introduced as foows; see aso [5]. The images of a sequence {I } L =0 can be registered simutaneousy using finite dispacements {x } L =1 by minimizing: where: J (L) fd (x 1,...,x L ) = S () fd (x 1,...,x L ) = L =1 j =1 S () fd (x 1,...,x L ) + R fd (x ) (4.36) R N [I j(x j (ξ)) I (x (ξ))] dξ (4.37) and, as in the derivation of F fd, a images are extended smoothy by their background intensities outside their domains, Ω, which are defined anaogousy as discussed in Section. The end indices = 0 and = L in (4.36) correspond to pairwise registration with the singe near neighbor. When (4.36) has been minimized, the point x i (ξ) Ω i has been matched to the point x j (ξ) Ω j. To minimize J (L) fd with respect to x whie a other transformations are hed fixed, repace F fd in (3.1) and (3.7) with F () fd = 1 δs fd () /δx: F () fd (x, x) = R N j =1 [I j (x j (ξ)) I (x (ξ))] x I (x (ξ)) T x(ξ)dξ. (4.38) The functiona of (4.36) can be minimized by freezing a current transformations except for one, minimizing the functiona with respect to the seected transformation, updating that transformation immediatey (Gauss-Seide strategy) or ese updating a transformations simutaneousy (Jacobi strategy), and then repeating the process unti the updates have converged. Known transformations can remain frozen as fixed boundary conditions, e.g., at one or both of the end indices = 0 and = L in (4.36) when the position of one or both of the end images I 0 and I L is known. is just as we minimized with respect to x by registering the image I with the averaged image I n (ξ) = 0 j L j =1 I j(x j (ξ))/ 0 j L j =1. Anaogousy, the images {I } L =0 can be registered simutaneousy by computing autonomous optica fows {u } L =0 for the image pairs {[I,I n ]} L =0 according to the pairwise procedures of the present work, where the transformations {x } L =0 are computed by using their respective fows in (.0). The cacuation (4.38) shows that J (L) fd Then the fows and their corresponding transformations can be updated repeatedy unti convergence. The images {I } L =0 can be interpoated from autonomous optica fows {u } L 1+ν =0 using the semi-discretization defined on Q (L) = Ω (0,L): L u(x,z) = u (x)χ ν (z) (4.39) =0 where {χ ν }L 1+ν =0 is a basis for the canonica B-spines of degree ν defined on the grid {[, + 1]} =0 L 1 of [0,L] [14]. Then the transformations are given by the foowing modifications of (.0) and (.1): and ζ x(ξ,ζ) = ξ + u(x(ξ,ρ),ρ)dρ, ξ Ω, ζ [, + 1] (4.40) +1 y(η,ζ) = η + u(y(η,ρ),ρ)dρ, η Ω +1, ζ [, + 1] (4.41) ζ 1

13 and for Γ (L) = Q (L) \{Ω 0 Ω L } the intensity I is given by the foowing modifications of (3.19) and (3.0): I(x(ξ, ζ), ζ) = (4.4) { I 1 (ξ)[1 U(ξ,ζ, + 1)] + I (x(ξ, + 1))U(ξ,ζ, + 1), ξ Ω c I 1 (ξ)[1 U(ξ,ζ, ζ)] + I U(ξ,ζ, ζ), x(ξ, ζ) Γ (L), ξ Ω i 1 I(y(η, ζ), ζ) = (4.43) { I (η)[1 V (η,,ζ)] + I 1 (y(η,))v (η,,ζ), η Ω c 1 I (η)[1 V (η, `ζ,ζ)] + I 1 V (η, `ζ,ζ), y(η, `ζ) Γ (L), η Ω i 1 where U(ξ,ζ, ζ) is defined by (3.1) for ξ Ω, ζ [, ζ], and arbitrary ζ 0 [, ζ] and V (η, `ζ,ζ) is defined by (3.1) for η Ω +1, ζ [`ζ,+1], and arbitrary ζ 0 [`ζ,+1]. For instance, for ν = 0 χ 0 is the characteristic function for the interva [, + 1], and the above procedure corresponds to pairwise interpoation of the given images. When smoother trajectories and greater couping among images are desired, higher order spines can be used in (4.39), and (3.3) or (3.8) can be soved for {u } L 1+ν =0 with γ = 0 and Q repaced by Q (L). 5 Discretized Formuation In this section the numerica discretization of the equations determining intensities, finite dispacements and optica fows is expained. For this, et Q be divided into a grid of ces, each having dimensions (h,...,h,τ), in the x 1,...,x N, and z directions, respectivey, where h = p and τ = q for positive integers p and q. Specificay, with the integer-component N-dimensiona muti-indices i = (i 1,...,i N ), 0 = (0,...,0), and 1 = (1,...,1), define the ce corners by (x i+1/,z k+1/ ) = (ih,kτ), 0 i p 1, 0 k q, and the ce centroids by (x i,z k ) = ((i 1 )h,(k 1 )τ), 1 i p 1, 1 k q. For finite dispacements x, for the given images I 0 and I 1, and for autonomous optica fows u, ony the discretization of Ω (or Ω ) is required, and (I 0 ) i for instance denotes the vaue of I 0 at the ce centroid x i. Whie the natura generaization is studied in [1], ony autonomous optica fows are considered here numericay, but the corresponding intensity I is in genera z dependent. Then, I i,k denotes the vaue of I at the ce centroid (x i,z k ). Fractiona indices are used for ce boundaries. 5.1 Intensity Discretization and Associated Integrations First consider the intensity cacuations required for finite dispacements. In (3.3) mutiinear interpoation of the vaues (I 0 ) i and (I 1 ) i is used to evauate these intensities in the coordinates y(η) and x(ξ) respectivey. In (3.3) and (3.8) the gradients ( I 0 ) i and ( I 1 ) i are computed at ce centers x i by centra differences with natura one-sided differences at the boundary. Then mutiinear interpoation is used to evauate the gradients y I 0 and x I 1 in the coordinates y(η) and x(ξ) respectivey. The integrations in (3.3) and (3.8) invoving intensities are computed numericay from sums of grid vaues at peaks of basis functions as described in detai in Subsection 7.1. The derivative η y(η) is computed by centra differences with natura one-sided differences at the boundary. Now consider the intensity cacuations required for optica fow. As expained in [1], in order to obtain a sufficienty accurate intensity I in each ce for the optica fow cacuation, it is necessary to perform the trajectory integrations (.0), (.1), (3.1) and (3.) from each ce both toward Ω 0 and toward Ω 1. Nevertheess, a considerabe savings is achieved for these integrations when the optica fow is autonomous since trajectories emanating from ces at different depths overap in the optica fow phase space. These trajectory integrations are approximated using a Runge-Kutta method, where mutiinear interpoation is used to obtain distributed vaues of the optica fow and its divergence. Aso, optica fow derivatives are 13

14 computed using centra differences, with natura one-sided differences at the boundary. The terms I and I z are needed for the computation of optica fow in (3.4), (3.5) and (3.9), and these intensity derivatives are computed numericay as performed in [1]. Specificay, I is computed with centra differences with natura one-sided modifications at the boundary. Computationa experience shows that the derivative I z is best computed from (.10), i.e., (I z ) i,k = (di/dζ) i,k ( I) i,k ũ i, using the most current optica fow ũ and the trajectory derivative di/dζ avaiabe from (3.19) or (3.0). The integrations in (3.4), (3.5) and (3.9) invoving intensities are computed numericay from sums of grid vaues at peaks of basis functions as described in detai in Subsection Finite Eement Discretization Since ony autonomous optica fows are considered here numericay, each of the boundary vaue probems (3.7), (3.6) and (3.8) has the form: in which B and F possess the structure: B(ϕ,ψ) = F(ψ), ψ H ν (Ω,R N ) (5.1) B(ψ,ϕ) = B(ϕ,ψ) = B µ (ϕ,ψ) + B 0 (ϕ,ψ) (5.) B 0 (ϕ,ψ) = (g ϕ,g ψ), g L (Ω,R N ) (5.3) F(ψ) = (f,ψ) L (Ω,R N ), f L (Ω,R N ) (5.4) where the biinear form B 0 invoves no derivatives of its arguments and the biinear form B µ invoves reguarizing derivatives of its arguments. Furthermore, there exists a sef-adjoint operator L with Dom(L) = H ν (Ω,R N ) satisfying (Lϕ,ψ) L (Ω,R N ) = B(ϕ,ψ), ϕ,ψ Dom(L). Because of the natura correspondence in reguarity, the function space H ν (Ω,R N ) is approximated in the present work as we as in [0] by N-fod tensor products of B-spines of degree ν [14], Sh ν(ω,rn ) C ν 1 (Ω,R N ) H ν (Ω,R N ). However, it has been discovered recenty in joint work with the authors of [17] that the umping approach discussed beow in Subsection 7.1 might be circumvented by using Lagrangian eements in a discontinuous Gaerkin context, athough at the expense of arger systems, and these resuts wi be reported separatey. In the proofs of Theorems 1, and 3, the corresponding biinear form B is shown to be bounded and coercive, ˇβ ϕ H ν (Ω,R N ) B(ϕ,ϕ), B(ϕ,ψ) ˆβ ϕ H ν (Ω,R N ) ψ H ν (Ω,R N ), ϕ,ψ Hν (Ω,R N ) (5.5) on H ν (Ω,R N ) and the corresponding inear form F is shown to be bounded on H ν (Ω,R N ). Thus, (5.1) is uniquey sovabe on a cosed subspaces of H ν (Ω,R N ), and in particuar, the finite eement approximation to the soution ϕ of (5.1) is ϕ h Sh ν(ω,rn ) defined by: B(ϕ h,χ) = F(χ), χ S ν h (Ω,RN ) (5.6) According to Theorems 5.1- and 5.-1 of [] the B-spines used here approximate a H ν (Ω,R N ) functions to optima order h ν. Furthermore, according to Theorems 5.1- and 5.-3 of [], the B-spines possess the additiona inverse property (used to prove mutigrid convergence) that their H ν (Ω,R N )-norm is bounded in terms of their L (Ω, R N )-norm mutipied by h ν. With optima approximation properties, it foows with Céa s Lemma [6] that: ϕ ϕ h ν ch ν f L (Ω,R N ) (5.7) provided the soution ϕ to (5.1) possesses the additiona reguarity: ϕ H ν (Ω,R N ) c f L (Ω,R N ). (5.8) 14

15 For finite dispacements, the finite eement discretization outined above is impemented in particuar for (3.7) by setting ν =. Then the reguarity (5.8) of the soution to (3.7) foows from Theorem.5 of [0]. The biinear form D fd of (3.7) assumes the structure shown in (5.), where the first term in (3.8) corresponds to B µ and the remaining term in (3.8) corresponds to B 0. For the optica fow computation of (3.3), the finite eement discretization outined above is impemented by setting ν = 1. For tota variation reguarization, φ(s) = µ s, it is seen in exampes of [1] that the reguarity of the optica fow does not correspond to (5.8). On the other hand, for a fixed k and for a sufficienty reguar u k the soution to (3.6) must possess the reguarity corresponding to Gaussian reguarization, φ(s) = µs. For Gaussian reguarization, the reguarity (5.8) for soutions to (3.6) or (3.3) does not foow immediatey from Theorem.5 of [0], but such reguarity is assumed for the exampes of interest in the present work. The biinear form B of,1 of (3.3) and (3.6) assumes the structure of (5.), where the first integra in (3.4) (with γ = 0) corresponds to B 0 and the second integra of (3.4) (with integration over Q repaced by integration over Ω) corresponds to B µ. For the optica fow computation of (3.8), the finite eement discretization outined above is impemented by setting ν =. Then the reguarity (5.8) of the soution to (3.8) foows from Theorem.5 of [0]. The biinear form B of, of (3.8) assumes the structure shown in (5.), where the first integra in (3.9) (with γ = 0) corresponds to B 0 and the second integra in (3.9) (with integration over Q repaced by integration over Ω) corresponds to B µ. 6 Geometric Mutigrid Formuation The geometric mutigrid formuation deveoped for (5.1) in [0] and based upon [10] is adapted here to sove the finite eement discretizations of (3.7), (3.6) and (3.8). The usua mutigrid strategy is generay to enhance a convergent reaxation scheme by using its initia and rapid smoothing of the smaest error scaes representabe on finer grids and then to transfer the probem progressivey to coarser grids before reaxation is deceerated. The principa ingredients of the strategy incude the definition of a smoothing reaxation scheme and the definition of a coarse grid representation of the probem which can be used to provide an improvement or correction on a finer grid. For this, et the finest grid of Ω, as defined in Section 5, be divided into a nested sequence of coarser grids ranging from the coarsest at eve = 0 to the finest at = max. The grid at eve consists of pn ces having unit aspect ratio and width h = p where p +1 = p + 1 and p 0 is sma enough that a inear system with N( p 0 + ν) N unknowns can easiy be soved directy. Thus, the ce widths on adjacent eves satisfy h = h +1. As in Section 5, the ce corners are defined in the coarse grids by x i+ 1 = ih, 0 i p 1, and the ce centroids by x i = (i 1 )h, 1 i p 1. The spine basis for S ν (Ω,RN ) = S ν h (Ω,R N ) is N( p + ν) N -dimensiona as iustrated in Fig. 4 for N = 1, ν = 1,, p = 1 and p +1 =. Since in each case of (3.1), (3.6) and (3.8), the biinear form of (5.1) is bounded as dispayed in (5.5), the Lax-Migram Lemma [6] guarantees the existence of an operator L such that: (L χ,ψ) L (Ω,R N ) = B(χ,ψ), χ,ψ Sν (Ω,R N ). (6.1) Thus, the probem (5.6) is naturay formuated on the subspaces S ν (Ω,RN ) as foows. With the L -projection operator P : L (Ω,R N ) S ν (Ω,RN ) and for f = P f, find ϕ S ν (Ω,RN ) such that: L ϕ = f, where L h ϕ h = f h, = max. (6.) Denoting by I 1 the injection operator from (S ν 1 (Ω,RN ), L (Ω,R N ) ) into (Sν (Ω,RN ), L (Ω,R N ) ), the variationa probems on adjacent eves can be reated according to L 1 = I 1 L I 1 [10]. 15

16 (a) (c) (b) (d) Figure 4: Exampes of one-dimensiona nested grids: (a) ν = 1, p = 1, (b) ν = 1, p =, (c) ν =, p = 1, and (d) ν =, p =. The horizonta ine segments represent Ω = (0, 1), the vertica ine segments denote ce boundaries, and circes mark the peaks of basis functions. For an impementation in terms of basis function coefficients instead of on the operator eve, et R denote the N( p + ν) N -dimensiona Eucidean space of basis function coefficients Φ = {Φ,i R N : 1 i ( p + ν) 1} for functions ϕ S ν (Ω,RN ). Let K denote the bijective mapping from (R,[ ]) to (S ν (Ω,RN ), L (Ω,R N ) ) so that ϕ = K Φ hods. Aso, equip R with an h ν -weighted Eucidean inner product [X,Y ] = h N (X,Y ) and inner product [X ] = [X,X ] 1. Then the norms of coefficients in Φ R and of the corresponding functions in ϕ = K Φ S ν (Ω,RN ) are equivaent [0]. Thus, for a basis {χ ı } of S ν (Ω,RN ), which can be expressed as {K e ı } for unit vectors {e ı } R, the coefficient matrix representations of the operators L are given by: A,ıj = (L χ ı,χ j ) L (Ω,R N ) = (L K e ı,k e j ) L (Ω,R N ) = [K L K e ı,e j ] (6.3) or A = K L K. Aso the right side of (6.) has the coefficient representation f = K F and the probem (6.) takes the form: A Φ = F, where A h Φ h = F h, = max. (6.4) Now the variationa probems on adjacent eves are reated on the coefficient space R according to: A 1 = R 1 A E 1. (6.5) where E 1 : R 1 R and its transpose R 1 : R R 1 are the canonica proongation and restriction operators satisfying: I 1 K 1 = K E 1, (E 1) = R 1 (6.6) In words, the reation I 1 K 1 = K E 1 means that E 1 produces coefficients Φ = E 1 Φ 1 R from coefficients Φ 1 R 1 so that the function K Φ S ν (Ω,RN ) is identica to the function K 1 Φ 1 S ν 1 (Ω,RN ). Since in each case of (3.1), (3.6) and (3.8), the biinear form of (5.1) and (6.1) is coercive as dispayed in (5.5), it foows that the symmetric matrices A of (6.3) are positive definite [6]. Thus, with (6.6) and (6.5) the coarse grid probem and the intergrid transfer operators have been defined. It remains to identify a suitabe reaxation scheme and to define the mutigrid iteration. Since the matrices A are symmetric, it is natura to use a symmetric reaxation scheme. For this, the symmetric successive over-reaxation is used: Φ k+1 = S Φ k + ωw 1 F, S = I ωw 1 A (6.7) where in practice ω = 1 is used in the present work. Aso, the symmetric matrix W is given by W = (D + L )D 1 (D + L T ) where the diagona, stricty ower trianguar and stricty upper 16

17 trianguar parts of A are the respective terms in the sum A = D +L +L T. Since A is positive definite, it foows from W = A + L D 1 L T that W is positive definite. This reaxation scheme is vectorized for impementation in IDL [16] by using a muti-coored ordering of ces [0] [3]. Specificay, for a stenci diameter of (ν + 1) a set of same-coor ces is defined as those which are separated from one another in any of N coordinate directions by exacty ν ces. These ces have stencis which do not weight any other ces in the set; thus, each such set of ces may be updated simutaneousy in the reaxation. With this muticoored ordering, the reaxation scheme may be impemented by performing a Jacobi iteration on same-coored ces whie ooping in one direction and then the other over the coors. for c = 1,...,(ν + 1) N and then c = (ν + 1) N,...,1 do: Φ c Φ c ω[d 1 (A Φ F )] c (6.8) In this way, same-coored ces are updated simutaneousy. The convergence rate of the mutigrid scheme depends upon the vaue of θ in the foowing estimate: [W ] (1 + D 1 A )[A ] θ[a ] (6.9) which is estabished as foows. The first inequaity in (6.9) is obtained using W = A +L D 1 L T [0]. Since in each case of (3.1), (3.6) and (3.8), the biinear form of (5.1) is bounded and coercive (5.5) on H ν (Ω,R N ), D 1 A can be estimated as foows: D,ıı 1 A B(χ ı,χ j ) ˆβ χ ı H,ıj = ν (Ω,R N ) χ j H ν (Ω,R N ) B(χ ı,χ ı ) ˇβ χ ı H ν (Ω,R N ) ˆβ χ ı H ν (Ω,R N ) + χ j H ν (Ω,R N ) ˇβ χ ı ˆβ max ı χ ı H ν (Ω,R N ) ˇβ min H ν (Ω,R N ı χ ) ı. H ν (Ω,R N ) (6.10) Note that the basis functions {χ ı } are transation invariant and supported in Ω in such a way that the max and min above are computed from a number of cases near Ω which depend ony upon N and ν but not h or. Thus, D,ıı 1 A,ıj is bounded independenty of h and. Note that the support of any one of the canonica B-spines of S ν (Ω,R) overaps the support of at most (ν + 1) N of its transates; therefore, for the system defined on S ν(ω,rn ), the number of nontrivia eements in a given row of A is at most N(ν + 1) N. With this factor times the estimate of (6.10), D 1,ıı A,ıj in (6.9) is bounded independenty of h and. Note that θ in (6.9) depends most significanty upon ˆβ/ˇβ in (6.10). The quotient ˆβ/ˇβ in turn increases as the data support decreases, since the coercivity seen in (5.5) becomes ever weaker as ˇβ becomes ever smaer. With the above ingredients, a symmetric two-grid cyce TGC(,σ) between the finer eve and the coarser eve 1 is constructed as foows: 1) perform σ reaxation steps to update Φ, ) compute the coarse-grid residua D 1 = R 1 (F A Φ ), 3) sove the coarse grid probem A 1 Ψ 1 = D 1, 4) correct on the fine grid Φ Φ + E 1 Ψ 1, and finay 5) perform another σ reaxation steps to update Φ. Then a symmetric mutigrid cyce MGC(,σ,τ) is defined as with the two-grid cyce except that step (3) in TGC(,σ) is recursivey repaced with τ iterations of MGC( 1,σ,τ) uness for eve 1 = 0 where the coarse grid probem is soved exacty. The mutigrid cyce is a V (σ)-cyce if τ = 1 and a W(σ)-cyce if τ =. The nested or fu mutigrid iteration FMG( max,σ,τ) is given by first projecting the data to a coarse grids, F max = F, F 1 = R 1 F, = max,...,1 and soving the coarsest grid probem exacty, Φ 0 = A 1 0 F 0. Then with initiaization Φ = E 1 Φ 1, τ mutigrid cyces MGC(,σ,τ) are performed up to the finest grid, = 1,..., max. 17

18 Whie the FMG iteration is used in practice and foowed by further MGC cyces as necessary, the convergence of MGC is considered beow. The iteration matrix for the two-grid cyce can be written expicity as: M TGC (σ) = S σ [I E 1 A 1 1 R 1 A ]S σ. (6.11) and the iteration matrix for the mutigrid cyce as [10]: M MGC (σ,τ) = M TGC (σ) + S σ E 1 [MMGC 1 (σ,τ)] τ A 1 1 R 1 A S σ. (6.1) The convergence framework in [10] requires to estabish an approximation property and a smoothing property, which are given as foows (cf. Theorem , Remark and Lemma as we as cf. Theorem in [10]): Theorem 4 Under the conditions enumerated above, the foowing approximation property hods: 0 A 1 E 1A 1 1 R 1 θw 1. (6.13) Theorem 5 Under the conditions enumerated above, the smoother S, written in the form X = I A 1/ S A 1/ = A 1/ W 1 A 1/, satisfies: where η(σ) = σ σ /(1 + σ) (1+σ) satisfies im σ η(σ) = 0. Now (6.13) impies 0 A 1/ M TGC (σ)a 1/ 0 (I X ) σ X (I X ) σ η(σ)i (6.14) = [A 1/ S A 1/ [A 1/ S A 1/ ] σ {A 1/ ] σ [A 1/ [A 1 E 1 A 1 1 R 1 ]A 1/ }[A 1/ S A 1/ W 1 A 1/ ][A 1/ S A 1/ ] σ ] σ (6.15) = θ(i X ) σ X (I X ) σ θη(σ)i and thus with (6.14) the two-grid as we as the mutigrid convergence are given as foows (cf. Theorem and ( ) in [10]): Theorem 6 Under the conditions enumerated above, the two grid cyce satisfies: [A 1/ M TGC (σ)a 1/ ] θη(σ). (6.16) Theorem 7 Under the conditions enumerated above, the V (σ)-cyce and the W(σ)-cyce satisfy: [A 1/ M V (σ)a 1/ ] θ θ + σ, [A1/ M W (σ)a 1/ θ ]. (6.17) θ + σ 7 Impementation Aspects Here, impementation aspects are addressed which are not necessariy deduced from the theoretica foundation above. See aso the discussion of impementation aspects in [0] which appy equay we here, incuding the organization of a muti-coored ordering for vectorization, the treatment of foating point errors by the carefu computation of scaar products, the comparison with aternative finite difference formuations and the study of the effect of variations in data support and method parameters. The aspects considered in the subsections beow deserve specia treatment in the present context. 18

19 7.1 Lumping of Zero-Order Terms Consider the simpe case in (5.1)-(5.4) that N = 1, f is the dotted curve in each graph of Fig. 5, and g is the characteristic function of the support of f. Soutions to (5.6) using B- Figure 5: Soutions to (5.6) using B-spines in the pure finite eement method with the cases that µ is arge and sma shown respectivey on the eft and on the right. Here, N = 1, ν =, f is shown in the dotted curve, and g is the characteristic function for the support of f. spines in the pure finite eement method are shown in Fig. 5 on the eft and right respectivey for the cases that µ is arge and sma. Since the quotient f/g can be extended to an H ν (Ω,R) function outside the data support, it can be shown there is a unique weak imit ϕ H ν (Ω,R) of soutions ϕ (µ) to (5.1) as µ 0. In this case, ϕ is a quadratic function satisfying ϕg = f for noise-free data f and g. However, as seen in Fig. 5, the finite eement soution ϕ (µ) h does not converge to ϕ as µ 0 when the data g are discontinuous. Such convergence is of course desirabe in the case of very high signa-to-noise ratios such as in this simpe exampe. This convergence can be obtained using a umping approach as introduced in [19]. However, as discussed in Subsection 5., it has been discovered recenty in joint work with the authors of [17] that this umping might be circumvented by using Lagrangian eements in a discontinuous Gaerkin context, athough at the expense of arger systems, and these resuts wi be reported separatey. In the umping approach used here, the zeroth-order component of the coefficient matrix is diagonaized according to: B 0 (χ ı,χ j ) = (g χ ı,g χ j ) L (Ω,R N ) δ ıj(g ı g ı ) (7.1) where g ı denotes g evauated at basis function peaks as iustrated with the circes in Fig. 4. Specificay, the basis functions addressed in (6.3) and (7.1) are characterized as foows. Let χ ν (x) be the canonica B-spine function of degree ν with support in (0,ν + 1) and peak χ ν ( 1 (ν + 1)) = 1 whose diation and then transations, χ ν i(x) = χ ν (h 1 (x x i (ν+ 1 ))), 1 i (p + ν) 1, (7.) provide a basis for S ν h (Ω,R). Then, when e n R N denotes the unit vector with e (m) n = δ nm, {χ ı } = {e n χ ν i : 1 i ( p + ν) 1, 1 n N}. (7.3) Thus, when χ ı = e n χ ν i then g ı = g(x i ν ) since χν i peaks at x = x i ν. With the modification shown in (7.1), convergence in the imit of increasing data fideity and vanishing reguarization is obtained as demonstrated experimentay in [19]. Furthermore, when the umping in (7.1) is used, the grids, e.g., as iustrated in Fig. 4, can aternativey be taken to incude the ghost ces situated outside Ω in Fig. 4, the finite eement coefficients Φ h,i = {Φ (n) h,i } RN can be taken as soution vaues Φ h,i = ϕ h (x i ν ) at basis function peaks 19

20 marked with circes in Fig. 4, and the numerica soution is determined pointwise as the soution to: [ ] B µ (χ ν ie m,χ j e n ) + B 0 (χ 0 i ν e m,χ 0 j ν e n ) Φ (n) h,j = F(χ0 i ν e m ), 1 n N 1 j ( p + ν) 1 1 i ( p + ν) 1, 1 m N where B 0 (χ 0 i ν e m,χ 0 j ν e n ) = g m (x i ν )g n(x i ν )δ mn and F(χ 0 i ν e m ) = f m (x i ν ) foow from (5.3) and (5.4) with g and f extended by zero outside Ω. The scaar product of the eft side of (7.4) with the same coefficients Φ h can be expressed in terms of the functions ϕ ν h = 1 i ( p +ν) 1 Φ h,i χ ν i and ϕ0 h = 1 i ( p +ν) 1 Φ h,i χ 0 i ν as B µ (ϕ ν h,ϕν h ) + B 0(ϕ 0 h,ϕ0 h ). Thus, the coefficient matrix in (7.4) is symmetric and non-negative. For the particuar cases (3.), (3.9) and (3.4), the coefficient matrix is shown beow to be positive definite by showing that B 0 is coercive on the kerne of B µ, where B ν and B 0 are defined for these cases as expained at the end of Section 5. Theorem 8 For a given x k H (Ω 0 ), suppose that for every c R and for every matrix W R N N the image I 1 satisfies: 1 i ( p +ν) 1 (7.4) x I 1 (x k (x i ν )) (c + Wx i ν ) > 0 (7.5) uness c = 0 and W = 0 hod. Then the coefficient matrix A fd of (7.4) corresponding to D fd is symmetric and positive definite. Proof: Define x ν (ξ) = 1 i ( p +ν) 1 x i χν i (ξ) and x0 (ξ) = 1 i ( p +ν) 1 x i χ0 i ν (ξ) which are in the kernes of B µ and B 0 when A fd x = 0. Then x ν must be an affine function of ξ or x i must be an affine function of i. Thus x i = x0 (x i ν ) = c + Wx i ν hods for a c RN and a matrix W R N N. Since B 0 (x 0,x 0 ) satisfies (7.5) uness x 0 = 0, it foows that x = 0. Theorem 9 Suppose that for every c R and for every skew symmetric matrix W R N N the intensity I satisfies: 1 k q 1 i ( p +ν) 1 ( I) i ν,k (c + Wx i ν ) > 0 (7.6) uness c = 0 and W = 0 hod. Then the coefficient matrix A of,1 of (7.4) corresponding to B of,1 is symmetric and positive definite. Proof: Define u ν (x) = 1 i ( p +ν) 1 u i χν i (x) and u0 (x) = 1 i ( p +ν) 1 u i χ0 i ν (x) which are in the kernes of B µ and B 0 when A of,1 u = 0. Then u ν satisfies u ν (x) = c+wx for a c R N and a skew symmetric matrix W R N N [1]. Since ν = 1, u i = uν (x i ν ) = c + Wx i ν hods and impies u 0 (x i ν ) = u i = c + Wx i ν. Since B 0(u 0,u 0 ) satisfies (7.6) uness u 0 = 0, it foows that u = 0. Theorem 10 Suppose that for every c R and for every matrix W R N N the intensity I satisfies: ( I) i ν,k (c + Wx i ν ) > 0 (7.7) 1 k q 1 i ( p +ν) 1 uness c = 0 and W = 0 hod. Then the coefficient matrix A of, of (7.4) corresponding to B of, is symmetric and positive definite. 0

21 Proof: Define u ν (x) = 1 i ( p +ν) 1 u i χν i (x) and u0 (x) = 1 i ( p +ν) 1 u i χ0 i ν (x) which are in the kernes of B µ and B 0 when A of, u = 0. Then u ν must be an affine function of x or u i must be an affine function of i. Thus u i = u0 (x i ν ) = c + Wx i ν hods for a c RN and a matrix W R N N. Since B 0 (u 0,u 0 ) satisfies (7.7) uness u 0 = 0, it foows that u = 0. To consider the umping in (7.1) within the theoretica convergence framework, consider the decomposition L h = L µ h + L0 h where Lµ h and L0 h are defined as foows: (L µ h χ ı,χ j ) = B µ (χ ı,χ j ), (L 0 hχ ı,χ j ) = B 0 (χ ı,χ j ) δ ıj (g ı g ı ) (7.8) where the set {χ ı } is given in (7.3). Then L µ and L 0 are defined by the Gaerkin formuations, L µ 1 = I 1 Lµ I 1, L 0 1 = I 1 L0 I 1, for 0 max and A = A µ + A 0 are simiary decomposed according to (6.5) for 0 max. Aso, A µ is convenienty computed according to (6.3), A µ,ıj = B µ(χ ı,χ j ) (7.9) whie A 0 is computed iterativey according to (6.5), A 0 1 = R 1 A 0 E 1. (7.10) The reations (6.6) and (6.5) hod in any case for A, A µ and A 0. Then, instead of by (6.10), D 1,ıı A,ıj is estimated as foows: D 1,ıı A,ıj = B µ (χ ı,χ j ) + δ ıj (g ı g ı ) B µ (χ ı,χ ı ) + (g ı g ı ) δ ıj + (1 δ ıj ) B µ(χ ı,χ ı ) + B µ (χ j,χ j ) B µ (χ ı,χ ı ) to obtain (6.9). 7. Noninear Mutigrid B µ (χ ı,χ j ) δ ıj + (1 δ ıj ) B µ (χ ı,χ ı ) + (g ı g ı ) δ ıj + (1 δ ıj ) max ı B µ (χ ı,χ ı ) min ı B µ (χ ı,χ ı ) (7.11) Instead of using the agged diffusivity iteration (3.6) as an outer iteration and the mutigrid techniques of Section 6 as an inner iteration, it is natura to inquire whether these outer and inner iterations can be reversed in the usua manner of noninear mutigrid [8]. Such an aternative formuation was tested and found to be probematic because of natura boundary conditions as expained here. The coarse grid operator is determined naturay by (6.1) and (6.3) for the inear(ized) case. For the noninear case, an additiona question arises concerning the representation of u in the foowing counterpart to (6.1): (L (u)χ,ψ) L (Ω,R N ) = B(χ,u,ψ), χ,ψ Sν (Ω,RN ). (7.1) For (3.4), in particuar, the question concerns the computation of φ ( u T + u ) which appears as a diffusivity in the strong differentia form of (3.3). Given a suitabe approximation u +1 S+1 ν (Ω,RN ), the natura candidate for u in (7.1) is u = I u +1. However, consider that for the 1D inear spines shown in Fig. 4, the canonica proongation and restriction operators are given by: E 1 = , R 1 = (7.13)

22 Thus, the constant function with coefficients U = 1,1,1,1,1 T on the finer grid is transformed according to U 1 = R 1 U = 1 3,4,3 T to obtain a function on the coarser grid which is not ony non-constant but diminished on Ω. In genera, u = I u +1 is diminished on Ω in reation to u +1, and thus the diffusivity obtained on eve from restriction is significanty different on the boundary than the corresponding diffusivity on eve + 1. It is seen from a detaied examination of the computations that this difference corrupts the coarse grid correction of noninear mutigrid. Whie one might consider to deveop an improved boundary representation of diffusivities, it is demonstrated in [8] that an outer iteration update strategy can be combined with inear mutigrid in an inner iteration so that the tota computationa work is comparabe to that of noninear mutigrid when it functions. Such strategies can be impemented here in reation to (3.6). Nevertheess, no visibe advantage to noninear φ (non-constant diffusivity) coud be demonstrated for practica exampes in the present work, in spite of the apparent advantages evident from simpe exampes shown in [1]. 7.3 Pyramida Scheme To acceerate the minimization of J fd, J of,1 and J of, a pyramida scheme is impemented in this work as foows for a chosen min, 0 min max, in combination with the agorithmic outine given at the end of Section 3: For = max,..., min + 1 restrict I 0 and I 1 from mutigrid eve to eve 1. For = min,..., max : if = min initiaize the dispacement or optica fow triviay, ese proong the dispacement or optica fow from mutigrid eve 1 to eve. sove for the dispacement or optica fow on mutigrid eve. The efficiency of this pyramida scheme depends upon the accurate representation of the images on the coarse grids. The exampe given in Subsection 7. shows that the images are diminished at the image domain boundary by restriction. If the restricted images are not particuary representative of those given on the fine grid, the dispacement or the optica fow computed on a coarse grid may aso be correspondingy biased, and the bias may draw the soution toward an undesired oca minimum of the cost function on a finer grid. In such cases, stronger reguarization can be used on coarser grids or ese min must be chosen nearer to max. 8 Computationa Resuts In this fina section, the numerica methods deveoped and anayzed in previous sections are first appied to simpe test cases to demonstrate generaized rigid and generaized affine registration. A test case is aso used to demonstrate the mutigrid convergence guaranteed in Section 6 for each registration formuation. Then the methods are appied to reaistic medica images, where generaized affine registration is intended mainy for histoogica appications and generaized rigid registration and interpoation are intended mainy to faciitate medica examinations by dynamic imaging. In the test case of Fig. 6, the two given images I 0 and I 1 are shown respectivey on the eft and on the right of both of the top two rows. These images were constructed so that they may be interpoated by either an affine or by a rigid transformation. The intermediate images in the first row of Fig. 6 were interpoated by generaized affine optica fow (3.8), whie the intermediate images in the second row were interpoated by generaized rigid optica fow (3.3). The successfu computation of affine and rigid interpoations are evident visuay from the image sequences in Fig. 6, and these resuts are visuay independent of µ. The

23 Figure 6: Interpoated image sequences obtained by generaized affine optica fow (3.8) and by generaized rigid optica fow (3.3) are shown respectivey in the first and second rows. The sequences are to be read from eft to right. The same images I 0 and I 1 were used for both sequences and are shown respectivey at the eft and at the right of the sequences. The registration transformations obtained are appied to a uniform grid, and the transformed grids are shown in the third row. The two transformed grids on the eft were obtained by generaized affine registration, and the two transformed grids on the right were obtained by generaized rigid registration. resuting registration transformations were appied to a uniform grid, paced first on Ω 1 and transformed to Ω 0 and then paced on Ω 0 and transformed to Ω 1, and the transformed grids are shown in the third row of Fig. 6. The two transformed grids on the eft were obtained by generaized affine registration, and the two transformed grids on the right were obtained by generaized rigid registration. Note that the generaized affine transformation compresses a grid ines in one direction whie most grid ines in the other direction fow out of the domain. On the other hand, the generaized rigid transformation eaves grid ines orthogona to one another whie rotating most grid points out of the domain, incuding some points near the corners of the rectanguar regions corresponding to nontrivia pixes in the given images. The registration errors for the generaized affine interpoation are: I 0 I 1 L (Ω 0,R) = = I 1 I 0 L (Ω 1,R) and I 0 I 1 L (Ω 0,R) = 0.6 = I 1 I 0 L (Ω 1,R). The registration errors for the generaized rigid interpoation are: I 0 I 1 L (Ω 0,R) = = I 1 I 0 L (Ω 1,R) and I 0 I 1 L (Ω 0,R) = 1.0 = I 1 I 0 L (Ω 1,R). When the given images I 0 and I 1 are registered according to (3.1), the morphed images I 1 and I 0 are visuay indistinguishabe from I 0 and I 1 respectivey, and the registration errors are I 0 I 1 L (Ω 0,R) = = I 1 I 0 L (Ω 1,R) and I 0 I 1 L (Ω 0,R) = 1.0 = I 1 I 0 L (Ω 1,R). In the test case of Fig. 7, the two given images I 0 and I 1 are shown respectivey at the eft and at the right of both rows, and these are reated by an affine transformation. The intermediate images shown in the first row of Fig. 7 were interpoated by generaized affine optica fow (3.8) whie those shown in the second row were interpoated by generaized rigid optica fow (3.3). The successfu computation of an affine interpoation is evident visuay from the sequence of images and quantitativey from the registration errors, I 0 I 1 L (Ω 0,R) = 0.005, I 0 I 1 L (Ω 0,R) = 0.6, I 1 I 0 L (Ω 1,R) = 0.0 and I 1 I 0 L (Ω 1,R) = 0.0. The generaized affine interpoation is aso visuay independent of µ. The generaized rigid interpoation is obtained using a Gaussian penaty, φ(s) = µs, where µ is arge enough for the resut to be strongy rigid, i.e., the resuting interpoation in the second row of Fig. 7 has the appearance of a convex combination of rigid transformations of the two images. The bias toward rigidity is evident not ony visuay in the image sequence but aso from the arger registration errors, I 0 3

24 Figure 7: Interpoated image sequences obtained by generaized affine optica fow (3.8) and by generaized rigid optica fow (3.3) are shown respectivey in the first and second rows. The sequences are to be read from eft to right. The same images I 0 and I 1 were used for both sequences and are shown respectivey at the eft and at the right of the sequences. I 1 L (Ω 0,R) = 0.18, I 0 I 1 L (Ω 0,R) = 1.0, I 1 I 0 L (Ω 1,R) = 0.17 and I 1 I 0 L (Ω 1,R) = 1.0. When the given images I 0 and I 1 are registered according to (3.1), the morphed images I 1 and I 0 are visuay indistinguishabe from I 0 and I 1 respectivey, and the registration errors are I 0 I 1 L (Ω 0,R) = 0.005, I 0 I 1 L (Ω 0,R) = 0.5, I 1 I 0 L (Ω 1,R) = and I 1 I 0 L (Ω 1,R) = 0.4. The computations described above were carried out as outined in Subsection 7.3 with reference to the agorithmic outine given at the end of Section 3. In particuar, p = 56, max = 5, min = 0 and µ/h ν = 10 5 were used and MGC-V (1) iterations were performed for each eve of the pyramida scheme. In each case, many outer iterations were performed on the coarser pyramida eves where they are inexpensive, and then ony a few outer iterations were performed on the finer pyramida eves. For the fina outer iteration on the finest pyramida eve, graphica representations of the convergence of MGC-V (1) iterations for (3.1), (3.3) and (3.8) for the exampe of Fig. 7 are shown in Fig. 8 by potting on a og 10 -scae the differences L Norm between Iterates optica fow () optica fow (1) dispacements Energy Norm between Iterates optica fow () optica fow (1) dispacements Number of Iterations Number of Iterations Figure 8: Convergence histories for (3.1), (3.3) and (3.8) of e k h = Φk h Φ k 1 h are shown here for MGC-V (1) iterations (a) in the norm e k h on the eft and (b) in the norm A 1/ h ek h on the right. The soid curve is for (3.8), the dashed curve is for (3.3), and the dot-dashed curve is for (3.1). e k h = Φk h Φk 1 h as functions e k h and A 1/ h ek h of iteration number k. The mutigrid reduction factors ρ (k) = A 1/ h ek h / A 1/ h ek 1 h are ρ (3) = 0.14 for (3.8), ρ (5) = 0.17 for (3.3), and ρ (9) = 0.5 for (3.1). The smaer mutigrid reduction factors for optica fow can be understood from the fact that the data 1 0 I(x,z) I(x,z)T dz for optica fow have broader support than the data I 1 (x) I 1 (x) T for finite dispacements; thus, for optica fow the quotient ˆβ/ˇβ in (6.10) is smaer and consequenty θ in (6.9) is smaer. See aso the reated experiments in [0]. 4

25 In the exampe of Fig. 9, a sequence of images, I 0, I 1, I 1 x 1 and I from histoogica sections Figure 9: Images I 0, I 1, I 1 x 1 and I from histoogica sections of a mouse heart are shown from eft to right, where the image I 1 x 1 is registered to I 0 and I. of a mouse heart is shown, where the image I 1 x 1 is registered to I 0 and I by minimizing (4.36) with L = and with frozen end transformations x 0 (ξ) = ξ and x (ξ) = ξ. Again, p = 56, max = 5, min = 0, µ/h ν = 10 5 and MGC-V (1) iterations were used. As mentioned in Section 1, generaized affine registration is suitabe for histoogica appications because sections may be affine deformed in the process of sicing. On the other hand, without fixing certain images in (4.36), (generaized) affine transformations can expand or contract images so that an object is deformed essentiay to the same size throughout the entire stack. This effect can aso be controed by the number of updates performed in order to sove the entire couped optimaity system for (4.36). The effect may by controed more rigorousy by incorporating additiona penaties on voume changes, but this goa is achieved by generaized rigid registration [3]. The three raw images in Fig. 9 are actuay one tripe taken from a engthy sequence of histoogica sections of a mouse heart, such as used in [3], and the raw and registered sequences are shown in 3D in Fig. 10, where the sequence registration is performed by first shifting a images to the same center of mass and then proceeding with finite dispacements as described in Section 4. Fims of the raw and registered sequences can be viewed by downoading them respectivey from: In the exampe of Fig. 11, respiratory motion accompanies the sudden and widespread appearance of contrast agent, particuary in the kidneys, and it is required to interpoate between the two given images shown respectivey at the eft and at the right. The other images shown in the figure have been interpoated using generaized rigid optica fow; in fact, the scaing function approach of [18] is used to treat the appearance of contrast agent. Again, p = 56, max = 5, min = 0, µ/h ν = 10 5 and MGC-V (1) iterations were used. The soft tissues move as a resut of respiration but the spina coumn, for instance, remains rigid. Such an exampe appears to be a good candidate for a reaistic appication of tota variation reguarization as discussed in [1]. However, as indicated in Subsection 7., the resut shown in Fig. 11 is not perceptiby affected by the use of tota variation reguarization. The raw and interpoated images shown in Fig. 11 are actuay sampes taken from a engthy tempora sequence in which tempora resoution is ow enough for the raw sequence to be quite jerky but high enough to permit a smooth interpoation which faciitates the medica examination. The raw image sequence has been interpoated pairwise using generaized rigid optica fow, and the raw and interpoated fims can be viewed by downoading them respectivey from: References [1] R.A. Adams, Soboev Spaces, Academic Press, New York,

26 Figure 10: A sequence of histoogica sections of a mouse heart are shown unregistered on the eft and registered by finite dispacements on the right. Figure 11: The two given images I 0 and I 1 are shown on the eft and right respectivey. The intermediate images have been interpoated by generaized rigid optica fow. [] J. Aubin, Approximation of Eiptic Boundary-Vaue Probems, Krieger, Huntington, New York, [3] R. Burton, G. Pank, J. Schneider, A.J. Prass, J. Lee, V. Grau, N. Smith, N. Trayanova, P. Koh, H. Ahammer and S.L. Keeing, 3-Dimensiona Modes of Individua Cardiac Histo-Anatomy: Toos and Chaenges, to appear in the Annas of the New York Academy of Sciences. [4] D.R.J. Chiingworth, Differentia Topoogy with a View to Appications, Pitman, London, [5] G.E. Christensen, H.J. Johnson, Consistent Image Registration, IEEE Trans. Med. Imaging. Vo. 0, No. 7, Juy 001, pp [6] P.G. Ciaret, The Finite Eement Method for Eiptic Probems, North-Hoand, Amsterdam, [7] B. Fischer, J. Modersitzki, Curvature Based Image Registration, J. Math. Imaging and Vision 18, pp , 003. [8] B. Fischer, J. Modersitzki, Fast Inversion of Matrices Arising in Image Processing, Numerica Agorithms, pp. 1 11, [9] L. Garcin and L. Younes, Geodesic Image Matching: A Waveet Based Energy Minimization Scheme, EMMCVPR, 005. [10] W. Hackbusch, Iterative Soution of Large Sparse Systems of Equations, Springer,