Regularization of linear inverse problems with total generalized variation

Regularization of linear inverse problems with total generalized variation Kristian Bredies Martin Holler January 27, 2014 Abstract The regularization properties of the total generalized variation (TGV) functional for the solution of linear inverse problems by means of Tikhonov regularization are studied. Considering the associated minimization problem for general symmetric tensor fields, the wellposedness is established in the space of symmetric tensor fields of bounded deformation, a generalization of the space of functions of bounded variation. Convergence for vanishing noise level is shown in a multiple regularization parameter framework in terms of the naturally arising notion of TGV-strict convergence. Finally, some basic properties, in particular non-equivalence for different parameters, are discussed for this notion. Mathematical subject classification (2010): 65L09, 65F22, 65L20, 46G05, 46B26. Keywords: Linear ill-posed problems, total generalized variation, multiple parameter regularization, symmetric tensor fields, spaces of bounded deformation, a-priori parameter choice. 1 Introduction This paper is concerned with establishing the total generalized variation (TGV) as a regularization functional for ill-posed linear inverse problems Ku = f in an Tikhonov regularization framework, i.e., the study of the minimization problem min u L p () 1 q Ku f δ q Y + TGVk α(u) Institute for Mathematics and Scientific Computing, University of Graz, Heinrichstraße 36, A-8010 Graz, Austria. Email: kristian.bredies@uni-graz.at, martin.holler@uni-graz.at. 1

on a bounded domain IR d and for noisy data f δ as well as a suitable K. The total generalized variation, introduced in [9], is a functional which is able to regularize, according to the set of parameters α, on different scales of smoothness up to a certain order k in terms of the Radon norm. Like its basis, the total variation (TV), it allows for discontinuities but also accounts for higher-order information if k is greater than 1. The total generalized variation is therefore better suited for modeling piecewise smooth functions than TV which is commonly regarded as a model for piecewise constant functions. While TGV was first applied as a regularizer in variational imaging for solving denoising problems in L p () as well as in several other imaging contexts [7, 8, 16, 18, 19, 20], the second-order version TGV 2 α has been shown to give well-posed Tikhonov-regularized problems min u L p () 1 2 Ku f δ 2 H + TGV 2 α(u) where K maps continuously between a Lebesgue space and a Hilbert space [11]. These results heavily rely on the functional-analytic properties of the space of vector fields of bounded deformation BD() which contains all integrable vector fields whose symmetrized derivative is a Radon measure [27, 26]. For establishing generalizations for arbitrary orders k of the TGV-functional, extensions of BD() to symmetric tensor fields of order l have to be considered. However, the functional-analytic properties of these spaces became available only recently [6]. In this paper, the respective results are employed in order to extend the regularization properties to TGV of orders greater than 2. Furthermore, the notion of total generalized variation for scalar functions has also a natural generalization to symmetric tensor fields. While again denoising problems can be solved quite easily and may, for instance, be applied in diffusion tensor imaging (DTI) [29, 28], regularization properties for general tensor order l and arbitrary k are still open. As the treatment of general symmetric tensor fields is inherent for proving well-posedness even in the scalar case, the analysis in the current paper will also cover the tensor case. The main results can be summarized as follows: First, for any tensor order l, differentiation order k, any continuous linear operator K mapping from an L p -space (1 p d/(d 1)) on a bounded Lipschitz domain to a normed space, any f δ Y and q 1, a solution of min u L p (,Sym l (IR d )) 1 q Ku f δ q Y + TGVk,l α (u) (1.1) exists and depends stably on the data f. Here, α denotes the total generalized variation of order k for symmetric tensor fields of order l with weights α. For its precise definition, we ask for the reader s patience until Section 3. Second, choosing for each noise level δ an f δ such that f δ f Y δ for some exact data f and letting the parameters α tend to zero in an appropriate manner to the convergence towards a α -minimizing solution. The functional α has to be extended in this context to incorporate special cases which formally correspond to letting some of the values in α to be infinite. On 2

the way of proving these results, we establish auxiliary properties about the structure of TGV which are interesting on its own. The most important are generalizations of [11] to arbitrary orders k and l: We have the probably more convenient minimum representation as a differentiation cascade according to α (u) = min w i BD(,Sym l+i (IR d )), i=0,...,k, w 0 =u, w k =0 k α k i Ew i 1 w i M (1.2) where BD(, Sym l (IR d )) is the space of symmetric tensor fields of bounded deformation, E the weak symmetrized derivative and M denotes the Radon norm. Furthermore, the norm equivalence u BGVα = u 1 + α (u) u BD = u 1 + Eu M is established from which the coincidence of TGV-based Banach spaces with BD(, Sym l (IR d )) follows. This, in turn, can be used to obtain L p -coercivity estimates which are necessary for existence and stability of solutions for the Tikhonov-functional minimization problem (1.1). The framework for TGV-regularization with arbitrary order established in this paper may be applied to recover functions or symmetric tensor fields which can be modeled as piecewise polynomials of a prescribed order. For instance, for the regularization of height maps, it might be favorable to utilize order 3 or higher, since then, solutions tend to be piecewise polynomials of at least order 2 which appear smooth in three-dimensional visualizations. In contrast to that, TGV 2 favors piecewise linear solutions which typically exhibit polygonalization artifacts in 3D-plots (or second-order staircasing, see [10] for a discussion in dimension one). The results in this paper may moreover be useful for the development of numerical solution algorithms: A discrete version of (1.2) can directly be plugged into L 1 -type minimization methods. For instance, based on ideas in [5], one can easily compute, given a Tikhonov-functional with α regularization, an equivalent convex-concave saddle-point formulation and derive a convergent primal-dual algorithm from general saddle-point solvers such as, e.g., the one introduced in [12]. It is worth mentioning that, as the result of the great interest in the recent years, the study of regularization functionals in general, non-reflexive Banach spaces is now a wellestablished part of the theory of inverse problems (see, for instance, [15, 23, 25] and the references therein). In particular, regularization properties of the total variation and total deformation are already known for some time [1, 24]. While in most of the literature, a single regularization parameter is considered, there have also been some efforts to understand regularization and convergence behavior for multiple parameters and functionals. However, all efforts aim towards regularization where multiple functionals are summed up according to the regularization parameters [21, 17, 14] or regularization is achieved by the infimal convolution of two functionals weighted by the parameters [22]. To the best knowledge of 3

the authors, convergence for multiple parameters in a minimization cascade like (1.2) has not been considered so far. The paper is organized as follows. Section 2 introduces into the concepts which are the basis for the total generalized variation: Symmetric tensor fields and associated spaces. In particular, the needed results about spaces of symmetric tensor fields of bounded deformation are collected. In Section 3, the functional-analytic framework for TGV-regularization is established. In particular the above-mentioned minimum characterization, topological equivalence and coercivity are obtained. Section 4 then deals with showing that TGV can indeed be used as a regularizer for linear inverse problems by first obtaining existence and stability of regularized solutions according to the Tikhonov-functional (1.1) and noisy data and then showing convergence of these solutions to a TGV-minimizing solution as the noise vanishes. As in this case, also the values of the TGV-functional converge, we can speak of TGV-strict convergence. Section 5 discusses this convergence notion which is stronger than weak*-convergence but weaker than strong convergence. Finally, some conclusions are drawn in Section 6. 2 Preliminaries In the following, we will recall some notation and basic facts about the total generalized variation, mainly for the reader s convenience. More details can be found in [9, 6]. 2.1 Basic tensor calculus First, let us fix a space dimension d and introduce the tensor framework we are dealing with. As we do not consider tensors in an differential geometric setting, we only consider contravariant tensors, i.e., define the space T k (IR d ) of tensors of order k as all k-linear mappings, i.e., T k (IR d ) = {ξ : IR d IR }{{} d IR ξ k-linear}. k times Beside the basic vector space operations, we consider the tensor product : T k (IR d ) T l (IR d ) T k+l (IR d ) defined by (ξ η)(a 1,..., a k+l ) = ξ(a 1,..., a k )η(a k+1,..., a k+l ) the reverse ordering : T k (IR d ) T k (IR d ), ξ(a 1,..., a k ) = ξ(a k,..., a 1 ) as well as, for k 2, the trace tr : T k (IR d ) T k 2 (IR d ), tr(ξ)(a 1,..., a k 2 ) = d ξ(e i, a 1,..., a k 2, e i ). 4

A scalar product and Frobenius-type norm for T k (IR d ) is then given by ξ η = tr k (ξ η), ξ = ξ ξ, respectively. Finally, a tensor is symmetric if it is invariant under permutations of the arguments, i.e., ξ(a π(1),..., a π(k) ) = ξ(a 1,..., a k ) for all a 1,..., a k IR d, π S k. We denote by Sym k (IR d ) the subspace of symmetric tensors. The symmetrization : T k (IR d ) Sym k (IR d ) given by ( ξ)(a 1,..., a k ) = 1 ξ(a k! π(1),..., a π(k) ) π S k is then the orthogonal projection onto Sym k (IR d ). For a domain IR d, we call mappings from to T k (IR d ) and Sym k (IR d ) tensor fields and symmetric tensor fields, respectively. The Banach spaces of continuously differentiable (symmetric) tensor fields are then given in the usual way. In the following, we use the Banach space as well as C m (, X) = {u : X u is m times uniformly continuously differentiable}, u,m = max sup l=0,...,m x l u(x) C m c (, X) = {u C m (, X) supp u }, C m 0 (, X) = C m c (, X) in C m (, X) where X is either T k (IR d ) or Sym k (IR d ) and l u is the identification of the l-th derivative of u with a tensor field ( l u) : T k+l (IR d ) via ( l u)(x)(a 1,..., a k+l ) = D l u(x)(a 1,..., a l )(a l+1,..., a l+k ). Setting formally m = then means the intersection of the spaces for all m 0. Likewise, for a measure µ on, measurable functions u : X as well as the corresponding Lebesgue spaces L p µ(, X) are also defined the usual way and we drop the subscript if µ is the d-dimensional Lebesgue measure. Now, turning again to differentiation, note that the divergence div u : T k 1 (IR d ) of a differentiable k-tensor field u : T k (IR d ) is given by (div u) = tr( u). 5

Observe that div u is symmetric if u is symmetric. Of course, this does not hold for the derivative and we are in particular interested in the symmetrization of the derivative Eu = ( u). For distributions u D(, T k (IR d )), u D(, T k+1 (IR d )) and Eu D(, Sym k+1 (IR d )) are defined in the weak sense by u, ϕ = u, div ϕ, ϕ C c (, T k+1 (IR d )), Eu, ϕ = u, div ϕ, ϕ C c (, Sym k+1 (IR d )). This allows to define Sobolev spaces of weakly (symmetrically) differentiable functions. For the analysis of the total generalized variation, we are in particular interested in symmetric tensor fields whose weak symmetrized derivative is a Radon measure which will be described in the following. But first, we recall that by Riesz representation theorem, the space of Radon measures can be identified with the dual space of continuous functions vanishing at the boundary. We take this as a definition, i.e., M(, X) = C 0 (, X), X {T k (IR d ), Sym k (IR d )}. The Radon norm then corresponds to u M = sup { u, ϕ ϕ C 0 (, X), ϕ 1}. Besides the convergence associated with the norm, we naturally have a notion of weak*- convergence. Also note that then, a distribution u D(, X) then belongs to M(, X) if and only if the supremum of u, ϕ tested against all C c (, X) with ϕ 1 is finite, as this characterizes whether u can be extended to C 0 (, X) by density. 2.2 Symmetric tensor fields of bounded deformation Definition 2.1. Let l IN, l 0. The space BD(, Sym l (IR d )) of symmetric tensor fields of bounded deformation is defined as the Banach space BD(, Sym l (IR d )) = {u L 1 (, Sym l (IR d )) Eu M(, Sym l+1 (IR d ))}, u BD = u 1 + Eu M. Note that BD(, Sym l (IR d )) is indeed complete which is a consequence of the weak symmetrized derivative being closed with respect to L 1 (, Sym l (IR d )) and M(, Sym l+1 (IR d )). The Radon norm of Eu can also be expressed by the total deformation { } Eu M = TD l (u) = sup u div ϕ dx ϕ Cc 1 (, Sym l+1 (IR d )), ϕ 1. (2.1) For l {0, 1}, these spaces correspond to well-known spaces in the literature: BD(, Sym 0 (IR d )) = BV(), BD(, Sym 1 (IR d )) = BD() 6

where BV() denotes the space of functions of bounded variation [2] and BD() the space of vector fields of bounded deformation [27], which is studied, for instance, in mathematical plasticity [26]. Also, for scalar functions u, we have TD 0 (u) = TV(u). The general case l 0 has only recently been introduced in [6]. All spaces of bounded deformation are weaker than an analogous space of bounded variation which considers the full weak derivative instead of its symmetrization. In particular, symmetric tensor fields of bounded deformation are allowed to have jump discontinuities on hypersurfaces. Analogously to BV(), one can define a notion of weak* and strict convergence. Definition 2.2. A sequence {u n } in BD(, Sym l (IR d )) is said to converge to an u BD(, Sym l (IR d )) in the weak* sense, denoted u n u, if lim n un u 1 = 0 and Eu n Eu in M(, Sym l+1 (IR d )). It converges TD l -strictly to u if lim n un u 1 = 0 and lim n TDl (u n ) = TD l (u). These notions are motivated by the following embedding and compactness results. Theorem 2.3. Let be a bounded Lipschitz domain. Then, 1. BD(, Sym k (IR d )) L d/(d 1) (, Sym k (IR d )) continuously and 2. BD(, Sym k (IR d )) L p (, Sym k (IR d )) compactly for 1 p < d/(d 1). It is then clear that bounded sequences in BD(, Sym k (IR d )) admit a subsequence which converges in the weak*-sense. Also, we see that strict convergence implies weak*- convergence. In particular, the embeddings turn weak*-convergent sequences in BD(, Sym k (IR d )) to weakly converging sequences in L d/(d 1) (, Sym k (IR d )) (weakly* for d = 1) as well as to strongly converging sequences in L p (, Sym k (IR d )) for any p [1, d/(d 1)[ in a continuous manner. One can show that C (, Sym k (IR d )) is dense in BD(, Sym k (IR d )) with respect to strict convergence. Like functions of bounded variation, BD(, Sym k (IR d )) possess appropriate boundary traces and a Gauss-Green formula is valid. Theorem 2.4. Let IR d be a bounded Lipschitz domain. Then, there exists a boundary trace mapping i.e., γ L ( BD(, Sym l (IR d )), L 1 (, Sym l (IR d )) ), γu = u for all u C 1 (, Sym l (IR d )), which is sequentially continuous with respect to strict convergence in BD(, Sym l (IR d )) and strong convergence in L 1 (, Sym k (IR d )). 7

For each u BD(, Sym l (IR d )) and v C 1 (, Sym l+1 (IR d )) the Gauss-Green formula is valid, i.e. ( ) v deu + u div v dx = (γu) ν v dh d 1 where ν denotes the outward unit normal on. Moreover, we have a Sobolev-Korn-type inequality which relates the d/(d 1)-norm with the Radon-norm of the symmetrized derivative. Theorem 2.5. Let IR d be a bounded Lipschitz domain. Then: 1. The kernel of E in D(, Sym l (IR d )) is a finite-dimensional subspace of Sym l (IR d )-valued polynomials of degree less than l. 2. For any continuous projection R in L d/(d 1) (, Sym l (IR d )) onto ker(e), there exists a C > 0 only depending on and R, such that u Ru d/(d 1) C Eu M for all u BD(, Sym l (IR d )). Proofs of Theorems 2.3 2.5. See [6]. We will also need the result that a distribution is in BD(, Sym l (IR d )) as soon as its symmetrized derivative is a Radon measure. Theorem 2.6. Let IR d a bounded Lipschitz domain and u D(, Sym l (IR d )) such that Eu M(, Sym l+1 (IR d )) in the distributional sense. Then, u BD(, Sym l (IR d )). Proof. The proof is shown in the appendix. Finally, we like to mention higher-order spaces which base on the iterated symmetrized gradient E k = E... E. Definition 2.7. For k, l IN with k 1, l 0 define the spaces BD k (, Sym l (IR d )) = {u L 1 (, Sym l (IR d )) E k u M(, Sym l+k (IR d ))}, u BD k = u 1 + E k u M. By Theorem 2.6 we have that for a Sym l (IR d )-valued distribution u on, E k u M(, Sym l+k (IR d )) implies E k 1 u BD(, Sym l+k 1 (IR d )), hence E k u M = TD l+k 1 (E k 1 u). Naturally, this leads to a higher regularity which generalizes and is thus comparable to higher-order total variation. In particular, the spaces are nested in the sense that larger k leads to smaller spaces. Let us finally note that by considering weak*-convergence of {E k u n } and convergence {TD l+k 1 (u n )}, respectively, a notion of weak* and TD l+k 1 -strict convergence in BD k (, Sym l (IR d )) analogous to Definition 2.2 can be established. It is then immediate that strict convergence implies weak*-convergence and that bounded sets are weakly* sequentially compact. 8

3 Functional-analytic properties of TGV Having collected the necessary notions, we are now able to define and analyze the total generalized variation for symmetric tensor fields in a functional-analytic framework. As already mentioned, we will not only consider the scalar version introduced in [9], but also study a natural generalization to symmetric tensor fields of order l, first introduced in [29] and its underlying spaces. As we will see, this generality is in particular useful for the subsequent proofs. Definition 3.1. Let IR d a bounded Lipschitz domain, l IN, k IN with k 1, l 0 and α = (α 0,..., α k 1 ) with α i > 0 for each i = 0,..., k 1 a positive weight vector. The total generalized variation of order k of the l-symmetric tensor field u L 1 loc (, Syml (IR d )) is then given by { α (u) = sup u div k v dx v Cc k (, Sym k+l (IR d )), } div i v α i, i = 0,..., k 1. (3.1) The normed space of functions of bounded generalized variation BGV k (, Sym l (IR d )) is defined by Remark 3.2. BGV k (, Sym l (IR d )) = {u L 1 (, Sym l (IR d )) α (u) < }, u BGVα = u 1 + α (u). 1. It is immediate that fixing k and l but changing the weights α in α yields equivalent norms and hence the same spaces. We therefore denote BGV k (, Sym l (IR d )) without α. 2. Also note that for k = 1, α (u) = α 0 TD l (u) and hence BGV 1 (, Sym l (IR d )) = BD(, Sym l (IR d )). 3. Finally, observe that α α k 1 TD l : As one can see from the definition (2.1), the functional α k 1 TD l can be obtained by replacing the constraint ϕ 1 by ϕ α k 1 in the respective supremum. Now, for each v Cc k (, Sym k+l (IR d )) which is tested against in (3.1), div k 1 v satisfies div k 1 v α k 1, hence the supremum can only increase. We are going to prove, in the following, that the equivalence BGV k () = BD(, Sym l (IR d )) in the sense of Banach spaces actually holds true for all k. As a side-product, coercivity estimates of the type u Ru d/(d 1) C α (u) 9

where R is a projection in L d/(d 1) (, Sym l (IR d )) onto ker(e k ), will follow. Throughout this section, we assume that α is given according to Definition 3.1. In particular, is a bounded Lipschitz domain, k, l are appropriate orders and α is a positive weight vector. 3.1 The minimum representation We start with rewriting α in terms of a minimization problem which can be considered dual to (3.1). In order to derive this dual problem, some basic facts about the spaces BGV k (, Sym l (IR d )) are needed. Proposition 3.3. It holds that 1. α is lower semi-continuous in L 1 (, Sym l (IR d )), 2. α is a continuous semi-norm on BGV k (, Sym l (IR d )) with finite-dimensional kernel ker(e k ), 3. the space BGV k (, Sym l (IR d )) is a Banach space. Proof. Note that u u divk v dx is a continuous functional on L 1 (, Sym l (IR d )) for each v Cc k (, Sym k+l (IR d )). The functional α is just the pointwise supremum of a non-empty subset of these v and hence lower semi-continuous. For the same reason, α is convex. The positive homogeneity immediately follows from the definition, hence α is a semi-norm. As α (u 1 ) α (u 2 ) α (u 1 u 2 ) u 1 u 2 BGVα for each u 1, u 2 BGV k (, Sym l (IR d )), it is also continuous. Finally, for each v Cc k (, Sym k+l (IR d )) one can find a λ > 0 such that div i (λv) α i for i = 0,..., k 1. Hence, α (u) = 0 if and only if u div k v dx = 0 for each v Cc k (, Sym k+l (IR d )) which is equivalent to u ker(e k ) in the weak sense. As ker(e) considered on D(, Sym l+i (IR d )) is finite-dimensional for each i = 0,..., k 1, see Theorem 2.5, the latter has to be finitedimensional. Finally, it follows from standard arguments that for a lower semi-continuous semi-norm F on a Banach space X, the subspace where F is finite equipped with X +F is complete. Applied to L 1 (, Sym l (IR d )) and α, this yields the third statement. As a further preparation for employing duality, we note: Lemma 3.4. Let i 1 and w i 1 C i 1 0 (, Sym l+i 1 (IR d )), w i C i 0 (, Syml+i (IR d )) be distributions of order i 1 and i, respectively. Then Ew i 1 w i M = sup { w i 1, div ϕ + w i, ϕ ϕ C i c (, Sym l+i (IR d )), ϕ 1} (3.2) 10

with the right-hand side being finite if and only if Ew i 1 w i M(, Sym l+i (IR d )) in the distributional sense. Proof. Note that in the distributional sense, w i Ew i 1, ϕ = w i 1, div ϕ + w i, ϕ for all ϕ Cc (, Sym l+i (IR d )). Since Cc (, Sym l+i (IR d )) is dense in C 0 (, Sym l+i (IR d )), the distribution w i Ew i 1 can be extended to an element in C 0 (, Sym l+i (IR d )) = M(, Sym l+i (IR d )) if and only if the supremum in (3.2) is finite. In case of finiteness, it coincides with the Radon norm by definition. This enables us to derive the problem which is dual to the supremum in (3.1). We will refer to this problem as the minimum representation of α. Theorem 3.5. For α L 1 loc (, Syml (IR d )): defined according to Definition 3.1, we have for each u α (u) = min w i BD(,Sym l+i (IR d )), i=0,...,k, w 0 =u, w k =0 k α k i Ew i 1 w i M (3.3) with the minimum being finite if and only if u BD(, Sym l (IR d )) and attained for some w 0,..., w k where w i BD(, Sym l+i (IR d )) for i = 0,..., k and w 0 = u as well as w k = 0 in case of u BD(, Sym l (IR d )). Proof. First, take u L 1 loc (, Syml (IR d )) such that α (u) <. We will employ Fenchel-Rockafellar duality. For this purpose, introduce the Banach spaces X = C 1 0(, Sym 1+l (IR d ))... C k 0 (, Sym k+l (IR d )), Y = C 1 0(, Sym 1+l (IR d ))... C k 1 0 (, Sym k 1+l (IR d )), the linear operator Λ L ( X, Y ) v 1 div v 2, Λv =, v k 1 div v k and the proper, convex and lower semi-continuous functionals k F : X ], ], F (v) = u, div v 1 + I { αk i }(v i ), G : Y ], ], G(w) = I {(0,...,0)} (w), 11

where, for a given set Z, I Z denotes the indicator function of this set, i.e., I Z (z) = 0 if z Z and I Z (z) = if z / Z. With these choices, the identity α (u) = sup v X F (v) G(Λv) follows from the definition in (3.1). In order to show the representation of α (u) as in (3.3) we want to obtain α (u) = min w Y F ( Λ w ) + G (w ). (3.4) This follows from [3, Corollary 2.3] provided we can show that Y = λ 0 λ(dom(g) Λ dom(f )). For this purpose, let w Y and define backwards recursively: v k = 0 C k 0 (, Symk+l (IR d )), v i = w i div v i+1 C i 0 (, Symi+l (IR d )) for i = k 1,..., 1. Hence, v X and Λv = w. Moreover, choosing λ > 0 large enough, one can achieve that λ 1 v α k i for all i = 1,..., k, so λ 1 v dom(f ) and since 0 dom(g), we get the representation w = λ(0 Λλ 1 v). Thus, the identity (3.4) holds and the minimum is obtained in Y. Now Y can be written as Y = C 1 0(, Sym 1+l (IR d ))... C k 1 0 (, Sym k 1+l (IR d )), with elements w = (w 1,..., w k 1 ), w i C i 0 (, Symi+l (IR d )), 1 i k 1. Therefore, with w 0 = u and w k = 0 we get, as G = 0, ( F ( Λ w ) + G (w ) = sup Λ w, v + u, div v 1 v X ( = sup v X, v i α k i,,...,k u, div v 1 + k ( = α k i sup k ) I { αk i }(v i ) k 1 ) wi, div v i+1 + wi, v i v i C i 0 (,Symi+l (IR d )), v i 1 ) wi 1, div v i + wi, v i. From Lemma 3.4 we know that each supremum is finite and coincides with Ewi 1 w i M if and only if Ewi 1 w i M(, Symk+i (IR d )) for i = 1,..., k. Then, as wk = 0, according to Theorem 2.6, this already yields wk 1 BD(, Symk+l 1 (IR d )), in particular wk 1 M(, Sym k+l 1 (IR d )). Proceeding inductively, we see that wi BD(, Sym k+i (IR d )) for 12

each i = 0,..., k. Hence, it suffices to take the minimum in (3.4) over all BD-tensor fields which gives (3.3). In addition, the minimum in (3.3) is finite if u BD(, Sym l (IR d )). Conversely, if TD l (u) =, also Ew0 w 1 M = for all w1 BD(, Syml+1 (IR d )). Hence, the minimum in (3.3) has to be. In particular, by Theorem 3.5 we have a rigorous proof of the minimum representation for the scalar version discussed in [9, Remark 3.8]: Remark 3.6. For l = 1 it holds that TGV k α(u) = min w i BD(,Sym i (IR d )), i=0,...,k, w 0 =u, w k =0 k α k i Ew i 1 w i M. (3.5) Remark 3.7. The minimum representation also allows to define α TGV 1,l α 0 (u) = α 0 Eu M recursively: TGVα k+1,l (u) = min α k Eu w M + +1 w BD(,Sym l+1 (IR d α (w) )) (3.6) where α = (α 0,..., α k 1 ) if α = (α 0,..., α k ). Remark 3.8. The total generalized variation is monotonic with respect to its weights. Let α, β IR k + be two sets of weights such that α i β i for i = 0,..., k 1. Then, for each admissible w 0,..., w k in (3.3) it follows k α k i Ew i 1 w i M k β k i Ew i 1 w i M hence, α (u) (u) by taking the minimum on both sides. β 3.2 Topological equivalence and coercivity The next step is to examine the spaces BGV k (, Sym l (IR d )). Again, our aim is to prove that these space coincide with BD(, Sym l (IR d )) for fixed l and all k 1. We will proceed inductively with respect to k and hence vary k, l but still leave fixed. The induction step requires some intermediate estimates as follows. Lemma 3.9. For each k 1 and l 0 there exists a constant C 1 > 0, only depending on, k and l such that for each v BD(, Sym l (IR d )) and w ker(+1 α ) L 1 (, Sym l+1 (IR d )), ( ) E(v) M C 1 E(v) w M + v 1. 13

Proof. If this is not true for some k and l, then there exist (v n ) n N and (w n ) n N with v n BD(, Sym l (IR d )) and w n ker(+1 α ) such that 1 E(v n ) M = 1 and n v n 1 + E(v n ) w n M. This implies that (w n ) n N is bounded in terms of M in the finite dimensional space ker(+1 α ) = ker(e k ), see Proposition 3.3. Consequently, there exists a subsequence, again denoted by (w n ) n N and w ker(+1 α ) such that w n w with respect to L 1. Hence, E(v n ) w. Further we have that v n 0 and thus, by closedness of the weak symmetrized gradient, E(v n ) 0, which contradicts to E(v) M = 1. For what follows, we like to choose a family of projection operators onto the kernel of α = ker(e k ) (see Proposition 3.3). Definition 3.10. For each k 1 and l 0, let R k,l : L d/(d 1) (, Sym l (IR d )) ker(e k ) be a linear, continuous and onto projection. As ker(e k ) (on and for symmetric tensor fields of order l) is finite-dimensional, such a R k,l always exists and is not necessarily unique. With the help of these projection, the desired coercivity estimates can be formulated. Proposition 3.11. For each k 1 and l 0, there exists a constant C > 0 such that Eu M C ( u 1 + α (u) ) as well as u R k,l u d/(d 1) C α (u) (3.7) for all u L d/(d 1) (, Sym l (IR d )). Proof. We prove the result by induction with respect to k. In the case k = 1 and l 0 arbitrary, the first inequality is immediate while the second one is equivalent to the Sobolev- Korn inequality in BD(, Sym l (IR d )) (see [6, Corollary 4.20]). Now assume both inequalities hold for a fixed k and each l 0 and perform an induction step with respect to k, i.e., we fix l IN, (α 0,..., α k ) with α i > 0, as well as R k+1,l. We assume that assertion (3.7) holds for (α 0,..., α k 1 ) and any l IN. We will first show the uniform estimate for Eu M for which it suffices to consider u BD(, Sym l (IR d )), as otherwise, according to Theorem 3.5, TGV k+1,l α (u) =. Then, with the projection R k,l+1, the help of Lemma 3.9, the continuous embeddings BD(, Sym l+1 (IR d )) L d/(d 1) (, Sym l+1 (IR d )) L 1 (, Sym l+1 (IR d )) and the induction hypothesis, we can estimate for arbitrary w BD(, Sym l+1 (IR d )), Eu M C 1 ( Eu R k,l+1 w M + u 1 ) C 2 ( Eu w M + w R k,l+1 w d/(d 1) + u 1 ) C 3 ( Eu w M + +1 α (w) + u 1 ) C 4 (α k Eu w M + +1 α (w) + u 1 ) 14

for C 1, C 2, C 3, C 4 > 0 suitable. Taking the minimum over all such w BD(, Sym l+1 (IR d ) then yields Eu M C 3 ( u 1 + TGV k+1,l α (u) ) by virtue of the recursive minimum representation (3.6). In order to show the coercivity estimate, assume that the inequality does not hold true. Then, there is a sequence (u n ) n IN in L d/(d 1) (, Sym l (IR d )) such that Note that ker(tgv k+1,l α TGV k+1,l α u n R k+1,l u n d/(d 1) = 1 and 1 n TGVk+1,l α (u n ). ) = ker(e k+1 ) = rg(r k+1,l ), hence TGV k+1,l α (u n R k+1,l u n ) = (u n ) for each n. Thus, since we already know the first estimate in (3.7) to hold, E(u n R k+1,l u n ) M C 4 ( TGV k+1,l α (u n ) + u n R k+1,l u n 1 ), (3.8) implying, by continuous embedding, that (u n R k+1,l u n ) n IN is bounded in BD(, Sym l (IR d )). By compact embedding (see Proposition 2.3) we may therefore conclude that u n R k+1,l u n u in L 1 (, Sym l (IR d )) for some subsequence (not relabeled). Moreover, as R k+1,l (u n R k+1,l u n ) = 0 for all n, the limit has to satisfy R k+1,l u = 0. On the other hand, by lower semi-continuity (see Proposition 3.3) 0 TGV k+1,l α (u ) lim inf n TGVk+1,l α (u n ) = 0, hence u ker(e k+1 ) = rg(r k+1,l ). Consequently, lim n u n R k+1,l u n = u = R k+1,l u = 0. From (3.8) it follows that also E(u n R k+1,l u n ) 0 in M(, Sym l+1 (IR d )), so u n R k+1,l u n 0 in BD(, Sym l (IR d )) and by continuous embedding also in L d/(d 1) (, Sym l (IR d )). However, this contradicts u n R k+1,l u n d/(d 1) = 1 for all n, and thus the claimed coercivity has to hold. Remark 3.12. Note that from the proof one can see that the constant C only depends on k, l, α, and R k,l. This also applies for the C and c appearing in the following. We will, however, not explicitly mention the dependency. From Proposition 3.11 and Remark 3.2 we immediately get the following corollary: Corollary 3.13. For k 1 and l 0 there exist C, c > 0 such that for all u BD(, Sym l (IR d )) we have c ( u 1 + α (u) ) u 1 + TD l (u) C ( u 1 + α (u) ). (3.9) In particular, BGV k (, Sym l (IR d )) = BD(, Sym l (IR d )) in the sense of Banach space isomorphy. 15

This topological equivalence for TGV now allows us to obtain bounds on the w 1,..., w k 1 in (3.3) as the following two propositions show. Proposition 3.14. There exists a constant C > 0 such that for each u BD(, Sym l (IR d )) and w i BD(, Sym l+i (IR d )), i = 1,..., k 1 being the minimizers in the minimum representation (3.3) it holds that w i 1 + Ew i M C Eu M for all i = 1,..., k 1. Proof. We will first show the boundedness of the w i in L 1 (, Sym l+i (IR d )) by induction with respect to i. Note that α k 1 w 1 1 α k 1 Eu w 1 M + α k 1 Eu M α (u) + α k 1 Eu M 2α k 1 Eu M as α (u) α k 1 Eu M by plugging in zeros in (3.3). Now, assume that, for a i = 1,..., k 2, it holds that w i 1 C Eu M for all u BD(, Sym l (IR d )) and C > 0 independent of u. Then, by employing the equivalence (3.9) and the minimum definition (3.3), α k i 1 w i+1 1 α k i 1 Ew i w i+1 M + α k i 1 Ew i M α (u) + C TGV k i,l+i (α 0,...,α k i 1 ) (w i) + C w i 1 ( k ) ( k ) α k j Ew j 1 w j M + C α k j Ew j 1 w j M + C Eu M j=1 j=i+1 C α (u) + C Eu M C Eu M which implies the induction step. Hence, w i 1 is bounded in terms of C Eu M for all i = 1,..., k 1 and also for i = k as w k = 0. Finally, for i = 1,..., k 1, α k i 1 Ew i M α k i 1 Ew i w i+1 M + α k i 1 w i+1 1 which implies the desired estimate. α (u) + α k i 1 w i+1 1 α k 1 Eu M + C Eu M C Eu M This estimate can be interpreted as stability of the optimal higher-order information w i subtracted from the respective symmetrized gradients Ew i 1 in the differentiation cascade. In case the w 1,..., w k 1 are not optimal, they still can be bounded if the sum in (3.3) is bounded. Proposition 3.15. Let, for i = 0,..., k 1, {w n i } be sequences in BD(, Syml+i (IR d )) and denote by {w n k } = 0 the zero sequence in BD(, Symk+l (IR d )). If the sequences { Ew n 0 M } and { k } α k i Ewi 1 n wi n M are bounded, then {w n i } are bounded in BD(, Syml+i (IR d )) for i = 1,..., k 1. 16

Proof. We show that for each i = 1,..., k 1, it follows from { Ew n i 1 M} bounded that {w n i } is bounded in BD(, Syml+i (IR d )). Let 1 i k 1 and assume that { Ew n i 1 M} is bounded. We have, with α = (α 0,..., α k i 1 ), that for each n, α k i Ewi 1 n wi n M + TGV k i,l+i α (wi n ) k α k i Ewi 1 n wi n M, hence { Ewi 1 n wn i M} and {TGV k i,l+i α (wi n)} are bounded sequences. As { Ewn i 1 M} is bounded, { wi n 1} is bounded and by the norm equivalence result (3.9), {wi n } is bounded in BD(, Sym l+i (IR d )). Induction then shows the result. 4 Regularization of linear inverse problems with TGV 4.1 Existence and stability We are now able to prove existence and stability for TGV-regularized problems. We start with a general existence result and first note a sufficient condition on F for F + α to be coercive in appropriate L p -spaces. Again, we leave k and l as well as fixed, the latter being a bounded Lipschitz domain. Proposition 4.1. Let p [1, [ with p d/(d 1) and F : L p (, Sym l (IR d )) ], ]. If F is bounded from below and there exists an onto projection R : L d/(d 1) (, Sym l (IR d )) ker(e k ) such that for each sequence {u n } in L d/(d 1) (, Sym l (IR d )) it holds that (4.1) Ru n d/(d 1) and { u n Ru n d/(d 1) } bounded F (u n ), then F + α is coercive in L p (, Sym l (IR d )). Proof. We show for each sequence {u n } in L p (, Sym l (IR d )) that if {(F + α )(u n )} is bounded, then {u n } is bounded. Let such a sequence be given. Then, as F is bounded from below, it follows that {F (u n )} as well as { α (u n )} is bounded. The latter implies that each u n L d/(d 1) (, Sym l (IR d )) (see Theorem 2.3). For the projection R chosen in condition (4.1) for F, there exists a constant C > 0 such that u n Ru n d/(d 1) C α (u n ) for all n (see Proposition 3.11). Hence, { u n Ru n d/(d 1) } is bounded. Moreover { Ru n d/(d 1) } has to be bounded as otherwise, (4.1) yields that {F (u n )} is unbounded which is a contradiction. Finally, the continuous embedding of Lebesgue spaces gives that {u n } is bounded in L p (, Sym l (IR d )). With the coercivity of F + α, existence of minimizers is immediate. We present a proof for completeness. 17

Theorem 4.2. Let p [1, [ with p d/(d 1) and assume that 1. F : L p (, Sym l (IR d )) ], ] is proper, convex, lower semi-continuous, 2. F is coercive in the sense of (4.1). Then, there exists a solution of the problem min u L p (,Sym l (IR d )) F (u) + α (u). In case that there is a u BD(, Sym l (IR d )) such that F (u) <, the minimum is finite. Proof. Note that α (u) is finite if and only if u BD(, Sym l (IR d )). Thus, if F (u) = for all u BD(, Sym l (IR d )), the functional is constant and a minimizer trivially exists. Hence, assume in the following that F (u) < for some u BD(, Sym l (IR d )). Now, denote by G = F + α and consider a minimizing sequence {u n } for G which exists since G is bounded from below. Now, applying Proposition 4.1 for a p [p, d/(d 1)] and p > 1, it follows that there exists a weakly convergent subsequence of {u n } (not relabeled) with limit u L p (, Sym l (IR d )). As G is convex and lower semi-continuous (see Proposition 3.9), we deduce that u is a minimizer by weak lower semi-continuity. As G is proper, the minimum must be finite. This result can immediately be applied to linear inverse problems. Corollary 4.3. For each K : L p (, Sym l (IR d )) Y linear and continuous in some normed space Y, each f Y and each q [1, [ there exists a minimizer of the corresponding Tikhonov functional, i.e., a solution of min u L p (,Sym l (IR d )) 1 q Ku f q Y + TGVk,l α (u). (4.2) Proof. Denote by F (u) = 1 q Ku f q Y ker(k) ker(e k ) and X = and G = F + TGVk,l α. Observe that both X = { } u L p (, Sym l (IR d )) u v dx = 0 for all v X (4.3) are closed subspaces in L p (, Sym l (IR d )): Being a subset of ker(e k ) which is a finitedimensional space of polynomials, it is a finite-dimensional subspace of both L p (, Sym l (IR d )) and its dual space L p (, Sym l (IR d )). It is moreover easy to see that X complements X in L p (, Sym l (IR d )). Since G(u+v) = G(u) for each u L p (, Sym l (IR d )) and each v X, it is therefore sufficient to minimize G over X. 18

Hence, consider F = F + I X which is proper, convex lower semi-continuous and bounded from below. To verify the coercivity condition (4.1), let R : L d/(d 1) (, Sym l (IR d )) ker(e k ) be an arbitrary continuous and onto projection. Suppose that {u n } is such that Ru n d/(d 1) and { u n Ru n d/(d 1) } is bounded. First, assume that u n X for each n. Then, as K is injective on the finite-dimensional space ker(e k ) X, there is a constant C > 0 independent of n such that Ru n d/(d 1) C KRu n Y for all n. Consequently, one can find a M > 0 such that C 1 Ru n d/(d 1) M KRu n Y K u n Ru n d/(d 1) f Y Ku n f Y for all n. As the left-hand side tends to, the right-hand side must also, hence F (u n ) = F (u n ). Now, suppose u n / X for some n. Note that u n / X implies F (u n ) =, so we can apply the above argument to the subsequence of {u n } corresponding to the elements in X to achieve that F (u n ) for the whole sequence. The application of Theorem 4.2 then yields the desired statement. Remark 4.4. In case the norm Y is strictly convex, it is immediate that K injective implies that solutions of (4.2) are unique. In the general case, uniqueness may fail since is not strictly convex. α We will now turn to stability and regularization properties. The following is a consequence of abstract standard results. Theorem 4.5. The solutions of (4.2) are stable in the following sense: For a sequence {f n } which converges to some f in Y, there is a subsequence {u nν } of solutions {u n }, where each u n solves (4.2) with data f n, such that {u nν } weakly* in BD(, Sym l (IR d )) to a solution u of (4.2) with data f. Each sequence of bounded solutions {u n } possesses such a subsequence. Moreover, for a weak* convergent subsequence of solutions u nν u in BD(, Sym l (IR d )) we have that α (u nν ) α (u ) as ν. Proof. The existence of a subsequence with the desired properties is a direct consequence of [15, Theorem 3.2] applied to R = α +I X from Corollary 4.3 and weak* convergence in BD(, Sym l (IR d )). Moreover, choosing R = α and noting that bounded sequences in BD(, Sym l (IR d )) admit weakly* converging subsequences, we see from the proof of the same theorem that the limit u has to be a solution of (4.2) with data f. This also holds true for α (u nν ) α (u ) whenever u nν u as ν. We close the discussion on existence and stability by some immediate observations concerning special cases of the above theorem. Remark 4.6. 19

1. If ker(k) ker(e k ) = {0}, then each sequence of solutions {u n } with bounded data {f n } is bounded. Otherwise, the sequence of solutions constructed in the proof of Corollary 4.3, i.e. where u n X for each n, is bounded. 2. If 1 p < d/(d 1), then weak* convergence can be replaced by strong convergence in L p (, Sym l (IR d )) as BD(, Sym l (IR d )) L p (, Sym l (IR d )) is compact (see Theorem 2.3). 3. If the solution u of (4.2) for data f is unique, then the whole sequence {u n } converges weakly* to u. 4.2 Convergence for vanishing noise level Likewise, regularization properties can be noted. These are again based on the results in [15]. However, a slight modification accounting for multiple regularization parameters has to be made. Let us first take a look at how the total generalized variation behaves for fixed argument and varying parameters. For this purpose, we like also to allow the weights : We will therefore denote by α = I {0} if α =, for any norm on a normed space. Then, for α = (α 0,..., α k 1 ) ]0, ] k we set α (u) = min w i BD(,Sym l+i (IR d )), i=0,...,k, w 0 =u, w k =0 k α k i Ew i 1 w i M (4.4) which is obviously well-defined (by the direct method, using the boundedness result in Proposition 3.15) and coincides with (3.3) for finite weights. Also, we agree to set TGV 0,l = 0. Observe that a coefficient α k i being infinity can be interpreted as a restriction w i = Ew i 1 and consequently, a way of introducing powers of E into (3.3). Indeed, setting i 0 = 0, denoting by i 1,..., i M the elements of the set {i 1 i k, α k i < } in a strictly increasing manner as well as letting i M+1 = k, (4.4) can be rewritten as α (u) = min w j BD i j+1 ij (,Sym l+ij (IR d )), j=0,...,m, w 0 =u, E k i M w M =0 M α k ij E i j i j 1 w j 1 w j M. The observation now is that fixing u and choosing a convergent sequence of parameters α, the corresponding values of the TGV-functional converges, even if one allows in the limit. j=1 20

Proposition 4.7. Let u BD(, Sym l (IR d )) and {α n } be a sequence in ]0, [ k which converges to some α ]0, ] k. Then, αn(u) TGVk,l α (u) as n. Additionally, for each sequence {u n } in BD(, Sym l (IR d )) with u n u it holds that α (u) lim inf n α n(un ). Proof. We will first bound the limes superior by α (u). For this purpose, we may assume that α (u) <. This implies in particular that there are feasible w 0,..., w k such that the minimum in (4.4) is attained. Denoting by M = {i 1 i k, α k i < } and introducing c n = max i M αk i n /α k i (where we agree to set c n = 1 if M = ), one gets α n(u) k αk i n Ew i 1 w i M c n k α k i Ew i 1 w i M = c n α (u). Due to convergence of {α n }, we have c n 1 as n and consequently, lim sup n αn(u) TGVk,l α (u). An estimate on the limes inferior will follow from the proof of the second assertion by taking constant sequences. Hence, let {u n } be a sequence in BD(, Sym l (IR d )) which converges in the weak* sense to u. We may assume that lim inf n αn(u) <. Introduce w0 n,..., wn k the admissible tensor fields which admit the minimum in (3.3) for u n and α n, respectively, as well as α the weights according to α i = inf n IN αi n (which are positive due to αi n αi as n and α i being positive). Hence, lim inf n k α k i Ew n i 1 w n i M lim inf n k αk i n Ewn i 1 wi n M = lim inf n TGVk,l α n (u n ) <. It follows that the left-hand side is bounded for infinitely many n and as Ew0 n = Eun which is bounded, so Proposition 3.15 implies the existence of a subsequence of {(w0 n,..., wn k )} (not relabeled) such that each {wi n} is bounded in BD(, Syml+i (IR d )) and by weak* sequential compactness of bounded sets, one can also assume that wi n wi for some wi BD(, Syml+i (IR d )) and each i = 0,..., k. Now, for an i with αk i < we get from weak* lower semicontinuity and the invariance of the limes inferior under multiplication with a positive, convergent sequence that α k i Ew i 1 w i M lim inf n αn k i Ewn i 1 w n i M. For i such that αk i = we have that {αn k i Ewn i 1 wn i M} is bounded which implies that Ewi 1 n wn i M 0 since αk i n as n and hence, Ew i 1 w i M lim inf n Ewn i 1 w n i M = 0. 21

Together, α (u) k αk i Ew i 1 wi M k lim inf n αn k i Ewn i 1 w n i M lim inf n TGVk,l α n(un ) which shows the second assertion. Combining this with the estimate on the limes superior gives the stated convergence for αn(u). In particular, this includes the case TGVk,l (u) =. α Theorem 4.8. Let 1 p d d 1 with p <, q [1, [ and the operator K : L p (, Sym l (IR d )) Y be linear and continuous in some normed space Y. Take f Y and {f n } to be a sequence in Y which satisfies f n f Y δ n for a monotonically decreasing null sequence {δ n } of positive real numbers. Further take a sequence of positive weights {α n } and, denoting by β n = min {α n i 0 i k 1}, m = max {m 1 lim n β n αk i n = for all i < m } suppose that 1. β n 0 and δq n βn 0 as n, 2. there is a u BD m (, Sym l (IR d )) such that f = Ku. Then, there exists a sequence of solutions {u n } of (4.2) with data f n and weights α n possessing a BD(, Sym l (IR d ))-weakly* convergent subsequence {u nν } with limit u such that 1. u is a α -minimizing solution of Ku = f for some α [1, ] k and 1 2. lim ν β nν α nν (u nν ) = α (u ). Proof. At first note that, as shown in Corollary 4.3, there always exists a solution u n X of (4.2) with data f n and weights α n (where X is given by (4.3)). We denote by {u n } a sequence of such solutions. Then we define α n = αn β n ]0, [ k. For the i for which lim n α i n = we set αi = while for the remaining i, an accumulation point αi [1, [ of { αn i } can be found. This gives a α [1, ] k and a subsequence { α nν } such that α nν α as ν. From now on, we will consider {u nν }, the corresponding subsequence of solutions. Now let u BD m (, Sym l (IR d )) such that Ku = f which exists by assumption. Then, α nν (u ) is a null-sequence: Indeed, first note that α nν (u ) = β nν α nν (u ) according to the primal representation (3.3), for instance. Proposition 4.7 then implies lim ν TGVk,l α nν (u ) = α (u ) 22

for which the right-hand side is finite which can be seen by (4.4) plugging wi ν = E i u for 0 i < m, wi ν = 0 for m i k into the corresponding minimization problem. Consequently, α nν (u ) 0 as ν. With that, we get Ku nν f nν q Y q + α nν (u nν ) δq n ν q + TGVk,l α nν (u ) 0 as ν (4.5) and hence, Ku nν f in Y. Also, dividing the inequality in (4.5) by β nν leads to α nν (u nν ) Kunν f nν q Y β nν q + α nν (u nν ) δq n ν β nν q + TGVk,l α nν (u ) for which the assumption on {β n } yields ( lim sup δ q ) α (u ν nν nν nν ) lim sup ν β nν q + TGVk,l α (u ) = nν α (u ). (4.6) The next step is to bound {u nν } in terms of the BD(, Sym l (IR d ))-norm. For this purpose, let R : L d/(d 1) (, Sym l (IR d )) ker(e k ) be a linear, continuous, onto projection. Denoting 1 = (1,..., 1) IR k, we get, by Proposition 3.11 and monotonicity with respect to the weights (Remark 3.8) u nν Ru nν d/(d 1) C 1 (unν ) C α nν (u nν ) and hence boundedness of u nν Ru nν d/(d 1). Similar as in the proof of Corollary 4.3, injectivity of K on X implies the existence of a constant such that Ru nν d/(d 1) C KRu nν Y and, consequently, for C > 0 sufficiently large, C 1 Ru nν d/(d 1) C KRu nν Y K u nν Ru nν d/(d 1) f nν Y Ku nν f nν Y with right hand side being bounded. Thus also { Ru nν d/(d 1) } and hence { u nν d/(d 1) } is bounded. In order to obtain a bound on { Eu nν M }, we use Proposition 3.11 and monotonicity again to estimate Eu nν M C ( u nν 1 + 1 (unν ) ) C u nν 1 + C α nν (u nν ) (4.7) where the last term is bounded again due to (4.6) and the boundedness of u nν d/(d 1). Thus {u nν } is bounded in BD(, Sym l (IR d )) and possesses a subsequence (not relabeled) weakly* converging to some u BD(, Sym l (IR d )). By virtue of the embedding result of Theorem 2.3, u nν u also in L p (, Sym l (IR d )) and since K is weakly sequentially continuous, Ku nν Ku. As Ku nν f, it follows that Ku = f. To show the convergence of { α nν (u nν )}, observe that the addition in Proposition 4.7 gives α (u ) lim inf ν TGVk,l α (u nν nν ) 23

which in turn implies α (u ) lim inf ν TGVk,l α nν (u nν ) lim sup ν α nν (u nν ) α (u ) Now since u is arbitrary in BD m (, Sym l (IR d )) such that Ku = f, and since α (u) = for any u L p (, Sym l (IR d )) \ BD m (, Sym l (IR d )) (according to (4.4) and noting that αk i = for 1 i < m) we get that α (u ) α (u) for any u Lp (, Sym l (IR d )) such that Ku = f, meaning that u is a α -minimizing solution of Ku = f. Considering u in the estimate on the limes superior (4.6) then gives α (u nν nν ) α (u ) as ν. One can think of many refinements and extensions of the above theorem. Remark 4.9. 1. For any sequence {u n } of solutions which is bounded in L 1 (, Sym l (IR d )) there already exists a subsequence with the properties stated in Theorem 4.8. This is because then, for a subsequence where { α nν } converges, the corresponding {Eu nν } is already bounded (see (4.7)), leading to the desired boundedness in BD(, Sym l (IR d )). 2. If { 1 β n α n k m } is bounded for some m such that there is a u BD m (, Sym l (IR d )) with Ku = f, then α n(u ) 0 and Ku n f as n, i.e., for the whole sequence. In this case, we can extract from each subsequence of {u n } another subsequence {u nν } such that lim ν α nν = α for some α with αk m <. As u BD m (, Sym l (IR d )) we have lim ν α (u ) = nν α (u ) < again by Proposition 4.7. It follows that α nν (u ) 0 and Ku nν f as ν. As the initial subsequence was arbitrary, the convergence also holds for the whole sequence. 3. If the whole sequence { α n } converges to some α, then each accumulation point of {u n } is a α -minimizing solution of Ku = f. In case the latter is unique, we get weak*-convergence of the whole sequence {u n }. 4. In view of the conditions β n 0 and δq n βn 0 one might interpret {β n } as the regularization parameter and { α n } as fine-tuning parameters. Nevertheless, it is also possible to formulate conditions on {α n } which lead to the same conclusion without introducing {β n }: For instance, α n k m 0 for a 1 m k, δq n α n i 0 for each 1 i k and the existence of u BD k (, Sym l (IR d )) such that Ku = f is sufficient. 24