Multiscale Decompositions and Optimization

Transcription

1 Multiscale Decompositions and Optimization Xiaohui Wang arxiv: v2 [math.oc] 13 Mar 2013 June 19, 2018 Abstract In this thesis, the following type Tikhonov regularization problem will be systematically studied: (u t,v t ) := argmin { v X +t u Y }, u+v=f where Y is a smooth space such as a BV space or a Sobolev space and X is the space in which we measure distortion. Examples of the above problem occur in denoising in image processing, in numerically treating inverse problems, and in the sparse recovery problem of compressed sensing. It is also at the heart of interpolation of linear operators by the real method of interpolation. We shall characterize the minimizing pair (u t,v t ) for (X,Y) := (L 2 (),BV()) as a primary example and generalize Yves Meyer s result in [11] and Antonin Chambolle s result in [6]. After that, the following multiscale decomposition scheme will be studied: u k+1 := argmin u BV() L 2 () { 1 2 f u 2 L 2 +t k u u k BV }, where u 0 = 0 and is a bounded Lipschitz domain in R d. This method was introduced by Eitan Tadmor et al. and we shall improve the L 2 convergence result in [16]. Other pairs such as (X,Y) := (L p,w 1 (L τ )) and (X,Y) := (l 2,l p ) will also be mentioned. In the end, the numerical implementation for (X,Y) := (L 2 (),BV()) and the corresponding convergence results will be given. 1 Introduction: The Importance of Research Many problems in optimization and applied mathematics center on decomposing a given function f into a sum of two functions with prescribed properties. Typically, one of these functions is called a good function u and represents the properties of f we wish to maintain while the second part v represents error/distortion or noise in the stochastic setting. Examples occur in denoising in image processing, in numerically treating inverse problems, and in the sparse recovery problem of compressed sensing. The general problem of decomposing a function as a sum of two functions is also at the heart of interpolation of linear operators by the real method of interpolation. My research explores the mathematics behind such decompositions and their numerical implementation. 1

2 One can formulate the decomposition problem for any pair of Banach spaces X and Y with Y the space of good functions and X the space in which we measure distortion. Given a real number t > 0, we consider the minimization problem: K(f,t) := inf f=u+v { v X +t u Y }. (1.1) K(f,t) is called the K-functional for the pair (X,Y). The pair (u t,v t ) which minimizes K(f, t) is the Tikhonov regularization pair: (u t,v t ) := argmin u+v=f { v X +t u Y }. (1.2) One can usually prove (by compactness argument and strict convexity) that there exists a unique solution (u t,v t ) for problem (1.2). As we vary t, we obtain different decompositions. These decompositions describe how f sits relative to X and Y. There are many variants of (1.2) that are commonly used. For example, the norm of Y can be replaced by a semi-norm or a quasi-norm and sometimes the norm with respect to X is raised to a power. While the above formulation can be defined for any pair (X,Y) and any f in X + Y, in applications we are interested in specific pairs. One common setting, and the first one of interest to me, is when X = L p and Y is a smooth space such as a Sobolev space or a BV space. This particular case appears in many problems of image processing, optimization, compression, and encoding. We shall study various questions associated to such decompositions. The main problems to be investigated in this thesis are: (i) Given f and t > 0, characterize the minimizing pair (u t,v t ). (ii) Find an analytic expression for K(f, t) in terms of classical quantities and thereby characterize the interpolation spaces for a given pair (X, Y). (iii) Multiscale decompositions corresponding to the pair (X, Y) that can be derived from the characterization of the minimizing pair. (iv) Numerical methods for computing this decomposition or something close to it. The structure of this thesis is as following: Chapter 1: Introduction: The Importance of Research. Chapter 2: Basic properties of BV() and Hausdorff measure. Chapter 3: Decomposition for the pair (L 2 (),BV()), where is a bounded Lipschitz domain in R d. In this section, we first characterize the minimizing pair (u t,v t ) by studying the Euler-Lagrange equation associated to (1.2) which includes giving an appropriate setting for the boundary condition. We generalize Yves Meyer s result in [11] and Antonin Chambolle s result in [6] on the properties of (u t,v t ). Then the expression of K(f, t) follows as a simple consequence. In addition, we propose simpler proofs about characterizing the subdifferential of BV semi-norm which were first proved in [2]. 2

3 Chapter 4: Multiscale decompositions corresponding to the pair (L 2,BV). In this section, we study the scheme introduced by Eitan Tadmor et al. under the general framework of Inverse Scale Space Methods and improve the L 2 convergence result in [16]. Chapter 5: Decomposition for (X,Y) := (L p,w 1 (L τ )) with 1/τ := 1/p+1/d. Chapter 6: Decomposition for (X,Y) := (l 2,l p ) with 1 p <. Chapter 7: Numerical implementation for (X,Y) := (L 2 (),BV()) and the corresponding convergence results. 2 The Space BV() and Hausdorff Measure In this section, we will introduce some basic facts about the space BV(), where is a bounded Lipschitz domain. First, we need to give a definition of the Lipschitz domain. Definition 2.1 (Lipschitz Domain). An open set R d is a Lipschitz domain if for any x 0 thereexists r > 0andaLipschitz functionφ : R d 1 Rsuchthat uponrelabeling and reorienting the coordinates axis we have B(x 0,r) = {x B(x 0,r) : x d > Φ(x 1,...,x d 1 )}. In the following text, without specifically mentioned, we will assume is a bounded Lipschitz domain in R d and use the following notations: : Lebesgue measure of. x := (x 1,x 2,...,x d ). d x := i=1 x2 i. u := ( u x 1, u x 2,..., u x d ). D α u := α 1 x 1 α 2 x 2 α d x d u, where α := (α 1,α 2,...,α d ) and α i N {0} for 1 i d. Before introducing the space BV(), we need to give definitions of weak derivative and measure. Definition 2.2 (Weak Derivative). Let u L 1 (). For a given multi-index α, a function v L 1 () is called the α th weak derivative of u if vφ dx = ( 1) α ud α φ dx for all φ C 0 (). Here α := α 1 +α 2 + +α d. 3

4 Definition 2.3 (Measure). The α th weak derivative of u is called a measure if there exists a regular Borel (signed) measure µ on such that φ dµ = ( 1) α ud α φ dx for all φ C 0 (). In addition Dα u () denotes the total variation of the measure µ. Definition 2.4. BV() := L 1 () {u : D α u is a measure, D α u () <, α = 1}. In addition, the BV semi-norm u BV can be defined as: u BV = Du := sup { udiv(φ) dx : φ C0 (;R d ), φ(x) 1 for x }. Remark. It is easy to see, for u W 1 (L 1 ()), u BV = u dx. Theorem 2.5 (Coarea Formula). Let u BV() and define E t := {x : u(x) < t}. Then Du = dt Dχ Et. Proof. See Theorem 1.23 in [8]. Definition 2.6 (Hausdorff Measure). For set E R d, 0 k < and 0 < δ, we define Hk δ (E) := ω k2 k inf { (diam S j ) k : E S j,diam S j < δ} and j=1 j=1 H k (E) := limhk δ (E) = suphk δ (E), δ 0 δ where ω k = Γ( 1 2 )k /Γ( k 2 +1), k 0, is the Lebesgue measure of the unit ball in Rk. H k is called the k-dimensional Hausdorff measure. Example 2.7. Suppose E hasc 2 boundaryandconsiderχ E thecharacteristic function of E, then χ E BV = H d 1 ( E ). Proof. See Example 1.4 in [8]. For a bounded Lipschitz domain, the outward unit normal vector ν(x) := (ν 1,ν 2,...,ν d ) is defined H d 1 -a.e. on. Then we have the following generalized Gauss-Green theorem: div(f) dx = F(x) ν(x) dh d 1 whenever F C 1 ( ;R d ). For detailed exposition, please refer to [17]. 4

5 3 Decomposition for (X,Y) = (L 2 (),BV()) While there are many settings and potential decompositions that we shall discuss, a particular problem which is of high interest and is a primary example of the goal of my research is the problem of decomposing a function f L 2 (), where is a bounded Lipschitz domain in R d, into a sum of an L 2 function and a BV function. For any prescribed t > 0, we define the pair (u t,v t ) as the solution of the following minimization problem: (u t,v t ) := argmin u+v=f { 1 2 v 2 L 2 +t u BV }. (3.1) If we define T(u) := 1 f 2 u 2 L 2 +t u BV, then problem (3.1) is equivalent to the following problem: u t := argmin u BV() L 2 () T(u). (3.2) Problem (3.2) is closely related to the following constrained minimization problem: J(u) subject to f dx = u dx and f u L2 = σ, (3.3) min u BV() L 2 () where J(u) := u BV. It is widely used in image denoising where it is called Rudin-Osher- Fatemi model for R 2 (see [15]). If f is a given noisy image, then u t captures the main features of f and v t contains the oscillatory patterns of texture or the inherent noise in the image. In the stochastic setting, a central question is what is the best choice of t. Before introducing our main results, we need to spend a few words on the rigorous definition of the solution of (3.2). To derive the Euler-Lagrange equation associated to problem (3.2), we first consider a special case that u C 1 ( ) and is C 1. Set T(u) := 1 f 2 u 2 L 2 + tj(u) with J(u) := u dx. Consider the following minimization problem: where S := {u C 1 ( ) : is the admissible set. Notice ( x ) = x u ν min T(u), u S = 0 for x and u 0 for all x } for x 0. We can thus calculate the Gateaux derivative of J(u) for x = 0 on, when ǫ small enough, we have u S. Given u S, for any h C 1 ( ) with h ν 5

6 u+ǫh S. Hence: 1 δj(u;h) = lim ǫ 0 ǫ {J(u+ǫh) J(u)} 1 = lim { (u+ǫh) u } dx ǫ 0 ǫ 1 = lim { ( x ) x= u ǫ h+o(ǫ 2 )} dx ǫ 0 ǫ u = h dx u = div( u )h dx+ div( u h) dx u u = div( u h u )h dx+ u u ν ds = div( u )h dx, u where u := u ν. ν The reason why we choose Neumann boundary condition is to make u dx = f dx, which means the error/distortion v = f u has mean value zero. As we shall show below, u dx = f dx will automatically be satisfied when u is a minimizer for problem (3.2). The necessary condition for u to be a minimizer is: δt(u;h) = 0 which means tδj(u;h) (f u)h dx = 0 for any h C 1 ( ) with h = 0. Hence, we can informally write the Euler-Lagrange equation ν associated to problem (3.2) as: { u tdiv( u ) = f in u u = 0 on ν Now we come back to the more general case which u BV() L 2 () with J(u) := u BV = sup { udiv(φ) dx : φ V}, where V := {φ C0 1(;Rd ) : φ(x) 1 for x }. We can extend the domain of J(u) to L 2 () in the following way: { u BV u BV() L J(u) := 2 () + u L 2 ()\(BV() L 2 ()) In this way, we can also define T(u) := 1 2 f u 2 L 2 +tj(u) on L 2 (). In the following text, without specific mention, we will assume J(u) and T(u) defined on the whole space of L 2 () as above. It is easy to check that J(u) and T(u) defined in this way are proper convex functionals on L 2 (). 6

7 Lemma 3.1. J(u) is weakly lower semi-continuous with respect to L p (1 p ) topology, i.e. if u n u weakly in L p, we have: J(u) liminf n J(u n ). In addition, if liminf n J(u n ) <, we have u BV(). Proof. For any φ V, where V := {φ C0 1(;Rd ) : φ(x) 1 for x }, we have udiv(φ) dx = lim u n div(φ) dx n = liminf u n div(φ) dx n liminf n J(u n). Take φ over the set V, we get J(u) liminf n J(u n ). Now we give the existence and uniqueness result for problem (3.2) without invoking the associated Euler-Lagrange equation. Theorem 3.2. For t > 0, there exists a unique minimizer u t for problem (3.2). In addition, u t solves (3.3) for σ = f u t L2 f f L2 and u t dx = f dx, where f = 1 f dx. Proof. Let u n be a minimizing sequence for T(u). Since u L2 f L2 + 2T(u), { u n L2 } are bounded. Since L p is reflexive when 1 < p <, there exists a subsequence {u nj } such that u nj u t weakly in L 2. By the weakly lower semi-continuity of J(u) as in lemma 3.1, we have: tj(u t )+ 1 2 f u t 2 L 2 liminf {tj(u n j )+ 1 j 2 f u n j 2 L 2 } = min T(u). u BV() L 2 () Hence u t solves (3.2) and the uniqueness of the minimizer follows immediately from strict convexity of T(u). Suppose σ = f u t L2 > f f L2, then tj( f)+ 1 2 f f 2 L 2 < 1 2 f u t 2 L 2 tj(u t )+ 1 2 f u t 2 L 2, which is contradictory with the definition of the minimizer. So we have σ = f u t L2 f f L2. Similarly, if u t dx 1 f dx, let c = (f u t) dx. Then we have f (u t + c) L2 < f u t L2 while J(u t +c) = J(u t ), which is contradictory with the definition of the minimizer. Remark. There is an alternative way to get the existence proof by compactness argument for BV() (See [1]). To characterize the minimizer of problem (3.2), we have to come back to the PDE approach. The associated Euler-Lagrange equation can be formally written as: { u tdiv( Du ) = f in Du u = 0 on (3.4) ν To understand this equation correctly, we first need to give a rigorous definition of the boundary condition and the nonlinear operator div( Du Du ). 7

8 3.1 Definition of Neumann Boundary Condition Throughout this section we frequently make use of results shown by Anzellotti in [3]. To define the Neumann boundary condition in the sense of trace, we shall consider the following spaces: Set BV() p := BV() L p (). X() q = {z L (;R d ) : div(z) L q ()}, where div(z) is defined in the sense of distribution: div(z)φ dx = z φ dx for any φ C 0 (). Here z L := d i=1 z2 i L. p = 1 p q := 1 < p < p 1 1 p = If z X() q and w BV() p, we can define the functional (z,dw) : C0 () R by the formula: (z,dw),φ := wφdiv(z) dx wz φ dx. Theorem 3.3. The functional (z,dw) defined above for z X() q and w BV() p is a Radon measure on, and (z,dw) = z w dx for w W 1 (L 1 ()) L p (). Inaddition, we have B (z,dw) B (z,dw) z L B Dw for any Borel set B. Proof. Given w BV() p, we can find a sequence {w n } C () BV() p (see [8]) such that: w n w in L p () and limsup Dw n Dw for any open set A. n Ā Ā Take any φ C0 (A) and consider an open set V such that supp(φ) V A, then we have (z,dw n ),φ = w n div(φz) dx = φz w n dx. So Taking the limit for n, we get (z,dw),φ φ L (V) z L (V) V (z,dw n ),φ φ L (V ) z L (V) V Dw φ L z L V V w n dx. A Dw for any φ C 0 (A). So (z,dw) is a Radon measure and we have B (z,dw) B (z,dw) z L B Dw for any Borel set B. Since (z,dw),φ = wdiv(φz) dx = φz w dx for any w W1 (L 1 ()) L p () and φ C 0 (), we get (z,dw) = z w dx for w W1 (L 1 ()) L p (). 8

9 Theorem 3.4. Let z X() q, w BV() p and (z,dw) be defined as in Theorem 3.3. Then there exists a sequence {w n } n=0 C () BV() p such that w n w in L p () and (z,dw n ) (z,dw). Proof. For any ǫ > 0, take an open set A such that Dw < ǫ, \A and let g C0 () be such that 0 g(x) 1 in and g(x) 1 in A. We can find a sequence {w n } C () BV() p (see [8]) such that: w n w in L p () and limsup n w n dx Dw. Then where (z,dw n ) (z,dw) (z,dw n ),g (z,dw),g + (z,dw n ) (1 g)+ (z,dw) (1 g), lim sup n \A \A lim (z,dw n),g = (z,dw),g, n (z,dw n ) (1 g) z L limsup Dw n < ǫ z L, (z,dw) (1 g) So the theorem is proved, as ǫ is arbitrary. \A n \A (z,dw) < ǫ z L. Theorem 3.5. There exists a linear operator γ : X() q L ( ) such that 1. γ(z) L ( ) z L (). 2. γ(z)(x) = z(x) ν(x) H d 1 -a.e. on for z C 1 ( ;R d ). 3. z,w := wdiv(z) dx+ (z,dw) = γ(z)tr(w) dhd 1 for any w BV() p, where tr(w) L 1 ( ) is the trace of w on. Proof. Let z,w := wdiv(z)dx+ (z,dw), first wewant toshow z,w 1 = z,w 2 for any tr(w 1 ) = tr(w 2 ) and w 1,w 2 BV() p. We can find a sequence of functions {g n } C0 () such that g n w 1 w 2 in L p and (z,dg n) (z,d(w 1 w 2 )). Then we have: z,w 1 w 2 = (w 1 w 2 )div(z) dx+ (z,d(w 1 w 2 )) = lim { g n div(z) dx+ (z,dg n )} = 0. n 9

10 So z,w 1 = z,w 2. Now we want to show z,w z L tr(w) dhd 1. For bounded Lipschitz domain, given any u L 1 ( ) and ǫ > 0, we can find a function w W 1 (L 1 ()) such that tr(w) = u, w dx u dh d 1 +ǫ, w(x) = 0 for x ǫ, where ǫ := {x : dist(x, ) > ǫ}. Then for any tr(w) L 1 (), we can find ŵ W 1 (L 1 ()) such that tr(ŵ) = tr(w) with the above properties. So z,w = z,ŵ ŵdiv(z) dx + z L Dŵ ŵdiv(z) dx + z L { tr(w) dh d 1 +ǫ}. \ ǫ Since lim ǫ 0 \ ǫ ŵdiv(z) dx = 0, let ǫ goes to 0, we get z,w z L tr(w) dh d 1. (3.5) Now given a fixed z X() q, we can define the linear functional F z : L 1 ( ) R by F z (u) := z,w, where tr(w) = u. From (3.5), we know F z (u) z L () u L1 ( ). By Rieze Representation theorem, there exists γ(z) L ( ) such that F z (u) = γ(z)u dh d 1. So z,w = γ(z)tr(w) dhd 1 and γ(z) L ( ) z L (). When z C 1 ( ;R d ), z,w = div(wz) dx = tr(w)z ν dhd 1. So γ(z) = z ν H d 1 -a.e. on. Thus the function γ(z) is a weakly defined trace on of the normal component of z, we shall denote γ(z) by [z,ν]. In this way, the Neumann boundary condition can be expressed as [z,ν] = 0 H d 1 -a.e. on. 3.2 Definition of the Operator div( Du Du ) Let A : X 2 X be a multivalued mapping defined on a Banach space X, i.e., A assigns to each point u X a subset Au of X, where X is the dual space of X. In this paper we will simply call such mapping an operator. 1. The set D(A) := {u X : Au } is called the effective domain of A. When D(A), we say A is proper. 10

11 2. The set R(A) := u X Au is called the range of A. 3. The set G(A) := {(u,v) X Y : u D(A),v Au} is called the graph of A. In this paper, we briefly write (u,v) A instead of (u,v) G(A) and we will identify an operator A with its graph G(A). 4. AnoperatorAiscalledamonotoneoperator,if v 1 v 2,u 1 u 2 0for(u 1,v 1 ),(u 2,v 2 ) A. 5. A monotone operator A is called a maximal monotone operator, if for any monotone operator B that A B, we have A = B. Lemma 3.6. Let X be a Hilbert space and A : D(A) X 2 X. If A is a monotone operator, then for any λ > 0, (I + λa) 1 : X D(A) is nonexpansive, i.e. for v 1,v 2 D((I +λa) 1 ), we have (I +λa) 1 (v 1 ) (I +λa) 1 (v 2 ) X v 1 v 2 X. Proof. Let v 1 (I + λa)(u 1 ) and v 2 (I + λa)(u 2 ). Then w 1 = 1 λ (v 1 u 1 ) A(u 1 ) and w 2 = 1 λ (v 2 u 2 ) A(u 2 ). By the fact w 1 w 2,u 1 u 2 0, we have: v 1 v 2 2 X = (I +λa)(u 1 ) (I +λa)(u 2 ),(I +λa)(u 1 ) (I +λa)(u 2 ) = u 1 u 2 2 X +λ2 w 1 w 2 2 X +2λ w 1 w 2,u 1 u 2 u 1 u 2 2 X. Now we introduce the following operator A on L 2 (): Definition 3.7. v A(u) means: z X() 2 with z L 1,v = div(z) in D () such that (φ u)v dx = (z,dφ) u BV for any φ BV() L 2 (). Let s recall the set S that we used for deriving Euler-Lagrange Equation: S := {u C 1 ( ) : u ν = 0 for x and u 0 for all x }. For u S, A(u) = div( u ). Hence the operator A can be viewed as generalization of u div( u u ). Formally, we can write A(u) = div( Du Du ) for u BV() L 2(). To associate the operator A with our minimization problem (3.2), we need to introduce the concept about subdifferential. Let L : X [,+ ] be a functional on a real Banach space X. The functional u in X is called a subgradient of L at the point u if and only if L(u) ± and L(w) L(u)+ u,w u for any w X. For each u X, the set: L(u) := {u X : u is a subgradient of L at u} is called the subdifferential of L at u. Thus L is a multivalued mapping defined on X and L(u) = if L(u) = ±. 11

12 Theorem 3.8. Let L : X (, + ] be a proper convex and lower semi-continuous functional on the real Banach space X, then the subdifferential L : X 2 X is maximal monotone. Proof. See the fundamental paper by R. T. Rockafellar([14]). Theorem 3.9. Let X be a Hilbert space and A : D(A) X 2 X. Then the following two statements are equivalent: 1. A is a monotone operator and R(I +A) = X 2. A is a maximal monotone operator. Proof We only need to show that: if for any v A(u), we have v v 0,u u 0 0, then v 0 A(u 0 ). Since R(I +A) = X, we can find u 1 X such that u 0 +v 0 = u 1 +v 1 for some v 1 A(u 1 ). So v 1 v 0,u 1 u 0 = u 1 u 0 2 X 0. Consequently, we have u 0 = u 1, so v 0 A(u 0 ) See the fundamental paper by G. Minty([12]). Theorem The operator A defined above is a maximal monotone operator on L 2 (). To prove this theorem, we need to introduce a p-laplace type operator A p defined on L 2 (): (u,v) A p if and only if u W 1 (L p ()) L 2 (),v L 2 () and vφ dx = u p 2 u φ dx for any φ W 1 (L p ()) L 2 (). From the definition of A p, we can see that v = A p (u) = div( u p 2 u) in D () for u D(A p ). Lemma A p is a monotone operator and R(I + A p ) = L 2 (), i.e. A p is maximal monotone on L 2 (). Proof. (i) A p is monotone: let (u 1,v 1 ),(u 2,v 2 ) A p, then v 1 v 2,u 1 u 2 = v 1 u 1 +v 2 u 2 v 1 u 2 v 2 u 1 dx = u 1 p 2 u 1 u 1 + u 2 p 2 u 2 u 2 = 0. u 1 p 2 u 1 u 2 u 2 p 2 u 2 u 1 dx u 1 p + u 2 p u 1 p 1 u 2 u 2 p 1 u 1 dx ( u 1 p 1 u 2 p 1 )( u 1 u 2 ) dx 12

13 (ii)r(i+a p ) = L 2 (): foranyv L 2 (), weneedtofindu D(A p )suchthatu+a p (u) v. That is to say: given v L 2 (), we need to find u W 1 (L p ()) L 2 () such that (v u)φ dx = u p 2 u φ dx (3.6) forany φ W 1 (L p ()) L 2 (). It isactually aweak solution oftheeuler-lagrangeequation for the following minimization problem: min T(u) (3.7) u W 1 (L p()) L 2 () where T(u) := 1 v 2 u 2 L p u p L p. We will first prove there exists a minimizer for (3.7). Then we will show that such a minimizer is a solution for (3.6). Select a minimizing sequence {u k } k=1 for (3.7). Without loss of generality, we can assume (v u k) dx = 0 for every u k, otherwise if 1 (v u k) dx = c 0, then v (u k + c) 2 L 2 < v u k 2 L 2. Let m = inf u W 1 (L p()) L 2 ()T(u) <. Since T(u k ) m and u k Lp pt(u k ), we have sup k u k Lp <. Since u k u 1 dx = 0, applying Poincare inequality, we have: u k Lp u k u 1 Lp + u 1 Lp C u k u 1 Lp + u 1 Lp C u k Lp +C u 1 Lp + u 1 Lp. So {u k } k=1 is bounded in W1 (L p ()). In addition, since u k L2 2T(u k )+ v L2, we also have{u k } k=1 boundedinl 2(). Consequentlythereexistsasubsequence{u kj } j=1 {u k } k=1 and a function u W 1 (L p ()) such that u kj u weakly in W 1 (L p ()) and in L 2 (). So T(u) liminf j T(u kj ). By the fact that {u kj } j=1 is a minimizing sequence, we have T(u) m. But from the definition of m, m T(u). Consequently u is indeed a minimizer. Now we will show that u is a solution for (3.6). 1 δt(u;φ) = lim ǫ 0 ǫ {T(u+ǫφ) T(u)} 1 = lim ǫ 0 ǫ {1 2 ( v (u+ǫφ) 2 L 2 v u 2 L 2 )+ 1 p ( (u+ǫφ) p L p u p L p )} 1 = lim (u v)ǫφ+ u p 2 u (ǫ φ)+o(ǫ 2 ) dx ǫ 0 ǫ = (u v)φ+ u p 2 u φ dx. The necessary condition for u to be a minimizer of T(u) is δt(u;φ) = 0, which is (3.6). Roughly speaking, we want to see A(u) = lim p 1 A p (u). It is easy to show that A is monotone, to prove it is a maximal monotone operator on L 2 (), we also need the range condition: R(I +A) = L 2 (). We need the following two lemmas: Lemma A is a monotone operator on L 2 (). 13

14 Proof. Let (u 1,v 1 ),(u 2,v 2 ) A. We have: v 1 v 2,u 1 u 2 = v 1,u 1 + v 2,u 2 v 1,u 2 v 2,u 1 = v 1 u 1 +v 2 u 2 v 1 u 2 v 2 u 1 dx = (z 1,Du 1 )+(z 2,Du 2 ) (z 1,Du 2 ) (z 2,Du 1 ) = u 1 BV + u 2 BV (z 1,Du 2 ) (z 2,Du 1 ) Since (z 1,Du 2 )+(z 2,Du 1 ) u 2 BV + u 1 BV, we get v 1 v 2,u 1 u 2 0, which means A is monotone. Lemma R(I +A) = L 2 () Proof. We only need to show that: for any v L 2 (), there exists u BV() L 2 () such that (u,v u) A. By lemma 3.2, we know that: given v L 2 (), for any p > 1, there is u p W 1 (L p ()) L 2 () such that (u p,v u p ) A p. Hence we have (v u p )φ dx = u p p 2 u p φ dx (3.8) for every φ W 1 (L p ()) L 2 (). Since A p is maximal monotone, by lemma 3.6, and noticing 0 A p (0), we have u p L2 = (I +A p ) 1 v L2 v L2. (3.9) Take φ = u p and combine (3.9), we get the estimate: u p p dx = (v u p )u p dx v u p L2 u p L2 2 v 2 L 2 = M, (3.10) for any p > 1. By using Holder inequality we also have: u p dx ( 1 dx) 1 q ( u p p dx) p = q M p max{ M,,M,1} = M0. Hence {u p } p>1 is bounded in W 1 (L 1 ()) and we may extract a subsequence such that u p converges in L 1 () and almost everywhere to some u L 1 () as p 1+. By Fatou s lemma, we have u 2 dx liminf u 2 p dx p 1+ v 2 L 2. By lemma 3.1, we get Du liminf p u p dx M 0,

15 so we have that u BV() L 2 (). Since {u p } p>1 is bounded in L 2 (), without loss of generality, we can assume u p u weakly in L 2 () as p 1+. Now we want to show that { u p p 2 u p } p>1 is weakly relatively compact in L 2 (;R d ). First we need to obtain the following two estimates: u p p 1 dx ( u p p dx) p 1 p 1 p M p 1 p 1 p M0 and for any measurable subset E, u p p 2 u p dx u p p 1 dx M p p E p max{m,1} E 2, for E < 1 and 1 < p < 2. E E Hence { u p p 2 u p } p>1 is bounded and equiintegrable in L 1 (;R d ), and consequently weakly relatively compact in L 1 (;R d ). Thus without loss of generality, we can assume: there exists z L 1 (;R d ) such that, u p p 2 u p z as p 1+, weakly in L 1 (;R d ). Take φ C0 () in (3.8) and let p 1+, we obtain (v u)φ = z φ, which means v u = div(z) in D (). Now we need to prove z L 1. For any k > 0, let B p,k := {x : u p (x) > k}. By (3.10), we have: k p B p,k u p p dx u p p dx M. B p,k Hence B p,k M max{ M, M } for every 1 < p < 2 and k > 0. k p k k 2 As above, there is some g k L 1 (;R d ) such that u p p 2 u p χ Bp,k g k weakly in L 1 (;R d ) as p 1+. Now for any φ L (;R d ) with φ L 1, we can prove that u p p 2 u p χ Bp,k φ dx u p p 1 dx 1 u p p dx M B p,k k B p,k k. Since g k φ dx u p p 2 u p χ Bp,k φ dx + ( u p p 2 u p χ Bp,k g k ) φ dx, let p 1+, we get: g k φ dx M k. So g k dx M k for every k > 0. Hence g k 0 in L 1 (;R d ) and a.e.. Since we have: u p p 2 u p χ \Bp,k k p 1 max{k,k 2 } for 1 < p < 2, 15

16 there exists f k L (;R d ) and we can assume u p p 2 u p χ \Bp,k f k weakly in L (;R d ) when p 1+. Thus f k L liminf p 1+ u p p 2 u p χ \Bp,k L 1. Since for any k > 0, we can write z = f k +g k, then we have z f k = g k 0 a.e.. By the fact f k L 1, we get z L 1. To prove (u,v u) A, we also need to show (φ u)(v u) dx = (z,dφ) u BV for any φ W 1 (L 1 ()) L 2 (). For any φ W 1 (L 1 ()) L 2 (), let φ n C ( ) be such that φ n φ in W 1 (L 1 ()) L 2 () as n. Using φ n u p as a test function in (3.8), we get (v u p )(φ n u p ) dx = u p p 2 u p (φ n u p ) dx. Hence, (v u p )(φ n u p ) dx+ u p p dx = u p p 2 u p φ n dx. (3.11) Since u p u weakly in L 2 () as p 1+, we have u L2 liminf p 1+ u p L2, (v u p )φ n dx (v u)φ n dx and vu p dx vu dx. And since we have u p L1 u p Lp 1 1 p and u BV liminf p 1+ u p L1, then So u BV liminf u p Lp 1 1 p (liminf u 1 p Lp )( lim p 1+ p 1+ p 1+ 1 p ) = liminf u p Lp. p 1+ u BV (liminf u p p p 1+ L p )(limsup p 1+ u p 1 p L p ) (liminf u p p p 1+ L p )( lim M 1 p p ) = liminf u p p p 1+ p 1+ L p. Combining (3.11), we get: (v u)(φ n u) dx+ u BV Letting n, we get: (v u)(φ u) dx+ u BV z φ n dx. z φ dx. By Theorem 3.4, for any φ BV() L 2 (), we can find a sequence {φ n } W 1 (L 1 ()) L 2 () such that: φ n φ in L 2 () and (z,dφ n ) (z,dφ). So we can claim that (v u)(φ u) dx+ u BV 16 (z, Dφ)

17 holds for any φ BV() L 2 (). Consequently, we have: (v u)(φ u) dx+ u BV = (z,dφ). So the theorem is proved. Recall: J(u) := { u BV u BV() L 2 () + u L 2 ()\(BV() L 2 ()) J is a proper convex functional defined on L 2 (). Since (L 2 ()) = L 2 (), we have J L 2 () L 2 (). Now we can present the fundamental theorem in this subsection: Theorem J = A. Proof. Since the functional J(u) is proper convex and lower semi-continuous, by Theorem 3.8, we know J is maximal monotone. It is easy to see A J. As we proved above, A is maximal monotone. So J = A. Remark. The results in this section also holds for = R d. 3.3 Characterization of the Pair (L 2 (),BV()) From the definition of subdifferential, we get the following Minimum Principle: Theorem Let L : X (,+ ] be a proper functional on the real Banach space X. Then u is a minimizer for L(u) if and only if 0 L(u ). Since T(u) = t J(u) (f u), the necessary and sufficient condition for u t to be a minimizer of problem (3.2) is: t J(u t ) (f u t ) 0. (3.12) Since we have shown in Theorem 3.14 that A = J, (3.12) can be rewritten as: ta(u t ) (f u t ) 0. Theorem The following assertions are equivalent: 1. u t is a minimizer for problem (3.2). 2. u t BV() 2 and there exists z X() 2 with z L 1, f u t = tdiv(z) such that div(z)(φ u t ) dx = (z,dφ) u t BV for any φ BV() 2. 17

18 3. u t BV() 2 and there exists z X() 2 with z L 1, f u t = tdiv(z) such that { (z,du t) = u t BV [z,ν] = 0 Proof Since z L 1, we have (z,dφ) φ BV. Then: 1 t (f u t)(φ u t ) dx φ BV u t BV. Since 1 2t ((f u t) 2 (f φ) 2 ) = 1 t (f u t +φ )(φ u t ) 2 = 1 t (f u t)(φ u t ) 1 2t (φ u t) 2 1 t (f u t)(φ u t ), we have 1 f u 2t t 2 L 2 1 f 2t φ 2 L 2 φ BV u t BV for any φ BV() 2. This tells us that u t is a minimizer for problem (3.2) By Green s formula and the boundary condition [z,ν] = 0, we have div(z)(φ u t ) dx = (z,dφ) (z,du t ). Plug in (z,du t) = u t BV. We get div(z)(φ u t) dx = (z,dφ) u t BV Since u t is a minimizer for problem (3.2), we have ta(u t ) (f u t ). So there exists z X() 2 with z L 1, v t = 1 t (f u t) = div(z) such that: { v tφ dx = (z,dφ) for any φ BV() 2 (z,du t) = u t BV So z,φ = φdiv(z) dx+ (z,dφ) = 0 for any φ BV() 2. Hence [z,ν] = 0. Remark. From Theorem 3.16, we can see that the Neumann boundary condition is a natural assumption for problem (3.2) and f u t dx = tdiv(z) dx = t[z,ν] dhd 1 = 0 will automatically hold. In order to go further, we denote the homogeneous part of L p () by L p () := {v L p () : v dx = 0} and define Ẋ() p := {z X() p : [z,ν] = 0}. As a rather deep result of [4], Bourgain and Brezis prove that for every v L p () with p d, there exists z Ẋ() p such that v = div(z). This result is obviously not true for p < d and so we introduce the following norm for v L p () with 1 p : v Yp := inf {liminf k z k L : lim k div(z k ) v Lp = 0, z k Ẋ() p} and the corresponding normed vector space: Y() p := {v L p () : v Y p < }. Then we have the following characterization for Y() p : 18

19 Theorem For every v Y() p, there exists z Ẋ() p such that v = div(z), z L = v Yp. In addition, the unit ball U p := {v L p () : v Y p 1} is closed in L p norm topology. Proof. From the definition of the Yp norm, we can find a sequence {z k } Ẋ() p such that: lim div(z k) v Lp = 0, lim z k L = v Yp. k k Hence { z k L } is bounded, so up to an extraction, we can find z L (;R d ) such that z k converges to z weak-* in L (;R d ). Then for every φ C (), vφ dx = lim div(z k )φ dx = lim z k φ dx k k = z φ dx = div(z)φ dx [z,ν]φ dh d 1. Choose φ C0 (), we get div(z) = v L p () in the sense of distribution. So for any φ C (), we have: [z,ν]φ dh d 1 = 0. Hence [z,ν] = 0 H d 1 a.e. on. By weak-* lower semi-continuity of L, we get: z L lim k z k L = v Yp. By the definition of the Yp norm, we have z L v Yp. So z L = v Yp. Now let {v n } be a sequence in U p such that v n v for some v L p (), we want to show v U p. Since v n = div(z n ) with z n Ẋ() p and v n Yp = z n L, we have z n L 1. Thus we can find z L (;R d ) such that, up to an extraction, z n z weak-* in L (;R d ). So z L liminf n z n L 1. For any φ C (), we have: vφ dx = lim v n φ dx = lim z n φ dx = z φ dx. n n Since z φ dx = div(z)φ dx [z,ν]φ dhd 1, pick φ C0 Consequently, [z,ν] = 0. So v U p. (), we get v = div(z). Lemma For {v n } n=1 L p () and v Lp () with: we have v n v weakly in L p (). sup v n Lp < and lim v n v Yp = 0, n N n 19

20 Proof. Since {v n } n=1 is bounded in L p (), we can extract a subsequence {v nk } k=1 such that: v nk ˆv weakly in L p () (3.13) for some ˆv L p (). Since lim n v n v Yp = 0, without loss of generality, we can assume sup n N v n v Yp <. By Theorem 3.17, we can find {z n } Ẋ() p such that v n v = div(z n ) and z n L = v n v Yp. Since [z n,ν] = 0, combining Theorem 3.5, we have: div(z nk )φ dx = z nk φ dx+ = z nk φ dx [z nk,ν]tr(φ) dh d 1 for all φ W 1 (L 1 ()) L q (). Since lim k z nk L = 0, we have: (v nk v)φ dx = lim z nk φ dx = 0. lim k k This together with (3.13) shows v = ˆv. Consequently, every subsequence of {v n } n=1 has in turn a weakly convergent subsequence with limit v. This implies that v n v weakly in L p (). With these preliminaries in hand, I am able to give two characterizations of the minimizing pair (u t,v t ). The first is the following. Theorem Let (u t,v t ) be the minimizing pair of problem (3.1) and f := 1 f dx. We have: (i) f f Y2 t u t = f. (ii) f f Y2 t v t Y2 = t and v tu t dx = t u t BV. Proof. u t is a minimizer for problem (3.2) is equivalent to say (f u t ) ta(u t ). So there exists z X() 2 with z L 1, f u t = tdiv(z) in D () such that (z,du t) = u t BV. Since (z,du t) z L u t BV u t BV, the equality holds when u t BV = 0 or z L = When u t BV = 0, we have u = constant. To minimize f u t L2, we get u t = 1 f dx. So f f Y2 = tdiv(z) Y2 t z L t. 2. When u t BV > 0, we have z L = 1. So we have v t u t dx = t div(z)u t dx = t (z,du t ) = t u t BV. And we claim v t Y2 = t z L = t. Otherwise, there exists ẑ Ẋ() 2 with v t = tdiv(ẑ) and ẑ L < 1, then v tu t dx = t (ẑ,du t) < t u t BV, which contradicts our previous statement. 20

21 Remark. The above theorem is a generalization of Yves Meyer s result in [11] which was proved for the special case = R 2 by using techniques from harmonic analysis. Antonin Chambolle [6] has introduced another type of characterization for a finite dimensional minimization problem related to (3.1). I have generalized his result to the case (L 2,BV) as a consequence of theorem Theorem Given f L 2 (), the minimizing pair for problem (3.1) is (u t,v t ) = (f π tu2 (f),π tu2 (f)), where π tu2 (f) is the L 2 projection of f onto the set tu 2. Proof. 1. When f f Y2 t, we have v t = π tu2 (f) = f f tu When f f Y2 t, by Theorem 3.19, we have v t Y2 = t and v tu t dx = t u t BV. For any w tu 2, there exists z Ẋ() 2 such that w = div(z) and z L = w Y2 t. Thus w(f v t ) dx = wu t dx = div(z)u t dx = (z,du t ). So Thus we have: w(f v t ) dx z L u t BV t u t BV. (w v t )(f v t ) dx t u t BV t u t BV = 0 for any w tu 2. So we have v t = π tu2 (f) and u t = f π tu2 (f). Since we have characterized the minimizing pair (u t,v t ), we can now give an alternative expression for the K-functional: Theorem K(f,t) := inf f=u+v {1 2 v 2 L 2 +t u BV }. K(f,t) = π tu2 (f)f dx 1 2 π tu 2 (f) 2 L 2. Proof. From Theorem 3.19, we know that (u t,v t ) = (f π tu2 (f),π tu2 (f)) and v tu t dx = t u t BV. So K(f,t) = 1 2 π tu 2 (f) 2 L 2 + (f π tu2 (f))π tu2 (f) dx = π tu2 (f)f dx 1 2 π tu 2 (f) 2 L 2. By using Theorem 3.16, we can calculate minimizers explicitly for some simple cases. 21

22 Example Let = B(0,R) = {x R d : x R} and f = χ B(0,r) with 0 < r < R. When 0 t( B(0,r) / B(0,r) + B(0,r) /( B(0,R) B(0,r) )) 1, we have: and u t = (1 t d r ) χ B(0,r) +t d rd 1 R d r dχ B(0,R)\B(0,r) = (1 t B(0,r) B(0,r) ) B(0, r) χ B(0,r) +t χ B(0,R)\B(0,r) B(0,R) B(0,r) v t = t d r χ B(0,r) t d rd 1 R d r dχ B(0,R)\B(0,r) = t B(0,r) χ B(0, r) B(0,r) t χ B(0,R)\B(0,r). B(0,r) B(0,R) B(0,r) When t( B(0,r) / B(0,r) + B(0,r) /( B(0,R) B(0,r) )) > 1, we have: and u t = rd R dχ B(0,R) = B(0,r) B(0,R) χ B(0,R) v t = χ B(0,r) rd R dχ B(0,R) = χ B(0,r) B(0,r) B(0,R) χ B(0,R). Proof. We look for the minimizer u t with the form u t = aχ B(0,r) + bχ B(0,R)\B(0,r). Then tdiv(z) = (f u t ) = (a 1)χ B(0,r) + bχ B(0,R)\B(0,r). We take z = (a 1) x for x B(0,r), td then u t = f+tdiv(z) = a for x B(0,r). To construct z in B(0,R)\B(0,r), we will look for z with the form z = ρ( x ) x. Since z x L 1, we need ρ(r) = 1, this tells us (a 1) = 1. td r So a = 1 td. To make [z,ν] = 0, we require ρ(r) = 0. Since r x div(z) = ρ( x ) x +ρ( x )div( x x ) = ρ ( x )+ρ( x ) d 1, x we must have: t(ρ (s)+ρ(s) d 1 s ) = b for r < s < R ρ(r) = 1 ρ(r) = 0 Solve this ODE, we get: ρ(s) = Rd r d 1 R d r d s 1 d + rd 1 R d r d s and b = t d rd 1 R d r d. In addition, ρ (s) = ((d 1)( R s )d +1) rd 1 R d r d 0, so 1 ρ(s) 0 for r < s < R. Thus, u t = (1 t d r ) χ B(0,r) +t d rd 1 R d r d χ B(0,R)\B(0,r) and v t = t d r χ B(0,r) t d rd 1 R d r d χ B(0,R)\B(0,r). 22

23 To show u t is a minimizer for (3.2), we only need to check whether (z,du t) = u t BV. By Green s formula, we have: (z,du t ) = t dx = div(z)u 1 v t u t dx t = 1 t { (1 t d r )td r dx+ t d rd 1 d( td rd 1 R d r R d rd) dx} B(0,r) B(0,R)\B(0,r) = (1 t d r )d r B(0,r) td rd 1 R d r d d r d 1 R d r d( B(0,R) B(0,r) ) = (1 t d r td rd 1 R d r d)hd 1 ( B(0,r)) = u t BV. The above equality makes sense only when 1 t( d r + d rd 1 R d r d ) 0, which means 0 t( B(0,r) / B(0,r) + B(0,r) /( B(0,R) B(0,r) )) 1. Example Let f(x) = x for x [0,1]. For 0 t 1, we have: 8 2t 0 x 2t u t = x 2t x 1 2t 1 2t 1 2t x 1 For t > 1 8, we have: u t = 1 2 for x [0,1]. Proof. Since tz = v t and z L 1, we look for z with the following structure: 1 2t (x h) x h z = 1 h x 1 h 1 2t (x (1 h)) h x 1 We need to choose h such that z(0) = z(1) = 0(Neumann Boundary Condition). So we have 1 h2 = 0. Hence h = 2t. But we also require h 1, so 2t 1. 2t 2 2 Thus for t 1, we have: 8 1 (x 2t) x 2t 2t z = 1 2t x 1 2t 1 (x (1 2t)) t x 1 2t Then we have: 2t 0 x 2t u t = x 2t x 1 2t 1 2t 1 2t x 1 23

24 For t > 1, we have z = 4(x )2 +1 for x [0,1]. Then we have: u t = 1 for x [0,1]. To 2 show u t is a minimizer, we only need to check (z,du t) = u t BV : 1 (z,du t ) = 0 h 1 h z dx = 1 2h = u t BV. 4 Multiscale Decompositions The solution of minimization problems like (3.1) leads to multiscale decompositions of a general function f. In the case we have been considering, each f L 2 () is decomposed as f = k=0 w k where each w k is viewed as providing the detail of f at some scale. Currently, there are several ways to achieve this goal. The most common of these is to use a standard telescoping decomposition where w k := u tk u tk 1 and t k = 2 k. Other approaches to obtain multiscale decompositions were given by Eitan Tadmor et al. s work (see [16]) and Stanely Osher et al. s work (see [13]). Since v t Y2 for problem (3.1) depends on the parameter t, this gives us a way of decomposing a given function f L 2 () into different components based on the size of the Y2 norm of each component. This approach falls into a category of methods (called Inverse Scale Space Methods) that were introduced by Groetsch and Scherzer in [9]. It centers on using theabove Yp norm tomeasure the oscillationof v t inacetain sense. Inour language, the choice of components takes the following form: u k+1 := argmin { 1 2 f u 2 L 2 +t k J(u,u k )}, (4.1) where 1 f 2 u 2 L 2 is the L 2 -norm fit-to-data term and J(u,u k ) is a regularization term. Typically we initialize u 0 = 0 or u 0 = 1 f dx and we require that {u k} satisfies the inverse fidelity property: lim f u k L2 0. k If, as a special case, we consider BV minimization and choose J(u,u k ) as the Bregman distance defined by D(u,u k ) := u BV u k BV s(u u k ) dx, where s ( u k BV ), then (4.1) becomes the method introduced by Osher et.al. in [13]. This method has many promising properties for image denoising which were proved in [13]. The interpretation of the Bregman distance for image processing given above is somewhat ambiguous. Another possibility is to simply take J(u,u k ) := u u k BV. Roughly speaking, u k+1 u k BV measures the similarity between two images u k andu k+1. Forany choice t k > 0, u k+1 contains more detail than u k and is closer to f. We choose a sequence t 0 > t 1 > t 2 >... with lim n t n = 0. One sees that u k+1 := argmin u BV() L 2 () { 1 2 f u 2 L 2 +t k u u k BV }, (4.2) 24

25 with u 0 = 0. If we take w k+1 := u k+1 u k, w 0 := u 0 and v k := f u k, then (4.2) can be viewed as: (w k+1,v k+1 ) = argmin w+v=v k { 1 2 v 2 L 2 +t k w BV }, with v 0 = f. Thus u k+1 = k+1 n=1 w n will be a minimizer for (4.2). This is the hierarchical (L 2,BV) decomposition method introduced by Tadmor et.al. in [16]. Theorem 4.1. Let f L 2 () and {u k } k=1 be defined as in (4.2). Then we have: 1. f u k+1 L2 f u k L2. 2. f u k+1 Y2 t k Let v n := f u n, then n k=0 {2t k u k+1 u k BV + u k+1 u k 2 L 2 } = f 2 L 2 v n+1 2 L 2. Proof. 1. Since we have 1 2 f u k 2 L f u k+1 2 L 2 +t k u k+1 u k BV 1 2 f u k+1 2 L 2, f u k+1 L2 f u k L2. 2. By Theorem If v k c k Y2 < t k, where c k := 1 v k dx, then (u k+1 u k,v k+1 ) = (c k,v k c k ). Otherwise, we have: v k+1 Y2 = t k, v k+1,u k+1 u k := v k+1 (u k+1 u k ) dx = t k u k+1 u k BV. (4.3) Since u k+1 u k +v k+1 = v k, we get: v k 2 L 2 = (u k+1 u k )+v k+1,(u k+1 u k )+v k+1 = v k+1 2 L 2 + u k+1 u k 2 L 2 +2 v k+1,u k+1 u k. (4.4) Combining (4.3) and (4.4), we get: v k 2 L 2 v k+1 2 L 2 = u k+1 u k 2 L 2 +2t k u k+1 u k BV. (4.5) Sum (4.5) from k = 0 to k = n, we get: n {2t k u k+1 u k BV + u k+1 u k 2 L 2 } = f 2 L 2 v n+1 2 L 2. (4.6) k=0 In addition, we have the following L 2 convergence result: Theorem 4.2. Let f L 2 () with bounded Lipschitz domain in R d and t k = t 0 r k with 0 < r < 1, then we have: 25

26 1. lim k f u k L2 = k=0 {2t k u k+1 u k BV + u k+1 u k 2 L 2 } = f 2 L 2. Proof. 1. By Theorem 4.1, we know that { v n L2 } is a decreasing sequence. Hence, to prove v n L2 0, we only need to show v 2n+1 L2 0. Note v 2n+1 = v n 2n k=n (u k+1 u k ). Multiply v 2n+1 with itself, we get: v 2n+1 2 L 2 = v 2n+1, 2n k=n By Theorem 4.1, we know v 2n+1 Y2 t 2n. So A t 2n 2n k=n (u k+1 u k ) + v 2n+1,v n =: A+B. 2n (u k+1 u k ) BV t 2n u k+1 u k BV k=n 2n k=n t k u k+1 u k BV. From Theorem 4.1, we know that n k=0 t k u k+1 u k BV 1 2 f 2 L 2. Hence { n k=0 t k u k+1 u k BV } is a Cauchy sequence. So A 0 when n. Since v n = f u n = f n 1 k=0 (u k+1 u k ), we have: n 1 B = v 2n+1,f v 2n+1,u k+1 u k k=0 n 1 v 2n+1,f +t 2n u k+1 u k BV v 2n+1,f + t 2n t n k=0 n 1 t k u k+1 u k BV k=0 v 2n+1,f + t 2n 2t n f 2 L 2. t Since lim 2n n 0 2t n = 0, we have B 0 iff v 2n,f 0. Since u k+1 is a minimizer for (4.2), there exists z k+1 X() 2 with z k+1 L 1, v k+1 = f u k+1 = t k div(z k+1 ) = div( t k z k+1 ) L 2 () such that { (z k+1,d(u k+1 u k )) = u k+1 u k BV [z k+1,ν] = 0 Since we have sup k N v k L2 f L2 and v k+1 Y2 t k 0, by Theorem 3.18, we know v n 0 weakly in L 2 (). So we have v 2n+1,f 0. Consequently v 2n+1 L2 0. So lim k f u k L2 = Recall from Theorem 4.2, we have: n {2t k u k+1 u k BV + u k+1 u k 2 L 2 } = f 2 L 2 v n+1 2 L 2. k=0 26

27 Let n and notice v n L2 0, we have: {2t k u k+1 u k BV + u k+1 u k 2 L 2 } = f 2 L 2 k=0 Remark. In [16] (Tadmor et al.), the same result was proved under the assumption f BV()( R 2 ) or f (L 2,BV) θ with 0 < θ < 1. Here we have removed the smoothness assumption. 5 Decomposition for (X,Y) = (L p,w 1 (L τ )) Now let s consider the following (L p (),W 1 (L τ ())) decomposition with 1/τ := 1/p+1/d: (u t,v t ) := argmin u+v=f { 1 p v p L p +t u Lτ }, (5.1) where u Lτ := ( u τ dx) 1/τ and1 < p <. Inthissection, := R d orisabounded Lipschitz domain in R d. Thus by Sobolev embedding theorem, we have W 1 (L τ ()) L p (). Let { u Lτ u W J τ (u) := 1 (L τ ()) + u L p ()\W 1 (L τ ()) be a functional defined on L p (). Since (L p ) = L q for = 1, by Riesz representation p q theorem, any functional s defined on L p () can be represented as s,v := sv dx for any v L p (). To study the pair (L p (),W 1 (L τ ())), we first need to characterize the subdifferential of the functional J τ. For s L q () (1 + 1 = 1), we can define the norm p q Gτ by s Gτ := sup sv dx v Lτ 1 Theorem 5.1. (i) For u constant, J τ (u) := {s L q () : su dx = u L τ and s Gτ = 1}. (ii) For u = constant, J τ (u) := {s L q () : s G τ 1}. Proof. Given a function u W 1 (L τ ()), we can define a functional ŝ on Ru such that ŝ,cu = c u Lτ for any c R. By Hahn-Banach Theorem, we can extend the domain of the functional to W 1 (L τ ()). Let s say functional s with s Ru = ŝ and s,v v Lτ for any v W 1 (L τ ()). Since s Ru = ŝ, we have s,u = u Lτ. Consequently, we have s Gτ 1 for s L q ()(equality holds when u 0). Then if s L q (), we have: s,v u = s(v u) dx s Gτ v Lτ u Lτ J τ (v) J τ (u) 27

28 for any v W 1 (L τ ()). So s J τ (u). Conversely, if s J τ (u), then J τ (v) J τ (u) s,v u for any v W 1 (L τ ()). In addition, we have s L q (). By taking v = λu, we get (1 λ)( s,u u Lτ ) 0. By successively taking λ > 1 and λ < 1, we deduce that s,u = u Lτ. Therefore, s,v v Lτ for all v W 1 (L τ ()) and s,u = u Lτ. This implies that s Gτ 1(equality holds when u 0). Since s,c J τ (u + c) J τ (u) = 0 for any constant c, we have s dx = 0, which means s Lq (). Define the duality mapping J p : L p () L q () ( 1 p + 1 q J p (u) := u p 2 u. = 1) by: It is easy to check J p (u) = ( 1 p p L p )(u). Then we have the following theorem for the minimizing pair (L p (),W 1 (L τ ())). Theorem 5.2. Given f L p () and let (u t,v t ) be the minimizing pair of problem (5.1) and c f := argmin c R f c Lp. We have: 1. J p (f c f ) Gτ t u t = c f. 2. J p (f c f ) Gτ t J p (v t ) Gτ = t and J p(v t )u t dx = t u t Lτ. Proof. Since J p (f c f ),c 1 p ( f c f +c) p L p f c f p L p ) for any c R and the right handside of the inequality is always nonnegative, we get J p (f c f ),c = 0 for any c R, which means J p(f c f ) dx = 0. So J p (f c f ) L q (). u t is a minimizer for problem (5.1) is equivalent to say J p (f u t ) t J τ (u t ). So J p (v t ) L q (). In addition, we have J p(v t ) Gτ t and J p(v t )u t dx = t u t Lτ. 1. When J p (f c f ) Gτ t, we have J p (f c f ) t J τ (c f ). So u t = c f. 2. When J p (f c f ) Gτ > t, u t cannot be a constant. So J p (v t ) Gτ = t. 6 Decomposition for (X,Y) = (l 2,l p ) A special case that is important in analysis and in numerical methods is when X and Y are a pair of l p spaces. Such problems occur when we discretize the decomposition problems for Sobolev or Besov spaces and also when we develop numerical methods. In this chapter we shall study the minimizing pair for the case of X = l 2 := l 2 (Z) and Y = l p := l p (Z), 1 p <, i.e. the problem where b l 2. (x t,y t ) := argmin x+y=b { 1 2 y 2 l 2 +t x lp }, (6.1) 28

29 Theorem For x 0, ( x lp ) := {s l q : s x = x lp and s lq = 1}. 2. For x = 0, ( x lp ) := {s l q : s lq 1}. Proof. Given asequence x l p, we can define a functional ŝ onrxsuch that ŝ,cx = c x lp for any c R. By Hahn-Banach Theorem, we can extend the domain of the functional to l p. Let s say functional s with s Rx = ŝ and s,y y lp for any y l p. Since s Rx = ŝ, we have s,x = x lp. So s lq 1 and we have s lq = 1 for the case x 0. Then we have s (y x) s lq y lp x lp y lp x lp. So s ( x lp ). Conversely, if s ( x lp ), then y lp x lp s (y x). By taking y = λx, we get (1 λ)(s x x lp ) 0. By successively taking λ > 1 and λ < 1, we deduce that s x = x lp. Therefore, s y y lp for all y l p and s x = x lp. This implies that s lq 1 (equality holds when x 0). For the case p = 1, the minimizing pair (x t,y t ) can be obtained by the soft thresholding procedure which is widely used for wavelet shrinkage in image processing (See [7]). That is to say: x i t = sign(bi )max{0, b i t}, where x t = {x i t } and b = {bi }. It can be shown that the soft thresholding technique is a special case of the following characterization. Theorem 6.2. Given b l 2, let (x t,y t ) be the minimizing pair of problem (6.1). We have: 1. b lq t x t = b lq t y t lq = t and y t x t = t x t lp. Proof. (x t,y t ) isthe minimizing pair ofproblem (6.1)is equivalent to say: b x t t ( x t lp ). Consequently, we have y t x t = (b x t ) x t = t x t lp. 1. When b lq t, we have b t ( lp )(0). So x t = When b lq > t, x t cannot be the zero element. So y t lq = b x t lq = t. Let s define the convex set U q := {y l 2 : y lq 1} where q is the dual index to p (1/q+1/p = 1). Then the set U q is closed in l 2 norm topology, so the l 2 projection of a given sequence b l 2 onto the set U q is always well defined. Theorem 6.3. Given b l 2, the minimizing pair for problem (6.1) is (x t,y t ) = (b π tuq (b),π tuq (b)), where π tuq (b) is the l 2 projection of b onto the set tu q. 29

30 Proof. 1. When b lq t, we have y t = π tuq (b) = b tu q. 2. When b lq t, by Theorem 6.2, we have y t lq = t and y t x t = t x t lp. For any z t tu q, we have: (z t y t ) (b y t ) = z t x t y t x t z t lq x t lp t x t lp 0. So y t = π tuq (b). Consequently, x t = b π tuq (b). Remark. This result paved a road for studying (L 2,B α p (L p)) type decompositions, where B α p (L p) is a Besov space. Since the norms of L 2 and Besov spaces can be characterized by their wavelet coefficients, the (L 2,B α p(l p )) type decompositions can be derived from the decompositions for sequence spaces. 7 Numerical Implementation for (X,Y) = (L 2 (),BV()) The problem of finding a minimizing pair (u t,v t ) almost always is solved numerically. Typically, numerical methods are built through some discretization of the continuous problem. In this chapter, we will study the numerical implementation for (L 2 (),BV()) decomposition. Without loss of generality, we can assume = [0,1] d. Let D n := {2 n ([k 1,k 1 +1] [k 2,k 2 +1] [k d,k d +1]) : 0 k i 2 n 1, 1 i d} be dyadic cubes with length 2 n. We replace f by a piecewise constant approximation of f which has the following form: f n := I D n a I χ I. Thus, f n is a function in the linear space V n := {f L 2 () : f is constant on I, I D n }. The spaces {V n }, n 0, are nested, i.e., V 0 V n V n+1. As our approximation, we will take f n = P n (f), where P n (f) is the L 2 projection of f onto V n. The original problem (3.2) is then approximated by the finite-dimensional problem u n := argmin u V n { 1 2 f n u 2 L 2 +t u BV }. (7.1) We can compute f n u n L2 and u n BV exactly by discrete norms. Problem (7.1) then becomes a l 1 -minimization problem: x t := argmin x { b x 2 l 2 +t Mx l1 }, (7.2) where M is an m n matrix with the property M(x+c) = Mx for any constant vector c. Define K n (f,t) := inf { 1 u V n 2 f n u 2 L 2 +t u BV }. We have the following result about the convergence of the discrete solution to the continuous solution: 30

31 Theorem 7.1. Let u t be the minimizer of problem (3.2). We have: 1. u n u t weakly in L u n u t strongly in L p, where 1 p < d/(d 1). 3. lim n K n (f,t) = K(f,t). Proof. Since 1 2 f n u n 2 L f n u n 2 L 2 +t u n BV 1 2 f n 2 L 2, we have u n L2 f n L2. Similarly, we canget u n BV 1 2t f n 2 L 2. Since f n = P n (f) f inl 2, { u n L2 } and{ u n BV } are bounded sequences. So we can find a subsequence {u nk } such that: 1. u nk û weakly in L u nk û strongly in L p for 1 p < d/(d 1). (by the compact embedding BV() L p ()) 3. u nk BV û BV. for some û BV() L 2 (). Then we have: K(f,t) = 1 2 f u t 2 L 2 +t u t BV 1 2 f û 2 L 2 +t û BV liminf k K n k (f,t). (7.3) Given the minimizer u t BV() L 2 (), we can find a sequence {w n } with w n V n such that: w n u t in L 2 and w n BV u t BV. Since K n (f,t) = 1 2 f n u n 2 L 2 + u n BV 1 2 f n w n 2 L 2 + w n BV, we have: lim supk n (f,t) limsup n n { 1 2 f n w n 2 L 2 + w n BV } = K(f,t). (7.4) Combining (7.3) and (7.4), we know that û is a minimizer for problem (3.2) and lim supk n (f,t) = liminf K n k (f,t) = K(f,t). (7.5) n k By the uniqueness of the minimizer, we have û = u t. Since we can extract a further subsequence {u nk }, which satisfies (7.5) and has the limit u t, from any subsequence of {u n }, we have: 1. u n u t weakly in L u n u t strongly in L p, where 1 p < d/(d 1). 3. lim n K n (f,t) = K(f,t). 31