An analysis of wavelet frame based scattered data reconstruction

Transcription

1 An analysis of wavelet frame based scattered data reconstruction Jianbin Yang ab, Dominik Stahl b, and Zuowei Shen b a Department of Mathematics, Hohai University, China b Department of Mathematics, National University of Singapore, Singapore Abstract In real world applications many signals contain singularities, like edges in images. Recent wavelet frame based approaches were successfully applied to reconstruct scattered data from such functions while preserving these features. In this paper we present a recent approach which determines the approximant from shift invariant subspaces by minimizing an l -regularized least squares problem which makes additional use of the wavelet frame transform in order to preserve sharp edges. We give a detailed analysis of this approach, i.e., how the approximation error behaves dependent on data density and noise level. Moreover, a link to wavelet frame based image restoration models is established and the convergence of these models is analyzed. In the end, we present some numerical examples, for instance how to apply this approach to handle coarse-grained models in molecular dynamics. Keywords: framelet; scattered data reconstruction; l -regularized least squares; image restoration; asymptotic approximation analysis Introduction The task of scattered data reconstruction is to determine a function that approximates a given set of unorganized points. It finds applications in various fields, for instance, terrain modeling, surface reconstruction and the numerical solution of partial differential equations, see e.g., [39]. Moreover, it can be used to approximate sparse range data [24], and it can be even applied to fit coarse-grained force functions in structural biology [34, 3], as we will learn below. Let f : R d R be a function, which usually is only known on some scattered data sites Ξ = {ξ, ξ 2,..., ξ n } R d and additionally is disturbed by some noise, i.e., the given jbyang@hhu.edu.cn, corresponding author dostahl@gmail.com matzuows@nus.edu.sg

2 data is y(ξ i ) = f(ξ i ) + ɛ i for i =, 2,..., n. The task of scattered data reconstruction is now to determine a function f from some function space V that approximates the noisy data {ξ i, y(ξ i )} n i=. Most approaches determine the function f by solving a regularized least squares problem of the form min g V n (g(ξ i ) y(ξ i )) 2 + Γ(g), (.) i= where the first term measures the fitting error while the regularization term Γ(g) gives preferences to properties of the approximant f. It can for instance be chosen such that the roughness of f is penalized or such that f comes close to a piecewise continuous function, as we will present in this paper. There are also several choices for the function space V in (.), often considered spaces are the Beppo-Levi space H m, C 2 or as we will use in this paper, a shift invariant space S h (φ) := closure{ α Z d u(α)φ( h α) : u(α) R with u(α) = 0 for almost all α Zd }, which is spanned by integer translates of just one compactly supported function φ that in turn is scaled by a h > 0. Besides its structural simplicity shift invariant spaces have the beneficial property that for special choices of φ they provide good approximation orders to sufficiently smooth functions, see [25, 26, 29]. The compact support of φ also results in sparse system matrices which is of computational interest [29, 40]. Such scaling functions φ are for instance B-splines, which in turn give rise to associated wavelet frame systems, as we discuss below. Inspired by some recent wavelet frame based image restoration methodologies [7, 36, 24], we determine the approximating function f S h (φ) by minimizing the functional E h (u) := ξ Ξ ( α I u(α)φ( ξ h α) y(ξ))2 + ν diag(λ)wu l (Z d ), (.2) where u l (Z d ), I is some properly chosen index set, ν is a positive parameter, W is the discrete framelet transform and diag(λ) is a diagonal matrix based on the vector λ which scales the different wavelet channels. The basic idea behind the regularization term in (.2) is to make use of the interaction between the framelet transform and the l -norm. It is a known fact that the wavelet coefficient sequence of a signal, which is sampled from a piecewise smooth function, is sparse. Furthermore, because of the l -norm, the regularization term diag(λ)wu l gives preference to a solution u whose wavelet coefficient sequence is sparse, and to keep the singularities of functions. This property makes the approach (.2) predestined for applications like range data approximation, as it is demonstrated in [24]. However in [24] the location of the scattered data sites Ξ is only considered on regular grids, i.e., scaled subsets of Z 2 and no error analysis of the method is given. 2

3 The main focus of this paper is to extend this to bounded subsets Ω R d and to give a proper approximation analysis to the approach connected to (.2), i.e., let f W k, given some scattered data y(ξ i ) = f(ξ i )+ɛ i with ξ i Ξ Ω and some noise ɛ i, how does f f Lp(Ω) behave dependent on the data density, the noise level and the dilation h, where f := α u (α)φ( /h α) with u = arg min u E h (u) being the minimizer of (.2). Similar has been considered in [29] for the model min g S h (φ) ξ Ξ (g(ξ) y(ξ)) 2 + ν g 2 H m, whose minimizer can be seen as an approximation to the thin plate smoothing spline. Hence gives a smooth approximation to the scattered data, meaning that discontinuities are not displayed very well. Additionally in numerical considerations the regularization term g 2 H m has to be discretized, but no approximation result is given for this discrete case. This is another advantage of the approach (.2), because no discretization of the regularization term is needed as it is already in discrete form and so the approximation results which we present stay valid. Since in real world applications many signals contain singularities, the approach (.2) finds various applications. For instance almost every image contains sharp edges. In [9] a similar model to (.2) is applied to reconstruct images from uniformly sampled pixels, which is referred to as an image inpainting problem. An error estimation in terms of probability is presented in [9] but no asymptotic approximation estimation is given with regard to the resolution of images, which we will present in this paper. Furthermore, a connection between approach (.2) and wavelet frame based image restoration models [8, 5, 36, 7] is established, additionally convergence of these models is analyzed. The model (.2) is also used in [24] to approximate range data which is known to contain discontinuities. Range data is usually obtained by lasers or photometric approaches using for instance structured light, where the main concept behind these approaches is triangulation, see [2] for a tutorial. Photogrammetric technology evolved drastically in the last two decades, because of the need in the manufacturing industries, especially in casting, to quickly inspect whether a produced part is matching the original CAD model within the tolerances. Molecular dynamics is a field in computational physics which simulates the interactions of atoms based on Newton s equations of motion [9]. To model molecular interactions in big systems it can be necessary to simplify the system by combining several atoms to one group which is then treated as one single interaction site. The resulting model is then called coarse-grained model, which can also be used to investigate the long time behavior of a molecular system. In this paper we show that the model (.2) is a good approach to determine the force functions between several interaction sites in a coarse-grained model. The paper is organized as follows. Below we set up the notation and review some basic properties of B-splines and framelets. In Section 2.2 we establish a connection between the regularity of functions in S h (φ) and the discrete wavelet transform of their 3

4 coefficients, see Proposition 2.. Moreover, we give an approximation result dependent on the scattered data sites. Finally, we present an asymptotic approximation analysis of the model E h (u) and its minimizer. In Section 2.3 we provide some lemmas and technical details to give the proofs to the Propositions from Section 2.2. In Section 3 we apply our approach to wavelet frame based image restoration models and investigate the convergence of their solutions. In the last section we explain how to treat the minimization problem numerically and present some examples. 2 Analysis of the model 2. Notation and Preliminaries We use multi-index notation µ = (µ,..., µ d ), ν = (ν,..., ν d ) N d, where µ ν means µ j ν j for all j =,..., d. If µ ν and µ ν, we write µ > ν. Moreover, µ := µ + + µ d and D µ equals D µ Dµ d d, where D j denotes the partial derivative with respect to the j-th coordinate. We use X Y to denote that two variables X and Y are equivalent, that is, cy X CY for some positive constants c and C. Let φ be a real valued continuous and compactly supported function, and the shift invariant space S(φ) generated by φ can be defined by S(φ) := closure{φ a : a l 0 (Z d )}, where φ a := α Z d a(α)φ( α) is the semi-discrete convolution and l 0 (Z d ) denotes the set of all finitely supported sequences in Z d. The shift invariant space S(φ) can be refined by a dilation h > 0 S h (φ) := {f( h ) : f S(φ)}. Let φ L p (R d ) be a compactly supported function. We say that the shifts of φ are l p -stable ( p ) if there exist positive constants c p and C p such that for all sequences a l p (Z d ), c p a lp α Z d a(α)φ( α) Lp C p a lp. It is known from [27] that if the shifts of φ are l p0 -stable for some p 0, then they are l p -stable for all p. The Fourier transform of a function f L (R d ) is defined by ˆf(ξ) := f(x)e ix ξ dx, ξ R d, R d 4

5 where x ξ is the inner product of two vectors x and ξ in R d. The domain of the Fourier transform can be naturally extended to functions in L 2 (R d ) and tempered distributions. Similarly, if c l (Z d ), its Fourier series is defined by ĉ(ξ) := α Z d c[α]e iα ξ, ξ R d. We denote the Sobolev space that consists of all distributions f such that D µ f L (R d ) for all µ k by W k (Rd ) and the Sobolev semi-norm by f W k := µ =k Dµ f L. We say that f satisfies the Strang-Fix conditions [38] of order k if ˆf(0) 0, and D µ ˆf(2πα) = 0, α Z d \{0}, µ < k. The Fourier series of the first order difference filter := [, ] is 2ie i ξ 2 sin ( ξ 2 ). In general, the symmetric l-th order difference filter l can be defined by its Fourier series 2 l i l e ij l ξ 2 sin l ( ξ ), (2.) 2 where j l = {, l is odd, 0, l is even. (2.2) When l is odd, the difference filter l is symmetric around 2 ; when l is even, l is symmetric around the origin. Similarly, the symmetric l-th order average filter l can be defined by its Fourier series where := [, ]. 2 l e ij l ξ 2 cos l ( ξ 2 ), Let µ = (µ, µ 2,..., µ d ) N d, the difference filter µ on l (Z d ) is defined by the tensor product µ := µ e µ 2 e 2... µ d e d. by Let B m be the centered B-spline of order m, which is defined in the Fourier domain B m (ξ) = e ijm ξ 2 sin m ( ξ 2 ) ( ξ 2 )m, with j m as in (2.2). By the well-known recurrence formula of B-splines, the derivative of the centered B-splines can be computed in terms of lower order splines as follows, see also [, 37]. 5

6 When m is odd, d dx B m = B m ( ) B m ( ) = B m. For l < m, When m is even, for l < m and l is odd, For < l < m and l is even, d l dx l B m = B m l l. (2.3) d l dx l B m = B m l ( + ) l. (2.4) d l dx l B m = B m l l. It is known that B m is refinable with refinement mask ĥ 0 (ξ) = e ijm ξ 2 cos m ( ξ ). (2.5) 2 By B m and h 0, we define m framelet masks as (m ) ĥ l (ξ) := i l e ijm ξ 2 sin l ( ξ l 2 ) cosm l ( ξ ), l =, 2,..., m, (2.6) 2 with j m as in (2.2). See [35, 2] for more details on framelet masks. By the m + filters defined above, we can define the discrete wavelet frame transform in the multidimensional case by tensor product. Let the index i = (i, i 2,..., i d ) with 0 i, i 2,..., i d m. The wavelet frame filters (h i [k]) k Z d l (Z d ) are defined by h i [k] := h i [k ]h i2 [k 2 ]... h id [k d ], (2.7) where i r denotes the i r -th vanishing moment of h ir corresponding to the r-th variable and k = (k, k 2,..., k d ) Z d. For the (l + )-th level of wavelet frame transform, the filters are defined by h l,i := h l,i h l,i... h 0,i, where h l,i [k] = { h i [2 l k], k 2 l Z d, 0, k / 2 l Z d. Let u l (Z d ), the discrete framelet transform are defined by W l,i u := h l,i [ ] u. In general, we denote the undecimated framelet transform (see, e.g., [5, 7]) with L levels of decomposition as Wu = {W l,i u : 0 l L, 0 i, i 2,..., i d m}. 6

7 For simplicity, we choose L =, while our analysis can be extended to the general cases with L >. In this case we define W i u := W 0,i u = h i [ ] u for i = (i, i 2,..., i d ). (2.8) Let B(r) := { x < r, x R d }. The scattered data sites Ξ are assumed to lie in a bounded subset Ω R d. Suppose Ω has a Lipschitz boundary and the cone property, that is there exist positive constants ρ Ω, r Ω such that for all x Ω, there exists y Ω such that x y = ρ Ω and The separation distance in Ξ is defined by The density level of Ξ in Ω is defined by x + t(y x + r Ω B()) Ω, t [0, ]. sep(ξ) := inf{ ξ ξ : ξ, ξ Ξ, ξ ξ }. δ := δ(ξ, Ω) := sup inf x ξ. And the accumulation of Ξ in Ω is defined by x Ω ξ Ξ γ := γ(ξ, Ω, δ) := max #{ξ Ξ : x ξ δ}. x Ω 2.2 Asymptotic approximation analysis In this section we determine an asymptotic approximation analysis to the wavelet frame based approach (.2) to approximate noisy scattered data {y(ξ i )} n i= by a function f on a bounded subset Ω R d, where the scattered data is sampled at the sites ξ i Ξ Ω from an usually unknown function f, i.e., y(ξ i ) = f(ξ i ) + ɛ i with added noise satisfying y f l2 (Ξ) ɛ. More precisely, let f W k(rd ) with k d, we investigate how f f Lp(Ω) behaves dependent on the data density, the scaling parameter and the noise level, where f := α I u (α)φ( h α) and u being the minimizer of the functional E h (u). Recall, this functional is defined as E h (u) := n i=( α I u(α)φ( ξ i h α) y(ξ i)) 2 + ν diag(λ)wu l (Z d ), (2.9) where u l (Z d ), φ W k (Rd ) with stable shifts, W is the discrete framelet transform and diag(λ) is a diagonal matrix based on the vector λ which scales the different wavelet channels. This means that we select the approximant f from the shift invariant subspace S h (φ, Ω) spanned by the integer translates of φ( /h), i.e., S h (φ, Ω) = { α I u(α)φ( α) : u(α) R}, h 7

8 with I = {α Z d : supp φ( α) Ω }. h Before we present the asymptotic analysis to the minimizer of E h (u), which gives rise to the approximation f, we need to establish a connection between the regularity of functions g = α I u(α)φ( /h α) and the discrete framelet transform of their coefficients. This link is provided in the next Proposition. Proposition 2.. (I) Let φ W k (Rd ) be a compactly supported function which satisfies the Strang-Fix conditions of order k, and the shifts of φ are stable. Let g(x) = α Z d u(α)φ( x h α), where u l (Z d ). Then for any β N d and β k, D β g L h d β i β W i u l and g W k h d k W i u l. i k (II) If f W k(rd ) with k d and h > 0 is sufficiently small, then for any β N d and β k, D β f L h d β W i f(hα) l i β and where (f(hα)) α Z d f W k h d k W i f(hα) l, i k is the discrete function value sequence. Proof. See Section 2.3. Here, φ is assumed to satisfy the Strang-Fix conditions of order k, this ensures that f W k (Rd ) can be approximated by functions in S h (φ) with high order. An explicit example of such a generator is the tensor product of B-splines, φ(x) = B m (x )B m (x 2 ) B m (x d ), with x = (x, x 2,..., x d ) R d. Since B m W m (R) satisfies the m-th order Strang-Fix conditions, it holds that φ W k(rd ) and satisfies the k-th order Strang-Fix conditions if m k +. 8

9 This assumption on φ ensures that there exits b l 0 (Z d ) such that ϕ := φ b satisfies the Strang-Fix conditions of order k and the conditions ϕ q := α Z d ϕ( α)q(α) = q for all q πk d [26], where πd k denotes the set of all polynomials in d variables with degree k. For example, if φ is given by tensor product of B-splines, by [30, Chapter 4.2], we can explicitly construct a finitely supported sequence b such that ˆb(0) = and ϕ = φ b satisfies the conditions ϕ q = q for all q πk d. The main idea of construction those ϕ is to make ˆϕ very flat at the origin. Other examples of such ϕ are pseudo-splines [2, 4]. Let f S h (φ) be defined by f : = j Z d f(hj)ϕ( h j) = j Z d (f(h ) b)(j)φ( h j). Proposition 2.2. There exists δ 0 > 0 (depending only on ρ Ω and r Ω ) such that if Ξ Ω satisfies δ := δ(ξ, Ω) δ 0, then for f defined above and 0 < h, it holds that f f l2 (Ξ) C φ,k h (k d 2 ) δ d 2 γ f W k, f W k, where C φ,k is a positive constant dependent on φ and k. Proof. See Section 2.3. With Proposition 2. and Proposition 2.2 in hand, we are now ready to give an asymptotic approximation analysis to the minimizer of E h (u). Theorem 2.3. Suppose f W k(rd ) with k d. Given noisy data {ξ i, y(ξ i )} n i= R d R, where y(ξ i ) = f(ξ i ) + ɛ i. Suppose u l (Z d ) minimizes E h (u) defined by (2.9) with diag(λ) diag(h d k ) and Wu given by those W i u for which i k. Let f = u (α)φ( h α) Sh (φ, Ω), α I p and τ := max{p, 2}. Then, if h and δ are sufficiently small, f f Lp(Ω) C φ,p,k,ω( δ k d+d/p (ν h (2k d) δ d γ f 2 W k + δ d/τ (ɛ + h (k d 2 ) δ d 2 γ f W k + ν f 2 ) ), W k where C φ,p,k,ω is a constant dependent on φ, p, k and Ω. + ν ɛ 2 + f W k ) Proof. By Duchon s inequality (see [6, ], or Lemma 2.9), we have f f Lp(Ω) C p,k,ω ( δ k d+d/p f f W k (Ω) + δd/τ f ) f l2 (Ξ) C p,k,ω ( δ k d+d/p ( f W k (Ω) + f W k(ω)) + δd/τ ( f y l2 (Ξ) + y f l2 (Ξ)) ) C p,k,ω ( δ k d+d/p ( f W k (Ω) + f W k(ω)) + δd/τ ( f y l2 (Ξ) + ɛ) ). (2.0) 9

10 In the following, we estimate f W k (Ω) and f y l2 (Ξ). Since u is a minimizer of E h (u), we have f y 2 l 2 (Ξ) + ν diag(λ)wu l (Z d ) = E h (u ) E h (f(h ) b) = f y 2 l 2 (Ξ) + ν diag(λ)w(f(h ) b) l (Z d ) f y 2 l 2 (Ξ) + Cφ 2 ν diag(λ)w(f(h )) l (Z d ) 2( f f 2 l 2 (Ξ) + f y 2 l 2 (Ξ) ) + Cφ 2 ν diag(λ)w(f(h )) l (Z d ). By Proposition 2. (II), diag(λ)w i (f(h )) l (Z d ) C 3 f W k. i k This together with Proposition 2.2 implies f y 2 l 2 (Ξ) + ν diag(λ)wu l (Z d ) = E h (u ) E h (f(h ) b) 2 f f 2 l 2 (Ξ) + 2 f y 2 l 2 (Ξ) + Cφ 2 C 3 ν f W k C φ,k 4 h(2k d) δ d γ f 2 W k + 2ɛ 2 + C φ 2 C 3 ν f W k. Furthermore, by Proposition 2. (I), f W k (Ω) u (α)φ( h α) C φ 5 α Z d W k diag(λ)w i u l (Z d ). i k Hence, f y 2 l 2 (Ξ) + ν f W k (Ω) ( Cφ,k 6 h (2k d) δ d γ f 2 + ɛ 2 ) + ν f W k W k. (2.) Therefore, combining (2.0) and (2.), we conclude that f f Lp(Ω) C φ,p,k,ω ( 7 δ k d+d/p (ν h (2k d) δ d γ f 2 + ν ɛ 2 + f W k W k) + δ d/τ (ɛ + h (k d 2 ) δ d 2 γ f W k + ν f 2 ) ). W k By Theorem 2.3, it is obvious that if the dilation h of the shift invariant space is chosen equivalent to the data density δ, we have the following asymptotic approximation result. 0

11 Corollary 2.4. With the same conditions as in Theorem 2.3 it holds that f f Lp(Ω) C φ,p,k,ω( δ k d+d/p (ν δ (2k 2d) γ f 2 W k + ν ɛ 2 + f W k ) + δ d/τ (ɛ + δ k d γ f W k + ν f 2 ) ), W k if h δ, for p and τ := max{p, 2}. The generalized wavelet frame based scattered data reconstruction model can be given by minimizing the functional G h (u) := ξ Ξ α I u(α)φ( ξ h α) y(ξ) m + ν diag(λ)wu lq(z d ). In Theorem 2.3 we chose m = 2 and q = to keep the singularities of the approximation functions. In fact, if the underlying functions are smooth enough, q = 2 is recommended. An approximation analysis of G h (u) for m, q can be done similarly. In [29] the model (.) is investigated for m = 2 and q = 2 with the regularized term Γ(g) = g 2. There no approximation result for the discrete case is given, additionally the W2 k regularization term has to be discretized. In Theorem 2.3 we assumed that f W k(rd ). Furthermore, the approximation result was restrict to Ω R d, so no boundary conditions were considered. If we assume that f is defined on Ω, but on information about f outside Ω is known, the problem gets more difficult. In that case boundary wavelets should be considered. 2.3 Proof of Proposition 2. and 2.2 Here, we give the proofs of Proposition 2. and Proposition 2.2. We start by providing some technical details and lemmas. Lemma 2.5. Let φ(x) := B m (x )B m (x 2 ) B m (x d ), x = (x, x 2,..., x d ) R d, and g(x) = α Z d u(α)φ( x h α), where u l (Z d ), h > 0 is a dilation. For any β = (β, β 2,..., β d ) N d, with β = β + β β d < m, it holds that D β g L h d β β u l.

12 Proof. First, we consider the case that m is odd. By equation (2.3), the following applies D β g(x) = h β u(α)(d β φ)( x h α) α=(α,α 2,...,α d ) Z d = h β ( u(α) (B(m β ) β )( x h α )... (B (m βd ) β d )( x d h α d) ) α Z d = h β e β d e d u)(α)b (m β )( x h α )... B (m βd )( x d h α d). α Z d ( β Let θ(x) := B m β (x )B m β2 (x 2 ) B m βd (x d ), then D β g(x) = h β Thus, α Z d ( β u)(α)θ( x h α). D β g(x) dx = h β ( β u)(α)θ( x R d R d h α) dx α Z d = h d β R d α Z d ( β u)(α)θ(x α) dx. Since the shifts of θ(x) are stable, there exist two positive constants C φ and Cφ 2 independent of u such that C φ β u l ( β u)(α)θ(x α) dx C φ 2 β u l. R d α Z d Therefore, C φ hd β β u l D β g L C φ 2 hd β β u l. In case that m is even the proof works analogously. Lemma 2.6. Let the framelet masks h l be given by (2.5) and (2.6) for 0 l m. Let k N and 0 k m. For every sequence u l (Z), k u l m h l [ ] u l. Proof. By the unitary extension principle (UEP) [35], it holds that l=k Hence, m ĥ l (ξ)ĥl(ξ) =. l=0 2 k i k e ij k ξ 2 sin k ( ξ m 2 )û(ξ) = ĥ l (ξ)ĥl(ξ)2 k i k e ij k ξ 2 sin k ( ξ 2 )û(ξ). l=0 2

13 For simplicity, we use the notation ĉ(ξ) l := c l for c l (Z) in the following context. The l -norm ĉ(ξ) l is defined on the time domain essentially, and we use this notation for applying the propositions of Fourier transform easily. By (2.) and the discrete Young s inequality (see e.g. [3]), we conclude that k u l = C m ĥ l (ξ)ĥl(ξ)2 k i k e ij k ξ 2 sin k ( ξ 2 )û(ξ) l l=0 m l=0 ĥl(ξ)2 k i k e ij k ξ 2 sin k ( ξ 2 )û(ξ) l m (m ) C 2 ( i) l e ijm ξ 2 sin l+k ( ξ l 2 ) cosm (l+k) ( ξ 2 ) cosk ( ξ 2 )û(ξ) l l=0 ( m (m ) C 3 ( i) l e ijm ξ 2 sin l ( ξ l 2 ) cosm l ( ξ 2 )û(ξ) l + ĥm(ξ)û(ξ) ) l C 4 l=k m l=k h l [ ] u l. On the other hand, applying the discrete Young s inequality again, we have m h l [ ] u l = l=k = m ĥl(ξ)û(ξ) l l=k m (m ) ( i) l e ijm ξ 2 sin l ( ξ l 2 ) cosm l ( ξ 2 )û(ξ) l l=k m C 5 l=k C 6 k u l. 2 k i k e ij k ξ 2 sin k ( ξ 2 )û(ξ) l Here, the positive constants C i, i =,..., 6 are independent of the sequence u. Note, by the discrete Young s inequality, there exists a postive constant C such that for all sequences u l (Z), h l [ ] u l C k u l, when l k. But there does not exist a positive constant C such that the inverse inequality k u l C h l [ ] u l holds for all u l (Z). Lemma 2.7. Let h i be defined by (2.7) and the framelet transform W i be defined by (2.8) for i = (i, i 2,..., i d ) N d with each i r m. Then, for β = (β, β 2,..., β d ) N d with each β r m, β u l W i u l, i β 3

14 for every sequence u l (Z d ). Proof. By Lemma 2.6, the commutativity and associativity of convolution, the following applies β u l = β e ( β 2 e 2... β d e d u) l h i [ ] ( β2 e 2 β d e d u) l i β h i [ ] h i2 [ ] ( β3 e 3 β d e d u) l i β,i 2 β 2 i β,i 2 β 2,,i d β d h i [ ] h i2 [ ] h id [ ] u l = i β = i β h i [ ] u l W i u l. By Lemma 2.5 and Lemma 2.7, the following proposition holds. Proposition 2.8. Let φ(x) := B m (x )B m (x 2 ) B m (x d ), x = (x, x 2,..., x d ) R d. Let g(x) := α Z d u(α)φ( x h α), where u l (Z d ). Then D β g L h d β i β W i u l and g W k h d k W i u l i k for β N d and β < m. In Proposition 2.8, under the assumption that φ is given by the tensor product of splines, the regularity of functions in S h (φ) can be estimated by the norm of their framelet coefficients. In fact, for a general function φ, similar results hold, see Proposition 2.. Proof of Proposition 2.. (I) Let ω(x) = (D β φ)(x). Then ˆω(ξ) = (iξ ) β (iξ 2 ) β 2... (iξ d ) β d ˆφ(ξ), for ξ = (ξ, ξ 2,..., ξ d ). 4

15 Since φ satisfies the Strang-Fix conditions of order k, by Leibniz s rule, for every µ < β, it holds that D µ ˆω(2πα) = 0 for all α Z d. It follows from [22, Lemma 3.], there exists a function q having polynomial decay such that ω = q β. Since the shifts of φ are stable, ˆφ has no 2π-periodic zero points. Convolution Theorem, ˆω(ξ) ˆq(ξ) = ξ d l= (2β l i β le ij l2 βl = ˆφ(ξ) d l= (iξ l) β l d l= (2β l i β le ij βl ξ l2 ˆφ(ξ) d l= ξ β l l sin β l ( ξ l 2 ). sin β l ( ξ l 2 )) sin β l ( ξ l 2 )) Hence by the It can be verified that ˆq also has no 2π-periodic zero points. This implies that the shifts of q are stable (see e.g. [27]). Note that D β g(x) = α Z d h β u(α)(d β φ)( x h α) = α Z d h β u(α)ω( x h α) = α Z d h β u(α)(q β )( x h α) = α Z d h β ( β u)(α)q( x h α). Then by using the same method as in Lemma 2.5, we can prove that Thus, by Lemma 2.7, we have D β g L h d β β u l. D β g L h d β i β W i u l and g W k h d k W i u l. i k (II) First we consider the case that each β i and β i is odd. 5

16 Let η(x) := B β (x )B β2 (x 2 )... B βd (x d ), x = (x, x 2,..., x d ) R d. By (2.3), the following holds D β f, η( h α) = ( ) β d D (,,...,) f, D (β,β 2,...,β d ) η( h α) = ( ) β d h d β D (,,...,) f, (D (β ) B β )( x h α )... (D (β d ) B βd )( x h α d) = ( ) β d h d β D (,,...,) f, (B (β ) )( x h α )... (B (β d ) )( x d h α d), for all α = (α, α 2,..., α d ) Z d. Let the sequence c[α] := f(hα), α Z d. The Sobolev embedding theorem implies that f is continuous. Moreover, by a standard density argument, we can find a series of smooth functions to approximate B, and their derivatives approximate δ 0 δ. Hence the following applies D β f, η( h α) = ( ) β d h d β f, ( δ δ 0 ) (β ) ( x h h α )... ( δ δ 0 ) (βd ) ( x d h h α d) = ( ) β d h d β ( ( β (,...,) [ ] c)[α + (,..., )] ( β (,...,) [ ] c)[α] ). Furthermore, since β (,...,) is symmetric around the origin, we have D β f, η( h α) = ( ) β d h d β ( β c)[α + (,..., )]. Thus, by [25, Theorem 3.], it holds that D β f α Z d D β f, h d η( h α) φ( h α) L (R d ) = D β f α Z d ( ) β d h d β ( β c)[α + (,..., )] h d φ( h α) L (R d ) 0 as h 0. (2.2) Since the shifts of φ(x) are stable, we get α Z d ( ) β d h d β ( β c)[α + (,..., )] h d φ( h α) L (R d ) h d β α Z d ( β c)(α). (2.3) Combining (2.2) and (2.3) together yields if h is sufficiently small, D β f L h d β 6 α Z d β f(hα). (2.4)

17 If there exits some even β i, by (2.4), D (βi ) B βi = B ( + ) (βi ) and (βi ) is symmetric around 2, then [α + (,..., )] in (2.3) should be substituted by α plus some other integer point, (2.4) also holds. In case some component β i = 0, for example, β = 0, the proof works analogously by choosing η(x) := B (x )B β2 (x 2 )... B βd (x d ). Since f is continuous, D (0,β 2,...,β d ) f L h d β (0,β 2,...,β d ) f(x, hα 2,..., hα d )dx α=(0,α 2,...,α d ) {0} Z d h d β β f(hα). α Z d R Proof of Proposition 2.2. The idea of the proof follows [28]. Let C := [ 2, 2 )d, N := {j Z d : supp ϕ( j) C }, and r be a positive number such that N B(r) and C B(r). Let f h := f(h ) and f h := f(h ) = ϕ f(h ). Since f W k, by the generalized Poincaré inequality (see e.g. [7, Theorem 3.2]), for every α Z d, there exists q α πk d such that f h q α L (B(r)+α) C r,k f h W k (B(r)+α). Note that ϕ q = q for all q π d k. It follows that for any given α Zd, f h f h L (C+α) = f h q α (f h q α ) L (C+α) ( fh (j) q α (j) ) ϕ( j) L (C+α) + f h q α L (C+α) j N +α ( fh (α + j) q α (α + j) ) ϕ( j α) L (C+α) + f h q α L (C+α) j N ( + #(N ) ϕ L ) f h q α L (B(r)+α) C φ 2 f h q α L (B(r)+α) C r,k Cφ 2 f h W k (B(r)+α). Noticing that f h W k = h k d f W k, for any bounded set of points E R d, if µ := sep(e) > 0, it holds that f h f h 2 l 2 (E) Cd 3 µ d α Z d f h f h 2 L (C+α) C φ,k 4 µ d α Z d f h 2 W k (B(r)+α) C φ,k 5 µ d h 2(k d) f 2. W k 7

18 By virtue of [29, Lemma 2.], it is possible to partition Ξ as Ξ = N i= Ξ i such that N C d 6 γ and sep(ξ i) δ, i =, 2,..., n. Therefore, with Ξ i := h Ξ i, f f 2 l 2 (Ξ) = f N h f h 2 l 2 (h Ξ) = f h f h 2 l 2 ( Ξ i ) N i= i= C φ,k 5 sep( Ξ i ) d h 2(k d) f 2 W k C φ,k 5 Cd 6(h δ) d h 2(k d) γ f 2 W k C φ,k 7 h(2k d) δ d γ f 2. W k It follows that f f l2 (Ξ) C φ,k 7 h (k d 2 ) δ d 2 γ f W k. The following Duchon s inequality plays an important role in the proof of Theorem 2.3. It was first proved in [6], and the generalized form can be found in [, Theorem 4.]. Lemma 2.9. (Duchon s inequality) Let p, q, m. Suppose k d if q = ; k > d/q if < q < or k N if q =. Let τ := max{p, q, m}. Then there exists δ 0 > 0 (dependent on Ω, d, k) such that if Ξ Ω satisfies δ := δ(ξ, Ω) δ 0, it holds that g Lp(Ω) C p,k,ω,q,m( δ k d/q+d/p g W k q (Ω) + δ d/τ g lm(ξ)), g W k q (Ω), where C p,k,ω,q,m is a positive constant dependent on p, k, Ω, q, and m. 3 Applications to image restoration models 3. Image inpainting In this section, we link the image restoration problems to the scattered data reconstruction. Ideas of section 2.2 are applied to analyze the asymptotic behavior of the wavelet frame models in image restoration. Let f be a function on Ω := [0, ] [0, ] which represents an image. Given partial observations { f(2 n α) + ɛ[α], α Λ n, v[α] = (3.) unknown, α I n \Λ n, with I n := {α = (α, α 2 ) Z 2, 0 α, α 2 2 n }, Λ n I n, which are disturbed by some noise ɛ. The image inpainting problem is to recover f on Ω, or its discrete version on I n. 8

19 Image inpainting is a fundamental problem in image processing and has been widely investigated during the last decade (see e.g. [2, 6, 8, 5, 36, 9, 7]). It can be seen as a special case of the scattered data reconstruction problem if we consider Ξ = {2 n α, α Λ n } Ω as our scattered data sites and the pixel values as y(2 n α) = v[α] = f(2 n α) + ɛ[α]. Hence, similar to the scattered data reconstruction, the underlying image function f can be approximated from S h (φ, Ω). Additionally, if φ is chosen to be an interpolatory function, that is φ(0) = and φ(α) = 0 for all α Z 2 \{0}, the matrix A ξ,α := φ(2 n ξ α) with ξ, α I n is the identity matrix. Thus, the wavelet frame based inpainting model [8, 36, 9] min u v 2 u l 2 (Λ + diag(λ)wu n) l = min ( u(α)φ( ξ u h α) y(ξ))2 + diag(λ)wu l (3.2) α I n ξ Ξ matches our approach with h = 2 n. Let u n be a minimizer of (3.2). Then g n := α I n u n(α)φ( h α) approximates the image function f. In case Λ n is uniformly sampled from the image domain I n, an error analysis of u n f(2 n ) l2 in terms of probability is given in [9] based on the uniform law of large numbers. In this section, we consider the observations of image as scattered data sites and image restoration as a scattered data reconstruction problem. Under some mild conditions for f, the convergence of g n to f is given in section Image denoising In general the pixel values are considered as local weighted averages of the analog image function f. For example, let φ be the B-spline function and denote the scaled functions by φ n,α := 2 n φ(2 n α). Let f L 2 (Ω). Then each pixel value p is obtained by a discrete operator T φ,n (see [33, 0, 9, 7]), p[α] := T φ,n f[α] := 2 n f, φ n,α, α = (α, α 2 ), 0 α, α 2 2 n. (3.3) The consistency between pointwise sampling (3.) and local weighted averages sampling (3.3) is proved in [3, Lemma 6.]. We assume that p l M holds for all 9

20 n, where M is a given constant. In image denoising, an usually unknown image f is reconstructed or denoised, by observations v that are disturbed by additive noise ɛ v[α] = p[α] + ɛ[α], see [33, 7]. Let ɛ n := ɛ l2 (I n) denote the noise level. In [8, 5, 36, 7] a wavelet frame based approach has been used for image denoising, where the denoised image is obtained by a minimizer u n of the model g n := α I n u n(α)φ(2 n α) E n (u) := u v 2 l 2 (I n) + diag(λ)wu l (I n). Let h i be 2-D masks given by (2.7) with d = 2 and i := (i, i 2 ). Then the corresponding 2-D refinable function and framelets are defined by ψ i (x, y) = ψ i (x)ψ i2 (y), 0 i, i 2 m; (x, y) R 2, with ψ i := 2 2 k Z 2 h i[k]φ(2 k) and ψ 0 := φ = B m (x )B m (x 2 ). We denote by M n I n the index set which contains all α such that ψ i,n,α := 2 n ψ i (2 n α) is completely supported in Ω for all i. For our analysis below we use two assumptions, one on the smoothness of f in the interior of Ω, which can be characterized by the decay of the wavelet frame coefficients, see [23, 5]. Moreover, we assume the noise level to be relatively small. Hence, we assume A: There exits s > 0 such that α M n 2 2sn f, ψ i,n,α 2 is uniformly bounded for all i. A2: lim n 2 n ɛ n = 0. As most images possess sharp edges, they are usually modeled by piecewise smooth functions to preserve these discontinuities. Therefore, small s is desirable in order to reflect the low regularity of the underlying image. We assume 0 < s <. Let 0 < s 0 < s and A be satisfied, then by the Cauchy-Schwarz inequality, for i, lim n 2(s s 0)n ( 2 2n ) W i p[α] α M n = lim 2 (s s0 )n 2 n W i T φ,n f[α] n α M n = lim n 2(s s 0)n ( 2 2n W i f(2 n ), ) φ( α) α M n = lim 2 (s s0 )n f, ψ i,n,α n α M n lim n 2 s 0n ( 2 2sn f, ψ i,n,α 2) 2 α M n = 0. (3.4) 20

21 The decay of the wavelet coefficients links to the regularity of the underlying image function where the pixel values p derived from. It is reasonable to assume that for 2 2n pixel values, the average values of the discrete frame coefficients in M n decay to zero when n tends to infinity, as (3.4) indicates. Proposition 3.. Let u n be a minimizer of E n with λ 0,0 = 0 and λ i 2 (s s 0)n for some 0 < s 0 < s and all i, then if (A) and (A2) are satisfied. lim n g n f L2 (Ω) = 0 Proof. Applying the triangle inequality, we get α I n u n(α)φ(2 n α) f L2 (Ω) α I n f(2 n ), φ( α) φ(2 n α) f L2 (Ω) + α I n (u n(α) f(2 n ), φ( α) )φ(2 n α) L2 (Ω). By [7, Lemma 4.], we obtain lim f(2 n ), φ( α) φ(2 n α) f L2 (Ω) = 0. n α M n Since #{α : α I n \M n } C 2 n and the Lebesgue measure of {x R 2 : x supp φ(2 n α)} is less than C 2 2 2n, where C and C 2 depend only on the support of ψ i and φ, it holds lim n lim n M f(2 n ), φ( α) φ(2 n α) L2 (Ω) α I n\m n α I n\m n φ(2 n α) L2 (Ω) lim n M φ L C 2 n C 2 2 2n = 0. Therefore, lim f(2 n ), φ( α) φ(2 n α) f L2 (Ω) = 0. n α I n 2

22 Furthermore, since the shifts of φ are stable, we have α I n (u n(α) f(2 n ), φ( α) )φ(2 n α) L2 (Ω) α I n (u n(α) f(2 n ), φ( α) )φ(2 n α) L2 (R 2 ) C 3 2 n u n p l2 (I n) C 3 2 n ( u n v l2 (I n) + p v l2 (I n)) C 3 2 n ( u n v l2 (I n) + ɛ n ). Thus, to complete the proof it is left to show that lim n 2 n u n v l2 (I n) = 0. Since u n is a minimizer of E n, the following applies By assumption A2, we obtain that 2 2n u n v 2 l 2 (I n) 2 2n E n (u n) 2 2n E n (p) = 2 2n p v 2 l 2 (I + n) 2 2n diag(λ)wp l (I n) 2 2n ɛ 2 n + 2 (s s 0)n ( 2 2n ) W i p[α]. (3.5) α I n i lim n 2 2n ɛ 2 n = 0. (3.6) For the second term of (3.5), we first consider the terms with α M n. By assumption A, (3.4) holds, i.e. for i, lim n 2(s s 0)n ( 2 2n ) W i p[α] = 0. (3.7) α M n Finally, we consider the terms with α I n \M n in (3.5). Note that p is bounded, thus lim n 2(s s 0)n ( 2 2n ) W i p[α] lim C 2 (s s0)n 2 2n 2 n M = 0. (3.8) n α I n\m n Combining (3.6), (3.7) and (3.8), we conclude lim n 2 n u n v l2 (I n) = 0. This completes the proof of the Proposition. Proposition 3. shows that if the noise level is relatively small, λ i can be chosen smaller than 2 (s s 0)n. If the noise level is high, for example lim n 2 n ɛ n > 0, meaning that the average noise of each pixel does not converge to 0 as n tends to. In this case, 22

23 in order to fit the image function well and suppress noise, we need a larger weight for the regularization term. In the following, we show that for different parameters in E n, g n has a different asymptotic approximation behavior. Following the line of [7], let f be an analog (noisy) image function, v := T φ,n f be the observed digital image and u n be a minimizer of E n with λ 0,0 = 0 and λ i = 2c i 2 n for i, where c i (see [7, P054]) are positive constants which are only dependent on ψ i. Let gn := u n(α) φ(2 n α), α I n where φ is the dual of φ. By [7, Propsition 3.], g n is a minimizer of F n (g) := T φ,n g T φ,n f 2 l 2 + diag(λ)wt φ,n g l, with g W (Ω). Furthermore, by [7, Theorem 3.2], 2 2n F n Γ-converges to the total variational model F (g) := g f 2 L 2 (Ω) + g W (Ω). Thus, g n approximates the minimizer of F (g) in case n is sufficiently large [7, Corollary 3.]. The essential difference of the asymptotic state between g n and g n is due to the different choice of parameters. Compared to g n, in the model corresponding to g n, we introduced a larger weight for the regularization term. This makes sense when the noise level increases proportional to the number of samples. But when the noise level is bounded, or even if the samples are from a clean analog image function, g n may not converge to f. As a result, the wavelet frame based approach has a relaxed asymptotic state behavior, and it can approximate various models by choosing a proper wavelet frame transform and parameters [7]. 3.3 Image inpainting (Continued) Let f L 2 (Ω) be an analog image function and p := T φ,n f be the pixel values. Suppose the noisy observations are given by { p[α] + ɛ[α], α Λ n, v[α] = (3.9) unknown, α I n \Λ n, with Λ n I n and ɛ[α] represents the noise. Besides the smoothness assumption (A), we need assumptions on the density of known pixels and on the noise level, under which we can prove that f can be recovered by observations v[α]. Let ɛ n := ɛ l2 (Λ n). We assume A3: The density of known pixels δ := δ(λ n, Ω) (2 ( s 3 n ) 2 )2 n for some 0 < s < s. 23

24 A4: lim n 2 (s 2)n ɛ 2 n = 0. By assumption A3, for every sub-square matrix with length 2δ, that is 2 2s 3 n total pixel positions, there exists at least one known pixel value. As f gets smoother, s becomes larger and thus δ can be larger. Note that assumption A4 is different from A2. Here, noise ɛ is defined on Λ n and we expect a lower bound. Inpainting can also be done by a wavelet frame based approach by minimizing Q n (u) := u v 2 l 2 (Λ n) + diag(λ)wu l (I n). Proposition 3.2. Let the assumptions (A), (A3) and (A4) be satisfied. Moreover, let u n be a minimizer of Q n where λ is chosen such that λ 0,0 = 0 and 0 < λ i 2 (s s 0)n for some s < s 0 < s, of the same order and satisfying lim n λ i 2 (s 2)n ɛ 2 n = 0 for all i. Let gn := u n(α)φ(2 n α), α I n then lim n g n f L2 (Ω) = 0. Proof. Similar to Proposition 3., we only need to prove that lim n 2 n u n p l2 (I n) = 0. The proof is organized as follows, first we prove then lim n 2 n u n p l2 (Λ n) = 0, lim n 2 n u n p l2 (I n\λ n) = 0. Since u n is a minimizer of Q n, it holds that 2 (s 2)n Q n (u n) 2 (s 2)n Q n (p) 2 (s 2)n (ɛ 2 n + diag(λ)wp l (I n)) 2 (s 2)n ( ɛ 2 n + 2 (s s 0)n ( W i p l (M n) + W i p ) l (I n\m n)) i 2 (s 2)n ɛ 2 n + 2 (s s 0 )n 2 sn 2 n f, ψ i,n,α + 2 (s+s s0 2)n C 2 n M i α M n 2 (s 2)n ɛ 2 n + 2 (s s 0 )n ( 2 2sn f, ψ i,n,α 2 ) (s+s s 0 )n C M. α M n i 24

25 By assumption A4, lim n 2 (s 2)n ɛ 2 n = 0. By assumption A, we have Thus, we obtain In particular, lim n i 2 (s s 0 )n ( 2 2sn f, ψ i,n,α 2 ) 2 = 0. α M n lim n 2(s 2)n u n p 2 l 2 (Λ lim n) n 2(s 2)n Q n (u n) = 0. (3.0) lim n 2 n u n p l2 (Λ n) = 0. Similarly, we have diag(λ)wu n l (I n) Q n (u n) Q n (p) = ɛ 2 n + diag(λ)wp l (I n). Since all λ i are of the same order, we obtain Wu n l (I n) C(λ i ɛ 2 n + Wp l (I n)). (3.) Next, we prove lim n 2 n u n p l2 (I n\λ n) = 0. Let the image domain Ω be partitioned into some sub-squares with equal length in [2δ, 4δ). Then, by assumption A3, for any α I n \Λ n, we can find α Λ n such that α and α are in the same sub-square and u n(α) p(α) 2 = u n(α) u n( α) p(α) + p( α) + u n( α) p( α) 2 3( u n(α) u n( α) 2 + p(α) p( α) 2 + u n( α) p( α) 2 ). Suppose α = (α, α 2 ) and α = α + (k, l) = (α + k, α 2 + l) with 0 k, l < 4δ 2 n. Then u n(α) u n( α) 2 8δ 2 n ( u n (α +, α 2 ) u n(α, α 2 ) 2 + u n(α + 2, α 2 ) u n(α +, α 2 ) 2 + u n(α + k, α 2 ) u n(α + (k )), α 2 ) 2 + u n(α + k, α 2 + ) u n((α + k), α 2 ) u n(α + k, α 2 + l) u n((α + k), α 2 + (l )) 2) k l 2 s n 3 4( e u n(α + m, α 2 ) 2 + e2 u n(α + k, α 2 + q) 2 ) m= 2 s 3 n+ 2 s 3 n+ 2 s n 3 4( e u n(α + m, α 2 ) 2 + e2 u n(α + k, α 2 + q) 2 ). m= If 4δ 2 n < k (or l) < 0, similar results can be obtained, and by the same method 2 s 3 n+ 2 s 3 n+ p(α) p( α) 2 2 s n 3 4( e p(α + m, α 2 ) 2 + e2 p(α + k, α 2 + q) 2 ). m= q= q= q= 25

26 For each sub-square matrix with length < 4δ, there exist at most 2 2s n+2 3 total pixel positions. It follows that u n(α) p(α) 2 α I n\λ n ( 2 s n 3 2 2s n 3 48 ( e u n(α) 2 + e2 u n(α) 2 ) + ( e p(α) 2 + e2 p(α) 2 ) α I n α I n + ) u n( α) p( α) 2 α Λ n ( 2 sn 48M ( e u n(α) + e2 u n(α) ) + ( e p(α) + e2 p(α) ) α I n α I n ) + u n p 2 l 2 (Λ n). This together with Lemma 2.7 implies u n(α) p(α) 2 2 sn ( 48M ( W i u n l (I n) + W i p l (I n)) + u n p 2 l 2 (Λ n)). α I n\λ n i Therefore, by (3.) we conclude lim n 2 2n u n p 2 l 2 (I n\λ n) lim n 2s n 2 2n 48M ( ( W i u n l (I n) + W i p l (I n)) + u n p 2 ) l 2 (Λ n) i lim n 2(s 2)n 48M(C + ) ( λ i ɛ 2 n + i lim 48M(C + )( 2 (s 2)n λ n i ɛ 2 n + i W i p l (I n) + u n p 2 ) l 2 (Λ n) 2 (s 2)n W i p l (I n) + 2 (s 2)n u n p 2 ) l 2 (Λ n) lim 48M(C + )( 2 (s 2)n λ n i ɛ 2 n + 2 (s 2)n W i p l (M n) + 2 (s 2)n W i p l (I n\m n) = 0, + 2 (s 2)n u n p 2 ) l 2 (Λ n) i where the last equality follows from (3.4) and (3.0). Remark: The parameters λ i in the model Q n (u) are chosen of the same order to make all of the wavelet channels W i u n l (M n) decay for i (0, 0). They should be neither too large in order to fit the observations, nor be too small in order to permeate the missing region. If f is continuous and the observed values v in (3.9) are defined by v[α] = f(2 n α) + ɛ[α], α Λ n, an asymptotic analysis of the inpainting model can be done similarly. 26

27 4 Algorithm and Experiments In this section we explain how to numerically solve the minimization problem min u n i=( α I u(α)φ( ξ i h α) y(ξ i)) 2 + diag(λ)wu l (4.) and further present some numerical experiments. The problem (4.) can be written in matrix vector form as min Au u R y 2 m l 2 + diag(λ)wu l, (4.2) with y = [y(ξ ), y(ξ 2 ),..., y(ξ n )] T, A ij = φ(ξ i /h k j ), Ξ = {ξ,..., ξ n } and I = {k,..., k m }. So, (4.2) is an ordinary least squares problem with an l -regularization term. It means that this problem cannot be solved straight forward by solving only one system of equations. But there are iterative solvers like the split Bregman algorithm which splits the l 2 -problem from the l -problem and then uses the Bregman iteration, see, e.g., [20, 8]. The iteration step i i + of the split-bregman algorithm in terms of problem (4.2) reads u i+ = arg min Au y 2 u l 2 + µ 2 Wu di + b i 2 l 2 (4.3) d i+ = T λ/µ (Wu i+ + b i ) (4.4) b i+ = b i + Wu i+ d i+ (4.5) with initial u 0 = 0, d 0 = 0 and b 0 = 0. For µ > 0, T λ/µ is the soft-threshold operator T λ/µ (x) := [t λ/µ (x ), t λ/µ (x 2 ),..., t λ/µ (x M )], with t λ/µ (x i ) := sgn(x i ) max{0, x i λ µ }. The stoping criteria of the iteration is d i Wu i l2 ɛ for some positive constant ɛ. The solution to (4.3) can be determined by solving the system of equations (2A T A + µw T W)u = 2A T y + µw T (d i b i ) which, because of W T W = I, can be simplified to (2A T A + µi)u = 2A T y + µw T (d i b i ). (4.6) Since (2A T A + µi) is symmetric positive definite, the system of equations (4.6) can be efficiently solved by applying a conjugate gradient method. Further note, that W T (d i b i ) in (4.6) is determined by performing the inverse framelet transform rather than by using its matrix representation, similar in the iterations (4.4) and (4.5). 27

28 Wavelet frame based image restoration has found various applications, for instance image denoising, inpainting and scene reconstruction, see [6, 8, 24, 36, 7, 3] and the references therein. Below, we also present two numerical examples. First we reconstruct a piecewise continuous function from some scattered data samples; second we fit the interaction force function between water and water molecules in terms of their distance, which is a simple example in molecular dynamics simulations [32]. 4. Piecewise continuous function In the first experiment we show the advantage of the approach (.2) by reconstructing a piecewise continuous function on Ω = [0, ] 2 from some noisy samples. We construct a testfunction by using the well-known Franke function, which is the weighted sum of four exponential functions franke(x, y) = 3 4 e ((9x 2)2 +(9y 2) 2 )/ e ((9x+)2 )/49 (9y+)/0 + 2 e ((9x 7)2 +(9y 3) 2 )/4 5 e (9x 4)2 (9y 7) 2. The function was introduced in [8] and is widely used in scattered data approximation benchmarks. To obtain our piecewise continuous test function f we substract 0.2 from the Franke function on the domain [0.25, 0.75] 2, i.e., { franke(x, y) 0.2 if (x, y) [0.25, 0.75] 2 f(x, y) =. (4.7) franke(x, y) else We randomly sample sites {ξ i } 4000 i= from Ω and choose the scattered data samples as y(ξ i ) = f(ξ i ) + ɛ i with ɛ i drawn from a normal distribution with 0 mean and standard deviation As scaling parameter we choose h = /80. We choose the cubic B-spline x 3 /6 if 0 x < ( 3x 3 + 2x 2 2x + 4)/6 if x < 2 B 4 (x) = (3x 3 24x x 44)/6 if 2 x < 3 (4 x) 3 /6 if 3 x < 4 0 else as generator for the shift invariant subspace and its associated tight wavelet frame system with the corresponding masks h = [/6, /4, 3/8, /4, /6], h 2 = [ /8, /4, 0, /4, /8], h 3 = [ 6/6, 0, 6/8, 0, 6/6] and h 4 = [ /8, /4, 0, /4, /8]. In figure a the approximation which is obtained by minimizing (2.9) is depicted, whereas in figure b the approximation with a Laplacian regularization is depicted, i.e., Au y 2 l 2 + νu Lu with L the discrete Laplacian is minimized to determine the approximant. It can be well seen that the Laplacian regularization which punishes the 28

29 roughness is not able to preserve the discontinuities, whereas the approach (2.9) is able to do so. (a) Approximation by minimizing (2.9) (b) Approximation using Laplacian regularization Figure : Approximation of 4000 samples from testfunction (4.7) 4.2 Application to CG models in Molecular dynamics Molecular dynamics (MD) numerically simulates the interactions of atoms. This has contributed key insight into structural biology, one of the most vibrant research fields in science, which advanced also because of the technological progress in spectroscopy and microscopy. Simulating the interactions of atoms inside large system is computationally very challenging. Coarse grained (CG) models [34] were introduced to provide a computational efficient concept. A CG-model is a simplified version of all atom representation, it simplifies according to the molecule structure and combines several atoms in one single interaction site. These simplified models are used for the rapid investigation of long time- and length-scale processes in many important biological and soft matter processes. To parameterize the interactions between different CG sites is an important research area in MD. These interactions (functions) are usually represented in term of their distance, bond angle or dihedral angle, etc. Assume a system with water molecules (H 2 O) at constant temperature and volume. The simplified CG-model handles every water molecule as one CG site W. Hence, the non-bonded interactions of types O-O, O-H, H-H and bonded interactions of O-H, H-O-H in all atom water system are not needed to be investigated. The only interaction type in the CG model which have to be considered are the non-bonded interaction W-W. (See figure 2a) In our numerical example a cubic box containing 999 water molecules was simulated to form atomistic trajectory using the GROMACS MD simulation package [4]. The simulation time step is set to 2 fs. The force matching method [34, 3] was applied to the atomistic configurations to generate the CG potentials, and 0 ns of the trajectory was used for analysis. 29

30 In the CG-model three atoms in one water molecule are combined to one single CG-site W. For every frame of the trajectory, the position of each CG site W (r i = (x i, y i, z i )) 999 i= is obtained by the centers of geometry of three atoms in the corresponding water molecule. In this way, each site W i at position r i = (x i, y i, z i ) is calculated by the trajectory of the water atoms. The net force F i = (Fi x, F y i, F i z ) which is acting on each W i is calculated based on Newton s equations of motion, which should equal to the sum of the forces W j (j i) acting upon W i. Given the net force F i of each W i, our aim is to derive the interaction force function between W and W in terms of their distance. Let e x i,j := x j x i r j r i, ey i,j := y j y i r j r i and e z i,j := z j z i r j r i. The intermolecular radial distribution functions (RDF) ( r j r i ) j i are considered as our scattered data sites. We approximate the force function f between W and W in terms of Euclidean distance from S h (φ, Ω) by solving min u i ζ {x,y,z} ( N j=,j i g( r j r i )e ζ i,j F ζ ) 2 i + diag(λ)wu l, in which g = α u(α)φ( h α), φ = B 4, W is the framelet transform with cubic framelets, N = 999 and h = nm. If u is a minimizer, then f can be approximated by α u (α)φ( h α). The distribution of the RDF represents the probability of finding the relevant molecules at a distance of (r, r + dr) as a function of the distance r. Most of the CG sites are located in a low energy state, whereas only a few CG sites are located in the high energy state, by the Boltzmann distribution in statistical mechanics [9]. Thus, the data sites ( r j r i ) j i in our model are very scattered. Moreover, due to inadequate sampling of phase space and random fluctuations in the measurements, the data r i = (x i, y i, z i ) and F i = (Fi x, F y i, F i z ) are usually very noisy. In order to fit the force well while suppress noise, the parameters λ are chosen inversely proportional to the RDF of experimental molecules, which is the cardinality of #{ r j r i : r j r i supp φ( h α)}. The idea is that for the poor sampling region, we put less trust, and large λ are chosen. On the contrary, for the relatively adequate sampling region, we put more trust, and small λ are chosen. This trust region regularization method can suppress noise over signal, provide appropriately smoothed results, and fit the important features of the force curve. In figure 2b the curve of fitting with efficient data can be seen as a benchmark, which was based on 0000 frames of the trajectory data. While for 30 frames of the trajectory data, it can be seen that compared with Laplacian regularization our approach preserves the minima better, which is important for MD simulations. 30

31 (a) Water molecules in a box (b) Approximation of interaction force of Water-Water by 999 water molecules Figure 2: Approximation of interaction force of Water-Water molecules Acknowledgement: The work of Jianbin Yang was partially supported by the research grant #020 from NSFC and the Fundamental Research Funds for the Central Universities, China; the work of Dominik Stahl was partially supported by the German Academic Exchange Service (DAAD); and the work of Zuowei Shen was partially supported by Singapore MOE AcRF Research Grant MOE20-T2--6 and R References [] R. Arcangli, M. C. L. de. Silanes, J. J. Torrens. An extension of a bound for functions in Sobolev spaces, with applications to (m, s)-spline interpolation and smoothing. Numer. Math., 07(2): 8-2, [2] M. Arigovindan, M. Su hling, P. Hunziker and M. Unser. Variational image reconstruction from arbitrarily spaced samples: A fast multiresolution spline solution. IEEE Trans. Image Process., 4(4): , [3] W. Beckner. Inequalities in Fourier analysis. Ann. of Math., 02(): 59-82, 975. [4] H. J. C. Berendsen, D. van der Spoel and R. van Drunen. GROMACS: A messagepassing parallel molecular dynamics implementation. Comput. Phys. Commun., 9(): 43-56, 995. [5] L. Borup, R. Gribonval, M. Nielsen. Bi-framelet systems with few vanishing moments characterize Besov spaces. Appl. Comput. Harmon. Anal., 7(): 3-28, [6] J. F. Cai, R. H. Chan, and Z. Shen. A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal., 24 (2): 3-49,