Numerische Mathematik

Transcription

1 First pubished in: Numer. Math. 75: (1997) Numerische Mathematik c Springer-Verag 1997 Eectronic Edition A waveet mutieve method for i-posed probems stabiized by Tikhonov reguarization Andreas Rieder Fachbereich Mathematik, Geb. 38, Universität des Saarandes, D Saarbrücken, Germany; e- mai: andreas@num.uni-sb.de Received March 6, 1995 / Revised version received December 27, 1995 Summary. An additive Schwarz iteration is described for the fast resoution of inear i-posed probems which are stabiized by Tikhonov reguarization. The agorithm and its anaysis are presented in a genera framework which appies to integra equations of the first kind discretized either by spine functions or Daubechies waveets. Numerica experiments are reported on to iustrate the theoretica resuts and to compare both discretization schemes. Mathematics Subject Cassification (1991): 65R20, 65R30 1. Introduction Iterative schemes for soving the compact operator equation (1.1) of the first kind, (1.1) Kf = g, are mosty used in the sense of reguarization methods due to the i-posedness of (1.1), that is, they are used to baance the data error and the approximation error, see e. g. Louis [18]. From this point of view the convergence speed and the performance of these iterative methods are of minor importance. In this paper we discretize equation (1.1) by appying the method of east squares and stabiize it by Tikhonov reguarization, see Pato and Vainikko [21]. Since the resuting finite dimensiona inear system is aready reguarized we are primariy interested in achieving an efficient iterative sover in terms of convergence speed, parae coding, and performance. Therefore, an additive Schwarz reaxation wi be the method of our choice. For the construction of the mutieve sover we wi spit the test function space into orthogona subspaces of increasing dimension. The number of subspaces invoved is caed the spitting eve and the subspace with smaest dimension is referred to as coarsest space. Numerische Mathematik Eectronic Edition page 501 of Numer. Math. 75: (1997) EVA-STAR (Eektronisches Votextarchiv Scientific Artices Repository)

2 502 A. Rieder This approach not ony offers a the advantages of mutieve spittings but aso yieds an asymptotic orthogonaity of the spitting spaces with respect to an inner product reated to probem (1.1). The atter fact wi be essentia for the presented convergence anaysis where we wi rey on we-known convergence resuts for Schwarz type methods, see e. g. Hackbusch [15], Oswad [20] and Yserentant [27]. A first study of mutieve agorithms in connection with i-posed probems was done by King in [17] where he proposed a method which is akin to the nested iteration known in mutigrid theory, cf. Hackbusch [15]. King appied his mutieve iteration as a reguarization technique in the sense mentioned above. This is one of the main differences to our approach. A more detaied comparison of King s agorithm with our agorithm is postponed to Sect. 5. The outine of this paper is as foows. In the next section we give a brief account on the adequate discretization and reguarization of equation (1.1) by the method of east squares. Aso in the next section we introduce the mutieve spitting of the approximation space and prove some of its properties. Section 3 is devoted to the additive Schwarz iteration. After a motivation we define and anayze the iteration in an abstract framework. We find two quaitative different convergence resuts: 1. For a fixed spitting eve, the convergence is getting faster as the discretization step-size decreases, that is, the dimension of the approximation space increases. 2. In case the coarsest space is fixed, the convergence rate is independent of the discretization step-size and of the spitting eve. We compete Sect. 3 with a representation of the agorithm with respect to waveet or pre-waveet spittings of the approximation space. In the remainder of the paper we appy the proposed iterative scheme to integra equations on L 2 (0, 1). Here, we present two famiies of test function spaces which satisfy the hypotheses of our abstract theory. These spaces are spine spaces and the spaces of the Daubechies scaing functions on the interva, see Cohen, Daubechies and Via [6]. The numerica reaization of the method in this setting is considered next. We show that approximate integration, which wi be necessary in a genera appication of the agorithm, does not deteriorate the convergence behaviour. Finay, an anaysis of the computationa compexity confirms the efficiency of the iteration and the presentation of various numerica experiments support the theoretica resuts. Waveets have aready been used for the treatment of inverse probems. For instance, we refer to Donoho [13], Dicken and Maaß [12] and to Xia and Nashed [26]. 2. Preiminary considerations 2.1. Discretization, reguarization and parameter seection Let K : X Y (X, Y rea Hibert spaces) be a compact non-degenerate operator. Then, it is we known that equation (1.1) is i posed, that is, the minimum norm soution f of (1.1) does not depend continuousy on the right hand side g. Numerische Mathematik Eectronic Edition page 502 of Numer. Math. 75: (1997)

3 A waveet mutieve method for i-posed probems 503 We now assume that ony noisy data g ε Y are avaiabe satisfying g g ε Y ε for a known error bound ε>0. A computabe approximation to f is then provided by the unique soution f ε,α of the finite dimensiona norma equation (2.1) (K K + αi ) f = K g ε, α > 0, which is stabiized using Tikhonov reguarization (throughout the paper I denotes either the identity operator or the identity matrix of appropriate size). In (2.1), K = KP where P : X V is the orthogona projection onto the subspace V X. In the seque we wi assume that the sequence {V } of finite dimensiona approximation spaces is expanding, i. e. V V +1, and that the union V is dense in X. Under these assumptions the quantity (2.2) γ := K K = K (I P ), which wi be crucia for the further anaysis, satisfies (2.3) γ +1 γ and γ 0as iff K is compact, see e. g. Groetsch [14]. The operator norm in (2.2) is defined by K = sup{ Ku Y u X, u X =1}. An a-priori (α = α(,ε)) as we as an a-posteriori (α = α(,ε,g ε )) choice for the reguarization parameter α in (2.1) is estabished by Pato and Vainikko [21] to the minimum norm soution f as ε tends to zero and goes to infinity. Moreover, the resuting convergence rate is optima in ε. It is the goa of the paper to provide an efficient mutieve sover for equation (2.1) with one of the above mentioned parameter seection strategies under the genera assumption of a fixed noise eve ε. In this framework α is bounded beow by a positive constant α 0 (ε) uniformy in the discretization eve which gives that eading to the convergence of f ε,α (2.4) γ α 0 (ε) < α for sufficienty arge. The above inequaity guarantees a high performance of our mutieve sover as wi be discussed in the Sects. 4 and 5. Remark. From an abstract point of view the norma equation (2.1) is a symmetric operator equation of the second kind. Therefore, our agorithm appies to such a cass of probems in genera. Nevertheess we present our agorithm in the above context of i-posed probems since Tikhonov reguarization automaticay eads to inear probems (2.1) with symmetric and positive definite matrices Mutieve spitting of the approximation spaces The basis of a mutieve agorithms is the decomposition of the approximation space into subspaces. To this end we define the space W as the X orthogona compement of V with respect to the arger space V +1 : V +1 = V W where Numerische Mathematik Eectronic Edition page 503 of Numer. Math. 75: (1997)

4 504 A. Rieder denotes the X orthogona sum. Consequenty, we have the orthogona mutieve spitting (2.5) V = V min 1 j = min W j, min 1, which aso can be expressed in terms of projection operators (2.6) P = P min + 1 j = min Q j where Q j is the orthogona projection from X onto W j. Compact operators vanish asymptoticay on the compement spaces W. Lemma 2.1. Let V and W be the spaces defined above and et K : X Y be a compact inear operator. Then, where γ is defined in (2.2). K Q γ 0 as Proof. The orthogonaity of V and W gives P Q = 0. Therefore, K Q = K(I P )Q K(I P ) =γ. The reguarized norma equation (2.1) can be reformuated as a variationa probem (2.7) find f ε,α V : a( f ε,α,v )= Kg ε,v X for a v V where the biinear form a : X X R, (2.8) a(u,v):= Ku, K v Y + α u,v X, is symmetric and positive definite. The operators A = K K + αp and B = Q K KQ + αq are associated to a via a(u,v )= A u,v X for a u,v V and a(w, z )= B w,z X for a w, z W, respectivey. Later, we wi rey on the foowing strong Cauchy inequaity which basicay says that the spaces V and W are not ony X orthogona but aso asymptoticay orthogona with respect to the inner product on X induced by the biinear form a (2.8). The corresponding norm 2 a=a(, ) is caed energy norm on X. Theorem 2.2. Let V and W m be defined as above and et m. The strong Cauchy inequaity a(v,w m ) min{1,γ m / α} v a w m a hods true for a v V and for a w m W m. Numerische Mathematik Eectronic Edition page 504 of Numer. Math. 75: (1997)

5 A waveet mutieve method for i-posed probems 505 Proof. Since v and w m are orthogona in X we have that a(v,w m ) = Kv,Kw m Y. Further, a(v,w m ) = K A 1/2 A 1/2 v, KQ m Bm 1/2 Bm 1/2 w m Y K A 1/2 A 1/2 v X KQ m Bm 1/2 Bm 1/2 w m X K A 1/2 v a KQ m Bm 1/2 w m a. Using arguments from spectra theory it is easy to verify that K A 1/2 1 and Bm 1/2 α 1/2. Thus, the strong Cauchy inequaity is proved by KQ m γ m (Lemma 2.1). Coroary 2.3. Let j <. Then, a(w j,w ) min{1,γ / α} w j a w a for a w j W j and for a w W. Proof. Because W j V for j < the statement foows readiy from Theorem The additive Schwarz iteration 3.1. Abstract formuation and convergence anaysis The genera phiosophy behind any mutieve iteration with respect to a given spitting is to repace the origina arge scae probem by auxiiary probems on the subspaces which can be soved cheapy. If the subprobems are chosen in an adequate way, their combination shoud yied a reasonabe approximation to the origina probem, see e. g. Oswad [20] and Rüde [22]. We introduce some notation to carify this mutieve concept in our setting. Recaing the statement of Lemma 2.1, the auxiiary biinear forms b j : W j W j R defined by (3.1) b j (w j, u j ):=α w j,u j X, approximate a (2.8) reasonaby we on W j, at east for j arge. Furthermore, et T j : V W j be given by the variationa probem ( min j 1) (3.2) b j (T j v,w j )=a(v,w j ) for v V and for a w j W j. In operator notation we may write T j = α 1 Q j A. On the coarsest approximation space V min we keep the origina biinear form a. Here, we get R min : V V min by (3.3) a(r min v,v min )=a(v,v min ) for v V and for a v min V min or R min = A 1 min P min A in operator notation. Now, we sum up the T j s and R min, Numerische Mathematik Eectronic Edition page 505 of Numer. Math. 75: (1997)

6 506 A. Rieder R min + 1 j = min T j = ( A 1 min P min + α 1 1 j = min Q j ) A, } {{ } =: C,min and consider C,min as an approximate inverse of A with respect to the spitting (2.5). Starting with an initia guess u 0 V we define the additive Schwarz iteration for the approximate soution of (2.1) resp. (2.7) by (3.4) u µ+1 = u µ ( ω C,min A u µ K g ε), µ =0,1,2,..., with a damping parameter ω R. IfC,min is cose to A 1 we may expect fast convergence of {u µ } µ 0 to f ε,α. For a convergence anaysis we introduce a new norm on V reative to the spitting (2.5): 1 v 2 := P min v 2 a + α Q j v 2 X. j = min Theorem 3.1. Suppose that (3.5) Γ u v 2 v 2 a Γ o v 2 for a v V hods true for some numbers 0 <Γ u Γ o. Then, f ε,α u µ a ρ ω µ f ε,α u 0 a, µ =1,2,..., where ρ ω = max { 1 ωγ u, 1 ωγ o }. The convergence rate ρ ω attains its minimum ρ ω opt =(Γ o Γ u )/(Γ o +Γ u )<1for ω opt =2/(Γ u +Γ o ). Proof. See e. g. Oswad [20], Theorem 18 on page 79. See aso Hackbusch [15] and Yserentant [27]. Providing γ (2.2) tends to zero sufficienty fast we can estimate the bounds Γ u and Γ o reasonaby we. Theorem 3.2. Let η be an upper bound of γ (γ η ) satisfying (3.6) η η 1 and 1 j = min η j C η η min with a positive constant C η which does neither depend on nor on min. Then, the norm equivaence (3.5) hods true for 1 Γ u = ( ), 1+2C η 1+(Cη +1)σ min σmin Γ o = ( 1+σ 2 )( ) 1+2Cη min σ min, where σ min = η min / α. The convergence rate ρ 1 of the undamped (ω = 1) iteration (3.4) fufis ( ρ 1 2 C η σ min 1+σmin (1 + max{c η,σ min }) ). Numerische Mathematik Eectronic Edition page 506 of Numer. Math. 75: (1997)

7 A waveet mutieve method for i-posed probems 507 The above theorem, which wi be proved in the appendix, has to be interpreted in the foowing ways: 1. For a fixed spitting eve L = min 1 and a fixed noise eve ε>0 the convergence rate ρ 1 tends to zero. Moreover, (3.7) ρ 1 () =O ( η L / α ) as, where the constant in the above O expression does neither depend on and L nor on α. 2. For a given noise eve ε > 0 the undamped iteration admits a convergence rate ρ 1 which is bounded smaer than 1 uniformy in provided that η min < α/ ( 2C η (C η +2) ). In this case min depends on ε and it increases when ε decreases. 3. If min is fixed independenty of ε and then the undamped iteration may even diverge for ε too sma. Sti, we can use the undamped iteration as a preconditioner for the conjugate gradient method appied to (2.1). For more detais on preconditioning the conjugate gradient method we refer e. g. to Deufhard and Hohmann [11] and Hackbusch [15]. As an initia guess for the iteration (3.4) we suggest to take u 0 = f ε,α min, the soution of (2.7) with respect to V min. The effort to cacuate f ε,α min is ess than the computationa work for one iteration step. Lemma 3.3. Let u 0 = f ε,α min. Then, f ε,α u 0 a K 2 + α inf f ε,α v V v X. min Proof. The proof is anaogous to the proof of Céa s emma, see e. g. Ciaret [5] Representation in coordinates In this section we present a matrix version of the iteration (3.4) given suitabe Riesz-bases in V and W. Therefore, we assume that X is a function space over the compact interva [a, b] R; X = L 2 (a, b) for exampe. The resuts obtained can easiy be generaized to mutipe tensor products of X with itsef. Let ϕ X be a compacty supported function satisfying the refinement equation (3.8) ϕ(x) = 2 M ϕ 1 k=0 h k ϕ(2x k) with coefficients h k R (ϕ is caed scaing function in the waveet terminoogy, see e. g. Daubechies [9]). For the ease of presentation we negect at the present time necessary boundary modifications and suppose that (3.9) V = span{ϕ,k k =0,...,n 1} X Numerische Mathematik Eectronic Edition page 507 of Numer. Math. 75: (1997)

8 508 A. Rieder for a > 0 where ϕ,k (x) :=2 /2 ϕ(2x k). Further et there exist another function ψ, the waveet, such that W = span{ψ,k k =0,...,m 1} and (3.10) ψ(x) = 2 M ψ 1 k=0 g k ϕ(2x k) with coefficients g k R. Since the sum of two functions f = k c k ϕ,k V and g = k d k ψ,k W is in V +1, it can be expressed by f +g = k c+1 k ϕ +1,k. Appying both refinement equations (3.8) and (3.10) we get the reation c +1 k = i h k 2i c i + j g k 2j d j which we write in matrix notation as (3.11) c +1 = H t +1c + G t +1d. Ceary, H +1 : R n+1 R n and G +1 : R n+1 R m. The soution f ε,α of the variationa probem (2.7) resp. of the norma equation (2.1) can be expanded in the basis of V as f ε,α = k (ξ ) k ϕ,k. The vector ξ R n of the expansion coefficients is the unique soution of the inear system (3.12) A ξ = β where the entries of the positive definite matrix A and of the right hand side β are given by (3.13) (A ) i,j = K ϕ,i, K ϕ,j Y + α ϕ,i,ϕ,j X and (β ) j = g ε, K ϕ,j Y. Lemma 3.4. Let T j and R min be the operators introduced in (3.2) and (3.3), respectivey. Further define the restrictions H,j := H j +1 H j H 1 H :R n R nj, G,j := G j +1 H j H 1 H :R n R mj for j 2 and set H, 1 := H and G, 1 := G. For v = k c k ϕ,k V we have that T j v = n 1 ( ) α 1 G,j t B 1 j G,j A c ϕ,k, min j 1, k k=0 where B j is the Gramian matrix (B j ) r,s = ψ j,r,ψ j,s X, and R min v = n 1 ( ) H, t min A 1 H,min A c ϕ,k. min k k=0 Numerische Mathematik Eectronic Edition page 508 of Numer. Math. 75: (1997)

9 A waveet mutieve method for i-posed probems 509 n mj Proof. Since T j ϕ,k W j there exists a matrix T R with entries T k,i such that T j ϕ,k = i T k,i ψ j,i. A function w j = k d j k ψ j,k W j admits the expansion w j = ( k G t,j d ) j k ϕ,k which foows inductivey from (3.11). Hence, b j (T j v,w j )=α B j Tc,d j R m j and a(v,w j )= G,j A c,d j R m j. Finay, the equation (3.2) gives T = α 1 B 1 j G,j A and we get T j v = i (Tc ) i ψ j,i = k (G,j t Tc ) k ϕ,k. The representation of R min can be proved in the same way. Now, the abstract iteration (3.4) transated into an iteration acting on (3.12) reads (3.14) z µ+1 = z µ ω C,min ( A z µ β ), µ =0,1,2,..., where (3.15) C,min = H t, min A 1 min H,min + α 1 1 G,j t B 1 j G,j j = min and with an arbitrary starting guess z 0 Lemma 3.3 is R n. The starting guess due to (3.16) z 0 = H t, min ξ min = H t, min A 1 min H,min β. In the next section we wi see that the appication of B 1 j to a vector can be reaized in a very efficient way. and hence of C,min Remark. Readers famiiar with mutieve preconditioners in the context of finite eement discretizations of eiptic PDEs may wonder why we do not go one step further and repace A 1 as we as B 1 min j, min j 1, in (3.15) by the spectray equivaent identity matrix I. In doing so, C,min woud certainy gain a simper structure but we woud oose property (3.7). For a fixed spitting eve L the convergence speed of (3.14) woud not be improved when the discretization eve increases. As a consequence the mutieve scheme (3.14) woud not be attractive any more as an iteration in its own right. In this situation C,min coud serve as a preconditioner for the conjugate gradient method appied to (2.1) eading to a eve independent convergence rate. However, the conjugate gradient method acting on (2.1) without any preconditioning resuts aready in a eve independent convergence speed. This is true because α α 0 (ε) > 0 uniformy in, cf. (2.4). Hence, the iteration (3.14) for arge scae probems (2.1) is meaningfu ony if (3.7) hods, see aso Sect In other words: the X orthogonaity of the spitting (2.5) is a crucia ingredient for our mutieve agorithm and cannot be reaxed. Numerische Mathematik Eectronic Edition page 509 of Numer. Math. 75: (1997)

10 510 A. Rieder 4. Appication to integra equations We demonstrate the performance of our proposed iterative scheme for the soution of integra equations. For simpicity we imit ourseves to probems on the unit interva, but the agorithm can be carried over to probems on Cartesian products of intervas. Let K : L 2 (0, 1) L 2 (0, 1) be an integra operator with non-degenerate kerne k which is square integrabe over the unit square [0, 1] 2. Then, the integra equation of the first kind, (4.1) Kf ( ) = 1 0 k(,y)f(y)dy = g( ), is i posed. To estimate the decay rate of γ as goes to infinity, we consider scaing function spaces V (3.9) of order N, that is, the poynomias up to degree N 1 restricted to [0, 1] are in V. We associate the discretization step-size δ =2 to V and we wi use D r to denote the (generaized) differentia operator of order r. Lemma 4.1. Let V be the spaces (3.9) of order N. Further, et K be the integra operator (4.1) and suppose that the composition D r K is a bounded operator on L 2 (0, 1) for one r {1,...,N}. Then, (4.2) γ = (I P )K C N D r K δ r. Proof. If f is sufficienty smooth we have the foowing approximation resut: (I P ) f L 2 C N D r f L 2 δ r for r =1,...,N, see e. g. Strang [23], Strang and Fix [24]. The estimate for γ is a straightforward consequence. In the framework of scaing function spaces of order N 1, Theorem 3.2 appies because the requirements (3.6) are met for η = C N D r K δ r with r 1 and the decay rate (3.7) reads now (4.3) ρ 1 () =O ( δ L/ r α ) as. The constant in (4.3) is independent of, L and α Waveet and spine spaces Let V, 0, be the space that contains a functions being constant on the intervas [2 k, 2 (k + 1)[ (k =0,...,2 1). This is a simpe but admissibe choice: V is of the form (3.9) when we set ϕ = χ [0,1[, the characteristic function of the interva [0, 1[. The compement spaces W have aso the dimension 2 and they are spanned by the Haar-waveet ψ = χ [0,1/2[ χ [1/2,1[, see Daubechies [8]. The refinement equations (3.8) and (3.10) hod true with h 0 = h 1 =1/ 2 and g 0 = g 1 =1/ 2, respectivey (M ϕ = M ψ = 2). Ceary, the order of V is N =1. Numerische Mathematik Eectronic Edition page 510 of Numer. Math. 75: (1997)

11 A waveet mutieve method for i-posed probems q 0.2 ρ Fig. 1. Soid curve: convergence rates ρ 1 of the undamped iteration (3.14), min = 4, with respect to the Haar-waveet and with α = The underying kerne is k 1 (x, y) =x y,ifx y, and k 1 (x, y) = 0, otherwise. Dashed curve: the rate q (4.4). The theoretica bound 0.5 for q is drawn as a dashed straight ine. On eve = 6 the iteration diverges The asymptotic behaviour (4.3) is iustrated in Fig. 1 where approximations to ρ 1 (), L = 4, are potted for severa and α = The underying kerne is k 1 (x, y) =x y,ifx y, and k 1 (x, y) = 0, otherwise. Besides the convergence speed we potted the rate (4.4) q = ρ 1 () /ρ 1 ( 1). Since (4.3) is vaid with r = 1 and L = 4 we expect q to be bounded: q 0.5 for arge. To improve the poor decay rate of ρ 1 obtained in the Haar-waveet case we introduce scaing function spaces containing poynomias of higher degree. We present two different famiies of functions which both can be considered as further deveopments of the introductory exampe above Daubechies waveets on the interva We wi briefy reca the waveet systems on the interva [0, 1] constructed by Cohen et a. in [6]. This construction is a modification of the Daubechies waveet famiy on the rea ine, see Daubechies [8]. Let ϕ = ϕ N and ψ = ψ N be the Daubechies scaing and waveet function of order N 2 which both have compact support in [1 N, N ]. Define to be the smaest integer such that 2 2N. For there exist 2N edge scaing functions ϕ 0 k, ϕ1 k, and 2N edge waveets ψ0 k, ψ1 k, such that each of the sets X = { ϕ 0,k, 0 k N 1 } { ϕ,k N k 2 N 1 } { ϕ 1,k 2 N k 2 1 }, Y = { ψ,k 0 0 k N 1 } { ψ,k N k 2 N 1 } { ψ,k 1 2 N k 2 1 }, is a famiy of orthonorma functions ( for convenience we use the notation f,k 0 (x) =2/2 fk 0(2 x) and f,k 1 (x) =2/2 fk 1(2 (1 x)) ony for edge functions). The spaces V d and W d of dimensions n = m =2, Numerische Mathematik Eectronic Edition page 511 of Numer. Math. 75: (1997)

12 512 A. Rieder (4.5) V d := span X and W d := span Y, have a properties required in the previous sections, for instance, the order of V d is N. The edge functions satisfy as the interior functions a kind of refinement equation. The corresponding coefficients are tabuated in [6]. Remark. Since the waveet-basis in W d is an orthonorma basis the corresponding Gramian matrix B coincides with the identity matrix which simpifies the structure of C,min (3.15) Spine spaces on the interva The N th order B spine S N is defined as the N fod convoution of χ [0,1]. The B spines have compact support in [0, N ] and they satisfy the refinement reation S N (x) = N ( N 2 1 N k k=0 ) S N (2x k). The corresponding B spine-waveet of order N is we known, see e. g. Chui [2] for a comprehensive introduction to spine-waveets. As in the case of the Daubechies waveets we wi need edge spines and edge spine-waveets to yied approximation spaces of order N on [0, 1]. For N = 2 we wi give expicit expressions. The genera case is considered by Chui and Quak in [3]. The graph of the inear B Spine S 2 is shown in Fig. 2. We have that (4.6) S 2 (x) :=S(x)= 1 ( ) S(2x) +2S(2x 1) + S (2x 2). 2 To span the inear functions on [0, 1], we need two edge spines S 0 and S 1 given by S 0 (x) =1 x,if0 x 1, S 0 (x) = 0, otherwise, and S 1 (x) =S 0 (2 x). The modified scaing equation is The space V s of dimension n =2 +1, S 0 (x) =S 0 (2x) S(2x). (4.7) V s := span{ S 0, S 1, S,k k =0,...,2 2}, where S 0(x) :=S0 (2 x) and S 1 (x) :=S0 (1 x) ( 1), coincides with the space of continuous functions which are affine inear on [2 k, 2 (k + 1)[. The interior waveet ψ s given by, see Chui and Wang [4], ψ s (x) = 1 10 S (2x) 3 S (2x 1) + S (2x 2) (4.8) S (2x 3) + S (2x 4) 5 10 Numerische Mathematik Eectronic Edition page 512 of Numer. Math. 75: (1997)

13 A waveet mutieve method for i-posed probems Fig. 2. The inear B Spine S 2 (4.6), its corresponding (pre-)waveet ψ s (4.8) and the edge spinewaveet ψ 0 (4.9) for the eft boundary ( from eft to right) has compact support in [0, 3], cf. Fig. 2. The edge waveets ψ 0 and ψ 1 can be cacuated directy and they fufi (4.9) ψ 0 (x) =S 0 (2x) S (2x) +1 2 ψ 1 (x) =ψ 0 (2 x), 1 S(2x 1) S (2x 2), 12 cf. Fig. 2. Again, we set ψ 0(x) :=ψ0 (2 x), ψ 1(x) :=ψ0 (1 x) ( 1), and define W s := span{ ψ 0,ψ 1,ψ s,k k=0,...,2 3} for = 2. The dimension of W s is m =2 and the reader may convince himsef that V+1 s = V s W s,. Since the constructed bases in V s and W s are ony Riesz-bases the functions ψ 0, ψ 1, ψ s, are usuay caed semi-orthogona waveets or pre-waveets in the waveet iterature. Remark. The Gramian matrix B with respect to the basis in W s is a band matrix with band width 3 and B differs from a Toepitz matrix ony at both corners of its diagona. Utiizing these specia features together with the Choesky decomposition, see e. g. Deufhard and Hohmann [11], it is possibe to compute the action of B 1 on a vector with O(m ) operations. The additiona storage is independent of the dimension Numerica reaization and computationa compexity Matrix generation The sensitive point in using the proposed method ies in the computation of the entries of the matrix A (3.13). Since the integras f,ϕ,j L 2 can be cacuated ony approximatey we choose a quadrature rue of the form (4.10) Q,j ( f )=δ 1/2 m w t f (x,t,j ) with weights w t R and abscissae x,t,j [0, 1], see Swedens and Piessens [25]. If Q,j is exact for poynomias up to degree m and if f C m+1, then t=0 Numerische Mathematik Eectronic Edition page 513 of Numer. Math. 75: (1997)

14 514 A. Rieder (4.11) f,ϕ,j L 2 Q,j (f) C Q max f (m+1) (x) δ m+3/2 x [0,1] where the positive constant C Q does not depend on j. The error estimate (4.11) foows from Peano s theorem, see e. g. Davis and Rabinowitz [10]. Approximating the action of K on ϕ,j by the integration rue (4.10) we have to dea with a perturbation K = KP of K where K ϕ,j (x) :=Q,j ( k(x, ) ) =2 /2 m w t k(x, x,t,j ). Provided the degree of accuracy m (4.11) of the quadrature rue is sufficienty high, we wi show that the convergence anaysis of the previous section carries over to the perturbed inear system t=0 (4.12) Ã ξ = β with (Ã ) i,j = K ϕ,i, K ϕ,j L 2 + α ϕ,i,ϕ,j L 2 (4.13) and ( β ) j = δ 1/2 = δ m s=0 m w s w t t=0 Lemma 4.2. Let V be either V d 1 0 t w t g ε (x) k(x, x,t,j ) dx, cf. (3.13). k(x, x,s,i ) k(x, x,t,j ) dx + α ϕ,i,ϕ,j L 2 (4.5) or V s d (4.7) and et K be defined as above with the quadrature rue (4.10) of accuracy m. Further, suppose that the kerne k of the operator K (4.1) is m +1 times continuousy differentiabe with respect to y. Then, γ := K K = O ( δ m+1 ). Proof. Obviousy, K K γ + K K. Using Cauchy s inequaity for sums and the fact that we have a Riesz-basis on V, it is straightforward to verify that (K K ) f 2 C f 2 L 2 n max 0 j n 1 1 (4.11) C f 2 L 2 n C 2 Q max x,y [0,1] 0 (K K )ϕ,j (x) 2 dx m+1 y m+1 k(x, y) 2 δ 2m+3. The assumption on the kerne gives K K = O ( ) δ m+1 as we as γ = O ( ) δ m+1 by Lemma 4.1. Lemma 2.1 hods aso true for K quaitativey: KQ = KP +1 (I P )Q K +1 K + K K = γ +1 + γ. Atogether we found: The convergence resut stated in Theorem 3.2 appies aso to the iteration (3.14) when A is repaced by Ã. Moreover, the optima order r of the decay rate (4.3) is not affected by numerica integration as ong as the quadrature rue has the degree of accuracy r 1. Numerische Mathematik Eectronic Edition page 514 of Numer. Math. 75: (1997)

15 A waveet mutieve method for i-posed probems Computationa compexity and impementation issues We demonstrate the efficiency of the mutieve agorithm (3.14) for the soution of (4.12) by comparing its compexity with the compexity of the Choesky decomposition which is a suitabe direct sover to tacke (4.12). We make the foowing genera assumptions: We wi not incude the computation of à (4.13) into the operation count of the agorithm because this effort has to be raised independent of the specific sover we use for (4.12). The appication of B 1 j in (3.15) is performed according to the remark at the end of Sect Further, the auxiiary inear system with matrix à min is soved using Choesky decomposition where the decomposition is computed once before the iteration starts. Finay, the underying scaing function spaces have order N and the degree of accuracy of the used quadrature rue is N 1. The different parts of the agorithm have the foowing operation count (we consider the eading terms ony): 1. Computation of the matrix à min (4.13): N 2 n 2 /2. Here, we did not take into min account the evauation of the kerne at the quadrature points and we supposed that the integra and the inner products are cacuated exacty. 2. Choesky decomposition of à min and computation of the starting guess (3.16): n 3 /6. min 3. One step of the iteration: n 2. Since the appication of C, min (3.15) to a vector can be reaized by O(n +n 2 ) operations, the evauation of the defect dominates min the compexity. Thus, s steps of the iteration take essentiay O s := sn 2 + n 3 min /6+N 2 n 2 min /2 operations. If we fix the spitting eve L = min then we have on one side that the convergence rate tends to zero for arge and ony a few steps of the iteration yied the soution, cf. Tabe 1. On the other side O s is dominated by n 3 = min n L 3 which gives the unfavourabe compexity O s = O ( ) n 3. However, with a sophisticated impementation we can achieve both, a decreasing convergence rate and an optima compexity. The idea is as foows: we aow the spitting eve L to grow but not too fast. We therefore define (4.14) L() := /3 for +1, where denotes the greatest integer and is the smaest possibe approximation eve, cf. Sects and Now the asymptotic reation (4.3) reads (4.15) ρ 1 () =O ( η L() / α ) =O ( δ 2r/3 / α ) as. So, we sacrificed one third of the optima decay rate of ρ 1 to have the foowing operation count (n =2 ) O s (s+4/3) n 2 +2N 2 n 4/3. Numerische Mathematik Eectronic Edition page 515 of Numer. Math. 75: (1997)

16 516 A. Rieder q Fig. 3. Soid curves: convergence rates ρ 1 of the iteration (3.14), min = 5, with respect to the kerne k 1 and with α =0.001 ( : Daubechies waveet, N =2; : inear spine). Dashed curves: the rate q (4.4). The theoretica bound 0.25 for q is drawn as a dashed straight ine In the atter impementation the undamped mutieve iteration can be viewed as a direct sover with compexity order O ( ) n 2 which outperforms the Choesky sover. Remark. Since C,min (3.15) is a sum of matrices its action on a vector can be performed in parae. This eads to a speed up when the iteration is impemented on a parae machine. Moreover, the iteration can even be acceerated with the hep of matrix compression techniques presented by Harten and Yad-Shaom [16], see aso Beykin, Coifman and Rokhin [1] and Dahmen, Prößdorf and Schneider [7]. The matrixvector product à v can be cacuated with ony O ( n ) or O ( n og n ) operations as far as the kerne k satisfies some additiona requirements Numerica experiments We sha provide numerica approximations to the convergence rates to demonstrate the theoretica resuts as we as the performance of the Schwarz reaxation (3.14). For convenience, we imit ourseves to the Daubechies waveet system of order N = 2 and to the inear spine system of the same order. In both cases we computed the weights of the quadrature rues such that affine inear functions are integrated exacty. The quadrature points are of the form x,t,j =2 (j+t). In a presented experiments the integra in (4.13) is evauated exacty and a kernes are smooth enough to yied γ = O ( ) δ 2. The asymptotic behaviour (4.3) is iustrated in Fig. 3 where approximations to ρ 1, L = 5, are potted for severa and the reguarization parameter α = The underying kerne is k 1 (x, y) =x y,x y,k 1 (x,y) = 0, otherwise. Next, we compute approximations to ρ 1 where the coarsest eve min =2is fixed. We know that ρ 1 < 1 uniformy in if α is not too sma. Here, α is numericay sufficient, see Fig. 4. In both exampes the convergence rates with respect to the inear spines are ceary smaer than those with respect to the Daubechies waveet. The main reason for this observation is the higher reguarity of the inear B spine: the Numerische Mathematik Eectronic Edition page 516 of Numer. Math. 75: (1997)

17 A waveet mutieve method for i-posed probems Fig. 4. Convergence rates ρ 1 with respect to the fixed coarsest eve min =2( : Daubechies waveet, N =2; : inear spine). Left: k 2 (x, y) = cos(π xy), Right: k 1 (x, y) =x y,x y,k 1 (x,y)=0, otherwise. Soid curves: α = 0.001, dashed curves: α = Tabe 1. Necessary iterations to guarantee a reative accuracy smaer than 10 4 (ε =0.04, α =0.001, Daubechies waveet, starting guess either z 0 =0orz 0 =ũ min := H, t minã 1 min H, min ξ min (3.16)) min = 5 min =3 =7 =8 =9 =10 =11 =12 z 0 = z 0 = ũ min z 0 = z 0 =ũ min µ. Here, {z } µ denotes the sequence of iterates generated by (3.14) with starting guess either z 0 =0or =ũ min := H, t à 1 min H,min ξmin (3.16). Our stopping criterion guarantees min Daubechies waveet of order 2 is Höder continuous with exponent 0.55, see Daubechies [9], whereas the inear B spine is Lipschitz continuous. Our ast exampe iustrates the performance of the iteration (3.14) by an approximate soution of (4.1) with kerne k 1 and exact right hand side g(x) = x 2 (x 2 4x+9)/12+(1 cos(3πx))/π 2 /18. Thus, f (x) =(1 x) 2 +cos 2 (3πx/2). In our computations we supposed that g ε is known ony at the discrete points j δ with g ε (j δ ) = g(j δ )+ε j where the random errors {ε j } are distributed uniformy in [ ε, ε]. The integras in the ( β ) j s are evauated by the trapezoida rue. Tabe 1 contains the number s of iteration steps to yied an Eucidean norm of the residue smaer than 10 4 α z s z 0 a reative accuracy z s ξ / z s bounded by The approximate soutions with respect to the discretization eves = 6 and = 11 together with the minimum norm soution f are dispayed in Fig. 5. We considered an absoute error ε = 0.04 and got the optima reguarization parameter by tria and error. It took 14 ( =6)and1(= 11) iteration steps (L = 4) to bound the reative accuracy by Discussion and concusion Our proposed mutieve iteration works most efficienty in its impementation discussed in Sect if Numerische Mathematik Eectronic Edition page 517 of Numer. Math. 75: (1997)

18 518 A. Rieder Fig. 5. Approximate soutions (soid ines) and minimum norm soution (dashed ines) of (4.1) with kerne k 1 (ε =0.04, Daubechies waveet). Left: =6,α=0.001, right: = 11, α = Each α is optima in the sense that the L 2 error is minimized (5.1) η L() α, cf. (4.15). In view of our genera assumption that the noise eve ε>0 is fixed, the requirement (5.1) hods for such arge that (5.2) η L() α 0 (ε), where α 0 (ε) is as in (2.4). The arger the more efficient is the agorithm. The crucia condition ε>0is reasonabe for rea-ife appications where noise cannot be avoided due to the nature of the specific experiment and due to the imitations of the measuring instrument. Readers famiiar with i-posed probems might see a serious drawback of the agorithm in condition (5.2) which is satisfied for arge dimensions of the system (2.1). At a first gance, this seems to be contradiction to the rue: Don t discretize too fine to avoid noise ampification!. The rue appies in the case the reguarization is obtained ony by discretization, see e. g. Natterer [19]. However, in our setting the stabiity of (2.1) comes from Tikhonov reguarization and of (2.1) is pouted by a discretization error as we as by the unavoidabe noise error. Increasing the dimension reduces the discretization error without ampifying the noise. For here it pays to refine the discretization: the soution f ε,α arge, the remaining error f ε,α f is caused ony by the noise provided the reguarization parameter α is seected according to one of the strategies mentioned in Sect This argument is supported by the error estimates given by Pato and Vainikko in [21] and by Fig. 5 where the approximate soution on the eft contains a reativey arge discretization error as we as the noise error. In contrast, the discretization error of the approximate soution shown on the right side is negigibe. Hence, this approximation is much coser to the exact minimum norm soution. In the remainder of this section we focus on the comparison of King s mutieve method [17] with our agorithm. There are severa differences: the most important one is as aready mentioned in the introduction that King considers his iterative scheme as a reguarization technique, that is, he uses the number of iteration steps to baance the discretization and the data error. In his setup reation (5.2) is superfuous, however, for the price of a sow convergence speed of his fu mutieve agorithm. Indeed, even if (5.2) is satisfied, King s iteration Numerische Mathematik Eectronic Edition page 518 of Numer. Math. 75: (1997)

19 A waveet mutieve method for i-posed probems 519 has a spitting eve dependent convergence rate, see [17, Coroary 3.4], which deteriorates for an increasing number of eves. In our case, provided (5.2) appies, the convergence becomes even faster with increasing discretization eve and hence with increasing spitting eve L() (4.14). Of course, our agorithm can aso be performed when (5.1) is strongy vioated, for instance, if ε is zero. Its performance and reguarization abiity in the atter situation remains sti to be investigated. Besides the above comments our method outperforms King s method with respect to the foowing aspects which we ony touch briefy. No damping parameter is needed to gain optima convergence resuts. Each iteration step is paraeizabe due to the additive structure. One iteration step is much cheaper (King s agorithm is a mutipicative iteration which requires two matrix-vector mutipications on each eve of the mutieve process). At the end we ike to emphasize that it is uncear how the performance of King s method improves (if it improves at a!) with the smoothness of the operator and the order of the approximation spaces. Appendix: Proof of Theorem 3.2 The proof of Theorem 3.2 wi be given by providing bounds Γ u and Γ o for the inequaity (3.5). We note the estimate (A.1) b j (w j,w j ) a(w j,w j ) ( 1+γ 2 min /α ) b j (w j,w j ) for a w j W j, j min, which is an immediate consequence from the definition of b j (3.1) and from Lemma 2.1. First, we consider the upper bound Γ o. Let v V then v = P min v + j Q j v due to (2.6). For the foowing estimate we wi use the statements of Theorem 2.2 and Coroary 2.3 as we as the inequaities (A.1), (3.6), and 2 xy x 2 +y 2, x,y R: a(v, v) = a(p min v, P min v)+ 1 1 j= min a(q j v, Q j v) +2 a(p min v, Q j v)+2 j= min ( 1+η 2 min /α ) v j= min i=j +1 a(q j v, Q i v) j= min η j a(p min v, P min v) 1/2 a(q j v, Q j v) 1/2 / α Numerische Mathematik Eectronic Edition page 519 of Numer. Math. 75: (1997)

20 520 A. Rieder j= min i=j +1 ( 1+η 2 min /α ) v j = min η j 2 j= min i=j +1 η i a(q j v, Q j v) 1/2 a(q i v, Q i v) 1/2 / α ( a(pmin v, P min v)+a(q j v, Q j v) ) / α 1 ( 1+η 2 min /α ) v 2 η i ( a(qj v, Q j v)+a(q i v, Q i v) ) / α 1 ) + C η η min (a(p min v, P min v)+ a(q j v, Q j v) / α j= min 1 +C η η min a(q j v, Q j v)/ α j = min ( 1+η 2 min /α ) v 2 ( +2C η ηmin / α )( 1+η 2 min /α ) v 2. Thus, we have verified the right inequaity in (3.5) with Γ o from Theorem 3.2. Now, we concentrate on the ower bound Γ u. The first inequaity of the foowing estimate comes from (A.1) and the second one foows by the spitting v = P min v + j Q j v for v V : (A.2) v 2 a(p min v, P min v)+ a(v, v) j= min i=j j= min a(q j v, Q j v) j= min a(p min v, Q j v) a(q j v, Q i v). To proceed we suppy Lemma A.1. Lemma A.1. Let P and Q be the projections defined in Sect. 2 and et a be the biinear form (2.8). If i /=j then Further, a(q i v, Q j v) ( γ i γ j /α ) a(v, v) for a v X. a(p v, Q j v) ( 1+γ / α )( γ j / α ) a(v, v) for j and for a v X. Numerische Mathematik Eectronic Edition page 520 of Numer. Math. 75: (1997)

21 A waveet mutieve method for i-posed probems 521 Proof. Writing A for K K + αi we have that a(q i v, Q j v) = KQ i v, KQ j v Y = KQ i A 1/2 A 1/2 v, KQ j A 1/2 A 1/2 v Y ( ) KQ i KQ j /α a(v, v). Lemma 2.1 impies now the first statement. Taking into account that KP A 1/2 KA 1/2 + K(I P )A 1/2 1+γ / α one can prove the second statement anaogousy. Appying Lemma A.1 to (A.2) we achieve v 2 a(v, v) +2 ( 1+η min / α ) ( C η η min / α ) a(v, v) +2 ( Cηη 2 2 min /α ) a(v, v). Hence, Theorem 3.2 is proved. References 1. Beykin, G., Coifman, R., Rokhin, V. (1991): Fast waveet transforms and numerica agorithms I. Comm. Pure App. Math., 44, Chui, C.K. (1992): An Introduction to Waveets. Waveet Anaysis and its Appications. Academic Press, Boston 3. Chui, C.K., Quak, E. (1992): Waveets on a bounded interva. In D. Braess, Schumaker, L.L., editors, Numerica Methods in Approximation Theory, Vo. 9, Internationa Series of Numerica Mathematics, Vo. 105, pages Birkhäuser Verag, Base 4. Chui, C.K., Wang, J.Z. (1992): On compacty supported spine waveets. Trans. Amer. Math. Soc., 330, Ciaret, P.G. (1978): The Finite Eement Methods for Eiptic Probems. Sudies in Mathematics and its Appications. North-Hoand, Amsterdam 6. Cohen, A., Daubechies, I., Via, P. (1993): Waveets on the interva and fast waveet transforms. App. and Comp. Harm. Ana., 1, Dahmen, W., Prößdorf, S., Schneider, R. (1993): Waveet approximation methods for pseudodifferentia equations II: Matrix compression and fast soution. Adv. in Comp. Math., 1, Daubechies, I. (1988): Orthonorma bases of compacty supported waveets. Comm. Pure App. Math., 41, Daubechies, I. (1992): Ten Lectures on Waveets. CBMS-NSF Series in Appied Mathematics. SIAM Pubications, Phiadephia 10. Davis, P.J., Rabinowitz, P. (1975): Methods of Numerica Integration. Computer Science and Appied Mathematics. Academic Press, New York 11. Deufhard, P., Hohmann, A. (1994): Numerica Anaysis: A First Course in Scientific Computation. de Gruyter Textbook. de Gruyter, Berin, New York 12. Dicken, V., Maaß, P. (1996): Waveet-Gaerkin methods for i-posed probems. J. Inv. I-Posed Probems, to appear 13. Donoho, D.L. (1995): Noninear soution of inear inverse probems by waveet-vagueette decomposition. App. and Comp. Harm. Ana., 2, Groetsch, C.W. (1984): The theory of Tikhonov reguarization for Fredhom equations of the first kind. Pitman, Boston Numerische Mathematik Eectronic Edition page 521 of Numer. Math. 75: (1997)

22 522 A. Rieder 15. Hackbusch, W. (1994): Iterative soution of arge sparse systems of equations. Appied Mathematica Sciences. Springer-Verag, New York 16. Harten, A., Yad-Shaom, I. (1994): Fast mutiresoution agorithms for matrix-vector mutipication. SIAM J. Numer. Ana., 31, King, J.T. (1992): Mutieve agorithms for i-posed probems. Numer. Math., 61, Louis, A.K. (1989): Inverse und schecht gestete Probeme. Studienbücher Mathematik. B. G. Teubner, Stuttgart, Germany (Engish transation in preparation) 19. Natterer, F. (1977): Reguarisierung schecht gesteter Probeme durch Projektionsverfahren. Numer. Math., 28, Oswad, P. (1994): Mutieve Finite Eement Approximation: Theory and Appications. Teubner Skripten zur Numerik. B. G. Teubner, Stuttgart, Germany 21. Pato, R., Vainikko, G.M. (1990): On the reguarization of projection methods for soving iposed probems. Numer. Math., 57, Rüde, U. (1993): Mathematica and Computationa Techniques for Mutieve Adaptive Methods. Frontiers in Appied Mathematics. SIAM, Phiadephia 23. Strang, G. (1989): Waveets and diation equations: A brief introduction. SIAM Review, 31, Strang, G., Fix, G. (1973): A Fourier anaysis of the finite eement variationa method. In Constructive Aspects of Functiona Anaysis, Rome. Edizioni Cremonese 25. Swedens, W., Piessens, R. (1994): Quadrature formuae and asymptotic error expansions for waveet approximations of smooth functions. SIAM J. Numer. Ana., 31, Xia, X.-G., Nashed, M.Z. (1994): The Backus-Gibert method for signas in reproducing Hibert spaces and waveet subspaces. Inverse Probems, 10, Yserentant, H. (1993): Od and new convergence proofs for mutigrid methods. In Acta Numerica, pages Cambridge University Press, New York This artice was processed by the author using the LaT E X stye fie pjour1m from Springer-Verag. Numerische Mathematik Eectronic Edition page 522 of Numer. Math. 75: (1997)