Fractional Tikhonov regularization for linear discrete ill-posed problems

Transcription

1 BIT manuscipt No. (will be inseted by the edito) Factional Tikhonov egulaization fo linea discete ill-posed poblems Michiel E. Hochstenbach Lotha Reichel Received: date / Accepted: date Abstact Tikhonov egulaization is one of the most popula methods fo solving linea systems of equations o linea least-squaes poblems with a seveely ill-conditioned matix A. This method eplaces the given poblem by a penalized least-squaes poblem. The pesent pape discusses measuing the esidual eo (discepancy) in Tikhonov egulaization with a seminom that uses a factional powe of the Mooe-Penose pseudoinvese of AA T as weighting matix. Popeties of this egulaization method ae discussed. Numeical examples illustate that the poposed scheme fo a suitable factional powe may give appoximate solutions of highe quality than standad Tikhonov egulaization. Keywods Ill-posed poblem egulaization factional Tikhonov weighted esidual nom filte function discepancy pinciple solution nom constaint. Mathematics Subect Classification (2000) 65F10 65F22 65R30 Föbeg, Böck, Ruhe: A Golden Baid fo 50 Yeas of BIT. 1 Intoduction This pape is concened with the appoximate solution of linea least-squaes poblems min Ax b (1.1) x Rn Michiel E. Hochstenbach Depatment of Mathematics and Compute Science Eindhoven Univesity of Technology, PO Box 513, 5600 MB, The Nethelands hochsten Lotha Reichel Depatment of Mathematical Sciences Kent State Univesity, Kent, OH 44242, USA eichel@math.kent.edu

2 2 Michiel E. Hochstenbach, Lotha Reichel with a matix A R m n of ill-detemined ank, i.e., A has many singula values of diffeent odes of magnitude close to the oigin. In paticula, A is seveely illconditioned and may be singula. Least-squaes poblems with a matix of this kind ae often efeed to as discete ill-posed poblems. Fo notational convenience, we assume that m n, howeve, the methods discussed also can be applied when m < n. Thoughout this pape denotes the Euclidean vecto nom. The vecto b R m epesents available data, which is contaminated by an eo e R m. The eo may stem fom measuement inaccuacies o discetization. Thus, b = ˆb+e, (1.2) whee ˆb is the unknown eo-fee vecto associated with b. We will assume the unavailable eo-fee system Ax = ˆb (1.3) to be consistent and denote its solution of minimal Euclidean nom by ˆx. We would like to detemine an appoximation of ˆx by computing a suitable appoximate solution of (1.1). Due to the ill-conditioning of the matix A and the eo e in b, the solution of the least-squaes poblem (1.1) of minimal Euclidean nom is typically a poo appoximation of ˆx. Tikhonov egulaization is a popula appoach to detemine an appoximation of ˆx. This method eplaces the minimization poblem (1.1) by a penalized least-squaes poblem. We conside penalized least-squaes poblems of the fom min x R n{ Ax b 2 W + µ x 2 }, (1.4) whee x W = (x T Wx) 1/2 and W is a symmetic positive semidefinite matix. The supescipt T denotes tansposition. The poblem (1.4) has a unique solution x µ fo all positive values of the egulaization paamete µ. The value of µ detemines how sensitive x µ is to the eo e in b, and how much x µ diffes fom the desied solution ˆx of (1.3). We popose to let W = (AA T ) (α 1)/2 (1.5) fo a suitable value of α > 0, whee A T denotes the tanspose of A. When α < 1, we define W with the aid of the Mooe-Penose pseudoinvese of AA T. The seminom W allows the paamete α to be chosen to impove the quality of the computed solution x µ,α of (1.4). We efe to (1.4) with W given by (1.5) as the weighted Tikhonov method o as the factional Tikhonov method. Standad Tikhonov egulaization based on the Euclidean nom is obtained when α = 1. Then W is the identity matix. Recently, Klann and Ramlau [7] poposed a factional Tikhonov egulaization method diffeent fom (1.4) (1.5). We comment on thei appoach in Sections 2 and 6. The nomal equations associated with the Tikhonov minimization poblem (1.4) with W defined by (1.5) ae given by ((A T A) (α+1)/2 + µi)x = (A T A) (α 1)/2 A T b. (1.6)

3 Factional Tikhonov egulaization 3 Thei solution x = x µ,α is uniquely detemined fo any µ > 0 and α > 0. When A is of small to modeate size, x µ,α can be conveniently computed fom (1.6) with the aid of the singula value decomposition of A; see Section 3. Lage-scale poblems can be solved by fist poecting them onto a subspace of small dimension, e.g., a low-dimensional Kylov subspace, and then applying the appoach of Section 3 to the poected poblem. This is descibed in Section 4. The pesent pape is oganized as follows. Section 2 discusses popeties of filte functions associated with (1.4) and othe egulaization methods. The detemination of µ and α so that the solution of (1.4) satisfies the discepancy pinciple is consideed in Section 3 fo small poblems and in Section 4 fo lage ones. Petubation bounds ae deived in Section 5, and Section 6 epots a few computed esults. Section 7 contains concluding emaks. 2 Filte functions Intoduce the singula value decomposition (SVD), A = UΣV T, (2.1) whee U = [u 1,u 2,...,u m ] R m m and V = [v 1,v 2,...,v n ] R n n ae othogonal matices, and Σ = diag[σ 1,σ 2,...,σ n ] R m n. The singula values ae odeed accoding to σ 1 σ 2... σ > σ +1 =... = σ n = 0, whee the index is the ank of A; see, e.g., [4] fo discussions on popeties and the computation of the SVD. We fist eview filte functions fo some popula solution methods in Subsections Note that this list is fa fom complete; fo instance, we do not mention the exponential appoach of [2]. Some desiable popeties of filte functions ae summaized in Subsection 2.4, and Subsection 2.5 discusses popeties of filte functions associated with (1.6). 2.1 Tuncated SVD Appoximate solutions of (1.1) detemined by the tuncated SVD (TSVD) ae of the fom k 1 x tsvd = (u T b)v (2.2) σ fo some cut-off paamete k. It is convenient to expess appoximate solutions x of (1.1) with the aid of filte functions ϕ, i.e., x = n ϕ(σ )(u T b)v ; (2.3)

4 4 Michiel E. Hochstenbach, Lotha Reichel see, e.g., [5] fo a discussion on filte functions. Fo instance, the appoximate solution (2.2) can be witten as x tsvd = n ϕ(σ )(u T b)v, whee ϕ(σ) = ϕ tsvd (σ) = { 1/σ if σ τ, 0 othewise, and σ k+1 < τ σ k is abitay. 2.2 Tikhonov Standad Tikhonov egulaization (1.4) (with W = I) coesponds to the filte function ϕ tikh (σ) = σ σ 2 + µ, whee µ > 0 is the egulaization paamete. The asymptotics of this function ae ϕ tikh (σ) = σ µ +O(σ 3 ) (σ ց 0), ϕ tikh (σ) = σ 1 +O(σ 3 ) (σ ). Figue 2.1 displays functions ϕ tikh fo seveal values of µ µ=10 3 µ=10 2 µ=10 1 µ=10 0 Fig. 2.1 Filte functions ϕ tikh (σ) = σ σ 2 +µ fo µ = 10 3,10 2,10 1,10 0, and σ Note the logaithmic scales.

5 Factional Tikhonov egulaization Klann and Ramlau s filte functions Klann and Ramlau [7] conside the family of filte functions ϕ KR (σ) = with paamete γ > 1/2. Its asymptotics ae ϕ KR (σ) = σ 2γ 1 σ 2γ 1 (σ 2 + µ) γ (2.4) µ γ +O(σ 2γ+1 ) (σ ց 0), ϕ KR (σ) = σ 1 +O(σ 3 ) (σ ). Standad Tikhonov egulaization is ecoveed fo γ = 1. The analogue of the nomal equations (1.6) associated with the filte function (2.4) is given by (A T A+ µi) γ x = (A T A) (γ 1) A T b. (2.5) Fo γ 1, this equation is not elated to a simple minimization poblem of the fom (1.4). 2.4 Desiable popeties of filte functions We now discuss some desiable popeties of filte functions. Equation (2.3) yields x 2 = b A x 2 = n n (ϕ(σ )) 2 (u T b) 2, (1 σ ϕ(σ )) 2 (u T b)2 + m (u T b)2. (2.6) =n+1 To get a small esidual nom fo matices with lage singula values, we equie in view of (2.6) that ϕ(σ) = σ 1 + o(σ 1 ) (σ ). (2.7) Moeove, we would like the filte function to satisfy ϕ(σ) = o(1) (σ ց 0). (2.8) This ensues that the computed appoximate solution x of (1.1) only contains small multiples of singula vectos associated with small singula values. These singula vectos usually epesent high-fequency oscillations. The above filte functions diffe in how quickly they convege to zeo when σ deceases to zeo. Fast convegence implies significant smoothing of the computed appoximate solution (2.3). The inteest of Klann and Ramlau [7] in the filte functions (2.4) stems fom the fact that they povide less smoothing fo 1/2 < γ < 1 than ϕ tikh.

6 6 Michiel E. Hochstenbach, Lotha Reichel 2.5 Factional Tikhonov We tun to the family of filte functions associated with weighted Tikhonov egulaization (1.4) with W given by (1.5). The following popeties ae easy to show and illustate that these filte functions satisfy the desiable popeties of filte functions stated in the pevious subsection. Poposition 2.1 The filte function fo weighted Tikhonov egulaization (1.4) with W defined by (1.5) fo some α > 0 is given by It has the asymptotics ϕ tikh,w (σ) = σ α σ α+1 + µ. (2.9) ϕ tikh,w (σ) = σ 1 +O(σ (α+2) ) (σ ), ϕ tikh,w (σ) = σ α µ +O(σ 2α+1 ) (σ ց 0). In paticula, ϕ tikh,w satisfies (2.7) and (2.8). The asymptotic behavio of ϕ tikh,w (σ) as σ ց 0 shows this function to povide less smoothing than ϕ tikh fo 0 < α < 1. Figue 2.2 displays the behavio of the functions ϕ tikh,w fo µ = 10 2 and seveal values of α. A compaison with Figue 2.1 shows that components of the solution (2.3) associated with tiny singula values ae damped less by the function ϕ tikh,w than by ϕ tikh. This often yields computed appoximate solutions of (1.1) of highe quality than with standad Tikhonov egulaization α=1.5 α=1 α=0.5 α=0.25 Fig. 2.2 Filte functions ϕ tikh,w (σ) = σα σ α+1 +µ fo α = 0.25,0.5,1,1.5, µ = 10 2, and σ 10 4.

7 Factional Tikhonov egulaization 7 3 Choosing µ and α We fist investigate the dependence of the solution x µ,α of (1.6) on the paametes µ and α. This is conveniently caied out with the help of the SVD of A. Subsequently, we detemine µ with the discepancy pinciple and study how the computed solutions vay with α. The situation when x µ,α is equied to be of specified nom is also consideed. Substituting the SVD (2.1) into (1.6) yields ((Σ T Σ) (α+1)/2 + µi)y = (Σ T ) α U T b. Denote the solution by y µ,α. Then x µ,α = Vy µ,α solves (1.6), and x µ,α 2 = y µ,α 2 = whee is the ank of A. Thus, µ x µ,α 2 = 2 σ 2α + µ) 2 (ut b)2, (3.1) σ 2α + µ) 3 (ut b) 2. (3.2) Clealy, µ x µ,α 2 is a monotonically deceasing function. Similaly, α x µ,α 2 = 2µ log(σ )σ α (σ + µσ α ) 3 (ut b)2. We may escale the poblem (1.1) so that A < 1. Then log(σ ) < 0 and it follows that α x µ,α 2 is monotonically deceasing. We assume this scaling in the pesent section. The choice of the egulaization paamete µ depends on the amount of eo e in b. Conside fo the moment standad Tikhonov egulaization, i.e., the situation when α = 1. Geneally, the lage e, the lage µ should be; see, e.g., Poposition 3.1 below. Howeve, it follows fom (3.2) that inceasing µ deceases the nom of the computed solution x µ,1. Theefoe, the computed solution may be of significantly smalle nom than the desied solution ˆx. This difficulty can be emedied by choosing α < 1, because this inceases the nom of the computed solution. Computed examples in Section 6 illustate that, indeed, α < 1 typically yields moe accuate appoximations of ˆx than α = 1. We tun to the situation when a faily accuate bound fo the eo in b, e ε, is available. Then we can apply the discepancy pinciple to detemine a suitable value of the egulaization paamete µ. Let α > 0 be fixed and define δ = ηε, (3.3)

8 8 Michiel E. Hochstenbach, Lotha Reichel whee η > 1 is a use-supplied constant independent of ε. We would like to detemine µ > 0, so that the solution x µ,α of (1.4) satisfies b Ax µ,α = δ. (3.4) Then the vecto x µ,α is said to satisfy the discepancy pinciple; see, e.g., [5] fo discussions on this choice of egulaization paamete. The change of vaiable λ = µ 1 gives a simple expession fo the coefficients in the leftmost sum in (2.6) with ϕ = ϕ tikh,w, σ α 1 σ σ α+1 + µ = 1 λ σ α+1 λ σ α = 1 λ σ α The solution of (3.4) fo µ > 0 is equivalent to the computation of the positive zeo of the function F α (λ) = (λ σ α+1 + 1) 2 (u T b)2 + m (u T b)2 δ 2. (3.5) =+1 We ae in a position to show how µ, such that x µ,α satisfies (3.4) fo fixed α > 0, depends on δ. Poposition 3.1 Let µ = µ(δ) > 0 be such that x µ,α satisfies (3.4) fo fixed α > 0. Then dµ/dδ > 0. Poof Conside λ(δ) = 1/µ(δ). It follows fom (3.5) that the invese function satisfies δ(λ) 2 = (λ σ α+1 + 1) 2 (u T b) 2 + m (u T b) 2. =+1 Diffeentiating with espect to λ yields 2δ(λ)δ (λ) = 2 σ α+1 (λ σ α+1 + 1) 3 (ut b) 2. It follows that δ (λ) < 0. Consequently, λ (δ) < 0 and µ (δ) > 0. We conside popeties of Newton s method when applied to the computation of the positive zeo of the function (3.5). Howeve, othe zeo-findes also can be used. A discussion of Newton s method and othe zeo-findes fo the situation when α = 1 is povided in [9]. Poposition 3.2 Newton s method applied to the computation of the positive zeo of F α with initial iteate λ 0 = 0 conveges quadatically and monotonically. Poof The quadatic convegence is a consequence of the analyticity of F α (λ) in a neighbohood of the positive eal axis in the complex plane. The monotonic convegence follows fom the fact that fo evey fixed α > 0 and λ 0, the function F α satisfies F α (λ) < 0 and F α (λ) > 0.

9 Factional Tikhonov egulaization 9 Let α > 0 and let µ = µ(α) be detemined so that x µ,α satisfies the discepancy pinciple. The following esult shows how x µ,α depends on α > 0. Poposition 3.3 Let fo α > 0 the egulaization paamete µ = µ(α) be such that x µ,α satisfies (3.4). Then thee is an open eal inteval Ω containing unity such that agmin x µ(α),α = 1. α Ω Poof The equation F α (λ) = 0 can be expessed as µ 2 (σ α+1 + µ) 2 (ut b)2 = δ 2 m (u T b)2. (3.6) =+1 We may conside µ = µ(α) a function of α. Implicit diffeentiation of (3.6) with espect to α yields 2µ which, since µ > 0, implies that whee Intoduce the function Then G (α) = = σ α+1 (µ µ log(σ )) + µ) 3 (u T b)2 = 0, (3.7) ξ = G(α) = x µ(α),α 2 = 2σ 2α log(σ ) ξ (µ µ log(σ )) = 0, (3.8) σ α+1 + µ) 3 (ut b) 2. (3.9) 2σ 2α (log(σ )µ µ ) (σ α+1 (u T + µ) 3 b)2 = 2 ξ σ α 1 (µ log(σ ) µ ). σ 2α + µ) 2 (ut b) 2. + µ) 2σ 2α (σ α+1 log(σ )+ µ ) (σ α+1 (u T + µ) 3 b) 2 It follows fom (3.8) that G (1) = 0. Moeove, diffeentiating (3.8) yields { ξ (µ µ log(σ ))+ξ (µ µ log(σ )) } = 0. (3.10)

10 10 Michiel E. Hochstenbach, Lotha Reichel Since G (α) = 2 we obtain, in view of (3.10), σ α 1 { (ξ + ξ log(σ ))(µ log(σ ) µ )+ξ (µ log(σ ) µ ) }, G (1) = 2 ξ log(σ )(µ log(σ ) µ ). The above sum is obtained by multiplying the tems in (3.8) by the positive weights log(σ ); the lagest weights multiply the lagest tems. Theefoe, G (1) > 0. By continuity, G (α) is positive in a neighbohood Ω of α = 1. Thus, G(α) has a local minimum at α = 1. Fo some linea discete ill-posed poblems (1.1) an estimate of the nom of the desied solution ˆx may be known. Then it may be desiable to equie the computed solution x µ,α to be of the same nom, i.e., = x µ,α. (3.11) This type of poblems is discussed in [3,8,10]. The following esult sheds light on how Ax µ,α b depends on α fo solutions that satisfy (3.11). Poposition 3.4 Let, fo α > 0, the egulaization paamete µ = µ(α) be such that x µ,α satisfies (3.11). Then thee is an open eal inteval Ω containing unity, such that agmin b Ax µ(α),α = 1. α Ω Poof This esult is shown in a simila fashion as Poposition 3.3. Diffeentiating the ight-hand side and left-hand side of (3.11) with espect to α, keeping in mind that µ = µ(α), gives analogously to (3.8) the equation ζ (µ log(σ ) µ ) = 0, whee ξ is defined by (3.9). Intoduce the function H(α) = b Ax µ(α),α 2 = µ 2 + µ) 2 (ut b)2 + ζ = ξ σ α 1, (3.12) m (u T b)2, (3.13) =+1 whee the ight-hand side is obtained by substituting (2.9) into (2.6). Then (cf. (3.7)) H (α) = 2µ and it follows fom (3.12) that H (1) = 0. The epesentation H (α) = 2µ ζ σ 1 α (µ µ log(σ )) ξ (µ µ log(σ ))

11 Factional Tikhonov egulaization 11 conveniently can be diffeentiated to give H (α) = 2µ +2µ Diffeentiating (3.12) yields ξ (µ µ log(σ )) {ξ (µ µ log(σ ))+ξ (µ µ log(σ ))}. (3.14) ζ (µ log(σ ) µ )+ζ (µ log(σ ) µ ) = 0. (3.15) Let α = 1. Then ζ = ξ fo all. Using this popety when substituting (3.15) into (3.14) gives, in view of (3.12), H (1) = 2µ (ξ ζ )(µ µ log(σ )). (3.16) It follows fom ξ = ζ σ 1 α that, fo α = 1, ξ = ζ ζ log(σ ). Substituting the latte expession into (3.16) yields H (1) = 2µ ζ (µ µ log(σ ))log(σ ). Compaing this sum with (3.12) shows that H (1) > 0, similaly as the analogous esult fo G (1) in the poof of Poposition 3.3. By continuity, H is convex in a neighbohood Ω of α = 1. Popositions 3.3 and 3.4 show the choice α = 1, which coesponds to standad Tikhonov egulaization, to be quite natual; by Poposition 3.3 this choice minimizes x µ(α),α locally when the esidual nom b Ax µ(α),α is specified and by Popositions 3.4 the esidual nom has a local minimum fo α = 1 when x µ(α),α is specified. We emak that the value of δ used in Poposition 3.3 does not have to be defined by (3.3) and, similaly, the value of in Poposition 3.4 does not have to be close to ˆx. Howeve, despite these popeties of standad Tikhonov egulaization, numeical examples of Section 6 illustate that α < 1 may yield moe accuate appoximations of ˆx. 4 Lage-scale poblems The solution method descibed in the pevious section, based on fist computing the SVD of A, is too expensive to be applied to lage poblems. We theefoe popose to poect lage-scale poblems onto a Kylov subspace of small dimension and then apply the solution method of Section 3 to the small poblem so obtained. Fo instance, application of l steps of Lanczos bidiagonalization to A with initial vecto b/ b yields the decompositions AV l = U l+1 C l, A T U l = V l C T l, U l+1e 1 = b/ b, (4.1)

12 12 Michiel E. Hochstenbach, Lotha Reichel whee the matices U l+1 R m (l+1) and V l R n l have othonomal columns, and the lowe bidiagonal matix C l R (l+1) l has positive subdiagonal enties. Moeove, U l R m l is made up of the l fist columns of U l+1, C l R l l consists of the fist l ows of C l, and e 1 = [1,0,...,0] T denotes the fist axis vecto. The columns of V l span the Kylov subspace K l (A T A, A T b) = span{a T b, (A T A)A T b,..., (A T A) l 1 A T b}; (4.2) see, e.g., [1] fo a discussion. The numbe of bidiagonalization steps, l, is geneally chosen quite small; we assume l to be small enough so that the decompositions (4.1) with the stated popeties exist. It follows fom (4.1) that min Ax b = min C l y e 1 b. (4.3) x K l (A T A,A T b) y R l Thus, application of l steps of Lanczos bidiagonalization educes the lage minimization poblem (1.1) to the small minimization poblem in the ight-hand of (4.3). We apply the factional Tikhonov method to the latte poblem as descibed in Section 3. A numeical illustation can be found in Section 6. 5 Sensitivity analysis This section studies the sensitivity of the egulaization paamete µ in (1.6) to petubations in the discepancy δ = ηε in (3.4) and to changes in = ˆx. (5.1) Ou analysis is motivated by the fact that only appoximations of ε and may be available. In this section A is abitaily lage. It is convenient to let µ d denote the solution of (3.4) and to let x d = x µd,α be the associated solution of (1.6). Since we keep the paamete α fixed in this section, we will not explicitly indicate the dependence of µ d and x d on α. Similaly, let µ n denote the value of the egulaization paamete such that x µn =, whee is given by (5.1), and define x n = x µn. We will also need the esidual eo d = b Ax d. It can be shown that fo δ sufficiently lage, µ n < µ d, x d < x n. The following bounds shed some light on the sensitivity of µ n = µ n ( ) and µ d = µ d (δ) to petubations in and δ, espectively. The lowe bound involves the constant which can also be expessed as δ 2 = δ 2 = δ 2 µ 2 d + µ d ) 2 (ut b)2, m (u T b) 2 ; =+1

13 Factional Tikhonov egulaization 13 cf. (3.13). In paticula, δ 2 = δ 2 fo consistent least-squaes poblems (1.1). When A is squae, the discete ill-posed least-squaes poblems consideed ae typically consistent with a seveely ill-conditioned matix. Poposition 5.1 The following bounds hold, µ n µ n ( ) A α+1 + µ n (5.2) and { max δ A 1 α x d 2, δ µ2 d δ 2 } µ d (δ). (5.3) Poof To show the inequalities (5.2), we expess the constaint x n 2 = 2 in tems of the singula value decomposition (2.1), σ 2α + µ n ) 2 (ut b)2 = 2 ; (5.4) cf. (3.1). Consideing µ n a function of and diffeentiating (5.4) with espect to gives ( ) µ n( ) σ = 2α 1 (σ α+1 + µ n ) 3 (ut b) 2. Theefoe, µ n( ) < 0 and µ n ( ) (σ α µ n )( Moeove, µ n( ) µn ( σ 2α + µ n ) 2 (ut b)2 σ 2α + µ n ) 2 (ut b) 2 ) 1 = A α+1 + µ n. ) 1 = µ n. We tun to the lowe bounds (5.3). The discepancy pinciple detemines the egulaization paamete µ d = µ d (δ) so that d 2 = δ 2, which can be witten as µ 2 d + µ d ) 2 (ut b)2 + Diffeentiating this expession with espect to δ yields µ d ( (δ) = µ 1 d δ σ α+1 m (u T b)2 = δ 2. =+1 + µ d ) 3 (ut b) 2 ) 1.

14 14 Michiel E. Hochstenbach, Lotha Reichel It follows fom that σ α+1 + µ d ) 3 (ut b) 2 = 1 µ 2 d 1 µ 2 d σ α+1 µ2 d + µ d (σ α+1 + µ d ) 2 (ut b) 2 σ α+1 µ d 2 µ d (δ) δ µ d δ 2. Altenatively, we may substitute the bound into to obtain µ d ( (δ) = µ 1 d δ ( µ d (δ) δ σ 1 α 1 µ d µ d σ 1 α σ α+1 σ α+1 σ 1 α + µ d µ d σ 1 α + µ d ) 2 (ut b)2 = δ 2 µ d 2 σ 2α + µ d (σ α+1 + µ d ) 2 (ut b)2 σ 2α + µ d ) 2 (ut b)2 ) 1 = ) 1 δ A 1 α x d 2. Using elementay computations, we can also bound the sensitivity of the solution and esidual noms to petubations in µ to fist ode. Coollay 5.1 We have and A α+1 + µ n (µ n ) µ n { A δ 1 α x d 2 (µ d ) min, δ δ 2 δ µ d }. 6 Computed examples We show numeical expeiments caied out fo ten linea discete ill-posed poblems fom Regulaization Tools [6]. These poblems ae discetized Fedholm integal equations of the fist kind. The matices A fo all poblems ae squae. The small poblems solved by the method of Section 3 ae of ode 100; the lage-scale poblems solved as descibed in Section 4 ae of ode MATLAB codes in [6] detemine these matices and the solutions ˆx fom which we compute the eo-fee ight-hand side (assumed unknown) of (1.3) by ˆb = Aˆx. The vecto b in (1.1) is detemined fom (1.2), whee the enties of the eo vecto e ae nomally distibuted andom numbes with zeo mean, scaled to coespond to a desied eo-level e / ˆb. In the expeiments, we conside the eo-levels 1%, 5%, and 10%.

15 Factional Tikhonov egulaization 15 Expeiment 6.1 We show the pefomance of the method descibed in Section 3 when applied to poblems (1.1) with matices of ode 100. Tables display the elative eos (qualities) x µ,α ˆx / ˆx fo standad Tikhonov (α = 1, labeled Tikh ), fo factional Tikhonov fo the α-values 0.8, 0.6, and 0.4 (labeled Fac ), as well as the atios Fac / Tikh. The egulaization paamete µ is detemined by the discepancy pinciple, i.e., so that x µ,α satisfies (3.4) with δ given by (3.3), whee ε = e and η = 1.1. The tables show impoved accuacy of the computed solutions fo the vast maoity of the poblems. The choice α = 0.8 gives bette esults than α = 1 fo almost all examples. Smalle values of α, such as α = 0.5 wok even bette fo some examples, at the cost of yielding wose esults fo othes. A geneal ule-of-thumb is that the lage the eo-level, the moe advantageous it is to let α < 1; fo smalle eo-levels, factional Tikhonov can be seen to pefom best fo α-values close to unity. Table 6.1 Qualities of Tikhonov, factional Tikhonov, and thei atios fo vaious n = 100 examples fo diffeent eo-levels (1%, 5%, 10%) and α = 0.8. Eo 1% 5% 10% Poblem Tikh Fac Ratio Tikh Fac Ratio Tikh Fac Ratio baat 2.1e-1 2.0e-1 9.7e-1 3.3e-1 3.1e-1 9.6e-1 3.6e-1 3.5e-1 9.8e-1 deiv e-1 2.7e-1 9.8e-1 3.8e-1 3.7e-1 9.7e-1 4.4e-1 4.3e-1 9.7e-1 deiv e-1 2.6e-1 9.7e-1 3.8e-1 3.7e-1 9.7e-1 4.4e-1 4.3e-1 9.6e-1 deiv e-2 3.4e-2 9.0e-1 7.0e-2 6.0e-2 8.5e-1 9.3e-2 8.2e-2 8.9e-1 foxgood 4.3e-2 4.2e-2 9.6e-1 1.4e-1 1.2e-1 8.7e-1 2.0e-1 1.8e-1 8.8e-1 gavity 3.7e-2 2.9e-2 8.0e-1 7.4e-2 6.5e-2 8.7e-1 1.1e-1 1.0e-1 9.0e-1 heat 1.5e-1 1.5e-1 1.0e-0 3.1e-1 3.1e-1 9.9e-1 4.4e-1 4.3e-1 9.7e-1 ilaplace 1.6e-1 1.5e-1 9.6e-1 2.0e-1 1.9e-1 9.6e-1 2.2e-1 2.1e-1 9.7e-1 phillips 2.8e-2 3.1e-2 1.1e-0 6.5e-2 6.2e-2 9.6e-1 1.1e-1 1.0e-1 9.5e-1 shaw 1.5e-1 1.5e-1 9.4e-1 1.8e-1 1.8e-1 9.8e-1 2.0e-1 2.0e-1 9.7e-1 To shed some moe light on the significance of the paamete α, Figue 6.1 povides plots that show the optimal α as a function of the eo-level (1%, 2%,..., 10%) fo each of the test poblems in Tables Fo a given andom eo vecto, we detemine the best value fo α fom the discete set 0.01, 0.02,..., 1; that is, the α fo which the coesponding solution has the smallest elative eo compaed to ˆx. The gaphs show the aveages of the optimal α-values ove 100 uns with diffeent andom eo vectos. The figues suggest that fo many of the test poblems, the value of the optimal α does not dastically vay with the eo level. Also, it is clea that the optimal value of α is smalle than unity fo all poblems, often even much smalle. Next, we compae ou factional Tikhonov method with the appoach in [7]. We compae (2.9) with paamete α = 0.6 with (2.4) fo γ = 1 2 (α + 1) = 0.8. These values of α and γ yield the same powe of A T A in the ight-hand sides of (1.6) and (2.5), espectively, and ende the filte functions (2.4) and (2.9) the same asymptotic be-

16 16 Michiel E. Hochstenbach, Lotha Reichel Table 6.2 Qualities of Tikhonov, factional Tikhonov, and thei atios fo vaious n = 100 examples fo diffeent eo-levels (1%, 5%, 10%) and α = 0.6. Eo 1% 5% 10% Poblem Tikh Fac Ratio Tikh Fac Ratio Tikh Fac Ratio baat 2.1e-1 1.9e-1 9.1e-1 3.3e-1 3.0e-1 9.0e-1 3.6e-1 3.4e-1 9.4e-1 deiv e-1 2.7e-1 9.6e-1 3.8e-1 3.6e-1 9.3e-1 4.4e-1 4.1e-1 9.3e-1 deiv e-1 2.5e-1 9.5e-1 3.8e-1 3.5e-1 9.2e-1 4.4e-1 4.1e-1 9.2e-1 deiv e-2 4.6e-2 1.2e-0 7.0e-2 6.3e-2 8.9e-1 9.3e-2 8.1e-2 8.7e-1 foxgood 4.3e-2 4.0e-2 9.3e-1 1.4e-1 1.0e-1 7.3e-1 2.0e-1 1.5e-1 7.4e-1 gavity 3.7e-2 2.3e-2 6.4e-1 7.4e-2 5.6e-2 7.6e-1 1.1e-1 9.0e-2 8.2e-1 heat 1.5e-1 1.6e-1 1.1e-0 3.1e-1 3.1e-1 9.9e-1 4.4e-1 4.2e-1 9.5e-1 ilaplace 1.6e-1 1.5e-1 9.2e-1 2.0e-1 1.8e-1 9.2e-1 2.2e-1 2.0e-1 9.4e-1 phillips 2.8e-2 4.8e-2 1.7e-0 6.5e-2 7.3e-2 1.1e-0 1.1e-1 1.0e-1 9.8e-1 shaw 1.5e-1 1.3e-1 8.6e-1 1.8e-1 1.7e-1 9.5e-1 2.0e-1 1.9e-1 9.3e-1 Table 6.3 Qualities of Tikhonov, factional Tikhonov, and thei atios fo vaious n = 100 examples fo diffeent eo-levels (1%, 5%, 10%) and α = 0.4. Eo 1% 5% 10% Poblem Tikh Fac Ratio Tikh Fac Ratio Tikh Fac Ratio baat 2.1e-1 1.7e-1 8.1e-1 3.3e-1 2.7e-1 8.2e-1 3.6e-1 3.2e-1 8.8e-1 deiv e-1 2.8e-1 1.0e-0 3.8e-1 3.6e-1 9.4e-1 4.4e-1 4.0e-1 9.1e-1 deiv e-1 2.6e-1 9.8e-1 3.8e-1 3.4e-1 9.0e-1 4.4e-1 3.9e-1 8.9e-1 deiv e-2 1.1e-1 2.9e-0 7.0e-2 1.3e-1 1.9e-0 9.3e-2 1.4e-1 1.5e-0 foxgood 4.3e-2 4.8e-2 1.1e-0 1.4e-1 8.6e-2 6.3e-1 2.0e-1 1.3e-1 6.2e-1 gavity 3.7e-2 4.3e-2 1.2e-0 7.4e-2 6.6e-2 8.9e-1 1.1e-1 9.3e-2 8.4e-1 heat 1.5e-1 1.9e-1 1.2e-0 3.1e-1 3.3e-1 1.1e-0 4.4e-1 4.3e-1 9.7e-1 ilaplace 1.6e-1 1.4e-1 9.0e-1 2.0e-1 1.8e-1 9.0e-1 2.2e-1 2.0e-1 9.2e-1 phillips 2.8e-2 1.1e-1 3.7e-0 6.5e-2 1.3e-1 2.0e-0 1.1e-1 1.5e-1 1.4e-0 shaw 1.5e-1 1.2e-1 7.9e-1 1.8e-1 1.7e-1 9.3e-1 2.0e-1 1.9e-1 9.2e-1 Table 6.4 Qualities of the Klann Ramlau appoach of [7], factional Tikhonov, and thei atios fo vaious n = 100 examples fo diffeent eo-levels (1%, 5%, 10%) and α = 0.6. Eo 1% 5% 10% Poblem KR Fac Ratio KR Fac Ratio KR Fac Ratio baat 2.1e-1 1.9e-1 9.2e-1 3.2e-1 3.0e-1 9.3e-1 3.5e-1 3.4e-1 9.6e-1 deiv e-1 2.7e-1 9.7e-1 3.8e-1 3.6e-1 9.5e-1 4.4e-1 4.1e-1 9.5e-1 deiv e-1 2.5e-1 9.6e-1 3.7e-1 3.5e-1 9.4e-1 4.3e-1 4.1e-1 9.4e-1 deiv e-2 4.6e-2 1.2e-0 6.8e-2 6.3e-2 9.2e-1 8.8e-2 8.1e-2 9.2e-1 foxgood 4.4e-2 4.0e-2 9.2e-1 1.3e-1 1.0e-1 7.5e-1 1.9e-1 1.5e-1 7.8e-1 gavity 3.4e-2 2.3e-2 6.9e-1 7.0e-2 5.6e-2 8.0e-1 1.1e-1 9.0e-2 8.5e-1 heat 1.6e-1 1.6e-1 1.0e-0 3.2e-1 3.1e-1 9.8e-1 4.4e-1 4.2e-1 9.5e-1 ilaplace 1.6e-1 1.5e-1 9.3e-1 1.9e-1 1.8e-1 9.4e-1 2.1e-1 2.0e-1 9.5e-1 phillips 3.1e-2 4.8e-2 1.6e-0 6.7e-2 7.3e-2 1.1e-0 1.1e-1 1.0e-1 9.7e-1 shaw 1.5e-1 1.3e-1 8.9e-1 1.8e-1 1.7e-1 9.6e-1 2.0e-1 1.9e-1 9.4e-1

17 Factional Tikhonov egulaization baat deiv2-1 deiv deiv2-3 foxgood gavity heat ilaplace phillips shaw Fig. 6.1 Optimal α (vetical axes) fo eo-levels 1%, 2%,..., 10% (hoizontal axes) fo the test poblems of Tables (aveage taken ove 100 andom eo vectos). havio, in tems of the powe of σ, at the oigin. Table 6.4 shows factional Tikhonov to usually ende solutions of highe quality. Expeiment 6.2 We illustate the pefomance of the method of Section 4 fo lage poblems. The same poblems as in the pevious tables ae consideed, but now fo m = n = We poect these lage poblems onto the Kylov space (4.2) of dimension l = 20. This value of l is quite abitay; othe values typically give simila esults. Computed esults ae epoted in Table 6.5. Table 6.5 shows that the factional Tikhonov appoach with α = 0.6 is bette than the standad Tikhonov method in all cases epoted.

18 18 Michiel E. Hochstenbach, Lotha Reichel Table 6.5 Qualities of Tikhonov, factional Tikhonov, and thei atios fo vaious n = 5000 examples poected onto 20-dimensional Lanczos bidiagonalization spaces fo diffeent eo-levels (1%, 5%, 10%) and α = 0.6. Eo 1% 5% 10% Poblem Tikh Fac Ratio Tikh Fac Ratio Tikh Fac Ratio baat 2.1e-1 1.9e-1 9.2e-1 3.3e-1 3.0e-1 9.1e-1 3.7e-1 3.5e-1 9.4e-1 deiv e-1 2.6e-1 9.2e-1 3.8e-1 3.5e-1 9.3e-1 4.4e-1 4.1e-1 9.3e-1 deiv e-1 2.5e-1 9.2e-1 3.7e-1 3.4e-1 9.2e-1 4.2e-1 3.9e-1 9.1e-1 deiv e-2 3.1e-2 7.0e-1 8.8e-2 6.7e-2 7.7e-1 1.1e-1 9.5e-2 8.6e-1 foxgood 4.7e-2 4.1e-2 8.8e-1 1.5e-1 1.2e-1 7.6e-1 2.2e-1 1.8e-1 7.9e-1 gavity 4.2e-2 3.1e-2 7.4e-1 7.7e-2 6.3e-2 8.2e-1 1.1e-1 9.1e-2 8.5e-1 heat 1.1e-1 9.3e-2 8.1e-1 2.6e-1 2.3e-1 9.0e-1 3.6e-1 3.3e-1 9.1e-1 ilaplace 2.5e-1 2.4e-1 9.6e-1 2.8e-1 2.7e-1 9.6e-1 2.9e-1 2.8e-1 9.7e-1 phillips 2.5e-2 1.8e-2 7.0e-1 6.1e-2 4.8e-2 7.8e-1 9.5e-2 8.0e-2 8.4e-1 shaw 1.6e-1 1.3e-1 8.5e-1 1.9e-1 1.8e-1 9.4e-1 2.3e-1 2.1e-1 9.3e-1 7 Conclusions We have studied a family of factional Tikhonov egulaization methods which depend on a paamete α > 0. Standad Tikhonov egulaization is obtained fo α = 1. We have shown how the solution depends on α and, in paticula, investigated how the choice of α affects solutions that satisfy the discepancy pinciple. The nom of these solution has a local minimum fo α = 1. Analogously, if the computed solution is equied to be of specified nom, the nom of the esidual eo has a local minimum fo α = 1. This indicates that the choice α = 1 is quite natual. Howeve, it is known that standad Tikhonov gives ove-smoothed solutions and we popose to emedy this by choosing α < 1. Extensive numeical expeiments suggest that letting α be smalle than, but close to unity, such as α = 0.8, gives bette esults than α = 1 fo almost all examples. Smalle values of α, such as α = 0.5, may wok even bette fo some examples, at the cost of endeing wose esults fo othes. A geneal ule-ofthumb is that the lage the eo-level, the moe advantageous it is to let α < 1. Ou computed examples illustate the pefomance of the method in conunction with the discepancy pinciple. We emak that factional Tikhonov can be applied with othe selection ules fo the egulaization paamete as well. Finally, the techniques also can be used fo lage-scale poblems by fist poecting them onto low-dimensional Kylov spaces. Acknowledgements. LR would like to thank MH fo an enoyable visit to TU/e duing which wok on this pape was caied out. The authos thank the efeee fo useful suggestions. Refeences 1. Å. Böck, Numeical Methods fo Least Squaes Poblems, SIAM, Philadelphia, D. Calvetti and L. Reichel, Lanczos-based exponential filteing fo discete ill-posed poblems, Nume. Algoithms, 29 (2002), pp

19 Factional Tikhonov egulaization D. Calvetti and L. Reichel, Tikhonov egulaization with a solution constaint, SIAM J. Sci. Comput., 26 (2004), pp G. H. Golub and C. F. Van Loan, Matix Computations, 3d ed., Johns Hopkins Univesity Pess, Baltimoe, P. C. Hansen, Rank-Deficient and Discete Ill-Posed Poblems, SIAM, Philadelphia, P. C. Hansen, Regulaization tools vesion 4.0 fo Matlab 7.3, Nume. Algoithms, 46 (2007), pp E. Klann and R. Ramlau, Regulaization by factional filte methods and data smoothing, Invese Poblems, 24 (2008), J. Lampe, M. Roas, D. Soensen, and H. Voss, Acceleating the LSTRS algoithm, Beicht 138, Institute of Numeical Simulation, Hambug Univesity of Technology, Hambug, Gemany, July V. A. Moozov, Methods fo Solving Incoectly Posed Poblems, Spinge-Velag, New Yok, M. Roas and D. C. Soensen, A tust-egion appoach to egulaization of lage-scale discete foms of ill-posed poblems, SIAM J. Sci. Comput., 23 (2002), pp