An efficient method to set up a Lanczos based preconditioner for discrete ill-posed problems

Transcription

1 Appled Mathematcal Appled Mathematcal Modellng (214) 1 16 Modellng An effcent method to set up a Lanczos based precondtoner for dscrete ll-posed problems Shervan Erfan a, Al Tavakol a,, Davod Khojasteh Salkuyeh b a Mathematcs Department, Val-e-Asr Unversty of Rafsanjan, Iran. b Faculty of Mathematcal Scences, Unversty of Gulan, P.O. Box 1914, Rasht, Iran Abstract Rezgh and Hossen [M. Rezgh and S. M. Hossen, Computng 88(21), 79-96] presented a Lanczos based precondtoner for dscrete ll-posed problems. Ther precondtoner s constructed by usng few steps (e.g., k) of the Lanczos bdagonalzaton and correspondng computed sngular values and rght Lanczos vectors. In ths artcle, we propose an effcent method to set up such precondtoner. Some numercal examples are gven to show the effectveness of the method. c 211 Publshed by Elsever Ltd. Keywords: Lanczos bdagonalzaton, Ill-posed problems, Precontoner, Regularzaton, Sngular value decomposton. 2 MSC: 35J65, 58J9 1. Introducton We consder large scale dscrete ll-posed problems,.e., lnear systems of equatons of the from Ax = b, A R n n, x, b R n. (1) The coeffcent matrx A s of full-rank and typcally ll-condtoned, the rght-hand sde vector b s perturbed by an error such that b = ˆb + e, whch may occur by measurement or dscretzaton errors. Here e and ˆb denote the Gaussan nose and unknown error-free rght-hand sde vectors, respectvely. These problems typcally arse, for nstance, n dscretzng lnear ll-posed problems, such as Fredholm ntegral equaton of the frst knd wth a smooth kernel. Also n these problems the two followng crtera are satsfed: (1) The sngular values of A decay gradually to zero wth no partcular gap n the spectrum. (2) The rato between the largest and the smallest nonzero sngular values s large [7]. Correspondng author Emal addresses: sh.erfan@stu.vru.ac.r (Shervan Erfan), tavakol@mal.vru.ac.r (Al Tavakol), khojasteh@gualn.ac.r, salkuyeh@gmal.com (Davod Khojasteh Salkuyeh) URL: tavakol.vru.ac.r (Al Tavakol) 1

2 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) We would lke to obtan an approxmate soluton ˆx of the lnear system Aˆx = ˆb wth error-free unknown rghthand sde. Snce the matrx A s severely ll-condtoned, the soluton of system (1) typcally s not a sgnfcant approxmaton of ˆx even f the nose e s small. So, we use the regularzaton methods to determne a soluton that approxmates the exact soluton ˆx. The conjugate gradent method appled to normal equatons (CGLS) s a well-known teratve method for solvng ll-posed problems. Snce, teratve methods, such as CGLS, have low rate of convergence, t s possble to speed up the convergence by a sutable precondtoner [15]. All regularzaton methods use one or more regularzaton parameters specfc to the regularzaton method that controls the amount of the stablzaton mposed on the soluton, and n most cases t s necessary to choose ths parameter from gven the problem and the gven set of data. A large regularzaton parameter makes a new wellcondtoned problem, but ts soluton may be far from the exact soluton. A small regularzaton parameter generally yelds a soluton very close to the nose-contamnated exact soluton of (1), and hence ts dstance from the nosefree soluton also can be large. Thus, one can choose a regularzaton parameter to balance the error due to nose wth the error due to regularzaton. A good choce of regularzaton parameter s clearly crucal to determne useful approxmate soluton for ll-posed problems [12]. In [15], Rezgh and Hossen proposed a new precondtoner for dscrete ll-posed problems whch s produced by a few steps of Lanczos bdagonalzaton process. The computaton of ther precondtoner depends on the large sngular values of A. In ths paper, we propose a new and effcent numercal method to estmate the large sngular values of the matrx A and construct the new form of Lanczos based precondtoner, whch can be obtaned n a very small number of operatons. The paper s organzed as follows. In Secton 2, we study Fredholm ntegral equaton of the frst knd and ntroduce the quadrature rule and Galerkn method for dscretzaton of these equatons. Also, we study the treatment of the sngular values on the matrx resulted by dscretzaton. In Secton 3, we defne Lanczos bdagonalzaton for constructng Least squares va Lanczos bdagonalzaton (LSQR) method and precondtoner to approxmate the soluton of the ll-posed problems. In Secton 4, we ntroduce a new method for constructng the new form of Lanczos based precondtoner and fndng regularzaton parameter based on large sngular values of the sngular spectrum. Fnally, n Secton 5 some numercal experments are gven. 2. Fredholm Integral Equaton of the Frst Knd 2.1. The Sngular Value Expanson A classcal example n lnear ll-posed problems s a Fredholm ntegral equaton of the frst knd wth a square ntegrable kernel b a K(s, t)f(t)dt = g(s), c s d, (2) where the rght-hand sde g and kernel K are known functons and f s an unknown soluton. Fredholm equaton of the frst knd can be classfed nto two categores. The frst one arses f the kernel functon K(s, t) s smooth. In ths case, Fredholm equaton s very often extremely ll-condtoned,.e., small changes n the data cause huge changes n the unknown functon f. That means the soluton f s extremely senstve to small changes n g(s) and so specal numercal technques called regularzaton methods are requred. In the second category, the kernel k(s, t) s a sngular functon [8]. The superor analytcal tool for analyss of the frst-knd Fredholm ntegral equaton (2) wth square ntegrable kernels s the sngular value expanson (SVE) of the kernel. A kernel K s called square ntegrable f the norm K 2 = b d a c K(s, t) 2 dsdt, s bounded. By means of the SVE, any square ntegrable kernel K can be wrtten as the followng nfnte sum [7]: K(s, t) = µ u (s)v (t). =1 2

3 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) The functons u and v are sad to be the sngular functons of K. They are orthonormal wth respect to the usual nner product,.e., { 1, f = j, (u, u j ) = (v, v j ) =, f j, where (.,.) s defned by (φ, ψ) = b a φ(t)ψ(t)dt. The numbers µ s are the sngular values of K, whch are nonnegatve and can always be ordered n non-ncreasng order so that µ 1 µ 2 µ 3. And, f there exsts a postve real number α such that the sngular values satsfy µ = O( α ), then α s called the degree of ll-posedness, and problems s characterzed as mldly or moderately ll-posed f α 1 or α > 1, respectvely. On the other hand, f µ = O(e α ), then the problem s termed severely ll-posed (see [7]). By the defnton of K, we have K 2 = = b d a c =1 b d µ u (s)v (t) µ j u j (s)v j (t)dsdt a c =1 j=1 j=1 µ µ j u (s)u j (s)v (t)v j (t)dsdt. (3) So, we have K 2 = µ 2 <. =1 Snce the sequence {µ } =1 s nonnegatve and r =1 µ2 for all r s bounded, then =1 µ2 s convergent. Also, we have =1 α s convergent f α > 1 and dvergent f α 1. Ths shows that the µ 2 must decay faster than 1. In general, f K s a contnuous partal dervatves of order p then µ s approxmately O( p 1 2 ) [11, 17]. So, we conclude that f the kernel K s smoother, then the sngular values µ s decay to zero rapdly. Ths yelds an ncrease n the number of small sngular values Dscretzaton In order to solve Fredholm ntegral equaton of the frst knd we have to dscretze t. There are essentally two man classes of methods, namely, quadrature and Galerkn methods. In the quadrature method, a quadrature rule lke the mdpont rule, the trapezodal rule and the Smpson rule, s used. The dscretzed form of (2) reads n ω j K(s, t j )f(t j ) = g(s ), = 1, 2,..., n, j=1 wth the weghts {ω j } n j=1 and the nodes {t } n =1. So, Eq. (2) s approxmated by the n n lnear system of equatons Ax = b, where A s the n n matrx wth the entres a j = ω j K(s, t j ), x s the n-vector x = [f(t 1 ), f(t 2 ),..., f(t n )] T, and b s the n-vector b = [g(s 1 ), g(s 2 ),..., g(s n )] T. 3

4 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) In the Galerkn method, one can choose two dfferent orthonormal bass functons, {ψ } =1,...,n and {ϕ j } j=1,...,n, to expand the functons g and f respectvely, by Thus, so that, g(s) = g = n g ψ (s) and f(t) = =1 g(s) = n f j ϕ j (t). j=1 n b f j K(s, t)ϕ j (t)dt, j=1 n b f j j=1 a d The system may be wrtten n the matrx form as Ax = b, where a j = b d wth b = [g 1, g 2,..., g n ] T and x = [f 1, f 2,..., f n ] T. a c c a K(s, t)ψ (s)ϕ j (t)dsdt. K(s, t)ψ (s)ϕ j (t)dsdt, 2.3. The Sngular Value Decomposton Let A R n n. Then, the SVD of A s a decomposton of the from A = UΣV T = n u v T, where U = (u 1, u 2,..., u n ) and V = (v 1, v 2,..., v n ) are untary matrces,.e., U T U = V T V = I n, and where Σ = dag(σ 1, σ 2,..., σ n ) has non-negatve dagonal entres appearng n non-ncreasng order such that =1 σ 1 σ 2 σ n. The numbers, = 1,..., n are the sngular values of A. In contnuaton, we descrbe that an ncrease n the dmensons of A wll only ncrease the number of small sngular values. To ths end, let µ 1, µ 2,..., µ k be the large sngular values of the kernel K. As we sad n the last subsecton, f K s a smooth functon, there exsts a few large sngular values µ that yeld k n. We have (see [1]): σ 1 µ 1 σ 2 µ 2... σ k µ k. On the other hand, snce µ s are fxed, then the large sngular values would be unchanged whenever the dmenson of the matrx A s ncreased. In connecton wth dscrete ll-posed problems, two characterstc features of SVD are very often found (1) The sngular value decay gradually to zero wth no partcular gap n the spectrum. An ncrease of the dmensons of A wll ncrease the number of small sngular values. (2) The left and rght sngular vectors u and v tend to have more sgn changes n ther elements as the ndex ncreases (.e., as s decreases). Both features are consequences of the fact that the SVD of A s closely related to SVE of the underlyng kernel K. In fact, n the sngular values of A denote to many cases approxmatons of the sngular values µ of K[2]. Let us consder, for example, the test problem derve2(1,1) n [8]. The kernel K of the ntegral equaton (2) s Green s functon for the second dervatve: { s(t 1), s t, K(s, t) = t(s 1), s t, 4

5 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) µ Fgure 1. A comparson between and µ for derve test problem. and both ntegraton ntervals are [, 1]. The sngular values and functons are gven by (see for example [2]) µ = (π) 2, u (s) = ± 2 sn(πs), = 1, 2,..., v (t) = 2 sn(πt). (4) Snce the sngular values are proportonal to 2, The problem s moderately ll-posed. Fgure 1 shows that the sngular values of dscretzed matrx of the test problem derv2(1,1) wth dmenson 1. It s clear from ths fgure that the sngular values of matrx A and the correspondng µ of kernel K are almost the same. Snce A s a full rank matrx, then ts nverse s gven by A 1 = n =1 v σ 1 u T, and therefore the soluton of Ax = b s x = A 1 b = n =1 u T b v. (5) Moreover, snce there are dscretzaton errors or lnear approxmaton errors, the dscrete ll-posed problems always contan perturbatons and error components created n tenson all sngular vectors of A. So, we cannot compute a stablzed soluton. By (5), the errors n the vector b are amplfed wth coeffcent σ 1. If s close to zero then the soluton s affected by the error e n the vector b, b = ˆb + e, ˆb e. In ths case, we have x = V Σ 1 U T b = V Σ 1 U Tˆb + V Σ 1 U T e = ˆx + x e, where ˆx = V Σ 1 U Tˆb and x e = V Σ 1 U T e. If the sngular values decay gradually to zero, then the term x e overcomes the exact soluton. Therefore, Fourer coeffcents u T b correspondng to small sngular values wth lower rate tend to zero. Hence, the terms correspondng to small sngular values overshadow the soluton. We should use the regularzaton methods to subtract or elmnate the soluton correspondng to small sngular values. 5

6 3. Lanczos Bdagonalzaton S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) For a gven matrx A R n n, the Lanczos Bdagonalzaton algorthm generates two orthogonal matrces U n and V n such that U T n AV n = B n, where B n s a real lower bdagonal matrx. Ths algorthm s descrbed as follows: Algorthm 1. Lanczos Bdagonalzaton 1. Let β 1 := b 2, u 1 := b/β 1 and v := 2. For = 1 to n do 3. p := A T u β v 1 4. α := p 2 5. v := p /α 6. q := Av α u 7. β +1 := q 2 8. u +1 := q /β Endfor The vectors u and v produced by Lanczos Bdagonalzaton algorthm are called the left and rght Lanczos vectors, respectvely. Denotng U k = [u 1, u 2,..., u k ] and V k = [v 1, v 2,..., v k ], the followng relatons can be establshed: (1) b = β 1 u 1 = β 1 U k+1 e 1, (2) AV k = U k+1 B k, (3) A T U k+1 = V k Bk T + α k+1v k+1 e T k+1, (4) Vk T V k = I k, (5) Uk+1 T U k+1 = I k+1, where I j denotes the dentty matrx of order j, e s the th unt vector and Now, suppose we want to solve B k = α 1 β 2 α 2 β αk β k+1 mn b Ax 2, x S where S denotes the k-dmensonal subspace spanned by the frst k Lanczos vectors v, = 1,..., n. The soluton whch we seek s of the from x (k) = V k y (k) for some vectors y (k) R k. Let r (k) = b Ax (k) be the correspondng resdual. From the above relatons, we get Snce the columns of U k+1 are orthonormal, we have In the step k of the LSQR algorthm we are gong to solve. r (k) = β 1 u 1 AV k y (k) = U k+1 (β 1 e 1 B k y (k) ). r (k) 2 = β 1 e 1 B k y (k) 2. mn β 1 e 1 B k y (k) 2. y (k) 6

7 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) Now, we wrte the exact SVD of the computed B k as k B k = H k ΓQ T k = h γ q T. =1 Then we obtan where k s typcally small. x (k) = β 1 V k k =1 h 1 γ q, (6) The best value of k can be computed by one of the followng four ways: (1) In [15] the parameter k s consdered as the smallest nteger for whch σ k < τσ 1, (7) where τ s the square root of the machne precson. Here, s are the sngular values of B k [15]. (2) Generalzed Cross-Valdaton (GCV) The parameter k s chosen by GCV method that mnmzes the followng functon (see [12]): ψ k = β 1e 1 B k y (k) 2 2 (n k) 2 (3) L-Curve To determne the L-curve assocated wth LSQR, estmates of r (k) and x (k) are needed for several values of k. In the method, the corner of the L-curve gves a good balance of the soluton sze and resdual sze, so the parameter k can be chosen by the corner ths curve (see [12]). (4) Dscrete Pcard Condton A standard tool for analyzng the dscrete ll-posed problems s the dscrete Pcard plot, whch s a plot of the quanttes, u T b and ut b that arse n (5). In order to derve a meanngful regularzaton soluton n a dscrete ll-posed problem, t must satsfes the dscrete pcard condton,.e the Fourer coeffcent u T b on the average should decay to zero faster than the sngular values [8]. In ths paper, we use the dscrete Pcard condton to fnd the parameter k. Fgure 2 shows plots of the frst 2 sngular values, Fourer coeffcents u T b, and coeffcents ut b soluton for perturbed problem shaw [8] wth Gaussan nose n whch e ˆb = and e ˆb of the = As we observe, one can choose k = 8 and k = 7 for shaw problem wth e = ˆb and e = ˆb 1.1 1, respectvely. We moreover see that the dfferent perturbatons contan a lttle change n the value of parameter k. In [15], a new regularzed precondtoner obtaned by k steps of Lanczos bdagonalzaton for dscrete ll-posed problems has been ntroduced. The constructon of ths precondtoner s not based on any partcular structure of the matrces. The matrx M = V k (B T k B k ) 1 V T k + (I V k V T k ) R n n, (8) clusters approxmately the large sngular values around 1 and leaves the others unchanged.the dea here s based on the fact that the precondtoner s constructed by k n steps of Lanczos bdagonalzaton. 7

8 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) Pcard plot(a) 1 2 Pcard plot(b) 1 15 u T b u T b /σ 1 15 u T b u T b /σ k=8 1 5 k= Fgure 2. The frst 2 sngular values, Fourer coeffcents u T b, and coeffcents ut b for the shaw test problem wth two perturbatons e = ˆb (a) and e = ˆb (b ). 4. An effectve method for computng optmal Lanczos based precondtoner We know, the matrx A can be produced by dscretzaton of Fredholm ntegral equaton of the frst knd. Solvng these equatons by methods lke Galerkn or quadrature rules usually produce an ll-posed system Ax = b that yelds a huge condton number of A. The soluton of the system Ax = b converges well, f the dmensons of A are suffcently large (see for nstance [3]). However, n ths case, the sngular values of A generated by SVD are not exact, and hence make the selecton of parameter k dffcult. In the sequel, we ntroduce a crteron for choosng a sutable regularzaton parameter k. To do so, we dscretze the Fredholm ntegral equaton on a coarse grd that produces the system Ā x = b. Then the matrx Ā resulted from ths knd of dscretzaton has less dmensons than those of A (e.g, half of A) and ts sngular values can be absolutely computed more easly than the orgnal matrx. On the other hand, as we mentoned n Secton 2, an ncrease n the dmensons of A wll ncrease the number of small sngular values. Thus, the number of large sngular values (.e., k) for A and Ā are almost the same. Moreover the number k (for the matrx Ā) can be derved va the methods lsted n Secton 3. All we need s to compute the rght Lanczos vectors V k. To do ths, we frst note that such vectors are avalable for Ā. Let v = ( v1, v 2,..., v m ) T be a rght Lanczos vector correspondng to a coarse grd. We use an nterpolaton (prolongaton) operator I k as follows: I k v = w, where w 2j 1 := v j and w 2j := 1 2 ( vj + v j+1 ), (9) for j = 1, 2,..., m. In fact, the values of coarse grd ponts are mapped unchanged to the fne grd and the values of the fne grd ponts whch are not on the coarse grd, are equal to the average of the pont values n ther neghborhood of coarse grd. In ths way, we obtan a vector of the form w = (w 1, w 2,..., w 2m 1 ) T whch s correspondng to a fne grd. It should be noted that one may apply the operator I k several tmes (e.g., r) to obtan the rght Lanczos vector for the matrx A. The followng algorthm descrbes the procedure clearly: 8

9 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) for n=2 for n= Fgure 3. A comparson between sngular values of shaw test problem for dfferent dmensons of A. Algorthm 2. Computng rght Lanczos vectors of A from v 1. Input A = [a 1,, a n ], Ā = [ā 1,, ā m ], b R n, b R m, v R m and r N. 2. For s = 1 to r do 3. m := length(v) 4. w := zeros(2m 1, 1) 5. j := 1 6. For = 1 to m do 7. w j := v 8. If m, then 9. w j+1 = (v + v +1 )/2 1. j = j Endf 12. Endfor 13. v := w 14. EndFor 15. q := If ā T b q and a T q b, then 17. v q1 = āt b q, ĀT b 18. v q1 = at q b A T b, 19. α = vq1 v q1, 2. Else 21. q := q Go to Lne EndIf 24. Return αw R 2r (m 1)+1 Remark 1: We note that 2 r (m 1) + 1 = n and so m = (n 1)2 r + 1. (1) Hence, n dscretzaton of ll-posed problems lke Fredholm equaton of the frst knd, one can choose n such that (n 1)2 r + 1 s a natural number for some small r (e.g r < 1). 9

10 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) v 1 R 2 v 1 R 25 v 3 R 2 v 3 R 25 v 5 R 2 v 3 R v 2 R 2 v 2 R 25 v 4 R 2 v 4 R 25 2 v R 6 v 6 R Fgure 4. A comparson between rght Lanczos vectors of shaw test problem for dfferent dmensons of A. Convergence of Algorthm 2 In order to show the convergence of Algorthm 2, we take smply for = 1, 2,..., r, the vectors u and v to be the exact and approxmated rght Lanczos vectors of the dscretzed system on the -th coarse grd, respectvely. In addton, let I +1 be the lnear nterpolaton operator such that t takes vectors v +1 n the ( + 1)-th coarse grd and produces vectors v n the -th fne-grd accordng to the rule I +1 v +1 = v. Now, we assume that whch ɛ denotes the accuracy of the approxmaton. We have: u I +1u +1 ɛ, (11) v = I +1I Ir 1 r v r, = 1, 2,..., r. (12) On the other hand, because of the boundedness of the nterpolaton operator, there exsts a constant α such that for any vectors v and w n the -th grd, I +1v +1 I +1w +1 = I +1(v +1 w +1 ) α v +1 w +1, (13) holds. Then, by (11), (12) and (13), we have: Smlarly, we have: Hence, u 1 v 1 = u 1 I 1 2 v 2 u 1 I 1 2 u 2 + I 1 2 u 2 I 1 2 v 2 ɛ 1 + α 1 u 2 v 2. u v ɛ + α u +1 v +1, = 1, 2,..., r 1. u 1 v 1 ɛ 1 + α 1 (ɛ 2 + α 2 (...(ɛ r 1 + α r 1 u r v r )...)). For the coarsest level r, we assume that u r v r ɛ r. So, ɛ = ɛ 1 + α 1 (ɛ 2 + α 2 (...(ɛ r 1 + α r 1 ɛ r )...)) s an upper bound for u 1 v 1. Now, f ɛ for = 1, 2,..., r, then u 1 v 1 and convergence s guaranteed. 1

11 Remark 2: We note that S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) u 1 v 1 = u 1 I 1 2 I I r 1 r v r, (14) where v r = v. Then, by (14) we can measure the dfference between the exact rght Lanczos vector u 1 and that of approxmated vector obtaned usng v n Algorthm 2 for dfferent values of m. Algorthm 3. Left-Precondtoned CGLS M MG A T Ax = M MG A T b 1. Let r () = b Ax (), p () = s () = M MG (A T r () ), γ = s () For j = o untl convergence do 3. t (j) = M MG p (j), 4. q (j) = At (j), 5. α (j) = γj, q (j) x (j+1) = x (j) + α j t (j), 7. r (j+1) = r (j) α j q (j), 8. s (j+1) = M MG (A T r (j+1) ), 9. γ j+1 = s (j+1) 2 2, 1. β j = γj+1 γ j, 11. p (j+1) = s (j+1) + β k p (j), 12. Endfor Let V k = [ v 1,, v k ] wth v, = 1,, k, as the rght Lanczos vectors for the coarsest grd. We defne Ik r V k = [Ik r v 1,, Ik r v k], where Ik r s the prolongaton operator and t s appled r-tmes n Algorthm 2. In fact, by Algorthm 2, Ik r v = w. Now, the new form of the precondtoner can be defned as M MG = α 2 (I r k V k )(B T k B k ) 1 (I r k V k ) T + (I α 2 (I r k V k )(I r k V k ) T ). We call ths new precondtoner by subscrpt M G, because ths precondtoner s based on an dea of prolongaton and restrcton operators n Multgrd algorthms. The constructon of regularzed nverse precondtoners M and M MG need O(kn 2 ) and O(km 2 + rm) operatons, respectvely. We know, k and r are very small n comparson wth m and n. On the other hand, by (1), the value of m s much smaller than n. Therefore, computatonal cost of the new form Lanczos based precondtoner M MG for large n s more less than that of the precondtoner M. Ths precondtoner can be used as a left or rght precondtoner wth the CGLS method [14, 15]. Algorthm 3 shows the scheme of a left precondtoned CGLS method. In fact, CGLS s a semconvergent method: For some j, n the frst j teratons, the method converges to the exact soluton, and then suddenly starts to dverge and the nose begns to enter the soluton. Methods for fndng the optmal value of such j (e.g. L-curve) can be found n [7, 9]. To llustrate the effcency of the proposed method, we consder the shaw test problem for two cases of dmensons 25 and 2. Fgure 3 shows the sngular values obtaned by SVD from the dscretzed matrx of shaw problem, wth the dmensons of 2 and 25 for the matrces A and Ā, respectvely. As we observe, the value and number of large sngular values of A and Ā are almost the same. Also, Fgure 4 shows a comparson between rght Lanczos vectors of shaw problem for dfferent dmensons of A wth the modfer coeffcent α = v11 v Numercal results In ths secton, we present some numercal examples. All experments were carred out usng MATLAB and Hansen Regularzaton [8]. We nvestgate the performance of our proposed regularzaton parameter (k) for ntroduced precondtoner and LSQR method. In the followng examples, taken from [8] the contamnated vector b and the Gaussan nose e are consdered as b = ˆb + e, where e = 1 3 ω ˆb. ω, 11

12 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) Exact Lanczos based precondtoner wth k=4 No precondtoner Fgure 5. A comparson between the exact soluton and the soluton obtaned by precondtoned CGLS method at step 2 and the selecton k = 4 of bdagonalzaton Lanczos for derve2(2,3) problem. and ω s Gaussan random vector wth mean and standard devaton 1. In all the followng examples, we consder the coarse grd wth dmensons On ths coarse grd, the parameter k s obtaned by (7). Moreover, by usng the nterpolaton operators, the ngredents of the precondtoner M,.e. V k, σ k and B k are obtaned for fne grd. Example 5.1 Consder the ntegral equaton where 1 K(s, t)f(t)dt = g(s), s 1, K(s, t) = { s(t 1), s < t, t(s 1), s t, f(t) = g(s) = { t, t < 1 2, 1 t, t 1 2, (4s 3 3s) 24, s < 1 2, ( 4s 3 +12s 2 9s+1) 24, s 1 2. Ths s a mldly ll-posed problem;.e., ts sngular values decay slowly to zero. We dscretze ths ntegral equaton, usng the functon derve2(2,3) (see [8]), to obtan the matrx A R 2 2. We apply the ntroduced methods n Secton 4 for selected sutable regularzaton parameter on ths coarse grd. In ths problem, we have only used k = 4 steps of Lanczos bdagonalzaton for constructng the precondtoner. Fgure 5 shows the soluton of Lanczos precondtoned obtaned for k = 4 steps of the b-dagonalzaton Lanczos and unprecondtoned systems after two teraton steps. Also, Fgure 6 shows that the relatve error for precondtoned lnear systems wth selecton k = 4 s better than that of the orgnal system. Example 5.2 The nverse heat equaton used here s a Volterra ntegral equaton of the frst knd wth [, 1] as ntegraton nterval. The kernel s K(s, t) = K(s t) wth k(t) = t 3 2 2kappa π exp( 1 4kappa 2 t ). 12

13 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) No precondtoner Lanczos based precondtoner wth k= Fgure 6. Relatve errors versus teraton steps for A T Ax = A T b of derve2(2,3) problem 1.2 Exact Lanczos based precondtoner wth k=8 No precondtoner Fgure 7. A comparson between the exact soluton and the soluton obtaned by precondtoned CGLS method wth 4 teratons and the selecton k = 8 of Lanczos bdagonalzaton process for A T Ax = A T b of heat problem. Here, the parameter kappa controls the ll-condtonng of the matrx A: kappa =5 gves a well-condtoned matrx, kappa =1 gves an ll-condtoned matrx. The default s kappa=1. The ntegral equaton s dscretzed by means of smple collocaton and the mdpont rule wth n grd ponts (see [5, 6]). An exact soluton x s constructed, and then the rght-hand sde b s produced as b = Ax. We use the code heat(n,kappa) from [8] to dscretze ths ntegral to obtan the matrx A R 2 2. In ths problem, we used the regularzaton parameter k = 8. Fgure 7 compares the soluton obtaned of precondtoned system at 4 teratons wth k = 8 steps of Lanczos bdagonalzaton and wthout precondtoner. Fgure 8 shows the relatve error wth and wthout precondtoner. Example 5.3 We dscretze a Fredholm ntegral equaton of the frst knd (2) wth [ π 2, π 2 ] as both ntegraton nterval. The kernel K and soluton f are gven by K(s, t) = (cos s + cos t) 2 ( sn u u )2, u = π(sn s + sn t), f(t) = a 1 exp( c 1 (t t 1 ) 2 ) + a 2 exp( c 2 (t t 2 ) 2 ). 13

14 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) No precondtoner Lanczos based precondtoner wth k= Fgure 8. Relatve errors versus teraton steps, for A T Ax = A T b of heat problem. 2.5 Exact Lanczos based precondtoner wth k=7 No precondtoner Fgure 9. A comparson between the exact soluton and the soluton obtaned by precondtoned CGLS method wth 3 teratons and the selecton k = 7 of bdagonalzaton Lanczos for A T Ax = A T b of shaw problem. The parameters a 1, a 2, c 1, c 2, t 1 and t 2 are constants that determne the shape of the soluton f. In ths mplementaton, we use a 1 = 2, a 2 = 1, c 1 = 6, c 2 = 2, t 1 =.8 and t 2 =.5. The kernel and the soluton are dscretzed by smple collocaton wth n ponts to produce A and x. Then the dscrete rght-hand sde s produced by b = Ax. We used the code shaw from [7] to obtan A R 2 2. In Fgure 9, we plotted the soluton obtaned by Lanczos based precondtoner wth k = 7 at step 2 and unprecondtoned systems. Fgure 1 shows, the relatve errors from teraton steps of CGLS method for precondtoned systems wth k = 7 and orgnal system. Example 5.4 Defne the functon { 1 + cos πx φ(x) = 3, x < 3,, x 3. Then kernel K, the soluton f, and the rght-hand sde g are gven by K(s, t) = φ(s t), f(t) = φ(t), g(s) = (6 s )( πx cos 3 ) + 9 π s sn 2π 3.

15 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) No precondtoner Lanczos based precondtoner wth k= Fgure 1. Relatve errors versus teraton steps, for A T Ax = A T b of shaw problem..16 Exact LSQR method Fgure 11. A comparson between the exact soluton and the soluton obtaned by LSQR method at k = 6 for mn b Ax of phllps problem. Both ntegraton ntervals are [ 6, 6]. We dscretze ths problem, usng the functon phllps(2)[12], to obtan the matrx A R 2 2. In the example, we used (6) to compute the approxmated soluton. Fgure 11, dsplays an exact soluton and the soluton obtaned by LSQR method for k = 6 steps of Lanczos bdagonalzaton. 6. Concluson We have studed a recently proposed precondtoner for ll-posed lnear system of equatons whch s based on the Lanczos bdagonalzaton method. Then, we have presented an effcent method to produced such precondtoner. Fnally, some numercal results have been presented to show the effectveness of the proposed method. References [1] R.C. Allen, W.R. Boland, V. Faber, G.M. Wng, Sngular values and condton numbers of Galerkn matrces arsng from lnear ntegral equatons of the frst knd, J. Math. Anal. Appl. 19(1985) [2] A. Bjorck, Numercal methods for least square problems, SIAM, Phladelpha, [3] S.C. Brenner, L.R. Scott, The mathematcal theory of fnte element methods, 3rd edton, Sprnger, 28. [4] D. Calvett, G. Golub, L. Rechel, Estmaton of the L-curve va Lanczos bdagonalzaton, BIT. 39(1999) [5] A.S. Carasso, Determnng surface temperatures from nteror observatons, SIAM J. Appl. Math. 42(1982) [6] L. Elden, The numercal soluton of a non-characterstc Cauchy problem for a parabolc equaton, n: Deuflhart, P., Harer, E. (Eds.) Numercal Treatment of Inverse Problems n Dfferental and Integral Equatons, Brkhauser, Boston,

16 S. Erfan, A. Tavakol and D. K. Salkuyeh / Appled Mathematcal Modellng (214) [7] P.C. Hansen, Rank-defcent and dscrete ll-posed problems, SIAM, Phladelpha, [8] P.C. Hansen, Regularzaton tools: a MATLAB package for analysss and soluton of dscrete ll-posed problems, Numer. Alg. 6(1994) [9] P.C. Hansen, T.K. Jensen, G. Rodrguez, An adaptve prunng algorthm for the dscrete L-curve crteron, J. Comput. Appl. Math. 198(27) [1] P.C. Hansen, Computaton of the Sngular Value Expanson, Computng 4(1998) [11] F.R. de Hoog, Revew of Fredholm equatons of the frst knd, The applcaton and numercal soluton of ntegral equatons.sjthoff and Noordhoff, pp Leyden, Netherlands, 198. [12] M.E. Klmer, D.P. Oleary, Choosng regularzaton parameter n teratve methods for ll-posed problems, SIAM J. Matr. Anal. Appl. 22(21) [13] A. Kleefeld, Numercal results of lnear Freadholm ntegral equatons of the frst knd over surfaces, Unversty of Wsconsn-Mlwaukee, 27. [14] D.P. Oleary, J.A. Smmons, A bdagonalzaton-regularzaton procedure for large scale dscretzatons of ll-posed problems, SIAM J. Sc. and Stat. Comput. 2(1981) [15] M. Rezagh, S.M. Hossen, Lanczos based precondtoner for dscrete ll-posed problems. Computng. 88(21) [16] Y. Saad, Iteratve Methods for Sparse lnear Systems, PWS press, New York, [17] F. Smthes, The egenvalues and sngular values of ntegral equatons, Proc. London Math. Soc. 43(1937),