On an Orthogonal Method of Finding Approximate Solutions of Ill-Conditioned Algebraic Systems and Parallel Computation

Transcription

1 Proceedings of the World Congress on Engineering 03 Vol I, WCE 03, July 3-5, 03, London, UK On an Orthogonal Method of Finding Approximate Solutions of Ill-Conditioned Algebraic Systems and Parallel Computation M Otelbaev, B I Tuleuov, D Zhussupova Abstract A new method of finding approximate solutions of linear algebraic systems with ill-conditioned or singular matrices, using Schmidt orthogonalization, is presented This method can be effectively used for arranging parallel computations for matrices of large size Index Terms ill conditioned matrices, eigenvalues, approximate solutions, parallel computation, Schmidt orthogonalization This work is to continue [], where we have considered an equation Ax = f, ) here A is a quadratic matrix of order n and f is n- dimensional vector In in the paper [] we solved a problem of parallelization of the problem ) solving process and constructed sufficiently effective parallel algorithm for matrix A with bounded inverse In this paper we suggest a method for finding approximate solutions and parallel computation of the problem ), when matrix A is noninvertible or ill-conditioned Efficiency increase problem of solving large system of linear equations depends on development of high-effective calculating technics Today multiprocessor systems and supercomputers is highly developed Distribution of calculations into parallel branches implies the increase of solving general problem Parallel computation of linear algebraic problems have been considered, for example, in monographs [-4], and program implementation issues in [5] The difference of an offered method from the known consists that existence of zero eigenvalues of a matrix A doesn t influence in any way efficiency of iterative process Only small but nonzero eigenvalues of A A are important Besides, estimates obtained by us in the theorem 3 for the solution doesn t depend on small and nonzero eigenvalues of a matrix A A Also we will notice that the iterative formula 3), as far as we know, in computing practice wasn t applied earlier We denote by A adoint matrix of A Nonnegative square roots of eigenvalues of nonnegative matrix A A we denote by s A) = s, =,, ) and numerate them in non-increasing order taking into account their multiplisities Manuscript received March 3, 03 M Otelbaev is with the Department of Mechanics and Mathematics, L N Gumilyov Eurasian National University, Astana, Kazakhstan otelbaevm@mailru B I Tuleuov is with the Department of Mechanics and Mathematics, L N Gumilyov Eurasian National University, Astana, Kazakhstan berik t@yahoocom D Zhussupova is with the Department of Mechanics and Mathematics, L N Gumilyov Eurasian National University, Astana, Kazakhstan zhusdinara@gmailcom Orthonormal eigenvectors of operator A A corresponding to s we write as e =,, ; A Ae = s e ) Numbers s A) s A) s n A) are called singular values of matrix A Note that the notion ill-conditioned matrix is relative, which often depends on hardware capabilities and its boundary moves away together with the power increase of computers Let s say that the matrix A is ill-conditioned, if A A is large, when A is nonsingular Further vector s Euclidean norm and modulus of the number we write as, and operator s norm of matrix as, and scalar product as, Let A and f be from ), for ε 0 consider functional J ε x) = Ax f + ε x We will find x, which will be a solution of the problem inf J ε x) = J ε x) ) In the left-hand side ) infimum is taken with respect to all vectors x in R n Since unit ball in R n is compact, solution of ) exists Remark : If matrix A is invertible, for ε = 0 problem ) has unique solution x = A f, if A is noninvertible, A x gives the best approximation f by elements Ax If ε 0 and A is invertible, x is the approximate solution of equation Ax = f If ε = 0 and matrix A is noninvertible, the problem may have several solutions, in this case we search for solution with minimal norm Lemma : If ε 0 and x is the solution of ), A A x f) + ε x = 0 Proof Let x be the solution of ) and ω = A A x f)+ ε x 0 Consider J ε x + δω) We have J ε x+δω) = J ε x)+δ A A x f)+ε x, ω +δ Aω +ε ω ) = J ε x) + δ ω + δ Aω + ε ω ) Let a number δ satisfy the following conditions δ < 0, δ > δ Aω + ε ω ω Such a choice is possible by assumption ω A A x f) + ε x 0 Then we get J ε x + δω) < J ε x) and it is contradiction Lemma is proved Lemma : Let ε 0 and x be the solution of ) Then for all x H we have εx + A Ax f) = ε + A A)x x) Proof It easily follows from Lemma ISBN: ISSN: Print); ISSN: Online) WCE 03

2 Proceedings of the World Congress on Engineering 03 Vol I, WCE 03, July 3-5, 03, London, UK Now we define sequence x =,, ) by the following formula x = δ [E δa A + εe)] k A f, 3) where δ satisfies the condition 0 < δ < A A + ε 4) Theorem : Let ε 0, δ given by 4) and x be the solution of ), x be constructed by 3) Then x x = [E δa A + εe)] x 5) and x converges to x as + at the geometric rate, ie there exists ρ > 0 and x x C ρ, 6) where C is the constant which depends on δ and ε Proof By using lemma we have A f = ε x + A A x Substituting A f to 3) we get x = δ [E δa A + εe)] k [ε x + A A x] = [E δa A + εe)] k [E E + δε + A A)] x = E δa A + ε)) x + x It implies 5) Further, since matrix E δa A+εE) is self-adoint, its norm is equal to maximum of modulus of eigenvalues Its eigenvalues are δs + ε), =,,, n) If for each =,,, n eigenvalues satisfy we get < δs + ε) < E δa A + εe) < 7) These inequalities hold if conditions δ max s +ε) < =,,,n and δ > 0 take place But max s = A A The =,,,n condition 4) follows 7) and by 7) we get 6) The proof of theorem is complete Note that results similar to theorem for linear ill-posed problems have been obtained in [6] see [6], p 38) The space generated by eigenvectors of matrix A A corresponding to zero eigenvalues we denote by R n) 0, i e if x R n) 0 x = n k= 0 x e and A Ae k = 0 for k = 0,, n R n) 0 is the kernel of matrix A A If matrix A is invertible, the space R n) 0 is empty Lemma 3: If x R n) 0 A f, x = 0, i e A f belongs to R n) R n) 0 which is the orthogonal complement of R n) 0 Proof For ε = 0 by using lemmas and we obtain A f = A A x Let x R n) 0, A f, x = A A x, x = x, A Ax = 0 This completes proof of the lemma Lemma 4: If ε > 0 and x is the solution of ), x R n) 0, x, x = 0, ie x belongs to R n) R n) 0 Proof It easily follows from Lemmas and 3 Note that for ε = 0 the solution of ) is determined up to a term, which is the solution of equation Ax = 0, but sequence x =,, ) by lemma converges to the solution of ) belonging to R n) R n) 0 In further for ε = 0 we take as x0) the limit of sequence x from 3) Obviously the solution of ) depends on ε So sometimes we write x = xε) We have Lemma 5: If x0) is the solution of ), for every ε > 0 and δ 0 xε) = A A + εe) A A xo) = A A + εe) A f, x0) = E + εa A) ) xε), xε) = A A + εe) A A + δe) xδ), xε) xδ) = δ ε)a A + εe) xδ) Proof It easily follows from Lemma From proved lemmas and theorem we state Theorem : a) The solution xε) of ) continuously depends on ε > 0 and xε) = A A + εe) A f b) For + the limit of sequence x ε) from 3) continuously depends on ε 0 c) If s s s 0 > 0, s 0+ = s 0+ = = 0 are the eigenvalues of matrix A A and e, e,, e n are corresponding orthonormal system of eigenvectors, xε) ε > 0) is the solution of ) and x ε) from 3), xε), x ε) R n) R n) 0, =,, ) x k ε) x k ε) = δs k + ε)) x k 0), k 0, x k ε) = x k ε), 0 + k 8) Here x k ε) = x ε), e k, x k ε) = xε), e k d) Number ρ > 0 from theorem is defined by ρ = max{ δs 0 + ε)), δ A A + ε))} < Note that if matrix A A hasn t zero eigenvalues 0 is taken as n The item c) of theorem implies that vector xε) for ε = 0 has minimal norm among all solutions of problem Furthermore for each ε 0 xε) and x ε) =,, ) belong to subspace R n) R n) 0, where Rn) 0 is the kernel of matrix A A Below we suggest one method of parallel computation for solving problem ) based on theorems and Let n be large enough integer and we have N +-processor system Let k 0, k,, k N are integers such that k m + < k m, m = 0,,, N, k 0 = 0, k N = n We define matrices A m a km +, a km +, a km +,3 a km +,n a km, a km, a km,3 a km,n ISBN: ISSN: Print); ISSN: Online) WCE 03

3 Proceedings of the World Congress on Engineering 03 Vol I, WCE 03, July 3-5, 03, London, UK and A ) m ã km +, ã km +, ã km +,3 ã km +,n, ã km, ã km, ã km,3 ã km,n m =,,, N, in such a way that lines numerated from k m + to k m coincide with those of matrices A and A respectively Here ã k and a k are elements of A and A such that ã k = a k Also we use vectors ω = [E δa A + εe)] ω, ω 0 = δa f, =,, Then formula 3) can be written in the following way x + = x + ω, x = ω 0, =,, Before computation matrices A m and A ) m are passed to processors C m m =,,, N) and vector ω 0 = δa f to root processor C N+ Algorithm works as follows ) Processor C N+ forms -th approximation x and passes vector ω to processors C m m =,,, N) Each processor C m calculates A m ω spending k m k m )n multiplications, k m k m )n ) additions and sends vector to C N+ ) C N+ forms vector Aω = N A m ω and m= spends n )N additions C N+ transmits vector Aω 3) Processor C m calculates A ) m Aω and sends it to C N+, spending k m k m )n multiplications and k m k m )n ) additions C m transmits A ) m Aω to processor C N+ 4) Summing up recieved vectors C N+ gets A Aω ) = N A ) m Aω Root processor m= calculates ω = δε)ω δa ) m Aω and forms approximate solution x + = x + ω, =,, It spends n multiplications and n )N + n additions Amount of operations per cycle which root processor C N+ carries out consists of n multiplications and n )N + n additions Each processor C m m =,,, N) spends k m k m )n multiplications and k m k m )n ) additions per cycle All processors spend n multiplications and nn ) additions per cycle Since s iterations all computers spend sn +n +n multiplications and snn )+nn ) additions Division is absent It follows from theorem that for effectivicity of iteration formula 3) the existence of small but nonzero eigenvalues of matrix A A are important, but existence of zero eigenvalues of the matrix A doesn t play any role! Therefore we come to the question: Is it possible to reduce noises due to nonzero small eigenvalues of matrix A A? It turns out that it is possible We demonstrate it by simple example Let ε = 00 and A = ), f = ) ) Matrix A A = ) ) has a small nonzero eigenvalue Iterative process by formula 8) may last long However, if matrix A is replaced with its approximation ) ) à =, f = 0) we have à à = ) ) = 0 ) The eigenvalues of this matrix are equal to λ = 0, λ = 0 and its norm is 0 So δ may be taken from interval 0, 0 ) Let s take δ = 0 < 0 Then by theorem we obtain ρ = 0 Therefore the problem with matrix à and vector f from 0) is solved in one step ) The solution of problem is vector Equation ) x à = x 6) ) x has solution vector such that x x +x = Vector ) satisfies it and has a minimal norm among all vectors The vector found will be the approximate solution of ) ) with matrix A and vector f from 9) Indeed ) ) ) 0 = 3 300) We have A x f = Recall that we try to reduce a norm Ax f increasing ) the norm ) x not too much) x 4 The vector x = = is the actual solution of x 6 system ) ) ) x = x 6006 For x = we get ) And for ) A x f + ε x = 0 ) ) 4 6 =0006) A x f + ε x = ) = 04 ) Therefore for ε = 00 vector x = is closer to solution ) 4 of problem than x = 6 This simple idea tracked on simple example we will develop in the next work with matrices arisen in solving numerically ill-posed direct and inverse problems of mathematical physics In general this effect is not always possible But we have Theorem 3: Let ε 0 and x =,, ) be the sequence of vectors from 3) Then ISBN: ISSN: Print); ISSN: Online) WCE 03

4 Proceedings of the World Congress on Engineering 03 Vol I, WCE 03, July 3-5, 03, London, UK a) If γ > 0 and is chosen to satisfy δγ + ε)) γ, the following inequality Ax ε) f f γ + γ xε) + A xε) f holds b) If is given by δγ + ε)) γ, A A x ε) xε)) γ [ A A x + x ] ; c) If is given by δγ + ε)) 5 A γ, A A x ε) xε)) 8γ f ; d) For ε = 0, if is given by Proof Let ε 0 and inf {x} δγ) 5 A γ, A A x 0) f) 8γ f ) Ax f + ε x ) = A xε) f + ε xε) For any vector u we have Au = Au, Au = A Au, u = A A) u Therefore, as A A) e k = s k e k and using 8) from theorem we have A x ε) xε)) = A A) x ε) xε)) = s k x k ε) x k ε)) = Hence for all γ > 0 we get A x ε) xε)) = s k γ s k s k δs k + ε) ) xk ε) s k >γ s k δs k + ε) ) xk ε) δs k + ε) ) xk ε) δγ + ε)) s k x k ε) + γ x k ε) s k >γ s k γ δγ + ε)) n s k x k ε) + γ x k ε) = δγ + ε)) A A) xε) + γ xε) = δγ + ε)) A xε) + γ xε) ) But ) A xε) = A xε) f + f A xε) f + f = = [inf {x} Ax f + ε x )) ] + f f + f ) = 4 f 3) Using this estimate and ) we arrive at the estimate A x ε) xε)) 4 [ δγ + ε)] f + γ xε) Then Ax ε) f = A x ε) xε)) + A xε) f If it take place conditions [ A x ε) xε)) + A xε) f ] f δγ + ε)) + xε) γ + A xε) f δγ + ε)) γ we have from the last inequality Ax ε) f f γ + γ xε) + A xε) f It implies item a) of theorem Further, using 8) we have A A x ε) xε)) = s 4 k δs k + ε) ) xk ε) = s k >γ s 4 k s k γ s 4 k s k >γ s 4 k s k γ s 4 k It follows from the inequality above that δs k + ε) ) xk ε) δs k + ε) ) xk ε) δs k + ε) ) xk ε) δs k + ε) ) xk ε) 4) A A x ε) xε)) δγ + ε)) A A x + γ x If is taken from we obtain δγ + ε)) γ, A A x ε) xε)) γ [ A A x + x ] It implies assertion of item b) of theorem By 4) we have also A A x ε) xε)) + δγ + ε)) s k x k ε) s k x k ε) = δγ + ε)) A A x k ε) + γ A A x k ε) = δγ + ε)) A A x k ε) f) + A f +γ A x k ε) δγ + ε)) A A x k ε) f) + A f ) But in view of 3) we get following inequalities + γ A x k ε) A A x k f) A A x k + f ) 5 A f, A x k ε) 4 f ISBN: ISSN: Print); ISSN: Online) WCE 03

5 Proceedings of the World Congress on Engineering 03 Vol I, WCE 03, July 3-5, 03, London, UK Therefore A A x ε) xε)) δγ + ε)) [ 5 A f + A f ] + 4γ f Choosing from we get 0 δγ + ε)) A f + 4γ f δγ + ε)) 0 A 4γ, A A x ε) xε)) 8γ f In the case of ψ + = 0 or ψ + e + α k A ψ k ε put ψ + = 0 Calculation of A ψ k doesn t meet difficulty If ψ k = 0, we get A ψ k = 0, otherwise ψ k 0, in view of formula ψ k+ = Ae k+ we obtain the representation k α l ψ l, l= ψ k+ = AΘ k+, This completes assertion of item c) of the theorem For ε = 0 using lemma we have A A x = Af Therefore the item c) of the theorem implies the item d) This completes proof of the theorem Usually it is important in practice to reduce difference Ax f less important to find solution of equation Ax = f!) Thus the theorem allows to solve problem ) effectively Note that usage of formula for x doesn t require ε > 0 Much more suitable case is ε = 0 Now we can suggest the next numerical algorithm for the problem ) based on the theorem 3 We can form sufficiently effective process of solving problem ) with ill-conditioned or non-invertible matrix The algorithm will distinguished be from above one only by these points: It is chosen γ > 0 stands for accuracy) The number ε is chosen to be zero Item d) condition of the theorem 3 is verified after every cycle iteration Computation is finished when condition ) holds Now we will describe in brief Schmidt orthogonalization process Let e,, e n be a basis in H and A is ill-conditioned matrix Then Ae = a, a,, a n ), A = {a i } We will orthogonalize {Ae } n = by Schmidt Fix a number ε > 0 If Ae > ε, we put ψ = Ae ) Ae Otherwise, if Ae ε, put ψ = 0 Let vectorsψ,, ψ are constructed We define where α k = Now we define ψ + If ψ + = Ae + α k ψ k, { 0, if ψ k = 0 Ae +, ψ k, if ψ k 0 where Θ k+ is defined recurrently Finally we get ψ,, ψ n, which satisfy the conditions ψ i, ψ = 0, as i, ψ = or ψ = 0 Now put f = f, ψ ψ = f, ψ AΘ = ψ 0 ψ = A f, ψ Θ ψ 0 ψ We get now as the approximate solution x to the problem the vector A inf Ax f = x f x H x = f, ψ Θ ψ 0 ψ It can be shown that the inequality A x f Cε), holds with Cε) 0 as ε 0 Some of results of this work have been announced in [7] see also [8]) REFERENCES [] Zh A Baldybek, M Otelbaev Problem of Parallelization for a Linear Algebraic System // Matematicheskii Zhurnal, Vol, No 39), pp 53-58, in Russian [] V V Voevodin and Vl V Voevodin, Parallel Computations BKhV- Peterburg, St Petersburg, 00, p608), in Russian [3] JM Ortega, Introduction to parallel and vector solution of linear systems Plenum Press, New York, 98 [4] Dimitri P Bertsekas, John N Tsitsiklis Parallel and Distributed Computation: Numerical Methods, 997 [5] Brucel P I Parallel programming An introduction Prentice Hall International UK) Limited, 993 p 70 [6] Israel Gohberg, Seymour Goldberg, Marinus A Kashoek Basic Classes of Linear Operators Birkhauser Verlag, 003, p 38 [7] Otelbaev M, Tuleuov B, Zhusupova D On a Method of Finding Approximate Solutions of Ill-conditioned Algebraic Systems and Parallel Computation //Eurasian Mathematical Journal, Vol, No, 0, pp49-5 [8] Otelbaev M, Zhusupova D, Tuleuov B, Parallelization of linear algebraic system with the invertible matrix // Vestnik Bashkirsk Univ, vol 6, No 4, 0, pp9-33, in Russian ψ + 0 we put ψ + e + α k A ψ k > ε, ψ + = ψ + ψ+ ISBN: ISSN: Print); ISSN: Online) WCE 03