ON THE TWO-LEVEL PRECONDITIONING IN LEAST SQUARES METHOD

Transcription

1 PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical and Matheatical Sciences 04,, p. 7 5 ON THE TWO-LEVEL PRECONDITIONING IN LEAST SQUARES METHOD M a t h e a t i c s Yu. A. HAKOPIAN, R. Z. HOVHANNISYAN Chair of Nuerical Analysis and Matheatical Modeling YSU, Arenia In the present paper an approach to construct algebraic two-level preconditioners for the atrices of noral systes arising in data fitting by least squares ethod with piecewise linear basis functions is proposed. The approach is based on using hierarchical grids with their subdivision into substructures and corresponding partition of the atrices. Estiates for condition nubers of preconditioned atrices are obtained. MSC00: 65D0; 65F08. Keywords: least squares ethod, noral syste, condition nuber.. Introduction. Least squares ethod can be considered as a ethod of data fitting. The proble consists in deterining the curve that best describes the relationship between expected and observed sets of data by iniizing the sus of the squares of deviations between observed and expected values (see [], for exaple). Matheatically, it can be stated as a proble of finding an approxiate solution to an overdeterined syste of linear equations. Suppose a set of points (data points) (x i,y i ), i =,,...,N, is given. The least squares proble is forulated as follows: find a function f (x) that depends on soe paraeters which iniizes the su N i= [ f (x i ) y i ]. (.) In the least squares ethod the approxiating function f (x) is often sought as a linear cobination of basis functions ϕ j (x), j =,,...,n, naely, n f (x) = c j ϕ j (x). j= Usually n N is taen. As a result, the proble is reduced to a noral syste A T Ac = A T y, (.) E-ail: yuri.haopian@ysu.a E-ail: ruzan.hovhannisyan@ysu.a

2 8 Proc. of the Yerevan State Univ., Phys. and Math. Sci., 04,, p where A = [ϕ j (x i )], i =,,...,N, j =,,...,n, (.3) is an N n atrix and c = [c c...c n ] T, y = [y y...y N ] T (for exaple, []). One of the drawbacs of the noral syste (.) is that it is usually ill-conditioned. Furtherore, the condition nuber of the atrix A T A increases with the increase of the nuber of basis functions [3]. This creates certain difficulties for nuerical solution of the noral syste (.). Fro now on, the atrix A T A of the noral syste (.) will be referred to as NS-atrix. The basis functions ϕ j (x) can be chosen in different ways. In practice, functions with local supports are convenient, e.g., basis splines. In the paper linear basis splines are considered. The ill-conditionedness of corresponding noral systes is discussed in [4]. In this article we develop a new approach that consists in constructing preconditioners for the atrices of noral systes. The concept of preconditioning and its role in the solution of linear systes can be found in [5], for exaple. Algebraic ultilevel preconditioning ethod is one of the ost efficient tools for nuerical solution of large-scale linear systes arising in finite eleent approxiation of partial differential equations. The ain ideas of the ethod have been proposed in [6, 7]. A ultilevel approach supposes a recursive decoposition of the grid until a coarse one, where the condition nuber of corresponding atrix is rather sall and the syste of grid equations can be solved easily enough. An iportant part of getting ultilevel preconditioners are so-called two-level preconditioners. The construction of that preconditioners is based on a special two-level ordering of the nodes on each grid refineent level and corresponding partitioning of the stiffness atrices. The present wor is the first attept to disseinate ultilevel technique for partial differential equations to the case of noral systes.. Hierarchical Partition of the Segent and NS-Matrices. Let (a,b) be an interval containing all the points of the set X = {x,x,...,x N }. First we describe a hierarchical partition of the segent [a,b]. Choose an integer n 0 and carry out an initial partition of the segent [a,b] using the points t (0) = a + ( )h 0, =,,...,n 0, (.) (b a) where h 0 = is the eshsize of the partition. The points (.) will be referred (n 0 ) to as the nodes of the initial partition. They for the initial grid σ 0 = {t (0) } n 0 =. With the initial partition we associate the intervals (0) = (t (0),t (0) + ), =,,...,n 0. Let us fix soe integer p > 3 and construct a hierarchical sequence of grids σ 0, σ,..., σ p, where each successive grid σ = {t () } n =,, with nodes t() is } n = by bisection procedure. We say that the grid σ corresponds to the -th level of partitioning of the segent [a,b]. For the nuber of nodes n and the eshsize h of the grid σ the forulae n = (n 0 ) +, h = h 0 / (.) obtained fro the previous grid σ = {t ( )

3 Haopian Yu. A., Honhannisyan R. Z. On the Two-Level Preconditioning... 9 hold. We introduce the following notation for the intervals associated with -th level of the hierarchical partition: () (t (),t () + h ), () [t (),t () + h ). (.3) At the partition level we will distinguish two types of nodes of the grid σ, that is, old nodes (those of the grid σ ) and new nodes which appear in a result of bisection procedure. The following ordering of the nodes is used: the old nodes preserve their nubers while the new nodes are nubered in consequtive order fro left to right. By construction, for the grid σ includes the grid σ. Therefore, the partitioning σ = σ () σ () (.4) can be defined in a natural way, where σ () is the subgrid of the old nodes and σ () is the subgrid of the new nodes. If n (i), i =,, is the nuber of nodes in the subgrid σ (i), then n() = n, n () = n n. For all levels = 0,,..., p, let G be the space of grid functions defined on the grid σ. These functions can be considered as vectors of lenght n. If, in accordance with the rule of node nubering, a grid function v G can be represented in the for [ v v = v ], v i G (i), i =,, (.5) where G (i) is the space of grid functions defined on the subgrid σ (i). Further, let V, 0 p, be the space of piecewise-linear functions associated with the grid σ. To each node t () σ, n, we put into correspondence a basis function ϕ () (x) V such that ϕ () (t () j ) = δ j, j =,,...,n, where δ j is the Kronecer sybol. The basis just defined is referred to as nodal hierarchical basis. Obviously, there is a one-to-one correspondence between piecewise-linear functions ˆv V and grid functions fro v G. In this connection we say that the function ˆv is the prolongation of the grid function v and write: ˆv = prol(v G : V ). By considering on each partition level a least squares proble of the type (.), we obtain a sequence of NS-atrices L () A ()T A (), = 0,,..., p, (.6) where, in accordance with (.3), A () = [ϕ () (x i )], i =,,...,N, =,,...,n, (.7) is an N n atrix. It follows fro (.6) and (.7) that the entries syetric atrix L () are coputed by the forula s () N q = i= s () q of the ϕ () (x i )ϕ () q (x i ),,q =,,...,n. (.8) Let us ae now the following assuption: the finest grid ω p is such that each interval (p) (see (.3)) contains at least one point fro the set X. Under this assuption the atrices L () are positive definite [4].

4 0 Proc. of the Yerevan State Univ., Phys. and Math. Sci., 04,, p If, then in accordance with the partitioning (.4) of the grid σ, the atrix A () can be written in the bloc for A () = [A () A() ] with subatrices A() and A () of sizes N n () and N n () correspondingly. In this case the atrix L () defined in (.6) is represented in the bloc for [ ] L () L () = L () L () L (), (.9) where L () i j = A ()T i A () j, i, j =,. Note that the blocs L () and L() are nonsingular diagonal atrices. 3. Two-Level Preconditioners. The bloc representation (.9) of the atrix L () can be written as where L () = [ S () + L () L() L () L () L () L () ], (3.) S () L () L() L() L () (3.) is the Schur copleent (c.f. [8]). For all values =,,..., p, taing into account the hierarchical structure of the grids, let us consider the atrix B () = [ L ( ) + L () L() L () L () L () L () ] (3.3) as a two-level preconditioner for the atrix L (). By construction, the atrix B () is positive definite [8]. Now we describe the process of a syste solution with atrix B (), p. To that end, let us consider the syste [ v where v = v ] [ g,g = g ] B () v = g, (3.4), v i,g i G (i), i =,. By using the bloc structure (3.3) of the atrix B (), we obtain the following coputational procedure. Procedure B () /TL. c o p u t e f = g L () L() g ;. s o l v e t h e s y s t e L ( ) v = f ; (3.5) 3. c o p u t e v = L () (g L () v ). End Thus, the solution of the syste (3.4) is reduced to the solution of the syste (3.5) with atrix L ( ). Recall that L () is a nonsingular diagonal atrix.

5 Haopian Yu. A., Honhannisyan R. Z. On the Two-Level Preconditioning Estiating the Spectral Condition Nuber. As it is nown, one of the ost iportant paraeters in preconditioning technique is the condition nuber of the considered atrix with respect to the corresponding preconditioner (c.f. [5, 8]). In accordance with the above entioned, let us deterine first the bounds of spectra of the atrices B () L (). To this end, consider the generalized eigenvalue proble L () u = λb () u. (4.) The sallest and the largest eigenvalues of the proble (4.) are denoted by λ () in and λ ax () respectively. L e a 4.. For all =,,..., p, the largest eigenvalue λ ax () =. P r o o f. Using the technique of transition fro the nodal hierarchical basis to the so-called two-level hierarchical basis in the space V (c.f. [6,9]), it can be proved that u T L () u u T B () u u G. (4.) Note, that a siilar inequality has been proved in [6, 0] when constructing preconditioners for finite eleent atrices. The inequality (4.) eans that for the eigenvalues of the proble (4.) the estiate λ holds []. On the other hand, λ = is eigenvalue of the proble (4.). In fact, proceeding fro the bloc structures (3.) and (3.3) of the atrices L () and [ B () ], respectively, it u can be readily seen that λ = and arbitrary grid function u =, where u = 0, u 0, are solution of the generalized eigenvalue proble (4.). Thus, λ ax () =. To ove further, we ae an assuption. Let d (p) be the nuber of points fro the set X, which belong to the interval (p) (see (.3)). We assue that for all values of the inequalities N c n p N d(p) c (4.3) n p hold, where c and c are soe positive constants. L e a 4.. For all values =,,..., p 3, is valid the estiate λ () in c c ( p )( p 4) 4( p + )( p ). (4.4) P r o o f. By Lea 4., λ = is an eigenvalue of the proble (4.). Procceding fro the bloc structures (3.) and (3.3) of the atrices L () and B (), respectively, it can be readily shown that, under the condition λ, the proble (4.) is reduced to the generalized eigenvalue proble [8] S () u = λl ( ) u, u G (). (4.5) Thus, λ () in can be considered as the sallest eigenvalue of the proble (4.5). Hence, this eigenvalue can be coputed through the following generalized Rayleigh quotient: λ () in = vt S() v v T L( ) v, (4.6) u

6 Proc. of the Yerevan State Univ., Phys. and Math. Sci., 04,, p where nonzero v G () us define a grid function v G () [ ] v grid function v = v is the corresponding eigenvector (c.f. [5, ]). Further, let G, for which as follows: v = L () L () v. Then we obtain a L () v + L () v = 0. (4.7) By construction, v T L () v = v T S() v. Thus, λ () in = vt L () v v T. (4.8) L( ) v Now consider ( )-th level of the hierarchical partition of the segent [a,b]. We have the intervals ( ) [t ( ),t ( ) + h ) associated with that level (see (.3)). As follows fro (.6) (.8), v T L () v = ˆv (x i ), (4.9) ( ) where ˆv(x) = prol(v G : V ). Siilarly, v T L ( ) v = where ˆv (x) = prol(v G () ( ) x i ( ) x i ( ) ˆv (x i ), (4.0) : V ). As a result, fro (4.8) (4.0) we get that λ () in = ( ) ( ) x i ( ) x i ( ) ˆv (x i ) ˆv (x i). Taing into account the above assuption that each interval (p) point fro the set X, it can be proved that λ () in ˆv (x i ) x i ( ) x i ( ) contains at least one ˆv (x i), (4.) where ( ) is an interval, in which function ˆv (x) is not identically zero. Let us introduce the notations J () ˆv (x i ), J () ˆv (x i ). (4.) x i ( ) x i ( ) Then the inequality (4.) can be written as follows: λ () in J() J (). (4.3) So, our tas is to estiate the quantities J () and J (). Let start with J(). First we perfor soe auxiliary constructions and introduce corresponding notations. We have the nodes t ( ) σ (), t () t ( ) + h σ (), t ( ) t ( ) + h σ ()

7 Haopian Yu. A., Honhannisyan R. Z. On the Two-Level Preconditioning... 3 of the grid σ associated with the considered interval ( ) Correspondingly, let ϕ (x) ϕ () (x), ϕ (x) ϕ () (x), = (t ( ),t ( ) ϕ 3 (x) ϕ () (x). + h ). According to the procedure of grid generation, the interval ( ) contains p + intervals of the p-th level of the hierarchical partition of the segent [a,b]. The points µ i = t ( ) + ih p, i = 0,,..., p +, are nodes of the p-th level. The values of piecewise linear functions can be easily coputed: ϕ (µ i ) = (p ) i, ϕ 3 (µ i ) = (p ) i, i = 0,,..., p, ϕ (µ i ) = + (p ) i, ϕ 3 (µ i ) = (p ) i, i = p, p +,..., p +. (4.4) In each of the intervals [µ i, µ i+ ), i =,,..., p +, we choose exactly s points of the set X. In accordance with (4.3) and (.), N c p (n 0 ) s c N p (n 0 ). (4.5) Let us denote the set of selected points by X (). Obviously, J () ˆv (x i ). (4.6) x i X () Now we introduce the following notations: w ˆv(t ( ) ) = ˆv (t ( ) ), w ˆv(t ( ) ) = ˆv (t ( ) ), w 3 ˆv(t () ). Then we can write the piecewise linear function ˆv(x) in the segent [t ( ),t ( ) ] in the following for: ˆv(x) = w ϕ (x) + w ϕ (x) + w 3 ϕ 3 (x). Proceeding fro (4.6), we get J () s w + s w + s 33 w 3 + s w w + s 3 w w 3 + s 3 w w 3, (4.7) where s αβ ϕ α (x i )ϕ β (x i ), α,β =,,3. (4.8) x i X () It can be easily seen that s = 0. Above we have established the relation (4.7) for our grid function v G. Then, using the forula (.8) and the notation (4.8), we obtain the relation s 3 w + s 3 w + s 33 w 3 = 0. (4.9) So, fro (4.7) and (4.9) we arrive at the inequality J () s w + s w s 33w 3. Again, using the relation (4.9), let us eliinate the quantity w 3 in the right-hand side of the last inequality. As a result we get J () ( s s 3 s 33 ) w + ( s s 3 s 33 ) w. (4.0) Thus, it reains to estiate the quantities s αβ defined in (4.8). For exaple, we estiate s. Using the expressions (4.4), by siple calculation we obtain s = x i X () ϕ (x i ) s p ϕ (µ i ) = s (p )( p ) i= 6 p. (4.)

8 4 Proc. of the Yerevan State Univ., Phys. and Math. Sci., 04,, p The reaining quantities are estiated siilarly, naely, and s s (p )( p ) 6 p, s 33 s (p )( p ) 3 p (4.) s α 3 s (p + )( p ) p, α =,. (4.3) Fro (4.0) (4.3) we obtain the inequality J () s ( p )( p 4) 8( p (w + w ) ). Taing into account (4.5),we obtain J () N ( p )( p 4) c p (n 0 ) 8( p (w + w ) ). (4.4) By a siilar reasoning, we obtain the inequality J () N p + c p (w + w (n 0 ) ). (4.5) Finally, the required estiate (4.4) follows fro the inequalities (4.3), (4.4) and (4.5). It is easy to verify that the quantity in the right-hand side of the inequality (4.4) increases with decreasing. Now we can estiate the spectral condition nuber of the atrices B () L (), which is defined as follows: κ(b () L () ) λ ax/λ () () in ( [8, ]). As a direct consequence of Leas 4. and 4., we get the following stateent. T h e o r e. For all values =,,..., p 3, the estiate κ(b () L () ) c 4( p + )( p ) c ( p )( p (4.6) 4) is valid. Consider closely the quantity δ 4( p + )( p ) ( p )( p fro the righthand side of the inequality (4.6). The strightforward calculation shows that 4) 4 < δ < δ < < δ p 3 = (4.7) 5 Thus, the spectral condition nuber of the atrices B () L () decreases with decreasing. Moreover, as follows fro (4.6) and (4.7), the condition nubers are bounded fro above by a value independent of the nuber of refineent levels of the initial grid. 5. Concluding Rears. In this paper we have discussed an idea of constructing algebraic two-level preconditioners for NS-atrices arising in data fitting by least squares ethod using piecewise linear basis functions. The results obtained can be useful for constructing ultilevel preconditioners. They also serve as a basis when considering ore sooth approxiating functions. We will study these issues in subsequent publications. Received

9 Haopian Yu. A., Honhannisyan R. Z. On the Two-Level Preconditioning... 5 R E F E R E N C E S. Bjorc A. Nuerical Methods for Least Squares Probles. SIAM, Philadelphia, PA, Ortega J.M., Poole W.G. Jr. An Introduction to Nuerical Methods for Differential Equations. Pitan Publishing Marshfield, MA Watins D.S. Fundaentals of Matrix Coputations. New Yor, John Wiley& Sons, Inc., Hovhannisyan R.Z. On the Behaviour of the Condition Nuber for Matrices of Noral Systes on the Sequences of Grids. Matheatics in Higher School, 0, v. 8,, p. 4 5 (in Russian). 5. Quarteroni A., Sacco R., Saleri F. Nuerical Matheatics. Springer, Axelsson O., Vassilevsi P. Algebraic Multilevel Preconditioning Methods. Part I. // Nuerische Matheati, 989, v. 56, p Haopian Yu.R., Kuznetsov Yu. A. Algebraic Multigrid/Substructuring Preconditioners on Triangular Grids. // Soviet Journal of Nuerical Analysis and Matheatical Modelling, 99, v. 6, 6, p Axelsson O. Iterative Solution Methods. Cabridge University Press, Axelsson O., Gustafsson I. Preconditioning and Two-Level Multigrid Methods of Arbitrary Degree of Approxiation. // Matheatics of Coputation, 983, v. 40, 6, p Axelsson O., Haopian Yu. Algebraic Multilevel Preconditioners for Perturbed Finite Eleent Matrices. // East-West Journal of Nuerical Matheatics, 997, v. 5,, p... Horn R.A., Johnson C.R. Matrix Analysis. Cabridge University Press, 986.