THE INTERPLAY BETWEEN CLASSICAL ANALYSIS AND (NUMERICAL) LINEAR ALGEBRA A TRIBUTE TO GENE H. GOLUB. WALTER GAUTSCHI y

Transcription

1 THE INTERPLAY BETWEEN CLASSICAL ANALYSIS AND (NUMERICAL) LINEAR ALGEBRA A TRIBUTE TO GENE H. GOLUB WALTER GAUTSCHI y Dedicated in friendshi, and with high esteem, to Gene H. Golub on his 70th birthday Abstract. Much of the work of Golub and his collaborators uses techniques of linear algebra to deal with roblems in analysis, or emloys tools from analysis to solve roblems arising in linear algebra. Instances are described of such interdiscilinary work, taken from quadrature theory, orthogonal olynomials, and least squares roblems on the one hand, and error analysis for linear algebraic systems, element-wise bounds for the inverse of matrices, and eigenvalue estimates on the other hand. Key words. Gauss-tye quadratures, eigenvalue/vector characterizations, orthogonal olynomials, modication algorithms, olynomials orthogonal on several intervals, least squares roblem, Lanczos algorithm, bounds for matrix functionals, iterative methods AMS subject classications. 65D3, 33C45, 65D10, 15A45, 65F10 1. Introduction. It has been a rivilege for me to have known Gene Golub for so many years and to have been able to see his very extensive work unfold. What intrigues me most about his work at least the art I am familiar with is the imaginative use made of linear algebra in roblems originating elsewhere. Much of Golub's work, indeed, can be thought of as lying on the interface between classical analysis and linear algebra. The interface, to be sure, is directional: a roblem osed in analysis may be solved with the hel of linear algebra, or else, a linear algebra roblem solved with tools from analysis. Instances of the former tye occur in quadrature roblems, orthogonal olynomials, and least squares roblems, while examles of the latter tye arise in error estimates for the solution of linear algebraic systems, element-wise bounds for the inverse of a matrix, and in eigenvalue estimates of interest in iterative methods. It will not be ossible here to ursue all the ramications of this interesting interlay between dierent discilines, but we try to bring across some of the main ideas and will refer to the literature for variations and extensions.. Quadrature. Integration with resect to some given measure d on the real line R is certainly a toic R that belongs to analysis, and so is the evaluation or aroximation of integrals Rf(t)d(t). If one follows Gauss, one is led to orthogonal olynomials relative to the measure d, which is another vast area of classical analysis. How does linear algebra enter in all of this? It was in 1969 when the connection between Gauss quadrature rules and the algebraic eigenvalue roblem was, if not discovered, then certainly exloited in the now classical and widely cited aer [33]. We Exanded version of a lecture resented in a secial session honoring Professor Gene H. Golub at the Latsis Symosium 00 on Iterative Solvers for Large Linear Systems, celebrating 50 years of the conjugate gradient method, held at the Swiss Federal Institute of Technology in urich, February 18{1, 00. y Deartment of Comuter Sciences, Purdue University, West Lafayette, Indiana (wxg@cs.urdue.edu). 1

2 WALTER GAUTSCHI begin with giving a brief account of this work, and then discuss various extensions thereof made subsequently..1. Gauss quadrature. Assume d is a ositive measure on R, all (or suciently many) of whose moments (.1) r = R tr d(t); r = 0; 1; ; : : : ; exist with 0 > 0. The n-oint Gauss quadrature rule for d is (.) R f(t)d(t) = nx =1 f( ) + R n (f); where = (n), = (n) deend on n and d, and R n (f) = 0 whenever f is a olynomial of degree n 1, (.3) R n (f) = 0; f P n 1 : This is the maximum degree ossible. If f (n) is continuous on the suort of d and has constant sign, then (.4) R n (f) > 0 if sgn f (n) = 1; with the inequality reversed if sgn f (n) = 1. The connection between Gauss quadrature and orthogonal olynomials is well known. If k ( ) = k ( ; d), k = 0; 1; ; : : :, denotes the system of (monic) olynomials orthogonal with resect to the measure d, = 0 if k 6= `; (.5) R k(t)`(t)d(t) > 0 if k = `; then 1 ; ; : : : ; n are the zeros of n ( ; d), and the can be exressed in terms of the orthogonal olynomials as well. The former are all distinct and contained in the interior of the suort interval of d, the latter all ositive. What is imortant here is the well-known fact that the orthogonal olynomials satisfy a three-term recurrence relation, (.6) k+1 (t) = (t k ) k (t) k k 1 (t); k = 0; 1; ; : : : ; 1 (t) = 0; 0 (t) = 1; with well-determined real coecients k = k (d) and k = k (d) > 0. In terms of these, one denes the Jacobi matrix (.7) J(d) = in general an innite symmetric tridiagonal matrix. Its leading rincial minor matrix of order n will be denoted by (.8) J n (d) = [J(d)] [1:n;1:n] : ;

3 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 3 Here, then, is the connection between the Gauss quadrature formula (.) and the algebraic eigenvalue roblem: the Gauss nodes are the eigenvalues of J n (d), whereas the Gauss weights are (.9) = 0 v ;1 ; where v ;1 is the rst comonent of the normalized eigenvector v corresonding to the eigenvalue. The eigenvalue charaterization of the nodes is an easy consequence of the recurrence relation (.6) and has been known for some time rior to the 1960s. The characterization (.9) of the weights is more intricate and seems to have rst been observed in 196 by Wilf [45, Ch., Exercise 9], or even reviously, around 1954, by Goertzel [46]; it has also been used by the hysicist Gordon in [34,. 658]. The merit of Golub's work in [33] is to have clearly realized the great comutational otential of this result and in fact to have develoed a stable and ecient comutational rocedure based on the QL algorithm. It is useful to note that the quadrature sum in (.) for smooth functions f can be written in terms of J n = J n (d) as (.10) nx =1 f( ) = 0 e T 1 f(j n)e 1 ; e T 1 = [1; 0; : : :; 0] R n : This follows readily from the sectral decomosition of J n and (.9). Also, for the remainder R n (f) in (.) one has (cf. [1,. 91, (vii)]) (.11) jr n (f)j kf (n) k 1 (n)! R n(t)d(t) = kf (n) k n ; (n)! rovided f (n) is continuous on the suort su(d) of d. The 1-norm of f (n) is the maximum of jf (n) j on su(d), and (.11) holds regardless of whether or not f (n) has constant sign. The Jacobi matrix J n (d), and with it the Gauss quadrature rule, is uniquely determined by the rst n moments 0 ; 1 ; : : :; n 1 of the measure d. The Chebyshev algorithm (cf. [1, x.3]) is a vehicle for assing directly from these n moments to the n recursion coecients k, k, k = 0; 1; : : :; n 1. Although numerically unstable, the rocedure can be carried out in symbolic comutation to arbitrary recision. (A Male 5 scrit named cheb.mws can be found on the internet at htt:// Gauss-Radau and Gauss-Lobatto quadrature. If the suort of d is a nite interval [a; b], the Gauss quadrature formula can be modied by requiring that one or both of the endoints of [a; b] be quadrature nodes. This gives rise to Gauss-Radau res. Gauss-Lobatto formulae. Interestingly, both these formulae allow again a characterization in terms of eigenvalue roblems; this was shown by Golub in [3]...1. Gauss-Radau quadrature. If 0 = a is the rescribed node, the (n+1)- oint Gauss-Radau formula is (.1) b a f(t)d(t) = a 0 f(a) + n X =1 a f( a ) + R a n(f);

4 4 WALTER GAUTSCHI where the remainder now vanishes for olynomials of degree n, (.13) R a n(f) = 0; f P n : Dene a modied Jacobi matrix of order n + 1 by J R;a n+1 (d) = 4 J n(d) n e n (.14) n e T n R n 3 5 ; where J n (d) is the same matrix as in (.8), n = n (d) as in (.6), (.15) R n = a n n 1 (a) n (a) ; with m ( ) = m ( ; d), and e T n = [0; 0; : : :; 1] the nth canonical basis vector of R n. Then, the nodes in (.1) (including 0 = a) are the eigenvalues of J R;a (d), n+1 and the weights a, = 0; 1; : : :; n, are again given by (.9) in terms of the resective normalized eigenvectors. An analogous result holds for the Gauss-Radau formula with rescribed node n+1 = b, (.16) b a f(t)d(t) = nx =1 b f( b ) + b n+1 f(b) + Rb n (f): The only change is relacing a in (.15) by b, giving rise to a modied Jacobi matrix J R;b n+1 (d). Both quadrature sums in (.1) and (.16) allow a matrix reresentation analogous to (.10), with J n relaced by J R;a n+1 res. JR;b and the dimension of n+1 e 1 increased by 1. The remainders R a n, R b n of the two Gauss-Radau formulae have the useful roerty (.17) R a n (f) > 0; Rb n (f) < 0 if sgn f (n+1) = 1 on [a; b]; with the inequalities reversed if sgn f (n+1) = 1. This means that one of the two Gauss-Radau aroximations is a lower bound, and the other an uer bound for the exact value of the integral. It now takes n+1 moments 0 ; 1 ; : : : ; n to obtain J R;a n+1 (d), JR;b n+1 (d) and the (n+1)-oint Gauss-Radau formulae. Chebyshev's algorithm will rovide the recursion coecients needed to generate J n (d) in (.14), n, and the ratio of orthogonal olynomials in (.15). The case of a discrete measure d N suorted on N oints t k with a t 1 < t < < t N b, and having ositive jums wk at t k, (.18) R f(t)d N (t) := w k f(t k); is of some interest in alications. For one thing, the Gauss-Radau formulae (.1), (.16) (and, for that P matter, the Gauss formula (.) as well), rovide \comressions" N of the sum S = w k f(t k), i.e., aroximations of S by a sum with fewer terms if n < N 1. When f is a olynomial of degree n, the comressed sums in fact have the same value as the original sum. More imortantly, the formula (.1) with n < N together with the comanion formula (.16) furnish uer and lower bounds of

5 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 5 S if f (n+1) < 0 on [a; b]. Alications of this will be made in xx5.{5.4. Chebyshev's algorithm can again be used to generate (.1) and (.16) from the moments of d N. There is also a numerically stable alternative to Lanczos's algorithm (cf. x5.1), due to Gragg and Harrod [35], generating the Jacobi matrix J n (d N ) directly from the quantities w k and t k in (.18).... Gauss-Lobatto quadrature. Written as an (n + )-oint formula, the Gauss-Lobatto quadrature rule is (.19) b a f(t)d(t) = 0 f(a) + and has the exactness roerty nx =1 f( ) + n+1 f(b) + R a;b n (f); (.0) and the sign roerty R a;b n (f) = 0; f P n+1 ; (.1) R a;b n (f) < 0 if sgn f (n+) = 1 on [a; b]; with the inequality reversed if sgn f (n+) = 1. The aroriate modication of the Jacobi matrix is q L n+1 e n+1 (.) J L n+(d) = 4 J n+1(d) q L n+1 et n+1 L n+1 with notations similar as in (.14). Here, L n+1, L n+1 are dened as the solution of the linear system (.3) 4 n+1(a) n (a) n+1 (b) n (b) L n+1 L n ; 3 5 = 4 a n+1(a) b n+1 (b) Then the nodes of (.19) (including 0 = a and n+1 = b) are the eigenvalues of J L n+(d) and the weights, = 0; 1; : : :; n; n + 1, once again are given by (.9) in terms of the resective normalized eigenvectors. Hence, (.10) again holds, with J n relaced by J L n+ and e 1 having dimension n Gauss quadrature with multile nodes. The Gauss-Radau and Gauss- Lobatto formulae may be generalized by allowing an arbitrary number of rescribed nodes, even of arbitrary multilicities, outside, or on the boundary, of the suort interval of d. (Those of even multilicities may also be inside the suort interval.) The remaining \free" nodes are either simle or of odd multilicity. The quadrature rule in question, therefore, has the form (.4) R f(t)d(t) = nx Xs =1 =0 () f () ( ) + mx rx 1 =1 =0 3 5 : () f () (u ) + R n;m (f); where are the free nodes and u the rescribed ones, and the formula is required to have maximum degree of exactness (n + P s )+ P r 1. This has a long history, going back to Christoel (all s = 0 and r = 1) and including among its contributors Turan (m = 0, s = s for all ), Chakalov, Pooviciu, and Stancu (cf. [19, x.]).

6 6 WALTER GAUTSCHI The rescribed nodes u give rise to the olynomial u(t) =! my =1 (t u ) r ; where! = 1 is chosen such that u(t) 0 for t on the suort of d. For the formula (.4) to have maximum algebraic degree of exactness, the free nodes (\Gauss nodes") must be chosen to satisfy R ny =1 (t ) s+1 t k u(t)d(t) = 0; k = 0; 1; : : :; n 1: By far the simlest scenario is the one in which s = 0 for all. In this case, are the zeros of the olynomial n ( ; ud) of degree n orthogonal with resect to the (ositive) measure ud. This gives rise to the roblem of modication: given the Jacobi matrix of the measure d, nd the Jacobi matrix of the modied measure ud. An elegant solution of this roblem involves genuine techniques from linear algebra; this will be described in x3. The weights = (0) are comutable similarly as in (.9) for ordinary Gauss quadrature, namely [7, x6] = 0 v ;1 =u( ); = 1; ; : : :; n; where 0 = R R u(t)d(t) and v ;1 is the rst comonent of the normalized eigenvector of J n (ud) corresonding to the eigenvalue. For the comutation of the remaining weights () in (.4), see [37]. The case of multile Gauss nodes (s > 0) is a good deal more comlicated, requiring the iterative solution of a system of nonlinear equations for the and the solution of linear algebraic systems for the weights () []., () ; see, e.g., [7, x5] and.4. Gauss-Kronrod quadrature. The quadrature rules discussed so far are roducts of the 19th century (excet for the multile-node Gauss rules). Let us turn now to a truly 0th-century roduct the Gauss-Kronrod formula (.5) R f(t)d(t) = nx =1 X K f( G ) + n+1 =1 K f( K) + RK n (f); where G are the nodes of the n-oint Gauss formula for d, and the n + 1 remaining nodes, called Kronrod nodes, as well as all n + 1 weights K, K are determined by requiring maximum degree of exactness 3n + 1, i.e., (.6) R K n (f) = 0; f P 3n+1: This was roosed by Kronrod [39] in the 1960s in the secial case d(t) = dt on [ 1; 1] as an economical way of estimating the error of the n-oint Gauss-Legendre quadrature rule. The formula (.5) nowadays is widely used in automatic and adative quadrature routines ([43], [17]). Remarkably enough, there is an eigenvalue/vector characterization similar to those in xx.1,. also for Gauss-Kronrod quadrature rules. This was discovered in 1997 by Laurie [40]. He assumes that there exists a ositive Gauss-Kronrod formula (i.e., K > 0, K > 0, and K R), which need not be the case in general.

7 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 7 (Indeed, the Kronrod nodes and all weights may well be comlex.) The modied Jacobi matrix is now a symmetric tridiagonal matrix of order n + 1 and has the form (.7) J K n+1 (d) = 6 4 J n (d) n e n 0 n e T n n n+1 e T 1 0 n+1 e 1 with notation similar as before and J n a symmetric tridiagonal matrix. The structure of J n diers according as n is even or odd. For deniteness, suose that n is even. Then 3 J n = 6 4 J [n+1:3n=](d) (.8) n e n= 7 5 (n even); n e T J K n= [(3n+)=:n] where J [:q] (d) denotes the rincial minor matrix of J(d) that has diagonal elements ; +1 ; : : : ; q, and similarly for J K [:q]. Thus, the uer left square block of J n (of order n=) may be assumed known, and the rest, including the constant n, is to be determined. Laurie devised an algorithm that determines the unknown elements of J n in such a way that the Gauss nodes G and Kronrod nodes K are the eigenvalues of J K n+1(d) and the weights are given by J n (.9) K = 0 [u K ;1] ; = 1; : : : ; n; K = 0 [u K n+;1] ; = 1; : : :; n; n + 1; where u K 1 ; uk ; : : : ; uk n+1 are the normalized eigenvectors of J K n+1(d) corresonding to the eigenvalues G 1 ; : : :; n G ; K 1 ; : : : ; n+1, K and u K 1;1 ; uk ;1 ; : : : ; uk n+1;1 their rst comonents. Moreover, J n in (.7) has the same eigenvalues as J n(d), i.e., the Gauss nodes G 1 ; : : : ; n G. If the Gauss nodes G are already known, as is often the case, there is some redundancy in Laurie's algorithm, inasmuch as it regenerates them all. In a joint aer with Calvetti, Gragg, and Reichel [8], Golub removes this redundancy by focusing directly on the Kronrod nodes. The basic idea is to observe that the trailing matrix J n in (.7) as well as the leading matrix J [n+1:3n=] (d) in (.8) (again with n assumed even) have their own sets of orthogonal olynomials and resective Gauss quadrature rules, the measures of which, however, are unknown. Since the eigenvalues of J n are the same as those of J n (d), the former Gauss rule has nodes G, = 1; ; : : :; n, and ositive weights, say, while the latter has certain nodes ~ and weights ~, = 1; ; : : :; n=. Let the matrices of normalized eigenvectors of J n (d) and J n be v = [v 1 ; v ; : : : ; v n ] and v = [v 1 ; v ; : : : ; v n ], resectively. The new algorithm will make use of the last comonents v 1;n ; v ;n ; : : : ; v n;n of the eigenvectors in v (assumed known) and the rst comonents v 1;1 ; v ;1 ; : : :; v n;1 of those in v. The latter, according to (.9), are related to the Gauss weights through [v ;1] = ; = 1; ; : : :; n;

8 8 WALTER GAUTSCHI where the underlying measure is assumed normalized to have total mass 1, and one comutes X n= = =1 `(~ ) ~ ; = 1; ; : : :; n; in terms of the second Gauss rule (for J [n+1:3n=] (d)) and the elementary Lagrange interolation olynomials ` associated with the nodes G 1 ; G ; : : : ; G n. Therefore, (.30) v ;1 = ; = 1; ; : : :; n: Now let (.31) J n = v Dv T ; D = diag ( G 1 ; G ; : : : ; G n ) be the sectral decomosition of J n, and dene (.3) V = 4 v T 1 0 T 0 0 v 3 5 ; a matrix of order n + 1. From (.7), one gets V T (J K n+1(d) I)V = 6 4 D I n e T n v 0 n v T e n 0 n n+1 e T 1 v n+1 v T e 1 D I ; where the matrix on the right is a diagonal matrix lus a Swiss cross containing the known elements e T n v and the elements et 1 v that were comuted in (.30). A further (cosmetic) orthogonal similarity transformation involving a ermutation and a sequence of Givens rotations can be alied to yield (.33) ~V (J K n+1 (d) I) ~ V = 6 4 D I D I c 0 T c T n where V ~ is the transformed matrix V and c a vector containing the entries in ositions n + 1 to n of the transformed vector [ n e T v; n n+1 e T 1 v ; n ]. Eq. (.33) now reveals that one set of eigenvalues of J K (d) is n+1 f G 1 ; G ; : : : ; n G g, while the remaining eigenvalues are those of the trailing block in (.33). From 6 4 D I c c T n = 6 4 I 0 c T (D I) D I c 0 T f() ; ;

9 where (.34) CLASSICAL ANALYSIS AND LINEAR ALGEBRA 9 f() = n + nx =1 c G ; ct = [c 1 ; c ; : : : ; c n ]; it follows that the remaining eigenvalues, i.e., the Kronrod nodes, are the zeros of f(). It is evident from (.34) that they interlace with the Gauss nodes G. The normalized eigenvectors u K 1 ; uk ; : : : ; uk n+1 required to comute the weights K, K via (.9) can be comuted from the columns of V by keeing track of the orthogonal transformations. 3. Orthogonal olynomials. The connection between orthogonal olynomials and Jacobi matrices (cf. x.1) gives rise to several interesting roblems: (a) Given the Jacobi matrix for the measure d, nd the Jacobi matrix for the modied measure d mod = rd, where r is either a olynomial or a rational function. (b) Given the Jacobi matrices for two measures d 1 and d, nd the Jacobi matrix for d = d 1 + d. (c) Let [c j ; d j ] be a nite set of P intervals, disjoint or not, and d j a ositive measure on [c j ; d j ]. Let d(t) = j [c j;d l](t)d j (t), where [cj;d j ] is the characteristic function of the interval [c j ; d j ], 1 if t [cj ; d [cj;d j ](t) = j ]; 0 otherwise: Knowing the Jacobi matrices J (j) for d j, nd the Jacobi matrix for d. The roblem in (b) is discussed in [13], where three algorithms are develoed for its solution. We do not attemt to describe them here, since they are rather technical and not easily summarized. Suce it to say that linear algebra gures rominently in all three of these algorithms. A secial case of Problem (a) modication of the measure by a olynomial factor a roblem discussed in [7] and [38], is considered in xx3., 3.3. It is related to a classical theorem of Christoel (cf., e.g., [19,. 85]), which exresses the orthogonal olynomials for the modied measure in determinantal form in terms of the orthogonal olynomials of the original measure. For algorithmic and comutational uroses, however, the use of Jacobi matrices is vastly suerior. The case of rational r, in articular r(t) = (t x) 1 with real x outside the suort of d, and r(t) = [(t x) + y ] 1, y > 0, is treated in [15], where algorithms are develoed that are similar to those in [0] but are derived in a dierent manner. Problem (c) is dealt with in x3.4. Note that Problem (b) is a secial case of Problem (c). We begin with an integral reresentation of Jacobi matrices and then in turn describe modication of the measure by a linear, quadratic, and higher-degree olynomial, and solution rocedures for Problem (c) Integral reresentation of the Jacobi matrix. Let ~ 0 ; ~ 1 ; ~ ; : : : be the system of orthonormal olynomials with resect to to the measure d, that is, ~ k (t) = k(t) (3.1) k k k ; k kk = R k(t)d; with k as in (.5), (.6). They satisfy the recurrence relation k+1 ~ k+1 (t) = (t k )~ k (t) k ~ k 1 (t); k = 0; 1; ; : : : ; (3.) ~ 1 (t) = 0; ~ 0 (t) = 1= 0 ;

10 10 WALTER GAUTSCHI with recursion coecients k, k as in (.6) and 0 = R R d(t) (= 0). From (3.) and the orthonormality of the olynomials ~ k one easily checks that (3.3) R t~ k(t)~`(t)d(t) = 8 < : 0 if jk `j > 1; k+1 if jk `j = 1; k if k = `: This allows us to reresent the Jacobi matrix J = J n (d) of order n (cf. (.8)) in integral form as (3.4) where J = R t(t)t (t)d(t); (3.5) T (t) = [~ 0 (t); ~ 1 (t); : : : ; ~ n 1 (t)]: Orthogonality, on the other hand, is exressible as (3.6) R (t)t (t)d(t) = I; where I = I n is the unit matrix of order n, and the rst n recurrence relations in (3.) can be given the form (3.7) t(t) = J (t) + n ~ n (t)e n ; where e n = [0; 0; : : :; 1] T R n. 3.. Modication by a linear factor. The roblem to be studied is the eect on the Jacobi matrix J of modifying the (ositive) measure d into a measure d mod dened by (3.8) d mod (t) =!(t c)d(t); where c is a real constant outside, or on the boundary, of the suort interval of d and! = 1 chosen such that the measure d mod is again ositive. A solution of this roblem has been given already by Galant [16] and was taken u again, and simlied, in [7]. The symmetric matrix!(j ci) is ositive denite since by the assumtions made regarding (3.8) all its eigenvalues are ositive. It thus admits a Cholesky decomosition (3.9)!(J ci) = LL T ; where L is lower triangular and bidiagonal. By (3.4), (3.6), and (3.8) one has!(j ci) =! This may be written as!(j ci) = L R (t c)(t)t (t)d(t) = R (t)t (t)d mod (t): R L 1 (t) T (t)l T d mod (t)l T ;

11 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 11 which, since L 1!(J ci)l T = I by (3.9), imlies (3.10) where R mod (t)t mod (t)d mod(t) = I; (3.11) mod (t) = L 1 (t): This means that mod are the orthonormal olynomials with resect to the measure d mod. What is the corresonding Jacobi matrix J mod? First observe that from (3.7) one has (3.1) (t c)(t) = (J ci)(t) + n ~ n (t)e n : Using the analogues of (3.4), (3.6), one has, by (3.11) and (3.8), (3.13) J mod ci = Multilying (3.1) with its transose, one gets R (t c) mod(t) T mod(t)d mod (t) =!L 1 R (t c) (t) T (t)d(t)l T : (t c) (t) T (t) = [(J ci)(t) + n ~ n (t)e n ][ T (t)(j ci) + n ~ n (t)e T n ]; and observing that by (3.6) R (J ci)(t)t (t)(j ci)d(t) = (J ci) and by orthonormality R (J ci)(t)~ n(t)e T nd(t) = 0; one nds Thus, by (3.13), R ~ n(t)d(t) = 1; R (t c) (t) T (t)d(t) = (J ci) + n e n e T n : J mod ci =!L 1 (J ci) + n e n e T n L T : Substituting from (3.9) yields for the desired Jacobi matrix J mod = 1! LT L + ci +! n L 1 e n e T n L T and noting that L 1 e n = e n =(e T n Le n) nally (3.14) J mod = 1! LT L + ci + e n e T n ; =! n=(e T n Le n) :

12 1 WALTER GAUTSCHI Equations (3.9) and (3.14) allow the following interretation: The matrix J 1 := J mod e n e T n is the result of one ste of the symmetric LR algorithm with shift c, (3.15) J ci = 1! LLT ; J 1 = 1! LT L + ci: Note that J 1 diers from J mod only by one element in the lower right-hand corner. We could get rid of it by deleting the last row and last column of J 1. This would yield the desired Jacobi matrix of order n 1. If we are interested in the Jacobi matrix J n;mod of order n, we can aly the symmetric LR algorithm (with shift c) to J n+1 (d) and then obtain J n;mod by discarding the last row and last column in the resulting matrix Modication by a quadratic and higher-degree factor. Modication by a quadratic factor (t c 1 )(t c ) essentially amounts to two alications of (3.15), (3.16) followed by J c 1 I = 1! 1 L 1 L T 1 ; J 1 = 1! 1 L T 1 L 1 + c 1 I (3.17) J 1 c I = 1! L L T ; J = 1! L T L + c I: From the second and third of these equations one gets L L T 1 =! L T 1! L 1 + (c 1 c )I : 1 In articular, if c 1 = c = c and! =! 1 = 1, then (3.18) L L T = L T 1 L 1: Let (3.19) Then, using (3.18), one comutes Q = L T 1 L ; R = L T LT 1 : Q T Q = L T L 1 1 L T 1 L = L T (L T 1 L 1) 1 L = L T (L L T ) 1 L = L T L T L 1 L = I; so that Q is orthogonal. As a roduct of two uer triangular matrices, R is uer triangular. Since, again by (3.18), QR = L T 1 L L T LT 1 = L T 1 L T 1 L 1L T 1 = L 1 L T 1, the rst equation of (3.16) can be written as (3.0) J ci = QR and the second of (3.17) similarly as (3.1) J = RQ + ci: Thus, J is obtained by one ste of the QR algorithm with shift c. It is now clear how the modication (3.) d mod (t) = (t c) d(t)

13 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 13 is to be handled: aly one ste of the QR algorithm with shift c to the Jacobi matrix J n+ (d) of order n + and discard the last two rows and columns of the resulting matrix to obtain J n;mod. More generally, a modication d mod (t) = (t c) m d(t) with an even ower can be handled by m stes of the QR algorithm with shift c, discarding the aroriate number of rows and columns, and a modication d mod (t) = (t c) m+1 d(t) by an odd ower by means of m shifted QR stes followed by one ste of the symmetric LR algorithm as in x3.. In this way it is ossible to accomlish the modication d mod (t) = r(t)d(t) for any olynomial r with real roots and r(t) 0 for t on the suort of d. Alternative methods, not necessarily requiring knowledge of the roots, are develoed in [38] Polynomials orthogonal on several intervals. Here we describe two solution rocedures for Problem (c) based resectively on Stieltjes's rocedure (cf. [1, x.1]) and modied moments Solution by Stieltjes's rocedure. Suose we are interested in generating the Jacobi matrix J = J n (d) of order n for the measure d(t) = P j [c j;d j](t) d j (t). It is well known that the recursion coecients k = k (d), k = k (d) satisfy (3.3) k = (t k; k ) d ( k ; k ) d ; k = 0; 1; : : :; n 1; (3.4) k = ( k; k ) d ( k 1 ; k 1 ) d ; k = 1; ; : : :; n 1; where (3.5) (u; v) d = R u(t)v(t)d(t) is the inner roduct associated with d. We also recall the basic recurrence relation (cf. (.6)) (3.6) k+1 (t) = (t k ) k (t) k k 1 (t); k = 0; 1; : : :; n 1; 1 (t) = 0; 0 (t) = 1; satised by the (monic) orthogonal olynomials k ( ) = k ( ; d). For convenience, we let, as before, (3.7) 0 = R d(t): Stieltjes's rocedure consists in the following: Comute 0 from (3.3) with k = 0 and 0 from (3.7). Then use (3.6) with k = 0 to generate 1. Go back to Eqs. (3.3), (3.4) and use them for k = 1 to obtain 1, 1. Then (3.6) is realied with k = 1 to get, etc. This rocedure, alternating between (3.3), (3.4) and (3.6), is continued until n 1, n 1 are obtained. The rincial issue in this rocedure is the comutation of the inner roducts in (3.3), (3.4). Since they require integrating olynomials of degrees at most n 1,

14 14 WALTER GAUTSCHI one can use n-oint Gauss quadrature (3.8) dj c j (t)d j (t) = nx (j) (j) ( =1 ); P n 1 ; for the measure d j on each constituent interval [c j ; d j ] of d. It has been observed in [14] that the exlicit calculation of the Gauss nodes and weights is not required, but only matrix maniulations involving the Jacobi matrices J (j) (of order n) for d j (cf. (.10)). We illustrate this for the inner roduct (3.9) (t k ; k ) d = R Denote (j) d 0 = j c d j j (t) and let (3.30) R t k (t)d(t) = X j dj c j t k (t)d j(t): (j) k = k (J (j) )e 1 ; e T 1 = [1; 0; : : :; 0] R n : Then, using (3.9), (3.8), and (.10), one has (t k ; k ) d = X j (j) (j) k( (j) ) = X j (j) 0 et 1 J (j) [ k (J (j) )] e 1 = X j (j) 0 et 1 k(j (j) )J (j) k (J (j) )e 1 ; that is, (3.31) (t k ; k ) d = X j (j) 0 (j)t k J (j) (j) k : Similarly (in fact a bit simler), one nds (3.3) ( k ; k ) d = X j (j) 0 (j)t k (j) k : The udating of the (j) k from (3.6), required in Stieltjes's rocedure follows immediately (3.33) (j) = k+1 (J (j) k I) (j) k k (j) k 1 ; where I is the unit matrix of order n and (j) 1 = Solution by the modied Chebyshev algorithm. The desired recursion coecients k (d), k (d), k = 0; 1; : : :; n 1, can also be roduced from the rst n modied moments (3.34) m k = R k(t)d(t); k = 0; 1; : : :; n 1; where f k g is a system of olynomials satisfying a three-term recurrence relation (3.35) k+1 (t) = (t a k ) k (t) b k k 1 (t); k = 0; 1; : : :; n 1; 1 (t) = 0; 0 (t) = 1

15 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 15 with known coecients a k, b k. A rocedure accomlishing this is the modied Chebyshev algorithm (cf. [1, x.4]); this works also P if the olynomials f k g satisfy an k extended recurrence relation k+1 (t) = t k (t) j=0 c kj j (t), and even if the measure d is indenite (see, e.g., [6]). The comutation of the modied moments (3.34) by Gauss quadrature is entirely analogous to the comutation of inner roducts in x Letting now (3.36) z (j) k = k (J (j) )e 1 ; one nds (3.37) m k = X j (j) 0 z(j)t k e 1 : Udating the vectors z (j) k can again be done via the recurrence relation (3.35), (3.38) z (j) k+1 = (J (j) a k I)z (j) k b kz (j) k 1 ; z(j) 1 = 0: There is yet a third algorithm roosed in [14], which is based on a fast Cholesky decomosition. For this, we refer to the original source. We remark that the modied Chebyshev algorithm rovides an alternative way of solving Problem (a) for olynomial modications d mod (t) = r(t)d(t) (cf. [19,. 13], [15]). Indeed, if r P m, then r can be exressed in terms of the olynomials k as (3.39) r(t) = mx j=0 c j j (t): If one assumes that f k g are orthogonal relative to the measure d, then the modied moments m k = R R k(t)d mod (t) are simly (3.40) 8 < m k = : c k R R k (t)d(t) if k m; 0 if k > m: The modied Chebyshev algorithm, if m < k, in fact simlies, owing to the k m+1 zero modied moments in (3.40). 4. The least squares roblem. The olynomial least squares roblem P N is as follows: Given N data oints (t k ; y k ), k = 1; ; : : :; N, where t 1 ; t ; : : : ; t N are mutually distinct oints on the real line, and N ositive weights wk, nd a olynomial q 0 P n 1, n N, such that P N : w k yk q 0 (t k ) With Problem P N one associates the discrete inner roduct (4.1) (u; v) dn = R u(t)v(t)d N := w k (y k q(t k )) for all q P n 1 : w k u(t k)v(t k );

16 16 WALTER GAUTSCHI and the norm kuk d N = (u; u) dn, in terms of which P N can be written as ky q 0 k d N ky qk d N for all q P n 1 : It is well known that the roblem allows an elegant solution by means of the orthonormal olynomials ~ k ( ) = ~ k ( ; d N ). Recall that there are exactly N such olynomials, ~ 0 ; ~ 1 ; : : : ; ~ N 1 ; we dene (4.) ~ N (t) = Q N (t t k) ; k N 1 k where N 1 is the monic orthogonal olynomial Matrix formulation of the least squares roblem and its solution. Let J = J N (d N ) be the Jacobi matrix of order N for the measure d N (cf. (.8)) and T = [~ 0 ; ~ 1 ; : : : ; ~ N 1 ] the vector of the N discrete orthonormal olynomials. Then, similarly as in (3.7), (4.3) t(t) = J (t) + ~ N (t)e N ; where ~ N (t) is dened as in (4.). Note by (4.3) and (4.) that the eigenvalues of J are the knots t 1 ; t ; : : :; t N. Thus, if P = [(t 1 ); (t ); : : :; (t N )], then (4.4) JP = P ; = diag(t 1 ; t ; : : : ; t N ): As a consequence of dual orthogonality (cf. [41, x.4.6]), one has T (t k )(t k ) = wk ; k = 1; ; : : :; N; so that w k (t k ) are the normalized eigenvectors of J. Thus, if (4.5) D = diag(w 1 ; w ; : : :; w N ); the matrix P D is orthogonal, and one has (4.6) P T P = D ; P D P T = I: Finally, (4.7) e T 1 P D = [w 1; w ; : : : ; w N ]~ 0 : Now let t = [t 1 ; t ; : : :; t N ] T, y = [y 1 ; y ; : : : ; y N ] T, and let q(t) = T (t)c be any olynomial of degree N 1 with coecients c = [c 0 ; c 1 ; : : : ; c N 1 ] T in the basis of the orthonormal olynomials. One checks that q(t) = P T c. In terms of the Euclidean vector norm k k = k kr N, the squared error in Problem P N for the olynomial q, in view of (4.5), can be written as (4.8) ky qk d N = kd (y q(t)) k = kd(y P T c)k = kp D D(y P T c)k = kp D y ck ; where the orthogonality of P D and the second relation in (4.6) have been used in the last two equations. Choosing c = P D y drives the error to zero and yields the

17 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 17 interolation olynomial of degree N 1. The solution of the least squares roblem P N, on the other hand, requires c = (4.8) is equal to (4.9) cn0, where c n = [c 0 ; c 1 ; : : : ; c n 1 ] T, and by q 0 (t) = T (t) cn 0 ; c n = P [1:n] D y: Here, P [1:n] is the matrix formed with the rst n rows of P. 4.. Udating and downdating the least squares solution. Suose we adjoin to the N data oints considered in x4.1 an additional oint (t N+1 ; y N+1 ) and give it the weight w N+1. How can the solution of the least squares roblem P N for the original N data oints be used to obtain the solution of the least squares roblem for the augmented set of N + 1 data oints? This is the roblem of udating the least squares solution. There is an analogous roblem of downdating whereby a single data oint is deleted. An interesting treatment of these roblems by matrix methods is given in [1]. We discuss here udating techniques only and refer to the cited reference for similar downdating techniques. In essence, the roblem of udating can be considered as solved once we have constructed the Jacobi matrix J u = J N+1 (d N+1 ) of order N + 1 for the augmented measure d N+1 from the Jacobi matrix J = J N (d N ) for the original measure d N, the inner roduct for d N+1 being (4.10) (u; v) dn+1 = N+1 X R u(t)v(t)d N+1(t) := w k u(t k)v(t k ): Let (cf. (3.7)) (4.11) so that (4.1) 0 = R d N (t); 0;u = R d N+1(t); ~ 0 = 1= 0 ; ~ 0;u = 1= 0;u : There is a unique orthogonal matrix Q N+1 of order N +1 whose rst row is rescribed to be (4.13) e T 1 Q N+1 = 1 0;u ( 0 e T 1 + w N+1 e T N+1) and which accomlishes a similarity transformation of the matrix tridiagonal form, (4.14) J 0 Q N+1 0 T Q T N+1 = T N+1 ; T N+1 tridiagonal t N+1 (cf. [4,. 113, (7--)]). We claim that J 0 0 T to t N+1 (4.15) J u = T N+1 :

18 18 WALTER GAUTSCHI To see this, recall that, with Q = P D (orthogonal), Eq. (4.4) imlies J = P P 1 = QD 1 DQ 1, hence (4.16) J = QQ T : By (4.7), in view of the rst relation in (4.1), there holds (4.17) 0 e T 1 Q = [w 1; w ; : : : ; w N ]: The analogous relations for the augmented roblem are and J u = Q u u Q T u ; u = 0 0 T t N+1 (4.18) 0;u e T 1 Q u = [w 1; w ; : : :; w N ; w N+1 ]; where e 1 now has dimension N + 1. Dene Then Q = Q u Q T 0 0 T 1 Q J 0 0 T Q T Q T 0 J 0 Q 0 = Q t u N+1 0 T 1 0 T t N+1 0 T 1 Q T JQ = Q 0 u 0 T Q T u t ; N+1 hence, by (4.16), : Q T u Q J T Q T = Q t u N+1 0 T Q T t = u Q N+1 u u Q T = u J u: Furthermore, using (4.18), e T 1 Q = e T Q T 1 Q 0 u 0 T 1 which by (4.7) and the rst of (4.1) becomes e T 1 Q = = = 1 [ 0 e T 1 Q w 0;u N+1] Q T 0 1 0;u [ 0 e T 1 w N+1 ] = 1 [w 1 ; : : : ; w N ; w N+1 ] Q T 0 0;u 0 T 1 0 T 1 1 0;u [ 0 e T 1 + w N+1 e T N+1]: Thus, Q satises exactly the roerties dening Q N+1, showing that indeed J u = T N+1. The analogue of (4.3), t u (t) = J u u (t) + ~ N+1;u (t)e N+1 ; in combination with the second relation of (4.1) can now be used to generate the new discrete orthonormal olynomials, and with them the udated least squares solution by the analogue of (4.9). ;

19 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 19 Algorithmically, the transformation (4.14) can be imlemented by a sequence of aroriate Givens rotations (cf. [1, Eq. (4.7)]). The udating technique described here is not the only ossible one; for others, see [ibid., xx4.3{4.7]. Since the solution for the one-oint least squares roblem P 1 is trivially ~ 0 = 1=jw 1 j, J = [t 1 ], c = [jw 1 jy 1 ], one can use the udating technique to build u the least squares solutions of P N successively for N = ; 3; : : : without necessarily having to store the entire data set. 5. Linear algebraic systems. Many linear algebra roblems that involve a symmetric ositive denite matrix A R NN can be related to discrete orthogonal olynomials suorted on the sectrum of A. This rovides the link between linear algebra and analysis. It may be aroriate, at this oint, to recall that the use of discrete (and other) orthogonal olynomials in the context of linear algebra has been ioneered by Stiefel [44]; see also [36, x14]. For simlicity assume that A has distinct 1 eigenvalues n, (5.1) 0 < N < N 1 < < 1 ; and denote the resective (orthonormal) eigenvectors by v n, (5.) Av n = n v n ; v T n v m = nm ; n; m = 1; ; : : :; N: (There should be no danger of confusing these 's with the weights of the Gauss quadrature rule in (.).) Thus, with V = [v 1 ; v ; : : :; v N ], = diag( 1 ; ; : : : ; N ), there holds (5.3) AV = V ; = V T AV : Now consider a discrete measure d N dened by (5.4) f(t)d N (t) := R+ k f( k); where k are ositive weights, and assume, temorarily, that the measure d N normalized, (5.5) R+ d N (t) = 1: is It is ossible to generate the orthonormal olynomials ~ k ( ; d N ), k = 0; 1; : : :; N 1, res. the associated Jacobi matrix J N = J N (d N ), entirely by matrix-vector multilications involving the matrix A and by an initial vector (5.6) h 0 = k v k ; kh 0 k = 1; whose comonents in the basis of the normalized eigenvectors are the (ositive or negative) square roots of the weights k. (Here and in the following, k k denotes the Euclidean vector norm.) A method accomlishing this is the Lanczos algorithm, which is briey described in x5.1. The subsequent sections give alications of this algorithm when combined with quadrature methods. 1 Otherwise, some terms in (5.4) below consolidate, so that N becomes smaller.

20 0 WALTER GAUTSCHI 5.1. The Lanczos algorithm. Let h 0 be given as in (5.6), and dene h 1 = 0. The Lanczos algorithm is dened as follows: (5.7) for j = 0; 1; : : :; N 1 do 6 4 j = h T j Ah j ~h j+1 = (A j I)h j j h j 1 j+1 = k h ~ j+1 k h j+1 = h ~ j+1 = j+1 Note that 0 can be arbitrary, but is often dened, in accordance with (5.5), by 0 = 1, or, in accordance with (5.10) below, by 0 = 0. The vectors h j so generated are orthonormal, as one checks by induction, and it is evident from (5.7) that fh j g n j=0, n < N, forms an orthonormal basis for the Krylov sace One also veries by induction that K n (A; h 0 ) = san(h 0 ; Ah 0 ; : : : ; A n h 0 ): (5.8) h j = j (A)h 0 ; where j is a olynomial of degree j satisfying the three-term recurrence relation (5.9) j+1 j+1 () = ( j ) j () j j 1 (); j = 0; 1; : : :; N 1; 1 () = 0; 0 () = 1: We claim that k ( ) = ~ k ( ; d N ). Indeed, from the second relation in (5.3) one has hence, by (5.8), n () = V T n (A)V ; h n = V n ()V T h 0 : Orthonormality h T n h m = nm of the Lanczos vectors h j then yields h T 0 V n()v T V m ()V T h 0 = h T 0 V n() m ()V T h 0 = nm ; which, since V T h 0 = P N ke k by (5.6), imlies as claimed. k;`=1 = k e T k diag ( n ( 1 ) m ( 1 ); : : : ; n ( N ) m ( N )) `e` k;`=1 k `e T k n(`) m (`)e` = k n( k ) m ( k ) = nm ;

21 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 1 The recurrence relation (5.9), therefore, must be identical with the one in (3.), i.e., j = j. If the measure d N is not normalized and, as in (3.7), one uts (5.10) 0 = R+ d N (t); then the recurrence relation still holds, excet that one must dene 0 () = 1= Bounds for matrix functionals. Given A R NN ositive denite and f a function analytic on an interval containing the sectrum of A, the roblem to be considered is nding lower and uer bounds for the bilinear form (5.11) u T f(a)v; where u; v R N are given vectors. The solution of this roblem has many alications; some will be discussed in subsequent sections. For alications to constrained least squares roblems for matrices, see [9], and [7] for alications to the evaluation of suitable regularization arameters in Tikhonov regularization. The case f(t) = ( t) 1 with outside the sectrum of A is imortant in hysical chemistry and solid state hysics alications; for references, see [3, x1]. Let rst u = v. With V = [v 1 ; v ; : : : ; v N ] and as dened in (5.), (5.3), we let (5.1) u = k v k and for simlicity assume k 6= 0 for all k. Then u = V, = [ 1 ; ; : : : ; N ] T, and f(a) = V f()v T. Therefore, that is, (5.13) u T f(a)u = T V T V f()v T V = T f() = u T f(a)u = R+ f(t)d N (t); k f( k); where d N is the discrete measure dened in (5.4). The desired bounds can be obtained by alying Gauss, Gauss-Radau, or Gauss-Lobatto quadrature to the integral in (5.13), rovided the aroriate derivative of f has constant sign (cf. xx.1,.). The Lanczos algorithm (cf. x5.1) alied with h 0 = u=kuk furnishes the necessary (discrete) orthogonal olynomials, res. their recursion coecients. For Gauss formulae, the quality of the bounds, even when no secic information is known about the sign of derivatives of f, can be estimated in terms of the absolute values of these derivatives and the quantities j = j generated during the Lanczos rocess (cf. [6]). One simly makes use of (.11). The case u 6= v can be handled by using the olarization identity u T f(a)v = 1 4 (T f(a) q T f(a)q) where = u + v, q = u v (cf. [, x3.1.], [3,. 46], or, for a similar identity, [3, Eq. (3)]) and alying aroriate bounds to the rst and second term of the identity. Alternatively, a \nonsymmetric" Lanczos rocess can be alied in conjunction with Gauss-Radau quadrature [30].

22 WALTER GAUTSCHI For the imortant function f(t) = t 1 (see, e.g., (5.19) or (5.) below), the case of an arbitrary nonsingular matrix A can be reduced to the case of a symmetric ositive denite matrix by noting that (5.14) u T A 1 v = u T (A T A) 1 w; w = A T v (cf. [, x3.], [3,. 47]) Error bounds. We consider now the system of linear algebraic equations (5.15) Ax = b with A R NN symmetric and ositive denite. Given any aroximation x to the exact solution x = A 1 b, the object is to estimate the error x x in some norm. We begin with using the Euclidean vector norm k k. Let r be the residual of x, (5.16) r = b Ax : Since x x = A 1 r, we have (5.17) kx x k = r T A r; which is (5.11) with u = v = r and f(t) = t. Here the derivatives are f (n) (t) = (n + 1)! t (n+), f (n+1) (t) = (n + )! t (n+3), so that (5.18) f (n) (t) > 0; f (n+1) (t) < 0 for t R + : The n-oint Gauss formula (with n < N) alied to the integral in (5.13) (with f(t) = t ) thus roduces a lower bound for the squared error (5.17). If the sectrum of A can be enclosed in an interval [a; b], 0 < a < b, then the \left-sided" (n + 1)-oint Gauss-Radau formula yields an uer bound, and the \right-sided" formula a lower bound for (5.17). The Lanczos algorithm alied with h 0 = r=krk yields the recursion coecients for the orthogonal olynomials required for generating these quadrature rules. If instead of the Euclidean norm one takes the A-norm kuk A = ut Au (cf. [31]), then (5.19) kx x k A = r T A 1 r; which is (5.11) with u = v = r and f(t) = t 1. Since this function satises the same inequalities as in (5.18), the Gauss and Gauss-Radau formulae alied to the integral in (5.13) (with f(t) = t 1 ) roduce the same kind of bounds as in the case of the Euclidean norm. The dierence between the N-oint and n-oint Gauss quadrature aroximation equals kx x n k A =krk, where x n is the nth iterate of the conjugate gradient method started with r (cf. [3, Eq. (50)]). The conjugate gradient method, in fact, can be used not only as an alternative to the Lanczos algorithm to generate the recursion coecients of the orthogonal olynomials, but also to imrove the aroximation x. The A-norm of the imroved aroximation can then be estimated from below and above (see [11, x5]). For analogous error estimates in the Euclidean norm, see [9]. The idea of using Gauss-Radau quadratures in combination with (5.18) to get error bounds for linear systems goes back to Dahlquist, Eisenstat, and Golub [10].

23 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 3 They also suggest a rocedure based on linear rogramming when all eigenvalues are known (cf. [10, x]). This requires knowledge of the moments m of d N, which by (5.13) are given by m = R+ t m d N (t) = r T A m r: Thus, comuting the rst n + 1 moments 0 ; 1 ; : : : ; n amounts to generating the Krylov sequence r; Ar; : : :; A n r and comuting the inner roducts of its members with r. In view of kx x k = R+ t d N (t) = k k ; an uer bound can be found by solving the linear rogramming roblem (5.0) subject to the constraints (5.1) max! k k k m k = m ; m = 0; 1; : : :; n; k 0; k = 1; ; : : :; N: Here, n can be any integer < N. A lower bound can similarly be obtained by relacing \max" in (5.0) by \min". The same rocedure, with k in (5.0) relaced by 1 k, works for the A-norm. The ideas outlined above, and still other ideas from the theory of moments, are alied in [10] to obtain uer and lower bounds for the errors in the Jacobi iterative method. Bounds for matrix moments m = r T A m r are similarly obtained in [4] The diagonal elements of A 1. Given A R NN ositive denite, the roblem is to nd bounds for the diagonal elements (A 1 ) ii of A 1, i = 1; ; : : :; N. Here, (5.) (A 1 ) ii = e T i A 1 e i ; where e i is the ith canonical basis vector. This is (5.11) with u = v = e i and f(t) = t 1. As before, f satises the inequalities (5.18) Lower bound from Gauss quadrature. By virtue of (.4) and the rst of (5.18), the n-oint Gauss quadrature sum (cf. (.10), where 0 = 1) yields a lower bound for the integral, i.e., (5.3) (A 1 ) ii = R+ t 1 d N (t) > e T 1 J 1 n e 1; e T 1 = [1; 0; : : :; 0] R n ; where J n = J n (d N ). Consider n = ; we aly two stes of the Lanczos algorithm with h 0 = e i to generate J = :

24 4 WALTER GAUTSCHI According to (5.7) we have 0 = a ii ; (5.4) ~h 1 = (A 0 I)e i = [a 1i ; : : : ; a i 1;i ; 0; a i+1;i ; : : : ; a Ni ] T ; 1 = a ki =: s i; s X k6=i h 1 = ~ h 1 =s i ; 1 = 1 s i ~h T 1 A~ h 1 = 1 s i XX k6=i `6=i a k`a ki a`i : Since J 1 = ; 1 0 one has (5.5) e T 1 J 1 e 1 = ; and therefore, by (5.3) and (5.4), (5.6) (A 1 ) ii > a ii X k6=i `6=i XX X k6=i `6=i a k`a ki a`i a k`a ki a`i X k6=i It should be noted that this bound, in contrast to those given below in xx5.4.{5.4.3, does not require any information about the sectrum of A Uer and lower bounds from Gauss-Radau quadrature. If the sectrum of A can be enclosed in the interval [a; b], 0 < a < b, then by the second of (5.18) and the rst of (.17) (with the inequality reversed) the \left-sided" (n + 1)- oint Gauss-Radau quadrature sum in (.1) yields an uer bound, and similarly the \right-sided" quadrature sum in (.16) a lower bound for the integral. Taking n = 1 in (.14), (.15), one gets J R;a (d N ) = R 1 a ki 1 A : ; R 1 = a a ; where 0 = a ii, 1 = s i from (5.4). Relacing here a by b yields J R;b (d N ). From (5.5), where 1 is relaced by R 1, a simle comutation then gives (5.7) a ii b + s i =b a ii a iib + s i < (A 1 ) ii < a ii a + s i =a a ii a iia + s i : Uer bound from Gauss-Lobatto quadrature. The (n + )-oint Gauss-Lobatto quadrature sum in (.19), on account of (.1) and the rst of (5.18)

25 CLASSICAL ANALYSIS AND LINEAR ALGEBRA 5 (with n relaced by n + 1), yields an uer bound for the integral. Taking n = 0 in (.), one gets J L (d N ) = 0 L 1 L 1 L 1 where by (.3) the quantities L 1 and L 1 solve the system a 0 1 L 1 a(a 0 ) b 0 1 ( L 1 ) = ; b(b 0 ) 0 = a ii : Carrying out the solution and using (5.5) with 1, 1 relaced by L 1, L 1, yields (5.8) (A 1 ) ii < a + b a ii ab The results in xx5.4.1{5.4.3 are from [30, Thm. 5.1]. When n >, the quadrature sum e T 1 J 1 n e 1 can be comuted for all three quadrature rules in terms of quantities generated during the course of the Lanczos algorithm; see [30, Thm. 5.3]. For an alication to Vicsek fractal Hamiltonian matrices, see [5] The trace of A 1 and the determinant of A. In rincile, each method described in xx5.4.1{5.4.3 can be used to estimate the trace (5.9) tr (A 1 ) = i=1 : (A 1 ) ii of A 1 by alying the method to each term in the sum of (5.9), hence N times. For large sarse matrices there are, however, more ecient estimation rocedures based on samling and a Monte Carlo aroach (cf. [, x4]). Alternatively, we may note that (cf. [1]) (5.30) tr (A 1 ) = 1 k = R+ ; t 1 d N (t); where d N is the discrete measure (5.4) with k = 1, k = 1; ; : : :; N. As in x5.4., we may then aly Gauss-Radau quadratures on an interval [a; b] containing all eigenvalues k to get lower and uer bounds. The only dierence is that now d N is no longer normalized, in fact (5.31) 0 = R+ d N (t) = N; and the Lanczos algorithm, in accordance with (5.6), is to be started with h 0 = 1 N N X v k : Observing that (5.3) 1 = = R+ td N (t) = t d N (t) = R+ k = i=1 X N k = i;j=1 a ii = tr (A); a ij = kak F ;

26 6 WALTER GAUTSCHI from (5.7) one gets 0 = h T 0 Ah 0 = 1 N v T k Av` = 1 N k ; that is, `=1 (5.33) 0 = 1 N 1: Furthermore, and ~h 1 = (A 0 I)h 0 = 1 N (A 0 I) 1 = ~ h T 1 ~ h 1 = 1 N An elementary calculation yields ( k 0 )v T k v k = 1 N N X ( k 0 )v k ; `=1 (` 0 )v`: (5.34) 1 = 1 N ( 1 N 1): The rest of the calculation is the same as in x5.4., excet that, by (5.31), one has to include the factor 0 = N in (5.5). The result is N 1 + N b < 1 b tr b 1 N (A 1 ) < N 1 + N a (5.35) ; a a 1 with 1, given by (5.3). The same inequalities, in a dierent form, are derived in [1, Eq. (9)] by means of dierence equations. As far as the determinant det A is concerned, we note that the trace is invariant to similarity transformations, so that by (5.3) tr (ln A) = tr (V ln V T ) = tr (ln) = Since det A = Q k k, this yields (5.36) det A = ex(tr (ln A)): ln k = ln Here, the trace of ln A can be estimated as described for A 1, with the function f(t) = t 1 relaced by f(t) = ln t. This latter function has derivatives whose signs are oosite to those in (5.18), which gives rise to bounds whose tyes are oosite to those obtained in xx5.4.1{ Note that in lace of J 1 in the quadrature sum (5.5), we now require ln J. This can be dened by linear interolation at the eigenvalues 0 < < 1 of J (see, e.g., [18, Ch. 5]), ln J = In articular, therefore, (5.37) e T 1 ln J e 1 = NY 1 1 [(J I) ln 1 + ( 1 I J ) ln ]: 1 1 [( 0 ) ln 1 + ( 1 0 ) ln ]; where 0 is given by the rst of (5.4) res. by (5.33). k :