Error Estimation and Evaluation of Matrix Functions

Transcription

1 Error Estimation and Evaluation of Matrix Functions Bernd Beckermann Carl Jagels Miroslav Pranić Lothar Reichel UC3M, Nov. 16, 2010

2 Outline: Approximation of functions of matrices: f(a)v with A large and sparse. Polynomial approximation: Reduction to small problem by Arnoldi. Error bounds via the Faber transform. Rational approximation: Reduction to small problem by rational Arnoldi. A symmetric, one or several distinct poles: Derivation of short recursion formulas. Computed examples.

3 Approximation of functions of matrices A R n n large, sparse or structured. f nonlinear, v unit vector. Approximate w := f(a)v. Examples: f(t) = exp(t), f(t) = t, f(t) = ln(t).

4 If A small, then several approaches are possible, including the use of the spectral factorization and A = SΛS 1, Λ = diag[λ 1,λ 2,...,λ n ] f(a)v = Sf(Λ)S 1 v, f(λ) = diag[f(λ 1 ),f(λ 2 ),...,f(λ n )]. Using other factorizations (Schur, Cholesky), squaring and scaling also options. Reference for small problems: N. J. Higham, Functions of Matrices, SIAM, If A large, then first reduce to small matrix.

5 Polynomial Approximation m steps of the Arnoldi process with initial vector v gives AV m = V m H m + g m e T m, where V m = [v 1,v 2,...,v m ] R n m, v 1 = v, VmV T m = I, H m = VmAV T m Hessenberg, Vmg T m = 0, e m = [0,...,0, 1] T R m.

6 Define the Krylov subspace: K m (A,v) = span{v,av,...,a m 1 v}. Then range(v m ) = K m (A,v), V m e j = p j 1 (A)v, p j 1 P j 1. Approximate w := f(a)v by w m := V m f(h m )e 1. This is a polynomial approximant: V m f(h m )e 1 = p(a)v, p P m 1.

7 For any polynomial p P m 1, Therefore p(a)v = p(a)v m e 1 = V m p(h m )e 1. f(a)v V m f(h m )e 1 = (f p)(a)v V m (f p)(h m )e 1 (f p)(a) + (f p)(h m ). How can we bound the right-hand side?

8 Crouzeix 2006: There is a universal constant 2 C 11.5, such that for any A C n n and any function f analytic in the field of values W(A) = {y Ay : y C n, y = 1}, there holds f(a) C f L (W(A)). Corollary: Let f be analytic in W(A). Then f(a)v V m f(h m )e 1 23 min p P m 1 f p L (W(A)). Application of the Faber transform yields a sharper bound.

9 Error bounds via the Faber transform Let E be a convex compact set symmetric with respect to the real axis containing the field of values. Let E c = C\E. Example: When A R n n is symmetric, let E be the a real interval containing λ(a). Example: When A R n n is normal, let E be the convex hull of λ(a).

10 The Faber transform Φ maps the polynomial p(w) = a 0 w 0 + a 1 w a m w m, a j C, to the polynomial Φ(p)(z) = a 0 f 0 (z) + a 1 f 1 (z) a m f m (z), where f j is the Faber polynomial of degree j for E. Example: Let E = [ 1, 1]. The Faber polynomials are scaled Chebyshev polynomials of the first kind. Example: Let E = {z : z c r}. Then f m (z) = (z c) m /r m.

11 Let D closed unit disc in C and φ : E c D c unique conformal mapping with φ( ) =, φ ( ) > 0, ψ = φ 1. The Faber transform Φ is a bijection between functions F analytic in D and functions f analytic in E: z Int(E) : f(z) = Φ(F)(z) = 1 F(φ(ζ)) 2πi ζ z dζ, w Int(D) : F(w) = Φ 1 (f)(w) = 1 2πi E D f(ψ(u)) u w du.

12 Example: The Faber polynomials are given by f m (z) = Φ(P)(z) for P(w) = w m, m = 0, 1, 2,..., i.e., f m P m is the polynomial part of φ(z) m.

13 Theorem: f = Φ(F) = where 1 Φ 1 η m(f, D) η m (f, E) Φ η m (F, D), Φ 2 and η m (f, E) = η m (F, D) = min p P m f p L (E), min P P m F P L (D).

14 Theorem: Let E be convex and W(A) E. Define Φ + (F)(z) = Φ(F)(z) + F(0) for F analytic in D. Then Φ + 2. Moreover, for F analytic in D, Φ + (F)(A) 2 F L (D).

15 Proof: Use the representation Φ + (F)(A) = Φ(F)(A) + F(0)I = 1 π where K(w) := 1 2i D F(w)K(w) dw, ( ψ 1 dw (w)(ψ(w)i A) dw ψ (w)(ψ(w)i A 1 dw ) dw and A is the conjugate transpose of A. )

16 Corollary: Let E be convex and W(A) E. Let f A(E), F := Φ 1 (f), P P m, and Then p(z) := Φ + (P)(z) F(0). f(a) p(a) 2 F P L (D).

17 Theorem: Let W(A) E and let f = Φ(F) be analytic in E. Then f(a)v V m+1 f(h m+1 )e 1 4η m (F, D). The Arnoldi process gives accurate results if F can be approximated well by a polynomial of fairly low degree on D. Related results shown by Druskin, Knizhnerman, Hochbruck, Lubich,...

18 Rational Arnoldi Determine an orthonormal basis {v j } m+1 j=1 Krylov subspace of the rational (q(a)) 1 span{v,av,...,a m v} where q(z) := (z z 1 )(z z 2 ) (z z m ). Let z 0 C, z 0 z j, j 1, and let v 1 = v. For j = 1, 2,..., determine v j+1 by orthonormalizing (z j z 0 )(z j I A) 1 (A z 0 I)v j against available basis vectors v 1,v 2,...,v j.

19 This defines the coefficients h k,j in h j+1,j v j+1 = (z j z 0 )(z j I A) 1 (A z 0 I)v j +h 1,j v 1 + h 2,j v h j,j v j, j = 1, 2,.... In matrix notation, with z m+1 =, (A z 0 I)V m+1 (I+H m+1 D m+1 ) = V m+1 H m+1 +h m+2,m+1 v m+2 e T m+1, where D m+1 = diag[(z 1 z 0 ) 1, (z 2 z 0 ) 1,...,(z m+1 z 0 ) 1 ].

20 Projected matrix: A m+1 := Vm+1AV T m+1 = z 0 I + H m+1 (I + H m+1 D m+1 ) 1. Simplifies to the (standard) Arnoldi projection A m+1 = H m+1 used for polynomial approximation when z 0 = 0, z j =, j = 1, 2,....

21 Theorem: Let W(A) E and let f = Φ(F) be analytic in E. Let z 1,z 2,...,z m E and z m+1 =. Then where f(a)v V m+1 f(h m+1 )e 1 4η Q m(f, D), Q(w) = (w w 1 )(w w 2 ) (w w m ), w j = Φ 1 (z j ) and η Q m(f, D) = min P P m F P Q L (D)

22 Approximation of Markov functions Let dµ be a positive measure with support in [α,β], α < β <. Then f(z) = β α dµ(x) z x is a Markov function. Note: f analytic in C\[α,β]. Examples: log(1 + z) f(z) =, z f(z) = z γ, 0 < γ < 1, z C\R, are Markov functions.

23 Define the polynomial q(w) = m (w w j ), 1 < w j, j=1 with real or complex conjugate zeroes. Introduce the Blaschke product B(w) = wm q(1/w) q(w) = m j=1 1 w j w w w j.

24 Theorem: Let E be compact, convex, and symmetric with respect to the real axis. Let f be a Markov function with α < β < γ = min{re(z) : z E}. Then f = F 1 (f) is a Markov function, f(w) = β α φ (x)dµ(x) w φ(x) =: d µ(x) w x.

25 Theorem (cont d): Let R = P/ q with P P m 1 be a rational interpolant of f with prescribed poles w j at the reflected points 1/w j, j = 1, 2,...,m. Define r(w) = R(w) + B(w) Then r P m / q and ( f(1) R(1) 2B(1) η eq m(f 1 (f), D) f r L (D) f L (E) φ(β) + f( 1) R( 1) 2B( 1) max y φ([α,β]) ). 1 B(y). A bound for rational approximants of f on E is given by η q m(f, E) 2η eq m(f 1 (f), D).

26 Rational Lanczos with a fixed pole Inspired by Druskin and Knizhnerman (SIMAX, 98). Let A be symmetric and nonsingular. Determine orthonormal bases for the rational Krylov subspaces K l,m (A,v) := span{a l+1 v,...,a 1 v,v,av,...,a m 1 v}. We consider m = il for i = 1, 2, 3,...

27 A Lanczos-like method for orthogonalizing K 1,2 (A,v),K 2,3 (A,v),... for A SPD (i = 1). Let {v 0,v 1,v 1,v 2,...,v m 1,v m+1,v m } be an ON basis for K m,m+1 (A,v). Represent the v j in terms of monic orthogonal Laurent polynomials:

28 φ j (x) := x j + x j + j 1 k= j+1 j k=j+1 c j,k x k, j = 0, 1,...,m, c j,k x k, j = 1, 2,..., m + 1. with w j = φ j (A)v, v j = w j / w j.

29 Orthogonality with respect to the inner product (p,q) := (p(a)v) T (q(a)v) By symmetry of A: (xp,q) = (p,xq).

30 Theorem (Njåstad and Thron, 83): The orthogonal Laurent polynomials φ j satisfy short recursion relations. Survey: Jones and Njåstad, JCAM, 91 Recent work: Díaz-Mendoza, Gonzáles-Vera, Jiménez Paiz, and Njåstad. Simplest recursions when A SPD = trailing coefficient of every φ j is nonvanishing.

31 Algorithm: Compute ON basis {v k } m k= m+1 of Km,m+1 (A,v). δ 0 := v ; v 0 := v/δ 0 ; u := Av 0 ; α 0 := v T 0 u; u := u α 0v 0 ; δ 1 := u ; v 1 := u/δ 1 ; for k = 1,2,...,m 1 do w := A 1 v k ; β k+1 := v T k+1 w; w := w β k+1v k+1 ; β k := v T k w; w := w β kv k ; δ k := w ; v k := w/δ k ; u := Av k ; α k := v T k u; u := u α kv k ; α k := v T k u; u := u α kv k ; δ k+1 := u ; v k+1 := u/δ k+1 ; end

32 Let V 2m 1 = [v 0,v 1,v 1,v 2,...,v m 1,v m+1 ]. From the recursion formulas: H 2m 1 = V T 2m 1AV 2m 1, AV 2m 1 = V 2m H2m 1, G 2m = V T 2m A 1 V 2m, A 1 V 2m = V 2m+1 G2m. Odd numbered columns of H 2m 1 have at most 3 nontrivial elements and even numbered columns have at most 5.

33 Example: H 8 = * * * * * * * * * * * * * * * * * * * * * * * * * * * *.

34 Example: G 8 = * * * * * * * * * * * * * * * * * * * * * * * * * * * *. Both G 2m and H 2m are pentadiagonal.

35 Moreover, H 2m G 2m = I + e 2m u T 2m, where only the last two entries of u 2m may be nonvanishing.

36 A Lanczos-like method for orthogonalizing K 1,2 (A,v),K 2,3 (A,v),... for A indefinite (i = 1). The trailing coefficient of the φ j may vanish = new derivation of recursion formulas. There may be 5-term recursions.

37 Example: H 7 = * * * * * * * * * * * * * * * * * * * *

38 A Lanczos-like method for orthogonalizing K 1,2 (A,v),K 2,4 (A,v),... for A SPD. Orthogonal Laurent polynomials: φ j (x) := with x j + x j + j 1 k= (j 1)/2 2j k=j+1 c j,k x k, j = 0, 1,...,m, c j,k x k, j = 1, 2,..., m + 1. v j = φ j(a)v, j = 0, 1, 2, 1, 3, 4, 2, 5,.... φ j (A)v

39 Then {v 0,v 1,v 2,v 1,v 3,...,v m+1,v 2m 1 } an orthonormal basis for K m,2m (A,v). Example: The matrix H 10 is pentadiagonal:

40

41 A Lanczos-like method for orthogonalizing K 1,i (A,v),K 2,2i (A,v),... for A SPD, i 2. We want to determine orthonormal basis of v,av,a 2 v,...a i v,a 1 v,a i+1 v,...,a 2i v,a 2 v,a 2i+1 v,.... Associated orthogonal Laurent polynomials φ 0,φ 1,...φ i,φ 1,φ i+1,...,φ 2i,φ 2,...

42 of the form φ j (x) := x j + x j + j 1 k= (j 1)/i ij k=j+1 c j,k x k, j = 1, 2, 3,..., c j,k x k, j = 1, 2, 3,.... with φ 0 (x) := 1. Example: Let m = 2 and i = 3. Then H 8 of the form

43 x x x x x x x x x x x x x x x x x x x x x x x x x

44 Computed examples. A R , v R 1000 random. Tabulate error in approximations obtained by 42 steps of standard Lanczos or rational Lanczos.

45 Example 1. A = n 2 [ 1 2 1] tridiagonal, SPD; n = f(x) Lan. (42) Rat. Lan. (21,22) Rat. Lan. (14,29) exp(x) x exp( x) ln(x) exp(x)/x

46 Example 2. A = [a j,k ], a j,k = 1/(1 + j k ), Toeplitz, SPD, f(x) Lan. (42) Rat. Lan. (21,22) Rat. Lan. (14,29) exp(x) x exp( x) ln(x) exp(x)/x

47 Example 2 (cont d) f(x) = exp(x)/x. Approximation errors from top to bottom: Lanczos, rational Lanczos for i = 1, 2, f(x)=exp( x)/x; A is positive definite Toeplitz

48 Example 3. Matrix A R stems from the discretization of the differential operator L(u) = 1 10 u xx 100u yy on the unit square. f(x) Lan. (42) Rat. Lan. (21,22) Rat. Lan. (14,29) 1/ x

49 Example 3 (cont d) f(x) = exp(x)/x. Approximation errors from top to bottom: Lanczos, rational Lanczos for i = 1, 2, f(x)=exp( x)/x; A is generated from L(u)

50 Orthogonal rational functions with several fixed poles dµ: nonnegative measure on (part of) the real axis (f,g) = b a f(x)g(x)dµ : inner product P: space of all polynomials with real coefficients { Q = span } 1 : s N, α (x α k ) s k / [a,b] { } space of rational functions with real or complex conjugate poles α k. :

51 Assume poles ordered so that Im(α j ) > 0 Im(α j+1 ) = Im(α j ) Replace for all s = 1, 2,..., 1 (x α j ) s and 1 (x α j+1 ) s by where 1 (x 2 + p j x + q j ) s and x (x 2 + p j x + q j ) s, x 2 + p j x + q j = (x α j )(x α j+1 ), p j,q j R.

52 Define linear space P + Q = span{1, x s, 1 1 (x α k ) s, (x 2 + p j x + q j ) s, x (x 2 + p j x + q j ) s : s N, α k R\[a,b],α j C\R, α k, α j < }.

53 Let Ψ = {ψ 0,ψ 1,ψ 2,... } denote an elementary basis for P + Q: ψ 0 (x) = 1 and ψ l (x), l = 1, 2,..., one of the functions x s, 1 1 x (x α k ) s, (x 2 + p j x + q j ) s, (x 2 + p j x + q j ) s for some positive integers k, j, and s. Gram-Schmidt process applied to the basis Ψ yields basis of orthonormal rational functions Φ = {φ 0,φ 1,φ 2,... }.

54 The recursion relations for the φ j depend on the ordering of the basis functions ψ j of Ψ. We say the ordering of Ψ is natural if for all integers s 1, all real poles α k, and all pairs {p j,q j }, x s 1 x s, 1 (x α k ) 1 s (x α k ) s+1,

55 1 (x 2 + p j x + q j ) x s (x 2 + p j x + q j ) s 1 (x 2 + p j x + q j ) s+1, x (x 2 + p j x + q j ) 1 s (x 2 + p j x + q j ) s+1 x (x 2 + p j x + q j ) s+1.

56 Theorem: Let the basis Ψ = {ψ 0,ψ 1,ψ 2,... } be naturally ordered. Let every sequence of m 1 consecutive basis functions ψ k,ψ k+1,...,ψ k+m1 1 contain at least one power x l, there be at most m 2 basis functions between every pair of functions { } 1 x (x 2 + p j x + q j ) s,, s = 1, 2,... (x 2 + p j x + q j ) s

57 Then the orthonormal Laurent polynomials φ 0,φ 1,φ 2,... satisfy a (2m + 1)-term recurrence relation of the form xφ k (x) = m i= m c k,k+i φ k+i (x), k = 0, 1, 2,..., with m = max{m 1,m 2 + 1}. Here c k,k+i and φ k+i with k + i < 0 are zero.

58 Note: If Q, then we may order the basis Ψ to get the smallest possible value of m, which is 2. This gives a 5-term recursion formula. If Q =, then m = 1 and we obtain the 3-term recusion formula for orthogonal polynomials.

59 Theorem: Let the basis Ψ = {ψ 0,ψ 1,ψ 2,... } be naturally ordered. Let every sequence of m 1 consecutive basis functions ψ k,ψ k+1,...,ψ k+m1 1 contain at least one power (x α l ) t, there be at most m 2 basis functions between every pair of functions { } 1 x (x 2 + p j x + q j ) s,, s = 1, 2,... (x 2 + p j x + q j ) s

60 Then the orthonormal Laurent polynomials φ 0,φ 1,φ 2,... satisfy a (2m + 1)-term recurrence relation of the form 1 x α l φ k (x) = m i= m c (l) k,k+i φ k+i(x), k = 0, 1, 2,..., with m = max{m 1,m 2 + 1}. Here c k,k+i and φ k+i with k + i < 0 are zero.

61 Note: Let P+Q = span{1,x,...,x l,x 1,x l+1,...,x 2l,x 2,x 2l+1,... } This defines a basis Ψ with m 1 = l + 1 and m 2 = 0. Therefore, φ k (x) x satisfies a recursion formula with 2l + 3 terms. Note that satisfies a 5-term recursion. xφ k (x)

62 Short recursion formulas for 1 x 2 + p j x + q j φ k (x), x x 2 + p j x + q j φ k (x) also can be established.

63 Extensions (work in progress): Application to the evaluation of matrix functions f(a)b for nonsymmetric matrices. New derivation of rational Gauss quadrature rules.

64 Muchas Gracias