How to Detect Definite Hermitian Pairs

Transcription

1 How to Detect Definite Hermitian Pairs Françoise Tisseur School of Mathematics The University of Manchester Joint work with Chun-Hua Guo and Nick Higham 17th Scottish Computational Mathematics Symposium, Edinburgh, September 10, 2008

2 Hermitian Positive Definite Matrix A C n n is Hermitian positive definite if x Ax > 0 for all x 0. In the eigenproblem Ax = λx, real, positive eigenvalues, unitary matrix of eigenvectors. Testing for definiteness: attempt a Cholesky decomposition, A = R R. Françoise Tisseur Definite Pairs 2 / 28

3 Generalized Eigenproblem Ax = λbx with A and B Hermitian and B positive definite. Equivalent to the standard eigenproblem Hy B 1/2 AB 1/2 y = λy. All eigenvalues real. A and B are simultaneously diagonalizable. These properties extend to definite pairs (A, B) defined by γ(a, B) := min z C n (z Az) 2 + (z Bz) 2 > 0. z 2 =1 γ is the Crawford number. Françoise Tisseur Definite Pairs 3 / 28

4 Definite Pairs Theorem Let (A, B) be a definite Hermitian pair, and for t R let A(t) = A cos t B sin t, B(t) = A sin t + B cos t. Then there is a t [0, 2π) s.t. B(t) > 0 and γ(a, B) = λ min (B(t)). Note that the eigenproblem A(t)x = µb(t)x share the same e vecs as Ax = λbx and their e vals are related by λ = µ cos t + sint cos t µ sin t. Françoise Tisseur Definite Pairs 4 / 28

5 Why Test for Definiteness? Both theoretical and computational advantages accrue in associated eigenproblem Ax = λbx: eigenvalues are real, A and B simultaneously diagonalizable, if t known s.t. B(t) = A sin t + B cos t > 0 then definiteness of B(t) is exploited to compute e vals. Detect hyperbolicity of quadratic matrix polynomials. Allow CG iterations for nonsymmetric saddle point linear systems. Françoise Tisseur Definite Pairs 5 / 28

6 Hyperbolic Quadratics Q(λ) = λ 2 A + λb + C, A, B, C C n n is hyperbolic if A, B, C Hermitian, A > 0, and (x Bx) 2 > 4(x Ax)(x Cx) for all nonzero x C n. Eigenvalues are all real and semisimple. Q > 0 Q indef. Q < 0 Q indef. Q > 0 + λ 2n λ n+1 λ n λ 1 Q is hyperbolic iff Q(µ) < 0 for some µ R. Françoise Tisseur Definite Pairs 6 / 28

7 Testing for Hyperbolicity Theorem Q(λ) = λ 2 A + λb + C with A, B and C Hermitian and A > 0 is hyperbolic iff the pair (A, B) is definite, where [ ] [ ] C 0 B A A =, B =. 0 A A 0 Proof. (A, B) definite iff αa+βb > 0, α 2 +β 2 = 1. For α 0 we have [ I β αa + βb = I ] [ α αc βb β 2 A 0 ] [ ] α I 0 0 I 0 αa β I I. α So αa + βb is congruent to α diag( Q(µ), A), µ = β/α. Françoise Tisseur Definite Pairs 7 / 28

8 Solving Hyperbolic Q(λ) = λ 2 A + λb + C Let µ s.t. Q(µ) < 0. Hence Q µ (λ) Q(λ + µ) = λ 2 A + λ(b + 2µA) + C + µb + µ 2 A = λ 2 A µ + λb µ + C µ, with C µ = Q(µ) < 0 and A µ = A > 0. The pencils [ ] Aµ 0 λ + 0 C µ [ ] Bµ C µ, λ C µ 0 [ ] [ ] 0 Aµ Aµ 0 + A µ B µ 0 C µ are both Hermitian definite linearizations of Q µ (they share the same e vals as Q). Françoise Tisseur Definite Pairs 8 / 28

9 Linear Systems in Saddle Point Problems Involve matrices of the form [ A B A = B C where A = A T R n n, A > 0 and C = C T R m m, C 0. A is indefinite: it has n positive e vals and rank(c + BA 1 B T ) negative e vals. Theorem (Liesen & Parlett 08) [ ] In 0 If the symmetric pair (A, J ) with J = is definite 0 I m and µ R s.t. A µj > 0 is known then there exists a well-define CG method for solving linear systems with J A. ], Françoise Tisseur Definite Pairs 9 / 28

10 How can we efficiently test whether (A, B) is definite, i.e., whether γ(a, B)>0? γ(a, B) := min (z Az) 2 + (z Bz) 2 z C n z 2 =1 = min z (A + ib)z z C n z 2 =1 ( ) = max max 0 t 2π λ min (A cos t + B sin t), 0. One-dimensional global optimization problem. Unfortunately, not a convex (or concave) maximization problem. Françoise Tisseur Definite Pairs 10 / 28

11 Numerical Methods for Testing Definiteness J-orthogonal Jacobi algorithm [Veselic, 1993]. Bisection algorithm [Higham, T & Van Dooren, 2002]. Level set algorithm [Higham, T & Van Dooren, 2002]. Chebfun package. Crawford & Moon algorithm [1983], improved [Guo, Higham & T, 2008]. Françoise Tisseur Definite Pairs 11 / 28

12 Finding Global Maximum Our aim is to compute max λ ( ) min(a cos t + B sin t) = maxλ min M(z), 0 t 2π z =1 where M(z) = (z 1 C + zc )/2 and C = A + ib Given ξ, is ξ = λ min (M(z)) for some z on unit circle? Françoise Tisseur Definite Pairs 12 / 28

13 Finding Global Maximum Our aim is to compute max λ ( ) min(a cos t + B sin t) = maxλ min M(z), 0 t 2π z =1 where M(z) = (z 1 C + zc )/2 and C = A + ib Given ξ, is ξ = λ min (M(z)) for some z on unit circle? For nonzero z, det(m(z) ξi) = 0 det(c 2ξzI + z 2 C ) = 0. For each e vals z j of C 2ξzI + z 2 C on the unit circle test if λ min (M(z j )) = ξ. Françoise Tisseur Definite Pairs 12 / 28

14 Bisection Algorithm Given Hermitian pair (A, B), determine whether (A, B) is definite. a = max{λ min (A), λ max (A),λ min (B), λ max (B)}. b = σ n ([ A B ]) if a > 0, quit (pair is definite), end if b 0, quit with a = b = 0 (pair is indefinite), end while b a > tol ξ = (a + b)/2 Compute eigenvalues z j of Q(z). If λ min (M(z j )) = ξ for some z j on unit circle a = ξ if a > 0,quit (pair is definite), end else b = ξ if b 0, quit with a = b = 0 (pair is indefinite), end end end Françoise Tisseur Definite Pairs 13 / 28

15 Bisection Algorithm, cont. Produces a bracket for γ(a, B) shrinking to zero. Does not compute t [0, 2π) such that B(t) > 0. Each iteration requires solution of a QEP and up to 2n Hermitian eigenproblems so is expensive. Françoise Tisseur Definite Pairs 14 / 28

16 Level Set Approach E vals of B(t) = A cos t + B sin t are piecewise analytic functions of t. Theorem (A, B) is definite if B(t) has exactly n consecutive increasing zero crossings followed by n consecutive decreasing zero crossings. The zero crossings are determined by the unit modulus e vals of Q(z) = z 2 C + C, where C = A + ib. The sign of t λ j( B(t) ) tk determines if λ j ( B(t) ) increases or decreases at each zero crossing. Françoise Tisseur Definite Pairs 15 / 28

17 Example A and B are random, 5 5, real and symmetric. Plot eigenvalues of A cosθ + B sin θ for θ [ π,π]: Eigenvalues of Acosθ+Bsinθ θ Françoise Tisseur Definite Pairs 16 / 28

18 Level Set Algorithm Given Hermitian pair (A, B), determine whether (A, B) is definite. C = A + ib Compute the eigenvalues of z 2 C + C. If there are 2n eigenvalues of unit modulus Compute the derivatives (using associated e vecs). If there are n consecutive increasing and n consecutive decreasing zero crossings The pair is definite; return end end The pair is not definite. Return an interval of t such that B(t) > 0. Françoise Tisseur Definite Pairs 17 / 28

19 Chebfun ( ) γ(a, B) = max max λ min(a cos t + B sin t), 0. 0 t 2π Use new Chebfun toolbox to solve the 1-variable minimization problem. f min(eig(cos(t)*a+sin(t)*b)); fc = chebfun(f,[0 2*pi]); [maxf,t opt] = max(fc); test = (maxf > 0) gamma = max(max(maxf,0)) Françoise Tisseur Definite Pairs 18 / 28

20 Algorithm of Crawford and Moon Bisection-like algorithm. Received little attention in the literature. PDFIND: Algorithm 646, ACM TOMS, Lack of clarity in the derivation and statement of alg and in the explanation of its properties. Our contributions: Give clearer derivation with no assumptions on definiteness of (A, B). Prove convergence for all (A, B) definite or not. Show that PDFIND is numerically unreliable. Formulate an improved algorithm. Françoise Tisseur Definite Pairs 19 / 28

21 Notations For a Hermitian pencil (A, B) define f(x) = x (A + ib)x x (A + ib)x, x Cn, x (A + ib)x 0. f(x) lies on the unit circle. For a, b of unit modulus, arc[a, b] is the shorter arc on the unit circle connecting a and b. θ[a, b] is the length of the arc [a, b]. When a = b, define arc[a, b] = π. Françoise Tisseur Definite Pairs 20 / 28

22 Range of f(x) = x (A + ib)x/ x (A + ib)x Property 1: When (A, B) is definite the range of f denoted by arc[ α, β] is a closed arc on the unit circle of length θ[ α, β] < π. Property 2 If c = e it and B(t) = A sin t + B cos t 0 then for any x 0 with x B(t)x 0 and x (A + ib)x 0, we have π θ[f(x), c] π/2. Property 3 If (A, B) is definite and c = cos t + i sin t is the midpoint of arc[ α, β] then B( t) > 0. Françoise Tisseur Definite Pairs 21 / 28

23 Pictorial Explanation b a c a b Françoise Tisseur Definite Pairs 22 / 28

24 Crawford and Moon algorithm Input: A and B; starting arc [a, b]. Output: t [0, 2π) when (A, B) definite. (Main loop) 1 c = (a + b)/ a + b = sin t + i cos t 2 if B(t) > 0, quit (pair definite), end 3 Find unit norm vector x s.t. x B(t)x 0. 4 d = f(x) 5 if d is in arc[ a, b] and d a 6 b = d, goto line 1 7 elseif d is in arc[a, b] and d b 8 a = d, goto line 1 9 else 10 quit (pair is indefinite) 11 end Françoise Tisseur Definite Pairs 23 / 28

25 Convergence Lemma On the kth step, θ[a k, b k ] π(1 2 k ), regardless of whether the pair (A, B) is definite or not. Unless alg. converges in finite number of steps, arc[a k, b k ] [π(1 2 k ),π] k. Hence sequence converges to π. (A, B) declared numerically indefinite if no determination is made in k steps and 2 k u (unit roundoff). Françoise Tisseur Definite Pairs 24 / 28

26 An Improved Crawford and Moon algorithm New starting phase; no assumption on definiteness of (A, B). (Main loop) 1 c = (a + b)/ a + b = sin t + i cos t 2 if B(t) > 0, quit (pair definite), end 3 Find unit norm vector x s.t. x B(t)x 0. 4 if x (A + ib)x = 0, quit (pair is indefinite), end 5 d = f(x) 6 if d is in arc[ a, b] and d a 7 b = d, goto line 1 8 elseif d is in arc[a, b] and d b 9 a = d, goto line 1 10 else 11 quit (pair is indefinite) 12 end Françoise Tisseur Definite Pairs 25 / 28

27 Testing Definiteness, Negative Curvature Is B(t) = A sin t + B cos t > 0? If not how to compute x 0 s.t. x B(t)x 0? Definiteness test: run Cholesky alg. until it completes or encounter a negative or zero pivot. Direction of negative curvature: at step k of Cholesky, [ ] k n k ( k n k R C = 11 ( ) k 0 0 R 11 R 12 + n k 0 S k R 12 If s (k) 11[ 0 factorization breaks down and R 1 x = 11 R ] 12 e I 1 satisfies x B(t)x 0. can suffers from numerical instability. Françoise Tisseur Definite Pairs 26 / 28 ),

28 Unreliability of Cholesky Approach: Example R = [ ] , A = R R + e 3 e T 4 + e 4 e T 3, B = diag(0, 1, 1, 1). γ(a, B) = 0.75 so the pair is safely definite. Crawford and Moon alg. attempt to Cholesky factorize B(π/2) = A which is indefinite. In exact arithmetic, x B(pi/2)x = 0. In floating point arithmetic, the computed x is s.t. fl( x B(π/2)x) = > 0, and the alg. breaks down. Françoise Tisseur Definite Pairs 27 / 28

29 Concluding Remarks Determining whether a given pair (A, B) is definite or not is not straightforward. Both theoretical and computational advantages accrue t such that A sin t + B cos t > 0 is known. The (modified) Crawford and Moon algorithm is more efficient than other numerical methods for testing definiteness. Work in progress: numerically stable way of computing the direction of negative curvature. For papers and Eprints, Françoise Tisseur Definite Pairs 28 / 28