Bounds for the Matrix Condition Number CIME-EMS School, Exploiting Hidden Structure in Matrix Computations. Algorithms and Applications Cetraro 22-26 June 2015 Sarah W. Gaaf Joint work with Michiel Hochstenbach (TU/e) Sarah W. Gaaf Eindhoven University of Technology 1
Uses of κ(a) Consider Ax = b, A R n n, A nonsingular. Problem: A, b perturbed Compute x = A 1 b Accuracy solution x? Sensitivity of linear system: x x κ(a) ( A A + b ) b Sarah W. Gaaf Eindhoven University of Technology 2
How to compute the condition number? Matrix 2-norm: A 2 = max x 0 Ax 2 x 2 = σ 1(A) = λ 1(A T A) = λ 1 ([ 0 A T A 0 ]) Matrix condition number: κ(a) = A 2 A 1 2 = σ1(a) σ n(a) = λ 1(A T A) λ n(a T A) Sarah W. Gaaf Eindhoven University of Technology 3
Bounds for largest singular value Lower bound θ: Krylov subspace methods (Lanczos bidiagonalization) Upper bounds: Bauer-Fike theorem (1960): There is a singular value in [θ, θ + C]. not guaranteed σ 1 Probabilistic bound [Hochstenbach 2013] Problem: Find upper and lower bound for smallest singular value Sarah W. Gaaf Eindhoven University of Technology 4
Extended Lanczos Bidiagonalization Procedure: Equations: v 0 A u 0 A T v 1 A T u 1 A T AV = V H T H (A T A) 1 V = V KK T v k = p k (A T A)v 0 AA T U = UHH T u k = q k (AA T )u 0 (AA T ) 1 U = UK T K Characteristics: H and K are tridiagonal, H K = I p k and q k are Laurent polynomials A 1... K m,m(a T A, v 0) = span{..., (A T A) 1 v 0, v 0, A T Av 0,...}. Sarah W. Gaaf Eindhoven University of Technology 5
Proposition: The matrix H is tridiagonal and of the form α 0 β 0 α 1 β 1 α 1 β 1 α 2, (1) β 2 α 2 β 2 α 3... where its entries satisfy h 2j,2j = α j = A T v j 1 = A T u j, h 2j+1,2j = β j = uj T Av j, h 2j+1,2j+1 = α j = uj T Av j, h 2j+1,2j+2 = β j = A T u j (uj T Av j)v j (uj T Av j)v j. Sarah W. Gaaf Eindhoven University of Technology 6
Lower bound for κ(a) Extended Lanczos Bidiagonalization: Largest singular value θ 1 of H approximates σ 1(A) Smallest singular value θ k of H approximates σ n(a) Lower bound for condition number: θ 1 θ k κ(a) Sarah W. Gaaf Eindhoven University of Technology 7
Probabilistic upper bound Let v 0 = then Thus n γ iy i, (y i right singular vectors of A) i=1 1 = v k 2 = p k (A T A)v 0 2 = 1 γ1 2 p k (σ1) 2 2, and 1 γ 1 p k(σ 2 1). n γi 2 p k (σi 2 ) 2. i=1 Sarah W. Gaaf Eindhoven University of Technology 8
p k Recall: 1 γ 1 p k(σ 2 1) Sarah W. Gaaf Eindhoven University of Technology 9
p k 1 γ 1 p k (σ 2 1) σ1 2 σup 2 Recall: 1 γ 1 p k(σ 2 1) Question: P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) Sarah W. Gaaf Eindhoven University of Technology 10
P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) y 1 v 0 y 2 Sarah W. Gaaf Eindhoven University of Technology 11
P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) y 1 γ 1 v 0 y 2 Sarah W. Gaaf Eindhoven University of Technology 12
P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) y 1 v 0 δ y 2 Sarah W. Gaaf Eindhoven University of Technology 13
Theorem: starting vector v 0 chosen randomly (uniform distribution over S n 1 ) ε (0, 1) user chosen δ be given by ε = P( γ 1 δ) = n 1 Binc(, 1, 2 2 δ2 ) B inc( n 1, 1, 1), 2 2 where B inc(x, y, z) = z 0 tx 1 (1 t) y 1 dt (incomplete Beta function) Then σup prob, the square root of the largest zero of the polynomial f δ 1 (t) = p k (t) 1 δ, is upper bound for σ 1 with probability at least 1 ε. Sarah W. Gaaf Eindhoven University of Technology 14
Conclusions Bounds for the condition number: lower bound: κ low = θ 1 θ k κ(a) probabilistic upper bound: κ(a) σprob 1 = κ σn prob up User chosen values: ε, probabilistic bound holds with probability at least 1 2ε ζ, method adaptively performs k steps such that κ up κ low ζ [SG, M. E. Hochstenbach, Probabilistic bounds for the matrix condition number with extended Lanczos bidiagonalization (Submitted)] Sarah W. Gaaf Eindhoven University of Technology 15
Matrix A Dim. κ κ low κ up k CPU LU CPU 1 utm5940 5940 4.35 10 8 3.98 10 8 7.21 10 8 4 0.13 61 0.12 grcar10000 10000 3.63 10 0 3.59 10 0 5.80 10 0 6 0.07 31 0.05 af23560 23560 1.99 10 4 1.93 10 4 2.82 10 4 6 0.98 74 0.88 rajat16 96294 5.63 10 12 5.69 10 12 5 9.34 97 9.19 torso1 116158 1.41 10 10 1.42 10 10 3 26.8 93 28.5 dc1 116835 2.39 10 8 4.59 10 8 5 6.05 93 5.57 xenon2 157464 4.29 10 4 8.14 10 4 7 20.1 82 19.6 scircuit 170998 2.40 10 9 4.69 10 9 7 2.05 54 1.39 transient 178866 1.02 10 11 2.00 10 11 8 7.70 86 7.12 stomach 213360 4.62 10 1 9.02 10 1 6 13.8 80 13.7 For ζ = 2 (i.e. κ up/κ low 2) ε = 0.01 (i.e. upper bound holds with probability at least 98%) CPU 1 indicates time of condest Sarah W. Gaaf Eindhoven University of Technology 16
Matrix A Dim. κ κ low κ up k CPU LU utm5940 5940 4.35 10 8 4.35 10 8 4.71 10 8 10 0.19 42 grcar10000 10000 3.63 10 0 3.62 10 0 3.97 10 0 13 0.13 21 af23560 23560 1.99 10 4 1.99 10 4 2.12 10 4 9 1.14 66 rajat16 96294 5.63 10 12 5.69 10 12 5 9.34 97 torso1 116158 1.41 10 10 1.42 10 10 3 26.8 93 dc1 116835 2.39 10 8 2.45 10 8 8 6.52 91 xenon2 157464 4.32 10 4 4.67 10 4 14 23.6 70 scircuit 170998 2.45 10 9 2.67 10 9 16 3.28 33 transient 178866 1.03 10 11 1.11 10 11 21 9.47 70 stomach 213360 4.82 10 1 5.24 10 1 14 17.5 63 For ζ = 1.1 (i.e. κ up/κ low 1.1) ε = 0.01 (i.e. upper bound holds with probability at least 98%) Sarah W. Gaaf Eindhoven University of Technology 17
References Hochstenbach, 2013 Halko, Martinsson, and Tropp, 2011 Jagels and Reichel, 2011 Knizhnerman and Simoncini, 2010 Jagels and Reichel, 2009 Simoncini, 2007 van Dorsselaer, Hochstenbach, and van der Vorst, 2000 Druskin and Knizhnerman, 1998 Kuczyński and Woźniakowski, 1992 Sarah W. Gaaf Eindhoven University of Technology 18