Bounds for the Matrix Condition Number

Transcription

1 Bounds for the Matrix Condition Number CIME-EMS School, Exploiting Hidden Structure in Matrix Computations. Algorithms and Applications Cetraro June 2015 Sarah W. Gaaf Joint work with Michiel Hochstenbach (TU/e) Sarah W. Gaaf Eindhoven University of Technology 1

2 Uses of κ(a) Consider Ax = b, A R n n, A nonsingular. Problem: A, b perturbed Compute x = A 1 b Accuracy solution x? Sensitivity of linear system: x x κ(a) ( A A + b ) b Sarah W. Gaaf Eindhoven University of Technology 2

3 How to compute the condition number? Matrix 2-norm: A 2 = max x 0 Ax 2 x 2 = σ 1(A) = λ 1(A T A) = λ 1 ([ 0 A T A 0 ]) Matrix condition number: κ(a) = A 2 A 1 2 = σ1(a) σ n(a) = λ 1(A T A) λ n(a T A) Sarah W. Gaaf Eindhoven University of Technology 3

4 Bounds for largest singular value Lower bound θ: Krylov subspace methods (Lanczos bidiagonalization) Upper bounds: Bauer-Fike theorem (1960): There is a singular value in [θ, θ + C]. not guaranteed σ 1 Probabilistic bound [Hochstenbach 2013] Problem: Find upper and lower bound for smallest singular value Sarah W. Gaaf Eindhoven University of Technology 4

5 Extended Lanczos Bidiagonalization Procedure: Equations: v 0 A u 0 A T v 1 A T u 1 A T AV = V H T H (A T A) 1 V = V KK T v k = p k (A T A)v 0 AA T U = UHH T u k = q k (AA T )u 0 (AA T ) 1 U = UK T K Characteristics: H and K are tridiagonal, H K = I p k and q k are Laurent polynomials A 1... K m,m(a T A, v 0) = span{..., (A T A) 1 v 0, v 0, A T Av 0,...}. Sarah W. Gaaf Eindhoven University of Technology 5

6 Proposition: The matrix H is tridiagonal and of the form α 0 β 0 α 1 β 1 α 1 β 1 α 2, (1) β 2 α 2 β 2 α 3... where its entries satisfy h 2j,2j = α j = A T v j 1 = A T u j, h 2j+1,2j = β j = uj T Av j, h 2j+1,2j+1 = α j = uj T Av j, h 2j+1,2j+2 = β j = A T u j (uj T Av j)v j (uj T Av j)v j. Sarah W. Gaaf Eindhoven University of Technology 6

7 Lower bound for κ(a) Extended Lanczos Bidiagonalization: Largest singular value θ 1 of H approximates σ 1(A) Smallest singular value θ k of H approximates σ n(a) Lower bound for condition number: θ 1 θ k κ(a) Sarah W. Gaaf Eindhoven University of Technology 7

8 Probabilistic upper bound Let v 0 = then Thus n γ iy i, (y i right singular vectors of A) i=1 1 = v k 2 = p k (A T A)v 0 2 = 1 γ1 2 p k (σ1) 2 2, and 1 γ 1 p k(σ 2 1). n γi 2 p k (σi 2 ) 2. i=1 Sarah W. Gaaf Eindhoven University of Technology 8

9 p k Recall: 1 γ 1 p k(σ 2 1) Sarah W. Gaaf Eindhoven University of Technology 9

10 p k 1 γ 1 p k (σ 2 1) σ1 2 σup 2 Recall: 1 γ 1 p k(σ 2 1) Question: P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) Sarah W. Gaaf Eindhoven University of Technology 10

11 P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) y 1 v 0 y 2 Sarah W. Gaaf Eindhoven University of Technology 11

12 P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) y 1 γ 1 v 0 y 2 Sarah W. Gaaf Eindhoven University of Technology 12

13 P( 1 < 1 δ γ 1 ) = P( γ1 < δ) = ɛ (ɛ is user-chosen) y 1 v 0 δ y 2 Sarah W. Gaaf Eindhoven University of Technology 13

14 Theorem: starting vector v 0 chosen randomly (uniform distribution over S n 1 ) ε (0, 1) user chosen δ be given by ε = P( γ 1 δ) = n 1 Binc(, 1, 2 2 δ2 ) B inc( n 1, 1, 1), 2 2 where B inc(x, y, z) = z 0 tx 1 (1 t) y 1 dt (incomplete Beta function) Then σup prob, the square root of the largest zero of the polynomial f δ 1 (t) = p k (t) 1 δ, is upper bound for σ 1 with probability at least 1 ε. Sarah W. Gaaf Eindhoven University of Technology 14

15 Conclusions Bounds for the condition number: lower bound: κ low = θ 1 θ k κ(a) probabilistic upper bound: κ(a) σprob 1 = κ σn prob up User chosen values: ε, probabilistic bound holds with probability at least 1 2ε ζ, method adaptively performs k steps such that κ up κ low ζ [SG, M. E. Hochstenbach, Probabilistic bounds for the matrix condition number with extended Lanczos bidiagonalization (Submitted)] Sarah W. Gaaf Eindhoven University of Technology 15

16 Matrix A Dim. κ κ low κ up k CPU LU CPU 1 utm grcar af rajat torso dc xenon scircuit transient stomach For ζ = 2 (i.e. κ up/κ low 2) ε = 0.01 (i.e. upper bound holds with probability at least 98%) CPU 1 indicates time of condest Sarah W. Gaaf Eindhoven University of Technology 16

17 Matrix A Dim. κ κ low κ up k CPU LU utm grcar af rajat torso dc xenon scircuit transient stomach For ζ = 1.1 (i.e. κ up/κ low 1.1) ε = 0.01 (i.e. upper bound holds with probability at least 98%) Sarah W. Gaaf Eindhoven University of Technology 17

18 References Hochstenbach, 2013 Halko, Martinsson, and Tropp, 2011 Jagels and Reichel, 2011 Knizhnerman and Simoncini, 2010 Jagels and Reichel, 2009 Simoncini, 2007 van Dorsselaer, Hochstenbach, and van der Vorst, 2000 Druskin and Knizhnerman, 1998 Kuczyński and Woźniakowski, 1992 Sarah W. Gaaf Eindhoven University of Technology 18