Ax=b. Zack 10/4/2013

Transcription

1 Ax=b Zack 10/4/2013

2 Iteration method Ax=b v 1, v 2 x k = V k y k Given (A, b) standard orthonormal base v 1, v 2 x k = V k y k For symmetric A, the method to generate V k is called Lanczos Method

3 Lanczos Goal of Lanczos: Find an orthonormal base in span(b, Ab A k 1 b) Note: this orthonormal base is unique. At k+1 iteration step, we want to generate v k+1, we have: β k+1 v k+1 = γ 0 b + γ 1 Ab + γ 2 A 2 b γ k 1 A k 1 b + A k b v i T v k+1 = 0 i k v k T v k+1 = 1 So v k+1 is unique.

4 Lanczos: Algorithm Given V k = v 1, v 2 v k β k+1 v k+1 = γ 0 b + γ 1 Ab + γ 2 A 2 b γ k 1 A k 1 b + γ k A k b = γ 1 v 1 + γ 2 v 2 γ k v k + γ k A k b = z 1 v 1 + z 2 v 2 z k v k + Av k = Av k + V k z Where z is a vector, β is a parameter to uniformization vector v

5 Lanczos: Algorithm β k+1 v k+1 = Av k + V k z v k T v k+1 = 0 v k T Av k + v k T V k z = 0 e k T z = v k T Av k v i T v k+1 = 0 i < k v i T Av k + v i T V k z = 0 e i T z = v i T Av k

6 Lanczos: Algorithm Notice that we have: β i+1 v i+1 = Av i + V i z v k T β i+1 v i+1 = v k T Av i + v k T V i z if i = k 1 β k = v k T Av i else 0 = v k T Av i e i T z = v i T Av k = v k T Av i = β k (i = k 1) 0 (i < k 1)

7 Lanczos: Algorithm β k+1 v k+1 = Av k v k T Av k v k β k v k 1 define α k = v k T Av k Av k = β k+1 v k+1 + α k v k + β k v k 1 AV k = V k+1 H k α 1 β 2 β 2 α 2 β 3 where, H k = β k α k β k+1

8 Lanczos: Algorithm We can also define a tridiagonal matrix T k : α 1 β 2 β T k = 2 α 2 β 3 ; β k α k AV k = V k T k + β k+1 v k+1 e k T Note: theoretically, iteration will stop when k = l < n, but in fact, l is always far more bigger than n, we need to set a reasonable stop rule.

9 Lanczos: Algorithm Define x k = V k y k is approximation of x. residual vector r k b Ax k = β 1 v 1 AV k y k = V k+1 β 1 e 1 H k y k = V k+1 t k+1 where t k+1 = β 1 e 1 H k y k

10 Choose Optimal x k Method Definition of optimal x k CG 2 min r k A 1 MINRES min r k 2 2 SYMMLQ min x x k 2 2

11 CG minimize function 2 r k A 1 = r k T A 1 r k 2 r k A 1 = b Ax k T A 1 b Ax k = b AV k y k T A 1 b AV k y k 2 r k A 1 has a stationary value at y k if V T k AV k y k = V T k b V T k (V k T k + β k+1 v k+1 e T k )y k = V T k β 1 v 1 T k y k = β 1 e 1

12 T k y k = β 1 e 1 Use LDL T decomposition, T k = L k D k L k T. L k is a bidiagonal lower matrix, D k is a diagonal matrix. L k D k L k T y k = β 1 e 1

13 Define z k = L k T y k, L k D k z k = β 1 e 1 Define W k T = L k 1 V k T x k = V k y k = W k L k T y k = W k z k

14 L L W V T T k 1 k 1 k 1 k 0 T T k 1 wk vk w w v T T T k k 1 k k T 0 0 L T k1 Dk 1 T T L D L Lk 1 k Lk 1Dk 1Lk 1 kdk 1 0 k 1 d k kdk 1 dk k d k 1 k k k k d d k k 1 k d 2 k k k 1 k L D z k k k Lk 1 Dk 1 z L k 1 k 1D k 1 zk 1 = e 0 1 d 0 d d k k k k k k 1 k 1 1 d d k k 1 k 1 k k 0 zk 1 xk Wk zk Wk 1 wk Wk 1zk 1 wkk xk 1 wkk k

15 A, b, v k 2, v k 1 v k (Lanczos) d k 1, β k λ k d k 1, λ k, α k d k (LDL decomposition) λ k, w k 1, v k w k (W T k = L 1 k V T k ) d k 1, d k, λ k, ζ k 1 ζ k (z k = L T k y k ) x k 1, ζ k, w k x k (x k = W k z k )

16 Problem in CG method A should be positive definite If A is indefinite, in practice, CG would still give out answer but no longer be relied upon numerically. It used LDL decomposition if A is an indefinite symmetric matrix, LDL T decomposition can be tried, often success, but it does not always exist.

17 Method Definition of Subproblem Factorization Estimate of x optimal x k CG 2 T min r k A 1 T k y k = β 1 e 1 T k = L k D k L k x k C = V k y k MINRES min r k 2 2 min β 1 e 1 H k y k Qk H k = R k 0 x k M = V k y k SYMMLQ min x x k 2 2 min y k s.t. H k 1 H k 1 T y k = β 1 e 1 = L k 1 0 T T Q k 1 x k L = V k y k

18 CG A must be positive definite matrix MINRES Any symmetric A, r M k decreases monotonically Risk of cancellation error when A is indefinite SYMMLQ QR factor, Any symmetric A, except Ax=b must be consistent, Very little cancellation error, method of choice for indefinite consistent system

19

20 Unsymmetric and rectangular iteration Set α, β as normalized parameters of u, v, β 1 u 1 = b, α 1 v 1 = A T u 1 follow this iteration: u Av u k 1 k 1 k k k v A u v T k 1 k 1 k 1 k 1 k generate 2 orthonormal base U k = u 1 u 2 u k V k = v 1 v 2 v k

21 Unsymmetric and rectangular iteration PROOF: assume that we have orthonormal vectors u 1 u 2 u k, v 1 v 2 v k generate u k+1, v k+1 as: u Av u k 1 k 1 k k k v A u v T k 1 k 1 k 1 k 1 k

22 Unsymmetric and rectangular iteration u u u Av u u T T T i k 1 k 1 i k i k k T T T T vk ivi vk A ui vk ivi When i=k 1 u u u Av u u v v + v v u u 0 T T T T T T k k 1 k 1 k k k k k k k k k k k 1 k k k When i<k u u u Av u u = v v + v v u u =0 T T T T T T i k 1 k 1 i k i k k k i i k i i1 i k k So u k u u, we can also prove that v k1 v1 vk 1 1 k

23 Unsymmetric and rectangular iteration AV k = U k+1 B k = U k L k + β k+1 u k+1 e k T A T U k+1 = V k B T T k + α k+1 v k+1 e k+1 T = V k+1 L k+1 L k = α 1 β 2 α 2 B k = β k α k α 1 β 2 α 2 β k α k β k+1

24 Direct method: LUSOL Factor: l A / A u T : j A i: T ij L L l U U u A A lu T

25 Direct method: LUSOL Pivot strategies 1. Preserve sparsity

26 Direct method: LUSOL a Markowitz strategies is used to select potential pivot A ij. Pivot should have a low Markowitz merit function M ij r i 1 c j 1 where r i and c j are number of nonzero element in row i and column j.

27 The sparsest columns and rows are searched in turn: columns of length 1,rows of length 1, then columns of length 2, rows of length 2, and so on).

28 Direct method: LUSOL 2. Preserve stability. check (i,j) given by Markowitz strategies with stability test: Strategy Name Stability test Threshold Partial Pivoting TPP l Ltol Threshold Rook Pivoting TRP l and u/a ij Ltol Threshold Complete Pivoting TCP A max Ltol A ij

29 Example, Ltol=3.0 Name Stability test TPP l Ltol TRP l and u/a ij Ltol TCP A max Ltol A ij

30 Thank you