Shifted QR algorithm

Transcription

1 Lecture 5 The QR algorithm. - Practical QR algorithm. - Functions of Matrices. - Google Page Rank. Systems of Non-Linear Equations. - Newtons Method in one dimension: Existance. Convergence Speed. Error Estimate. The Secant Method. Shifted QR algorithm The convergence can be increased by using shifts. A k s k I = Q k R k, A k+1 = R k Q k +s k I. Lemma It holds that A k+1 = Q H k A kq k. Hence A k and A k+1 jas the same eigenvalues. Remark The element (A k ) i,i 1 tend to zero with a linear rate equal to γ = λ i s k λ i 1 s k. Hence if λ i s k we get very fast convergence. 1 2 Shift selection strategies Single shift Select s k = (A k ) n,n. Example 1 We select a Hessenberg matrix A and perform a few QR steps. In Matlab >> A0 = [ ; ; ; ]; >> s=a0(4,4); [Q,R]=qr(A0-s*eye(4)); A1=R*Q+s*eye(4) A = A1 = We perform a few more QR steps. >> s=a3(4,4);[q,r]=qr(a3-s*eye(4));a4=r*q+s*eye(4) A3 = A4 = Note that (A 3 ) 4,3 /(A 4 ) 4, Already fast convergence. Proceedingweobtain (A 5 ) 4,3 = and (A 6 ) 4,3 = The matrix A 1 is Hessenberg and the new shift s 1 =

2 Example 2 We select a new Hessenberg matrix A and perform several QR steps using s k = (A k ) 4,4. In Matlab >> s=a0(4,4); [Q,R]=qr(A0-s*eye(4)); A1=R*Q+s*eye(4);... A0 = A15 = The block A 15 (3:4,3:4) have eigenvalues λ 3,4 = ±1.9484i. Since A and s are real we can t get convergence to a complex eigenvalue. Can still use decoupling. The 2 2 case has an analytic solution. Double shift Select s k as an eigenvalue of the block (A k )(n 1:n,n 1:n). >> s=max(eig(a(3:4,3:4))), [Q,R]=qr(A0-s*eye(4)); A1=R*Q+s*eye(4);... A0 = A5 = i i i i i i i i i i i i Now s 4 = i is complex and (A 5 ) 4,3 = Remark If A is real we want to avoid complex numbers during the computations. 5 6 Practical QR algorithm 1. Hessenberg Reduction A := Hess(A). 2. for k=1,2,... Pick a shift s k. Factorize A s k I = Q k R k Multiply A := R k Q k +s k I. If any element A(j+1,j) < tol then set A(j+1,j)= 0 to obtain ( ) A1 B A :=. 0 A 2 3. end Apply the QR algorithm to A 1 and A 2. If we find a 2 2 block then we can compute the eigenvalues explicitly. Functions of Matrices The Taylor series representation of the scalar function f(t) is f(t) = For any matrix A R n n we define f(a) = k=0 k=0 f (k) (0) t k. k! f (k) (0) A k. k! Proposition If the power series representation of f(t) is absolute convergent for t < L then the corresponding series f(a) is absolute convergent for A < L. 7 8

3 Proposition If A is non defective then f(a) = Xf(D)X 1, where X is the eigenvector matrix, D = diag(λ i ) are the eigenvalues, and f(d) = diag(f(λ i )). Remarks The eigenvalue decomposition offers a relatively cheap and stable way to compute f(a). Since the transformation At + ci preserve the eigenvectors f(at+ic) = Xdiag(f(λ i t+c))x 1. Example If e A is defined by its Taylor series representation and X(t) = e At then X (t) = AX(t) and X(0) = I. So the initial value problem, has the solution y(t) = e At b. y (t) = Ay(t), y(0) = b, Remark In Matlab expm computes the matrix exponential. There are also functions sqrtm, logm, Google Page Rank Google ranks about webb pages (2011). The ability to identify high quality webb pages is a large part of Googles success. i The ranking is based on the link structure of the internet and has to be recomputed often. A Web Crawler downloads webb pages, collects keywoards for indexing, and finds links to, and from, webb pages. All webb pages relevant to a certain search phrase are retrived. They are displayed in the order given by the their PageRank. Each webb page is assigned an index i = 1,...,N. The PageRank r i ( [0,1]) is a quality measure for webb pages. It is based on the set of inlinks I i and outlinks O i. Idea Good webb pages get links from many other good webb pages

4 Definition The Google PageRank is ri for webb page i satisfies, Example 0 ri = 2000 X rj. Nj j Ii Remarks This means that the rank of a page j is divided equally between the its outlinks This is a matrix equation r = Ar, nz = The link structure for the domain Total number of links within the domain is Definition If page j lacks outlinks then change the corresponding column to A(:, j) = e/n, e = (1, 1,..., 1)T. Lemma The largest eigenvalue of the modified Google transition matrix is λmax = 1 and the corresponding eigenvector r has elements 0 ri 1. Remarks The proof is based on probability theory. We need to compute the largest eigenvalue and the corresponding eigenvector of a matrix A of dimension N = Ai,j = 1/Nj, if page j links to page i, 0, otherwise. Note If page j has at least one outlink then the corresponding column A(:, j) sums to 1. A is the Transition matrix. 14 The only realistic choice is the Power Method. The matrix is too large to fit in main memory, or even on one hard drive. In practice the PageRank computation takes about 2 weeks and it is recomputed about once each month. Remarks The idea is good but open to manipulation, e.g. link farms, so some hands-on work is needed. Other search engines are also based on the link structure. Google also promotes local content, uses your previous search history and other things. The site startpage.com forwards your searches to google. 16

5 Non-Linear Equations We want to solve an equation, Example Consider the equations: or, f(x) = f(x) = 0, x Ω. f(x) = x 3 2+e x = 0, ( ) ( ) x 2 1 x x 1 +x 2 = Theorem (Intermediate Value Theorem) If f(x) is continuous on [a, b] and c [f(a),f(b)] then there is a x [a,b] such that f(x) = c. f(a) a f(x) = 0 b f(b) Remark This is a practical solution to the existance of roots f(x) = 0 in one dimension. For systems of equations the situation is more complex. Remarks Often Ω = R n and f is a mapping from R n to R m. Existance, Uniqueness, and Stability is much more complicated than for linear systems Lemma A fixed point x of the Newton iteration, is a root of the equation f(x) = 0. x n+1 = x n f(x n) f (x n ), Example Solve the equation f(x) = e x x = 0 using Newton s Method. n x n f n /f n If it is difficult to compute f (x) we can approximate or, f (x n ) f(x n+h) f(x n h), 2h f (x n ) f(x n) f(x n 1 ). x n x n 1 Remarks The second option is called the Secant method. convergence but doesn t require f (x). Question How to generalize this to systems of non-linear equations, i.e. f(x) = 0, f : R n R m? Slower Convergence Speed? Error estimate? Stability? 19 20

6 Fixed Point Iteration Definition A fixed point iteration can be written as A fixed point satisfies x = ϕ(x ). x k+1 = ϕ(x k ), k = 0,1,2,... Theorem Suppose ϕ(x) has a real fixed point x and that ϕ (x) m < 1 near x. Then lim x k = x. k Theorem If the iteration x k+1 = ϕ(x k ) is convergent then x k+1 x m x k x where x is the fixed point. 21