Math 502 Fall 2005 Solutions to Homework 5. Let x = A ;1 b. Then x = ;D ;1 (L+U)x +D ;1 b, and hence the dierences

Transcription

1 Math 52 Fall 25 Solutions to Homework 5 (). The i-th row ofd ; (L + U) is r i =[ a i ::: a i i; a i i+ ::: a i M ] a i i a i i a i i a i i Since A is strictly X row diagonally dominant jr i j = j a i j j = X ja i j j < i = ::: M: a j6=i i i ja i i j j6=i Therefore kd ; (L + U)k = max jr ij < : im Let x = A ; b. Then x = ;D ; (L+U)x +D ; b, and hence the dierences e (n) = x (n) ; x satisfy e (n+) = ;D ; (L + U)e (n). By induction ke (n) k ; kd ; (L + U)k n ke () k n : Therefore x (n)! x,asn!. 2. ( ) ) Suppose ^x 2 K minimizes (x) over K. Lety 2 K be any other point in K and consider F (s) =(^x + sy), s 2 R. Clearly F () = (^x) F (s) for all s. Thus F () =. Since F (s) =y T r(^x + sy) this shows that for every y 2 K, y T r(^x) =. Hence r(^x)? K. ( ( ) Suppose r(^x)? K. Let x 2 K be another point ink. Then x ; ^x = y 2 K. Using the fact that r(^x) =A^x ; b we have (x) =(^x + y) = 2 (^x + y)t A(^x + y) ; (^x + y) T b 2 ^xt A^x + y T A^x + 2 yt Ay ; x T b ; y T b (^x)+y T r(^x)+ 2 yt Ay: Since r(^x)? K and A is positive denite, this shows (^x) (x), for all x 2 K. 3. The i j entry (AU) i j of the product is computed by the expression (AU) i j = ;u i; j ; u i+ j +4u i j ; u i j; ; u i j+ : If we assume, for convenience in counting, u j = u M+ j = for j = ::: M and u i = u i M+ =fori = ::: M then () is valid for i j M. Since the evaluation of () requires multiplication and 4 subtractions (inverse additions) computing all components in this way requires M 2 multiplications and 4M 2 subtractions. If we now eliminate the padded zeros, of which there are 4M, this reduces the number of subtractions to 4M 2 ; 4M. It is also easy to count ops using the block structure of A. Multiplication by T requires M multiplications and 2M ; 2 subtractions, and this is done M times. The o diagonal blocks of ;I require M subtractions for the rst and last blocks, and 2M subtractions for the other M ; 2blocks. The total is the same, M 2 multiplications and 4M 2 ; 4M subtractions. Earlier in the semester we showed that if A is an (m m) matrix and x is an (m ) column vector then computing Ax required m 2 multiplications and m 2 ; m additions. In the present context m = M 2,sotocomputeBU requires M 4 multiplications and M 4 ; M 2 additions.

2 2 4. Consider f(x) =; cos(x), x 2 [ ]. Using basic calculus we ndf(x) is strictly increasing with f() = and f() = 2. Also, if <a<b<then f(a) =minff(x) : x 2 [a b]g and f(b) = maxff(x) : x 2 [a b]g. From these observations we obtain ; cos( i M ) ; cos( ) ; cos( ) i = ::: M: M+ M+ M+ By a trigonometric identity Therefore cos( M (M+;) ) = cos( )=; cos( ): M+ M+ M+ min = min i j = 4( ; cos( i jm max = )) M+ max i j = 4( ; cos( M )) = 4( + cos( )): i jm M+ M+ Hence the spectral radius and 2-norm condition number are (A) = max = 4( + cos( )) M+ 2(A) = max = + cos( min ; cos( Since cos x ; 2 x2 for x near it follows that for M large so that M+ ) M+ ): + cos( M+ ) 2 ; 2 2(M +) 2 ; cos( M+ ) 2 2(M +) 2 (A) 8 ; 4 2 (M +) 4 2(A) 2 2 (M +) : 2 5. For convenience we simplify the notation describing the iteration to x k+ =(I ; A)x k + v with x = Av= v: Then x = v, x 2 =(I ; A)v + v =(; )v + v, and x 3 =(I ; A)[( ; )v + v]+v =(; ) 2 v +(; )v + v: Clearly each of these has the form x k = q k ()v, whereq k () is a polynomial of degree k ;. We show by induction that this holds in general with q k () =+(; )++(; ) k; = k; X j= ( ; ) j : We have already veried this for k = 2 3. Suppose it's valid for x k. Then x k+ =(I ; A)q k ()v + v = q k ()( ; )v + v P k and it follows that q k+ ()=+(; )q k () = ( ; j= )j. The sequence fx k g will converge if and only if lim q k() = k! X j= ( ; ) j is a convergent power series. Since this is a geometric series with ratio r = ; we know itconverges if and only if j ; j <, in which casethesumis=. In the context of the original problem is one of the eigenvalues of A, all of which lie in the interval ( 8). If i j 2 ( 2) and F = V i j then the

3 3 iteration will converge. However, the majority of the eigenvalues lie in the interval (2 8) and if F is a corresponding eigenvector then the iteration will not converge. Any F 2 R m can be represented in terms of the basis of eigenvectors fv i j g M i j=.from our work with projections we know F = MX i j= (Vi jf T )V i j : Using this F in the iteration scheme we obtain U (k) = h 2 M X i j= (Vi jf T )q k ( i j )V i j : If Vi j T F 6= for some eigenvector with eigenvalue i j 2 (2 8) then the sequence will not converge. In particular, if F is the column vector of ones the sequence will not converge. (You need to have explicit formulas for the eigenvectors to actually show this. Since I didn't provide them the general argument is sucient.) 6. The diagonal matrix D =4I, where I is the (m m) identity, is the diagonal portion of A in the splitting A = L+D+U. Using this we nd L+U = A;4I and D ; (L + U) = 4 (A ; 4I). Therefore the eigenvalues of D; (L + U) are ( i j ; 4)=4 where i j is an eigenvalue of A. From a problem 4 we knowthe minimum and maximum eigenvalues of A satisfy < min < 4 < max < 8, so that the eigenvalues of D ; (L + U) lie in (; ), with the smallest and largest being ( 4 min ; 4) = ; cos( ) ( M+ 4 max ; 4) = cos( ): M+ Clearly these have the same absolute value, so that the spectral radius of D ; (L + U) is (D ; (L + U))=cos( M+ ): Note that in general (A) = (;A). Using calculus it is easy to show cos x =; 2 x2 + O(x 4 ), as x!. Hence (D ; (L + U)) = ; 2 2(M +) 2 + O((M +);4 ) as M!: Thus (D ; (L + U)) approaches quadratically in h =(M +) ;. Below is a table showing the depence of (D ; (L + U)) on M. M m = M 2 (D ; (L + U))

4 4 7. From problem 4 we know 2 (A) = (+cos )=(;cos ), where = =(M+). Using the trigonometric identities + cos = 2 cos 2 (=2) and ; cos = 2sin 2 (=2) we have p 2 (A) = cos(=2) sin(=2) since 2 ( ). Therefore C = p 2 (A) ; cos(=2) ; sin(=2) p = 2 (A)+ cos(=2) + sin(=2) = cos( ) ; sin( 2(M+) cos( ) + sin( 2(M+) By using calculus we obtain cos(=2) ; sin(=2) cos(=2) + sin(=2) =; + O(2 ) as! + : Hence C = p 2 (A) ; p 2 (A)+ =; ) 2(M+) ): 2(M+) M + + O((M +);2 ) as M!: Thus C approaches linearly in h =(M +) ;.Belowisatableshowing the depence of C on M. M m = M 2 C A script for Jacobi iteration is: % Script File: Jacobi amp = 6 M = 3 h = /(M+) max_loops = 5*M*M epsilon = h*h rho = cos(pi/(m+)) x = (:M)*h y = x u = zeros(m,m) % initialize exact solution and f = zeros(m,m) % right hand side of differential equation for i = :M for j = :M u(i,j) = amp*x(i)*(-x(i))*y(j)*(-y(j)) f(i,j) = 2*amp*(x(i)*(-x(i)) + y(j)*(-y(j)))

5 5 U = zeros(m+2,m+2) V = zeros(m+2,m+2) F = (h^2)*f % approximate solution % temporary storage % right hand side of linear system err = zeros(max_loops,) e_start = max(max(abs(u - U(2:M+,2:M+)))) % data for plotting k = e = e_start while ((e > epsilon) & (k < max_loops)) k = k + for i = 2:M+ % compute next Jacobi iterate for j = 2:M+ V(i,j) =.25*(U(i-,j) + U(i+,j) +... U(i,j-) + U(i,j+) + F(i-,j-)) U = V e = max(max(abs(u - U(2:M+,2:M+)))) % compute norm of error err(k) = e if (e <= epsilon) disp(sprintf(' convergence achieved after %2d interations',k)) else disp(' maximum number of loops computed without convergence') plot(:k,[e_start err(:k)]) xlabel('') ylabel('max norm error') hold on plot(:k,[e_start,rho.^(:k)*e_start],'-.') leg('','\rho^k') plot(k,err(k),'*') hold off

6 6 9. A script for Conjugate Gradient iteration is: % Script File: mycg amp = 6 M = 5 h = /(M+) max_loops = 5*M*M epsilon = h*h alfa = pi/(m+) c = cos(alfa/2) s = sin(alfa/2) CKA = (c - s)/(c + s) x = (:M)*h y = x u = zeros(m,m) % initialize exact solution and f = zeros(m,m) % right hand side of differential equation for i = :M for j = :M u(i,j) = amp*x(i)*(-x(i))*y(j)*(-y(j)) f(i,j) = 2*amp*(x(i)*(-x(i)) + y(j)*(-y(j))) U_exact = zeros(m+2,m+2) U_exact(2:M+,2:M+) = u % pad u with zeros for computing A-norm U = zeros(m+2,m+2) R = zeros(m+2,m+2) P = zeros(m+2,m+2) V = zeros(m+2,m+2) % approximate solution % residual vector % search direction % temporary storage R(2:M+,2:M+) = h*h*f % initialization of R and P P = R rho_ = norm(r,'fro') % 2-norm of previous residual. This is rho_ = rho_*rho_ % used to avoid re-computing this quantity err = zeros(max_loops,) e_start = A_norm(U_exact - U) % data for plotting k = e = e_start while ((e > epsilon) & (k < max_loops)) k = k + for i = 2:M+ % compute V = A*p

7 7 for j = 2:M+ V(i,j) = 4*P(i,j) - P(i-,j) -... P(i+,j) - P(i,j-) - P(i,j+) sum = for i = 2:M+ for j = 2:M+ sum = sum + P(i,j)*V(i,j) % sum = A-norm of p squared alfa = rho_/sum U = U + alfa*p R = R - alfa*v rho_ = norm(r,'fro') rho_ = rho_*rho_ beta = rho_/rho_ P = R + beta*p rho_ = rho_ % update U % and the residual R % 2-norm of current residual % update search direction % save to avoid re-computing e = A_norm(U_exact - U) err(k) = e if (e <= epsilon) disp(sprintf(' convergence achieved after %2d interations',k)) else disp(' maximum number of loops computed without convergence') plot(:k,[e_start err(:k)]) xlabel('') ylabel('a-norm of error') hold on plot(:k,[2*e_start,2*(cka.^(:k))*e_start],'-.') leg('','2c^k') plot(k,err(k),'*') hold off

8 8.9 ρ k.8.7 max norm error Figure. The e n = ku;u n k in the Jacobi iterates U n for the model problem with M =, compared with the sequence of iterates n e where = (D ; (L+U)). Convergence was achieved after 46 iterations..9 ρ k.8.7 max norm error Figure 2. The e n = ku;u n k in the Jacobi iterates U n for the model problem with M = 2, compared with the sequence of iterates n e where = (D ; (L+U)). Convergence was achieved after 6 iterations.

9 9.9 ρ k.8.7 max norm error Figure 3. The e n = ku;u n k in the Jacobi iterates U n for the model problem with M = 3, compared with the sequence of iterates n e where = (D ; (L+U)). Convergence was achieved after 45 iterations C k A norm of error Figure 4. The e n = ku ; U n k A in the conjugate gradient iterates U n for the model problem with M =, compared with the sequence of iterates 2C n e where C =( p 2 (A);)=( p 2 (A)+). Convergence was achieved after 7 iterations.

10 C k A norm of error Figure 5. The e n = ku ; U n k A in the conjugate gradient iterates U n for the model problem with M = 3, compared with the sequence of iterates 2C n e where C =( p 2 (A);)=( p 2 (A)+). Convergence was achieved after 26 iterations C k A norm of error Figure 6. The e n = ku ; U n k A in the conjugate gradient iterates U n for the model problem with M = 5, compared with the sequence of iterates 2C n e where C =( p 2 (A);)=( p 2 (A)+). Convergence was achieved after 47 iterations.