MA3232 Numerical Analysis Week 9. James Cooley (1926-)

Transcription

1 MA umerical Analysis Week 9 James Cooley (96-) James Cooley is an American mathematician. His most significant contribution to the world of mathematics and digital signal processing is the Fast Fourier transform, which he co-developed with John Tukey (Cooley-Tukey FFT algorithm). John Tukey (95-000) John Tukey was an American statistician. One can FT anything often meaningfully. --- John W. Tukey FFT (Fast Fourier transform) FFT is an efficient way of calculating DFT. Assuming = p. DFT: F = { f j, j = 0,,, } Y = y k, k = 0,,, k j y k = f j ω, k = 0,,, j =0 where ω = ep iπ Cost of straightforward calculation =. { } Let us look at a recursive way of calculating Y = DFT( F). Theorem: Consider DFT: F Y Let F = f 0, f,, f F = f, f,, f Y = y 0, y,, y Then = y, y +,, y Y - -

2 Y = DFT( F )+ V.* DFT F Y = DFT( F ) V.* DFT F where V = ω 0, ω,, ω MA umerical Analysis ( ) ( ). (See handout # for the proof of this theorem) ote: This theorem provides a recursive way of calculating Y = DFT( F). FFT (Fast Fourier transform) A recursive way of calculating Y = DFT( F) Let F = f 0, f,, f F = f, f,, f Y = y 0, y,, y = y, y +,, y Y Then we have Y = DFT( F )+ V.* DFT F Y = DFT( F ) V.* DFT F where V = ω 0, ω,, ω ( ) ( ). Let C( ) be the cost of the recursive algorithm on a vector of size. It satisfies C = 0 C ( ) = C + (a recursive equation) ==> ==> C( ) = C( ) = C C + + log ( ) log - -

3 MA umerical Analysis ==> C( ) C log ( ) = Consider the function g = g g g = 0 ==> g( ) = 0 ==> = C( ) C( ) log ( ) = 0 ==> C( ) = log ( ) log log ( ). We have otice that the cost of the recursive algorithm is much smaller than that of the straightforward calculation. log << Solving linear systems Consider the linear system where A = b b b b = b a a A = a a = is given (a vector of size ) is given (a matri of ) is known (a vector of size ) - -

4 MA umerical Analysis In Matlab, = A \ b; Principles of the method solving A = b Special case #: A is lower triangular. a 0 0 a a A = 0 a a a We start with the first row and go down a = b = b a ( a + a = b = b a ) a ( ) a a + + a ( ) + a = b = b a a This is called forward substitution. The cost of forward substitution is C = ( ) = ( j ) = Special case #: j = A is upper triangular. a a a 0 a a A = 0 0 a We start with the last row and go up a = b = b a ( ) a ( ) ( ) + a ( ) = b = b a a ( ) ( ) a a + a + + a = b = b a a - 4 -

5 MA umerical Analysis This is called backward substitution. The cost of backward substitution is C = Special case #: A = LU L: lower triangular U: upper triangular A = b ==> LU = b L y = b ==> U = y Solve L y = b using forward substitution Solve U = y using backward substitution Given A = LU, the cost of solving LU = b is C = + = Question: How to write A as A = LU? LU decomposition (Gauss elimination): To eliminate part of row # for i = :, end add a i a row # to row # i a a a a a a a a a To eliminate part of row # for i = :, end add a i row # to row # i a a a a 0 a a 0 a a - 5 -

6 MA umerical Analysis The cost of LU decomposition is C = ( ) + ( ) + = ( j) j = ote: ) A sum can be approimated by an integral. This is a convenient way of estimating a sum. ( j) j = 0 d = ( ) = 0 ) The cost of LU decomposition is much larger than the cost of solving LU = b. >> The cost of solving A = b by Gaussian elimination is. Solving linear systems with iterative methods Consider the linear system where A = b b b b = b a a A = a a = is given (a vector of size ) is given (a matri of ) is known (a vector of size ) - 6 -

7 MA umerical Analysis Jacobi Iteration Method and Gauss-Seidel Iteration Method Eample: Consider A = b where 9 0 A = 0, b 9 =. 4 0 The eact solution is =. We rewrite the linear system as = b = b = b In the equation numbered k, solve k in terms of the remaining unknowns. So we rearrange linear system as follows: = b = [ b ] = b = b = b = 4 [ ] [ b ] If we assume initial values for,,, and substitute these values into the right-hand of the above equations, we may determine new values for,,. The iterative process is continued by substituting these values of,, into the right-hand side of the equations. There are several variants of the process: () If we use old values of,, in the righthand side of the equations to determine ALL the new values of,, in the left-hand side of the equation, then this method is called Jacobi or simultaneous iteration. () Alternatively, we may use a new value of i in the right-hand side of the equation as soon as it is determined, to obtain the other values of i in the right-hand side. For eample, once is determined from the first equation above, it is used in the second equation, together with the old, to determine. These two new values of, are then used in the third equation to find. This is called Gauss-Seidel or cyclic iteration. This method usually converges faster than the Jacobi method (For our eample, it takes 0 iterations for Jacobi method to converge whereas it takes 5 iterations for Gauss-Seidel to converge to the eact solution =, =, =. ) - 7 -

8 MA umerical Analysis We stop the iteration when the latest iterates ( k + ) ( k ) tolerance. ( k ) + and ( k ) satisfy ( 0) ( 0) ( 0) Mathematically, let = be an initial guess of the true solution. Then the Jacobi ( 0) iteration method defines an iteration sequence: ( k + ) ( k ) ( k ) = b 9 = 0 b = b 4 k = 0,,,. ( k + ) ( k ) ( k ) ( k + ) ( k ) ( k ) The Gauss-Seidel iteration method generates an iteration sequence: ( k + ) ( k ) ( k ) = b 9 = 0 b = b 4 k = 0,,,. ( k + ) ( k + ) ( k ) ( k + ) ( k + ) ( k + ) The condition for convergence for Jacobi iteration and Gauss-Seidel iteration are n aii >> aij for i =,,, n. j= j i Thus these iterative methods are only guaranteed to work when the coefficient matri is diagonally dominant. Successive Overrelaation Method (SOR) Convergence in the Gauss-Seidel method can be speeded if we do what is called overrelaing. The term comes from an old hand method where a set of ``residuals were ``relaed to zero

9 MA umerical Analysis The successive overrelaation method (SOR) is derived by taking the form of a weighted average between the previous iterate and the computed Gauss-Seidel iterate successively for each component: ( + ) ( + ), ( ω ) = ω + k k GS k i i i + k, GS where i denotes a Gauss-Seidel iterate and ω is the overrelaation factor. The idea is to choose a value for ω that will accelerate the rate of convergence of the iterates to the solution. If ω =, the SOR method simplifies to the Gauss-Seidel method. A theorem due to Kahan (958) shows that SOR fails to converge if ω is outside the interval (0,). In general, it is not possible to compute in advance the optimal value of ω that will maimize the rate of ω = O h, where h convergence of SOR. Frequently, some heuristic estimate is used, such as is mesh size of the discretization of the underlying physical domain. For our eample, the SOR method gives ( k + ) ω ( k ) ( k ) = b 9 + ( k + ) ω ( k + ) ( k ) = b 0 + ( k + ) ω ( k + ) ( k + ) = b 4 + k = 0,,,. ( ω) ( k ) ( ω) ( ω) ( k ) ( k ) - 9 -