Fixed point iteration and root finding

Transcription

1 Fixed point iteration and root finding The sign function is defined as x > 0 sign(x) = 0 x = 0 x < 0. It can be evaluated via an iteration which is useful for some problems. One such iteration is given by x k+ = 3x k x 3 k. (4) a) Find all fixed points of this iteration. How many of the fixed points are stable? b) Find the order of convergence of the iteration at the stable fixed point(s). c) Show that for x [, ], x 0, the iteration (4) always converges. SOLUTION: a) Solving the equation x = (3x x 3 )/ g(x) gives the fixed points x = ± and x = 0 The fixed points x = ± are stable since g (±) = 0, and the fixed point x = 0 is unstable since g (0) = 3/ >. b) Let α denote the fixed point x = or x =. Define the error ɛ n x n α. Taylor expanding x n+ = g(x n ) around α gives us x n+ = g(x n ) = g(α) + g (α)ɛ n + g (α) ɛ n + O(ɛ3 n ). Since g(α) = α, we have that ɛ n+ = g(x n ) g(α) and hence ɛ n+ = g (α)ɛ n + g (α) ɛ n + O(ɛ 3 n). Since g (±) = 0 and g (α) = 3 we have that the lowest non-zero term is O(ɛ n ) and we have quadratic convergence. 3

2 c) We show that g([0, ]) [0, ] by first observing that g (x) = 3 3x > 0 for x (0, ). Therefore the function g(x) is monotonically increasing from g(0) = 0 to the local maximum g() = and hence g([0, ]) [0, ]. Since x k+ x k = 3x k x 3 k x k = 3 x k. we have that x k+ x k > for x (0, ). Hence, if x 0 (0, ) then {x k } is a monotonically increasing sequence in the interval [0, ] where it must have a point of accumulation. Since x = 0 and x = are the only fixed points in this interval, the monotonically increasing sequence must converge to x =. (The argument when x 0 [, 0) is analogous.) 4

3 Quadrature Gaussian quadratures f(x) dx = w k f(x k ) () with nodes including the endpoints (x 0 = and x N = ) are called Gauss- Legendre-Lobatto quadratures. a) Show that if the interior nodes x,..., x N in the quadrature are given by the roots of p N (x), where p N(x) denotes the N:th degree Legendre polynomial, then the quadrature is exact for polynomials up to degree N. Hint: The N :th degree Legendre polynomial can be written as p N (x) = s N (x)(x )p N (x) + r N(x) where s N (x) and r N (x) are N :th and N:th degree polynomials, respectively. b) Find the 4-point (N = 3) Gauss-Legendre-Lobatto quadrature (nodes and weights) for the integrals f(x) dx. Hint: The three term recursion for Legendre polynomials is given by P 0 (x) =, P (x) = x and (k + )P k+ (k + )xp k + kp k = 0. SOLUTION: a) We want to show that with the given nodes, weights can be found such that p N (x) dx = w k p N (x k ). From the hint we have that p N (x) dx = s N (x)(x )p N (x) dx + r N (x) dx. Integration by parts of the first integral to the right gives s N (x)(x )p N (x) dx = d ( sn (x)(x ) ) p N (x) dx. dx

4 Since d (s dx N (x)(x )) is a polynomial of degree N, it can be expanded into p 0 (x),..., p N (x). Since these polynomials are orthogonal to p N (x), the integral s N(x )p N (x) dx = 0, and hence p N (x) dx = r N (x) dx. () Since x 0 =, x N =, and {x k } N k= are the roots of p N (x), it follows that w k p N (x k ) = w k s N (x k )(x k )p N (x k) dx + w k r N (x k ) dx = w k r N (x k ) dx. To show () we see from () and (3) that we must require that r N(x) dx = N w kr N (x k ) for r N (x) = x j, j = 0,..., N. This gives a linear (Vandermonde) system for the weights w k which can be solved since the nodes are distinct. b) Using the recursion formula for Legendre polynomials we find that p 3 (x) = 5x 3 3x, and hence p 3 (x) = 5x 3 with the roots ± 5. Hence, the nodes are x 0 =, x = / 5, x = 5 and x 3 =. We find the weights by solving the system 3 w k x j k = x j dx, j = 0,..., 3 w 0 + w + w + w 3 = w 0 w 5 + w 5 + w 3 = 0 w 0 + w 5 + w 5 + w 3 = 3 w 0 w 5 + w w 5 3 = 0 which by using symmetry (w 0 = w 3 and w = w ) is reduced to { w0 + w = w 0 + w 5 = 3 with the solution w 0 = /6 and w = 5/6. (3)

5 Interpolation / Approximation: 3. a. Define what is meant by a cubic spline. b. Show that giving just function values f(x i), i =,..., N is insufficient to determine uniquely a cubic spline through the points. Work out how many additional conditions that are required for this. c. Two ways to obtain a unique cubic spline interpolant are known as 'natural spline' and 'not-a-knot' spline. Define these two spline types. d. A cubic B-spline is the cubic spline that is non-zero over the narrowest interval. In the case of a unit-spaced grid, it will look as shown in the figure to the right; non-zero over four subintervals only (taking values at the nodes and at subinterval midpoints as shown, when normalized in the usual fashion). If one knows the values of such a B-spline f (x) only at the node points, one can very easily tabulate it at increasingly denser sets of point locations by means of repeatedly using the relation /3 3/48 /6 /48 f (x) = 8 (f (x ) + 4f (x ) + 6f (x) + 4f (x + ) + f (x + )) () Show that () holds true for all values of x. Hint: One way to solve the problem, requiring only little algebra, is to let f(x) in the right hand side of () be the cubic B-spline, and then show that uniqueness of cubic splines implies that the whole expression must evaluate to the very same spline. Solution: a. A cubic spline s(x) is a piecewise cubic function which, at the nodes, feature continuity in s, s', s'' (but may jump in s'''). b. With f(x i), i =,..., N given, we need to determine N- cubics, i.e. 4N-4 parameters. The conditions we have are Function values at the ends of each sub-interval (N-) Continuity of s' at interior nodes N- Continuity of s'' at interior nodes N- for a total of 4N-6. We fall short of conditions. c. Natural spline: Add two conditions by imposing also s''(x ) = s''(x N) = 0. Not-a-knot spline: Remove two freedoms by not allowing the jump in s''' at x and x N-. d. Substituting the B-spline f(x) into the right hand side of () gives a piecewise cubic result, which we call h(x). It will clearly satisfy: - h(x)h0 for x < - and x > +. - Continuity of h, h', h'' everywhere, and possible jumps in h''' at x =, 3,,, 0,,, 3,. In particular, h''(-) = h''() = 0. - At the points x =, 3,,, 0,,, 3,, h(x) evaluates to the same values as are given in the figure for f(x). A cubic spline which takes certain values at the nodes and also satisfies zero second derivative end conditions is the unique natural spline. Both h(x) and f(x) (the B-spline) satisfies all the properties just listed. So they have to be the same.

6 Linear Algebra Let A be an m n matrix, with m n, and b and m vector. Consider the least-squares problem which can be written as min Ax x R b, n Ax b. () a. Explain why any solution of the normal equations is a solution of this problem, regardless of the rank of A. Show that the solution is unique if the rank of A is n. b. Describe the solution of this problem in terms of the singular value decomposition of A. Consider the regularized least-squares problem ( min Ax b + λ x ). x R n c. Derive a form of the normal equations for this problem. Use it to argue that the regularized problem has a unique solution whenever λ 0, regardless of the rank of A. (Hint: Write the regularized least-square problem in the form of equation () using an (m+n) n linear system.) d. Write the solution of the regularized problem in terms of the singular value decomposition of A. e. Assume that A has only one zero singular value and b is not orthogonal to the corresponding left singular vector of A. Describe the effect on the solution, x, of the regularized problem as λ 0. Solution: a. Any solution of the least-squares problem satisfies the orthogonality condition < Ax b, Az >= 0 z R n and, thus, x satisfies the normal equations, which are given by A t Ax = A t b. If A has rank n, then A t A is nonsingular and x = (A t A) A t b.

7 b. Let the singular value decomposition be given as A = UΛV where U is an m m unitary matrix, V is an n n unitary matrix and Λ is is an m n matrix such that Λ = diag {σ, σ,..., σ n }, 0 σ σ σ n Substituting in the equation above and simplifying, we have x = V Λ U b + z where Λ = diag { σ, σ,..., σ n }, σ j = { 0 σj = 0 0 and z is any null vector of A. Writing it out in terms of the columns of U and V yields x = v j u t j b + z c. Write the system as [ A λi 0 ] x ( b 0 The normal equations associated with the regularized problem are [ A t λi ] [ ] A x = [ A λi t λi ] ( ) b 0 ). which become ( A t A + λ I ) x = A t b. () Since, A t A is positive semidefinite and, if λ 0, λ I is positive definite, then A t A + λ I is nonsingular. This implies that equation() has a unique solution/ d. Plugging the singular value decomposition in to these new normal equations, we have x = V DU b where D = diag {..., } σj + λ,....

8 Writing it out in terms of the columns of U and V yields x = n j= σ j + λ v ju j t b = 0 σ j + λ v ju j t b e. Consider the formula above. If = 0, it play no role in the solution. Thus, lim x = v j u t j b λ 0 0 which is the minimal length least squares solution. 3

9 Numerical ODE: 5. A general linear multistep formula for solving y = f (t, y) takes the form p y n+ = j=0 q a j y n j + k j= b j f (t n j, y n j ) a. Derive the coefficients a j and b j for the second order Adams-Bashforth and Adams-Moulton methods. b. The figure to the right shows the stability domain for the second order Adams-Bashforth method (interior of the closed curve). Can this method be used to solve y = y, y = y, and/or y = i y? Explain! c. Determine the stability domain for the second order Adams-Moulton method Solution: a. The AB-formula for solving y = f(t, y) takes the form y n+ = a 0 y n + k[b 0 f n + b f n ]. It should be exact for example when k = and {yh, fh0}, {yht, fh}, {yht, fht}. This gives three equations: = a 0 +0b 0 +0b = 0a 0 +b 0 +b = 0a 0 +0b 0 b with the solution a 0 =, b 0 = 3, b =. For AM, we get similarly y n+ = a 0 y n + k[b f n+ + b 0 f n ], leading to = a 0 +0b +0b 0 = 0a 0 +b +b 0 = 0a 0 +b +0b 0 with the solution a 0 =, b =, b 0 =. b. The stability domain tells for which values of the time step k solutions to the ODE y = y can feature growing solutions. That is not the information we need to decide convergence. This instead requires - consistency (here OK; already first order accuracy would be slightly more than what is needed) - stability (root condition OK; the equation to test is r =, which obviously has as only root r = ) So the method can be used for all the cases mentioned. c. We find the stability domain by applying the method to y = y, giving y n+ + y n + k Writing [y n+ + y n ]. k =, this becomes y n+ = + / The relation holds precisely when Re ξ < 0. Hence, / y + / n. / < the stability domain is the left half plane.

10 6. Finite Difference Equations for PDEs ConsiderRichardson sdifferenceschemefortheheatequation,u t =u xx : k (u(x,t+k) u(x,t k))= h (u(x h,t) u(x,t)+u(x+h,t)) a. Show that this scheme has second-order truncation error. b. UseeitherODEprinciplesorvonNeumannanalysistoshowthatthis scheme is unconditionally unstable. c. Demonstrate a minor modification of the left-side of Richardson s scheme that yields a familiar unconditionally stable scheme and prove it. Solution a. Taylor expansion of left-hand side yields k (u(x,t+k) u(x,t k))=u t(x,t)+ 6 u ttt(x,t)k +O(k 4 ) Likewise, a Taylor expansion of the right-hand side yields h (u(x h,t) u(x,t)+u(x+h,t))=u xx(x,t)+ u xxxx(x,t)h +O(h 4 ) b. Using von Neumannanalysis, we set u(x,t) = a t e iωx andsolve for a k. This yields the equation a k a k = k h (e iωh +e iωh )= 4k h (cos(ωh))=8k h sin (ωh/) Multiplyingbya k yields (a k ) + 8k h sin (ωh/)a k =0 Plugging in the values a k = 0 and a k = demonstrates that the equationhasarealroota k <foranyk,h>0,andω. 3

11 c. The backward Euler scheme is k (u(x,t) u(x,t k))= h (u(x h,t) u(x,t)+u(x+h,t)) A von Neumann analysis yields a k =a kk h (e iωh +e iωh )=a k4k h (cos(ωh))=ak8k h sin (ωh/) Solvingfora k yileds a k = foranyk,h>0,andω. + 8k (sin (ωh/)) <.0 h 4