Accelerating Convergence

Transcription

1 Accelerating Convergence MATH 375 Numerical Analysis J. Robert Buchanan Department of Mathematics Fall 2013

2 Motivation We have seen that most fixed-point methods for root finding converge only linearly to a solution. Today we describe a technique which can be used accelerate the convergence of any linearly convergent sequence.

3 Linear Convergence Suppose sequence {p n } n=0 converges linearly to p.

4 Linear Convergence Suppose sequence {p n } n=0 converges linearly to p. p n+1 p 0 < lim = λ < 1 n p n p

5 Linear Convergence Suppose sequence {p n } n=0 converges linearly to p. p n+1 p 0 < lim = λ < 1 n p n p Assume the signs of p n+2 p, p n+1 p, and p n p are all the same.

6 Linear Convergence Suppose sequence {p n } n=0 converges linearly to p. p n+1 p 0 < lim = λ < 1 n p n p Assume the signs of p n+2 p, p n+1 p, and p n p are all the same. For large n then p n+2 p p n+1 p p n+1 p p n p.

7 Re-writing the Sequence (1 of 2) Solve for p. p n+2 p p n+1 p p n+1 p p n p (p n+1 p) 2 (p n+2 p)(p n p)

8 Re-writing the Sequence (1 of 2) Solve for p. p n+2 p p n+1 p p n+1 p p n p (p n+1 p) 2 (p n+2 p)(p n p) p p n+2p n p 2 n+1 p n+2 2p n+1 + p n

9 Re-writing the Sequence (2 of 2) p p n+2p n p 2 n+1 p n+2 2p n+1 + p n Add and subtract the terms p 2 n and 2p n p n+1 in the numerator.

10 Re-writing the Sequence (2 of 2) p p n+2p n p 2 n+1 p n+2 2p n+1 + p n Add and subtract the terms p 2 n and 2p n p n+1 in the numerator. p p2 n+1 + 2p np n+1 p 2 n + p n+2 p n + p 2 n 2p n p n+1 p n+2 2p n+1 + p n Now factor the numerator.

11 Re-writing the Sequence (2 of 2) p p n+2p n p 2 n+1 p n+2 2p n+1 + p n Add and subtract the terms p 2 n and 2p n p n+1 in the numerator. p p2 n+1 + 2p np n+1 p 2 n + p n+2 p n + p 2 n 2p n p n+1 p n+2 2p n+1 + p n Now factor the numerator. p (p n+1 p n ) 2 + p n (p n+2 + p n 2p n+1 ) p n+2 2p n+1 + p n (p n+1 p n ) 2 = p n p n+2 2p n+1 + p n

12 Aitken s 2 Method Define a new sequence {ˆp n } n=0 as ˆp n = p n We would like to show that (p n+1 p n ) 2 p n+2 2p n+1 + p n. lim ˆp n = p = lim p n, and n n sequence {ˆp n } n=0 converges to p faster than sequence {p n } n=0.

13 Aitken s 2 Method Define a new sequence {ˆp n } n=0 as ˆp n = p n We would like to show that (p n+1 p n ) 2 p n+2 2p n+1 + p n. lim ˆp n = p = lim p n, and n n sequence {ˆp n } n=0 converges to p faster than sequence {p n } n=0. We will come to call this iterative method as Aitken s 2 Method.

14 Example ( ) 1 Define p n = sin for n 1. Find the limit of this sequence n and compare its convergence to that of the sequence ˆp n.

15 Example ( ) 1 Define p n = sin for n 1. Find the limit of this sequence n and compare its convergence to that of the sequence ˆp n. n p n ˆp n

16 Forward Difference Definition Given sequence {p n } n=0, the forward difference denoted p n is defined as p n = p n+1 p n, for n 0. Higher powers of the operator are defined recursively as ) k p n = ( k 1 p n, for n 2.

17 Forward Difference (2 of 2) p n = p n+1 p n 2 p n = (p n+1 p n ) = p n+2 p n+1 (p n+1 p n ) = p n+2 2p n+1 + p n 3 p n = (p n+2 2p n+1 + p n ) = p n+3 3p n+2 + 3p n+1 p n.

18 Forward Difference (2 of 2) p n = p n+1 p n 2 p n = (p n+1 p n ) = p n+2 p n+1 (p n+1 p n ) = p n+2 2p n+1 + p n 3 p n = (p n+2 2p n+1 + p n ) = p n+3 3p n+2 + 3p n+1 p n. We can write Aitken s 2 Method as ˆp n = p n ( p n) 2 2 p n.

19 Convergence of Aitken s 2 Method Theorem Let {p n } n=0 be a sequence converging linearly to p with asymptotic error constant λ < 1. The sequence {ˆp n } n=0 converges to p faster than {p n } n=0 in the sense that ˆp n p lim n p n p = 0.

20 Proof (1 of 5) By assumption {p n } n=0 converges linearly to p, i.e. p n+1 p 0 < lim = λ < 1. n p n p Assume the signs of p k p are all the same for sufficiently large k.

21 Proof (1 of 5) By assumption {p n } n=0 converges linearly to p, i.e. p n+1 p 0 < lim = λ < 1. n p n p Assume the signs of p k p are all the same for sufficiently large k. Define a new sequence δ n = p n+1 p λ for n 0. Note that lim n δ n = 0. p n p

22 Proof (2 of 5) Consider ˆp n p p n p = 1 p n p ( p n ( p n) 2 2 p n p )

23 Proof (2 of 5) Consider ˆp n p p n p = 1 p n p = 1 p n p ( ) p n ( p n) 2 2 p p n ( ) p n p ( p n) 2 2 p n ( p n ) 2 = 1 (p n p) 2 p n

24 Proof (2 of 5) Consider ˆp n p p n p = 1 p n p = 1 p n p ( ) p n ( p n) 2 2 p p n ( ) p n p ( p n) 2 2 p n ( p n ) 2 = 1 (p n p) 2 p n (p n+1 p n ) 2 = 1 (p n p)(p n+2 2p n+1 + p n ) = 1 ((p n+1 p) (p n p)) 2 (p n p)((p n+2 p) 2(p n+1 p) + (p n p))

25 Proof (3 of 5) Recall p n+1 p p n p = δ n + λ p n+1 p = (δ n + λ)(p n p)

26 Proof (3 of 5) Recall p n+1 p p n p = δ n + λ p n+1 p = (δ n + λ)(p n p) This implies p 2 p = (δ 1 + λ)(p 1 p) p 3 p = (δ 2 + λ)(p 2 p) = (δ 1 + λ)(δ 2 + λ)(p 1 p). n p n+1 p = (p 1 p) (δ i + λ). i=1

27 Proof (4 of 5) Consequently ˆp n p p n p = 1 ((p n+1 p) (p n p)) 2 (p n p)((p n+2 p) 2(p n+1 p) + (p n p)) = 2 n n 1 (p 1 p) (δ i + λ) (p 1 p) (δ i + λ) i=1 i=1 1 ( n 1 n+1 n n 1 ) (p 1 p) (δ i + λ) (p 1 p) (δ i + λ) 2(p 1 p) (δ i + λ) + (p 1 p) (δ i + λ) i=1 = 1 n 1 (δ i + λ) = 1 i=1 n 1 i=1 i=1 i=1 ( n+1 i=1 n 1 i=1 n n 1 (δ i + λ) (δ i + λ) i=1 2 ) n n 1 (δ i + λ) 2 (δ i + λ) + (δ i + λ) i=1 2 (δ i + λ) (δ i + λ) 2 i=1 (δ n+λ 1) 2 ((δ n+1 +λ)(δ n+λ) 2(δ n+λ)+1) i=1 i=1

28 Proof (5 of 5) ˆp n p p n p = 1 = 1 n 1 i=1 n 1 i=1 2 (δ i + λ) (δ i + λ) 2 (δ n+λ 1) 2 ((δ n+1 +λ)(δ n+λ) 2(δ n+λ)+1) (δ n + λ 1) 2 (δ n+1 + λ)(δ n + λ) 2(δ n + λ) + 1

29 Proof (5 of 5) ˆp n p p n p = 1 = 1 n 1 i=1 Taking the limit as n : ˆp n p lim n p n p = lim n = 1 = 0 n 1 i=1 2 (δ i + λ) (δ i + λ) 2 (δ n+λ 1) 2 ((δ n+1 +λ)(δ n+λ) 2(δ n+λ)+1) (δ n + λ 1) 2 (δ n+1 + λ)(δ n + λ) 2(δ n + λ) + 1 [ 1 (δ n + λ 1) 2 ] (δ n+1 + λ)(δ n + λ) 2(δ n + λ) + 1 (λ 1)2 λ 2 2λ + 1

30 Steffensen s Method We can use Aitken s 2 Method to accelerate the convergence of any linearly convergent sequence generated by fixed-point iteration.

31 Steffensen s Method We can use Aitken s 2 Method to accelerate the convergence of any linearly convergent sequence generated by fixed-point iteration. Consider the fixed point problem g(x) = x and an initial approximation p 0. Calculate p 1 = g(p 0 ) p 2 = g(p 1 ) { ˆp 0 = 2} (p 0 ) p 4 = g(ˆp 0 ) p 5 = g(p 4 ) { ˆp 1 = 2} (ˆp 0 ) p 7 = g(ˆp 1 ) p 8 = g(p 7 ) { ˆp 2 = 2} (ˆp 1 )

32 Steffensen s Method We can use Aitken s 2 Method to accelerate the convergence of any linearly convergent sequence generated by fixed-point iteration. Consider the fixed point problem g(x) = x and an initial approximation p 0. Calculate p 1 = g(p 0 ) p 2 = g(p 1 ) { ˆp 0 = 2} (p 0 ) p 4 = g(ˆp 0 ) p 5 = g(p 4 ) { ˆp 1 = 2} (ˆp 0 ) p 7 = g(ˆp 1 ) p 8 = g(p 7 ) { ˆp 2 = 2} (ˆp 1 ) Every 3rd term is calculated using the 2 method, all other terms use fixed-point iteration.

33 Algorithm INPUT initial approximation p 0, tolerance ɛ, maximum iterations N. STEP 1 Set i = 1. STEP 2 While i N do STEPS 3 6. STEP 3 Set p 1 = g(p 0 ); p 2 = g(p 1 ); p = { 2} (p 0 ). STEP 4 If p p 0 < ɛ then OUTPUT p; STOP. STEP 5 Set i = i + 1. STEP 6 Set p 0 = p. STEP 7 OUTPUT Method failed after N iterations. ; STOP.

34 Example Use Steffensen s Method to approximate the solution to x 3 x = 0 for x [0, 1].

35 Example Use Steffensen s Method to approximate the solution to x 3 x = 0 for x [0, 1]. n Fixed-point Steffensen s

36 Final Result Remark: Steffensen s Method appears to generate a quadratically convergent sequence for root finding without requiring computation of a derivative.

37 Final Result Remark: Steffensen s Method appears to generate a quadratically convergent sequence for root finding without requiring computation of a derivative. Theorem Suppose g(x) = x has a solution p with g (p) 1. If there exists δ > 0 such that g C 3 [p δ, p + δ], the Steffensen s Method gives quadratic convergence for any p 0 [p δ, p + δ].

38 Homework Read Section 2.5. Exercises: 1, 2, 7, 9, 11