Numerical Analysis and Computing

Transcription

1 Numerical Analysis and Computing Lecture Notes #13 Rational Function Approximation Joe Mahaffy, Department of Mathematics Dynamical Systems Group Computational Sciences Research Center San Diego State University San Diego, CA jmahaffy Spring 21 Joe Mahaffy, Rational Function Approximation (1/21)

2 Outline 1 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 2 Finding the Optimal Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (2/21)

3 Polynomial Approximation: Pros and Cons. Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Advantages of Polynomial Approximation: [1] We can approximate any continuous function on a closed interval to within arbitrary tolerance. (Weierstrass approximation theorem) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (3/21)

4 Polynomial Approximation: Pros and Cons. Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Advantages of Polynomial Approximation: [1] We can approximate any continuous function on a closed interval to within arbitrary tolerance. (Weierstrass approximation theorem) [2] Easily evaluated at arbitrary values. (e.g. Horner s method) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (3/21)

5 Polynomial Approximation: Pros and Cons. Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Advantages of Polynomial Approximation: [1] We can approximate any continuous function on a closed interval to within arbitrary tolerance. (Weierstrass approximation theorem) [2] Easily evaluated at arbitrary values. (e.g. Horner s method) [3] Derivatives and integrals are easily determined. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (3/21)

6 Polynomial Approximation: Pros and Cons. Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Advantages of Polynomial Approximation: [1] We can approximate any continuous function on a closed interval to within arbitrary tolerance. (Weierstrass approximation theorem) [2] Easily evaluated at arbitrary values. (e.g. Horner s method) [3] Derivatives and integrals are easily determined. Disadvantage of Polynomial Approximation: [1] Polynomials tend to be oscillatory, which causes errors. This is sometimes, but not always, fixable: E.g. if we are free to select the node points we can minimize the interpolation error (Chebyshev polynomials), or optimize for integration (Gaussian Quadrature). Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (3/21)

7 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Moving Beyond Polynomials: Rational Approximation We are going to use rational functions, r(x), of the form r(x) = p(x) q(x) = 1 + n p i x i i= m q i x i j=1 and say that the degree of such a function is N = n + m. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (4/21)

8 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Moving Beyond Polynomials: Rational Approximation We are going to use rational functions, r(x), of the form r(x) = p(x) q(x) = 1 + n p i x i i= m q i x i j=1 and say that the degree of such a function is N = n + m. Since this is a richer class of functions than polynomials rational functions with q(x) 1 are polynomials, we expect that rational approximation of degree N gives results that are at least as good as polynomial approximation of degree N. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (4/21)

9 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Extension of Taylor expansion to rational functions; selecting the p i s and q i s so that r (k) (x ) = f (k) (x ) k =, 1,...,N. f (x) r(x) = f (x) p(x) f (x)q(x) p(x) = q(x) q(x) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (5/21)

10 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Extension of Taylor expansion to rational functions; selecting the p i s and q i s so that r (k) (x ) = f (k) (x ) k =, 1,...,N. f (x) r(x) = f (x) p(x) f (x)q(x) p(x) = q(x) q(x) Now, use the Taylor expansion f (x) i= a i(x x ) i, for simplicity x = : f (x) r(x) = m n a i x i q i x i p i x i i= i= q(x) i=. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (5/21)

11 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 Extension of Taylor expansion to rational functions; selecting the p i s and q i s so that r (k) (x ) = f (k) (x ) k =, 1,...,N. f (x) r(x) = f (x) p(x) f (x)q(x) p(x) = q(x) q(x) Now, use the Taylor expansion f (x) i= a i(x x ) i, for simplicity x = : f (x) r(x) = m n a i x i q i x i p i x i i= i= q(x) i=. Next, we choose p,p 1,...,p n and q 1,q 2,...,q m so that the numerator has no terms of degree N. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (5/21)

12 : The Mechanics. Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 For simplicity we (sometimes) define the indexing-out-of-bounds coefficients: { pn+1 = p n+2 = = p N = q m+1 = q m+2 = = q N =, so we can express the coefficients of x k in as m n a i x i q i x i p i x i =, i= i= i= k =, 1,...,N k a i q k i = p k, i= k =, 1,...,N. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (6/21)

13 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 : Abstract Example 1 of 2 Find the Padé approximation of f (x) of degree 5, where f (x) a + a 1 x +...a 5 x 5 is the Taylor expansion of f (x) about the point x =. The corresponding equations are: x a p = x 1 a q 1 + a 1 p 1 = x 2 a q 2 + a 1 q 1 + a 2 p 2 = x 3 a q 3 + a 1 q 2 + a 2 q 1 + a 3 p 3 = x 4 a q 4 + a 1 q 3 + a 2 q 2 + a 3 q 1 + a 4 p 4 = x 5 a q 5 + a 1 q 4 + a 2 q 3 + a 3 q 2 + a 4 q 1 + a 5 p 5 = Note: p = a!!! (This reduces the number of unknowns and equations by one (1).) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (7/21)

14 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 : Abstract Example 2 of 2 We get a linear system for p 1, p 2,...,p N and q 1, q 2,...,q N : a q 1 p 1 a 1 a 1 a q 2 a 2 a 1 a q 3 a 3 a 2 a 1 a q 4 p 2 p 3 p 4 = a 2 a 3 a 4. a 4 a 3 a 2 a 1 a q 5 p 5 a 5 If we want n = 3, m = 2: a a 1 a a 2 a 1 a a 3 a 2 a 1 a a 4 a 3 a 2 a 1 a q 1 q 2 p 1 p 2 p 3 = a 1 a 2 a 3 a 4 a 5. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (8/21)

15 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 : Abstract Example 2 of 2 We get a linear system for p 1, p 2,...,p N and q 1, q 2,...,q N : a q 1 p 1 a 1 a 1 a q 2 a 2 a 1 a q 3 a 3 a 2 a 1 a q 4 p 2 p 3 p 4 = a 2 a 3 a 4. a 4 a 3 a 2 a 1 a q 5 p 5 a 5 If we want n = 3, m = 2: a 1 a 1 a a 2 a 1 a 3 a 2 a a 4 a 3 a 1 a q 1 q 2 p 1 p 2 p 3 = a 1 a 2 a 3 a 4 a 5. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (8/21)

16 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 : Abstract Example 2 of 2 We get a linear system for p 1, p 2,...,p N and q 1, q 2,...,q N : a q 1 p 1 a 1 a 1 a q 2 a 2 a 1 a q 3 a 3 a 2 a 1 a q 4 p 2 p 3 p 4 = a 2 a 3 a 4. a 4 a 3 a 2 a 1 a q 5 p 5 a 5 If we want n = 3, m = 2: a 1 a 1 a 1 a 2 a 1 a 3 a 2 a 4 a 3 a q 1 q 2 p 1 p 2 p 3 = a 1 a 2 a 3 a 4 a 5. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (8/21)

17 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 : Abstract Example 2 of 2 We get a linear system for p 1, p 2,...,p N and q 1, q 2,...,q N : a q 1 p 1 a 1 a 1 a q 2 a 2 a 1 a q 3 a 3 a 2 a 1 a q 4 p 2 p 3 p 4 = a 2 a 3 a 4. a 4 a 3 a 2 a 1 a q 5 p 5 a 5 If we want n = 3, m = 2: a 1 a 1 a 1 a 2 a 1 1 a 3 a 2 a 4 a 3 q 1 q 2 p 1 p 2 p 3 = a 1 a 2 a 3 a 4 a 5. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (8/21)

18 Pros and Cons of Polynomial Approximation New Bag-of-Tricks: Rational Approximation : Example #1 : Abstract Example 2 of 2 We get a linear system for p 1, p 2,...,p N and q 1, q 2,...,q N : a q 1 p 1 a 1 a 1 a q 2 a 2 a 1 a q 3 a 3 a 2 a 1 a q 4 p 2 p 3 p 4 = a 2 a 3 a 4. a 4 a 3 a 2 a 1 a q 5 p 5 a 5 If we want n = 3, m = 2: a 1 a 1 a 1 a 2 a 1 1 a 3 a 2 a 4 a 3 q 1 q 2 p 1 p 2 p 3 = a 1 a 2 a 3 a 4 a 5. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (8/21)

19 Finding the Optimal : Concrete Example, e x 1 of 3 The Taylor series expansion for e x about x = is hence {a, a 1, a 2, a 3, a 4, a 5 } = {1, 1, 1 2, 1 6, 1 24, 1 12 }. k= ( 1) k k! x k, Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (9/21)

20 Finding the Optimal : Concrete Example, e x 1 of 3 The Taylor series expansion for e x about x = is hence {a, a 1, a 2, a 3, a 4, a 5 } = {1, 1, 1 2, 1 6, 1 24, 1 12 } / /6 1/2 1/24 1/6 q 1 q 2 p 1 p 2 p 3 = k= 1 1/2 1/6 1/24 1/12 which gives {q 1,q 2,p 1,p 2,p 3 } = {2/5, 1/2, 3/5, 3/2, 1/6} ( 1) k k! x k,, Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (9/21)

21 Finding the Optimal : Concrete Example, e x 1 of 3 The Taylor series expansion for e x about x = is hence {a, a 1, a 2, a 3, a 4, a 5 } = {1, 1, 1 2, 1 6, 1 24, 1 12 } / /6 1/2 1/24 1/6 q 1 q 2 p 1 p 2 p 3 = k= 1 1/2 1/6 1/24 1/12 ( 1) k k! x k,, which gives {q 1,q 2,p 1,p 2,p 3 } = {2/5, 1/2, 3/5, 3/2, 1/6}, i.e. r 3,2 (x) = x x2 1 6 x x x2. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (9/21)

22 Finding the Optimal : Concrete Example, e x 2 of 3 All the possible Padé approximations of degree 5 are: r 5, (x) = 1 x x2 1 6 x x x5 r 4,1 (x) = x+ 3 1 x x x x r 3,2 (x) = x+ 3 2 x2 1 6 x x+ 1 2 x2 r 2,3 (x) = x+ 1 2 x x+ 3 2 x x3 r 1,4 (x) = x x+ 3 1 x x x4 r,5 (x) = 1 1+x+ 1 2 x x x x5 Note: r 5, (x) is the Taylor polynomial of degree 5. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (1/21)

23 Finding the Optimal : Concrete Example, e x 3 of 3.1 The Absolute Error. R{5,}(x) e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

24 Finding the Optimal : Concrete Example, e x 3 of 3.1 The Absolute Error. R{5,}(x) R{4,1}(x) e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

25 Finding the Optimal : Concrete Example, e x 3 of The Absolute Error. R{5,}(x) R{4,1}(x) R{3,2}(x).1.1 1e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

26 Finding the Optimal : Concrete Example, e x 3 of The Absolute Error. R{5,}(x) R{3,2}(x) R{2,3}(x).1.1 1e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

27 Finding the Optimal : Concrete Example, e x 3 of The Absolute Error. R{5,}(x) R{2,3}(x) R{1,4}(x).1.1 1e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

28 Finding the Optimal : Concrete Example, e x 3 of The Absolute Error. R{5,}(x) R{2,3}(x) R{,5}(x).1.1 1e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

29 Finding the Optimal : Concrete Example, e x 3 of The Absolute Error. R{5,}(x) R{4,1}(x) R{3,2}(x) R{2,3}(x) R{1,4}(x) R{,5}(x).1.1 1e Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (11/21)

30 Finding the Optimal : Matlab Code. The algorithm in the book looks frightening! If we think in term of the matrix problem defined earlier, it is easier to figure out what is going on: % The Taylor Coefficients, a, a 1, a 2, a 3, a 4, a 5 a = [1 1 1/2 1/6 1/24 1/12] ; N = length(a); A = zeros(n-1,n-1); % m is the degree of q(x), and n the degree of p(x) m = 3; n = N-1-m; % Set up the columns which multiply q 1 through q m for i=1:m A(i:(N-1),i) = a(1:(n-i)); end % Set up the columns that multiply p 1 through p n A(1:n,m+(1:n)) = -eye(n) % Set up the right-hand-side b = - a(2:n); % Solve c = A\b; Q = [1 ; c(1:m)]; % Select q through q m P = [a ; c((m+1):(m+n))]; % Select p through p n Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (12/21)

31 Finding the Optimal Optimal? One Point Optimal Points Polynomials Taylor Chebyshev Rational Functions Padé??? From the example e x we can see that Padé approximations suffer from the same problem as Taylor polynomials they are very accurate near one point, but away from that point the approximation degrades. Chebyshev-placement of interpolating points for polynomials gave us an optimal (uniform) error bound over the interval. Can we do something similar for rational approximations??? Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (13/21)

32 Finding the Optimal Chebyshev Basis for the! We use the same idea instead of expanding in terms of the basis functions x k, we will use the Chebyshev polynomials, T k (x), as our basis, i.e. n k= r n,m (x) = p kt k (x) m k= q kt k (x) where N = n + m, and q = 1. Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (14/21)

33 Finding the Optimal Chebyshev Basis for the! We use the same idea instead of expanding in terms of the basis functions x k, we will use the Chebyshev polynomials, T k (x), as our basis, i.e. n k= r n,m (x) = p kt k (x) m k= q kt k (x) where N = n + m, and q = 1. We also need to expand f (x) in a series of Chebyshev polynomials: f (x) = a k T k (x), k= so that k= f (x) r n,m (x) = a kt k (x) m k= q kt k (x) n k= p kt k (x) m k= q. kt k (x) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (14/21)

34 Finding the Optimal The Resulting Equations Again, the coefficients p, p 1,...,p n and q 1, q 2,...,q m are chosen so that the numerator has zero coefficients for T k (x), k =, 1,...,N, i.e. m a k T k (x) q k T k (x) k= k= n p k T k (x) = k= We will need the following relationship: k=n+1 γ k T k (x). T i (x)t j (x) = 1 2 [ Ti+j (x) + T i j (x) ]. Also, we must compute (maybe numerically) a = 1 1 f (x) π dx and a k = 2 1 f (x)t k (x) dx, k x 2 π 1 1 x 2 Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (15/21)

35 Finding the Optimal Example: Revisiting e x with Chebyshev- 1/5 The 8 th order Chebyshev-expansion (All Praise Maple) for e x is P CT 8 (x) = T (x) T 1 (x) T 2 (x) T 3 (x) T 4 (x) T 5 (x) T 6 (x) T 7 (x) T 8 (x) and using the same strategy building a matrix and right-hand-side utilizing the coefficients in this expansion, we can solve for the Chebyshev-Padé polynomials of degree (n + 2m) 8: Next slide shows the matrix set-up for the r3,2 CP (x) approximation. Note: Due to the folding, T i (x)t j (x) = 1 [ 2 Ti+j (x) + T i j (x) ], we need n + 2m Chebyshev-expansion coefficients. (Burden- Faires do not mention this, but it is obvious from algorithm 8.2; Example 2 (p. 519) is broken, it needs P 7 (x).) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (16/21)

36 Finding the Optimal Example: Revisiting e x with Chebyshev- 2/5 [ T (x) : 1 2 [ T 1 (x) : 1 2 [ T 2 (x) : 1 2 [ T 3 (x) : 1 2 [ T 4 (x) : 1 2 [ T 5 (x) : 1 2 ] (2a + a 2 )q 1 + (a 1 + a 3 )q 2 2p 1 = 2a 1 ] (a 1 + a 3 )q 1 + (2a + a 4 )q 2 2p 2 = 2a 2 ] (a 2 + a 4 )q 1 + (a 1 + a 5 )q 2 2p 3 = 2a 3 ] (a 3 + a 5 )q 1 + (a 2 + a 6 )q 2 = 2a 4 ] (a 4 + a 6 )q 1 + (a 3 + a 7 )q 2 = 2a 5 ] Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (17/21)

37 Finding the Optimal Example: Revisiting e x with Chebyshev- 3/5 R CP 4,1 (x) = T (x) T 1 (x) T 2 (x) T 3 (x) T 4 (x) T (x) T 1 (x) R CP 3,2 (x) = T (x) T 1 (x) T 2 (x) T 3 (x) T (x) T 1 (x) T 2 (x) R CP 2,3 (x) = T (x) T 1 (x) T 2 (x) T (x) T 1 (x) T 2 (x) T 3 (x) R CP 1,4 (x) = T (x) T 1 (x) T (x) T 1 (x) T 2 (x) T 3 (x) T 4 (x) Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (18/21)

38 Finding the Optimal Example: Revisiting e x with Chebyshev- 4/ Error for Chebyshev Pade 4 1 Approximation x Error for Chebyshev Pade 3 2 Approximation x Error for Chebyshev Pade 2 3 Approximation x Error for Chebyshev Pade 1 4 Approximation x Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (19/21)

39 Finding the Optimal Example: Revisiting e x with Chebyshev- 5/ x 1 4 Error Comparison for 4 1 Approximations Error Comparison for 3 2 Approximations x Chebyshev Pade Pade Chebyshev Pade Pade x 1 4 Error Comparison for 2 3 Approximations Error Comparison for 1 4 Approximations x Chebyshev Pade Pade Chebyshev Pade Pade Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (2/21)

40 Finding the Optimal The Bad News It s Not Optimal! The Chebyshev basis does not give an optimal (in the min-max sense) rational approximation. However, the result can be used as a starting point for the second Remez algorithm. It is an iterative scheme which converges to the best approximation. A discussion of how and why (and why not) you may want to use the second Remez algorithm can be found in Numerical Recipes in C: The Art of Scientific Computing (Section 5.13). [You can read it for free on the web ( ) just Google for it!] ( ) The old 2nd Edition is Free, the new 3rd edition is for sale... Joe Mahaffy, mahaffy@math.sdsu.edu Rational Function Approximation (21/21)