Numerical Analysis: Approximation of Functions

Transcription

1 Numerical Analysis: Approximation of Functions Mirko Navara navara/ Center for Machine Perception, Department of Cybernetics, FEE, CTU Karlovo náměstí, building G, office 104a October 13, 2016 Requirements for the assessment Adequate presence at seminars (you may install Maple on your computer* and do the tasks elsewhere). * Contact Aleš Němeček for the license. Tasks 6 5 points 18 points suffice References [Navara, Němeček] Navara, M., Němeček, A.: Numerical Methods (in Czech). 2nd edition, CTU, Prague, [Knuth] Knuth, D.E.: Fundamental Algorithms. Vol. 1 of The Art of Computer Programming, 3rd ed., Addison- Wesley, Reading, MA, [KJD] Kubíček, M., Janovská, D., Dubcová, M.: Numerical Methods and Algorithms. Institute of Chemical Technology, Prague, [Num. Recipes] Press, W.H., Flannery, B.P., Teukolsky, S.A., Vetterling, W.T.: Numerical Recipes (The Art of Scientific Computing). 2nd edition, Cambridge University Press, Cambridge, [Stoer, Bulirsch] Stoer, J., Bulirsch, R.: Introduction to Numerical Analysis. Springer Verlag, New York, Handbook of Linear Algebra. Chapman & Hall/CRC, Boca Ra- [Handbook Lin. Alg.] Hogben, L. (ed.): ton/london/new York, Motivation Task: A polynomial can be factorized to the product of linear polynomials, but only for orders up to 4. Numerical solution is possible, but it is imprecise and division leaves a remainder. Also this is a subject of numerical anaysis. Task: Linear algebra: A system of linear equations Ax = b has a unique solution x = A 1 b if and only if det A 0. Programming: Real numbers cannot be tested for equality! Numerical anaysis: Imprecision of coefficients causes imprecision of outputs, which for det A 0 exceeds all bounds. This is called an ill-conditioned task. Task: Linear algebra: det A is a sum of products ( 1) sign(p) a 1,p(1) a 2,p(2)... a n,p(n) = ( 1) sign(p) a i,p(i), p p i n 1

2 where we sum up over all permutations p of indices 1,..., n. Numerical anaysis: This represents appr. n n! operations. n no. of operations Gaussian Elimination has a complexity proportional to n 3. APPROXIMATION OF FUNCTIONS Task: Estimate future currency exchange rates from the preceding ones. Task: A random variable with Gaussian normal distribution has a cumulative distribution function Φ(u) = 1 u ( ) t 2 exp dt. 2π 2 This is a transcendent function. Numerical integration gives an approximate result with given precision. (In fact, even the exponential function is computed only numerically, only the 4 basic arithmetical operations are implemented in the processor.) Faster evaluation can be done with a prepared table of values of Gauss integral. Task: Estimate the remaining capacity of a battery from its voltage. We use finitely many values, but we need an approximation on the whole interval. Moreover, we require monotonicity. Task: A digital image is zoomed, rotated, etc. This changes the number of pixels and their orientation. Draw the V-A characteristic of a diode from measured data: Motivating problems on approximation 1. Task 1: Dependence of an exchange rate on time. 2. Task 2: Fast estimation of Gauss integral (using a preprocessed table). 3. Task 3: V-A characteristic of a diode based on measured values. 2

3 4. Task 4: Temperatures from a meteorological station. Basic task of approximation Given: n different points x 0,..., x n 1 R (nodes) y 0,..., y n 1 R set of functions F, defined at least in the nodes x 0,..., x n 1 Task: Find a function ϕ F, which minimizes the differences y i ϕ(x i ), i = 0,..., n 1. Assumption: F is the linear hull of k known, so-called approximating functions ϕ 0,..., ϕ k 1, k n: { } F = ϕ 0,..., ϕ k 1 = c j ϕ j : c j R. It remains to determine real coefficients c j, j = 0,..., k 1, of the linear combination j<k ϕ = j<k c j ϕ j. Linearity of the procedure of approximation We mostly assume linear dependence of the output on the inputs ( superposition principle ); this is related to the independence of the output on the chosen scale. Than any approximation depends linearly on the entries of the arithmetical vector y = (y 0,..., y n 1 ) R n. The resulting approximation is ϕ = j<k c j ϕ j, where ϕ = (ϕ(x 0 ),..., ϕ(x n 1 )) R n, ϕ j = (ϕ j (x 0 ),..., ϕ j (x n 1 )) R n, j = 0,..., k 1. (No other values are used.) Vector formulation of approximation: For a given vector y, we look for the nearest vector ϕ from the linear hull of known vectors ϕ j, j = 0,..., k 1. Vector y has coordinates (y 0,..., y n 1 ) w.r.t. the standard basis ψ 0 = (1, 0,..., 0), ψ1 = (0, 1, 0,..., 0),..., ψn 1 = (0,..., 0, 1), 1. For j = 0,..., n 1, we can put y = ψ j and find approximating functions ϱ j, represented by vectors ϱ j. 2. For a general input vector y, the approximation is a linear combination of results of the special cases: ϕ = j<n y j ϱ j. Functions ϱ j, j = 0,..., n 1, give sufficient information about the solution for any y = (y 0,..., y n 1 ). Interpolation Special case of approximation, where we require exact match at nodes. Given: n different nodes x 0,..., x n 1 R y 0,..., y n 1 R set of functions F, defined at least at the nodes x 0,..., x n 1 Task: Select a function ϕ F such that ϕ(x i ) = y i, i = 0,..., n 1. Simple interpolation 3

4 Assumption: Substitution gives { } F = ϕ 0,..., ϕ n 1 = c j ϕ j : c j R. j<n c j ϕ j (x i ) = y i, i = 0,..., n 1, j<n which is a system of n linear equations for unknowns c j, j = 0,..., n 1. For uniqueness, we put k = n and we require linear independence of arithmetical vectors ϕ j = (ϕ j (x 0 ),..., ϕ j (x n 1 )), j = 0,..., k 1. The complexity is proportional to n 3 ; we shall achieve smaller complexity for special tasks. Polynomial interpolation We interpolate by a polynomial of degree less than n, i.e., n 1. It has n coefficients, which is needed for existence and uniquness of the solution. Interpolating functions can be chosen as ϕ j (t) = t j, j = 0,..., n 1. It may be advantageous to use another bases of the n-dimensional linear space of all polynomials of degree less than n. It is easy to integrate them, differentiate, etc. Solvability: Advantages of polynomial interpolation Theorem: There is exactly on polynomial of degree less than n which solves the interpolation task for n nodes. Universality: Weierstrass Theorem: Let f be a continuous function at a closed interval I and let ε > 0. Then there exists a polynomial ϕ such that t I : f(t) ϕ(t) ε. Disadvantages of polynomial interpolation Weierstrass Theorem does not say anything about the degree of the polynomial, thus it may be useless. Polynomial dependencies are rather rare in practice. Simple interpolation works, but we shall show better methods. Lagrange construction of the interpolating polynomial 1. We solve special cases when the jth entry of vectoru y is unit, others are zeros; this results in polynomials ϱ j, j = 0,..., n 1, { 1 if i = j, ϱ j (x i ) = δ ij = 0 if i j. (δ ij is the Kronecker delta) Polynomial ϱ j of degree n 1 has n 1 roots x 0,..., x j 1, x j+1,..., x n 1 ; ϱ j (t) = e j (t x 0 )... (t x j 1 )(t x j+1 )... (t x n 1 ) = e j (t x i ), i<n,i j 4

5 where e j follows from the equality ϱ j (x j ) = 1: 1 e j = (x j x i ), ϱ j(t) = i<n,i j i<n,i j i<n,i j 2. The general solution is the linear combination ϕ = y j ϱ j, j<n (t x i ) (x j x i ). Check: ϕ(t) = y j ϱ j (t) = y j j<n j<n (t x i ) i<n,i j i<n,i j (x j x i ). ϕ(x i ) = y j ϱ j (x i ) = y j δ ij = y i. j<n j<n The complexity is proportional to n 2. The idea of the Lagrange construction of the interpolating polynomial is applicable to more general tasks. Motivation for dissatisfaction: Approximating exchange rates, we continuously receive new data. Is it necessary to comput all again from scratch? Some intermediate results can be used, provided that we recorded them. The previous result can be used and only corrected by certain polynomial. Newton construction of the interpolating polynomial 5

6 It is still the same (unique) polynomial The constant polynomial c 0 = y 0 has the right value at x 0. The resulting polynomial ϕ is a sum of an appropriate polynomial ω 0 vanishing at x 0 : t (t x 0 ) ω 0 (t), where ω 0 is a polynomial of degree n 2: ϕ(t) = c 0 + (t x 0 ) ω 0 (t), c 0 = y 0, ω 0 (t) = ϕ(t) c 0 t x 0 (cancel out). Values at nodes: ω 0 (x i ) = y i c 0 x i x 0. Induction: Polynomial ω 0 is of the form (ω 1 is a polynomial of degree n 3): ω 0 (t) = c 1 + (t x 1 ) ω 1 (t), c 1 = ω 0 (x 1 ), ω 1 (t) = ω 0(t) c 1 t x 1, ω 1 (t) = c 2 + (t x 2 ) ω 2 (t), c 2 = ω 1 (x 2 ), ω 2 (t) = ω 1(t) c 2 t x 2, ω 2 (t) = c 3 + (t x 3 ) ω 3 (t), c 3 = ω 2 (x 3 ), ω 3 (t) = ω 2(t) c 3 t x 3,... ω n 2 (t) = c n 1 = ω n 3 (x n 2 ). 6

7 Back substitution gives Expansion (not recommended!): ϕ(t) = c 0 + (t x 0 ) [c 1 + (t x 1 ) [c (t x n 3 ) [c n 2 + (t x n 2 ) c n 1 ]...]]. ϕ(t) = c 0 + (t x 0 ) c 1 + (t x 0 ) (t x 1 ) c c n 1 = c j (t x i ). j<n i<j i<n 1 (t x i ) The complexity of the computation of coefficients is n 2, then each function value is obtained with the complexity n. Nevill s algorithm We need to evaluate the interpolating polynomial just for a single argument t; the coefficients are irrelevant. One point (x i, y i ) gives an approximation by a constant polynomial y i. Two points (x i, y i ), (x i+1, y i+1 ) are approximated by a linear polynomial; its value at t is a linear combination of values y i, y i+1, namely y i,i+1 = (t x i+1) y i + (x i t) y i+1 x i x i+1, etc. y i,i+1,...,i+m 1... the value at t of the interpolating polynomial with m nodes x i, x i+1,..., x i+m 1 y i+1,...,i+m 1,i+m... the value at t of the interpolating polynomial with m nodes x i+1,..., x i+m 1, x i+m The value at t of the interpolating polynomial with m + 1 nodes x i, x i+1,..., x i+m is y i,i+1,...,i+m = (t x i+m) y i,i+1,...,i+m 1 + (x i t) y i+1,...,i+m 1,i+m x i x i+m We proceed from values y 0, y 1,..., y n 1 to y 0,1,...,n 1 = ϕ(t) = result. Numerical errors can be reduced if, instead of the polynomial values, we compute only their differences, using recurrent formulas The result is, e.g., C i,m = y i,...,i+m y i,...,i+m 1, D i,m = y i,...,i+m y i+1,...,i+m, C i,m = (x i t) (C i+1,m 1 D i,m 1 ) x i x i+m, D i,m = (x i+m t) (C i+1,m 1 D i,m 1 ) x i x i+m. y 0,1,...,n 1 = y 0 + n 1 m=1 C 0,m. Error of approximation by the interpolating polynomial Zero at nodes (only numerical errors occur). Error in other points if we approximate a function f by the interpolating polynomial. Derivation of the error of approximation by the interpolating polynomial Assumptions: f has continuous derivatives up to order n at a closed interval I {x 0,..., x n 1 }, u I \ {x 0,..., x n 1 } 7

8 We define a function g u (t) = f(t) ϕ(t) P u W (t), }{{} error of interpolation at t f(u) ϕ(u) where P u = is a suitable constant, W (u) W (t) = (t x i ) is a polynomial of degree n with roots x 0,..., x n 1 and a unit coefficient at the highest i<n power, t n (these conditions determine the polynomial uniquely), Its nth derivative is W (n) (t) = n! g u = f ϕ P u W has roots u, x 0,..., x n 1 and a continuous nth derivative g (n) u = f (n) P u n! Between every two roots of g u, there is a root of its derivative g u. Between n + 1 roots of g u, there are n roots of g u, n 1 roots of g u,... 1 root of g (n) u, say ξ. 0 = g u (n) (ξ) = f (n) (ξ) P u n! = f (n) f(u) ϕ(u) (ξ) n! W (u) f(u) ϕ(u) = f (n) (ξ) n! f(u) ϕ(u) = f (n) (ξ) n! f(u) ϕ(u) = f (n) (ξ) n! W (u) W (u) W (u) We replace f (n) (ξ) by its upper estimate M n : t I : f (n) (t) M n f(u) ϕ(u) M n n! W (u) Using an upper estimate W of W, t I : W (t) W, we get an upper estimate independent of u: f(u) ϕ(u) M n n! W 8

9 (Red: upper estimate of error; green: real error.) Two cases: Extrapolation: approximation outside the interval I(x 0,..., x n 1 ) of nodes Interpolation (in the narrow sense): approximation inside I(x 0,..., x n 1 ) Recommendation for minimization of error: cosine distribution of nodes On interval [ 1, 1], choose nodes on a general interval [a, b] choose x i = b + a 2 z i = cos π(i ), i = 0,..., n 1, n + b a 2 z i = b + a 2 + b a 2 cos π(i ) n Approximation error of polynomial interpolation for a cosine distribution of nodes 9

10 (Red: upper estimate of error; green: real error.) Motivating problem - polynomial interpolation Dependence of interpolating polynomial on local changes 10

11 Hermite interpolating polynomial - example Given: x 0 = 0, x 1 = 1, y 0,0, y 1,0, y 0,1, y 1,1 R Task: Find a polynomial ϕ of degree at most 3, such that ϕ(x 0 ) = y 0,0, ϕ(x 1 ) = y 1,0, ϕ (x 0 ) = y 0,1, ϕ (x 1 ) = y 1,1 As in the Lagrange construction of the interpolating polynomial, we first find polynomials η, ϱ, σ, τ of degree 3: ψ ψ(0) ψ(1) ψ (0) ψ (1) η ϱ σ τ Polynomial η has a double root 1, thus it is of the form η(t) = (a 0 t + b 0 )(t 1) 2, 11

12 where a 0, b 0 can be determined from the values at 0: ϱ has a double root 0, thus it is of the form where a 1, b 1 can be determined from the values at 1: η(0) = b 0 = 1 η (0) = a 0 2b 0 = 0 a 0 = 2, b 0 = 1, η(t) = (2t + 1)(t 1) 2 ϱ(t) = (a 1 t + b 1 ) t 2, ϱ(1) = a 1 + b 1 = 1 ϱ (1) = 3a 1 + 2b 1 = 0 a 1 = 2, b 1 = 3, ϱ(t) = ( 2t + 3)t 2 σ has a double root 1 and a single root 0, thus it is of the form where c 0 can be determined from the derivative at 0: σ(t) = c 0 t (t 1) 2, σ (0) = c 0 = 1 σ(t) = t(t 1) 2 τ has a double root 0 and a single root 1, thus it is of the form where c 1 can be determined from the derivative at 1: τ(t) = c 1 t 2 (t 1), τ (1) = c 1 = 1 τ(t) = t 2 (t 1) 12

13 ϕ is a linear combination of η, ϱ, σ, τ, ϕ = y 0,0 η + y 1,0 ϱ + y 0,1 σ + y 1,1 τ Approximation by Taylor series (more exactly, by Taylor polynomial) Special case of Hermite interpolating polynomial with one node, where the first n derivatives (including the 0 s) are given. Given: Node x 0 n values y 0,0, y 0,1,..., y 0,n 1 R Task: Find a polynomial ϕ of degree less than n, such that The solution is of the form ϕ (j) (x 0 ) = y 0,j, j = 0,..., n 1. ϕ = y 0,j ψ j, j<n ψ (i) j (x 0 ) = δ ij, ψ j (t) = 1 j! (t x 0) j ϕ(t) = j<n y 0,j j! (t x 0 ) j 13

14 If y 0,0, y 0,1,..., y 0,n 1 are the derivatives of some function f at x 0, i.e., y 0,j = f (j) (x 0 ), then ϕ is a finite Taylor series with center x 0 : ϕ(t) = f (j) (x 0 ) (t x 0 ) j. j! j<n If f has a continuous nth derivative at interval I(u, x 0 ), then f(u) ϕ(u) = f (n) (ξ) n! (u x 0 ) n, where ξ I(u, x 0 ) f(u) ϕ(u) M n n! (u x 0 ) n The only difference from the error of the interpolating polynomial is that polynomial W (u) with roots x 0,..., x n 1 is replaced by polynomial (u x 0 ) n (of the same degree) with a single root x 0 of multiplicity n. Approximation by Taylor polynomial is very precise in the neighborhood of x 0, but less precise in more distant points. Approximations by Taylor polynomials of degrees 5, 10, and 15 for positive arguments. Approximation by Taylor polynomial 14

15 Approximations by Taylor polynomials of degrees 5, 10, and 15 for negative arguments. Approximation by Taylor polynomial 15

16 Approximations by Taylor polynomials of degrees 5, 10, and 15 for positive and negative arguments. Spline interpolation Disadvantage of polynomial interpolation: small change of input can cause big change of output at distant points (the influence is not localized). Spline is a piecewise polynomial functionãŋ. (It coincides with polynomials of small degree at particular intervals.) The simplest case is linear spline, which is the approximation by a piecewise linear function. Cubic spline Given: n nodes x 0,..., x n 1 ordered increasingly, n values y 0,..., y n 1. Task: Find function ϕ, defined on interval [x 0, x n 1 ], such that: ϕ(x i ) = y i, i = 0,..., n 1, at each interval [x i 1, x i ], i = 1,..., n 1, function ϕ coincides with a polynomial ϕ i of degree at most 3, the first two derivatives of ϕ are continuous at interval [x 0, x n 1 ]. (We shall find it necessary to make this task more precise.) It suffices to ensure continuity of derivatives at n 1 nodes x 1,..., x n 2 : ϕ i(x i ) = ϕ i+1(x i ), i = 1,..., n 2, ϕ i (x i ) = ϕ i+1(x i ), i = 1,..., n 2. Suppose that values ϕ (x i ) = ϕ i (x i) = ϕ i+1 (x i), i = 1,..., n 2 are known. Then ϕ is continuous. 16

17 Polynomials ϕ i, i = 1,..., n 1 can be found just as the Hermite interpolating polynomial in the previous example; the only difference is that the nodes 0, 1 are replaced by x i 1, x i. The general case is obtained by an easy linear transformation of coordinates: t = x i 1 + (x i x i 1 ) u, u = t x i 1 x i x i 1. On interval [x i 1, x i ], i = 1,..., n 1, we get ϕ i (t) = y i 1 η i (t) + y i ϱ i (t) + ϕ (x i 1 )σ i (t) + ϕ (x i )τ i (t), where η i, ϱ i, σ i, τ i are polynomials of degree at most 3 (computed as the polynomials η, ϱ, σ, τ in the example of Hermite interpolating polynomial). It remains to find ϕ (x i ), i = 0,..., n 1: y i 1 η i (x i ) + y i ϱ i (x i ) + ϕ (x i 1 )σ i (x i ) + ϕ (x i )τ i (x i ) = = y i η i+1(x i ) + y i+1 ϱ i+1(x i ) + ϕ (x i )σ i+1(x i ) + ϕ (x i+1 )τ i+1(x i ), where i = 1,..., n 2 (system of n 2 linear equations with n unknowns). There remain 2 undefined parameters. The usual choice ϕ (x 0 ) = ϕ (x n 1 ) = 0 leads to so-called natural spline. This choice influences the result only at the endpoints of the interval [x 0, x n 1 ]. The algorithm consists of 2 parts: 1. Computation of coefficients ϕ (x i ), i = 0,..., n Evaluation of the spline. The matrix of the system of linear equations is three-diagonal (its particular properties allow more effective solutions). Splins cannot be used for extrapolation! Still many implementations allow it. Comment: The choice of nodes has smaller influence than in polynomial interpolation. Influence of local changes on a spline Influence of local changes on a spline 17

18 Motivating problem - spline Given: n nodes x 0,..., x n 1 n values y 0,..., y n 1 k functions ϕ 0,..., ϕ k 1, k n, defined at least at the nodes. Task: Find coefficients c 0,..., c k 1 R of a linear combination of functions ϕ j ϕ = j<k c j ϕ j which minimizes H 2 = (ϕ(x i ) y i ) 2 = i<n i<n ( ) 2 c j ϕ j (x i ) y i j<k Modified criteria H 2w = i<n w i (ϕ(x i ) y i ) 2 can be solved analogously, H 1 = i<n ϕ(x i ) y i 18

19 not used, the solution is not unique, H 0 = max i<n ϕ(x i) y i is more difficult to solve (Chebyshev approximation). Solution of approximation by LSM We introduce an inner (=scalar) product on R n, defined for u = (u 0,..., u n 1 ), v = (v 0,..., v n 1 ) by u v = i<n u i v i. We have to approximate vector y by a linear combination ϕ = c j ϕ j, the criterion is H 2 = ( ϕ y) ( ϕ y) = ϕ y 2. Solution: The orthogonal projection; it satisfies the system of equations (for m = 0,..., k 1) j<k ( ϕ y) ϕ m, ( ϕ y) ϕ m = 0, ϕ ϕ m = y ϕ m. Vectors ϕ and y have the same inner products with vectors ϕ m. ( ) c j ϕ j ϕ m = y ϕ m, j<k c j ( ϕ j ϕ m ) = y ϕ m, m = 0,..., k 1; j<k system of linear equations for unknowns c 0,..., c k 1 - system of normal equations. Special case: Approximation by a polynomial of degree less than k; we may choose ϕ j (t) = t j, ϕ j ϕ m = i<n x j i xm i = i<n x j+m i. Motivating problem - LSM 19

20 Approximation of V-A characteristic of a diode by a linear combination of a constant and two exponencials with appropriate bases. In linear subspace P = ϕ 0,..., ϕ k 1, we find an orthogonal basis ( ψ 0,..., ψ k 1 ), ψ j ψ m = 0 for j m. Orthogonality depends not only on functions ϕ 0,..., ϕ k 1, but also on the choice of nodes! We look for a solution in the form ϕ = d j ψ j, where d j, j = 0,..., k 1 are coordinates w.r.t. a new basis. The matrix of the system of normal equations is diagonal: d j ( ψj ψ j ) = y ψ j, j = 0,..., k 1, j<k d j = y ψ j ψ j ψ = y ψ j j ψ, j = 0,..., k 1. j 2 Moreover, vectors ψ j can be chosen unit, then also the denominator is 1. Constructs an orthogonal basis ψ 0 = ϕ 0 ψ 1 = ϕ 1 + α 10 ψ0 ψ 1 ψ 0 = 0 ϕ 1 ψ 0 + α 10 ψ0 ψ 0 = 0 α 10 = ϕ 1 ψ 0 ψ 0 ψ 0 ψ 2 = ϕ 2 + α 20 ψ0 + α 21 ψ1 ψ 2 ψ 0 = 0 ϕ 2 ψ 0 + α 20ψ0 ψ 0 + α 21 ψ1 }{{ ψ } 0 = 0 0 α 20 = ϕ 2 ψ 0 ψ 0 ψ 0 20

21 ψ 2 ψ 1 = 0 ϕ 2 ψ 1 + α 20 ψ0 }{{ ψ } 1 +α 21ψ1 ψ 1 = 0 0 α 21 = ϕ 2 ψ 1 ψ 1 ψ 1... ψ j = ϕ j + m<j α jm ψm p, p < j : ψ j ψ p = 0 = ϕ j ψ p + m<j α jm ψm ψ p = ϕ j ψ p + α jp ψp ψ p α jp = ϕ j ψ p ψ p ψ p Approximation by LSM with the choice of approximating functions 1, cos 2π t T, sin 2π t T, cos 4π t T, sin 4π t T,... For equidistant nodes at an interval of length T, (at most n) vectors ϕ j are orthogonal. Given: Bounded interval I, continuous function f na I, k N Task: Find a polynomial ϕ of degree less than k minimizing x i = a + i T, i = 0,..., n 1, n H 0 = max ϕ(t) f(t) t I There are methods solving this task, but their complexity is much higher; instead of that, we usually apply modified LSM: For simplicity, we take interval I = 1, 1 ; generalization to any interval a, b is done by a linear transformation the inverse transformation is x = b + a 2 z = x b+a 2 b a 2 + b a 2 z, We choose n k nodes x 0,..., x n 1 1, 1 with cosine distribution: z i = cos π(i ) n., i = 0,..., n 1. For a basis of the linear space of all polynomials of degree less than k, we take Chebyshev polynomials: 21

22 Chebyshev polynomials They can be computed by a recurrent formula ϕ j (t) = cos(j arccos t), j = 0,..., k 1, k < n. ϕ 0 (t) = 1, ϕ 1 (t) = t, ϕ j (t) = 2tϕ j 1 (t) ϕ j 2 (t), j 2 The range of functions ϕ j, j 1, at interval I is 1, 1. The vectors ϕ 0,..., ϕ k 1 are orthogonal, 0 if j m, n ϕ j ϕ m = 2 if j = m > 0, n if j = m = 0. For k = n, we obtain the interpolating polynomial (with the recommended distribution of nodes). The solution for k < n differs by ignoring the summands of higher order. Coefficients c k,..., c n 1 are usually small (they depend on higher derivatives of the approximated function!). An upper estimate of the error at nodes is n 1 ϕ(x i ) f(x i ) c j. j=k Remarks on Chebyshev approximation We do not optimize the criterion H 0, but the result is not much different from the optimal solution. We cannot say much about the error at other points than the nodes; nevertheless, the method can be recommended. The recurrent formula is used not only for computation of Chebyshev polynomials, but also to their direct evaluation at given points. It is not recommended to expand the result to the standard form ϕ(t) = b j t j. Chebyshev approximation can be generalized to an approximation by a product of a known function and an unknown polynomial. j<k 22