Discrete Orthogonal Polynomials on Equidistant Nodes

Transcription

1 International Mathematical Forum, 2, 2007, no. 21, Discrete Orthogonal Polynomials on Equidistant Nodes Alfredo Eisinberg and Giuseppe Fedele Dip. di Elettronica, Informatica e Sistemistica Università della Calabria Arcavacata di Rende (CS Italy Abstract In this paper we give an alternative and, in our opinion, more simple proof for the orthonormal discrete polynomials on a set of equidistant nodes. Such a proof provides a unifying explicit formulation of discrete orthonormal polynomials on an equidistant grid and an explicit formula for the coefficients of the three-term recurrence relation. We show how to efficiently apply these formulas to the problem of least-squares fitting on equidistant nodes. Finally we investigate the problem to determine the effect of adding a new basis function or taking one away. This is useful in the process of trying to discover the optimal set of basis functions and the problem to update them when a moving window over the time series/data stream is used. Mathematics Subject Classification: 05E35, 93E24 Keywords: Orthogonal polynomials, Least-squares 1 Introduction Although orthogonal polynomials over discrete sets were considered as early as the middle of the nineteenth century by Chebyshev [5, 6, 10, 11], comparatively little attention had been paid to them until now. They appear naturally in combinatorics, genetics, statistics and various areas in applied mathematics (see, for example, [7, 11]. Expansion in orthogonal polynomials is used, for example, to avoid the difficulties with ill-conditioned systems of equations which occur in a least squares applications [1]. The Gram polynomials [3, 1], orthonormal on the set of equidistant nodes in [ 1, 1], are used

2 1008 A. Eisinberg and G. Fedele in all the applications in which equidistant measures must be used. In this paper we focus our attention on discrete orthonormal polynomials. We derive an analytic expression for orthonormal polynomials on a discrete grid Z n = {z r z r = a + hr, r =0, 1,..., n 1} and the coefficients of the threeterm recurrence relation by using some results of the pseudoinverse on integer nodes [4]. We show how to efficiently apply these formulas to the problem of least-squares fitting on equidistant nodes. Finally we investigate the problem to determine the effect of adding a new basis function or taking one away as part of a process of trying to discover the optimal set of basis functions and the problem to update the basis functions when a moving window over the time series/data stream is used. We want to emphasize that although high degree approximations on equidistant nodes have a bad reputation in numerical analysis, a lot of researches have been made for the design of accurate algorithms for polynomial approximation on such a set of nodes. There is a considerable interest on equidistant nodes because this choice frequently occurs in many applications. 2 Preliminary results Let f 1 and f 2 be real valued functions defined on the set of nodes X n = {x 1,x 2,..., x n } and introduce the inner product: f 1 (x,f 2 (x = n f 1 (x k f 2 (x k. (1 k=1 A family P = {p 1 (x,p 2 (x,..., p m (x} with m n of polynomials is orthogonal respect to this inner product if the following properties hold: p k (x,p q (x =0, k q; k, q =1, 2,..., m p k (x,p k (x = ξ k 0, k =1, 2,..., m. (2 If ξ k =1, k =1, 2,..., m then polynomials p k are said orthonormal on the set of nodes X n. A well known result from the general theory of orthogonal polynomials is that the orthogonal polynomials satisfy a three term recursion formula: p k (x =(α k x + β k p k 1 (x+γ k p k 2 (x, k =3, 4,..., m. (3 Proposition 1 Let X n be a set of n nodes and let P be a family of m orthogonal polynomials on X n, then

3 Discrete orthogonal polynomials on equidistant nodes 1009 if and only if p i (x j =( 1 i+1 p i (x n+1 j, j =1, 2,..., n; i =1, 2,..., m (4 α i (x j + x n+1 j +2β i =0, i =1, 2,..., m; j =1, 2,..., n. (5 Proof. The proof is made by induction on the polynomial index i. Suppose (5 is true for i 1 and i 2 then and p i 1 (x j =( 1 i p i 1 (x n+1 j, j =1, 2,..., n (6 p i 2 (x j =( 1 i+1 p i 2 (x n+1 j, j =1, 2,..., n (7 ( Taking into account (3, the (4 becomes: (α i x j + β i p i 1 (x j +γ i p i 2 (x j = ( 1 i+1 [(α i x n+1 j + β i p i 1 (x n+1 j +γ i p i 2 (x n+1 j ] (8 By using (6 and (7 the (5 follows. ( Taking into account (6 and (7, p i (x n+1 j can be written as p i (x n+1 j = ( 1 i (α i x n+1 j + β i p i 1 (x j + +( 1 i+1 p i 2 (x j. (9 Considered that (α i x n+1 j + β i = (α i x j + β i then (4 follows. Here we review a result useful in the sequel. Let B k be the matrix defined as follows: B k (i, j = n p i+j 2, i,j =1, 2,..., k (10 p=1

4 1010 A. Eisinberg and G. Fedele then its inverse can be factorized as [4]: B 1 k = W k Λ 1 k W T k. (11 In (11, W k and Λ 1 k have the following expressions: (j 1! W k (i, j = where ( 2j 2 j 1 n s 1 ( 1 i+j (! ( s + j 2 ( n s n j [s [ ] i are the Stirling numbers of the first kind [8], and j Λ 1 k { ( 2i 2 } i 1 = diag [(i 1!] 2 ( n+i 1 2i 1 i i=1,2,...,k 3 Discrete orthogonal polynomials ], i,j =1, 2,..., k, (12. (13 Let P = {p 1 (x,p 2 (x,..., p m (x} be the set of polynomials defined as p k (x = k c k,i x i 1, k =1, 2,..., m. (14 i=1 The set P is orthogonal on the set of nodes X n = {1, 2,..., n} if the following equalities hold: p k (x,x q 1 =0, k =2, 3,..., m; q =1, 2,..., k 1, (15 p k (x,x k 1 = ρ k 0, k =1, 2,..., m. (16 Let c k and b k (k =1, 2,..., m, be two vectors defined as follows: c k (i =c k,i, bk (i =ρ k δ k,i, i =1, 2,..., k i =1, 2,..., k (17 then by (15 and (16 and taking into account (11, it must be

5 Discrete orthogonal polynomials on equidistant nodes 1011 It is straightforward to note that: c k = W k Λ 1 k W T k b k. (18 and [ Λ 1 k W k T b k ]i = ρ k W T k b k = b k (19 ( 2k 2 k 1 [(k 1!] 2 ( n+k 1 2k 1 Then, by simple algebraic manipulations we have: δ k,i, i =1, 2,..., k. (20 c k,i = μ k ( 1 i+k k s=1 1 (! ( s + k 2 ( n s n k [s i ], i =1, 2,..., k, (21 where μ k = ρ k (k 1! (, k =1, 2,..., m. (22 n+k 1 2k 1 By using the identity [8]: [ s ( 1 i i i ] x i 1 =( 1 s (! ( x 1 (23 polynomial p k (x becomes: p k (x =μ k If we put k s=1 ( ( s + k 2 n s ( 1 s+k n k ( x 1, k =1, 2,..., m. (24 ( n + k 1 ρ k =(k 1!, k =1, 2,..., m (25 2k 1 then the family P of polynomials

6 1012 A. Eisinberg and G. Fedele p k (x = k s=1 ( ( s + k 2 n s ( 1 s+k n k ( x 1, k =1, 2,..., m. (26 is orthogonal on the set X n = {1, 2,..., n}. By an affine transformation on the variable x in (26 we have that the polynomials p k (x = k s=1 ( 1 s+k ( s + k 2 are orthogonal on the set ( n s n k, k =1, 2,..., m. (27 ( x a h Z n = {z r z r = a + hr, r =0, 1,..., n 1} (28 Such polynomials satisfy the following three-term recurrence relation: where p k (x =(α k x + β k p k 1 (x+γ k p k 2 (x, k =3, 4,..., m (29 α k = 4k 6, h(k 1 2 β k = [(2k 3(2a+h(n 1] h(k 1 2, γ k = (k 22 n 2 (k 1 2, k =3, 4,..., m (30 Proposition 2 The set of polynomials Q = {q 1 (x,q 2 (x,..., q m (x} where q k (x = 1 (n+k 1 ( 2k 2 2k 1 k 1 k ( ( s + k 2 n s ( 1 s+k n k s=1 ( x 1, (31 k =1, 2,..., m is orthonormal on the set of nodes X n = {1, 2,..., n}. Proof. It is straightforward to note that p k (x,p k (x = c k,k ρ k, (32

7 Discrete orthogonal polynomials on equidistant nodes 1013 then by using (21 and the expression (25 for ρ k, we have p k (x,p k (x = ( ( n + k 1 2k 2 2k 1 k 1 (33 and the (31 follows. Corollary 3.1 The generic polynomial t k (x in the family of orthonormal polynomials T = {t 1 (x,t 2 (x,..., t m (x} on the set of nodes can be written as Z n = {z r z r = a + hr, r =0, 1,..., n 1} (34 t k (x = 1 (n+k 1 ( 2k 2 2k 1 k 1 k s=1 ( 1 s+k ( s + k 2 k =1, 2,..., m ( n s n k ( x a h, (35 The t k satisfies a three-term recurrence relation t 1 (x = 1 n t 2 (x = 3 [2x h(n 1 2a] h n(n 2 1 t k (x =(α k x + β k t k 1 (x+γ k t k 2 (x k =3, 4,..., n where α k = 2 (2k 1(2k 3 h(k 1 n 2 (k 1 2 β k = 1 2 [2a + h(n 1] α k (36 γ k = α k α k 1 k =3, 4,..., n Such explicit formulas are particularly useful because they allow to compute efficiently the orthonormal polynomials without using the Stieltjes procedure which can be sensitive to roundoff errors [13].

8 1014 A. Eisinberg and G. Fedele 4 Least squares application In this section we consider the problem of least squares on a discrete set Z n = {z r z r = a + hr, r =0, 1,..., n 1} (37 by using polynomials p k defined in (27 which are orthogonal on Z n. Such polynomials satisfies the three-term recurrence relation (29 with parameters (30. Note that p k (x,p k (x is invariant under affine transformation, therefore its value can be computed by using the (33. Here we want to expand a given function y = f(x in terms of orthogonal polynomials [9, 12]: f(x = m w i q i (x. (38 i=1 and to determine the coefficients w 1,w 2,..., w m in (38 such that the Euclidean norm of the error function f f is minimized, n 1 f f = f(z j f(z j 2. j=0 Since q 1 (x,q 2 (x,..., q m (x form an orthogonal system, the coefficients are computed more simple by w j = q j(x,f(x, j =1, 2,..., m. (39 q j (x,q j (x If we define the following quantities: where b = H f (40 H(i, j =q i (z j, i =1, 2,..., m; j =0, 1,..., n 1, (41 and f =[f(z 0,f(z 1,..., f(z n 1 ] (42

9 Discrete orthogonal polynomials on equidistant nodes 1015 then g(i = ( 2i 2 i 1 ( n + i 1 2i 1, i =1, 2,..., m (43 w i = b(i, i =1, 2,..., m. (44 g(i Since the product b = H f is also invariant under affine transformation, then it would be computed by considering, for example, the set of orthogonal polynomials on [1, 2,..., n] for which the matrix H is H(i, j = i s=1 ( 1 s+i ( s + i 2 ( n s n i ( j 1, (45 i =1, 2,..., m; j =1, 2,..., n. The pseudo-code in Table 1 finds the coefficients w i,i=1, 2,..., m in (38. We compare our algorithm (EF applied to the orthogonal polynomials on the set of n equidistant nodes in [0, 1] with that proposed in [2] (CB which costs 10mn flops and with the Mathematica built-in function Fit (MA [15]. The first two algorithms have been implemented in Mathematica package, which allows arbitrary precision numbers. Our algorithm costs 3.5mn flops. Infact we build an half part of the matrix H by the three-term recurrence relation, and the product H f by using the property in Proposition 1. For some values of m with n = 1000, we have generated one thousand vectors f, with entries uniformly distributed in [ 1, 1], and have computed the exact solution of the problem (38 f(x = m h i q i (x = i=1 m r k x k 1 (46 using extended precision. For each algorithm we have computed the maximum componentwise relative errors k=1 E EF = max 1 i m ˆr EF i r i r i (47 E CB = max 1 i m ˆr CB i r i r i (48

10 1016 A. Eisinberg and G. Fedele Table 1: Pseudo-code for least-square problem by discrete orthogonal polynomials. function lmsort Input: a, h, m, f Output: w i,i=1, 2,..., m Calculate the quantities α k,β k and γ k in (30. Calculate g k in (43. Build the matrix C P of the coefficients of the polynomials p k by using the three-term recurrence relation (29. Build the elements i =1, 2,..., m, j =1, 2,..., n 2 of the matrix H. Build the two vectors f 1 = { f(i, i=1, 2,..., n 2 } and f 2 = { f(n +1 i, i=1, 2,..., n 2 }. { Build the vector b = b(i = n 2 j=1 if n is odd then c h =1 b(1 = b(1 + f((n +1/2 for i =3:2:m c h = γ(ic h ; b(i =b(i+c h f((n +1/2 end } H(i, j(f 1 (j+( 1 i+1 f 2 (j.

11 Discrete orthogonal polynomials on equidistant nodes 1017 E MA = max 1 i m ˆr MA i r i r i (49 where ˆr EF,ˆr CB and ˆr MA are the approximate solutions computed by EF, CB and MA algorithms respectively in machine precision. The mean and the maximum of E EF, E CB and E MA over one thousand runs have then been computed and reported in Table 2. Table 2 reports also the fraction of trials in which the proposed algorithm gives equal or more accurate result than CB and MA algorithm. Note that the matrix H in (45 has integer entries that can be stored without rounding errors and this is probably the cause of the robustness of this algorithm. 5 Some properties A common problem in least-square problems is to determine the effect of adding a new basis function, that is to determine the effect of using m +1 basis in (38. This is important as part of a process of trying to discover the optimal set of basis functions. The effect can, of course, be calculated by restarting the algorithm in the previous section. However in our case, simple analytical formulas can be derived to cover this situation. Let C Pm be the matrix of the coefficients of the polynomial p k (x, k = 1, 2,..., m in (27, that is C P m = c 1, c 2,1 c 2, c 3,1 c 3,2 c 3, c m,1 c m,2 c m,3... c m,m = [ CP m 1 0 w T m and let G m be the matrix of the g k,k=1, 2,..., m in (43: c m,m ] (50 G m = g g = [ Gm T g m ] ( g m If we consider the matrix H in (41 as H m = [ Hm 1 h T m ] (52

12 1018 A. Eisinberg and G. Fedele then the coefficients of the approximant polynomial can be expressed in matrix form as C P T m G m H m f = [ CP T m 1 G m 1H m 1 f + g m (h T m fw m c m,m g m (h T m f ]. (53 Formula (53 is useful for the updating of the degree of the approximant polynomial in least square sense since it gives the coefficients of the approximant polynomial of degree m 1 in terms of the approximant polynomial of degree m 2. In real-time applications, which process a stream of continuously arriving data, it is often required a processing in moving time windows. More specific, let Z 1 and Z 2 be two discrete sets of nodes such as Z 1 = {a + h r, r =0, 1,..., n 1}, (54 Z 2 = {a + p + h r, r =0, 1,..., n 1, p>0} (55 and C P1 and C P2 the matrix of the coefficients of the orthogonal polynomial on Z 1 and Z 2 respectively, then where C P2 = C P1 M p (56 ( i 1 M(i, j =( 1 i+j, i,j =1, 2,..., m. (57 j 1 Proposition 3 The p-th power of the matrix M is ( i 1 M p (i, j =( 1 i+j p i j, i,j =1, 2,..., m. (58 j 1 The matrix M p has the following useful recursive properties: M p (i, i =1, i =1, 2,..., m, M p (i, 1 = pm p (i 1, 1, i =2, 3,..., m, M p (i, j = pm p (i 1,j+M p (i 1,j 1, i =3, 4,..., m, j =2, 3,..., i 1. (59

13 Discrete orthogonal polynomials on equidistant nodes 1019 m CB MA EF max mean max mean max mean EF vs CB EF vs MA Table 2: Maximum and mean value of E CB, E MA and E EF n = 1000, f [ 1, 1]. Success rate of EF algorithm. over 1000 runs, 6 Conclusions In this paper we have given an alternative proof of the explicit formula for the discrete orthogonal polynomials on equidistant nodes. Such a formula, with its properties, allows us to design an efficient algorithm for solving least squares problem. The numerical experiments indicate that our approach is more stable compared with existing Conte-de Boor algorithm. We finally have shown some properties satisfied by these polynomials. References [1] A. Björck, Numerical Methods for Least Squares Problems, SIAM, [2] S.D. Conte and C. De Boor, Elementary Numerical Analysis, McGraw-Hill, Second. ed., [3] G. Dahlquist and A. Björck, Numerical Methods, Prentice Hall, Englewood Cliffs, [4] A. Eisinberg, P. Pugliese and N. Salerno, Vandermonde matrices on integer nodes: the rectangular case. Numer. Math., 87 (2001, [5] F.B. Hildebrand, Introduction to Numerical Analysis, New York: Mc- Graw-Hill, [6] C. Jordan, Calculus Of Finite Differences, Chelsea Publ. Comp., New York, [7] S. Karlin and J. McGregor, Linear growth models with many types and multidimensional Hahn polynomials, in Theory and applications of special functions, 261?288, ed. R. A. Askey, Academic Press, New York, 1975.

14 1020 A. Eisinberg and G. Fedele [8] D.E. Knuth, The Art of Computer Programming,(2nd ed., Addison- Wesley, Reading Mass., [9] B.M. Mohan and B. Srinath, On the identification of discrete-time systems via discrete orthogonal functions, Computers Elec. Engng., 23 (1997, [10] R. Mukundan, Image Analysis by Tchebichef Moments, IEEE Trans. Image Proc., vol.10 no.9 (2001, [11] A.F. Nikiforov, S.K. Suslov, V.B. Uvarov, Classical orthogonal polynomials of a discrete variable, Springer-Verlag, Berlin, [12] M. Okabe, Discrete orthogonal polynomials inherent to sampling points at equal intervals, Comput. Methods Appl. Mech. Engng., 123 (1995, [13] L. Reichel, Fast QR Decomposition of Vandermonde-Like Matrices and Polynomial Least Squares Approximation, SIAM J. Matrix Anal. Appl., 3 (1991, [14] G. Szego, Orthogonal Polynomials, 4th ed., Amer. Math. Soc., Providence, RI, [15] S. Wolfram, Mathematica: a System for Doing Mathematics by Computers, Second. ed., Addison-Wesley, Received: August 10, 2006