A chain rule for multivariate divided differences

Transcription

1 A chain rule for multivariate divided differences Michael S. Floater Abstract In this paper we derive a formula for divided differences of composite functions of several variables with respect to rectangular grids of points. Letting the points coalesce yields a chain rule for partial derivatives of multivariate functions. Math Subject Classification: 41A05, 65D05, 26A06 Keywords: Divided difference, chain rule, calculus of several variables, Faa di Bruno formula. 1 Introduction The product rule for divided differences generalizes the Leibniz rule of calculus. It goes back at least to Popoviciu [16, 17] and Steffensen [22], and has played a role in the development of spline theory [1]. More recently, motivated by the error analysis of parametric curve interpolation, a chain rule was derived for divided differences: a formula that expresses divided differences of a composite function g f in terms of divided differences of the two functions f and g separately. Somewhat curiously, there are (at least) two such formulas, both of which were found in [8]. In one formula, all divided differences of the inner function f are over consecutive points, while in the other, all divided differences of the outer function g are over consecutive points, which suggests referring to the formulas as the inner and outer chain rules respectively. The outer chain rule was found independently in [23]. Centre of Mathematics for Applications, Department of Informatics, University of Oslo, PO Box 1053, Blindern, 0316 Oslo, Norway, michaelf@ifi.uio.no 1

2 Divided differences of composite functions appear in error formulas for parametric curve fitting based on polynomials and splines [15, 6, 7]. When compared with derivative chain rules such as Faa di Bruno s formula [5], an advantage of the divided difference chain rules is that they can be applied to functions with little or no smoothness. Moreover, by letting the points in these difference formulas coalesce, they provide a new, and simple way of deriving the Faa di Bruno formula [5, 11, 18, 19, 12, 13, 20, 11]. The inner chain rule was later used in [9] to derive a rule for divided differences of inverses of functions. Surprisingly, the inverse rule can be interpreted in terms of partitions of convex polygons, and offers a new way of counting such partitions. Given these various applications it seems worthwhile looking for a divided difference chain rule for multivariate functions. In this paper we show that at least the inner chain rule has a natural extension to multivariate functions, when divided differences are defined over rectangular grids of points. The rule can be interpreted in terms of polygonal paths that pass through points in the grid and are monotonic with respect to each variable. Due to the fact that multivariate divided differences converge to normalized partial derivatives as the grid points coalesce, the rule yields a corresponding formula for partial derivatives of multivariate composite functions; cf. [4]. 2 The univariate formula We begin by recalling the inner chain rule of [8]. Let [x 0, x 1,...,x n ]f denote the usual divided difference of a real-valued function f at the real values x 0,..., x n ; see [3]. We define [x i ]f = f(x i ), and for distinct x i, [x 0,...,x n ]f = ([x 1,..., x n ]f [x 0,...,x n 1 ]f)/(x n x 0 ). We can allow any of the x i to be equal if f has sufficiently many derivatives to allow it. In particular, if all the x i are equal to x say, then [x 0, x 1,...,x n ]f = f (n) (x)/n!. Sometimes, in order to simplify expressions, we will use the shorter notation [x; i, j]f := [x i, x i+1,...,x j ]f, j i. (1) It was shown in [8] that for n 1 and f and g smooth enough, [x; 0, n](g f) = n [f i0, f i1,...,f ik ]g k=1 0=i 0 < <i k =n k [x; i j 1, i j ]f, (2) j=1 2

3 where f i := f(x i ). The formula is a sum over all sequences of integers (i 0, i 1,...,i k ) satisfying the condition 0 = i 0 < i 1 < < i k = n, and the product term is formed by filling the gaps between x ij 1 and x ij. For example, the case n = 3 is [x; 0, 3](g f) = [f 0, f 3 ]g [x; 0, 3]f + [f 0, f 1, f 3 ]g [x; 0, 1]f [x; 1, 3]f + [f 0, f 2, f 3 ]g [x; 0, 2]f [x; 2, 3]f + [f 0, f 1, f 2, f 3 ]g [x; 0, 1]f [x; 1, 2]f [x; 2, 3]f. (3) 3 Chain rule: inner function multivariate We begin the generalization of (2) by considering the case that the inner function f is replaced by a multivariate function while the outer function g remains univariate. Following the approach of ([10], sec. 6.6), if f is a function of p variables, f = f(x 1,...,x p ), we denote by [x 1 0, x 1 1,...,x 1 n 1; x2 0, x 2 1,...,x 2 n 2;...;xp 0, x p 1,...,x p n p]f the divided difference of f with respect to the real values x r k, r = 1,...,p, 0 k n r. This is the tensor-product kind of divided difference, in the sense that it is the leading coefficient of the unique tensor-product polynomial that interpolates f over the rectangular grid of points (x 1 k 1,...,xp k p), 0 kr n r. If these points are not distinct, the interpolating polynomial is understood to be the Hermite one. The difficult but interesting issues of multivariate polynomial interpolation over other configurations of points and associated notions of divided differences have been addressed by several authors [1, 2, 14, 21]. In analogy with the univariate case, if x r k r xr for all r, then [x 1 0, x1 1,...,x1 n 1;...; xp 0, x p 1,...,x p n p]f Dn f(x), n! where x := (x 1, x 2,...,x p ), n = (n 1, n 2,...,n p ), n! := n 1!n 2! n p!, and ( ) n 1 ( ) n 2 ( ) n p D n f(x) := f(x). x 1 x 2 x p 3

4 Using the vector notation i = (i 1,...,i p ) and j = (j 1,...,j p ), we can express the condition that i r j r for all r = 1,...,p as i j, in which case, in analogy with (1), we will sometimes make use of the shorthand [x;i,j]f := [x 1 i 1, x1 i 1 +1,...,x1 j 1;...;xr i r, xr i r +1,...,xr j r;...; xp i p, xp i p +1,...,xp j p]f. As in [8] we will need the product rule. In the multivariate case this is [x;0,n](fg) = [x;0,i]f [x;i,n]g, (4) 0 i n which is a straightforward generalization of the univariate product rule found by Popoviciu [16, 17] and Steffensen [22]. The formula can be viewed as a sum over polygonal paths that connect the point 0 to the point n in the grid of points i, 0 i n. Each path has two line segments, one connecting 0 to i, the other connecting i to n, either of which can degenerate to a point. Figure 1(a) illustrates an example of a path in the bivariate case, p = 2. The two shaded boxes indicate the points involved in the two divided differences in the sum in (4), the bottom left box applies to f, while the top right box applies to g. The rule extends in a simple way to deal with a product of r functions f 1, f 2,...,f r, by induction on r, ( r ) r [x;0,n] f j = [x;i j 1,i j ]f j. (5) j=1 0=i 0 i 1 i r=n j=1 Like (4), this formula is a sum over non-decreasing polygonal paths: polygonal paths that connect the point 0 to the point n in such a way that the vertices form a non-decreasing sequence in each of the p coordinates. The paths now have r line segments (possibly degenerate). Figure 1(b) illustrates the bivariate case when there are four functions, r = 4. We now derive a chain rule for divided differences when the inner function is multivariate. With the vector notation, the formula looks almost like the univariate one. Theorem 1 If f : R p R and g : R R are smooth enough, and f i := f(x i ), then [x;0,n](g f) = n k=0 0=i 0 < <i k =n [f i0,...,f ik ]g k [x;i j 1,i j ]f. (6) j=1 4

5 n n i 3 i i i 1 Figure 1: The bivariate product rule: (a) two functions, (b) four functions. Here, i 1 < i 2 means i 1 i 2 and i 1 i 2, and n := n 1 + n n p. Note that we allow n = 0 in this formula, which is why we include the case k = 0 in the first sum, unlike in (2). If n = 0 we interpret the condition 0 = i 0 < i 1 < < i k = n (7) as having the unique solution k = 0 and i 0 = 0 and with the convention that an empty product has the value 1, formula (6) reduces to the correct equation (g f)(x 0 ) = g(f(x 0 )). If on the other hand n 1, all solutions to (7) require k 1. So for n 1 we could replace the first sum in (6) by n k=1. However, we will need to allow the case n = 0 when we use (6) to prove Theorem 2. Proof. It is sufficient to prove (6) when g is a polynomial of degree n because if we replace g by the polynomial of degree n that interpolates it on the points f i, 0 i n, the divided differences in (6) are unchanged. Therefore, by the linearity of (6) in g, it is also sufficient to prove it for any monomial g(y) = y r, with r n. In this case, by the divided difference rule for the product of r functions, [x;0,n](g f) = [x;0,n](f r ) = 0=α 0 α r=n q=1 Now, by counting the multiplicities of the points α j we have (α 0,..., α r ) = (i 0,...,i }{{} 0,...,i k,...,i }{{} k 1+µ 0 r [x; α q 1, α q ]f. (8) 1+µ k ) 5

6 Figure 2: Chain rule with f bivariate and n = (1, 1). for some k with 0 k r and 0 = i 0 < i 1 < < i k = n and µ 0 + +µ k = r k, with µ j 0. The product on the right of (8) is then r [x; α q 1, α q ]f = f µ 0 (x i0 )f µ 1 (x i1 ) f µ k (x ik ) q=1 and we obtain k [x;i j 1,i j ]f, j=1 [x;0,n](g f) = r k=0 0=i 0 < <i k =n µ 0 + +µ k =r k f µ 0 i 0 f µ k i k k [x;i j 1,i j ]f. (9) j=1 Now, observe that µ 0 + +µ k =r k f µ 0 i 0 f µ k i k = [f i0,...,f ik ]g, the divided difference of the monomial g(y) = y r at the points f i0,...,f ik. We substitute this identity into (9), and, then, since [f i0,...,f ik ]g = 0 if k > r, we can replace the upper limit r in the first sum in (9) by n. Similar to the product rule (5), formula (6) can be thought of as a sum over non-decreasing polygonal paths of vertices that connect 0 to n in the grid of points i, 0 i n. The number of line segments in the paths now varies, and is given by k. As long as n 0, the shortest path has one line segment (k = 1), and the longest paths have k = n line segments. Unlike the product rule, none of the line segments are degenerate, due to the strict inequality i j 1 < i j. Figure 1(b) can be used to illustrate the structure of the chain rule: it shows a path in (6) in the case k = 4. The shaded boxes indicate the divided differences of f in the sum in (6). 6

7 Figure 3: Chain rule with f bivariate and n = (2, 1). We write out a couple of examples of (6). In the bivariate case p = 2 with n = (1, 1) the formula has three terms. There is one term with k = 1, corresponding to the path (i 0,i 1 ) = (00, 11) (the first row of Figure 2) and two terms with k = 2 corresponding to the paths (i 0,i 1,i 2 ) = (00, 10, 11) and (i 0,i 1,i 2 ) = (00, 01, 11) (the second row of Figure 2); [x; 00, 11](g f) = [f 00, f 11 ]g [x; 00, 11]f + [f 00, f 10, f 11 ]g [x; 00, 10]f [x; 10, 11]f + [f 00, f 01, f 11 ]g [x; 00, 01]f [x; 01, 11]f. (10) In the case n = (2, 1), the formula is a sum over eight terms, one for each of the paths shown in Figure 3. The three rows in the figure show respectively: the unique path with k = 1 segment, the four paths with k = 2 segments, and the three paths with k = 3 segments, and the formula is [x; 00, 21](g f) = [f 00, f 21 ]g [x; 00, 21]f + [f 00, f 10, f 21 ]g [x; 00, 10]f [x; 10, 21]f + [f 00, f 20, f 21 ]g [x; 00, 20]f [x; 20, 21]f + [f 00, f 01, f 21 ]g [x; 00, 01]f [x; 01, 21]f + [f 00, f 11, f 21 ]g [x; 00, 11]f [x; 11, 21]f + [f 00, f 10, f 20, f 21 ]g [x; 00, 10]f [x; 10, 20]f [x; 20, 21]f + [f 00, f 10, f 11, f 21 ]g [x; 00, 10]f [x; 10, 11]f [x; 11, 21]f + [f 00, f 01, f 11, f 21 ]g [x; 00, 01]f [x; 01, 11]f [x; 11, 21]f. (11) 7

8 4 Chain rule: both functions multivariate Now we treat the full multivariate case, building on Theorem 1. We consider a composition g f in which f is a multivariate, vector-valued function f : R p R q and g is a multivariate scalar-valued function g : R q R. Theorem 2 If f : R p R q and g : R q R are smooth enough and f = (f 1,...,f q ) then, defining f r,i := f r (x i ), [x;0,n](g f) = n k=0 0=i 0 < <i k =n 0=j 0 j q=k [f 1,ij0,...,f 1,ij1 ;...; f q,ijq 1,...,f q,ijq ]g q j r r=1 j=j r 1 +1 [x;i j 1,i j ]f r. (12) Proof. Similar to the proof of Theorem 1, it is sufficient to prove the formula when g is a polynomial of the form g(y 1,...,y q ) = y r 1 1 y rq q, where 0 r j n j. Therefore, it is also sufficient to prove the formula in the more general case that g is the product of q univariate functions, g i : R R, i = 1,...,q, g(y 1,...,y q ) = g 1 (y 1 ) g q (y q ). (13) To show this, observe that with g in this form, the composition g f is given by g(f(x)) = g 1 (f 1 (x)) g q (f q (x)). (14) Then, due to the product rule (5), we have q [x;0,n](g f) = [x; α r 1, α r ](g r f r ). 0=α 0 α 1 α q=n r=1 Next, since g r is univariate, we can expand the divided differences on the right using the chain rule of Theorem 1. Note that these divided differences may have order 0, due to the possibility that α r 1 = α r, and this is why we allowed n = 0 (and consequently k = 0) in formula (6). With the shorthand A r (i 0,...,i k ) := [f r,i0,...,f r,ik ]g r 8 k [x;i j 1,i j ]f r, (15) j=1

9 for points i 0,...,i k R p and r = 1,...,q, we find [x;0,n](g f) = q 0=α 0 α 1 α q=n r=1 α r α r 1 k r=0 α r 1 =i r 0 < <ir kr =αr A r (i r 0,...,ir k ). Next, observe that since the sequence i r 0 < < ir k r in this expression connects α r 1 to α r, when we put all q of these sequences together we obtain a single increasing sequence that connects 0 to n. This means that we can rearrange the formula to be a single sum over paths connecting 0 to n, and over all possible choices of q sub-paths, with each sub-path contributing to one of the functions g r : [x;0,n](g f) = n k=0 0=i 0 < <i k =n 0=j 0 j q=k q A r (i jr 1,...,i jr ). To complete the proof, we expand the term A r using (15), and hence we obtain equation (12) for g of the form (13) because with this choice of g, r=1 q [f r,ijr 1,...,f r,ijr ]g r = [f 1,ij0,...,f 1,ij1 ;...;f q,ijq 1,...,f q,ijq ]g. r=1 We look at four examples. First, if p = 1 and q = 2, and setting a = f 1 and b = f 2, the composition has the form g(f(x)) = g(a(x), b(x)), and the formula with n = n = 1 is [x; 0, 1](g f) = [a 0, a 1 ; b 1 ]g [x; 0, 1]a + [a 0 ; b 0, b 1 ]g [x; 0, 1]b, (16) and with n = n = 2, [x; 0, 2](g f) = [a 0, a 2 ; b 2 ]g [x; 0, 2]a + [a 0 ; b 0, b 2 ]g [x; 0, 2]b + [a 0, a 1, a 2 ; b 2 ]g [x; 0, 1]a [x; 1, 2]a + [a 0, a 1 ; b 1, b 2 ]g [x; 0, 1]a [x; 1, 2]b + [a 0 ; b 0, b 1, b 2 ]g [x; 0, 1]b [x; 1, 2]b. (17) 9

10 If p = 2 and q = 2, with a = f 1 and b = f 2, the composition has the form g(f(x)) = g(a(x), b(x)), with x R 2. Specializing to n = (1, 1) we then obtain a sum over the three paths of Figure 2. But each path is now split into q = 2 parts, and there are k + 1 ways of splitting a path with k segments. Thus the total number of terms is = 8: [x; 00, 11](g f) = [a 00, a 11 ; b 11 ]g [x; 00, 11]a + [a 00 ; b 00, b 11 ]g [x; 00, 11]b + [a 00, a 10, a 11 ; b 11 ]g [x; 00, 10]a [x; 10, 11]a + [a 00, a 10 ; b 10, b 11 ]g [x; 00, 10]a [x; 10, 11]b + [a 00 ; b 00, b 10, b 11 ]g [x; 00, 10]b [x; 10, 11]b + [a 00, a 01, a 11 ; b 11 ]g [x; 00, 01]a [x; 01, 11]a + [a 00, a 01 ; b 01, b 11 ]g [x; 00, 01]a [x; 01, 11]b + [a 00 ; b 00, b 01, b 11 ]g [x; 00, 01]b [x; 01, 11]b. (18) If instead, n = (2, 1), the formula is a sum over the eight paths of Figure 3. We do not write it out but the number of terms is = Chain rule for derivatives Letting the points x k in equation (12) converge to x immediately gives a multivariate chain rule for derivatives. The points i r in the second sum in (12) are no longer specifically needed in the derivative version; the order of the partial derivative of f r depends only on the vector difference α r := i r i r 1. Thus, summing over the α r instead of the i r, the derivative formula is D n (g f)(x) n! = n k=0 α 1 + +α k =n 0=j 0 j q=k D j g(f(x)) j! q j r r=1 j=j r 1 +1 D α j f r (x), (19) α j! where the second sum is over α r > 0, and j := (j 1 j 0,...,j q j q 1 ). This equation contains, in general, repeated terms, unlike Faa di Bruno s formula and the multivariate formula in [4]. It could be reformulated to avoid repeats by summing over partitions of vector integers using the same approach as that of the conversion of equation (25) of [8] to equation (26) of [8]. Instead we simply gives some examples, taking, for comparison, the same cases that we looked at earlier for divided differences. Specializing (19) 10

11 to the cases treated in (3), (10 11), (16 17), and (18), one obtains, after cancelling factorials and drawing together repeated terms, their respective derivative counterparts: (g f) = g f + 3g f f + g (f ) 3, D 11 (g f) = g D 11 f + g D 10 fd 01 f, D 21 (g f) = g D 21 f + g (D 20 fd 01 f + 2D 10 fd 11 f) + g (D 10 f) 2 D 01 f, (g f) = D 10 g a + D 01 g b, (g f) = D 10 g a + D 01 g b + D 20 g (a ) 2 + 2D 11 g a b + D 02 g (b ) 2, and D 11 (g f) = D 10 g D 11 a + D 01 g D 11 b + D 20 g D 10 a D 01 a + D 11 g D 10 a D 01 b + D 11 g D 01 a D 10 b + D 02 g D 01 bd 10 b. References [1] C. de Boor, A Leibniz formula for multivariate divided differences, SIAM J. Numer. Anal. 41 (2003), [2] C. de Boor, A multivariate divided difference, in Approximation Theory VIII, Vol 1: Approximation and Interpolation, C. K. Chui and L. L. Schumaker (eds.), World Scientific, Singapore, 1995, pp [3] C. de Boor, Divided differences, Surveys in Approximation Theory 1 (2005), [4] G. M. Constantine and T. H. Savits, A multivariate Faa di Bruno formula with applications. Trans. Amer. Math. Soc. 348 (1996), [5] C. F. Faa di Bruno, Note sur une nouvelle formule de calcul differentiel, Quarterly J. Pure Appl. Math, 1 (1857), [6] M. S. Floater, Arc length estimation and the convergence of parametric polynomial interpolation, BIT 45 (2005),

12 [7] M. S. Floater, Chordal cubic spline interpolation is fourth order accurate, IMA J. Numer. Anal. 26 (2006), [8] M. S. Floater and T. Lyche, Two chain rules for divided differences and Faa di Bruno s formula, Math. Comp. 76 (2007), [9] M. S. Floater and T. Lyche, Divided differences of inverse functions and partitions of a convex polygon, Math. Comp. 77 (2008), [10] E. Isaacson and H. B. Keller, Analysis of numerical methods, Wiley, [11] W. P. Johnson, The curious history of Faa di Bruno s formula, Amer. Math. Monthly 109 (2002), [12] C. Jordan, Calculus of finite differences, Chelsea, New York, [13] D. Knuth, The art of computer programming, Vol I, Addison Wesley, [14] C. A. Micchelli, A constructive approach to Kergin interpolation in R k : multivariate B-splines and Lagrange interpolation, Rocky Mountain J. Math. 10 (1979), [15] K. Mørken and K. Scherer, A general framework for high-accuracy parametric interpolation, Math. Comp. 66 (1997), [16] T. Popoviciu, Sur quelques propriétés des fonctions d une ou de deux variables reélles, dissertation, presented at the Faculté des Sciences de Paris, published by Institutul de Arte Grafice Ardealul (Cluj, Romania), [17] T. Popoviciu, Introduction à la théorie des différences divisées, Bull. Math. Soc. Roumaine Sciences 42 (1940), [18] J. Riordan, Derivatives of composite functions, Bull. Amer. Math. Soc. 52 (1946), [19] J. Riordan, An introduction to combinatorial analysis, John Wiley, New York,

13 [20] S. Roman, The formula of Faa di Bruno, Amer. Math. Monthly 87 (1980), [21] T. Sauer, Polynomial interpolation in several variables: lattices, differences, and ideals, in Topics in multivariate approximation and interpolation, K. Jetter et al. (eds.), Elsevier, 2006, pp [22] J. F. Steffensen, Note on divided differences, Danske Vid. Selsk. Math.- Fys. Medd 17 (1939), [23] X. Wang and H. Wang, On the divided difference form of Faà di Bruno s formula, J. Comp. Math. 24 (2006),