Tricks of the Trade. Edited by Paul Abbott. Interpolation with Noise Vittorio G. Caffa. The Mathematica Journal

Transcription

1 The Mathematica Journal Tricks of the Trade Edited by Paul Abbott This is a column of programming tricks and techniques, most of which, we hope, will be contributed by our readers, either directly as submissions to The Mathematica Journal or as an edited answer to a question posted in the Mathematica newsgroup, comp.soft-sys.math.mathematica. Interpolation with Noise Vittorio G. Caffa caffa@iabg.de In Interpolation, TMJ, 9(), 004, pp. 06 0, a method was presented for computing derivatives of a sampled function. This method only works if the data is unaffected by measuring error. If models of the data source and the measurement noise are available, you can use (for example) Kalman filtering techniques for computing an optimal estimation (see mathworld.wolfram.com/ KalmanFilter.html). Alternatively, you can achieve good results using a simple filtering technique. Consider again the exact function yhxl = x sinhxl - x ê. In[]:= yhx_l = x sinhxl - x ê ; Take x from a uniform random distribution over the interval Ha, bl = H, L, and append the endpoints. In[]:= 8a, b< = 8, <; SeedRandom@4D; xs = Join@8a<, Table@Random@Real, 8a, b<d, 8<D, 8b<D; Now sample yhxl over the interval adding Gaussian noise corresponding to measurement error. In[]:= Needs@"Statistics`ContinuousDistributions`"D; f Hx_L := 8x, yhxl + Random@NormalDistribution@0, 0.0DD<; data = f êû xs; Superimpose the sampled data over a plot of its interpolated function to visualie the 8x, y< data. It is clear that the interpolation is not smooth, so computing derivatives will be problematic. In[6]:= y int = Interpolation@dataD;

2 Tricks of the Trade 68 In[7]:= y int HxL, 8x, a, b<, Epilog Æ AbsolutePointSie@D, Point êû data<d A simple filtering technique involves computing a truncated Fourier series of the function yhxl, that is, n y n HxL = a 0 + Ha k coshk w xl + b k sinhk w xll. k= Requiring the wavelength l to satisfy l = p ê w > Hb - al avoids wrap-around effects. As only the frequencies up to n w are considered, the method acts like a low-pass filter. The value of the parameter n should be tuned to obtain the best results. This approach should be compared with that used in Spectral Analysis of Irregularly Sampled Data, TMJ, 7(), 997, pp Here is the Fourier basis set for arbitrary n and l. In[8]:= basis@n_, l_d := WithA9w = ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ p, k = Range@nD=, Join@8<, coshk w xl, sinhk w xlde N@lD The best truncated Fourier expansion, y n, for l =0 > Hb - al = and n = is computed using Fit. In[9]:= y = Function@x, Evaluate@Fit@data, basis@, 0D, xddd Out[9]= Function@x, cosh xl cosh xl cosh0.689 xl cosh xl -.8 cosh.047 xl sinh xl sinh xl -.0 sinh0.689 xl sinh xl +.6 sinh.047 xl D Here is the sampled data superimposed over a plot of y.

3 686 Edited by Paul Abbott In[0]:= y HxL, 8x, a, b<, Epilog Æ AbsolutePointSie@D, Point êû data<d Compare the first derivatives; the red curve denotes the exact value ( ), the green curve denotes the interpolated function ( ), and the blue curve denotes the truncated Fourier fit y ( ). The agreement between the exact function and the truncated Fourier fit is excellent. In[]:= PlotA8 y HxL, y int HxL, y HxL<, 8x, a, b<, PlotStyle Æ 9Hue@0D, HueA ÄÄÄÄÄ E, HueA ÄÄÄÄÄÄ E=E Using Reduce To Solve The Kuhn Tucker Equations Frank J. Kampas fkampas@msn.com The functions Maximie and Minimie give exact solutions to simple nonlinear optimiation problems with equality and inequality constraints. Another approach is to set up Lagrange multipliers for equality constraints and the Kuhn Tucker equations for inequality constraints (see mathworld.wolfram.com/ Kuhn-TuckerTheorem.html) and solve using Reduce. In some cases, this approach can solve problems which cannot directly be solved with Maximie and Minimie. The following function demonstrates this approach.

4 Tricks of the Trade 687 In[]:= KuhnTuckerHob_, cons_list, vars_, domain L := ModuleA8consconvrule = 8x_ y_ Æ y - x 0, x_ > y_ Æ y - x < 0, x_ y_ Æ x - y 0, x_ y_ Æ x - y 0<, stdcons, eqcons, ineqcons, lambdas, mus, numequals, numinequals, lagrangian, eqs, eqs, eqs<, stdcons = cons ê. consconvrule; eqcons = Cases@stdcons, x_ 0 xd; ineqcons = Cases@stdcons, x_ 0 xd; numequals = Length@eqconsD; numinequals = Length@ineqconsD; lambdas = Array@l, numequalsd; mus = Array@m, numinequalsd; lagrangian = ob + lambdas.eqcons + mus.ineqcons; eqs = i lagrangian ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ 0& y êû vars; eqs = Thread@mus 0D; k # { eqs = Table@musPiT ineqconspit 0, 8i, numinequals<d; Reduce@ Join@eqs, eqs, eqs, consd, Join@vars, lambdas, musd, domain, Backsubstitution Æ TrueDE Here is an example with an equality constraint in which both the maximum and minimum are obtained. Defining the obective function with a constraint and listing it first in the list of variables places the obective function first in the results. In[]:= Out[]= KuhnTucker@obf, 8obf ä x + y +, x + y + ä <, 8obf, x, y, <D i obf - Ì x - k Ì y - Ì - Ì y lhl - Ì lhl Î i obf Ì x { k Ì y y Ì Ì lhl - Ì lhl - Equivalent results are obtained by using Maximie and Minimie. In[]:= Out[]= In[4]:= Out[4]= Maximie@obf, 8obf ä x + y +, x + y + ä <, 8obf, x, y, <D êê Simplify 9,9obf Ø, x Ø, y Ø, Ø == Minimie@obf, 8obf ä x + y +, x + y + ä <, 8obf, x, y, <D êê Simplify 9-,9obf Ø-,x Ø-, y Ø-, Ø- == KuhnTucker can solve the problem when the righthand side of the constraint is changed from a number to a symbol. {

5 688 Edited by Paul Abbott In[]:= Out[]= 8obf ä x + y +, x + y + ä c<, 8obf, x, y, <D i obf - è!! è!! è!! c c c Ì x - k Ì y - Ì è!! c y - Ì lhl - Ì c 0 Ì lhl ÅÅÅÅ è!! Î c { i obf è!! è!! è!! è!! c c c c Ì x k Ì y Ì Ì y lhl - Ì c 0 Ì lhl - ÅÅÅÅ è!! c { Maximie and Minimie give an error message in this situation. In[6]:= Maximie@obf, 8obf ä x + y +, x + y + ä c<, 8obf, x, y, <D Maximie::consv : The constraint x + y + c contains a nonconstant expression c independent of variables 8obf, x, y, <. More Out[6]= Maximie@obf, 8obf x + y +, x + y + c<, 8obf, x, y, <D In the following example, the constraint x b is only active if b is less than. Therefore, the solution obtained covers b and b separately. In[7]:= KuhnTucker@obf, 8obf ä x - x, x b<, 8obf, x<d Out[7]= Hb Ï obf b - b Ï x b Ï lhl - Ï mhl - Hb - LL Í Hb Ï obf - Ï x Ï lhl - Ï mhl 0L When the solution is degenerate, all the results can be obtained. In[8]:= KuhnTucker@obf, 8obf ä x + y, x + y ä <, 8obf, x, y<d Out[8]= obf Ï y - x Ï lhl - Ï lhl - The following problem can be solved quite easily if the obective function is not defined as a constraint. Adding the extra constraint greatly increases the solution time. The problem cannot be solved by Minimie or NMinimie unless a change of variables is used to eliminate the square roots. In[9]:= KuhnTuckerA ÄÄÄÄÄÄ Jt h - "###### q h + t l - "##### q l N, 8t l 0 q l, t h q h, t l - 0 q l t h - 0 q h, t h - q h t l - q l <, 8q l, q h, t l, t h <E Out[9]= q l ÅÅÅ 776 Ì q h ÅÅÅÅÅÅÅÅ 6 Ì t l ÅÅÅ 444 Ì t h 8 ÅÅÅ 444 Ì mhl Ì mhl 0 Ì mhl 0 Ì mh4l ÅÅÅÅÅ Using Reduce with domain specified eliminates complex solution candidates.

6 Tricks of the Trade 689 In[0]:= 8obf == x + y, x + y ä <, 8obf, x, y<, RealsD êê FullSimplify Out[0]= obf ê Ì x Ì y Ì lhl - Ì lhl - ÅÅÅÅÅ Note, however, that using Lagrange multipliers and the Kuhn Tucker equations relies on assumptions that the code does not check. For instance, the solution set of equational constraints should be a manifold of dimension v - e, where v is the number of variables and e is the number of equations. If this is not satisfied, KuhnTucker may not find the extremum. In[]:= Out[]= KuhnTucker@x + y, 8 ä x - xy, ä y + xy, ä x + y <, 8x, y, <D False Maximie and Minimie work in this situation. In[]:= Minimie@x + y, 8 ä x - xy, ä y + xy, ä x + y <, 8x, y, <D Out[]= 80, 8x Ø 0, y Ø 0, Ø 0<< In[]:= Maximie@x + y, 8 ä x - xy, ä y + xy, ä x + y <, 8x, y, <D Out[]= 80, 8x Ø 0, y Ø 0, Ø 0<< Legendre Gauss Quadrature Gaussian quadrature gives the best estimate of an integral by picking optimal abscissas, x i, at which to evaluate the function f HxL. It is optimal because the n-point formula is exact for polynomials of degree n -. Legendre Gauss quadrature (see mathworld.wolfram.com/legendre-gaussquadrature.html ) is a Gaussian quadrature over the D with weighting function W HxL =, that is, - f HxL x º w i f Hx i L, where the abscissas, x i, for quadrature order n, satisfy P n Hx i L 0, and the weights are H - x i L w i ÅÅÅÅÅÅÅÅÅ Hn + L P n+ Hx i L. Define the abscissas as the roots of P n HxL using Root and save them as they are computed. This is an exact expression for the roots. In[]:= n i= x ê: x i_,n_ := x ê: x i,n = Root@Function@x, P n HxLD, id Define the weights. RootReduce expresses the weights as a single Root obect. Save the weights as they are computed.

7 690 Edited by Paul Abbott H - x i,n L In[]:= w ê: w i_,n_ := w ê: w i,n = ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ ÄÄÄÄÄÄÄÄÄÄÄÄ Hn + L P n+ Hx i,n L Here are the x i and w i for n =. In[]:= Out[]= With@8n = <, Table@8x i,n, w i,n <, 8i, n<dd i - "##### ÅÅÅÅ 0 "##### k ÅÅÅÅ ÅÅÅÅ 9 8 ÅÅÅÅ 9 ÅÅÅÅ 9 y { Here is x and w for n =, expressed as radicals. In[4]:= With@8n = <, 8x,n, w,n <êêtoradicalsd Out[4]= 9- ÅÅÅÅÅ $%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%!!! ÅÅÅÅÅ I - 70 M, 7!!! ÅÅÅÅÅÅÅÅÅÅÅ I + 70 M= 900 êê RootReduce Abscissas and weights for higher n are better left as Root obects (see Root versus Radicals, TMJ, 9(), 00, pp. 7). Note that, for fixed n, each w i can be expressed as a particular root of a polynomial of degree d ÅÅÅÅ n t. Here are the weights for n = 6. In[]:= With@8n = 6<, Table@w i,n, 8i, n<dd Out[]= 8Root@0000 # # + 69 # &, D, Root@0000 # # + 69 # &, D, Root@0000 # # + 69 # &, D, Root@0000 # # + 69 # &, D, Root@0000 # # + 69 # &, D, Root@0000 # # + 69 # &, D< n The weights satisfy i= In[6]:= Out[6]= Tr@%D êêrootreduce w i. Here we check this for n = 6. Now consider computing the integral of a polynomial of degree m, denoted p m HxL. In[7]:= p m_ Hx_L := a 0 + a x ; m = Compute the integral of p 7 HxL over - x. In[8]:= - p 7 HxL x Out[8]= a 0 + a + a 4 + a 6 7

8 Tricks of the Trade 69 As expected, an identical answer is obtained using Legendre Gauss quadrature with n = 4 since n - = 7. n In[9]:= WithA8n = 4<, w i,n p 7 Hx i,n LE êê FullSimplify Out[9]= a 0 + a i= + a 4 + a 6 7 Pascal Matrices Writing Pascal s triangle as a lower triangular matrix generates one type of Pascal matrix (see mathworld.wolfram.com/pascalmatrix.html). Such matrices have many interesting properties (web.mit.edu/8.06/www/pascal-work.pdf). Define the lower triangular matrix L n by 8L n < i, = i i y, where i, = 0,,, k { n -. In[]:= Here is L. L n_ := TableA i i y, 8i, 0,n - <, 8, 0,n - <E k { In[]:= L Out[]= i y k { And here is its inverse. In[]:= L - Out[]= i y k { In general, the inverse of L n has 8L - n < i, = H-L i+ i i y. k { Clearly 8L p < i, = p i- i i y. k {

9 69 Edited by Paul Abbott In[4]:= pd Out[4]= i y p p p 0 0 p p p 0 k p 4 4 p 6 p 4 p { Define v n HxL = 8x i < i=0,,, n. In[]:= v n_ Hx_L := x Range@nD- Using the binomial theorem, we can show that Ln p v n HxL = v n Hp + xl, that is, i y i y i y p x p + x p p x = Hp + xl. p p p x Hp + xl k ª ª ª ª { k ª { k ª { Prove the binomial theorem. In[6]:= SimplifyAHp + xl i i i y x p i-, i Œ E k { i =0 Out[6]= True Verify that Ln p v n HxL = v n Hp + xl for n =. In[7]:= MatrixPower@L, pd.v HxL êêfactor Out[7]= 8, p + x, Hp + xl, Hp + xl, Hp + xl 4 < In[8]:= Out[8]= % v Hp + xl True Writing Ln p = A n p, it follows that A n = loghl n L = lim pø0 HLn p - I n Lê p. Here is A. In[9]:= Out[9]= A = lim pæ0 i y k { MatrixPower@L, pd - IdentityMatrix@D ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ p In general, it can be shown that 8A n < i, = i d i, +. Check that L = A.

10 Tricks of the Trade 69 In[0]:= D Out[0]= i y k { Define the upper triangular matrix n by alternating the sign of the rows in L n T. In[]:= Here is. n_ := H-L Range@nD- L n T In[]:= Out[]= i y k { n is its own inverse (it is involutory). In[]:=. Out[]= i y k { Multiplying L n by its transpose yields a symmetric, positive-definite matrix, denoted S n, now with Pascal s triangle entries along each skew diagonal (see mathworld.wolfram.com/skewdiagonal.html). In[4]:= S n_ := L n.l n T In[]:= S 4 Out[]= i y k 4 0 0{ T Since S n = L n. L n and 8L T n < k, = 8L n <,k, we obtain that i 8S n < i, = 8L n < k=0 i,k 8L n <,k i i + y i i + y. k i { k {

11 694 Edited by Paul Abbott In[6]:= Out[6]= In[7]:= Out[7]= i k=0 i i y i y k k { k k { GHi + + L ÅÅÅÅÅÅÅ GHi + L GH + L FullSimplifyA% i i + y i i + y E k i { k { True The inverse of S n has integer entries. In[8]:= S - Out[8]= i y k { It can be shown that S n - n -. S n. n. That is S n - is similar to S n (see mathworld.wolfram.com/similarmatrices.html). Verify that S n - n -. S n. n for n =. In[9]:= S - -. S. Out[9]= True Since similar matrices have the same eigenvalues, the eigenvalues of S n must come in reciprocal pairs, l and l -. Moreover, if n is odd, then the middle eigenvalue must be unity. Here are the eigenvalues of S. In[0]:= Out[0]= Eigenvalues@S D 94 +!!!,, 4 -!!! = Confirm that the eigenvalues come in reciprocal pairs. In[]:= ê % Out[]= 9 In[]:= Out[]= 4 +!!!,, 4 -!!! = RootReduce@%D 94 -!!!,, 4 +!!! =

12 Tricks of the Trade 69 Generalied Padé Approximation It is often the case that, while an exact solution is difficult or impossible, series or asymptotic solutions to a particular problem can be obtained without too much difficulty. As is well known, Padé approximations (see mathworld.wolfram.com/ PadeApproximant.html) are usually superior to Taylor expansions when functions contain poles, because the use of rational functions allows them to be well represented. Define P N HxL and Q M HxL as polynomials of degrees N and M, with Q M H0L =. In[]:= P n_ Hx_L := p i x i i=0 In[]:= Q m_ Hx_L := + i= n m qi x i The coefficients in MD Padé approximant to f HxL are determined by requiring that the Taylor expansion of f HxL - P N HxLêQ M HxL is of order OHx N +M+ L. Suppose that you have obtained the following Taylor series up to OHx L for a function of interest. In[]:= taylor@x_d = - ÄÄÄÄÄÄÄÄÄÄ x + ÄÄÄÄÄÄÄÄÄÄÄÄÄ 4 x - ÄÄÄÄÄÄÄÄÄÄÄÄ 8 x x4 ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ 94 ; Here is D Padé approximant for this series. In[4]:= << Calculus` In[]:= pade@x_d = Pade@taylor@xD, 8x, 0,, <D Out[]= x - ÅÅÅÅÅÅ x ÅÅÅÅ 4 x ÅÅÅÅÅÅÅÅÅ + ÅÅÅÅÅÅÅ 4 x Verify that the Taylor expansion of f HxL - P HxLêQ HxL is of order OHx L. In[6]:= taylor@xd - pade@xd + O@xD Out[6]= OHx L A plot shows that the Taylor series ( ) diverges from the Padé approximant ( ) near x =.

13 696 Edited by Paul Abbott In[7]:= small = PlotA8taylor@xD, pade@xd<, 8x, 0,<, PlotStyle Æ 9Hue@D, HueA ÄÄÄÄÄ E=E Suppose now that you have also computed the following asymptotic series for the same function. In[8]:= asymptotic@x_d = ÄÄÄÄÄÄÄÄÄÄ x + ÄÄÄÄÄÄÄÄÄÄÄÄÄ 4 x + ÄÄÄÄÄÄÄÄÄÄÄÄ 8 x ; A plot shows that the Padé approximant about x = 0 ( ) and the asymptotic expansion ( ) intersect for x º.. In[9]:= << Graphics` In[0]:= large = DisplayTogetherA PlotAasymptotic@xD, 8x,,0<, PlotStyle Æ HueA ÄÄÄÄÄÄ EE, smalle

14 Tricks of the Trade 697 How can you incorporate information from expansions of a function about two or more points? One approach is to generalie the idea of computing Padé approximants []. Suppose f HxL has the asymptotic expansions f HxL ~ a i Hx - x 0 L i, x Ø x 0, i=0 f HxL ~ b i Hx - x L i, x Ø x, i=0 in the neighborhoods of distinct points x 0 and x, respectively. To compute a two-point Padé approximant to f HxL, we require that the rational function P N HxLêQ M HxL agrees with f HxL up to order J about x 0 and to order K about x, where J + K = N + M +. To use all the information at hand, note that for x 0 = 0, J =, and for x =, K = 4. In[]:= = Exponent@taylor@xD, xd + Out[]= In[]:= k = Exponent@asymptotic@xD, ê xd + Out[]= 4 The diagonal Padé approximant then has N = M = 4. In[]:= n = m = ÄÄÄÄÄ H + k - L Out[]= 4 It is straightforward to determine the coefficients p i and q i in terms of a i and b i. Here we obtain these coefficients using series arithmetic via Solve. In[4]:= SolveA 9taylor@xD - P n HxL ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ Q m HxL + O@xD ä 0, asymptotic@xd - P n HxL ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ Q m HxL + O@x, Dk ä 0=, Join@Table@p i, 8i, 0,n<D, Table@q i, 8i, m<dde Out[4]= 99p Ø ÅÅÅÅÅÅÅÅÅÅÅ 94, p Ø 8 94, p Ø ÅÅÅÅÅ 9, p 4 Ø 0, p 0 Ø, q Ø ÅÅÅÅÅÅÅÅÅ 8, q Ø ÅÅÅÅÅÅÅÅÅÅÅ, q Ø ÅÅÅÅÅ 8 9, q 4 Ø ÅÅÅÅÅÅÅÅÅÅÅ 6 94 == Here is the two-point Padé approximant to f HxL. In[]:= twopoint@x_d = P n HxL ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ ê. First@%D Q m HxL Out[]= 8 x ÅÅÅÅÅÅÅÅÅ x + ÅÅÅÅÅÅÅÅÅÅÅ + ÅÅÅÅÅÅÅ x ÅÅÅÅÅÅÅÅ 6 x 4 x + + ÅÅÅÅÅÅÅÅÅ 8 x + ÅÅÅÅÅÅÅ 8 x 94 ÅÅÅÅÅÅÅÅ ÅÅÅÅ ÅÅÅÅÅÅÅÅ + 9

15 698 Edited by Paul Abbott The two-point Padé approximant ( ) agrees with the Taylor series about x = 0 ( ), the one-point Padé approximant about x = 0 ( ), and the asymptotic expansion ( ) about x =. In[6]:= DisplayTogether@Plot@twopoint@xD, 8x, 0, 0<D, larged References [] C. M. Bender and S. A. Orsag, Advanced Mathematical Methods for Scientists and Engineers, New York: McGraw-Hill, 978, pp Paul Abbott School of Physics, M0 University of Western Australia Stirling Highway Crawley WA 6009, Australia tm@physics.uwa.edu.au physics.uwa.edu.au/~paul