Least-squares Solutions of Linear Differential Equations

Transcription

1 1 Least-squares Solutions of Linear Differential Equations Daniele Mortari dedicated to John Lee Junkins arxiv: v1 [mathca] 5 Feb 017 Abstract This stu shows how to obtain least-squares solutions to initial and boundary value problems to nonhomogeneous linear differential equations with nonconstant coefficients of any order However, without loss of generality, the approach has been applied to second order differential equations The proposed method has two steps The first step consists of writing a constrained expression, introduced in Ref [1], that has embedded the differential equation constraints These expressions are given in term of a new unknown function, gt), and they satisfy the constraints, no matter what gt) is The second step consists of expressing gt) as a linear combination of m independent known basis functions, gt) = ξ T ht) Specifically, Chebyshev orthogonal polynomials of the first kind are adopted for the basis functions This choice requires rewriting the differential equation and the constraints in term of a new independent variable, x [ 1, +1] The procedure leads to a set of linear equations in terms of the unknown coefficients vector, ξ, that is then computed by least-squares Numerical examples are provided to quantify the solutions accuracy for initial and boundary values problems as well as for a control-type problem, where the state is defined in one point and the costate in another point Acronyms used throughout this paper DE IVP BVP LS Differential Equation Initial Value Problem Boundary Value Problem Least-Squares I INTRODUCTION The n-th order nonhomogeneous ordinary linear Differential Equation DE) with nonconstant coefficients is the equation n f i t) di yt) dt i = ft), 1) i=0 where ft) and the n + 1 functions, f i t), can be any nonlinear continuous functions and t often the time) is the independent variable This kind of equations appear in many problems and in almost all scientific disciplines Equation 1) can be solved by the method of variation of parameters: using n linearly independent solutions, y 1 t),, y n t), of the homogenous part Then, the general solution is just a linear combination of the independent solutions plus the particular solution associated to the nonhomogeneous equation [] The variation of parameters method relies on the capability of finding the n linearly independent solutions Unfortunately, there is no general method to find these solutions Another method, called undetermined coefficients [], is restricted to the case of constant coefficient, only Finally, in the specific case if ft) and all the n + 1 functions, f i t), are polynomials, then an approximate solution can be found by power series [] However, in this stu, ft) and all the f i t) functions can be any nonlinear continuous functions that are nonsingular in the integration time range The proposed Least-Squares LS) method can be applied to solve Eq 1) for any value of n However, without loss of generality and for sake of brevity, the approach is here applied to second order nonhomogeneous linear DE with nonconstant coefficients, f t) d yt) dt + f 1 t) t) + f 0 t) yt) = ft) ) dt It is important to outline that, if functions f 1 t), f 0 t), and ft) are continuous and nonsingular within the integration range, the Initial Value Problems IVP) always admit solutions, while Boundary Value Problems BVP) may have a single, multiple, no, or infinite solutions Final special analysis is dedicated to particular BVP typical from optimal control) where the variable is a vector, {x T, λ T } T, and where the state vector is defined at initial time, xt 0 ) = x 0, and the costate vector at final time, λt f ) = λ f Professor, Aerospace Engineering, Texas A&M University, College Station, TX , USA mortari@tamuedu

2 II THE CONSTRAINED EXPRESSIONS The key idea of this stu is to search the solution of Eq 1) using constrained expressions, whose theory is presented in d di y Ref [1] These expressions have embedded all the DE constraints, = y di) t i, where the n-element vector, d, contains dt di t=ti the constraints derivatives orders and the n-element vector, t, indicates where the constraints are specified The constrained equations adopted in this stu are expressed as, yt) = gt) + n i=1 [ ] β i t, t) y di) t i g di) t i where: β d k) i t k, t) = δ ik 3) expressions that are linear functions in the unknown function gt) and in its derivatives, g di) t i, evaluated at constraints times and where δ ik is the Kronecker delta The β i t, t) are special functions of the time and constraints times defined by the vector t The β i functions given in Eq 3) are not unique but they are characterized by β d k) i t k, t) = δ ik Detailed derivations and presentations of these constrained expressions can be found in Ref [1] However, let s give three constrained expression examples Example #1 In the first example consider the function, yt) = gt) + t t t) t t 1 ) ẏ 1 ġ 1 ) + t t t 1) t t 1 ) ẏ ġ ), ) where β 1 t, t) = t t t) t t 1 ) and β t, t) = t t t 1) The first derivative of Eq ) is t t 1 ) ẏt) = ġt) + t t ẏ 1 ġ 1 ) + t t 1 ẏ ġ ) t t 1 t t 1 It is easy to verify that, when t = t1) = t 1 then ẏt 1 ) = ẏ 1 and when t = t) = t then ẏt ) = ẏ Therefore, no matter what gt) is, Eq ) can be used as constrained expression for functions subject to: ẏt 1 ) = ẏ 1 and ẏt ) = ẏ Example # This example is for a function subject to the following n = constraints, d y dt = ÿ t1, yt ) = y t, yt 3 ) = y t3, and t1 dt = ẏ t t where, d = {, 0, 0, 1} Let s select the constraint time vector as, t = { 1, 0,, } A constrained expression with embedded all four constraints is + t t yt) = gt) + t ÿ t1 g t1 ) + 8 t + 3 t + t 3 y t g t ) ) + 3 t t 8 t y t3 g t3 ) + 10 t + 3 t + t 3 ẏ t ġ t ) 1 where + t t β 1 t, t) = t, β t, t) = 8 t + 3t + t 3, 1 8 3t t β 3 t, t) = t, and β t, t) = 10t + 3t + t It is not difficult to verify that yt), as defined by Eq 5), has embedded all four constraints, ÿ t1, y t, y t3, ẏ t ), independent what gt) is Example #3 This example shows the constrained expression when the constraints are specified in a relative way, as for In this specific case, a constrained expression is yt) = gt) + yt 1 ) = yt ) and ẏt 1 ) = ẏt ) t g 1 g ) + t t 1 + t ) t ġ 1 ġ ) t t 1 t t 1 ) It is straightforward to prove this equation satisfies the two relative constraints, y 1 = y and ẏ 1 = ẏ These three examples show that the solution, yt), can be expressed in term of an unknown function, gt), such that yt) always satisfies all DE constraints This allows us to re-write the original DE in terms of the new function, gt), thus obtaining a DE with constraints alrea embedded in the DE This new DE has two interesting properties: it is not subject to external constraints and it is linear in gt) and its derivatives In this stu simple constrained equations have been provided to solve linear DE with nonconstant coefficients for IVP, in Eqs 1, 0, 1), and for BVP, in Eqs 3, 9, 30, 31, 3, 33, 3, 35, 36)

3 3 A The least-squares approach Since function gt) is free to be selected, then it can be expressed as a linear combinations of a set of m linearly independent basis functions, h k t), m gt) = ξ T ht) = ξ k h k t) 6) This means that two distinct functions, h i t) and h j t) with i j, must span different function spaces In addition, all functions, h k t), and their derivatives must be continuous and nonsingular within the time range By doing this, the coefficients ξ k of Eq 6) become our unknowns Once these coefficients are computed, the gt) function is known and, consequently, Eq ) provides the DE solution Examples of basis functions are polynomials eg, Lagrange, Legendre, monomial, Chebyshev, etc), Fourier series, monomial plus Fourier series, and any combinations of continuous and nonsingular functions spanning different function spaces By substituting the expression of yt), given in Eq 3), in the DE of Eq ), along with the expression of gt), given in Eq 6), and its derivatives, ġt) = ξtḣt) and gt) = ξtḧt), a linear equation in terms of the unknown coefficients, ξ, is obtained This equation can be then specialized for a set of N values of t j eg, uniformly distributed in the integration time range), obtaining m ξ k p k t j ) = p T t j ) ξ = λt j ) k=0 where pt j ) is an n-long known vector and j [1, N] and N m + 1 This set of N equations in m + 1 unknowns usually, N m) can be set in the matrix form p 0 t 1 ) p 1 t 1 ) p m t 1 ) ξ 0 λt 1 ) p 0 t ) p 1 t ) p m t ) ξ 1 λt ) P ξ = = = λ p 0 t N ) p 1 t N ) p m t N ) λt N ) admitting the LS solution k=0 ξ m ξ = P T P ) 1 P T λ 7) This LS solution is computed by scaling the P matrix is order to decrease the condition number of P T P and, consequently, the numerical errors This procedure is applied in detail for a second order nonhomogeneous linear DE with nonconstant coefficients for IVP and BVP, respectively B The selected basis functions In all the examples provided in this paper, the Chebyshev Orthogonal Polynomials COP) of the first kind have been selected to represent the basis functions set Note that, this selection may not be the best solution In fact, while COP are a versatile basis to describe almost any kind of function, the COP derivatives are affected by a sort of Runge s phenomenon, with one order of magnitude increase at each subsequent derivatives Figure 1 shows this effect for the first two derivatives Since COP are defined in term of a new variable, x [ 1, +1], we set x linearly related to t [t 1, t ], as x = t t 1 1 t = t 1 + x + 1)t t 1 ), 8) t t 1 where t is specifically defined in BVP while it can be considered as the integration upper limit in IVP Setting δt = t t 1, the derivatives in terms of the new variable are, dt = d y dt = d dt Therefore, Eq ) can be re-written as, dt = ) δt = d dt dt ) dt = d δt ) dt = δt d y 9) δt f x) d yx) + x) x) + f 0x) yx) = fx), 10) where the functions f, f 1, f 0, and f are now expressed in term of the new variable using Eq 8) By changing the integration variable, particular attention must be given to the constraints specified in term of derivatives In fact, the derivatives dt and

4 Fig 1 Chebyshev Orthogonal Polynomials and the first two derivatives d y dt are related to derivatives and d y as specified by Eq 9) Therefore, also the constraints, provided in term of the first and/or second derivatives, need to comply with the rules given in Eq 9), = δt x1 dt = δt d y ẏ1 = ẏ 1x and = d y δt x1 dt t1 = δt ÿ1 = ÿ 1x, t1 meaning that: the constraints on the derivatives in term of the new x variable now depend on the integration time range III LEAST-SQUARES SOLUTION OF INITIAL VALUE PROBLEMS Three distinct IVPs can be considered, depending on the DE constraints kind Consider first the most classic problem where the function and its first derivative are specified in one point A Initial Value Problems subject to: yt 1 ) = y 1 and ẏt 1 ) = ẏ 1 In this case, the constraints written in term of the new variable x) are, yx 1 = 1) = y 1 and and a simple constrained expression for this IVP is, x1= 1 = ẏ 1x = δt ẏ1, yx) = gx) + y 1 g 1 ) + x + 1)ẏ 1x ġ 1 ), 11) where g 1) = g 1 and ġ 1) = ġ 1 Again, the solution yx), as expressed by Eq 11), has embedded the constraints, no matter what the function gx) is Substituting yx), as expressed by Eq 11), in Eq 10) we obtain, δt f d g + ) dg ġ 1 + f 0 [g g 1 ġ 1 x + 1)] = f δtẏ1xf 1 f 0 [y 1 + ẏ 1x x + 1)] 1)

5 5 Now, let gx) be expressed as a linear combination of COPs of the first kind, m gx) = ξ k T k x), 13) which are defined by the recursive function, T k+1 = x T k T k 1 k=0 starting from: All derivatives of COP can be computed in a recursive way, starting from dt 0 = 0, dt 1 = 1 and d d T 0 d = dd T 1 = 0 d > 1), d while the subsequent derivatives of Eq 1) give for k > 1, { T0 = 1 T 1 = x 1) dt k+1 = T k +x dt k d T k+1 = dt k +x d T k dt k 1 d T k 1 15) d d T k+1 d = d dd 1 T k d 1 +x dd T k d In particular, it is easy to show that, T k 1) = 1) k dt k, = 1) k+1 k, x= 1 k=0 dd T k 1 d, d 1), d T k = 1) k k k 1) 16) x= 1 3 Therefore, substituting the expressions given in Eqs 13-16) in Eq 1), the following equation m { ξ k δt f d T k + [ ] δt f dtk 1 1)k+1 k [ + f 0 Tk 1) k 1) k+1 k x + 1) ]} = k= = f δtẏ1xf 1 f 0 [y 1 + ẏ 1x x + 1)]} is obtained However, particular attention must be given to Eq 17) because, for k = 0 and k = 1, all three terms of the RHS vanish, d T k = dt k 1)k+1 k = T k 1) k 1) k+1 k x + 1) = 0 for k = 0 and k = 1 which is equivalent to rewrite Eq 17) as m { ξ k δt f d T k + [ ] δt f dtk 1 1)k+1 k [ + f 0 Tk 1) k 1) k+1 k x + 1) ]} = 18) = f δtẏ1xf 1 f 0 [y 1 + ẏ 1x x + 1)]} The reason why in Eq 17) for k = 0 and k = 1, all three terms of the RHS vanish, derives from the fact that the first two terms of COP are constant and linear in x Now, the constrained expression of Eq 11) is derived using a constant plus a linear expression in x This means that the basis functions used for gx) cannot be composed using the same function spaces, namely, the constant and the linear expression in x, because alrea adopted to define the constrained expression d T k Note that, the two derivatives, and dt k, are specific known polynomials in x, derived using Eqs 13-15) Therefore, Eq 18) is a linear equation in terms of the m 1) unknown coefficients ξ k This allows us to estimate these coefficients by LS, by specifying Eq 18) for a set of N values x j, ranging from x 1 = 1 to x = +1 Specifically, we have p k x j ) = [ δt f x j ) d T k + xj x j ) dt k 1) k+1 k xj + f 0 x j ) [ T k x j ) 1) k 1) k+1 k x j + 1) ] λx j ) =fx j ) δtẏ1xf 1 x j ) f 0 x j )[y 1 + ẏ 1x x j + 1)] In the next subsection the proposed approach is applied to a DE with known analytical solution Accuracy comparison is provided with respect to the solution obtained by the Runge-Kutta-Fehlberg step-varying integrator MATLAB function ODE5) ] + 17)

6 6 B Accuracy tests Consider integrating the following IVP from t 1 = 1 to t =, t d y tt + ) + t + ) y = 0 dt dt { y1) = y1 = 1 subject to: ẏ1) = ẏ 1 = 0 This implies ẏ 1x = ẏ 1 3 = 0 The general solution of this equation is yt) = e t 1) t Equation 19) has been solved using the proposed LS approach with m = 16 and N = 1, 000) and integrated using MATLAB function ODE5, implementing the Runge-Kutta-Fehlberg variable step integrator The results are shown in Fig 19) Fig Results example with known solution m = 16 and N = 1, 000) In the left plot of Fig the absolute values of mean and standard deviation of the P ξ λ) residuals are shown as a function of m When the residuals standard deviation reaches the minimum at m = 17) 1 the LS approach provides the best accuracy results The errors with respect to the true solution for the LS approach and the errors obtained using MATLAB function ODE5, are shown in the right plot For this IVP, the LS method provides about five order of magnitudes accuracy gain with respect to ODE5 integrator C Initial Value Problems subject to: yt 1 ) = y 1 and ÿt 1 ) = ÿ 1 ÿ 1x = ÿ 1 δt Using the constrained equation, then Eq 10) becomes δt f yx) = gx) x y 1 g 1 ) + x + x ÿ 1x g 1 ) d ) g g 1 + dg = f δt f ÿ 1x 1 The value of m = 17 implies a size of matrix P T P ) ) + g x + 1 x ) + x 1 g 1 + f 0 g + g 1 x g 1 = ) x + 1 x ) 0) + x y 1 + ÿ 1x f 0 y 1 x + ÿ 1x

7 7 Note that, if y 1 and ÿ 1x are known, then ẏ 1x can be derived using Eq 10) evaluated at x 1 = 1 ẏ 1x = δt ft 1 ) f ÿ 1x δt f 0 t 1 ) y 1 t 1 ) provided that f 1 t 1 ) 0 Therefore, the solution of the DE given in Eq 10) with constraints y 1 and ÿ 1x can be solved as in the previous section with constraints y 1 and ẏ 1x, where ẏ 1x is provided as a function of ẏ 1 and integration time range δt D Initial Value Problems subject to: ẏt 1 ) = ẏ 1 and ÿt 1 ) = ÿ 1 Using the constrained equation, Eq 10) becomes ) x yx) = gx) + x ẏ 1x ġ 1 ) + + x ÿ 1x g 1 ) d ) δt f g g 1 + [ ] )] dg x ġ 1 g 1 x + 1) + f 0 [g ġ 1 x g 1 + x = = f δt f ÿ 1x )] x 1) [ẏ 1x + ÿ 1x x + 1)] f 0 [ẏ 1x x + ÿ 1x + x Again, if ẏ 1x and ÿ 1x are known, then y 1 can also be computed by specializing Eq 10) at x 1 = 1 y 1 = δt ft 1 ) f ÿ 1x t 1 ) ẏ 1x δt f 0 t 1 ) f 0 t 1 ) 0 ) Therefore, the solution of the DE given in Eq 10) with constraints ẏ 1x and ÿ 1x can be solved as in the previous section with constraints y 1 and ẏ 1x, where y 1 is provided by Eq ) IV LEAST-SQUARES SOLUTION OF BOUNDARY VALUE PROBLEMS Boundary Value Problems BVP) appear in many applications arising in science and engineering Examples are the modeling of chemical reactions, heat transfer, and diffusion A thorough survey of the existing solutions to this problem can be found in Ref [3], describing most of the existing methods with the exception of those using Bézier curves The use of implicit Bézier functions to obtain approximate solutions of BVP or IVP) is not a new idea, albeit it is quite recent 00) Venkataraman has attacked the problem using optimization techniques [], [5], [6] while Zheng uses analytical LS approach [7] Bézier curves have been adopted also to solve specific problems such as singular perturbed BVP [8] as well as integro-de [9] Two-point BVP are usually solved by iterative techniques The most common approaches are the shooting methods, transforming the BVP into IVP With respect to the analytical LS approach proposed in Ref [7], this paper has developed a practical, fast and easy to implement, numerical LS approach The proposed method does not require any sophisticated optimization technique to solve BVP applied to linear second-order nonhomogeneous DE Examples are provided with particularly emphasis on the approximate solutions accuracy levels Let s first consider the most common BVP whose constraints are y 1) = y 1 and y1) = y A constrained function is Substituting in Eq 10) we obtain δt f yx) = gx) + 1 x d g + y 1 g 1 ) x y g ) dg + g ) 1 g + f 0 g g 1 + g = f 1 y y 1 ) f 0 y1 + y In particular, using COP to describe gx), we have dt k T k 1) = 1, = k, x=1 Using these expressions in Eq 3) we obtain n { ξ k δt f d T k + k=0 [ dtk + 1)k 1 and = f 1 y y 1 ) f 0 y1 + y y 1 y x d T k + g ) 1 g x = ) 3) = k k 1) x=1 3 ] [ + f 0 T k 1)k + 1 y 1 y x ]} + 1)k 1 x = ) )

8 8 However, for k = 0 and k = 1, all three terms on the RHS of Eq ) becomes zeros d T k = dt k + 1)k 1 For this reason Eq ) must be selected as m { ξ k δt f d T k + k= = T k 1)k + 1 [ dtk + 1)k 1 = f 1 y y 1 ) f 0 y1 + y + 1)k 1 x = 0 for k = 0, 1 ] [ + f 0 T k 1)k + 1 The m 1) coefficients ξ k of Eq 5) are then computed by LS using Eq 7) A Numerical accuracy tests with known solution Consider the following BVP with constant coefficients, d y dt + dt + y = 0 y 1 y x ]} + 1)k 1 x = ) 5) { y0) = 1 subject to: y1) = 3, 6) whose solution is, yt) = e t +3e 1)te t, with derivatives, ẏt) = e t 3et t 3e+), and ÿt) = e t 3et t 6e+3) Figure 3 show the LS approach results for this test in terms of mean and standard deviation of P ξ λ) residuals top, left) and the condition number of matrix P T P top, center) as a function of the number of COP adopted m 1) to solve the LS problem The residuals of the DE of Eq 6) are provided top, right), and the errors of the LS solution, yt) bottom, left), the first derivative, ẏt) bottom, center), and the second derivative, ÿt) bottom, right), with respect to the true solution, are provided Fig 3 IVP least-square results for Eq 6) and for m [3, 3] Shooting methods are transforming BVP into IVP Numerical integrations of IVP, provide subsequent estimates based on previous estimates This implies that the error, in general, is accumulating In the contrary, the accuracy provided by LS solution is uniformly distributed within the integration bounds If more accuracy is desired on a specific range, then by increasing the number of points on that range or by providing greater weights to the points on that range, the accuracy increase is obtained where desired

9 9 B Tests with unknown solution, with no solution, and with infinite solutions Consider the DE with unknown solution, 1 + t) d y dt + cos t 3t + 1 ) dt + 6 sin t e cos3t)) y = [1 sin3t)]3t π), t subject to y0) = and y1) = In this case the LS solution results are given in the plots of Fig with the same meaning to those provided in Fig 3 The LS solution accuracy increases up to m = 1-degree COP The standard deviation of the residuals reaches about 10 1 accuracy level while the DE residuals are lower than Fig Results with unknown solution Consider the following BVP with no solution, d y dt 6 dt + 5 y = 0 { y0) = 1 subject to: yπ) = 7) In fact, the general solution of Eq 7) is yt) = [a cos t) + b sin t)] e 3 t, where the constraint y0) = 1 gives a = 1 while the constraint yπ) = gives = e 3 π, a wrong identity! Figure 5 shows the results when trying to solve by LS the problem given in Eq 7) For this example the number of COP terms has been increased up to when the matrix P T P become numerically singular, at m =, with a condition number value above The mean and standard deviation of the P ξ λ) residuals show no convergence while the condition number of P T P indicates that the problem has no solution It is possible to show that, even in the no solution case, the proposed LS approach provides anyway a solution complying with the DE constraints! Finally, consider the BVP with infinite solutions, d y dt + y = 0 { y0) = subject to: yπ) = 8) In fact, the general solution of Eq 8) is, yt) = a cos t) + b sin t), consisting of infinite solutions as b can have any value Results of the LS approach are given in Fig 6 showing the convergence, but not at machine level accuracy Note the differences between the two cases of no and infinite solutions Both of them experience bad condition number, but the convergence is experienced in the infinite solution case, only

10 10 Fig 5 Results with NO solution C Constraints: yt 1 ) = y 1 and ẏt ) = ẏ ẏ x = ẏ δt yx) = gx) + y 1 g 1 ) + x + 1)ẏ x ġ ) can be used Substituting this equation in Eq 10), we obtain δt f d g + ) dg ġ + f 0 [g g 1 ġ x + 1)] = = f ẏ x f 0 [y 1 + ẏ x x + 1)] 9) Then, using the expressions provided in Eqs 13-16) in Eq 9) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 9) has been tested, providing excellent results, which are not included for sake of brevity D Constraints: yt 1 ) = y 1 and ÿt ) = ÿ ÿ x = ÿ δt yx) = gx) x y 1 g 1 ) + x + x ÿ x g ) can be used Substituting this equation in Eq 10), we obtain d ) δt f g g + ) dg + g x + 1 x ) + x 1 g + f 0 g + g 1 x g = = f δt ÿxf ) δt f x + 1 x ) 30) + x 1 y 1 + ÿ x f 0 y 1 x + ÿ x Then, using the expressions provided in Eqs 13-16) in Eq 30) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 30) has been tested, providing excellent results, which are not included for sake of brevity

11 11 Fig 6 Results with infinite solutions E Constraints: ẏt 1 ) = ẏ 1 ẏ 1x = ẏ 1 δt and yt ) = y yx) = gx) + y g ) + x 1)ẏ 1x ġ 1 ) can be used Substituting this equation in Eq 10), we obtain δt f d g + ) dg ġ 1 + f 0 [g g ġ 1 x 1)] = = f δtẏ1xf 1 f 0 [y + ẏ 1x x 1)] 31) Then, using the expressions provided in Eqs 13-16) in Eq 31) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 31) has been tested, providing excellent results, which are not included for sake of brevity F Constraints: ẏt 1 ) = ẏ 1 ẏ 1x = ẏ 1 δt and ẏt ) = ẏ ẏ x = ẏ δt yx) = gx) + x 1 x ) ẏ 1x ġ 1 ) + x 1 + x ) ẏ x ġ ) can be used Substituting this equation in Eq 10), we obtain d ) δt f g + ġ1 ġ + [ dg ġ11 ] x) + ġ x + 1) + { + f 0 g x [ġ 1 1 x ) x )]} + ġ + 1 = f δt ẏ x ẏ 1x )f 1 δt f x 1 [ẏ 1x 1 x) + ẏ x x + 1)] f 0 [ẏ 1x 1 x ) x )] + ẏ x + 1 3)

12 1 Then, using the expressions provided in Eqs 13-16) in Eq 3) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 3) has been tested for the special case of the Mathieu s DE [11] G Constraints: ẏt 1 ) = ẏ 1 ẏ 1x = ẏ 1 δt and ÿt ) = ÿ ÿ x = ÿ δt x ) yx) = gx) + x ẏ 1x ġ 1 ) + x + 1 ÿ x g ) can be used Substituting this equation in Eq 10), we obtain d ) δt f g g + [ ] dg x ) ] ġ 1 g x + 1) + f 0 [g ġ 1 x g + 1 x = = f δt ÿxf [ x ) ] 33) [ẏ 1x + ÿ x x + 1)] f 0 ẏ 1x x + ÿ x + 1 x Then, using the expressions provided in Eqs 13-16) in Eq 33) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 33) has been tested, providing excellent results, which are not included for sake of brevity H Constraints: ÿt 1 ) = ÿ 1 ÿ 1x = ÿ 1 δt and yx ) = y yx) = gx) + x y g ) + x x 1)ÿ 1 g 1 ) can be used Substituting this equation in Eq 10), we obtain d ) δt f g g 1 + ) dg g x 1 x ) x g 1 + f 0 g g x g 1 = = f δt ÿ1f ) δt f x 1 x ) 3) x 1 y + ÿ 1 f 0 y + ÿ 1 Then, using the expressions provided in Eqs 13-16) in Eq 3) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 3) has been tested, providing excellent results, which are not included for sake of brevity I Constraints: ÿt 1 ) = ÿ 1 ÿ 1x = ÿ 1 δt and ẏt ) = ẏ ẏ x = ẏ δt yx) = gx) + x ẏ x ġ ) + x x )ÿ 1 g 1 ) can be used Substituting this equation in Eq 10), we obtain d ) δt f g g 1 + [ ] )] dg x ġ g 1 x 1) + f 0 [g ġ x g 1 x = f δt ÿ1f [ẏ x + ÿ 1 x 1)] f 0 = )] x 35) [ẏ x x + ÿ 1 x Then, using the expressions provided in Eqs 13-16) in Eq 35) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 35) has been tested, providing excellent results, which are not included for sake of brevity J Constraints: ÿt 1 ) = ÿ 1 ÿ 1x = ÿ 1 δt and ÿt ) = ÿ ÿ x = ÿ δt yx) = gx) + x 1 3 x)ÿ 1x g 1 ) + x x)ÿ x g ) can be used Substituting this equation in Eq 10), we obtain [ d δt f g g ] 11 x) + g x + 1) + + [ dg g 1x x ) + g x ] + x) + [ +f 0 g g 13x x 3 ) + g x 3 + 3x ] ) = 1 = f δt f [ÿ 1x 1 x) + ÿ 1x x + 1)] ÿ 1 x x ) + ÿ x x + x) f 0 ÿ 1x 3x x 3 ) + ÿ x x 3 + 3x ) 1 36)

13 13 Then, using the expressions provided in Eqs 13-16) in Eq 36) the LS solution can be obtained using the procedure described in Eqs 6-7) Equation 36) has been tested, providing excellent results, which are not included for sake of brevity K Optimal control example: state known at initial time and costate at final time This example consists of the linear DE, {ẋ } [ ] { } A11 t) A = 1 t) x λ A 1 t) A t) λ subject to: with the constrained expressions, { xt) = gx t) + x 0 g x0 ) λt) = g λ t) + λ f g λf ) Assuming, x = {x, ẋ} T and λ = {λ x, λẋ} T, then [ ] ht t) g x t) = α ḣ T t) and [ ] β T g λ t) = ht) γ T and the constrained expressions become, [ ] ht h T 0 xt) = x 0 + α ḣ T ḣt 0 and [ ] β T λt) = λ f + h h γ T f ) Then, the namic equation becomes, [ḣt ] [ ] ht h T 0 α A 11 [ ḧ T ] [ ḣ T ḣt 0] β T ht h T 0 ḣ A γ T 1 ḣ T ḣt 0 which can be written in matrix form, where [ ] β T α A 1 h h γ T f ) = A 11 x 0 + A 1 λ f α A [ β T γ T ] h h f ) = A 1 x 0 + A λ f, { xt0 ) = x 0 λt f ) = λ f, 37) α [ ] { } M β = A11 A 1 x0, 38) A γ 1 A λ f [ḣt ] [ ] [ ] [ ] ht h T 0 ht h T f 0 T A 11 A M = ḧ T ḣ T 1 A ḣt 0 0 T 1 h T h T [ ] f ht h T [ḣt ] [ ] [ ] [ ] 0 ht h T f 0 T 0 T A 1 A ḣ T ḣt 0 0 T A 0 T ḣ T h T h T f Equation 38) is linear in the unknown coefficients vectors: α, β, and γ) and, therefore, it can be solved by LS as done in the previous numerical examples Equation 37) can be subject to different constraints The following are two examples that can be solved by LS using the corresponding constrained expressions, { { xt0 ) = x 0 xt) = gx t) + x 0 g x0 ) + t t 0 )ẋ 0 ġ x0 ) ẋt 0 ) = ẋ 0 { xt0 ) = x 0 λt f ) = xt f ) λt) = g λ t) { xt) = gx t) + x 0 g x0 ) λt) = g λ t) + g xf + x 0 g x0 g λf ) CONCLUSIONS AND FUTURE WORK This stu presents a new approach to provide least-squares solutions of linear nonhomogeneous differential equations of any order with nonconstant coefficients, continuous and non singular in the independent variable integration range For sake of brevity and without loosing generality, the implementation of the proposed method has been applied to second order differential equations This least-squares approach can be adopted to solve initial and boundary value problems with constraints given in terms of the function and/or derivatives The proposed method is based on searching the solution with a specific expression, called constrained expression, which is a function with embedded differential equation constraints This expression is given in terms of a new unknown function, gt) The original differential equation is rewritten in terms of gt), thus obtaining a new differential equation where the constraints are embedded in the differential equation itself Then, the gt) function is expressed as a linear combination of basis functions, ht) The coefficients of this linear expansion are then computed by least-squares by specializing the new differential equation for a set of N different values of the independent variable In this stu the Chebyshev orthogonal polynomial of the first kind have been selected as basis functions This choice may not be a good choice because each subsequent derivative degree

14 1 increases the range by approximately one order of magnitude and because polynomials are in general a bad choice to describe potential periodic solutions) Numerical tests have been performed for initial value problems with known solution A direct comparison has been made with the solution provided by MATLAB function ODE5, implementing the Runge-Kutta-Fehlberg variable-step integrator In this test, the least-squares approach shows five orders of magnitude accuracy gain Numerical tests have been performed for boundary value problems for the four cases of known, unknown, no, and infinite solutions In particular, the condition number and the residual mean of the least-square approach can be used to discriminate whether a boundary value problem has no solution or infinite solutions The proposed method is easy to implement, not iterative and, as opposed to classic numerical approaches, the solution error distribution do not increase along the integration but it is approximately uniformly distributed in the integration range In addition, the proposed technique is identical to solve initial and boundary value problems The method can also be used to solve higher order linear differential equations with linear constraints This stu is not complete as many investigations should be performed before setting this approach as a standard way to integrate linear differential equations Many research areas are still open, full of question marks A few of these open research areas are: 1) Extension to weighted least-squares; ) Nonuniform distribution of points and optimal distributions of points to increase accuracy in specific ranges of interest; 3) Comparisons with different function bases and identification of an optimal function base if it exists!); ) Analysis using Fourier bases; 5) Accuracy analysis of number of basis functions versus points distribution; 6) Extension to nonlinear differential equations; 7) Extension to partial differential equations 8) Extension to nonlinear constraints This stu does not provide answers to the above questions, but it provides suggestions of important area of research and basic tools to dig REFERENCES [1] Mortari, D The Theory of Connections Part 1: Connecting Points, AAS of 017 AAS/AIAA Space Flight Mechanics Meeting Conference, San Antonio, TX, February 5-9, 017 [] Strang, G Differential Equations and Linear Algebra, Wellesley-Cambridge Press, 015, ISBN , [3] Lin, Y, Enszer, JA, and Stadtherr, MA Enclosing all Solutions of Two-Point Boundary Value Problems for ODEs, Computers and Chemical Engineering, 008, pp [] Venkataraman, P A New Class of Analytical Solutions to Nonlinear Boundary Value Problems, DETC , 5th Computers and Information in Engineering CIE) Conference, Long Beach, CA, September, 005 [5] Venkataraman, P Explicit Solutions for Linear Boundary Value Problems using Bézier Functions, DETC , 6th Computers and Information in Engineering CIE) Conference, Philadelphia, PA, September, 006 [6] Venkataraman, P and Michopoulos, JG Explicit Solutions for Nonlinear Partial Differential Equations, DETC , 7th Computers and Information in Engineering CIE) Conference, Las Vegas, NA, September, 007 [7] Zheng, JM, Sederberg, TW, and Johnson, RW Least-Squares Methods for Solving Differential Equations using Bézier Control Points, Applied Numerical Mathematics, Vol 8, No, Feb 00, pp 37-5 [8] Evrenosoglu, M, Somali, S Least-Squares Methods for Solving Singular Perturbed Two-Point Boundary Value Problems Using Bézier Control Points, Applied Mathematics Letters, Vol 1, No 10, Oct 008, pp [9] Ghomanjani, F, Kamyad, AV, and Kiliman, A Bézier Curves Method for Fourth-Order Integro-Differential Equations, Abstract and Applied Analysis, Vol 013, Article ID 67058, 5 pages, 013 doi:101155/013/67058 [10] Doha, EH, Bhrawy, AH, and Saker, MA On the Derivatives of Bernstein Polynomials: An Application for the Solution of High Even-Order Differential Equations, Hindawi Publishing Corporation, Boundary Value Problems, Vol 011, Article ID 8953, 16 pages, doi:101155/011/8953 [11] Mathieu, E Mémoire sur Le Mouvement Vibratoire d une Membrane de Forme Elliptique, Journal de Mathématiques Pures et Appliquées, , 1868 [1] Meixner, J and Schäfke, FW Mathieusche Funktionen und Sphäroidfunktionen Springer, 195