Received: 30 July 2017; Accepted: 29 September 2017; Published: 8 October 2017

Transcription

1 mathematics Article Least-Squares Solution o Linear Dierential Equations Daniele Mortari ID Aerospace Engineering, Texas A&M University, College Station, TX 77843, USA; mortari@tamu.edu; Tel.: This paper is an extended version o our paper published in Mortari, D. Least-squares Solutions o Linear Dierential Equations. In Proceedings o 7th AAS/AIAA Space Flight Mechanics Meeting Conerence, San Antonio, TX, USA, 5 9 February 17. Received: 3 July 17; Accepted: 9 September 17; Published: 8 October 17 Abstract: This study shows how to obtain least-squares solutions to initial value problems IVPs), boundary value problems BVPs), and multi-value problems MVPs) or nonhomogeneous linear dierential equations DEs) with nonconstant coeicients o any order. However, without loss o generality, the approach has been applied to second-order DEs. The proposed method has two steps. The irst step consists o writing a constrained expression, that has the DE constraints embedded. These kind o expressions are given in terms o a new unknown unction, gt), and they satisy the constraints, no matter what gt) is. The second step consists o expressing gt) as a linear combination o m independent known basis unctions. Speciically, orthogonal polynomials are adopted or the basis unctions. This choice requires rewriting the DE and the constraints in terms o a new independent variable, x [ 1, +1]. The procedure leads to a set o linear equations in terms o the unknown coeicients o the basis unctions that are then computed by least-squares. Numerical examples are provided to quantiy the solutions accuracy or IVPs, BVPs and MVPs. In all the examples provided, the least-squares solution is obtained with machine error accuracy. Keywords: linear least-squares; interpolation; embedded linear constraints 1. Introduction The purpose o this article is to provide a uniied ramework to obtain analytical, least-squares, approximated solutions o nonhomogeneous linear dierential equations DEs) o order n 1 with nonconstant coeicients subject to a set o n constraints: n k t) dk y k= dt k = t) subject to: d i y dt i = y i) j 1) tj where k t) and t) can be any unctions, continuous and nonsingular, within the range o integration, and y i) j indicates the ith derivative speciied at time t j. The proposed method uniies how to solve initial value problems IVPs), boundary value problems BVPs), or multi-point value problems MVPs). While the proposed method works or linear DEs o any order, or brevity o exposition, the method is presented to solve second-order DEs: t) d y dt + 1t) dy dt + t) y = t) ) This is because the proposed technique does not change i n increases, other than by adding additional terms. Mathematics 17, 5, 48; doi:1.339/math5448

2 Mathematics 17, 5, 48 o Background on the Existing Approaches Various techniques exist to provide numerical approximations o the DEs given in Equation 1). The most widely used approaches are the Runge Kutta methods [1] or IVPs and shooting methods or BVPs. The shooting methods solve BVPs essentially by transorming BVPs into IVPs. These methods provide enough precision or most o the practical problems. However, the estimated solution is not analytic and is provided at discrete points, usually with an error growing along the range o integration. Reerences [,3] provide a collection o the most well-known methods or solving IVPs and BVPs. Other existing alternative integration approaches include, but are not limited to, the Gauss Jackson method [4], collocation techniques [5], and the Modiied Chebyshev Picard Iteration [6] a path-length integral approximation), which has recently been proven to be highly eective. All o these methods are based on low-order Taylor expansions, which limits the step size that can be used to propagate the solution. Higher-order Taylor methods have also been analyzed [7]. More contemporary approaches are the ollowing: Collocation Techniques. In these techniques, the solution components are approximated by piecewise polynomials on a mesh. The coeicients o the polynomials orm the unknowns to be computed. The approximation to the solution must satisy the constraint conditions and the DEs at collocation points in each mesh subinterval. We note that the placement o the collocation points is not arbitrary. A modiied Newton-type method, known as quasi-linearization, is then used to solve the nonlinear equations or the polynomial coeicients. The mesh is then reined by trying to equidistribute the estimated error over the whole interval. An initial estimate o the solution across the mesh is also required. However, a common weakness o all these collocation techniques low-order Taylor expansion models) is that they are not eective in enorcing algebraic constraints. Enorcing the DE constraints is the strength o the proposed method, as the DE constraints are embedded in the searched solution expressions. This means that the analytical solution obtained, even i completely wrong, perectly satisies all the DE constraints. Spectral methods. Spectral methods are used to numerically solve DEs by writing the solution as a sum o certain basis unctions e.g., Fourier series) and then choosing the coeicients in the sum in order to satisy the DEs as well as possible e.g., by least-squares). Additionally, in spectral methods, the constraints must then be enorced. This is done by replacing one or more) o the least-squares equations with the constraint conditions. The proposed method proceeds in the reverse sequence. It irst builds a special unction, called a constrained expression, that has the boundary conditions embedded, no matter what the inal estimation is. Then, it uses this expression to minimize the DE residuals by expressing the ree unction o the constrained expression, gt), as a sum o some basis unctions e.g., orthogonal polynomials). The resulting estimation always perectly satisies the boundary conditions as long as they are linear. To provide an example, a linear constraint involving our times t 1, t, t 3 and t 4 ) can be: π = 7yt 1 ) e ẏt 1 ) yt ) 5 ẏt 3 ) + 3 ÿt 3 ) yt 4 ) It is not clear how the spectral methods can enorce this kind o general linear constraint. Additionally, the spectral methods provide solutions approximating the constraints. The solution o the proposed method is an analytical unction ully complying with the DE constraints, and it approximates the DE by minimizing the residuals in a least-squares sense. 1.. The Idea Behind the Proposed Method The least-squares solution o Equation ) is obtained using constrained expressions, which are introduced in [8] by the theory o connections, a methodology to provide general interpolation unctions

3 Mathematics 17, 5, 48 3 o 18 with an embedded set o n linear constraints. These expressions are ound using the ollowing general orm: yt) = gt) + n k=1 η k h k t) = gt) + η T ht) 3) where h k t) are n assigned unctions and gt) is an unknown unction, independent rom the ht) unctions; η k are n coeicients that are derived by imposing the n linear constraints on the yt) unction. The linear constraints considered in the theory o connections [8] are any linear combination o the unction and/or its derivatives evaluated at speciied values o the independent variable, t. For an assigned set o n independent unctions, h k t), the n constraints o the DE allow us to derive the unknown vector o coeicients, η. By substituting this expression into Equation 3), we obtain an equation satisying all the constraints, no matter what gt) is. The problem o solving the DE is, thereore, to ind the unction gt). Because the DE is linear in yt) and yt) is linear in gt), then, by substituting Equation 3) into the DE, we obtain a new DE, linear in gt). In this paper, the solution, gt), is then expressed as a linear combination o orthogonal polynomials. The coeicients o this linear combination constitute the solution o the problem. Orthogonal polynomials are excellent basis sets to describe the gt) ree unction appearing in Equation 3). However, most o these orthogonal polynomials, such as Chebyshev or Legendre orthogonal polynomials, are deined in some speciic deined range, x [x, x ]. Speciically, Chebyshev and Legendre orthogonal polynomials are deined in x [ 1, +1].) Thereore, in order to adopt them, the DE must be rewritten in terms o the new variable, x, linearly related to t [t, t ] as: x = x x t t t t ) + x t = t t x x x x ) + t and by introducing the integration range ratio, c = x x t t, these relationships become: x = c t t ) + x t = x x c where t is speciied in BVPs while in IVPs it is the integration upper limit. This change o variable also aects the derivatives. Using the notation ẏ or the derivative with respect to t and y or the derivative with respect to x, we obtain: ẏ = dy dx + t dx dt = c y and ÿ = c y 4) By changing the integration variable, particular attention must be given to the constraints speciied in terms o derivatives. In act, the constraints provided in terms o the irst and/or second derivatives also need to comply with the rules given in Equation 4), meaning that the constraints on the derivatives in terms o the new x variable depend on the integration time range, through the integration range ratio, c. Functions t), 1 t), t) and t) are also rewritten in terms o x, where, or instance, t) = [t + x + 1)/c], which can be indicated by x). This means that, using the new variable, we can write t) = x), 1 t) = 1 x), t) = x), and t) = x). Thereore, Equation ) can be rewritten as: c x) y x) + c 1 x) y x) + x) yx) = x) 5)

4 Mathematics 17, 5, 48 4 o 18. Least-Squares Solution o IVPs A second-order DE has two constraints. Thereore, once the DE is written as in Equation 5), the constrained expression is searched, having the orm: For IVPs, with constraints, yx) = gx) + η 1 px) + η qx) 6) The values o η 1 and η are: yt ) = y 1) = y and ẏt ) = ẏ = c y 1) = c y { } [ ] 1 { } { } { } η 1 p = q y g Y η p q y = h g Y 11 g + h 1 g h 1 g + h g Substituting into Equation 6), yx) = gx) + Y h 11 g h 1 g ) px) + Y h 1 g h g ) qx) and setting αx) = h 11 px) + h 1 qx) and βx) = h 1 px) + h qx), we obtain: Then, substituting into Equation 6), yx) = gx) αx) g βx) g + Y px) + Y qx) cc [g α g β g ] + c 1[g α g β g ] + [g αg βg ] = c [Y p + Y q ] c 1 [Y p + Y q ] [Y p + Y q], The DE, written in terms o gx), becomes: c x) g x) + c 1 x) g x) + x) gx) m 1 x) g m x) g = bx) 7) where: m 1 x) = c x) α x) + c 1 x) α x) + x) αx) m x) = c x) β x) + c 1 x) β x) + x) βx) bx) = x) c x)[y p + Y q ] c 1 x)[y p + Y q ] x)[y p + Y q] 8) The unction gx) can be set as a linear combination o m basis unctions, gx) = ξ T hx), where hx) is the vector o m basis unctions, and ξ is a vector o m unknown coeicients. Thereore, Equation 7) becomes: [ c h + c 1 h + h m 1 h m h ] T ξ = at x) ξ = bx) 9) Equation 9), or any value o x, is linear in terms o the unknown coeicients ξ). Thereore, this equation can be speciied or a set o N values o x ranging rom the initial value, x = x, to the inal

5 Mathematics 17, 5, 48 5 o 18 value, x = x. This can be done, or instance, by a uniorm x step. This set o N equations in m unknowns usually, N m) can be set in the matrix orm: where: a 11 a 1 a 1m ξ 1 b 1 a 1 a a m ξ b A ξ = = = b 1)... a N1 a N a Nm ξ m a ij = c x i ) h j x i) + c 1 x i ) h j x i) + x i ) h j x i ) m 1 x i ) h m x i ) h The linear system o Equation 1) allows us to compute the solution coeicient vector ξ) by least-squares. The simplest way to write a constrained expression subject to yx ) = y and y x ) = y = ẏ /c is to select px) = 1 and qx) = x. This provides the values o η 1 and η by inverting a matrix with its determinant equal to 1. For the variety o constraints considered in this paper, this property can be obtained by selecting the unctions px) and qx) to be those provided in Table 1. Table 1. Simplest selection o px) and qx) or various initial value problem and boundary value problem constraints. b N Dierential Equation Constraints px) qx) [y, y ] or [y, y ] or [y, y ] or [y, y ] 1 x [y, y ] or [y, y ] or [y, y ] 1 x / [y, y ] or [y, y ] or [y, y ] or [y, y ] x x / [y, y ] x / x 3 /3.1. Accuracy Tests We consider integrating the ollowing IVP rom t = 1 to t = 4: t ÿ tt + ) ẏ + t + ) y = subject to: { y1) = y = 1 ẏ1) = ẏ = 11) whose general solution is yt) = e t 1) t. We consider solving this IVP using Chebyshev orthogonal polynomials. This implies that, in terms o the new variable, the constraint derivative becomes y = 3 ẏ =. Equation 11) has been solved using the proposed least-squares approach with m = 18 basis unctions and a discretization made o N = 1 points, and it has been integrated using the MATLAB unction ode45 The numerical comparisons made in this paper are limited to ode45 because it is the most widely used method in commercial applications.), implementing a Runge Kutta variable-step integrator ormula [1]. The results obtained are shown in Figure 1. In the top-let plot, the L norm o Aξ b) is shown as a unction o the number o basis unctions, while the bottom-let plot provides the condition number to solve the least-squares. I the solution requires just m = 9 basis unctions and m = 18 is used, then the coeicients rom m [1, 18] are all zeros. The values o the L norm o the least-squares residuals identiy the minimum number o basis unctions. The accuracies obtained using MATLAB s ode45 [1] integrator and using the least-squares solution The least-squares problem has been solved

6 Mathematics 17, 5, 48 6 o 18 by QR decomposition o a scaled A matrix.) with m = 18 basis unctions and N = 1 points are shown in the right plot. The least-squares solution shows about 9 orders o magnitude o accuracy gain. In the ollowing subsections, the constrained expressions or two other kinds o IVP known-unction and irst-derivative and known irst- and second-derivative are provided. Figure 1. Results o example with known solution m = 18 and N = 1)... Second-Order DE Subject to yt ) = y and ÿt ) = ÿ In terms o the new x variable, the initial conditions are y and y = ÿ /c, while a constrained expression is: yx) = gx) x y g ) + x + x y g ) Then Equation 5) becomes: c g g ) + c 1 = c y c 1 g + g g y + y ) x ) x + 1 g + g x g y x + y x + x x ) + x ) and the least-squares solutions ollows the same steps provided in Equations 7) 1). We note that, i y and y are known, then y can be derived using Equation 5) evaluated at x : y = t ) c y t ) y c 1 t ) provided that 1 t ) =. Thereore, the least-squares solution o the DE given in Equation 5) with constraints y and y can be solved as in the previous section with constraints y and y..3. Second-Order DE Subject to ẏt ) = ẏ and ÿt ) = ÿ In terms o the new variable, the initial conditions are y = ẏ /c and y = ÿ /c. In this case, a constrained expression is: yx) = gx) + x y x g ) + ) + x y g )

7 Mathematics 17, 5, 48 7 o 18 Equation 5) becomes: c g g ) + c 1 [g g g = c y c 1 [y + y x + 1)] [ x x + 1)] + g g x )] g + x [ x y x + )] y + x and the least-squares solution ollows the same steps provided in Equations 7) 1). Again, i y and y are known, then y can also be computed by specializing Equation 5) at x : y = t ) c y c 1t ) y t ) where t ) = Thereore, the solution o the DE given in Equation 5) with constraints y solved as in the previous section with constraints y and y. 3. Least-Squares Solutions o BVPs and y can also be A two-point BVP, already written in terms o the new variable and subject to the constraints yx ) = y and yx ) = y, has a constrained expression that can be obtained using Equation 6), where the values o η 1 and η are: that is, { } [ ] 1 { } { } { } η 1 p = q y g Y = h 11 g + h 1 g η p q y g Y h 1 g + h g Substituting into Equation 6), yx) = gx) + Y h 11 g h 1 g ) px) + Y h 1 g h g ) qx) yx) = gx) [px) h 11 + qx) h 1 ] g [px) h 1 + qx) h ] g + Y px) + Y qx) which is an equation that can be written as: yx) = gx) αx) g βx) g + Y px) + Y qx) Substituting this expression into the DE, and we obtain: c x) y x) + c 1 x) y x) + x) yx) = x) c x) g x) + c 1 x) ġx) + x) gx) m 1 x) g m x) g = bx) 1) where: m 1 x) = c x) α x) + c 1 x) α + x) αx) m x) = c x) β x) + c 1 βx) + x) βx) bx) = x) c x)[y p + Y q ] c 1 [Y p + Y q ] [Y p + Y q] 13)

8 Mathematics 17, 5, 48 8 o 18 Thereore, the procedures to solve IVPs and BVPs have little dierences. In act, or BVPs, we obtain the linear equation: a T x) ξ = [ c x) h x) + c 1 x) hx) + x) hx) m 1 x) h m x) h ] T ξ = bx) 14) Once the least-squares has been solved and, thereore, the coeicients vector ξ has been computed, then the solution is: yx) = ξ T hx) + η 1 px) + η qx) y x) = ξ T h x) + η 1 p x) + η q x) 15) y x) = ξ T h x) + η 1 p x) + η q x) Numerical integrations o IVPs provide subsequent estimates based on previous estimates. This implies that the error, in general, increases along the integration range. On the contrary, as shown by the next numerical tests, the accuracy provided by the least-squares solution is uniormly distributed along the integration range. In addition, i greater accuracy is desired or some speciic regions, then by increasing the number o points in those regions, accuracy increase is obtained where desired. Alternatively, by adopting weighted least-squares, the weight s value gives us an additional option to control the accuracy distribution. In the ollowing subsections, the proposed least-squares approach is tested, or the our cases o BVPs with known, unknown, no, and ininite solutions Numerical Accuracy Tests or a BVP with Known Solution We consider the ollowing BVP: ÿ + ẏ + y = subject to: { y) = 1 y1) = 3 16) whose solution is yt) = e t + 3e 1)te t, with derivatives ẏt) = e t 3et t 3e + ) and ÿt) = e t 3et t 6e + 3). Figure shows the least-squares-approach results or this test in terms o the L norm o A ξ b) residuals top-let) and the condition number o matrix A T A bottom-let) as a unction o the number o basis unctions considered. The errors with respect to the true solution obtained with m = 14 basis unctions Legendre orthogonal polynomials) and N = 1 points discretization) are provided in the right plot. These plots clearly show that the machine error precision is obtained with excellent numerical stability low condition number).

9 Mathematics 17, 5, 48 9 o 18 Figure. Least-squares-approach results known analytical solution). 3.. Numerical Accuracy Tests or a BVP with Unknown Solution We consider the DE completely invented) with an unknown analytical solution: ) 1 + t)ÿ + cos t 3t + 1 ẏ + 6 sin t e cos3t)) [1 sin3t)]3t π) y = 4 t subject to y) = and y1) =. In this case, the least-squares-solution results are given in the plots o Figure 3 with the same explanation as or those provided in Figure. The L norm o the residuals top-let plot) decreases up to m = 31-degree basis unctions Chebyshev orthogonal polynomials o the irst kind) while the condition number remains at good values bottom-let plot). The machine error values o the residuals, obtained or m = 31-degree basis unctions and N = 1 points, are shown in the right plot.

10 Mathematics 17, 5, 48 1 o 18 Figure 3. Least-squares-approach results unknown analytical solution) Numerical Accuracy Tests or a BVP with No Solution We consider the ollowing BVP with no solution: ÿ 6 ẏ + 5 y = subject to: { y) = 1 yπ) = 17) In act, the general solution o Equation 17) is yt) = [a cos4 t) + b sin4 t)] e 3 t, where the constraint y) = 1 gives a = 1, while the constraint yπ) = gives = e 3 π, a wrong identity. Figure 4 shows the results when trying to solve, by least-squares, the problem given in Equation 17). For this example, the number o orthogonal polynomials has been increased up to that or which the condition number o the matrix A T A becomes too large numerical instability), at an upper bound o 1 8. The number o basis unctions has been increased up to m = by keeping the number o points to N = 1. By increasing the number o basis unctions, the L norm o A ξ b) does not converge, and the condition number o A T A becomes huge. These are two potential indicators that the problem has no solution. However, even in this no-solution case, the proposed least-squares approach provides a solution complying with the DE constraints.

11 Mathematics 17, 5, o 18 Figure 4. Least-squares-approach results no solution) Numerical Accuracy Tests or a BVP with Ininite Solutions Finally, we consider the BVP with ininite solutions: ÿ + 4 y = subject to: { y) = yπ) = 18) In act, the general solution o Equation 18) is yt) = a cos t) + b sin t), consisting o ininite solutions, as b can have any value. The results o the least-squares approach are given in Figure 5, showing the convergence, but not at machine-level accuracy. Figure 5 shows the results obtained or this ininite-solutions case. The main dierence with respect to the no-solution case is in the unstable convergence o the L norm o A ξ b). In act, by increasing the number o basis unctions rom m = to m = 5, the L norm actually increases. This unusual behavior is still under analysis, and no explanation is currently available. The instability may be explained by several ininite) solutions being possible, and some o these may be close rom a convergence numerical point o view. In this ininite-solutions case, as in the no-solution case, the value o the condition number increases to very high values. In this example, an upper bound has been deined or the number o basis unctions m = 3), while there is no upper bound or the condition number value.

12 Mathematics 17, 5, 48 1 o 18 Figure 5. Least-squares-approach results ininite solutions) Constrained Expressions or BVP with Dierent Constraints In this section, constrained expressions are provided or BVPs o a second-order DE subject to all nine) possible constraints. 1) The irst constraints considered are the unction at the initial time, yt ) = y, and the irst derivative at the inal time, ẏt ) = ẏ. For this case, the inal constraint in terms o the new variable, x [ 1, +1], becomes y = ẏ /c, and the constrained expression: yx) = gx) + y g ) + x + 1)y g ) can be used. Substituting this into Equation 5), we obtain: c g + c 1 g g ) + [g g g x + 1)] = c 1 y [y + y x + 1)] 19) Then, the least-squares solution can be obtained using the procedure described by Equations 1) 15). Equation 19) has been tested, providing excellent results, which are not included here or sake o brevity. ) The second constraints are the unction at the initial time, yt ) = y, and the second derivative at the inal time, ÿt ) = ÿ. For this case, the inal constraint in terms o the new variable is y = ÿ /c, and the constrained expression is: yx) = gx) x y g ) + x + x y g )

13 Mathematics 17, 5, o 18 Substituting this into Equation 5), we obtain: c g g ) + c 1 g + g g = c y c 1 y + y Then, it ollows as in the irst case. ) x x + 1 g + g x g ) y x + y x + x x ) + x ) 3) The third constraints are the irst derivative at the initial time, ẏt ) = ẏ, and the unction at the inal time, yt ) = y. For this case, the initial constraint in terms o the new variable is y = ẏ /c, while the constrained equation: yx) = gx) + y g ) + x 1)y g ) can be used. Substituting this into Equation 5), we obtain: c g + c 1 g g ) + [g g g x 1)] = c y 1 [y + y x 1)] and the least-squares solution can be obtained using the procedure described by Equations 1) 15). 4) The ourth constraints are the irst derivative at the initial and inal times, ẏt ) = ẏ and ẏt ) = ẏ. This special equation is known as Mathieu s DE [9,1]. For this case, the constraints in terms o the new variable are y = ẏ /c and y = ẏ /c, while the constrained equation: yx) = gx) + x 1 x ) y g ) + x 1 + x ) y g ) can be used. Substituting this into Equation 5), we obtain: c g + g g = c y y ) c 1 ) [ + c 1 g 1 x)g + x + ] 1)g [ 1 x)y + x + 1)y { + g x [ 1 x ) x ) ]} g g ] x [ 1 x ) x ) ] y y and the least-squares solution can be obtained using the procedure described by Equations 1) 15). 5) The ith constraints are the irst derivative at the initial time, ẏt ) = ẏ, and the second derivative at the inal time, ÿt ) = ÿ. For this case, the constraints in terms o the new variable are y = ẏ /c and y = ÿ /c, while the constrained equation: yx) = gx) + x y g ) + x x + 1 ) y g ) can be used. Substituting this into Equation 5), we obtain: [ c g g ) + c 1[g g g x + 1)] + g g x g x ) = c y c 1 [y + y x + 1)] [x y x + 1 ) ] x y ] x = and the least-squares solution can be obtained using the procedure described by Equations 1) 15). 6) The sixth constraints are the second derivative at the initial time, ÿt ) = ÿ, and the unction at the inal time, yt ) = y. For this case, the irst constraint in terms o the new variable is y = ÿ /c, while the constrained equation: yx) = gx) + x y g ) + x x 1)y g )

14 Mathematics 17, 5, o 18 can be used. Substituting this into Equation 5), we obtain: c g g ) + c 1 g g g = c y c 1 y + y ) x 1 + g g x g x ) x ) x 1 y + y x ) x and the least-squares solution can be obtained using the procedure described by Equations 1) 15). 7) The seventh constraints are the second derivative at the initial time, ÿt ) = ÿ, and the irst derivative at the inal time, ẏt ) = ẏ. For this case, the constraints in terms o the new variable are y = ÿ /c and y = ẏ /c, while the constrained equation: yx) = gx) + x y g ) + x can be used. Substituting this into Equation 5), we obtain: [ c g g ) + c 1[g g g x 1)] + [ ] = c y c 1 y + y x 1) x )y g ) x g g x )] g x = x )] x [y x + y and the least-squares solution can be obtained using the procedure described by Equations 1) 15). 8) The inal eighth) constraints are the second derivative at the initial and inal times, ÿt ) = ÿ and ÿt ) = ÿ. For this case, the constraints in terms o the new variable are y = ÿ /c and y = ÿ /c, while the constrained equation: yx) = gx) + x 3 x)y 1 g ) + x 3 + x)y 1 g ) can be used. Substituting this into Equation 5), we obtain: [ ] 1 c g x)g + x + 1)g + c 1 [g x x )g + x + x)g ] 4 [ + g 3x x 3 )g + x3 + 3x )g ] 1 = c [y 1 x) + y x + 1)] c x x )y + x + x)y 3x x 3 )y 1 + x3 + 3x )y 4 1 and the least-squares solution can be obtained using the procedure described by Equations 1) 15). 4. MVP Example Finally, to show the potential o the proposed least-squares approach, we consider solving the ollowing third order DE:... y + sin t ÿ + 1 t) ẏ + t y = t) ) where: t) = t 1) sin t + + t t cos t) sin t + tt 1) cos t and is subject to the multi-point constraints: yt 1 = 1) =, ẏt = π/) = 1, and yt 3 = π) =,

15 Mathematics 17, 5, o 18 whose analytical solution is yt) = 1 t) sin t. We consider integrating this DE rom t = to t = 4 beyond the constraint bounds, [1, π]) using Legendre orthogonal polynomials x [ 1, +1]). Thereore, t = x + 1) x = t 1 c = t t = 1 Equation ) becomes: c 3 y + c sin t y + c 1 t) y + t y = t) 1) subject to: yx 1 ) =, y x ) = 1/c =, and yx 3 ) = ) where x 1 = t 1 / 1 = 1/, x = t / 1 = π/4 1, and x 3 = t 3 / 1 = π/ 1. A constrained expression [8] satisying Equation ) can be ound as: yx) = gx) + η 1 h 1 x) + η h x) + η 3 h 3 x) 3) where h 1 x) = 1, h x) = x, and h 3 x) = x have been selected as the irst three monomials. The three coeicients, η 1, η, and η 3, can be derived rom the DE constraints: gx 1 ) 1 x 1 x g x ) gx 3 ) = 1 1 x 1 x 3 x3 Now, setting g = gx 1 ) = ξ T h 1, g = g x ) = ξ T h, and g 3 = gx 3 ) = ξ T h 3, the coeicients η 1, η and η 3 are: η 1 η η 3 = M 1 M 1 h 1 h h 3 T ξ = Then, substituting in the constrained expression, we obtain: whose derivatives are: v 1 v v 3 η 1 η η 3 m 11 h 1 + m 1 h + m T 13h 3 m 1 h 1 + m h + m 3h 3 ξ m 31 h 1 + m 3 h + m 33h 3 yx) = ξ T hx) + v 1 + v x + v 3 x m 11 + m 1 x + m 31 x )h T 1 ξ + m 1 + m x + m 3 x )h T ξ m 13 + m 3 x + m 33 x )h T 3 ξ y x) = ξ T h x) + v + v 3 x m 1 + m 31 x)h T 1 ξ m + m 3 x)h T ξ m 3 + m 33 x)h T 3 ξ y x) = ξ T h x) + v 3 m 31 h T 1 ξ m 3h T ξ m 33 h T 3 ξ y x) = ξ T h x) These expressions are then substituted back into Equation 1), and the linear equation provided by the least-squares approach is obtained. The numerical results o this procedure are given in Figure 6. The top-let plot o Figure 6 shows the analytic true solution black line) with the three constraints: two or the unction black markers) and one or the irst derivative red line). We note that the range o integration, t [, 4], goes beyond the constraints coordinate range, [1, π]. This is purposeully done to show that or the proposed least-squares approach, the range o integration, t [, 4], is independent rom the constraint range, [1, π]. The bottom-let plot shows that as the number o basis unctions considered increases, the condition number to solve the least-squares problem has values that guarantee the numerical stability o the least-squares solution. The number o basis unctions continues to increase until the L

16 Mathematics 17, 5, o 18 norm o Aξ b) decreases, as shown in the top-right plot. The minimum is reached when m = 1. However, the true number o basis unctions adopted to describe gx) is 18 and not 1. The reason is that, to build the constrained expression appearing in Equation 3), the unctions or h 1 x), h x), and h 3 x) are constant, linear, and quadratic, respectively. These three unctions cannot be included to express gx), which is consequently expressed as: gx) = 1 ξ k h k x) k=3 The bottom-right plot o Figure 6 shows the absolute error obtained when m = 1 basis unctions are used to express yx), while the number o Legendre polynomials used or gx) is m = 18. This plot clearly shows that, using 18 basis unctions or gx), the least-squares approach provides a machine error solution. Figure 6. Multi-point value problem example results. 5. Conclusions This study presents a new approach to provide least-squares solutions o linear nonhomogeneous DEs o any order with nonconstant coeicients, both continuous and nonsingular in the integration range. Without loss o generality and or the sake o brevity, the implementation o the proposed method has been applied to second-order DEs. This least-squares approach can be adopted to solve IVPs, BVPs, and MVPs with constraints given in terms o the unction and/or derivatives. The proposed method is based on imposing that the solution has a speciic expression, called a constrained expression, which is a unction with the DE constraints embedded. This expression is given in terms o a new unknown unction, gt). The original DE is rewritten in terms o gt), thus obtaining a new DE or which the constraints are embedded in the DE itsel. Then, the gt) unction is expressed as a linear combination o basis unctions, ht). The coeicients o this linear expansion are then computed by least-squares by specializing the new DE or a set o N dierent values o the

17 Mathematics 17, 5, o 18 independent variable. In this study, the Chebyshev and Legendre orthogonal polynomials have been selected as basis unctions. Numerical tests have been perormed or IVPs with known and unknown solutions. A direct comparison has been made with the solution provided by the MATLAB unction ode45, implementing a Runge Kutta variable-step integrator [1]. In this test, the least-squares approach shows about 9 orders o magnitude o accuracy gain. Numerical tests have been perormed or BVPs or the our cases o known, unknown, no, and ininite solutions. In particular, the condition number value and the L norm o the residual can be used to discriminate between whether or not a BVP has a single solution. Finally, a numerical test has been perormed or a third-order MVP with constraints two on unctions and one on the irst derivative) deined inside the integration range. In all our tests, the machine error precision has been reached with a limited number o basis unctions and the discretization number. All the computations were perormed on a laptop using MATLAB sotware. The elapsed times in seconds) required to obtain all the solutions are speciied in the igures or all the test cases considered. The proposed method is not iterative, is easy to implement, and, as opposed to numerical approaches, is based on truncated Taylor series such as Runge Kutta methods); the solution error distribution does not increase along the integration, but it is approximately uniormly distributed in the integration range. In addition, the proposed technique uniies the way to solve IVPs, BVPs, and MVPs. The method can also be used to solve higher-order linear DEs, as long as the DE constraints are linear. This study is not complete, as many investigations should be perormed beore setting this approach as a standard method to integrate linear DEs. In addition, many research areas are still open and under question. A ew o these open research areas are the ollowing: 1) Extension to weighted least-squares. ) Error-bound estimates or the problems admitting solutions. 3) Nonuniorm distribution o points and optimal distributions o points to increase accuracy in speciic ranges o interest. 4) Comparisons between dierent unction bases and identiication o an optimal unction base i it exists). 5) Analysis using Fourier bases. 6) Accuracy analysis o number o basis unctions versus points distribution. 7) Extension to nonlinear DEs. 8) Extension to partial DEs. 9) Extension to nonlinear constraints. Acknowledgments: The author would like to thank Rose Sauser or checking and improving the use o English in the manuscript. Conlicts o Interest: The author declares no conlict o interest. Abbreviations The ollowing abbreviations are used in this manuscript: DE IVP BVP MVP Dierential equation Initial value problem Boundary value problem Multi-point value problem

18 Mathematics 17, 5, o 18 Reerences 1. Dormand, J.R.; Prince, P.J. A Family o Embedded Runge-Kutta Formulae. J. Comp. Appl. Math. 198, 6, Strang, G. Dierential Equations and Linear Algebra; Wellesley-Cambridge Press: Wellesley, MA, USA, 15; ISBN Lin, Y.; Enszer, J.A.; Stadtherr, M.A. Enclosing all Solutions o Two-Point Boundary Value Problems or ODEs. Comput. Chem. Eng. 8, 3, Berry, M.M.; Healy, L.M. Implementation o Gauss-Jackson integration or orbit propagation. J. Astronaut. Sci. 4, 5, Elgohary, T.A.; Junkins, J.L.; Atluri, S.N. An RBF-Collocation Algorithm or Orbit Propagation. In Proceedings o the 5th AAS/AIAA Space Flight Mechanics Meeting, Williamsburg, VA, USA, January Junkins, J.L.; Bani Younes, A.; Woollands, R.; Bai, X. Picard Iteration, Chebyshev Polynomials, and Chebyshev Picard Methods: Application in Astrodynamics. J. Astronaut. Sci. 15, 6, , doi:1.17/s Jorba, A.; Zou, M. A sotware package or the numerical integration o ODEs by means o high-order Taylor methods. Exp. Math. 5, 14, Mortari, D. The Theory o Connections. Part 1: Connecting Points. In Proceedings o 7th AAS/AIAA Space Flight Mechanics Meeting Conerence, San Antonio, TX, USA, 5 9 February Mathieu, E. Mémoire sur Le Mouvement Vibratoire d une Membrane de Forme Elliptique. J. Math. Pures Appl. 1868, 13, Meixner, J.; Schäke, F.W. Mathieusche Funktionen und Sphäroidunktionen; Springer: Berlin, Germany, c 17 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions o the Creative Commons Attribution CC BY) license