An Optimization Method for Numerically Solving Three-point BVPs of Linear Second-order ODEs with Variable Coefficients

Transcription

1 An Optimization Method for Numericall Solving Three-point BVPs of Linear Second-order ODEs with Variable Coefficients Liaocheng Universit School of Mathematics Sciences , Liaocheng P.R. CHINA Abstract: It is known that most numerical methods for solving differential equations are based on iterative methods or Talor epansion methods. This paper tries to stud a numerical method from a new perspective optimization method. B means of the idea of kernel ε-svr, the paper constructs an optimization model for a class of threepoint boundar value problems (BVPs) of linear second-order ordinar differential equations (ODEs) with variable coefficients and proposes a novel numerical method for solving them. The proposed method has a certain versatilit and can be used to solve some other kinds of differential equations and integral equations. In order to verif the effectiveness of the proposed method, comparative eperiments with si specific linear second-order ODEs are performed. Eperimental results show that the proposed method has a good approimation propert. Ke Words: Optimization modeling, numerical method, ordinar differential equation, kernel ε-support vector regression, Lagrange function Introduction Differential equations can be found in the mathematical formulation of phsical phenomena in a wide variet of applications especiall in science and engineering. Depending on the form of the boundar conditions to be satisfied b the solution, problems involving ordinar differential equations (ODEs) can be divided into two main categories, initial value problems (IVPs) and boundar value problems (BVPs). Eact solutions for these problems are not generall available and hence numerical methods must be applied. Multi-point BVPs for ODEs can arise in solving linear partial differential equations b using the separation variable method []. Moreover, in the engineering problem to increase the stabilit of a rod, one also imposes a fied interior point ecept for the ends of the rod [2]. Along this line, the solutions of multipoint BVPs for ODEs have great significance in mathematical theor and practical applications [3-4]. It is seen that the three-point BVPs of nonlinear secondorder ODEs have attracted much attention [5-8] and it is significant to provide the solutions and Green s functions for these problems [9-0]. As shown in mentioned works, the eistence and uniqueness of the solution are alwas focused on b using a fied point theorem. However, from the viewpoint of practical Corresponding author. applications, one should provide eplicit solutions or s for multi-point BVPs. In this paper, inspired b work of S. Mehrkanoon et. al [], we tr from a new perspective to stud a class of threepoint BVPs of linear second-order ODEs with variable coefficients () + p() () + q()() = g(), [a, b], (a) = p 0, (b) + λ(µ) = q 0, µ (a, b), () where the known functions p() C [a, b], q(), g() C[a, b] and λ, µ are the constants, and research how to obtain its approimal solutions b using an optimization method based on kernel support vector regression (KSVR). Kernel support vector regressions (KSVRs) based on optimization modelings are a powerful methodolog for solving pattern recognition and function estimation problems, which based on Vapnik and Chervonenkis structural risk imization principle [2]. The show better generalization abilit compared to other machine learning methods on a wide variet of real-world problems, such as optimal control [3], image segmentation [4], image denoising [5], time series prediction [6], cber securit problems [7], network traffic predictive [8] and so on. E-ISSN: Volume, 205

2 KSVRs are used to deal with nonlinear regression problems b means of kernel skill. Data can be mapped into a high dimensional feature space b using a kernel function and then linear regression problems can be solved, which results in solving quadratic programg problems (QPPs). The main challenge in developing a useful regression modeling is to capture accuratel the underling functional relationship between the given inputs and their output values. Once the resulting model is obtained, it can be used as a tool for analsis, simulation and prediction. KSVR aims at detering a regressor for a given set of data points and a kernel function. The basic idea of kernel ε-svr with ε-insensitive loss function proposed b Vapnik [2] is to find a linear function () in the higher dimensional feature space such that, on the one hand, more mapped data samples locate in the ε- intensive tube between () ε and () + ε and on the other hand, the function () is as flat as possible, which leads to introduce the regularization term. Thus, the structural risk imization principle is implemented. The paper mainl researches how to use kernel ε- SVR method to obtain numerical solutions of Eq.(). It can be seen from the derivation process that some IVPs and BVPs of general linear ODEs ma be studied b using the similar wa. The paper does not involve nonlinear ODEs, which will be our net work. The remainder of this paper is organized as follows: Section 2 briefl reviews kernel ε-svr and some basic concepts. Section 3 presents a new approimation method for the solution of Eq.(). In order to verif the effectiveness of the presented method, a series of comparative eperiments with si specific linear second-order ODEs are performed in Section 4 and some concluding remarks are given in Section 5. 2 Kernel ε-svr This section briefl recalls kernel ε-svr, for details see [2]. Let T = {( i, i )} l i= be a set of sample data, where i R n and i R are input and output of the ith sample data respectivel. Let k : R n R n R be a kernel function with the reproducing kernel Hilbert space (RKHS) H and the nonlinear feature mapping φ : R n H. The space H is a higher dimensional space and can be epressed as an epended space of the mapped inputs {φ( i )} l i=, that is, H = span{φ( ),, φ( l )}. It is known that k(u, v) = φ(u), φ(v) for all u, v R n, where, denotes the inner product in RKBS H. Let K = [k( i, j )] R l l be the kernel matri and K i denote the ith column of K. ε-svr is based on ε-insensitive loss function defined b L ε () = { 0 if f () ε, f () ε otherwise, where f () and are the predicted value and true value of the input R n, respectivel. The regression function obtained b using kernel ε-svr has the form f () = w T φ() + b, where unknown normal vector w H and bias b R can be obtained b solving the following optimization problem: w,b 2 w 2 + C l (ξ i + ξ 2i ), i= s.t. w T φ( i ) + b i ε + ξ i, i =,, l, i (w T φ( i ) + b) ε + ξ 2i, i =,, l, ξ i, ξ 2i 0, i =,, l, (2) where C > 0 is a pre-specified value and ξ i, ξ 2i, i =,, l are slack variables. Due to H = span{φ( ),, φ( l )}, we can set w = l i= β i φ( i ). Put β = (β,, β l ) T, = (,, l ) T, e l = (,, ) T R l, ξ = (ξ,, ξ l ) T R l, ξ 2 = (ξ 2,, ξ 2l ) T R l, then w T φ( i ) = Ki T β and the problem (2) can be rewritten as the matri form: w,b 2 βt Kβ + Ce T l (ξ + ξ 2 ), s.t. Kβ + be l εe l + ξ, (Kβ + be l ) εe l + ξ 2, ξ, ξ 2 0. B solving the Wolfe dual form of the problem (3): α,α 2 2 (α α 2 ) T K(α α 2 ) T (α α 2 ) +εe T l (α + α 2 ) s.t. e T l (α α 2 ) = 0, 0 α, α 2 Ce l, we can obtain the optimal solution (α, α 2 ) and then or β = α α 2, b = i K T i β + ε for some i : 0 < α i < C, b = k K T k β ε for some k : 0 < α 2k < C. Therefore, the optimal regression function is f () = [k(, ),, k( l, )]β + b. (3) E-ISSN: Volume, 205

3 3 A numerical method based on Kernel ε-svr It is known that KSVRs based on optimization model are a powerful methodolog for solving function estimation problems and show better generalization abilit than other machine learning methods on a wide variet of real-world problems. But the main challenge in developing a useful regression model is to capture the underling functional relationship between the given inputs and their output values accuratel. This section tries to stud how to seek the numerical solutions of Eq.() b using kernel ε-svr. Specificall, it assumes that numerical solutions of Eq.() have the form ŷ() = w T φ() + b for a given kernel function k : R R R with RKHS H and feature mapping φ : R H and then learns unknown w H and b R b kernel ε-svr. For this end, the domain [a, b] of Eq.() needs to be discretized into a set of collocation points a = < 2 < < l = b with the same stepsize h = b a l such that µ = n (, l ), and then the points { i } l = are selected as input values. Since there are no available output values to learn (w, b) from Eq.(), we have to substitute the function ŷ() = w T φ() + b into Eq.(). So, it needs to define the derivative of the kernel function. Making use of the Mercer s Theorem [9], derivatives of the feature map can be written in terms of derivatives of the kernel function [20]. Define the following differential operator which will be used in subsequent sections, m n k(u, v) = n+m k(u, v) u n, u, v R, n, m =, 2,. vm B the definition of kernel function, it has and then m n k(u, v) = (φ (n) (u)) T φ (m) (v), ŷ() = w T φ() + b = l j= β j φ( j ) T φ() + b = l j= β j k( j, ) + b, ŷ () = w T φ () = l j= β j φ( j ) T φ () = l j= β j 0 k( j, ), ŷ () = w T φ () = l j= β j φ( j ) T φ () = l j= β j 2 0 k( j, ). In the sequel, put K 0 0 = [ 0 0 k( i, j )] = [k( i, j )] = K R l l, K m n = [ m n k( i, j )] = [ m n k(u, v) u=i,v= j ] R l l, and denote b (K m n ) i and K i the ith columns of the matrices K m n and K, respectivel. Let L(()) = () + p() () + q()(). If ŷ() is an eact solution of Eq.(), then ŷ(a) = p 0, ŷ(b) + λŷ(µ) = q 0 and L(ŷ( i )) = g( i ) for all i =,, l. But in general, ŷ(a), ŷ(b) + λŷ(µ) and L(ŷ( i )) are not necessaril equal to p 0, q 0 and g( i ), respectivel. So, we hope the smaller the better of the absolute values ŷ(a) p 0, ŷ(b) + λŷ(µ) q 0 and L(ŷ( i )) g( i ) for i =,, l. For this end, choose {g( i )} l i= as output values and construct the following optimization problem: w,b 2 w 2 + C l i= (ξ i + ξ 2i ), s.t. L(ŷ( i )) g( i ) ε + ξ i, ξ i 0, i =,, l, g( i ) L(ŷ( i )) ε + ξ 2i, ξ 2i 0, i =,, l, ŷ( ) = p 0, ŷ( l ) + λŷ( n ) = q 0, (4) where C > 0 is a pre-specified value and ξ i, ξ 2i, i =,, l are slack variables. In the problem (4), it assumes the boundar conditions being satisfied and hopes the most values of {L(ŷ( i ))} l i= satisfing L(ŷ( i)) g( i ) < ε for some given sufficientl small ε > 0, which indicates that the errors between eact values and estimated values at most of discrete points { i } l i= are smaller than ε. B discussion in the previous section, it knows that the numerical solution ŷ() = w T φ() + b is continuous if the kernel function k : R R R is continuous. According to the smbol propert of continuous functions, it can conclude that the error between eact solution and numerical solution in the whole domain [a, b] is smaller than ε as long as the stepsize h is taken sufficientl small. In general, ε is often taken as In this paper, take ε = 0 2. According to the discussion in Section 2, we can let w = l j= β j φ( j ) and then deduce that ŷ( i ) = l j= β j k( j, i ) + b = K T i β + b, ŷ( l ) + λŷ( n ) = (K l + λk n ) T β + ( + λ)b, ŷ ( i ) = (K 0 )T i β, ŷ ( i ) = (K 2 0 )T i β, L(ŷ( i )) = [(K 2 0 ) i + p( i )(K 0 ) i + q( i )K i ] T β + q( i )b, i =,, l. Put m i = (K 2 0 ) i + p( i )(K 0 ) i + q( i )K i R l, i =,, l, M = [m,, m l ] R l l, g = (g( ),, g( l )) T R l, q = (q( ),, q( l )) T R l, then the problem (4) can be rewritten as the matri E-ISSN: Volume, 205

4 form: β,b,ξ,ξ 2 βt Kβ + Ce T l (ξ + ξ 2 ) 2 s.t. M T β + qb g εe l + ξ, ξ 0, g M T β qb εe l + ξ 2, ξ 2 0, K T β + b = p 0, (K l + λk n ) T β + ( + λ)b = q 0. (5) Considering the Lagrange function of the problem (5) L(β, b, ξ, ξ 2, η, η 2, ς, ς 2, α, α 2 ) = 2 βt Kβ + Ce T l (ξ + ξ 2 ) +η T (MT β + qb g εe l ξ ) +η T 2 (g MT β qb εe l ξ 2 ) ς T ξ ς 2 T ξ 2 + α (K T β + b p 0) +α 2 ((K l + λk n ) T β + ( + λ)b q 0 ), where η, η 2, ς, ς 2 R l and α, α 2 R are the vectors of Lagrange multipliers, and letting L L ξ 2 = 0, it can deduce that β = L b = L ξ = Kβ + M(η η 2 ) + α K + α 2 (K l + λk n ) = 0, q T (η η 2 ) + α + ( + λ)α 2 = 0, Ce l η ς = 0 0 η Ce l, Ce l η 2 ς 2 = 0 0 η 2 Ce l. (6) Since the kernel matri K is smmetric and nonnegative definite, it can be assumed as a nonsingular matri without loss of generalit. Otherwise, to take care of problems due to possible ill-conditioning, a regularization term K + δi l can be introduced with a sufficientl small number δ > 0 and a l order unit matri I l. Put D = [M, M, K, K l + λk n ] R l (2l+2), γ = [η T, ηt 2, α, α 2 ] T R 2l+2, d = [q T, q T,, ( + λ)] T R 2l+2. It can get β = K Dγ from (6). Substituting (6) and β = K Dγ into the Lagrange function, it has where L(γ) = 2 γt D T K Dγ z T γ, z = ((εe l + g) T, (εe l g) T, p 0, q 0 ) T R 2l+2. Consequentl, the Wolfe dual form of the problem (5) can be written as γ 2 γt D T K Dγ + z T γ s.t. d T γ = 0, 0 γ j C, j =,, 2l. (7) After getting the optimal solution γ of the problem (7), it has β = K Dγ, b = p 0 K T β, and then the numerical solution of Eq.() is ŷ() = [k(, ),, k( l, )]β + b. The specific procedure is as follows. Algorithm. Step. Discrete the domain [a, b] b a = < 2 < < l = b with a sufficientl small same stepsize h = b a l such that µ = n (, l ). Step 2. Select a proper kernel function. Step 3. Choose proper kernel parameters and model parameters. Step 4. Solve the problem (7) and obtain the optimal solution γ. Step 5. Compute β = K Dγ and b = p 0 K T β. Step 6. construct the numerical solution of Eq.() b ŷ() = [k(, ),, k( l, )]β + b. 4 Numerical eamples In order to demonstrate the effectiveness of Algorithm, in this section, a series of comparative eperiments are performed between numerical solutions and eact solutions in the following si elaborated three-point BVPs of linear second-order ODEs with variable coefficients. The first five eamples come from a the second kind Fredholm integral equation [0] ϕ() 2 0 ( + )e ϕ()d = e e (+), 0, and have the same eact solution ϕ() = e. The sith eample is also taken from [0] and has the eact solution ϕ() = e. Eample. Consider a three-point BVP of linear { ϕ () + ϕ () + ϕ() = e, 0, ϕ(0) =, ϕ() + ϕ(/2) = e + e /2. Eample 2. Consider a three-point BVP of linear { ϕ () ( + sin )ϕ() = e sin, 0, ϕ(0) =, ϕ() + ϕ(/2) = e + e /2. Eample 3. Consider a three-point BVP of linear { ϕ () + 2 ϕ () ϕ() = ( 2 )e, 0, ϕ(0) =, ϕ() + ϕ(/2) = e + e /2. E-ISSN: Volume, 205

5 Eample 4. Consider a three-point BVP of linear { ϕ () + ( + )ϕ () = e, 0, ϕ(0) =, ϕ() + ϕ(/2) = e + e /2. Eample 5. Consider a three-point BVP of linear ϕ () + sin ϕ () + 2 ϕ() = ( sin + 2 )e, 0, ϕ(0) =, ϕ() + ϕ(/2) = e + e /2. Eample 6. Consider a three-point BVP for a { ϕ () ( + sin )ϕ() = e sin, [0, ], ϕ(0) =, ϕ() + ϕ(/2) = e + e /2. All computations are implemented in Matlab 200b on a PC with 2.5 GHz CPU and 4 G btes memor. The Gaussian RBF kernel function k(u, v) = ep{ (u v)2 2σ 2 }, u, v R is chosen in all eperiments and the model parameter C and kernel parameter σ are taken as C = and σ = 2. The eperiment results are listed in Table and the diagrams of comparisons are listed in Figure. Because most of known methods for solving differential equations numericall are focus on Talor epansion or discrete approaches, almost no use of optimization method, it can onl compare the errors between numerical solutions and eact solutions in this paper. From Table, it can see that the proposed numerical method has good accurac, which can be seen more obvious from Figure. According to the eperiment results, we can conclude that the proposed method is feasible and effective for solving numerical three-point BVPs of linear second-order ODEs with variable coefficients. 5 Conclusions This paper tries to seek the numerical solutions of three-point BVPs of linear second-order ODEs with variable coefficients b using optimization technolog. Although kernel ε-svr based on optimization model is a powerful methodolog for solving function estimation problems, the main challenge in developing a useful regression model is to capture the underling functional relationship between the given inputs and their output values accuratel. In order to construct optimization model corresponding to kernel ε-svr, this paper selects the discrete points of the domain of Eq.() as input values and selects values of the function L(ŷ()) at the discrete points as output values. B solving the Wolfe dual problem of the model, a numerical method is proposed. eperiment results indicates that the proposed method has a good approimation propert. From the derivation process, it can be seen that the proposed method has a certain versatilit and can be used to stud some other kinds of linear ODEs. The paper onl involves linear ODEs and nonlinear ODEs problems will be our net work. 6 Acknowledgements The research was supported b Science and Technolog Project of Higher Education of Shandong Province, P.R. China (grant No.L3LI0). References: [] F.V. Atkinson, Multiparameter Eigenvalue Problems, Academic Press, New York, 972. [2] A. Boucherif, Nonlinear three-point boundar value problems, J. Math. Anal. Appl. vol. 77, pp , 980. [3] C.Z. Bai, J.X. Fang, Eistence of multiple positive solutions for nonlinear m-point boundarvalue problems, Appl. Math. Comput. vol. 40, pp , [4] X.J. Liu, J.Q. Liu, Three positive solutions for second-order m-point boundar value problems, Appl. Math. Comput. vol. 56, no. 3, pp , [5] X. Xu, Multiplicit results for positive solutions of some semi-positone three-point boundar value problems, J. Math. Anal. Appl. vol. 29, pp , [6] X. Xu, Three solutions for three-point boundar value problems, Nonlinear Anal. vol. 62, pp , [7] H. Chen, B. Li, Three-point boundar value problems for second-order ordinar differential equations in Banach spaces, Comput. Math. Appl. vol. 56, pp , [8] P.D. Phung, L.X. Truong, On the eistence of a three point boundar value problem at resonance in Rn, J. Math. Anal. Appl. vol. 46, pp , 204. E-ISSN: Volume, 205

6 (a) Comparison for Eample. Eact solution Numerical solution Absolute error Table : Comparison for si eamples (b) Comparison for Eample 2. Eact solution Numerical solution Absolute error (c) Comparison for Eample 3. Eact solution Numerical solution Absolute error (e) Comparison for Eample 5. Eact solution Numerical solution Absolute error (d) Comparison for Eample 4. Eact solution Numerical solution Absolute error (f) Comparison for Eample 6. Eact solution Numerical solution Absolute error E-ISSN: Volume, 205

7 WSEAS TRANSACTIONS on SIGNAL PROCESSING eact solution eact solution. eact solution (a) Comparison for Eample (b) Comparison for Eample (c) Comparison for Eample 3. eact solution eact solution eact solution (d) Comparison for Eample (e) Comparison for Eample (f) Comparison for Eample 6. Figure : diagrams of comparisons for si eamples [9] Z. Zhao, Solutions and Greens functions for some linear second-order three-point boundar value problems, Comput. Math. Appl. vol. 56, pp. 04-3, [0] Xian-Ci Zhong, Qiong-Ao Huang, Approimate solution of three-point boundar value problems for second-order ordinar differential equations with variable coefficients, Applied Mathematics and Computation, vol. 247, pp. 8-29, 204. [] S. Mehrkanoon, T. Falck, J.A.K. Sukens, Approimate Solutions to Ordinar Differential Equations Using Least Squares Support Vector Machines, IEEE Transactions on Neural Networks and Learning Sstems, vol. 23, no. 9, pp , 202. [2] V. N.Vapnik, The Nature of Statistical Learning Theor, Springer, New York, [3] J. A. K. Sukens, J. Vandewalle, B. De Moor, Optimal control b least squares support vector machines, Neural Networks, vol. 4, no., pp , 200. [4] S. Chen, M. Wang, Seeking multi-thresholds directl from support vectors for image segmentation, Neurocomputing, vol. 67, pp , [5] Z. Liu, S. Xu, C. L. P. Chen, Y. Zhang, X. Chen, Y. Wang, A three-domain fuzz support vector regression for image denoising and eperimental studies. IEEE Transactions on Cbernetics, vol. 44, no. 4, pp , 204. [6] K. W. Lau, Q. H. Wu,. Local prediction of nonlinear time series using support vector regression, Pattern Recognition, vol. 4, no. 5, pp , [7] Y. Shang, Optimal control strategies for virus spreading in inhomogeneous epidemic dnamics. Canadian Mathematical Bulletin, vol. 56, no. 3, pp , 203. [8] A. K. Sharma, P. S. Parihar, An effective DoS prevention sstem to analsis and prediction of network traffic using support vector machine learning. International Journal of Application or Innovation in Engineering and Management, vol. 2, no. 7, pp , 203. [9] V. Vapnik, Statistical learning theor, New York: Wile, 998. [20] M. Lázaro, I. Santamaria, F. Pérez-Cruz, A. Artés-Rodriguez, Support vector regression for the simultaneous learning of a multivariate function and its derivative, Neurocomputing, vol. 69, pp. 42-6, E-ISSN: Volume, 205