Design Collocation Neural Network to Solve Singular Perturbed Problems with Initial Conditions

Transcription

1 Article International Journal of Modern Engineering Sciences, 204, 3(): International Journal of Modern Engineering Sciences Journal homepage: ISSN: Florida, USA Design Collocation Neural Network to Solve Singular Perturbed Problems with Initial Conditions Luma. N. M. Tawfiq* & K. M. M. Al-Abrahemee Department of Mathematics, College of Education for pure science / Ibn Al-Haitham, University of Baghdad * Author to whom correspondence should be addressed; drluma_m@yahoo.com Article history: Received 2 April 204, Received in revised form 5 May 204, Accepted 8 May 204, Published 20 May 204. Abstract: The aim of this paper is to design artificial neural networks to solve singular perturbation problems with initial conditions. We design a multi-layer collocation neural network having one hidden layer with 5 hidden units (neurons) and one linear output unit the sigmoid activation function of each hidden unit is ridge basis function where the network trained by back propagation with different training algorithms such as quasi-newton, Levenberg-Marquardt, and Bayesian Regulation. Finally the results of numerical experiments are compared with the exact solution in illustrative examples to confirm the accuracy and efficiency of the presented scheme. Keywords: Artificial neural network, back propagation training algorithm, singular perturbed problems.. Introduction Singularly perturbed problems (SPP) with initial conditions in ordinary differential equations (ODE) are characterized by the presence of a small parameter that multiplies the highest derivative. These problems are stiff and are characterized by an initial layer or inner solution which satisfies the initial condition and an outer solution which matches the inner one. These problems have been treated numerically by means of exponential-fitting, adaptive meshes, and ideas based on the method of matched Copyright 204 by Modern Scientific Press Company, Florida, USA

2 Int. J. Modern Eng. Sci. 204, 3(): asymptotic expansions [, 2]. Because of their stiff nature, the use of explicit numerical methods for SPP with initial conditions requires very small time steps because the time step is controlled by the eigenvalue whose magnitude is largest. On the other hand, the use of stable, implicit methods for nonlinear, SPP with initial conditions may require a large number of iterations for convergence within each time step. In addition, most of standard finite difference methods are not uniformly convergent in the small perturbation parameter [3]. Perturbation problems are problems which rely on there being a dimensionless parameter in the problem that is relatively small: <<. The most common example you may have seen before is that of high-reynolds number fluid mechanics, in which a viscous boundary layer is found close to a solid surface. Note that in this case the standard physical parameter Re is large: our small parameter is = Re [4]. This paper is organized as follows: the next section introduce definition of the singularly perturbed problems with initial conditions, in section three, we define neural network and its architect, section four, described the structure of neural network, in section five, the Illustration of the design network is introduced, in section eight, report our numerical finding accuracy of the suggested network, finally conclusions are introduced in the last part of the paper. 2. Singularly Perturbed Problems In this section we consider a system of ordinary differential equations (to gather with appropriate initial conditions) in which the highest derivative is multiplied by a small, positive parameter (usually denoted by ϵ). In what follows we give the general form of the 2 nd order singularly perturbed problems (SPPs) are: ε y = f(x, y, y ) a x b () with IC: y(a) = A, y (a) = A The above problem is called a singular perturbation problem (SPP) with initial condition, and is characterized by the fact that its order reduces when the problem parameterϵ equals zero. In such a situation, the problem becomes singular since, in general, not all of the original initial conditions can be satisfied by the reduced problem [5]. 3. Artificial Neural Networks An Artificial Neural Network (Ann) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this Copyright 204 by Modern Scientific Press Company, Florida, USA

3 Int. J. Modern Eng. Sci. 204, 3(): paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurons) working in unison to solve specific problems. Ann's, like people, learn by example. An Ann is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning in biological systems involves adjustments to the synaptic connections that exist between the neurons. This is true of Ann's as well. That is Ann's are relatively crude electronic models based on the neural structure of the brain. The brain basically learns from experience. It is natural proof that some problems that are beyond the scope of current computers are indeed solvable by small energy efficient packages. This brain modeling also promises a less technical way to develop machine solutions. This new approach to computing also provides a more graceful degradation during system overload than its more traditional counterparts. These biologically inspired methods of computing are thought to be the next major advancement in the computing industry. Even simple animal brains are capable of functions that are currently impossible for computers. Computers do rote things well, like keeping ledgers or performing complex math. But computers have trouble recognizing even simple patterns much less generalizing those patterns of the past into actions of the future [6]. Artificial neural networks characterized by [7]: I: Architecture: its pattern of connections between the neurons. II: Training Algorithm: its method of determining the weight on the connections. III: Activations function. In an Ann expressions structure, architecture or topology, express the way in which computational neurons are organized in the network. Particularly, these terms are focused in the description of how the nodes are connected and in how the information is transmitted through the network. As it has been mentioned, the distribution of computational in the following[8]: Number of layers: neurons in the neural network is done forming levels or layers of a determined number of nodes each one. As there are input, output and hidden neurons, we can talk about an input layer, an output layer and single layer or multilayer hidden layers. By the peculiarity of the behavior of the input nodes some authors consider just two kinds of layers in the ANN, the hidden and the output. Connection patterns: Depending on the links between the elements of the different layers. the ANN can be classified as: totally connected, when all the outputs from a level get to all and each one of the nodes in the following level, if some of the links in the network are lost, then we say that the network is partially connected. Information flow: Another classification of the ANN is obtained by considering the direction of the flow of the through the layers, when any output of the neurons is input of neurons of the same level or preceding levels, the network is described as feed forward. In counter position if there is at least one Copyright 204 by Modern Scientific Press Company, Florida, USA

4 Int. J. Modern Eng. Sci. 204, 3(): connected exit as entrance of neurons of previous levels or of the same level, including themselves, the network is denominated of feedback. 4. Description of the Structure Network In this section we will explain how this approach can be used to find the approximate solution the singular perturbation with initial conditions (SPIVPs), (2) where a subject to certain IC s D R denotes the domain and y(x) is the solution to be computed. If yt(x, p) denotes a trial solution with adjustable parameters p, the problem is transformed to a discrete form: (3) subject to the constraints imposed by the IC s. In the proposed approach, the trial solution yt employs a Ann and the parameters p corresponding to the weights and biases of the neural architecture. We choose a form for the trial function yt(x) such that it satisfies the IC s. This is achieved by writing it as a sum of two terms: (4) wheren(x, p) is a single-output Ann with parameters pand ninput units fed with the input vector x. The term (x) contains no adjustable parameters and satisfies the ICs, the second term Gis constructed so as not to contribute to the ICs, since y(x) satisfy them. This term can be formed by using an Ann whose weights and biases are to be adjusted in order to deal with the minimization problem. 5. Illustration of the Design Network In this section we describe solution of SPIVPs using ANN. To illustrate the method, we will consider the 2 nd order SPIVPs: with initial condition (5) where x [0,] and the IC: y(0) = A, y (0) = A ; a trial solution can be written as: y t (x, p) = A + A x + x 2 N(x, p) (6) Copyright 204 by Modern Scientific Press Company, Florida, USA

5 Int. J. Modern Eng. Sci. 204, 3(): wheren(x, p) is the output of an Ann with one input unit for xand weights p. Note that (x) satisfies the IC by construction.the error quantity to be minimized is given by where the xi [0, ]. Since, dy t (x, p) dx (7) = A dn(x, p) 2 + x + 2xN(x, p) dx d 2 y t (x,p) dx 2 = x 2 d2 N(x,p) dx 2 + 2N(x, p) + 4x dn(x,p) dx (8) It is straight forward to compute the gradient of the error with respect to the parameters p. 6. Numerical Results In this section we report some numerical result and the solution of number of model problem. In all cases we used a multi-layer Ann having one hidden layer with 5 hidden units (neurons) and one linear out output unit. The sigmoid activation of each hidden is tansig. (ridge basis function). For each test problem the exact analytic solution ya(x) were known in advance. Therefore we test the accuracy of obtained solutions computing the mean square error (MSE). Example Consider the following 2 nd order singular perturbed problem: 2 y + 3e x2 y + 4(2 x) = x 2 e x With I.C: y(0) =, y (0) =, x [0,] and the analytic solution is: y = x2 e x 2 x + 3e x 2e 2x According to the equation (6) the trial neural form of the solution is: y t (x) = + x + x2 N(x, p). The ANN trained using a grid of ten equidistant points in [0,], we get =0-4. Figure () display the analytic and neural solution with different training algorithms. The neural result with different types of training algorithms such as: Levenberg Marquardt (trainlm), quasi Newton ( trainbfg ), Bayesian Copyright 204 by Modern Scientific Press Company, Florida, USA

6 Int. J. Modern Eng. Sci. 204, 3(): Regulation (trainbr)are introduced in Table () and its errors are giving in Table (2), Table (3) gave the performance of the train for epoch and time, Table (4) gave the initial weight and bias of the design network. Figure : Analytic and neural solution of example using different training algorithm Table : Analytic and Neural solution of example Input x Analytic solution ya(x) Out of suggested Ann yt(x) for different training algorithm Copyright 204 by Modern Scientific Press Company, Florida, USA

7 Int. J. Modern Eng. Sci. 204, 3(): Table 2: Accuracy of the Solutions for example The error E(x) yt(x) ya(x) w h e r e yt(x) computed by the following training algorithm e e e e e e e e e e e e e e e e e e e-06 Table 3: The performance of the Train with epoch and time Train Function Performance of train Epoch Time MSE 4.2e-3 2 0:00: e e :00: e-05.87e :00: Table 4: Initial weight and bias of the network for different training algorithm Weights and bias for trainlm Net.IW{,} Net.LW{2,} Net.B{} Weights and bias for trainbfg Net.IW{,} Net.LW{2,} Net.B{} Weights and bias for trainbr Net.IW{,} Net.LW{2,} Net.B{} Copyright 204 by Modern Scientific Press Company, Florida, USA

8 Int. J. Modern Eng. Sci. 204, 3(): Example 2 Consider the following singular perturbed problem: y = y (y )cosx, x [0,] with IC: y(0) = 0.5, and the analytic solution: y = +e sinx According to the equation (6) the trial neural form of the solution is taken to be: y t (x) = x N(x, p) The Ann trained using a grid of ten equidistant points in [0, ], we get =0-3.Figure (2) display the analytic and neural solution with different training algorithms. The neural result with different types of training algorithms such as: Levenberg Marquardt (trainlm), quasi Newton ( trainbfg), Bayesian Regulation (trainbr) are introduced in Table (5) and its errors are giving in Table (6), Table (7) gave the performance of the train for epoch and time, Table (8) gave the initial weight and bias of the design network. Figure 2: Analytic and neural solution of example 2 using different training algorithms. Table 5: Analytic and Neural Solution of Example 2 Input x Analytic solution ya(x) Out of suggested Ann yt(x) for different training algorithm Copyright 204 by Modern Scientific Press Company, Florida, USA

9 Int. J. Modern Eng. Sci. 204, 3(): Table 6: Accuracy of solution for example 2 The error E(x) yt(x) ya(x) w h e r e yt(x) computed by the following training algorithm e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e-06 Train Function Table 7: The performance of the train with epoch and time Performance of train 2.52e-33.65e e-0 Epoch Time MSE 478 0:00: e :00: e :00: Table 8: Initial weight and bias of the network for different training algorithm Weights and bias for trainlm Net.LW{2, Net.B{} } Net.IW{,} Weights and bias for trainbfg Net.IW{,} Net.LW{2,} Net.B{} Weights and bias for trainbr Net.IW{,} Net.LW{2,} Net.B{} Copyright 204 by Modern Scientific Press Company, Florida, USA

10 Int. J. Modern Eng. Sci. 204, 3(): Conclusion This paper present new design of Ann to solve st and2 nd order nonlinear singular perturbed problems with initial condition, the suggested architecture of the network is efficient and more accurate than other numerical method and the practical results show which contain up to a few hundred weights the Levenberg-Marquardt algorithm (trainlm) will have the fastest convergence, then trainbfg and then trainbr. However, trainbr it does not perform well on function approximation on problems. The performance of the various algorithms can be affected by the accuracy required of the approximation. Reference [] OReilly, M.J., A uniform scheme for the singularly perturbed Riccati equation, Numer. Math. J.,50(987): [2] Salvekumar, K., A computational method for solving singular perturbation problems using exponentially fitted finite difference schemes, Appl. Math. Comput. J., 66(994): [3] Salvekumar, K., Optimal uniform finite difference schemes of order one for singularly perturbed Riccati equation, Commun. Numer. Methods Eng., 3(997): -2. [4] Mohan, K. K., Arora, P., Gupta, V., Collocation method using artificial viscosity for solving stiff singularly perturbed turning point problem having twin boundary layers, Computers and Mathematics with Applications, 6(20): [5] Abdelhameed, N. A., Numerical Solution of Stiff and Singularly Perturbed Problems for Ordinary Differential and Volterra-type Equations, Springer, 20. [6] Stergiou, C., and Siganos, D., Neural Networks, IEEE Neural Networks Council, [7] Hristev, R. M., The ANN Book, Edition, GNU Public License, 998. [8] Haykin, S., Neural networks: A comprehensive foundation, Edition, Springer, 993. Copyright 204 by Modern Scientific Press Company, Florida, USA