Using collocation neural network to solve nonlinear singular differential equations

Transcription

1 216; 2(1): 5-56 ISSN: Maths 217; 2(1): Stats & Maths Receved: Accepted: Luma. N.M Tawfq Department of Mathematcs, College of Educaton Ibn Al- Hatham, Baghdad Unversty, Iraq Usng collocaton neural network to solve nonlnear sngular dfferental equatons Luma. N.M. Tawfq Abstract The am of ths paper s to desgn collocaton neural network to solve second order nonlnear boundary value problems n sngular ordnary dfferental equatons. The proposed network traned by back propagaton wth dfferent tranng algorthms quas-newton, Levenberg Marquardt, and Bayesan Regulaton, were the desgned network traned wth those algorthms usng the avalable expermental data as the tranng set and the proposed network contanng a sngle hdden layer of fve nodes. The next objectve of ths paper was to compare the performance of aforementoned algorthms wth regard to predctng ablty. Keywords: Ann, Back propagaton, Tranng Algorthm, ordnary dfferental equatons Correspondence Luma. N.M Tawfq Department of Mathematcs, College of Educaton Ibn Al- Hatham, Baghdad Unversty, Iraq 1. Introducton Many methods have been developed so far for solvng dfferental equatons, some of them produce a soluton n the form of an array that contans the value of the soluton at a selected group of ponts [1]. Others use bass functons to represent the soluton n analytc form and transform the orgnal problem usually to a system of algebrac equatons [2]. Most of the prevous study n solvng dfferental equatons usng artfcal neural network(ann) s restrcted to the case of solvng the systems of algebrac equatons whch result from the dscretzaton of the doman [3]. Most of the prevous works n solvng dfferental equatons usng neural networks s restrcted to the case of solvng the lnear systems of algebrac equatons whch result from the dscretzaton of the doman. The mnmzaton of the networks energy functon provdes the soluton to the system of equatons [4]. Lagars et al. [5] employed two networks: a multlayer perceptron and a radal bass functon network to solve partal dfferental equatons (PDE) wth boundary condtons (Drchlet or Neumann) defned on boundares wth the case of complex boundary geometry. Mc Fall et al. [6] compared weght reuse for two exstng methods of defnng the network error functon, weght reuse s shown to accelerate tranng of ODE, the second method outperforms the fals unpredctably when weght reuse s appled to accelerate soluton of the dffuson equaton. Tawfq [7] proposed a radal bass functon neural network (RBFNN) and Hopfeld neural network (unsupervsed tranng network) as a desgner network to solve ODE and PDE and compared between them. Malek et al. [8] reported a novel hybrd method based on optmzaton technques and neural networks methods for the soluton of hgh order ODE whch used three layered perceptron network. Akca et al. [9] dscussed dfferent approaches of usng wavelets n the soluton of boundary value problems (BVP) for ODE, also ntroduced convenent wavelet representatons for the dervatves for certan functons and dscussed wavelet network algorthm. Mc Fall [1] presented mult-layer perceptron networks to solve BVP of PDE for arbtrary rregular doman where he used logsg transfer functon n hdden layer and purelne n output layer and used gradent decent tranng algorthm, also, he used RBFNN for solvng ths problem and compared between them. Junad et al. [11] used Ann wth genetc tranng algorthm and log sgmod functon for solvng frst order ODE, Zahoor et al. [12] has been used an evolutonary technque for the soluton of nonlnear Rccat dfferental equatons of fractonal order and the learnng of the unknown parameters n neural network has been acheved wth hybrd ntellgent algorthms manly based on genetc algorthm (GA). ~5~

2 Abdul Samath et al. [13] suggested the soluton of the matrx Rccat dfferental equaton (MRDE) for nonlnear sngular system usng Ann. Ibraheem et al. [14] proposed shootng neural networks algorthm for solvng two pont second order BVP n ODEs whch reduced the equaton to the system of two equatons of frst order. Hoda et al. [4] descrbed a numercal soluton wth neural networks for solvng PDE, wth mxed boundary condtons. Majdzadeh [15] suggested a new approach for reducng the nverse problem for a doman to an equvalent problem n a varatonal settng usng radal bass functons neural network, also he used cascade feed forward to solve two - dmensonal Posson equaton wth back propagaton and levenberg - Marqwardt tran algorthm wth the archtecture three layers and 12 nput nodes, 18 tansg transfer functon n hdden layer, and 3 lnear nodes n output layer. Orab [16] desgned feed forward neural networks (FFNN) for solvng IVP of ODE. Al [17] desgn fast FFNN to solve two pont BVP. Ths paper proposed FFNN to solve two pont sngular boundary value problem (TPSBVP) wth back propagaton (BP) tranng algorthm. 2. Sngular Boundary Value Problem The general form of second order two pont boundary value problem (TPBVP) s: y + P(x)y + Q(x)y =, a x b (1) y(a) = A and y(b) = B, where A, B ϵ R there are two types of a pont x [,] : Ordnary Pont and Sngular Pont. A functon y(x) s analytc at x f t has a power seres expanson at x that converges to y(x) on an open nterval contanng x. A pont x s an ordnary pont of the ODE (1), f the functons P(x) and Q(x) are analytc at x. Otherwse x s a sngular pont of the ODE. On the other hand f P(x) or Q(x) are not analytc at x then x s sad to be a sngular pont [18, 19]. There s at present no theoretcal work justfyng numercal methods for solvng problems wth sngular ponts. The man practcal occurrence of such problems seems to be sem - analytc technque [2]. 3. Artfcal Neural Network Ann s a smplfed mathematcal model of the human bran, It can be mplemented by both electrc elements and computer software. It s a parallel dstrbuted processor wth large numbers of connectons, t s an nformaton processng system that has certan performance characters n common wth bologcal neural networks [21]. The arrvng sgnals, called nputs, multpled by the connecton weghts (adjusted) are frst summed (combned) and then passed through a transfer functon to produce the output for that neuron. The actvaton (transfer) functon acts on the weghted sum of the neuron s nputs and the most commonly used transfer functon s the sgmod functon (tansg.) [17]. There are two man connecton formulas (types): feedback (recurrent) and feed forward connecton. Feedback s one type of connecton where the output of one layer routes back to the nput of a prevous layer, or to same layer. Feed forward (FFNN) does not have a connecton back from the output to the nput neurons [22]. There are many dfferent tranng algorthms but the most often used s the Delta-rule or back propagaton (BP) rule. A neural network s traned to map a set of nput data by teratve adjustment of the weghts. Informaton from nputs s fed forward through the network to optmze the weghts between neurons. Optmzaton of the weghts s made by backward propagaton of the error durng tranng phase. The Ann reads the nput and output values n the tranng data set and changes the value of the weghted lnks to reduce the dfference between the predcted and target (observed) values. The error n predcton s mnmzed across many tranng cycles (teraton or epoch) untl network reaches specfed level of accuracy. A complete round of forward backward passes and weght adjustments usng all nput output pars n the data set s called an epoch or teraton. If a network s left to tran for too long, however, t wll be over traned and wll lose the ablty to generalze. In ths paper, we focused on the tranng stuaton known as supervsed tranng, n whch a set of nput/output data patterns s avalable. Thus, the Ann has to be traned to produce the desred output accordng to the examples. In order to perform a supervsed tranng we need a way of evaluatng the Ann output error between the actual and the expected output. A popular measure s the mean squared error (MSE) or root mean squared error (RMSE) [23]. 4. Descrpton of the Method In the proposed approach the model functon s expressed as the sum of two terms: the frst term satsfes the boundary condtons (BC) and contans no adjustable parameters. The second term can be found by usng Ann whch s traned so as to satsfy the dfferental equaton and such technque called collocaton neural network. In ths secton we wll llustrate how our approach can be used to fnd the approxmate soluton of the general form a 2 nd order TPSBVP: x m y"(x) = F( x, y(x), y'(x)) (2) where a subject to certan BC s and m Z, x R, D R denotes the doman and y(x) s the soluton to be computed. If y t(x, p) denotes a tral soluton wth adjustable parameters p, the problem s transformed to a dscretze form : Mn G(x, (x,p), (x,p), (x,p)) 2 t t t p x ˆ Mn p D F(x, y t(x,p), y t'(x,p)) (3) subject to the constrants mposed by the BC s. ~51~

3 In the our proposed approach, the tral soluton y t employs a Ann and the parameters p correspond to the weghts and bases of the neural archtecture. We choose a form for the tral functon y t(x) such that t satsfes the BC s. Ths s acheved by wrtng t as a sum of two terms: y t(x, p) = A(x) + G( x, N(x, p)) (4) where N(x, p) s a sngle-output Ann wth parameters p and n nput unts fed wth the nput vector x. The term A(x) contans no adjustable parameters and satsfes the BC s. The second term G s constructed so as not to contrbute to the BC s, snce y t(x) satsfy them. Ths term can be formed by usng a Ann whose weghts and bases are to be adjusted n order to deal wth the mnmzaton problem. 5. Computaton of the Gradent An effcent mnmzaton of (3) can be consdered as a procedure of tranng the network, where the error correspondng to each nput x s the value E(x ) whch has to forced near zero. Computaton of ths error value nvolves not only the Ann output but also the dervatves of the output wth respect to any of ts nputs. Therefore, n computng the gradent of the error wth respect to the network weghts consder a multlayer Ann wth n nput unts (where n s the dmensons of the doman ) one hdden layer wth H sgmod unts and a lnear output unt. For a gven nput x the output of the Ann s: N H 1 (z ), where z n j1 w x b j j w j denotes the weght connectng the nput unt j to the hdden unt v denotes the weght connectng the hdden unt to the output unt, b denotes the bas of hdden unt, and σ (z) s the sgmod transfer functon (tansg.). The gradent of Ann, wth respect to the parameters of the Ann can be easly obtaned as : N N b N w j (z ) (5) v (z ) (6) v (z ) x j (7) Once the dervatve of the error wth respect to the network parameters has been defned, then t s a straght forward to employ any mnmzaton technque. It must also be noted, the batch mode of weght updates may be employed. 6. Illustraton of the Method In ths secton we descrbe soluton of TPSBVP usng Ann. To llustrate the method, we wll consder the 2 nd order TPSBVP: x m d 2 y(x) / dx 2 f( x, y, y') (8) where x [a, b] and the BC: y(a) A, y(b) = B, ( Drchlet case ) or y (a) A, y (b) = B, ( Neumann case) or y(a) A, y(b) = B, (Mxed case) a tral soluton can be wrtten as: y t(x, p) (ba ab) / (b a) + (B A)x /(b a) + (x a)(x b)n(x, p) (9) where N(x, p) s the output of a FFNN wth one nput unt for x and weghts p. Note that y t(x) satsfes the BC by constructon. The error quantty to be mnmzed s gven by: n 2 d t(x ) f (x, t(x )) E[p] 1 dx d 2 y t(x, p) / dx 2 f(x, y t(x, p), dy t(x, p) / dx ) } 2 (1) ~52~

4 where the x [a, b]. Snce : dy t(x, p) /dx (B A)/(b a) + {(x a) + (x b)}n(x, p) + (x a) (x b) and dn(x,p) dx dn(x,p) dx d 2 y t(x, p) /dx 2 = 2N(x, p) + 2{(x a) + (x b)} + (x a) (x b) d 2 N(x, p) /dx 2 It s straghtforward to compute the gradent of the error wth respect to the parameters p usng (5) (7). The same holds for all subsequent model problems. 7. Example In ths secton we report numercal result, usng a mult-layer Ann havng one hdden layer wth 5 hdden unts (neurons) and one lnear output unt. The sgmod actvaton of each hdden unt s tansg, the analytc soluton y a(x) was known n advance. Therefore we test the accuracy of the obtaned solutons by computng the devaton: y(x) y t(x) y a(x). In order to llustrate the characterstcs of the solutons provded by the neural network method, we provde fgures dsplayng the correspondng devaton y(x) both at the few ponts (tranng ponts) that were used for tranng and at many other ponts (test ponts) of the doman of equaton. The latter knd of fgures are of major mportance snce they show the nterpolaton capabltes of the neural soluton whch to be superor compared to other soluton obtaned by usng other methods. Moreover, we can consder ponts outsde the tranng nterval n order to obtan an estmate of the extrapolaton performance of the obtaned numercal soluton. Example 1 Consder the followng 2 nd order TPSBVP: y = 5x3(5x5ey x 5) - ( 1 +1) y 4+x5 x wth BC: y (), y (1) = 1 and x [, 1]. Ths problem s an applcaton of oxygen dffuson wth the analytc soluton s: y a(x) - ln( x 5 + 4), accordng to (9) the tral neural form of the soluton s taken to be : y t(x) x + x (x - 1) N(x, p). The Ann traned usng a grd of ten equdstant ponts n [, 1]. Fgure(1) dsplay the analytc and neural solutons wth dfferent tranng algorthm. The neural results wth dfferent types of tranng algorthm such as: Levenberg Marquardt ( tranlm ), quas Newton ( tranbfg ), Bayesan Regulaton ( tranbr ) ntroduced n table (1) and ts errors gven n table (2), table (3) gves the performance of the tran wth epoch and tme and table(4) gves the weght and bas of the desgner network. Fg 1: analytc and neural soluton of example 1 usng: tranbfg, tranbr, and tranlm tranng algorthm. ~53~

5 Table 1: Analytc and Neural soluton of example 1 Input Analytc soluton Out of suggested FFNN yt(x) for dfferent tranng algorthm x ya(x) Tranbfg Tranbr Table 2: Accuracy of solutons for example 1 The error E(x) yt(x) ya(x) where yt(x) computed by the followng tranng algorthm t r a n l m t r a n b f g Tranbr e e e e e e e e e e e e e e e e e e e e e e e e e e e-8 Table 3: the performance of the tran wth epoch and tme Tran Fcn Tranbfg Tranbr Performance of tran 7.4e e e-13 Epoch Tme ::17 ::14 :1:31 MSE 1.e e-1 4.e-11 Table 4: Weght and bas of the network for dfferent tranng algorthm Weghts and bas for tranlm Net. IW{1,1} Net. LW{2,1} Net. B{1} Weghts and bas for tranbfg Net. IW{1,1} Net. LW{2,1} Net. B{1} Weghts and bas for tranbr Net. IW{1,1} Net. LW{2,1} Net. B{1} Rasheed [2] solved ths problem usng sem-analytc technque and gave the followng seres soluton: P 15 = x x x x x x x x x Abukhaled et al. n [24] applyng L'Hoptal's rule to overcome the sngularty at x =, then used the modfed splne approach and get maxmum error 7.79e -4 and resoluton ths problem usng fnte dfference method then gves maxmum error 1.46e -3. Example 2 Consder the followng 2 nd order TPSBVP: y + ( 1 x ) y + exp(y) = wth BC: y (), y(1) = and x [, 1]. The analytc soluton s: y a(x) 2ln( ) accordng to (9) the tral neural form of the soluton s : x2+1 y t(x) x (x - 1) N(x, p). The Ann traned usng a grd of ten equdstant ponts n [, 1]. Fgure (2) dsplay the analytc and neural solutons wth dfferent tranng algorthm. The neural results wth dfferent types of tranng algorthm such as : Levenberg Marquardt (tranlm), quas Newton ( tranbfg ), Bayesan Regulaton (tranbr) ntroduced n table (5) and ts errors gven n table (6), table(7) gves the performance of the tran wth epoch and tme and table(8) gves the weght and bas of the desgner network. ~54~

6 Fg 2: analytc and neural soluton of example 2 usng : tranbfg, tranbr, and tranlm tranng algorthm. Table 5: Analytc and Neural soluton of example 2 Input x Analytc soluton ya(x) Out of suggested FFNN yt(x) for dfferent tranng algorthm Tranbfg Tranbr e e-8 Table 6: Accuracy of solutons for example 2 The error E(x) yt(x) ya(x) where yt(x) computed by the followng tranng algorthm t r a n l m t r a n b f g tranbr e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e-8 Table 7: The performance of the tran wth epoch and tme TranFcn Tranbfg Tranbr Performance of tran 2.48e e e-13 Epoch Tme ::3 ::48 ::41 MSE 3.687e e e-11 Table 8: Weght and bas of the network for dfferent tranng algorthm Weghts and bas for tranlm Net.IW{1,1} Net.LW{2,1} Net.B{1} Weghts and bas for tranbfg Net.IW{1,1} Net.LW{2,1} Net.B{1} Weghts and bas for tranbr Net.IW{1,1} Net.LW{2,1} Net.B{1} ~55~

7 Ramos n [25] solved ths example usng C 1 - lnearzaton method and gave the absolute error e 3, also, Kumar n [26], solved ths example by the three-pont fnte dfference technque and gave the absolute error 2.e Concluson From the above problems t s clear that the network whch proposed can handle effectvely TPSBVP and provde accurate approxmate soluton throughout the whole doman and not only at the tranng ponts. As evdent from the tables, the results of proposed network are more precse as compared to method suggested n [2, 24, 25, 26]. In general, the practcal results on Ann show whch contan up to a few hundred weghts the Levenberg-Marquardt algorthm (tranlm) wll have the fastest convergence, then tranbfg and then tranbr. However, "tranbr" t does not perform well on functon approxmaton on problems. The performance of the varous algorthms can be affected by the accuracy requred of the approxmaton. 9. References 1. Mohammed KM. On Soluton of Two Pont Second Order Boundary Value Problems by Usng Sem-Analytc Method, MSc thess, Unversty of Baghdad, College of Educaton Ibn Al- Hatham, LeVeque RJ. Fnte Dfference Methods for Dfferental Equatons, Unversty of Washngton, AMath 585, Wnter Quarter, Agatonovc-Kustrn S, Beresford R. Basc concepts of artfcal neural network modelng and ts applcaton n pharmaceutcal, research. J. Pharm. Bomed. Anal., 2; 22: Hoda SA, Nagla HA. On Neural Network Methods for Mxed Boundary Value Problems, Internatonal Journal of Nonlnear Scence. 211; 11(3): Lagars IE, Lkas AC, Papageorgou DG. Neural-Network Methods for Boundary Value Problems wth Irregular Boundares, IEEE Transactons on Neural Networks. 2; 11(5): Mc Fall KS, Mahan JS. Investgaton of Weght Reuse n Mult-Layer Perceptron Networks for Acceleratng the Soluton of Dfferental Equatons, IEEE Internatonal Conference on Neural Networks. 24; 14: Tawfq LNM. Desgn and Tranng Artfcal Neural Networks for Solvng Dfferental Equatons, PhD thess, Unversty of Baghdad, College of Educaton Ibn Al- Hatham, Malek A, Bedokht RS. Numercal soluton for hgh order dfferental equatons usng a hybrd neural network-optmzaton method, Appled Mathematcs and Computaton. 26; 183: Akca H, Al-Lal MH, Covachev V. Survey on Wavelet Transform and Applcaton n ODE and Wavelet Networks, Advances n Dynamcal Systems and Applcatons. 26; 1(2): Mc Fall KS. An Artfcal Neural Network Method for Solvng Boundary Value Problems wth Arbtrary Irregular Boundares, PhD Thess, Georga Insttute of Technology, Junad A, Raja MAZ, Quresh IM. Evolutonary Computng Approach for The Soluton of Intal value Problems n Ordnary Dfferental Equatons, World Academy of Scence, Engneerng and Technology. 29; 55: Zahoor RMA, Khan JA, Quresh IM. Evolutonary Computaton Technque for Solvng Rccat Dfferental Equaton of Arbtrary Order, World Academy of Scence, Engneerng and Technology. 29; 58: Abdul Samath J, Kumar PS, Begum A. Soluton of Lnear Electrcal Crcut Problem usng Neural Networks, Internatonal Journal of Computer Applcatons. 21; 2(1): Ibraheem KI, Khalaf BM. Shootng Neural Networks Algorthm for Solvng Boundary Value Problems n ODEs, Applcatons and Appled Mathematcs: An Internatonal Journal. 211; 6(11): Majdzadeh K. Inverse Problem wth Respect to Doman and Artfcal Neural Network Algorthm for the Soluton, Mathematcal Problems n Engneerng. 211, Orab YA. Desgn Feed Forward Neural Networks For Solvng Ordnary Intal Value Problem, MSc Thess, Unversty of Baghdad, College of Educaton Ibn Al-Hatham, Al MH. Desgn Fast Feed Forward Neural Networks to Solve Two Pont Boundary Value Problems, MSc Thess, Unversty of Baghdad, College of Educaton Ibn Al-Hatham, Rachůnková I, Staněk S, Tvrdý M. Solvablty of Nonlnear Sngular Problems for Ordnary Dfferental Equatons, New York, USA, Shampne J. Kerzenka, Rechelt MW. Solvng Boundary Value Problems for Ordnary Dfferental Equatons n Matlab wth bvp4c, Rasheed HW. Effcent Sem-Analytc Technque for Solvng Second Order Sngular Ordnary Boundary Value Problems, MSc. Thess, Unversty of Baghdad, College of Educaton - Ibn - Al - Hatham, Galushkn IA. Neural Networks Theory, Berln Hedelberg, Mehrotra K, Mohan CK, Ranka S. Elements of Artfcal Neural Networks, Sprnger, Ghaffar A, Abdollah H, Khoshayand MR, Soltan Bozchaloo I, Dadgar A, Rafee-Tehran M. Performance comparson of neural network tranng algorthms n modelng of bmodal drug delvery, Internatonal Journal of Pharmaceutcs. 26; 327: Abukhaled M, Khur SA, Sayfy A. A Numercal Approach for Solvng a Class of Sngular Boundary Value Problems Arsng n Physology, Internatonal Journal of Numercal Analyss and Modelng. 211; 8(2): Ramos JI. Pecewse quaslnearzaton technques for sngular boundary value problems, Computer Physcs Communcatons. 24; 158: Kumar M. A three-pont fnte dfference method for a class of sngular two-pont boundary value problems, J Comput. Appl. Math. 22; 145: ~56~