Improved Gene Expression Programming to Solve the Inverse Problem for Ordinary Differential Equations

Transcription

1 Improved Gene Expression Programming o Solve he Inverse Problem for Ordinary Differenial Equaions Kangshun Li a, Yan Chen a,, Wei Li a,b, Jun He c, Yu Xue d a Souh China Agriculural Universiy, College of Mahemaics and Informaics Guangzhou, Guangdong, , CN b Jiangxi Universiy of Science and Technology, School of Informaion Engineering, Ganzhou, , CN c Aberyswyh Universiy, Deparmen of Compuer Science Aberyswyh, Ceredigion, UK d Nanjing Universiy of Informaion Science and Technology, School of Compuer and Sofware, Nanjing, Jiangsu, CN Absrac Many complex sysems in he real world evolve wih ime. These dynamic sysems are ofen modeled by ordinary differenial equaions in mahemaics. The inverse problem of ordinary differenial equaions is o conver he observed daa of a physical sysem ino a mahemaical model in erms of ordinary differenial equaions. Then he model may be used o predic he fuure behavior of he physical sysem being modeled. Geneic programming has been aken as a solver of his inverse problem. Similar o geneic programming, gene expression programming could do he same job since i has a similar abiliy of esablishing he model of ordinary differenial sysems. Neverheless, such research is seldom sudied before. This paper is one of he firs aemps o apply gene expression programming for solving he inverse problem of ordinary differenial equaions. Based on a saisic observaion of radiional gene expression programming, an improvemen is made in our algorihm, ha is, geneic operaors should ac more ofen on he dominan par of genes han on he recessive par. This may help mainain populaion diversiy and also speed up he convergence of he algorihm. Experimens show ha his improved algorihm performs much beer han geneic programming and radiional gene expression programming in erms of running ime and predicion precision. Keywords: gene expression programming, sysem of ordinary differenial equaions, inverse problem, Runge-Kua algorihm 1. Inroducion There are many complex sysems or non-linear phenomena varying wih he ime in he real world. Such sysems are called dynamic sysems, including weaher change, populaion increase, disease diffusion and so on. In order o predic he developmen rend of such dynamic sysems, i is ofen required o esablish heir mahemaical models, ha is, o esablish he funcional relaionship or changing rend among variables of he sysems. I is difficul o find he funcional relaions among variables in complicaed changing processes, bu i is sill possible o find ou he change rae or differenial coefficiens of some variables, and hen o model hem by ordinary differenial equaions (ODEs). If here are more han one unknown funcions, we need o esablish a group of ordinary differenial equaions (ODEs). Through he ODEs model of a physical sysem, i is possible o learn he developmen rend of he sysem and apply he predicion in he real world. The problem of convering observed daa of a physical sysem ino a mahemaical model in erms of differenial equaions is known as he inverse problem [1, 2] of differenial equaions [3, 4, 5, 6]. For insance, if we have previous daa of a sock marke, we may creae an ODEs model for he sock marke using previous daa and hen predic he developmen rend of he sock marke. The inverse problem of Corresponding auhor address: cheny@scau.edu.cn (Yan Chen) Preprin submied o Journal of LATEX Templaes March 28, 2018

2 differenial equaions plays an imporan role in many areas from scienific experimens o sock markes. However, given observed daa, i is no an easy ask o creae models of ODEs for complex dynamical sysems, because hese problems are very complicaed and usually belong o non-linear sysems, so i is difficul o deermine he srucure of ODEs and parameers in ODEs in order o creae a correc model. In his paper, an improved Gene Expression Programming (GEP) is pu forward o solve he inverse problems of ordinary differenial equaions. GEP is a kind of evoluionary algorihms based on genome and phenomena and referred o he gene expression rule in he geneics [7, 8]. I inends o combine he advanages of boh GP and GA [9]. Unlike GP where an individual is expressed in he form of a ree, an individual in GEP is represened by he Isomeric linear symbols. GEP [10] has been successfully applied in problem solving [7], combinaorial opimizaion [11], real parameer opimizaion [12], evolving and modeling he funcional parameers [13], classificaion [14, 15], even selecion in high energy physics [16]. Choosing GEP is based on several reasons. Firs, an GEP algorihm adops a muli-gene srucure, where each gene sands for an ODE and each chromosome for a group of differenial equaions. This is differen from radiional algorihms in which an individual canno be used o represen a group of ODEs direcly. Secondly, previous experimens show ha GEP algorihms have a beer predicion effec in he shorer ime and is ime cos is so sable ha i is seldom influenced by he complexiy of dynamical sysems. In addiion, an improvemen is made in our GEP algorihm. I is more suiable for sudying he inverse problems of ordinary ODEs han he radiional ones because more geneic operaions are cenered a he dominan segmen of he gene and fewer geneic operaions are cenered a he recessive segmen[17]. The remainder of his paper is organized as follows: Secion 2 inroduces inverse problems of ODEs. Secion 3 presens an improved GEP algorihm for solving he inverse problems of ODEs. Secion 4 gives compuer experimen resuls. Secion 5 concludes he whole paper. 2. Inverse Problems for Ordinary Differenial Equaions A dynamic sysem is represened by n correlaed funcions: x 1 (), x 2 (),, x n () where denoes ime. The sysem has a series of observed daa colleced a imes j = 0 + j, (j = 0, 1,, m 1), where 0 represens he saring ime, he ime incremen, and x i ( j ) he observed value of x i a he ime j. Wrie he observaion daa in a marix form: Denoe X m := x 1 ( 0 ), x 2 ( 0 ),, x n ( 0 ) x 1 ( 1 ), x 2 ( 1 ),, x n ( 1 ),,,. (1) x 1 ( m 1 ), x 2 ( 0 ),, x n ( 0 ) x() := [x 1 (), x 2 (),..., x n ()] T, (2) f(x, ) := [f 1 (x, ), f 2 (x, ),..., f n (x, )] T (3) where f j (x, ) = f j (x 1 (), x 2 (),..., x n (), ) (j = 1, 2,..., n) is a composie funcion of several elemenary funcions involving of x i (i = 1,, n) and. Le F denoe he se of all possible composie funcions. A sysem of ordinary differenial equaions (ODEs) in he form of dx i () = f i (x 1,, x n, ), i = 1,, n (4) can be wrien in he vecor form dx() = f(x, ). (5) The goal of he inverse problem of ODEs is o find a mahemaical model which is represened by a sysem of ODEs dx () = f(x, ) (6) 2

3 such ha where he marix norm min{ X m X m ; f F} (7) X m X m := m 1 n (x i ( j) x i ( j )) 2 (8) j=0 i=1 The above he marix norm represens he difference beween he observed daa and he corresponding values derived from he ODEs model. Then we may use he obained ODEs (6) o predicae he fuure rend of he sysem. The above problem is called he inverse problem of ODEs. Differen approaches have applied o solving he inverse problem of ODEs. Linear modeling, such as Auoregressive model, Moving Average model, Auoregressive Moving Average model, are simple and popular [18, 19, 20]. However, here exis several resricions for linear models. Firsly, hey are linear models so ha hey can no represen non-linear dynamical sysems. Secondly, idenificaion and esimaion of linear models requires srong mahemaical knowledge and experise, which ofen lacks in pracice. Finally, once a model is esablished, i is no easy o consanly adjus he srucure and parameers of he model based on updaed observaion daa. Anoher simple modeling approach is o ake a form of differenial equaions which are pre-seleced by experience, and hen a numerical mehod is used o deermine he variables [21]. However, how o pre-selec he righ differenial equaion model is a difficul ask, especially for he differenial equaions whose number of variables increases. Evoluionary modeling [22, 23, 24, 25] has been successfully used in sudying he inverse problems of ordinary differenial equaions. Curren evoluionary modeling are mainly based on Geneic Programming [22, 25, 26], [27] where an equaion is represened in he form of ree. A hybrid evoluionary mehods are proposed for evolving ODEs, for example, o predic small-ime scale raffic measuremens daa in [28], which uses ree model o evolve bu is speed is also slow. ECSID [29] found good models for linear pendulum, non-linear pendulum wih fricion, coupled mass-spring, and linear circui. Bu he difference beween model found by ECSID and original model becomes large when he model becomes complex. GEP- SWPM is proposed in [30], and he predicion is based on several generaions of daa before, so i is seriously affeced by he noise. A new mehods of GEP was proposed in [13, 31], which is very effecive for idenifying parameer funcions. Bu i is based on he assumed model and doesn provide a common soluion and i can be exended o mos siuaions. In his paper, we proposed an improved GEP algorihm for solving he inverse problem of ODEs. 3. Improved GEP for he Inverse Problem of ODEs 3.1. Gene Represenaion in GEP Algorihm The geneic codes of GEP is he isomeric linear symbols (GEP chromosome). Each chromosome can be composed of several genes. GEP gene consiss of a head and a ail, where he former may conain boh he funcional symbols and erminaion symbols, while he laer only has he erminal symbols. For example, *+-aq*+aababb baab is a legal gene, of which, * sands for he muliplicaion operaion, Q he square roo operaion, he segmen wihou underline belongs o he head, while he underlined segmen is he ail. Figure 1 shows he expression of he gene in he form of a ree. For each problem, he lengh of he ail is a funcion of he lengh of he head h and he number of argumens of he funcion wih he mos argumens n, deermined using he following formula [32]. = h(n 1) + 1 (9) 3.2. The Flowchar of GEP Algorihm for Soling he Inverse Problem of ODEs GEP used o solve he inverse problem, is shown in Algorihm1. The deails of procedures are described. 3

4 Figure 1: Expression ree 1 Algorihm 1 Improved GEP algorihm for inverse problem of ODEs 1: Deermine he erminaion symbol se, operaor se and conrol parameers; 2: Iniialize a populaion; 3: while he erminaion requiremen is no me do 4: Conver he chromosome ino he expression ree (his sep migh be replaced by he GRCM algorihm inroduced laer); 5: Calculae he finess; 6: Apply geneic operaion; 7: Obain a new populaion; 8: end while 9: Oupu he opimal soluion; 3.3. Iniializaion The firs ask in he iniializaion is o se conrol parameers, including he lengh of a gene s head and ail, he number of genes, a erminaion symbol se and a funcional se. The erminaion se used for he inverse problem of ODEs is {, 1, 2, 3, } where i sands for x i. The funcional se is {+,,,, s, c, Q, e, ln}, where s = sin, c = cos, Q =, e = exp. These ses are deermined specifically for he inverse problem. The seings of he lengh of a gene s head and ail, and he number of he gene rely on he problem. In our experimens, he head lengh is se o 8. Since he maximal number of he operaors is 2, he ail lengh is se 9 according o (9). The second ask in he iniializaion is o creae an iniial populaion. A gene is generaed a random using he erminaion se and funcional se, subjec o he consrains on he head and ail lenghs of genes. A chromosome has k genes where k is fixed. For example, *+-1Q* is a gene, and he chromosome consiss of 3 genes: *+-1Q* *-*1+*+* *+*1Q* Then a number of chromosomes or individuals are generaed, where he number of chromosomes is called he populaion size Finess Evaluaion and Chromosomes Ranking The finess evaluaion of an individual is raher complex in he inverse problem of ODEs [33]. Firsly, we produce a sysem of ODEs or an ODEs model from each individual. Secondly, he model is used o produce predicion daa using Runge-Kua s mehod. Finally, he finess is calculaed by comparing predicion daa and acual daa. The deail is described as follows. Calculae genes The radiional GEP mehod is o conver a chromosome ino an expression ree, and hen solve i via sacks. This is a complicaed process. In his paper, we adop an alernaive mehod, called Gene Read 4

5 & Compue Machine (GRCM) algorihm [34] for calculaing genes. The soluions of a chromosome can be achieved direcly wihou convering he chromosome ino an expression ree. GRCM algorihm is described in Algorihm2. Algorihm 2 GRCM algorihm 1: Calculae he valid lengh of genes and deermine he valid geneic sequence; 2: Read he valid geneic lengh forward one by one from he las operaor unil he firs operaor is achieved; record is posiion P) 3: while he valid lengh of he gene is 1 do 4: if P arges o he unary operaor hen 5: Read he geneic sequence in valid lengh from he las characer and find ou a erminal symbol, which is used as a parameer for operaion, subsiue he above operaor wih he operaion resuls and conver he former operaor ino a erminal symbol; 6: When he valid lengh of gene is subraced by 1; 7: The posiion of P moves forward by 1; 8: else if P arges o he binary operaor hen 9: Read he geneic sequence in valid lengh from he las characer and find ou wo erminal symbols, which is used as a parameer for operaion, subsiue he above operaor wih he operaion resuls and conver he former operaor ino a erminal symbol; 10: When he valid lengh of gene is subraced by 2; 11: The posiion of P moves forward by 1; 12: else 13: No operaion is conduced; 14: Posiion P moves forward by 1; 15: end if 16: end while 17: reurn he value of he erminal symbol Comparing wih he radiional mehods, GRCM is easy for undersanding and convenien for operaion, wha s more, he operaion can be conduced a high speed. The more complicaed he chromosome is, he greaer advanages he algorihm has. Generae raining and predicion daa The Runge-Kua [35] is adoped in his paper o accomplish he predicion daa. The Runge-Kua is a ieraion mehod for simulaing he ODE soluions. A presen, he commonly used Runge-Kua is RK4, which is used in he condiion ha he differenial coefficien of equaion and he original value are known and omis he complicaed process of solving he differenial equaion via he compuer simulaion. The Runge-Kua can be used o achieve he precise soluion especially for he complicaed non-linear differenial equaion group and he non-linear ordinary differenial equaion group ha is oo complicae o obain he precise soluion. For he ordinary differenial equaion wih one variable x() dx() = f(x(), ), (10) 5

6 he RK4 formula is shown in (11). x( 0 ) is he original value K 1 = f(x( j ), j ) K 2 = f(x( j ) K 1, j ) K 3 = f(x( j ) K 2, j ) K 4 = f(x( j ) + K 3, j + ) x( j+1 ) = x( j ) (K 1 + 2K 2 + 2K 3 + K 4 ) (11) where = j+1 j. For he ordinary differenial equaion wih n variables x 1 (),, x n (), dx i () = f i (x 1 (),, x n (), ), i = 1,, n (12) he RK4 formula is shown in (13). x 1 ( 0 ), x 2 ( 0 ),, x n ( 0 ) are orignal values, K i,1 = f i (x 1 ( j ), x 2 ( j ),, x n ( j ), j ), K i,2 = f i (x 1 ( j ) K 1,1, x 2 ( j ) K 2,1,, x n ( j ) K n,1, j ) K i,3 = f i (x 1 ( j ) K 1,2, x 2 ( j ) K 2,2,, x n ( j ) K n,2, j ) K i,4 = f i (x 1 ( j ) + K 1,3, x 2 ( j ) + K 2,3,, x n ( j ) + K n,3, j + ) x i ( j+1 ) = x i ( j ) (K i,1 + 2K i,2 + 2K i,3 + K i,4 ) i = 1, 2,, n The calculaion is as follows: firs, achieve he marix X from he observed daa: x 1 ( 0 ) x 2 ( 0 ) x n ( 0 ) X = x 1 ( 1 ) x 2 ( 1 ) x n ( 1 ) (14) x 1 ( m 1 ) x 2 ( m 1 ) x n ( m 1 ) Then, selec he daa a firs line of he marix X as he original values of RK4, and calculae he predicion daa X from he achieved model via he RK4 mehod as per he (14) by adoping he incremen same wih ha of marix X, see (15). X = x 1 ( 0 ) x 2 ( 0 ) x n ( 0 ) x 1( 1 ) x 2( 1 ) x n( 1 ) x 1( m 1 ) x 2( m 1 ) x n( m 1 ) (13) (15) The firs row of marix X are he original values of RK4 and hey are same as he daa a firs line of marix X. The remaining daa of marix X are generaed using (13). Consrucion of finess funcion Consrucion of finess funcion is o ensure he algorihm o evolve in he required direcion; i is he major drive for he evolvemen of GEP groups. Differen finess funcion has differen influences on he evolvemen qualiy. The beer he finess funcion is, he individual is more adapable o he environmen 6

7 and he greaer probabiliy ha he individual evolves o he nex generaion [36]. The finess funcion in his paper is consruced by he difference beween marix X and X, i.e. = X X. is he difference of he wo marixes, and he smaller i is, he beer i is. Therefore, he greaer he finess is, he beer i is afer he conversion as per (16). finess = (16) 3.5. Geneic Operaors Selecion The Roulee selecion is adoped in he paper. The beer he finess, he greaer probabiliy an individual is reproduced o he nex generaion. The reproducion imes will be deermined as per he Roulee principle in he process and meanwhile he populaion size is unchanged. Muaion operaor design The muaion operaion can be performed a any posiion of he chromosome and, according o relevan sipulaions, he operaor a he head can be muaed ino any funcion or erminal symbol, bu ha a he ail can only be muaed ino he erminal symbol, such muaion can ensure he newly generaed chromosome is in he valid srucure. In accordance wih relevan requiremens, he muaion probabiliies se in his paper are 0.044, which is used for performing he muaion operaion for he chromosome. Transposiion operaor design The ransposiion operaion are performed as per he se ransposiion probabiliy and lengh, here are hree kinds of ransposiion operaions in his paper. In addiion, he ransposiion probabiliy is se as 0.1 and lengh is 5 as per he relevan experience. Inserion Sequence Transposiion: The conversion segmen is seleced randomly from he chromosome and he segmen can be insered a any posiion of he head excep for he original posiion. Roo Inserion Sequence Transposiion: I is similar o he IS ransposiion, bu is conversion segmen can only be insered a he original posiion of he gene. Gene Transposiion: The complee gene is seleced as he conversion segmen and insered a he original posiion of he chromosome; however, he seleced gene will be deleed in he new chromosome [32, 37]. Resrucuring operaor design There are also hree kinds of resrucuring operaors in his paper. They are single-poin resrucuring, double-poin resrucuring and gene resrucuring. Single-poin resrucuring operaor: This is similar o he single poin muaion in GA, in his paper, we randomly seleced an exchange poin for wo chromosomes, and hen exchanged he chromosome segmen behind he poin. Double-poin resrucuring operaor: This is similar o he double-poin muaion in GA, we randomly seleced wo exchange poins for wo chromosomes, and hen exchange he wo srings beween he exchange poins. Gene resrucuring operaor: This operaor only acs on chromosomes of muliple genes. We randomly seleced a gene for wo polygene chromosomes, and hen exchanged he wo chromosome of he corresponding genes. This is similar o he double-poin resrucuring operaor, excep ha he swap subsring mus be a complee gene. Selec randomly he resrucuring segmen or gene from he paren chromosome as per he specific resrucuring operaor, and hen exchange he seleced segmens or genes. Care shall be aken ha he gene resrucuring operaor canno generae he new gene: he newly-generaed individual is he array or combinaion of he exising gene. The probabiliies for single-poin resrucuring, double-poin resrucuring and gene resrucuring se in his paper are 0.3, 0.3 and 0.1 respecively. 7

8 3.6. Terminaion condiions The algorihm will sop once any of he following requiremens is me. The maximum number of generaions is reached; The finess of he bes individual reaches he a predefined value, or i is unchanged for a predefined number of generaions Improvemen of GEP algorihm A GEP gene has coding regions and noncoding regions. For example, consider he gene Q*+*a*Qaababbaababa ab, he par wihou underline is is Head while he underlined segmen is he ail. Figure 2 illusraes he expression ree associaed wih he gene. Figure 2: Expression ree 2 From he expression ree in Figure 2, we see ha he las 10 characers of he gene are noncoding regions. This is he reason why GEP is a non-linear segmen and is phenoype form and genoype form of he genes may be differen. Obviously, only when he muaion, ransposiion and resrucuring operaors are applied ino he dominan segmen of he gene which can affec he finess direcly, hey increase he diversiy of he populaion remarkably. If hey are applied ino he recessive segmen which can no affec he finess direcly, hey can only increase he probabiliy for populaion diversiy before performing he muaion operaor. For he four sandard geneic operaors: muaion, IS ransposiion, single-poin resrucuring and double-poin resrucuring, all characers have he equal probabiliy o become he posiion of he operaor, herefore, he righ selecion of he posiion of hese characers has a grea impac on he validiy of he geneic operaors. To verify he above saemen, we demonsrae he average valid lengh of GEP genes used in recen references by Table 1. 8

9 Reference Head lengh Table 1: The valid lengh and he average uilizaion rae of gene zero-param funcion one-param funcion wo-param funcion valid lengh of gene lengh of gene Average use raio of gene [38] % [38] % [38] % [39] % [40] % [41] % [42] % [43] % [44] % [44] % [44] % [45] % [45] % From he umos righ column of Table 1, we see ha he average uilizaion rae of gene is very low, especially for a funcion wihou parameer. The greaer proporion of he funcion wihou parameer is, he lower he average uilizaion rae of gene is. In his case, geneic operaors wih varying posiions may ake place a he recessive segmen of he gene wih grea probabiliy even hough he occurrence probabiliy is me. I will grealy reduce he validiy of geneic operaors. Therefore, we propose ha mos of geneic operaors can be cenered a he dominan segmen. This may help mainain populaion diversiy and also increase he convergence rae of GEP. The seing of he occurrence probabiliies of he dominan and recessive pars of he gene is deermined by he problem. In he paper, his probabiliy is assigned o 0.8, ha is, 80% of geneic operaion occurs a he dominan par of he gene and he remaining 20% a he recessive segmen. 4. Compuer Experimens 4.1. Seings of Experimenal Parameers The parameers used in he experimens are lised as follows: Se of erminaion symbols: I-II:{1,2,3,0}, III:{1,0}, IV:{1,2,0} Se of funcional symbols: {+,,,, sin, cos, Q, exp, ln} Number of ieraions: Size of Populaions: 50 Number of gene: 3 Head lengh of gene: 8 Selecion algorihm: Roulee Probabiliy of gene resrucuring: 0.1 Probabiliy of muaion: Probabiliy of IS ransposiion: 0.1 9

10 Lengh of IS ransposiion : 5 Probabiliy of RIS ransposiion: 0.1 Lengh of RIS ransposiion: 5 Lengh of Gene Transposiion: 0.1 Probabiliy of single-poin resrucuring: 0.3 Probabiliy of double-poin resrucuring: Experimen 1 We searched carefully wih oher sae of he ar solvers he inverse problem of ordinary differenial equaions including based he proposed mehod. There is only one lieraure [46] o discuss his inverse problem of ordinary differenial equaions. The raining daa se in experimen 1 is chosen from [46] which is consruced from he soluion (18) o he ODEs (17). dx 1 = x 1 dx 2 = x 1 + x 2 + x 3 (17) dx 3 = 2x 1 x 2 + 3x 3 x 1 = e x 2 = e 2 ( + 1) (18) x 3 = e 2 ( + 2) e Le 0 = 0, = 0.01, hen we ge daa I described in Table 2. The par X m is used as he observaion daa, and he par X p as he predicion daa for evaluae he qualiy of he model. For he daase lised in Table 2, we performed 100 experimens and choose he bes ODEs model produced in he experimens. Then, Runge-Kua ieraion is performed on he ODEs. The resuls of he bes model obained by using GEP is lised in Table 3. E m represens he error of he ODE model on he raining daa and E p he error of he ODE model under modelling on he predicion daa. Table 2: Daase I where 0 = 0, = 0.01 Observed daa Modeling daa x 1 () x 2 () x 3 () x 1 () x 2 () x 3 () X m X p

11 Table 3: Resul I: E m and E p represen he error analysis resuls of comparison beween improved GEP and Kang. e al. GP Improved GEP Kang. e al. GP x 1 () x 2 () x 3 () x 1 () x 2 () x 3 () E m E p Regression sandard error Predicion sandard error The experimen resuls in Table 3 demonsraes ha he error of improved GEP has smaller han he Kang. e al. GP. This indicaes he effeciveness of he improved GEP algorihm for solving he inverse problem of differenial equaions Experimen 2 In experimen 2, he raining daa se is sill consruced from he soluion (18) o he ODEs (17). Se 0 = 0 and = 0.05, hen we ge daa II described in Table 4. The par X m is aken as observaion daa, and he par X p as he predicion daa for evaluaing he soluion qualiy of he model. Table 4: experimenal daa II where 0 = 0, = 0.05 Observed daa Modeling daa x 1 () x 2 () x 3 () x 1 () x 2 () x 3 () X m X p For he daase lised in Table 4, we performed 100 experimens and choose he bes ODEs model generaed in he experimens. Table 5 gives he error beween he predicion by he bes model and he soluion o he original ODEs. E m and E p represen he error on raining daa and predicion daa respecively. 11

12 Table 5: resul II: E m and E p represen he error analysis resuls of comparison beween improved GEP and Kang. e al. GP Improved GEP Kang. e al. GP x 1 () x 2 () x 3 () x 1 () x 2 () x 3 () E m E p Regression sandard error Predicion sandard error Experimen resuls in Table 5 demonsraes ha predicion provided by he model show good agreemen wih he original ODEs. I indicaes ha he improved GEP algorihm is applicable for solving he inverse problem of ordinary differenial equaions. As can be seen from Table 2 o Table 5, mos of he resuls are beer han he Kang. e al. GP algorihm, excep for individual modeling resuls and predicive resuls. In addiion, due o explore he improved GEP algorihm can ransfer complex nonlinear curve ino a linear problem, herefore, he use of improved GEP algorihm for ordinary differenial equaion have he more advanage of making he convergence faser han Kang. e al. GP algorihm in erms of compuaional complexiy Experimen 3 The raining daa III, given in Table 6, come from [47]. The par X m is aken as observaion daa, and he par X p as he predicion daa for evaluaing he qualiy of he model found by he improved GEP algorihm. Table 6: experimenal daa III where 0 = 0, = 0.01 III x() x() X m X p For he daase lised in Table 6, we performed 100 experimens and choose he bes model for predicion. The resuls of he bes models obained by using GEP are lised in Table 7. 12

13 Table 7: resul III: E m and E p represen he error of he ODE model and sysem under modeling on raining daa and predicion daa respecively { dx Model = (xx)+((x+) ) III x() x() E m E p Training sandard error Predicion sandard error The experimen resuls in Table 7 demonsraes ha he daase can predic he original ordinary differenial equaions by using he improved GEP algorihm, which indicaes ha i is feasible o use he improved GEP algorihm for he inverse problem of differenial equaions Experimen 4 The raining daa IV, given in Table 8, come from [47]. The par X m is aken as observaion daa, and he par X p as he predicion daa for evaluae he qualiy of he model. Table 8: experimenal daa IV where 0 = 0, = 0.01 IV x 1 () x 2 () X m X p For he daase lised in Table 8, we performed 100 experimens obain he beer model. The resuls of he bes models obained by using GEP are lised in Table 9. The experimen resuls in Table 9 demonsraes ha he daase can predic he original ordinary differenial equaions by using he improved GEP algorihm, which indicaes ha i is feasible o use he improved GEP algorihm for he inverse problem of differenial equaions A Comparison of GP, Sandard GEP and Improved GEP for he Inverse Problem of ODEs To verify he advanages, disadvanages and exising problems of applying he GEP algorihm o he inverse problem of ordinary differenial equaions, we also conduced a comparaive experimen. A presen, sae-of-he-ar of he inverse problems using ordinary differenial equaions are solved by mahemaical analysis. Furhermore, he limiaions of inverse problems and complex differenial equaions are difficul o consruc. There are differen mehod of analyic soluion in mahemaics. The mehod of arificial inelligence is applied o he inverse problem of ordinary differenial equaions in his paper. Therefore, he 13

14 Table 9: resul IV: E m and E p represen he error of he ODE model and sysem under modeling on raining daa and predicion daa respecively. { dx1 Model = ((x 2 x 1 ) + x 1 )(x 1 /x 2 ) x 2 ( + x 1 ) dx 2 = ((x 2 x 1 ) (x 2 + x 2 ))/ sin(x 2 /x 1 ) + x 1 IV x 1 () x 2 () E m E p Training sandard error Predicion sandard error comparison experimen in his paper only chooses wo kinds of algorihms of he same mehod, and analyzes he experimenal resuls wih his improved algorihm. All algorihms in his sudy in 100 experimen runs using he four daases. The experimen resuls were compared wih he GP, he sandard GP and he improved GEP. The saisical errors lised in Table 10 were obained. Table 10: The resul of 100 experimen runs using he four daases for GP, Sandard GEP and Improved GEP Daa GP Sandard GEP Improved GEP Training I Sandard II Deviaion III IV Predicion I Sandard II Deviaion III IV I Running II Time III IV From Table 10 we can observe ha among he average running imes (in seconds) of he four daases, he sandard deviaion using sandard GEP algorihm are significanly less han hose for he GP algorihm. As he sandard deviaion when using he sandard GEP algorihm are smaller han when using he GP algorihm, which indicaes ha i is excellen o apply he sandard GEP algorihm o ordinary differenial equaions and ha is predicion effecs and compuaion efficiency are beer han he GP algorihm. In addiion, i can be seen from Table 10 ha, in differen daa ses, GP algorihm s running ime flucuae a lo, on he oher hand, GEP algorihm s running ime flucuae a few. Table 10 shows ha for hese four daases, he minimum predicion errors generaed by he improved GEP algorihm were smaller han hose generaed by he sandard GEP algorihm and ha he maximum predicion errors generaed by he improved GEP algorihm were also smaller han hose generaed by he sandard GEP algorihm. Thus, he improvemen of he sandard GEP algorihm was no only effecive 14

15 bu also overcame he problems of insabiliy of he sandard GEP algorihm. In erms of average running ime (in seconds), here is no significan difference beween he improved GEP algorihm and he sandard GEP algorihm. However, he running ime is decreased. Therefore, in erms of running efficiencies, he improved GEP algorihm is beer han he GP algorihm Summary of Experimen Resuls The described experimen resuls can be summarized as follows: For each daase, using he GEP algorihm o solve he inverse problem of ordinary differenial equaions are efficien, and is able o produce a beer model. GEP is effecive improvemen sraegies. In erms of running ime, here was no significan difference beween he improved GEP and he sandard GEP. However, he running ime of GEP is significanly less han ha of he GP algorihm. In erms of he sabiliy, adoping he improved GEP algorihm o solve he inverse problem of ordinary differenial equaions is beer han using he GP algorihm, paricularly for complex problems. In erms of predicion accuracy, he improved GEP algorihm are beer han sandard GEP on he error of he model, while he GEP algorihm is superior o he GP algorihm on he sandard deviaion. As he consan iems increase, he use of GEP algorihm is effecive, and he overall effec is sill beer han he GP algorihm. 5. Conclusions In his sudy, we proposed a GEP based algorihm o solve he inverse problem of ordinary differenial equaions, and GEP algorihm s shorcoming, evoluion operaion in he recessive segmen conribues lile is overcome. In view of he effeciveness of he improved GEP algorihm, his paper mainly discusses is applicaion o pracical problems, alhough he effec is good, bu here is no heoreical issues in-deph discussion, so many problems need furher sudy. Through experimens, we verified ha he algorihm, in erms of solving he inverse problem of ordinary differenial equaions, was very effecive wih respec o raining sandard deviaion, predicion sandard deviaion and running ime. And improvemen sraegies can furher enhance he efficiency of he algorihm convergence. The work of his paper is only a sar, he problem of solving despie he engineering background, bu he pracical applicaion of he projec here is sill a grea disance. Our sudy on algorihms provides a powerful ool o auomae he process of knowledge discovery for dynamic daa, and we expec he approach o be promoed and applied in acual problems relaed o ime series and ime fields, such as weaher forecasing, marke predicion and ecological predicion. Acknowledgemens This work is suppored by he Naional Naural Science Foundaion of China wih he Gran No , he Science and Technology Planning Projec of Guangdong Province wih he Gran No. 2017A , he Fund of Naural Science Foundaion of Guangdong Province of China wih he Gran No. 2014A , he Science and Technology Planning Projec of Guangdong Province of China wih he Gran No. 2015A , he Science and Technology Research Projec of Jiangxi Province wih he Gran No. GJJ160631, he Science Foundaion of Jiangxi Universiy of Science and Technology under he gran No. NSFJ2015-K13. 15

16 6. References [1] R. O. Alan L. Jennings, Opimal inverse funcions creaed via populaion-based opimizaion, Cyberneics, IEEE Transacions on 44 (6) (2014) [2] F. O.-T. Riemann Ruiz-Cruz, Edgar N. Sanchez, Paricle swarm opimizaion for discree-ime inverse opimal conrol of a doubly fed inducion generaor, Cyberneics, IEEE Transacions on 43 (6) (2013) [3] V.G.Romanov, Inverse Problems of Mahemaical Physis, Neherlands, VNU Science Press BV, [4] C. W. Groesch, Inverse problems in he mahemaical sciences, Vol. 52, Springer, [5] V. Isakov, Inverse problems for parial differenial equaions, Vol. 127, Springer, [6] J. L. Guofeng Feng, Bo Han, Widely convergen generalized pulse-specrum mehods for 2-d wave equaion inversion, Chinese Journal of Geophysics 46 (2) (2003) [7] C. Ferreira, Gene expression programming: A new adapive algorihm for solving problems, Complex Sysems 13 (2) (2001) [8] C. Ferreira, Gene Expression Programming: Mahemaical Modeling by an Arificial Inelligence, Springer-Verlag, [9] Q. S. X. D. Jun Li, Mengchu Zhou, Colored raveling salesman problem, Cyberneics, IEEE Transacions on PP (99) (2014) 1. [10] M. J. K. C. T. Su Nguyen, Mengjie Zhang, Auomaic programming via ieraed local search for dynamic job shop scheduling, Cyberneics, IEEE Transacions on 45 (1) (2014) [11] C. Ferreira, Combinaorial opimizaion by gene expression programming: inversion revisied, in: Proceedings of he Argenine symposium on Arificial Inelligence, 2002, pp [12] R. T. Kaikuo Xu, Yinian Liu, A novel mehod for real parameer opimizaion based on gene expression programming, Applied Sof Compuing 9 (2) (2009) [13] L. K. Dazhi Jiang, Zhijian Wu, Parameer idenificaions in differenial equaions by gene expression programming, in: Naural Compuaion, ICNC Third Inernaional Conference on, Vol. 5, IEEE, 2007, pp [14] X. Wen, L. Shao, Y. Xue, W. Fang, A rapid learning algorihm for vehicle classificaion, Elsevier, Informaion Sciences 295 (1) (2015) [15] J. S. Z. F. X. L. N. L. Qi Liu, Weidong Cai, A speculaive approach o spaial-emporal efficiency wih muli-objecive opimizaion in a heerogeneous cloud environmen, Wiley Online Library, Securiy and Communicaion Nework 9 (17) (2016) [16] L.Teodorescu, Gene expression programming approach o even selecion in high energy physics, Nuclear Science, IEEE Transacions on 53 (4) (2006) [17] D. X. Q. W. H. Z. Yuhui Zheng, Byeungwoo Jeon, Image segmenaion by generalized hierarchical fuzzy c-means algorihm, IOS Press, Journal of Inelligen & Fuzzy Sysems 28 (2) (2015) [18] T. C. Mills, Time Series Techniques for Economiss, Cambridge Universiy Press, [19] D. B. Percival, A. T. Walden., Specral Analysis for Physical Applicaions, Cambridge Universiy Press, [20] S. W. Sudhakar M.Pandi, Time Series and Sysem Analysis wih Applicaions, John Wiley and Sons, [21] W.F.Lucas, Differenial Equaion Models, New York: Springer-Verlag, [22] Y. C. Hongqing Cao, Lishan Kang, The evoluionary modeling of higher-order ordinary differenial equaions for ime-series analysis, Mini-Micro Sysem (4) (2000) [23] H. L. Theodore W.Cornforh, Inference of hidden variables in sysems of differenial equaions wih geneic programming, Geneic Programming and Evolvable Machines 14 (2) (2013) [24] Y. Ikeda, Esimaion of chaoic ordinary differenial equaions by coevoluional geneic programming, Elecronics and Communicaions in Japan (Par III Fundamenal Elecronic Science) 86 (2) (2003) [25] T. G. Y. C. Hongqing Cao, Lishan Kang, A wo-level hybrid evoluionary algorihm for modeling one-dimensional dynamic sysems by higher-order ode models, IEEE Transacions on Sysems, Man, and Cyberneics, Par B: Cyberneics 30 (2) (2000) [26] Y. Zhang, X. Sun, B. Wang, Efficien algorihm for k-barrier coverage based on ineger linear programming, IEEE, China Communicaions (7) (2016) [27] L. K. Aimin Zhou, Hongqing Cao, The auomaic modeling of complex funcions based on geneic programming, Journal of Sysem Simulaion (6) (2003) [28] Q. M. Y. Z. A. A. Yuehui Chen, Bin Yang, Time-series forecasing using a sysem of ordinary differenial equaions, Informaion Sciences 181 (2011) [29] M. G. Juan J. Flores, Sysem idenificaion using geneic programming and gene expression programming, Compuer and Informaion Sciences - ISCIS (2005) [30] C. L. C. Y. A. C. Jie Zuo, Changjie Tang, Time series predicion based on gene expression programming, Advances in Web-Age Informaion Managemen 3129 (2004) [31] K. Z. Z. D. Fangyu Li, Shouqian Sun, Improved gene expression programming algorihm for solving inverse problems in parial differenial equaions, Journal of Zhejiang Universiy (Engineering Science) 43 (11) (2009) [32] C. Ferreira, Gene expression programming: a new adapive algorihm for solving problems, Complex Sysems 13 (2) (2001) [33] Q. L. L. Z. J. S. Zhangjie Fu, Xingming Sun, Achieving efficien cloud search services: muli-keyword ranked search over encryped cloud daa supporing parallel compuing, The Insiue of Elecronics, Informaion and Communicaion Engineers, IEICE Transacions on Communicaions 98 (1) (2015) [34] L. K. Dazhi Jiang, Zhijian Wu, New mehod used in gene expression programming: Grcm, Journal of Sysem Simulaion (6) (2006)

17 [35] H.Md.Azamahulla, Gene-expression programming o predic fricion facor for souhern ialian rivers, Neural Compuing and Applicaions 23 (5) (2013) [36] B. Z. T. M. Yu Xue, Jiongming Jiang, A self-adapive arificial bee colony algorihm based on global bes for global opimizaion, Springer, Sof Compuing (2017) [37] C. Ferreira, Gene expression programming in problem solving, Sof Compuing and Indusry: Recen Applicaions (2002) [38] Y. L. Wenyin Gong, Zhihua Cai, Auomaic modeling of complex funcions based on gene expression programming, Journal of Sysem Simulaion 18 (6) (2006) [39] Z. C. Chong Li, Qiong Gu, Hyperspecral remoye sensing image classificaion based on de and gep, Microelecronics and Compuer 29 (11) (2012) [40] X. L. Xianfeng He, Mining and applicaion of reservoir landslide risk model based on gep, Yellow River 34 (3) (2012) [41] Z. H. Z. D. Dong, Predicion of gases conen dissolved in power ransformer oil based on gep sliding window model, Journal of Norh China Elecric Power Universiy 39 (4) (2012) [42] Z. D. Zhongmin Niu, Jiao Niu, Transformer faul diagnosis based on muli-gep classifier and dga, Journal of Norh China Elecric Power Universiy 39 (3) (2012) [43] T. L. Mengwei Liu, Xia Li, A gene expression programming algorihm for populaion predicion problem, Aca Scieniarum Nauralium Universiais Sunyaseni 49 (6) (2010) [44] X. Qian, Predicion in sock-price index based on improved gene expression programming, Compuer Engineering 35 (5) (2009) [45] Y. Wang, An improved gene expression programmingmehod and applicaion, Journal of Qingdao Agriculural Universiy (Naural Science) 26 (3) (2009) [46] Y. C. Lishan Kang, Hongqing Cao, The evoluionary modeling algorihm for sysem of ordinary differenial equaions, Chinese Journal of Compuers-Chinese Ediion 22 (1999) [47] Y. C. Hongqing Cao, Lishan Kang, A hybrid evoluionary modeling algorihm for dynamic sysem, Journal of Compuer Research and Developmen (8) (1999)