Factor Sensitivity Analysis with Neural Network Simulation based on Perturbation System

Transcription

1 1402 JOURNAL OF COMPUTERS, VOL. 6, NO. 7, JULY 2011 Factor Senitivity Analyi with Neural Network Simulation baed on Perturbation Sytem Runbo Bai College of Water-Conervancy and Civil Engineering, Shandong Agricultural Univerity, Tai an, China Daozhen Zhang Shandong Water Polytechnic, Rizhao, China Hailei Jia College of Civil and Traffic Engineering, Hohai Univerity, Nanjing, China Abtract Perturbation ytem i often ued in the factor enitivity analyi with neural network deign. The two major and key problem: the enitivity definition and the input perturbation ratio are invetigated in thi tudy. Beide, four model of enitivity analyi are conidered in the invetigation. Through comparion and analyi, reult how that the definition of enitivity derived from the partial derivative i relatively more rational than other and, the optimum range of the input perturbation ratio could be [, ] for a general cae. Additionally, the effect of quality of model on the prediction accuracy of the enitivity i dicued in thi paper, and their correlation i revealed. Index Term perturbation ytem, input perturbation ratio, artificial neural network, enitivity analyi I. INTRODUCTION Senitivity analyi i a fundamental iue in neural network deign, from which the evaluation on the degree that the enitivity of model output to change in parameter value i implemented for a collection of method [1]. Several approache have been developed on the quantitative meaurement of enitivity. For intance, the partial derivative method [2], the weight method [3], the perturbation method [4] and the profile method [5]. Of which, the perturbation method i widely ued, which aee the enitivity of neural network model through oberving the effect upon the overall error with the perturbation of each input. The perturbation i realized by adding noie to a certain input. Although uch a method i of extenive prevalence, ome diputation are till available in the literature. Firt, the cot function ued in the enitivity of a network i inconitent, which mean the definition of the enitivity i not uniform [4] [6] [7] [8] [9]. The major problem i how to reflect the repone of neural network output to the perturbation of it input. Second, one ha to take a ubjective cope about how much noie to add. Many reearche how that for the perturbation method, the reult of enitivity i related to the mode of input perturbation [10] [11] [12] [13]. Therefore, the input perturbation ratio i another important problem deerving of tudy. The goal of thi tudy are twofold: (1) explore the rational definition of enitivity, epecially in the neural network imulation, and (2) dicu the optimum range of the input perturbation ratio. Beide, the factor affecting the aement of the accuracy of the enitivity are alo hereinto elaborated. II. DEFINITION OF SENSITIVITY For the neural network-baed perturbation method, the enitivity of a ytem i etimated by the change of the output with repect to the input perturbation. The baic idea i that the input to the network are hifted lightly and the correponding change in the output i reported either a a percentage or a an abolute difference [14]. The key i to elect the reaonable index meauring the change in the output. Variou indice are available in preent tudie. Scardi et al. [4] adopt the mean quare error to repreent the change of the output, and thu the enitivity of the kth output with repect to the jth input i defined a ( ( ) ) ( ) ( ) ( ) D o D u u D o D (1) = = jk k j j k Where, D (ok) i the variance of the kth output correponding to the perturbation of the jth input change; D* mean the ideal variance without noie input perturbation; uj i a mall incremental change in the jth input variable; uj* i the original value of the jth input variable; i the input perturbation ratio. Choi et al. [6] define the enitivity of input perturbation a the ratio of the tandard deviation of output error to the tandard deviation of the input error under the condition that the latter tend to zero. The formula i doi: /jcp

2 JOURNAL OF COMPUTERS, VOL. 6, NO. 7, JULY = σ ( ) σ ( ) ( ( u j ) 0) o u jk k j σ (2) where σ (o k ) and σ (u j ) are the tandard deviation of the output error and the input error, repectively. Jiang Quan et al. [7] ue the ratio of the output relative error to the input relative error to repreent the enitivity, expreed a ( ) ( ) ( ) = o o u u = o o (3) jk k j j k in which o k i the correponding change in the value of the kth output variable due to u j and o * i the original value of the kth output variable. Mathematically, for a caual-to-reult ytem, the enitivity of a certain independent variable to one dependent variable i calculated by the partial derivative of the dependent variable with repect to the independent variable. The partial derivative method in neural network enitivity analyi i jut originated from the above idea. Neverthele, two major weaknee can be found in the partial derivative method. Firtly, it cannot implement neural network with non-differentiable activation function and econdly, it i inadequate to conider the magnitude effect of the parameter in enitivity aement [8]. So the numerical computation intead of analytical expreion i ued by ome reearcher to obtain the enitivity of model. Reddy et al. [9] give a definition of the enitivity in the form = o u (4) jk k j where o k reflect the network output error more directly and preciely than the variance doe. Thi definition i cloer to the one of the partial derivative compared to the other definition. It i expected that thi definition i more reaonable. In the above formula, o = ( yˆ y ) N k ik ik i= 1. In all the aforementioned literature, N i the number of the training object, y ik i target output for the i object and k output, and y i the correponding network etimate of ˆik the target value with the variou input perturbation. In fact, y ik hould not be the original value of the output but the network etimate of the target value without the input perturbation. Thi i eaily accountable; comparion mut be a kind, which i neural network compared with neural network, not the real target value. For the ake of convenience, we till ue the above form of the formula (1) ~ (4) while the ignificance of y ik i different in the following calculation. III. INPUT PERTURBATION RATIO OF INFLUENTIAL FACTORS The perturbation method adjut a certain dependent variable through adding noie at a time while keeping all the other untouched. The change ratio of the output variable with regard to the perturbation of the input variable i evaluated. The input variable with mot ignificant change ratio turn out the one that ha the tronget effect on the ytem analyzed. To obtain an objective aement on the enitivity of dependent variable, the optimum range of input perturbation ratio hould be determined a a precondition. If the perturbation i overlarge, the enitivity pectra may appear clipped. Generally, the farther a perturbation move from the bae cae value, the le reliable the reult become. However, if the perturbation i underize, the enitivity pectra may have no noie and ometime no ignal. The input perturbation ratio are added from -30% to 30% in tep of 1 in [7] and from -40% to 40% in tep of in [10], and from to in tep of in [11]. In [4], the range of input perturbation magnitude i [, ] and the correponding range i [0., ] in [12]. In [13], the input perturbation ratio ued i. In the cae of thi paper, the input perturbation ratio cover a wide range [-, ] o a to explore the appropriate range of input perturbation ratio in the enitivity analyi of neural network deign. The input perturbation i added a the following example: =0.1; pv= P_input(k,: )* ; P_input(k,: )= P_train(k,: )+pv; That mean the kth input factor value which are added the perturbation value with the ratio. We can change the value, for example, -0.01, to add the perturbation ratio a. If the invetigated factor i the (k+1)th, the k can be changed a k+1 to achieve the goal. Auming there are n input factor in number which needed to be tudied, through changing the from - to and the k from 1 to n, and combing the above definition of the enitivity, the enitivitie of all the input factor to the output can be calculated. IV. NUMERICAL CASE STUDY Two numerical example are run in thi ection to etimate the enitivity formula preented in ectionⅡ and to judge the appropriate range of input perturbation ratio. (1) Cae 1 The firt example employ the following equation: y=a 1 x 1 +a 2 x 2 +a 3 x 3 +a 4 x 4 with a 1 =1; a 2 =2; a 3 =-4; a 4 =8. (5) Thi i a imple linear function whoe value i the um of four function. The enitivity of x 1, x 2, x 3 and x 4 can be calculated by əy/əx 1, əy/əx 2, əy/əx 3, and əy/əx 4, and the reult are 1 =1, 2 =2, 3 =-4 and 4 =8. Thee reult can be regarded a a tandard with which the following neural network analyi reult are compared. The network tructure ued in thi example contain three layer with 4 input neuron, 6 hidden neuron and 1 output neuron. In thi cae, back-propagation algorithm with Bayeian training method i adopted, which offer an efficient tool to avoid overffitting o a to improve the generalization of the network [15]. The performance goal wa et to a very mall value 1E-5.

3 1404 JOURNAL OF COMPUTERS, VOL. 6, NO. 7, JULY 2011 One hundred and fifty group of data baed on (5) were randomly et a the ideal data for training. To calculate the enitivity pectra, each input variable wa perturbed by ±, ±0., ±, ±, ±, ±, ±, ± and ± of it original data. After the network model i well trained, the enitivity value of cae 1 at increaing level of input perturbation ratio from - to are calculated according to the formula (1) ~ (4) and are hown in Fig. 1 (a) ~ (d) repectively. It can be found from Fig. 1 that the formula (1) can reflect the relative order of each variable enitivity, but can not accurately meaure the ize-relationhip among variou factor; while the formula (2) can correctly calculate the magnitude of the enitivitie, but can not reflect their direction, that i the enitivitie according to the formula (2) are all poitive and can not reflect the poible inhibitory effect of variable; the formula (3) can reflect the relative magnitude and the direction of the enitivitie, but the value are not directly equal to the aforeaid tandard value, which need a converion of calculating the ratio among the variable; the formula (4) i perfect, which can correctly calculate the magnitude and the direction of the enitivitie. So the formula (4) i the bet for calculating the enitivity in thi cae. It can alo be found from Fig. 1 that the accuracy of enitivity i decreaed a the noie level increae. The reaonable range of the input perturbation ratio i [, ]. There i no much difference in the enitivity meaurement within thi range E E E E E E (a). formula (1) (b). formula (2) (c). formula (3) (d). formula (4) Fig. 1. The enitivitie of cae 1 according to the formula (1) ~ (4) at increaing level of input perturbation ratio. In order to how the different repreentation of the calculating enitivity when y ik in the formula (1) ~ (4) denote the target output of the model and the network etimate of the target value without the input perturbation, repectively. Fig. 2 give the enitivitie of cae 1 according to the formula (4) with y ik denoting the target output of the model. It i clear that the accuracy of enitivitie in Fig. 2 doe not alway increae with the noie level decreae. There i a big deviation when the input perturbation ratio i more (for example ±) or le (for example ±). The trend of change in the enitivity i irregular when the input perturbation ratio

4 JOURNAL OF COMPUTERS, VOL. 6, NO. 7, JULY change in a regular way under thi ituation. So it i not uitable to calculate the enitivity with y ik denoting the target output of the model. Herein lie another problem the accuracy of the neural network imulation. Fig. 3 how the enitivitie of cae 1 according to the formula (4) when the performance goal i 1E-2. Compared with the Fig. 1 (d), the accuracy of enitivitie with 1E-2 performance goal i lower than that with 1E-5 performance goal. The trend of accuracy of enitivitie calculated by the formula (4) i conitent with the accuracy of the neural network imulation Fig. 2 The enitivitie of cae 1 according to the formula (4) with y ik denoting the target output of the model Fig. 3 The enitivitie of cae 1 according to the formula (4) with the performance goal i 1E-2. (2) Cae 2 The econd example employ the following equation: x4 x y = 2x + 5x + 10inx cox + 5e (6) The enitivity of x 1, x 2, x 3 and x 4 can be calculated by əy/əx 1, əy/əx 2, əy/əx 3, and əy/əx 4. All the variable enitivitie are the function of the input data except 1. The average enitivity of x 2 in the input domain i 12.21, and for x 3, 2.51 for x 4. So 1 =2, 2 =12.21, 3 =-4.69 and 4 =2.51 can be regarded a a tandard with which the following neural network analyi reult are compared. The neural network ued for realizing thi function approximation i a three-layered ytem with 4 input neuron, 8 hidden neuron and 1 output neuron. Back-propagation algorithm with Bayeian training method wa adopted, and the performance goal wa et to 1E-5. One hundred and fifty group of data baed on equation (6) were randomly et a the ideal data for training. To calculate the enitivity pectra, each input variable wa perturbed by ±, ±0., ±, ±, ±, ±, ±, ± and ± of it original data. Fig. 4 how the enitivity value of cae 2 at increaing level of input perturbation ratio from - to according to the formula (1) ~ (4). A the cae 1, the ame concluion that the formula (4) i the bet and the reaonable range of the input perturbation ratio i [, ] can be drawn. In fact, the excellence of formula (4) i not accidental but baed on the conitency of it definition with the one of partial derivative. Fig. 5 i the enitivitie of cae 2 according to the formula (4) with y ik denoting the target output of the model. The reult alo how that it i unuitable to calculate enitivity when y ik denoting the target output of the model. The concluion drawn from Fig. 6 which how the enitivitie of cae 2 according to the formula (4) with the performance goal being 1E-2 i the ame a cae 1. The accuracy of the neural network imulation ha an important influence on the accuracy of enitivity calculated by the formula (4). V. CONCLUSION The perturbation method on enitivity wa dicued in thi paper, epecially for it definition and the problem of it input perturbation ratio. Our finding are a follow. (1) The formula (4) in thi paper i the bet on calculating the enitivity in comparion with other formula which ued in the publihed literature. The formula (4) can correctly calculate the magnitude and the direction of the enitivity while other formula can not both atify thee two point. It hould be noted that y ik in thee formula i not the target output of the model but the network etimate of the target value without the input perturbation, which i different from the other literature. (2) The input perturbation ratio i an important factor affecting the enitivity of the model. The reaonable range of the input perturbation ratio i [, ] according to the two example in thi paper. There i no much difference in the enitivity meaurement and the right anwer can be obtained within thi range. (3) The accuracy of the neural network imulation ha an important influence on the accuracy of enitivity calculated by the formula (4). The more accurate the neural network imulation i, the more accurate the

5 1406 JOURNAL OF COMPUTERS, VOL. 6, NO. 7, JULY 2011 enitivity of the model i. In that way, the performance goal of the neural network model hould be limited a mall a poible E E E E E E+008 (a). formula (1) (b). formula (2) (c). formula (3) (d). formula (4) Fig. 4. The enitivitie of cae 2 according to the formula (1) ~ (4) at increaing level of input perturbation ratio Fig. 5 The enitivitie of cae 2 according to the formula (4) with y ik denoting the target output of the model Fig. 6 The enitivitie of cae 2 according to the formula (4) with the performance goal being 1E

6 JOURNAL OF COMPUTERS, VOL. 6, NO. 7, JULY ACKNOWLEDGMENT Thi work i upported in part by a grant from the National Natural Science Foundation of China (Grant No ) and China Outtanding Potdoctor Science Foundation (Grant No ). REFERENCES [1] Swartzman, G.L., Kaluzny, S.P., Ecological Simulation Primer. Macmillan Publihing Company, NewYork, 370pp [2] Dimopoulo, I., Chronopoulo, J., Chronopoulou-Sereli, A., Lek, S., Neural network model to tudy relationhip between lead concentration in grae and permanent urban decriptor in Athen city (Greece). Ecol. Model. 120, [3] Garon, G.D., Interpreting neural network connection weight. Artif. Intel. Expert 6, [4] Scardi, M., Harding, L.W., Developing an empirical model of phytoplankton primary production: a neural network cae tudy. Ecol. Model. 120, [5] Lek, S., Delacote, M., Baran, P., Dimopoulo, I., Lauga, J., Aulagnier, S., Application of neural network to modelling nonlinear relationhip in ecology. Ecol.Model. 90, [6] J. Y. Choi and C.-H. Choi, Senitivity analyi of multilayer perceptron with differentiable activation function, IEEE Tranaction on Neural Network, vol.3, pp , Jan [7] Jiang Quan,Feng Xiating,Su Guohao,Chen Guoqing. Intelligent Back Analyi of Rock Ma Parameter for Large under Ground Cavern under High Earth Stre Baed on EDZ and Increment Diplacement. Chinee Journal of Rock Mechanic and Engineering, 26(Supp.1): (in Chinee) [8] AntonyY. Cheng and DanielS. Yeung. Senitivity Analyi of Neocognitron, IEEE Tranaction on Sytem, Man and Cybernetic --Part C: Application and Review, v 29, n 2, p , [9] Reddy, N.S.; Lee, C.S.; Kim, J.H.; Semiatin, S.L. Determination of the beta-approach curve and beta-tranu temperature for titanium alloy uing enitivity analyi of a trained neural network. Material Science and Engineering A, v 434, n 1-2, p , October 25, [10] S. Jaiwal, E.R. Benon, J.C. Bernard, G.L. Van Wicklen. Neural Network Modelling and Senitivity Analyi of a Mechanical Poultry Catching Sytem, Bioytem Engineering, 92(1), [11] Muriel Gevrey, Ioanni Dimopoulo, Sovan Lek. Review and comparion of method to tudy the contribution of variable in artificial neural network model. Ecological Modelling, 160, [12] Peter de B. Harrington and Chuanhao Wan. Senitivity Analyi Applied to Artificial Neural Network: What ha my neural network actually learned? IJIMS 5(2002)2,1-18. [13] Michele Scardi. Artificial neural network a empirical model for etimating phytoplankton production, Marine ecology progre erie. 139: [14] D. Lamy, Modeling and enitivity analyi of neural network, Mathematic and Computer in Simulation, 40, p , [15] Yuanchang Xie, Dominique Lord, Yunlong Zhang. Predicting motor vehicle colliion uing Bayeian neural network model: An empirical analyi, Accident Analyi and Prevention, Volume 39, Iue 5, September 2007, pp Runbo Bai wa born in Tai an, China, in He received hi Ph.D. in College of Civil Engineering from Hohai Univerity, China in Dr. Bai joined Shandong Agricultural Univerity after hi Ph.D. a a lecturer. He ha been actively engaged in collaborative reearch project in the area of artificial neural network, hydraulic and civil engineering. Daozhen Zhang wa born in Zibo, China, in He graduated in the College of Water-Conervancy and Civil Engineering, Shandong Agricultural Univerity, Tai an, China, in 2004, and preently i a lecture in Shandong Water Polytechnic, Rizhao, China. Hi main interet are dynamic model, nonlinear ytem and automaton. Hailei Jia wa born in Zibo, China, in He graduated in the College of Water-Conervancy and Civil Engineering, Shandong Agricultural Univerity, Tai an, China, in 2007, and preently i a potgraduate tudent in the College of Civil and Traffic Engineering, Hohai Univerity, Nanjing, China. Hi main interet are artificial neural network, biofilm and biofouling.