Evaluating the effect of salinity on corn grain yield using multilayer perceptron neural network

Transcription

1 Evaluating the effect of salinity on corn grain yield using ultilayer perceptron neural netork O. M. Ibrahi Plant Production Departent, Arid Lands Cultivation Research Institute, City of Scientific Research and Technological Applications (SRTA-City), Ne Borg El-Arab, Aleandria 1934, Egypt Abstract Multilayer perceptron neural netork (MLP) is a poerful statistical odeling technique in the agricultural sciences. The ai of the relative iportance analysis is to separate eplained variance aong ultiple predictors to better understand the role played by each predictor or independent variable in relation to the dependent variable. To assess the relative iportance of independent variables (Na +, K +, Ca +, and Mg + ) hich have negative relationship ith the dependent variable (corn grain yield) in ultilayer perceptron neural netork the current study as carried out to ake a coparison aong different algoriths, the Connection Weights Algorith, Modified Connection Weights Algorith, Most Squares Algorith, Doinance Analysis, Garson s Algorith, Partial Derivatives, Profile Method, and Multiple Linear Regression. The perforance of these algoriths is studied for epirical data. The Most Squares Algorith as found to be a better algorith in coparison to the above entioned algoriths and see to perfor uch better, and agree ith the results of ultiple linear regression in ters of the partial R and see to be ore reliable, Most Squares Algorith cleared that Na + had the ain ipact on corn grain yield folloed by Mg +, K +, and Ca +, respectively. Keyords: salinity, relative iportance, ultilayer perceptron neural netork 1. INTRODUCTION The nature of the relationships beteen independent and dependent variables in agriculture are alost very coplicated. One of the ost successful ethods to illustrate these relationships is ultilayer perceptron neural netork. A nuber of authors have shon the interest of using Artificial Neural Netorks (ANNs) instead of linear statistical odels (Özesi and Özesi, 1999, Ibrahi, 01, Ibrahi, 013, Ibrahi et al., 013, Ibrahi et al., 014). The ain application of ANNs is to develop predictive odels to predict future values of a dependent variable fro a set of independent variables. The study of the contributions of input variables in ANNs odels has been attepted only by fe authors. (Ibrahi, 013) has presented a ne ethod to understand ho yield coponents of corn as inputs are correlated to the grain yield as output by the MLP. He has tested this ne ethod on epirical dataset and has shon that the results correspond ell to the partial least squares ethod interpretation for ultiple linear regression odels. (Olden and Jackson, 00) have revieed a nuber of ethods (Neural Interpretation Diagra, Garson s algorith and sensitivity analysis) and deonstrated utility of these ethods for interpreting neural netork connection eights. (Olden et al., 004) have provided a coparison of different ethodologies for assessing variable contributions in artificial neural netorks using siulated data ehibiting defined nueric relationships beteen a response variable and a set of predictor variables. They proposed an approach called connection eights ethod to outperfor other ethods in quantifying iportance of variables. They have shon that the connection eight ethod is the least biased aong 400 P a g e 8 F e b r u a r y g j a r. o r g

2 others. This ethod has also been used in coparison to other available ethods in assigning the relative contribution of input variables in prediction of the output by (Watts and Worner, 008).. MATEALS AND METHODS An eperient as set up during the suer seasons of 01 and 013 to investigate the effect of three levels of ater salinity (0.5,.75, 5.5 ds/.) on yield and yield coponents of Maize as ell as the relative iportance of Calciu (Ca + ), Magnesiu (Mg + ), Sodiu (Na + ), and Potassiu (K + ) as the ajor cations. Grain of Maize cultivar (Geeiza 1) as obtained fro Field Crops Research Institute, Agricultural Research Center, Egypt. The soil as analed according to (Chapan and Pratt, 1978) before soing and had the folloing echanical and cheical characters. Soil teture as sandy loa here Sand (74%), silt (10.4%), and clay (15.6%). Electrical conductivity (1.8 ds/), ph (7.53), Saturation percent (43.3%). The eperiental design as Randoized Coplete Block Design (RCBD) ith four replicates; the eperiental unit as a ceented plot ith a diension of 150 c in long and 75 c in ide ith an area of Every ceented plot contains four ros, the grain ere soing in first May and before soing the ceented plot ere prepared by adding calciu superphosphate 15.5% P O 5 at a rate of 100 kg/feddan (Hectare =.38 feddan) and potassiu sulphate 48% K O 4 at a rate of 50 kg/feddan, the nitrogen fertilizer rates ere added at the rate of 15 kg N/feddan of aoniu sulphate 0.5% N at three doses, the first at soing, the second at the first irrigation, and the third at the second irrigation. At the end of the eperient the folloing characters ere easured: Nuber of grain/ro, 100 kernel eight (g), and grain yield (g/plot). The data as subjected to analysis of standardized and stepise ultiple linear regression according to (Snedecor and Cochran, 198) and doinance analysis using SAS coputer softare v SP4, 003. A progra as ritten by the author, based on Microsoft Access 007 and containing odules ritten in visual basic coputer language, as used to perfor the coparison aong the different relative iportance ethods (Figure ). Neural netork training and architecture A three layer feed forard ultilayer perceptron neural netork is considered and trained using back propagation algorith. The three layered feed forard netork contains one input layer, one hidden layer and one output layer. The input layer contains 4 neurons corresponding to 4 independent variables, Ca +, Mg +, Na +, K + and the output layer contains one neuron corresponding to one dependent variable, corn grain yield (GY). The sigoid activation function as used at both the hidden layer and the output layer. The structure of the ultilayer perceptron neural netork used in this study and its connections beteen layers is shon in Figure (1). Fig.1. Structure of the ultilayer perceptron neural netork used in this study. 401 P a g e 8 F e b r u a r y g j a r. o r g

3 Fig.. Interface of the progra used to perfor the coparison aong the studied ethods. 3. RESULTS AND DISCUSSION Relative iportance of independent variables ethods The relative iportance of a predictor or input variables is the contribution of each of the variables for predicting the dependent variable. 1- Connection eights algorith (CW) The connection eights algorith (Olden and Jackson, 00) calculates the su of products (Table 3) of final eights of the connections fro input neurons to hidden neurons (Table 1) ith the connections fro hidden neurons to output neuron (Table ) for all input neurons. The relative iportance of a given input variable can be defined as Where y1 is the relative iportance of input neuron, y1 is su of product of final eights of the connection fro input neuron to hidden neurons ith the connection fro hidden neurons to output neuron, y is the total 40 P a g e 8 F e b r u a r y g j a r. o r g

4 nuber of hidden neurons, and z is output neurons. This approach is based on estiates of netork final eights obtained by training the netork. It is observed that these estiates of final eights ay vary ith the change in the initial eights used for starting the training process. Table (1). Input-hidden final connection eights Hidden1 Hidden Hidden3 Hidden4 Hidden5 Hidden6 Ca Mg Na K Table (). Hidden-output final connection eights Hidden1 Hidden Hidden3 Hidden4 Hidden5 Hidden6 GY Table (3). Connection eights products, relative iportance and rank of inputs. Absolute Relative Rank Hidden1 Hidden Hidden3 Hidden4 Hidden5 Hidden6 Su su iportance Ca % 4 Mg % Na % 1 K % 3 Su % - Modified Connection Weights (MCW) This ethod as proposed by (Ibrahi, 013). In this algorith, the connection eights of the artificial neural netork odel is obtained after training the netork, after the calculation of su of product of final eights of the connections fro input neurons to hidden neurons ith the connections fro hidden neurons to output neuron for all input neurons, a correction ter (partial correlation) is ultiplied by this su and the absolute value is taken, this is called the corrected su, (Table 4), then to calculate the relative iportance of each input, the corrected su of each input is divided by the total corrected su as illustrated in the folloing equation. here y1 n 1 y1 r ij. k is the relative iportance of neuron, y1 input neuron to hidden neurons ith the connection fro hidden neurons to output neuron, input i ith output j after input k, n 1 y1 is su of product of ra eights of the connection fro is the total corrected su of all inputs. r ij. k is partial correlation of 403 P a g e 8 F e b r u a r y g j a r. o r g

5 The correction ter is the partial correlation; the partial correlation is the correlation that reains beteen to variables after reoving the correlation that is due to their utual association ith the other variables. The correlation beteen the dependent variable and an independent variable hen the linear effects of the other independent variables in the odel have been reoved fro both. The partial correlation for first order is illustrated in the folloing equation. r ij. k here output j, output j. r ij r ki r kj r 1 r 1 ki kj r ij. k is partial correlation of input i ith output j after input k, rij is the siple correlation beteen input i and r ki is the siple correlation beteen input k and input i, r kj is the siple correlation beteen input k and Table (4). Absolute corrected su and relative iportance of inputs. Absolute Rank Inputs partial corrected Relative Su correlation Su iportance Ca % 3 Mg % Na % 1 K % 4 Total % 3- Most Squares (MS) This ethod as proposed by (Ibrahi, 013). In this algorith, the connection eights beteen hidden layer and the output layer ere not used; hoever, the connection eights beteen input layer and hidden layer ere used, both the initial eights before start training (Table 5) and the final eights after training the netork (Table 1). The second step is to su the squared difference beteen initial and final eights for each input. The third step is to divide the su of squared difference for each input on the total su of all inputs (Table 6). The folloing equation is used to calculate the relative iportance of each input. here 1 1 n i f y1 i f i f is the relative iportance of neuron, 1 is the su squared difference beteen initial n i f connection eights and final connection eights fro input layer to hidden layer, and of su squared difference of all inputs. 1 y1 is the total 404 P a g e 8 F e b r u a r y g j a r. o r g

6 Table (5). Input-hidden initial connection eights Hidden1 Hidden Hidden3 Hidden4 Hidden5 Hidden6 Ca Mg Na K Table (6). Squared difference beteen initial and final connection eights, relative iportance, and rank of inputs. Hidden1 Hidden Hidden3 Hidden4 Hidden5 Hidden6 Su Relative iportance Rank Ca % 4 Mg % Na % 1 K % 3 Su % 4- Multiple linear regression (MLR) A coparison beteen MLR and the other ethods as ade in order to judge their predictive capacities. The stepise ultiple regression technique (Toassone et al., 1983) as coputed especially to define the significant variables and their contribution. In fact the influence of each input variable can be assessed by checking the final values of the regression coefficients. Standardized regression coefficient has been suggested as a easure of relative iportance of input variables by any authors (Afifi and Clarke, 1990). For each input variable, standardized regression coefficient is obtained by standardizing the variable to zero ean and unit standard deviation before ultiple regression is carried out. The results of ultiple linear regression are shon in (Table 7). The results revealed that Na + as the ost iportant variable ith a partial R of folloed by Mg + (0.1531), Ca + (0.0003), and K + (0.0001). To ake a coparison ith the other ethods, the relative iportance of each input as coputed by dividing its partial R on the su of R for all inputs hich is equal to regression R (86.01), for eaple the relative iportance of Na + is coputed as: = (0.7067/0.8601)*100 = 8.16 %. Table (7). Results of ultiple linear regression and correlation Paraeter estiates Standardized estiates Significance Partial Intercept R Relative iportance based on contribution to regression R Siple Correlation Ca ns % Mg ** % Na ** % K ns % Partial Correlation 5- Doinance analysis (DA) Doinance analysis deterines the doinance of one input over another by coparing their additional contributions across all subset odels. Doinance analysis (Azen and Budescu, 003) approaches the proble of relative iportance by eaining the change in R resulting fro adding an input to all possible subset regression odels. By averaging all of the possible odels (average squared sei partial correlation), e can obtain general doinance eight of an input, 405 P a g e 8 F e b r u a r y g j a r. o r g

7 hich reflects the contribution by itself and in cobination ith the other inputs, and overcoing the probles associated ith correlated inputs, the results of doinance analysis are presented in (Table 8). To ake a coparison ith the other ethods, the relative iportance of each input as coputed by dividing its overall average contributions on the su of all overall average contributions for all inputs hich is equal to regression R (86.01), for eaple the relative iportance () of Na + is coputed as follos: = (0.5571/0.8601)*100 = 64.78%. Table (8). Relative iportance and rank of inputs. Inputs Overall average Relative iportance based on contribution Rank contributions of inputs to regression R Ca % 4 Mg % Na % 1 K % 3 Total % 6- Garson s algorith (GA) (Garson, 1991) proposed a ethod for partitioning the neural netork connection eights in order to deterine the relative iportance of each input variable in the netork. It is iportant to note that Garson s algorith uses the absolute values of the final connection eights hen calculating variable contributions, and therefore does not provide the direction of the relationship beteen the input and output variables as illustrated in the folloing equation. here n 1 y1 is the relative iportance of neuron, y1 is su of product of final eights of the connections fro input neurons to hidden neurons ith the connections fro hidden neurons to output neurons. (Table 9) shos the contribution of each input to each hidden neuron, for eaple the contribution of Ca + is (.41) resulted fro dividing its products (0.580) on the su of products of all inputs (0.40) in (Table 3) and take the absolute result. Table (9). Relative contribution of each input neuron to each hidden neuron, relative iportance, and rank of inputs. Hidden1 Hidden Hidden3 Hidden4 Hidden5 Hidden6 Su Relative iportance Rank Ca % 3 Mg % Na % 1 K % % 406 P a g e 8 F e b r u a r y g j a r. o r g

8 The results in (Table 10) reveal that Garson s algorith produced inaccurate results here it tends to under estiate the ost iportant input and over estiate the loest iportant input. 7- Partial derivatives (PD) To obtain the relative iportance of each input, the partial derivatives of the MLP output ith respect to the inputs ere coputed (Diopoulos et al., 1999). In a netork ith n i inputs, one hidden layer ith n h neurons, and one output neuron, the partial derivatives of the output y j ith respect to input j and N total nuber of observations are: here S j is the derivative of the output neuron ith respect to its input, I hj is the response of the h th hidden neuron, ho and ih are the connection eights beteen the output neuron and h th hidden neuron, and beteen the i th input neuron and the h th hidden neuron. If the partial derivative is negative the output variable ill tend to decrease hile the input variable increases. Inversely, if the partial derivatives are positive, the output variable ill tend to increase hile the input variable also increases. The relative iportance of the ANN output is calculated by a su of the square partial derivatives, SSD, obtained per input variable as follos: The input variable that has the highest SSD value is the ost iportant variable, and inversely the input variable that has the loest SSD value is the loest iportant variable. The results of partial derivatives ethods are shon in (Table 10). 8- Profile Method (PM) Profile ethod involves varying each input variable across its entire range hile holding all other input variables constant; so that the individual contributions of each variable are assessed. This approach is soehat cubersoe, hoever, because there ay be an overheling nuber of variable cobinations to eaine. As a result, it is coon first to calculate a series of suary easures for each of the input variables (iniu, aiu, quartiles, percentiles), and then vary each input variable fro its iniu to aiu value, in turn, hile all other variables are held constant at each of these easures (Özesi and Özesi, 1999). The results of profile ethod are shon in (Table 10). Table (10). Relative iportance and rank of inputs. Partial derivatives Profile Method Inputs Relative iportance Rank Relative iportance Rank Ca % % 3 Mg % 19.18% Na % % 1 K % 3.33% 4 Total 100% 100% Coparison of the 8 ethods used for quantifying relative iportance of input variables: According to the results of Multiple Linear Regression (MLR), the Most Squares ethod as found to ehibit the best overall perforance copared to the other ethods ith regard to its accuracy, here the degree of dissiilarity coefficient beteen MLR and Most Squares (MS) as the loest, (Table 11), they ere in the sae cluster (Fig. 4), and produced coparable results (Fig. 3). The perforance of Modified Connection Weights (MCW) as less than 407 P a g e 8 F e b r u a r y g j a r. o r g

9 MS here the degree of dissiilarity coefficient beteen MCW and MLR as , Connection Weights (CW), Partial Derivatives (PD), (Gervey et al., 003), and Profile Method (PM) shoed oderate perforance (dissiilarity 1.750, , and.1446 respectively), hereas Garson s Algorith (GA), and Doinance Analysis (DA) perfored poorly (dissiilarity and respectively). Furtherore, MS ehibits acceptable precision copared to the other ethods, as indicated by the sall variation around its ean (Fig. 3). In contrast, CW, GA, and PM ehibit large variation around their eans. MS ethod as found to accurately quantifying the relative iportance of input variables and should be favored over the other ethods tested in this study. This ethod successfully identified the relative iportance of all input variables in the neural netork, including variables that ehibit both strong and eak partial correlations ith the dependent variable. GA as the poorest perforing ethod, because it uses absolute connection eights to calculate variable relative iportance, and therefore does not account for negative connection eights. Hoever, all other ethods used ra connection eights to calculate variable relative iportance. Fig. provides an illustration about the key difference aong the ethods and shos ho GA can result in incorrect estiates of variable relative iportance. Table (11). Distance atri based on Euclidian dissiilarity coefficient for the 8 relative iportance ethods. CW=connection eight, MCW=Modified connection eight, MS= ost squares, MLR=ultiple linear regression, DA=doinance analysis, GA=garson s algorith, PD=partial derivatives, PM=profile ethod 408 P a g e 8 F e b r u a r y g j a r. o r g

10 CW=connection eight, MCW=Modified connection eight, MS= ost squares, MLR=ultiple linear regression, DA=doinance analysis, GA=garson s algorith, PD=partial derivatives, PM=profile ethod Fig.3. Relative iportance of the inputs according to each of the 8 ethods. CW=connection eight, MCW=Modified connection eight, MS= ost squares, MLR=ultiple linear regression, DA=doinance analysis, GA=garson s algorith, PD=partial derivatives, PM=profile ethod Fig.4. Dendrogra shoing cluster analysis (Ward ethod) of the 8 relative iportance ethods. 409 P a g e 8 F e b r u a r y g j a r. o r g

11 4. CONCLUSION The current study provides a coparison of the perforance of 8 different ethods for assessing variable relative iportance. The perforance of the algoriths are studied using epirical data sets having negative relationships beteen one or/and ore of the independent variables ith the output. Most Squares (MS) is seeed to perfor uch better than the other ethods, and agree ith the results of ultiple linear regression in ters of the partial R and see to be ore reliable. Most Squares Algorith cleared that Na + had the ain ipact on corn grain yield folloed by Mg +, K +, and Ca REFERENCES [1] Afifi, A.A. and V. Clarke (1990). Coputer aided ultivariate analysis, nd ed., Van Nostrand Reinhold, Ne York. [] Azen, R. and D.V. Budescu (003). The doinance analysis approach for coparing predictors in ultiple regression, Psychological Methods 8, [3] Chapan, H. O. and P.E. Pratt (1978). Methods of analysis for soil, plants and aters. Univ. Calif. Div. Agic. Sci. [4] Diopoulos, I., Chronopoulos, J., Chronopoulou Sereli, A., and and S. Lek (1999). Neural netork odels to study relationships beteen lead concentration in grasses and peranent urban descriptors in Athens city (Greece). Ecological Modelling 10, [5] Garson, G.D. (1991). Interpreting neural netork connection eights. Artif. Intell. Epert 6, [6] Gevrey, M., Diopoulos, I., and S. Lek (003). Revie and coparison of ethods to study the contribution of variables in artificial neural netork odels. Ecol. Model., 160: [7] Ibrahi, O.M. (01). Siulation of Barley grain yield using artificial neural netorks and ultiple linear regression odels. Egypt. J. Appl. Sci., vol. 7, No.1, p [8] Ibrahi, O.M. (013). A coparison of ethods for assessing the relative iportance of input variables in artificial neural netorks. Journal of Applied Sciences Research, 9(11): [9] Ibrahi, O.M., Bakry, A. B., Asal, M. Waly and Elha, A. Badr (014). Modeling grain and stra yields of heat using back propagation and genetic algorith. Asian Acadeic Research Journal of Multidisciplinary. 1(8): [10] Ibrahi, O.M., A.T. Thalooth and Elha A. Badr (013). Application of Self Organizing Map (SOM) to Classify Treatents of the First Order Interaction: A coparison to Analysis of Variance.World Applied Sciences Journal. 5 (10): [11] Olden, J.D. and D.A. Jackson (00). Illuinating the black bo : a randoization approach for understanding variable contributions in artificial neural netorks, Ecological Modelling 154, [1] Olden, J.D. Joy, M.K. and R.G. Death (004). An accurate coparison of ethods for quantifying variable iportance in artificial neural netorks using siulated data, Ecological Modeling 178, [13] Özesi, S.L., and U. Özesi (1999). An artificial neural netork approach to spatial habitat odeling ith inter specific interaction. Ecol. Model. 116, [14] SAS Institute Inc. (00). SAS Softare v sp4, Cary, NC: SAS Institute Inc. [15] Snedecor, G. W and W.G. Cochran (198). Statistical Methods 7 th ed., Ioa state Press Ioa, USA. [16] Toassone, R., Lesquoy, E., and C. Miller (1983). La re gression,nouveau Regards sur une ancienne e thode Statistique.INRA, Paris. [17] Watts, M.J. and S.P. Worner (008). Using artificial neural netorks to deterine the relative contribution of abiotic factors influencing the establishent of insect pest species, Ecological Inforatics 3, P a g e 8 F e b r u a r y g j a r. o r g