LR-KFNN: Logistic Regression-Kernel Function Neural Networks and the GFR-NN Model for Renal Function Evaluation

Transcription

1 LR-KF: Logistic Regression-Kernel Function eural etworks and the GFR- Model for Renal Function Evaluation Qun Song, Tianmin Ma and ikola Kasaov Knowledge Engineering & Discovery Research Institute Auckland University of Technology Private Bag 9006, Auckland 00, ew Zealand Astract This paper introduces a novel knowledge ased neural network models that incorporate and adapt oth existing logistic regression formulas and kernel functions in there structures to improve the learning and adaptation aility of a connectionist model when there is an existing knowledge on the prolem in the form of a logistic regression. Different from standard feed-forward neural networks, the proposed model uses several non-linear function pairs as neurons in its hidden layer. Each of these pairs consists of one knowledge-ased function and one distance ased function. An existing gloal model a logistic regression formula, is transformed into a set of su-models each representing a cluster of the prolem space. These su-models constitute new, local knowledge functions. The Gaussian kernel functions are taken as the distance functions. After modifying each su-model individually, all these su-models are aggregated through incremental learning. The gradient descent method is applied for parameter optimization. The method is illustrated on a model called GFR-, for creating an accurate estimation of glomerular filtration rate (GFR) ased on an existing formula MDRD and real data from the clinic. The performance of the GFR- is compared with the MDRD formula, MLP neural network, AFIS and DEFIS fuzzy models on the same data. The results show that the GFR- is effective on this task and superior over the other models. Keywords: Logistic regression neural networks, kernel function neural networks, glomerular filtration rate; renal function evaluation.. Introduction: Logistic Regression-Kernel Function eural etworks (LR-KF) The introduced here LR-KF can e represented as the following generic function and its structure is shown in Fig.: y(x) G (x) F (x) + G (x i ) F (x)+ + G M (x) F M (x) () where, x [x, x,, x P ] is the input vector; y is the output vector; G l are kernel functions and F l, are logistic regression functions, for l,, M. Using different G l and F l, Eq. can represent different kinds of neural networks: if the G l are Gaussian kernel functions and the F l are constants it is a RBF neural network; if the G l are sigmoid transform functions and the F l are constants it is a generic MLP neural network; if the G l are fuzzy rule ased weighted functions and the F l are linear functions it is a first-order TSK fuzzy inference model; and in the simplest case, if each G l represents a single input variale and the F l are constants it is a linear regression function.

2 Fig. A generic structure of the LR-KF The proposed LR-KF can also e represented y Eq.. Different from standard feedforward ack-propagation (MLP) or radial asis function (RBF) neural networks, which usually use sigmoid transfer functions or Gaussian kernel functions as the neurons in their hidden layer(s), in the case of LR-KF, the F l are non-linear functions that represent the knowledge in local areas, and the G l are Gaussian kernel functions that control the contriution of each F l to the system output. In this paper we will develop as an illustration of the LR-KF a model that incorporate oth existing formula (the MDRD [4]) and new clinical data for the evaluation of renal functions. The model is called GFR-. The MDRD formula [4], which will e introduced in Section 3, is popular to use in the clinic and in research of nephrology for calculating the estimate value of glomerular filtration rate (GFR), and the modified MDRD formula is taken as F l the knowledge functions in the LR- KF. The paper is organized as follows: Section presents the LR-KF learning algorithm. Sections 3 illustrates the implementation of the LR-KF on the GFR data collected at hospitals in ew Zealand and Australia and conclusions are drawn in Section 4. LR-KF Learning Algorithm The LR-KF learning procedure performs the following steps: () Cluster the whole training data set into M clusters and each of these clusters includes a su-dataset. () In each cluster l, l,,, M, take an existing logistic formula (e.g. MDRD) as the initial function F l and then, using gradient descent method to modified the function on the su-dataset. (3) Create a Gaussian kernel function as the distance function G l : the cluster centre and radius are respectively taken as the initial values of the centre and width of G l. (4) Aggregate F l and G l and optimize the parameters in the LR-KF using the gradient descent method on the whole data set.

3 The parameter optimisation procedure is descried elow: Consider the system having P inputs, one output, and M neuron pairs in the hidden layer, the output value of the system can e calculated on input vector x i [x, x,, x P ] y the rewrote Eq. : y(x i ) G (x i ) F (x i ) + G (x i ) F (x i )+ + G M (x i ) F M (x i ) In the case of the GFR model an MDRD-type logistic regression formula is used: F l l l lp ( x i) l0 x x x, (the MDRD-type logistic regression formula) () p P ( xij m ) and G l( x i) αl exp[ ] (Gaussian kernel function) (3) σ j In the LR-KF learning algorithm, the following indexes are used: Training Data pairs: i,,, ; Input Variales: j,,, P; euron pairs in the hidden layer: l,,, M; Learning Iterations: k,, Suppose the LR-KF is given the training input-output data pairs [x i, t i ], the system minimizes the following ojective function (an error function): E i [ y ( x ) ] (4) i t i The gradient descent algorithm (ack-propagation algorithm) is used to otain the recursions for updating the parameters,, m and such that E of Eq. 4 is minimized: l 0 ] { Gl( i) y ( i) ti } ( k + ) l0( x x (5) l 0( i { log( xij) Gl( i) y ( i) ti] } ( k + ) ( x x (6) i { Gl( xi) y ( i) ti] } α αl( k + ) αl( x (7) α ( l i m ( k + ) m ( σ ( k + ) σ ( m m i Gl( xi) y ( xi) ti]( x m) i σ Gl( xi) y ( xi) ti]( x m) 3 σ (8) (9) here,,, m, and are learning rates for updating the parameters,, m and respectively.

4 3 KF for the GFR Estimation the GFR- Model Here, the LR-KF is applied for the creation of a knowledge ased neural network model for the evaluation (identification) of renal functions of patients in a renal clinic the GFR- model. Real data is collected in a clinical environment. The accurate evaluation of renal function is fundamental to sound nephrology practice. The early detection of renal impairment will allow for the institution of appropriate diagnostic and therapeutic measures, and potentially maximize preservation of intact nephrons [4]. Glomerular filtration rate (GFR) is traditionally considered the est overall index to determine renal function in healthy and in diseased people. Most clinicians rely upon the clearance of creatinine (CrCl) as a convenient and inexpensive surrogate for GFR. CrCl can e determined y either timed urine collection, or from serum creatinine using equations developed from regression analyses such as that y Cockcroft-Gault formula, ut the accuracy of CrCl is limited y methodological imprecision and the systematic ias [4]. Recently, the Modification of Diet in Renal Disease (MDRD) study group developed a new formula to more accurately evaluate the GFR [4]. The formula uses six input variales: age, gender, Screat, Race, Sal and Surea and is defined as follows: GFR 70 Screat Age (if female).8(if race is lac Surea -0.7 Sal 0.38 (0) In Eq. (0) Screat (Serum creatinine) is a protein which is expected to e filtered in the kidneys and the residual of it - released into the lood. The creatinine level in the serum is determined y the rate it is eing removed in the kidney and is also a measure of the kidney function. Surea (Serum urea) is a sustance produced in the liver as a means of disposing of ammonia from protein metaolism. It is filtered y the kidney and can e reasored to the loodstream. Sal (Serum alumin) is the protein of the highest concentration in plasma. Decreased serum alumin may result from kidney disease, which allows alumin to escape into the urine. Decreased alumin may also e explained y malnutrition or liver disease. However, prediction of GFR with the use of the existing formulas that constitute gloal and fixed models, can e misleading as to the presence and progression of renal disease [4]. Using proposed model on a GFR data set (447 samples) collected in hospitals in ew Zealand and Australia, we have otained more accurate results than with the use of the MDRD formula or the use of other connectionist models. For comparison, the results produced y the MDRD function [4], MLP and RBF neural networks [5], adaptive neural fuzzy inference system (AFIS) [] and dynamic evolving neural fuzzy inference system (DEFIS) [3] along with the results produced y proposed GFR-, are also listed in Tale. The results include the numer of fuzzy rules (for AFIS and DEFIS), or neurons in the hidden layer (for RBF and MLP), the testing RMSE (root mean square error), and the testing MAE (mean asolute error). All experimental results reported here are ased on 0 cross-validation experiments with the same model and parameter values, and the results are averaged. In each experiment, 70% of the whole data set is randomly selected as training data and another 30% as testing data.

5 The proposed model, GFR-, performs etter than the other neural networks and fuzzy models. This is a result of the fine tuning of each local model in GFR- and the fine aggregating of these local models. Tale. Experimental results on GFR data Model eurons or rules RMS E MAE MDRD MLP AFIS DEFIS RBF GFR-.3 (Average) Conclusions This paper presents a knowledge ased neural network LR-KF for the integration of sumodels and new data related to the same prolem resulting in incrementally adaptive model. The GFR- model is an application of LR-KF for the GFR Estimation. The GFR- performs local generalization and its learning is a procedure of knowledge-insertion, knowledge-modification and knowledge extraction. The new knowledge can e extracted from the GFR-: () In each local area, there is a modified MDRD function (or su-model) that can represent the knowledge est in this area. () For a new input vector, the contriution, each local function gives to the output, can e known. Further directions for research include: () in each local area, the est one will e selected as the knowledge function from several different formulas, e.g. MDRD, CG, Gates and Walser [4]; () the weighted data normalization [6] will e applied to the GFR-. Acknowledgements The research presented in the paper is funded y the ew Zealand Foundation for Research, Science and Technology under grant ERF/AUTX0-0. The authors acknowledge the assistance of Dr Mark Marshal from the Middlemore hospital in Auckland for providing data and expertise for the analysis and the validation of the results. References [] Fuzzy Logical Toolox User s Guide, The Math Works Inc., ver., 00. [] Jang, R. AFIS: adaptive network ased fuzzy inference system, IEEE Trans. on Syst.,Man, and Cyernetics, vol. 3, o.3, pp , 993.

6 [3] Kasaov,. and Song, Q. "DEFIS: Dynamic, evolving neural-fuzzy inference systems and its application for time-series prediction," IEEE Trans. on Fuzzy Systems, vol. 0, pp , 00. [4] Levey, A.S., Bosch, J,P., Lewis, J.B., Greene, T., Rogers,., Roth, D. for the Modification of Diet in Renal Disease Study Group, A More Accurate Method To Estimate Glomerular Filtration Rate from Serum Creatinine: A ew Prediction Equation, Annals of Internal Medicine, vol. 30, pp , 999. [5] eural etwork Toolox User s Guide, The Math Works Inc., ver. 4, 00. [6] Song, Q. and Kasaov,. "Weighted Data ormalizations and feature Selection for Evolving Connectionist Systems Proceedings, Proc. of The Eighth Australian and ew Zealand Intelligence Information Systems Conference (AZIIS003), pp , Sydney, Australia, Decemer, 003.