Onyeagu, Sidney I. Nnamdi Azikiwe University, Awka, Nigeria. Okeke, Joseph Uchenna Anambra State University, Uli, Nigeria

Transcription

1

2 Comparatve Study Of Double Dscrmnant Analyss And Logstc Regresson Based On Bnary And Contnuous Varables By Okonkwo, Evelyn Nkruka Nnamd Azkwe Unversty, Awka, Ngera Onyeagu, Sdney I. Nnamd Azkwe Unversty, Awka, Ngera Okeke, Joseph Uchenna Anambra State Unversty, Ul, Ngera Nwabueze, Joy Choma. Mcheal Okpala Unversty of Agrculture,Umudke Ngera Ogbonna, Blessng Nnamd Azkwe Unversty, Awka, Ngera ABSTRACT In the classfcaton of an observaton consstng of both bnary and contnuous varables, double dscrmnant analyss and logstc regresson had been consdered approprate by most researchers. In ths study, these two technques were etensvely dscussed and compared usng two real lfe data sets. The average value of PMC for the two data sets showed that logstc regresson s optmal to double dscrmnant analyss n classfyng objects whose eogenous varable consst of dscrete and contnuous varables. Key words: Double dscrmnant analyss (DDA), Logstc regresson, Regressor, and Probablty of msclassfcaton..0 INTRDUCTION Logstc regresson allows one to predct outcome such as group membershp from a set of varables that may be contnuous, dscrete, dchotomous, or a m (Tabachnck and Fdell, 996). The problem of dscrmnant analyss arses when one wants to predct group membershp on

3 the bass of feature vector. From the above two sentences, t s obvous that the same research questons can be answered by both methods. The logstc regresson may be better sutable for cases when the dependant varable s dchotomous such as yes/no, pass/fal, nfected/not nfected, defectve/good lfe/death, etc., whle the ndependent varable can be on any scale. The dscrmnant analyss mght be better suted when the dependant varable has two groups or more wth the requrement that the ndependent varables wll be normally dstrbuted, lnearly related, and each group has the same varance and covarance for the varables. Several authors have formally compared these two technques. For eample, Halpern et al (97) obtaned results from several attrbuted type eplanatory varable and noted only small dfference n the classfcaton ablty between the two analytc procedures. Press and Wlson (978) concluded that each analytc technque served a unque functon: dscrmnant analyss was useful for classfcaton of observatons nto one or two or more populatons, whereas logstc regresson was useful for relatng a qualtatve (bnary) dependent varable to one or more eplanatory varables by a logstc dstrbuton functonal form of P(E). Klenbaum et al (98) compared classfcaton ablty of logstc regresson and dscrmnant analyss and noted that logstc model was slghtly superor. Afuecheta et al (00) compared these two methods on three data sets of normal and nonnormal data and concluded that logstc regresson s the more fleble and more robust method n the case of volaton of lnear dscrmnant assumptons. The objectve of ths paper s to compare the performance of double dscrmnant functon obtaned usng pont-bseral model developed by Chang and Aff (974), and logstc regresson based on bnary and contnuous eplanatory varables for classfyng subjects nto one of two populatons..0 DOUBLE-DISCRIMINANT ANALYSIS An observaton consstng of both bnary and contnuous varables may be classfed nto one of two populatons by the double-dscrmnant functon based on the pont-bseral model. When the parameters are unknown or partally known, a sample double-dscrmnant functon s obtaned by replacng the unknown parameters by ther sample estmates.

4 X W Y Suppose an observaton s to be classfed nto one of two populatons, =, where Y s p vector of contnuous varates and X s a Bernoull varate wth P(X=) and P(X=0) = f W belongs to. We assume that W follows a pont-bseral model, that s, that the condtonal N(, ) dstrbuton of Y gven X (0 or ) s when W, where s a pp postve defnte matr.. Under the pont-bseral model, and gven X, the lkelhood rato procedure s to classfy the observaton W nto f C [ Y ]' k ln ( ) /, where (.) = 0,, and k s a gven constant. Otherwse the observaton s classfed nto.the dscrnnant functon n (.) s called the double-dscrmnant functon by Chang and Aff (974). If the parameters are unknown, we may replace them by ther sample estmates. Let X j W j,,and j,,... N Y j be two sequences of observaton vectors ndependently drawn from. We shall add a subscrpt on ndcate those values correspondng to. The sample doubledscrmnant functon s defned accordng to whether s known or unknown. When s known but, and are unknown, the sample double-dscrmnant s where X j T [ Y Y Y ]' S Y Y k Y j and (.) N C to N j j Y N Y, N N X j j 3

5 N N N, 0 N j j j. S M Y Y Y Y M N N N s number of observatons n populaton, and N s the number of observatons n (or 0) class of th populaton. When all the parameters are unknown, the double-dscrmnant functon s U [ Y Y Y ]' S Y Y k N ] ln N NN. ', (.3) It can be easly shown that T and U are nvarant under any nonsngular lnear transformaton of the Y s. As sample szes N tend to nfnty,, Y S, and n probablty. Hence the lmtng dstrbuton of T tends to that of C (Tu N D, D 978), or from or,where N D, D D = ' (.4) dependng on whether W comes It should be noted that some of the N may assume small values f s close to zero or one. Effort should be to ensure that N > p (the number of parameters). 3.0 Logstc Regresson Logstc regresson s part of a category of statstcal models called generalzed lnear models (Agrest, 996). Logstc regresson, more commonly called logt model, deals wth the bnary case, when the response varable s dchotomous (.e., bnary 0 or ). The predctor varables may be quanttatve, categorcal, or a mture of the two. Ths model s manly used 4

6 to dentfy the relatonshp between one or more eplanatory varables X and response varable Y. It has been used for predcton and determnng the most nfluencng eplanatory varable(s) on the varables (Co and Snell, 994). Instead of a straght lne, logstc regresson seems preferable to ft some knd of sgmod curve to the observed ponts. The tals of sgmod curve level off before reachng P(E) =0 or P(E) =, so that the problem of mpossble values of P(E) s avoded. The basc form of the logstc functon s P Z e (3.) where Z s the predctor varable(s) and e s the base of the natural logarthm, equal.788,p s estmated probablty of event occurrng. In the multvarate case, Z nstead of beng a sngle predctor varable, s a lnear functon of a set of predctor varables: Z b b X b X b X 0... p p. (3.) 3. One Regressor Assumng that we have a sngle regressor, let us try to wrte a smple lnear regresson model as Y X 0 (3.3) We would logcally let Y = 0 f the unt does not have the characterstc that Y represents, and Y = f the unt does have the characterstc. It thus follows that can also take on only two values: 0X f Y = and - 0 X f Y = 0. Therefore cannot be appromately normally dstrbuted. Consequently, the model (3.3) s napplcable for a bnary dependent varable. In smple lnear regresson, the startng pont n determnng a model s a scatter plot of Y. Consequently; t s necessary to consder other plots. One such plot s a plot of E(Y\X) aganst X. Rather than plottng ponts, we must postulate a relatonshp between Y and X varables snce the ordnate of the plot s not related to the data. It s customary to let E(Y \X ) =,whch s P(Y = ), where Y s bnomal. Here represents the probablty of, for eample, someone dyng wthn a stated tme perod who has B.P gven by 5

7 X. Gven one regressor the probablty of an event, say E, for a gven value of X, P(E) = ( X ) s ep 0 X X ep 0 X (3.4) The model gven by (3.4) satsfes the mportant requrement that 0 and wll be satsfactory model n many applcatons. The model n terms of Y would be wrtten as Y X It follows from (3.4) that so ep 0 X log X 0 (3.5) Snce (3.5) results from usng a logstc transform (also called a logt transform), the model s called a logstc regresson model. The left sde of (3.5) s called log odds rato, and ths can be eplaned as follows. Snce = P(Y=), t follows that - = P(Y=0), and so s the rato of the two probabltes, whch, when stated n the form the of odds, gves the odds of havng Y=, for a gven value of X. Odds are frequently stated n terms of aganst rather than for so the odds aganst havng Y= would be obtaned from. The absence of error on the rght sde of (3.5) s because the left sde s a functon of (Y/X), nstead of Y, whch serves to remove the error term. The nterpretaton of s naturally somewhat dfferent from the nterpretaton n the lnear regresson. In (3.5) obvously represents the amount by whch the log odds change per unt change n X. Ths mples that a unt ncrease n X ncreases the odds by the multplcatve factor e. 4.0 DATA AND THEIR ANALYSIS 4. The Data for ths study are two real lfe data sets. One set was collected from Amaku General Hosptal, Awka, Anambra state, Ngera. The data s on fastng and non-fastng blood sugar level (FBS and NFBS respectvely) 6

8 of dabetcs and non-dabetcs patents wth ther gender randomly selected from the cases reported at the Hosptal n 009. The laboratory reference ranges for adults are: Glucose (fastng): 9 4mmol/l and Glucose (non-fastng):4 8mmol/l The other set s on four albno CD- Sprague-Dawley rats at the weanng age wth smlar weghts. The data s avalable n Tu and Han (98). 4. FINDINGS 4. Result of dabetc and non-dabetc patent data Double dscrmnant analyss The sample double dscrmnant functons (DDF) are: T 0 =.4773y +.379y for male. T =.3330y y for female. The above two sample double dscrmnant functons when appled on the orgnal data gave the probablty of msclassfcaton as: 0.05 Result of Logstc Regresson The logstc regresson model for the data s Z= y y The p- value of the model s: The probablty of correct classfcaton s.00. Here y=fastngs blood glucose level, y= non-fastngs blood glucose level, and = se 4. Result of four albno CD- Sprague-Dawley rats The sample double dscrmnant functons (DDF) are: 7

9 T 0 = y y for male. T = y y for female. The above two sample double dscrmnant functons when appled on the orgnal data gave the probablty of msclassfcaton as: 0.37 Result of Logstc Regresson The logstc regresson model for the data s Z= y y.67589x The p- value of the model s: The probablty of correct classfcaton s 0.74 wth tes of 0.04 and the probablty 0f msclassfcaton s 0. Here y=body weght, y= total length, and = se Summary of fndngs The frequences of msclassfcatons are lsted n the followng table. Table 4. Sample data DDF Logstc Dabetc and non-dabetc patents Albno CD-Sprague-Dawley rats Average CONCLUSIONS AND DISCUSSION. Double dscrmnant analyss (DDA) and logstc regresson are used when the observatons of eogenous varable consst of bnary and contnuous 8

10 varables wth dchotomous dependent varable. We gave etensve dscusson on the smlartes and dssmlartes of the two methods n the lterature. From the average value of probablty of msclassfcaton of and 0.3 for DDA and logstc regresson analyss respectvely, we can conclude that logstc regresson s optmal to DDA n classfyng objects whose eogenous varable consst of dscrete and contnuous varables. Reference. Afuecheta, E.O., Ogum, G.E.O., Osuj, G.A., and Utaz, C.E. (00). Comparson of Lnear Dscrmnant Analyss and Logstc regresson n Classfcaton Problems. Conference Proceedngs Ngera Statstcal Assocaton, Agrest, A. (996). An ntroducton to Categorcal Data Analyss. John Wley& Sons, Inc., New York. 3. Chang, P.C., and Aff, A. A.(974). Classfcaton Based on Dchotomous and Contnuous Varables. Journal of the Amercan Statstcal Assocaton, 69: Co, D.R., and Snell, E.J. (994). Analyss of Bnary Data. Chapman& Hall, London. 5. Halperne, M., Blackwelder W.E., and Verter, J.I (97). Estmaton of the Multvarate Logstc Rsk Functon and Mamum Lkelhood Approach. Journal of Chron. Ds. 4: Klenbaum D.G., Kupper, L.L., and Morgenstern, H. (98). Epdemology Research: Prncples and Quanttatve Methods. 9

11 Toronto: Lfetme Learnng Krzanowsk, W.J. (993). Prncples of Multvarate Analyss. Oford Unversty Press Inc., New York Press, S.J., and Wlson, S. (978). Choosng Between Logstc Regresson and Dscrmnant Analyss. Journal of Amercan Statstcal Assocaton 73: Tabachnck, B.G. and Fdell, L.S. (996). Usng Multvarate Statstcs Harper Collns, New York. 0. Tu, C.T. (978). Dscrmnant Analyss Based on Bnary and Contnuou Varables Unpublshed Ph.D Dssertaton, Iowa State Unversty. Tu, C.T., and Han C.P. (98). Dscrmnant analyss Based on Bnary and contnuous varable. Journal of Amercan Statstcal Assocaton 77: