ISSN - 77-0593 UNAAB 00 Journal of Natural Scences, Engneerng and Technology ERROR RATES STABILITY OF THE HOMOSCEDASTIC DISCRIMINANT FUNCTION A. ADEBANJI, S. NOKOE AND O. IYANIWURA 3 *Department of Mathematcs, Kwame Nkrumah Unversty of Scence and Technology, Kumas, Ghana. Department of Appled Mathematcs and Computer Scence, Unversty for Development Studes, Navrongo, Ghana. E-mal: nokoe_bomaths@yahoo.co.uk. 3Department of Statstcs, Unversty of Ibadan, Ibadan, Ngera. Emal: jo_yanwura@yahoo.com *Correspondng author: tnuadebanj@yahoo.com ABSTRACT In ths study the stablty of the observed error rates of the homoscedastc dscrmnant functon relatve to the number of parameters n the model usng smulated data from multvarate normal populatons was nvestgated. Three models were consdered, the four, sx and eght varables models, each havng four values of the separator functon ( ). Equal and unequal pror probabltes were consdered for the dfferent number of parameter and separator functon confguratons. The asymptotc performance of the models was consdered usng the cross valdaton error rate estmaton procedure. Results ndcate the sx varable models as beng more stable (dsplayng less varablty n the estmated error rates) than the other models under consderaton. Less deteroraton was observed for the sx-varable model specfcaton as was evdent n the other models and ths was more pronounced for smaller values of. Keywords: Homoscedastc, Dscrmnant functon, pror probabltes, asymptotc. 000 Mathematcs Subject Classfcaton: 6H30, 65C0 INTRODUCTION Gven two or more groups of populatons and a set of assocated varables, one wants to locate a subset of the varables and assocated functons of the subset that leads to maxmum separaton among the centrods of the groups. The exploratory multvarate procedure of determnng varables and a reduced set of functons called dscrmnants or dscrmnant functons s called dscrmnant analyss. Dscrmnants that are lnear functons of the varables are called J. Nat. Sc. Engr. Tech. 00, 9():6-3 6 Lnear Dscrmnant Functons (LDF) or homoscedastc dscrmnant functon (derved from the assumpton of homoscedastcty of the varance-covarance matrx). In ths study, estmaton of the stablty (determned as a measure of wthn sample varablty) n the error rates of the dfferent models under consderaton s of nterest. Ths s an overall ndcator of the performance of the dscrmnant functon.
A. ADEBANJI, S. NOKOE AND O. IYANIWURA 3 Observatons are drawn from two multvarate normal populatons (groups) The Model MATERIALS AND METHODS R (,, R In the classcal homoscedastc model, the ). The mean vectors of lkelhood rato classfcaton functon for an p R (0,...,0) and are gven as observed vector x s derved as p f (,0,...0) ( x) exp x ( ) ( ) ( ) and respectvely and f( x) both matrces have dentty varance covar- pxp ance structures and. The performance of the functon s not dependent on the locaton of n the mean vector. Under a homoscedastc normal model for the group condtonal dstrbutons of the feature vector X on an entty, t s assumed that X () The th group-condtonal densty f s gven as p ( ) exp ( X ) ( X ) () where MNV (, ) (, ) f (, ) ( ;, ) conssts of the elements of {( ) ( )} / (3) (4) We assgn a new p-varable observaton x to R f P( P ) and the dstnct elements of. The square root of the Mahalanobs dstance s predetermned as,3,5 and 7 usng ( x ( )) ( ) k (otherwse assgn to ) (5) k log( C( ))/( C( )). (, ) s the pror probablty of an observed vector comng from populaton and C( j) C s the cost ncurred when an observaton from R R j s classfed as havng R j, j, come from ( ),. Thus C( ) C( j j) 0. In ths study, we assume equal cost of msclassfcaton and k reduces to the log of the rato of the pror probabltes. It may be remarked that a Bayes rule may result n a large probablty of msclassfcaton and several attempts have been made to overcome ths dffculty. When pror probabltes are known, the Bayes rule s optmum n the sense that t mnmses average expected cost (Gr - 004). For an observed vector x, f the plug-n rule J. Nat. Sc. Engr. Tech. 00, 9():6-3 7
s gven as approxmaton to the Bayes rule F s a good estmate of F. r (.) 0 s the classfcaton rule obtaned when sample X, X, estmates of and S of are plugged nto (5). The probablty that a randomly chosen en- R R j tty from s allocated to on the ba- r0 ( x; F) ss of, has an error rate specfc to the th group as And the overall error rate s, j, ( ) (6) (7) where s as earler defned. In usng error rates to measure the performance of a sample-based allocaton rule, t s the condtonal error rates that are of prmary concern once the rule has been formed from the tranng data (Johnson and Wchern-998). The overall error rate for equal prors, s gven by where and r0 ( x; F), ths provdes a good g j eo( F) eo ( F) g eo( F) eo ( F) j s as earler defned. r0 ( x; F) ef ( ) eof ( ) n { ( )/4}{ p 4( p) O } eo( F) ( ) ERROR RATES STABILITY OF THE HOMOSCEDASTIC... If the dmenson p s small, the sample szes n, n occurrng n practce wll probably be large enough to apply ths result. However, f p s not small, extremely large sample szes f wll probably be requred to make ths results relevant (Gr 004). (8) (9) J. Nat. Sc. Engr. Tech. 00, 9():6-3 8 The leave-one-out (cross valdaton) error rate estmaton procedure of Lachenbruch and Mckey (968) s used as the performance evaluator for the models under consderaton. The smulaton study From (7), we can determne approxmately how large n must be for a specfed and p n order for the uncondtonal error rate not to exceed too far the best obtanable as gven by the optmal error rate. Indeed, f n s small, then for p>, the error rate s not far short of ½, whch s the error rate for a randomzed rule that gnores the feature vector and makes a choce of groups accordng to the toss of a con (McLachlan 99). The sample szes have to be specfed, here we set the values of n at 5, 50, 75, 00, 00, 300, 400, 500, 600, 700, 800, 900, 000, 00, 00,,300, 400, 500, 000, 500, 3000 respectvely. The sze of n s decded by the predetermned sample sze ratos n :n. The ratos are : and : thus determnng the pror probabltes to be consdered. Numercal values were assgned to, the group centrod separator factor and Mahalanobs dstance determnant. These are, 3, 5 and 7 respectvely. The number of varables n the mult-normal dstrbutons to be generated s predetermned as 4, 6 and 8 followng Murray (977) that consdered the selecton of varables n
A. ADEBANJI, S. NOKOE AND O. IYANIWURA 3 Dscrmnant Analyss. By makng 00 replcates we am at attanng an accurate estmate of the msclassfcaton rate by reducng the between sample varablty. Hence 00 samples of random varates of the requred specfcaton are generated, and the analyss s carred out on the 00 samples. Thus resultng n,00 samples for each sample rato consderaton and four values of. Ths gves a total of 400x4 (6,800) samples of varous szes. The error rate estmates are then averaged over the number of replcates. The SAS V8 (996) package was used for generatng the matrces from a N (0,) dstrbuton for the predetermned models. Independent seres of normal devates of requred length are drawn and then transformed (standardzed) to have unt varance and zero covarance. Smlar smulaton experments had been constructed by Marks and Dunn (976) and He and Fung (000). These dd not consder varable effects and number of replcates and sample szes not ths many. Results of smulaton The total probablty of msclassfcaton, standard devaton and coeffcent of varaton are presented as decmals. The results of the smulaton are presented n a seres of fgures The frst set (. to.6) present the results for the four varable model wth equal pror probablty scheme presented n fgs. to.3 for dfferent values of. Fgures. to.3 are the total error rates, the standard devaton (SD) and coeffcent of varaton (CV) respectvely. Fgures.4 to.6 present the same results for the unequal pror probablty (n : n = J. Nat. Sc. Engr. Tech. 00, 9():6-3 9 :). The second seres of fgures are for the sx (6) varable model and are presented n the same format (that s equal and unequal pror probabltes) as fgures. to.3 and.4 to.6 respectvely. The last seres of fgures are for the eght (8) varable model and are presented n the same sequence as fgures 3. to 3.6 respectvely. In the equal pror scenaro, for the four varable model, error rates are less stable for the smaller sample szes than for the larger sample szes as evdenced n the coeffcents of varaton recorded. Stablty n the error rates deterorates remarkably asymptotcally as ncreased from to 7 and worsens when =7 when the samples are so far apart that t does not justfy the use of a classfcaton functon. Changng the number of varables n the model to sx resulted n some observable dfference n the performance of the LDF. A reducton n error rate for the respectve values was recorded. When =, the values recorded for the standard devaton were consstently lower than those for the mean total error rate. The reducton of the CV s rapd. The hghest value of 35.8% error rate was observed for sample sze 50 (rato :) and the smallest recorded was 8.59% for sample sze 6000 (rato :4). Reducton n error rates was more rapd for rato : than for :. When compared wth = (p=4 varables), the mnmum and maxmum recorded values are qute close. The four varable model (p=4) had values 33.76 and 8.6% for maxmum and mnmum msclassfcatons respectvely.
ERROR RATES STABILITY OF THE HOMOSCEDASTIC... J. Nat. Sc. Engr. Tech. 00, 9():6-3 0
A. ADEBANJI, S. NOKOE AND O. IYANIWURA 3 J. Nat. Sc. Engr. Tech. 00, 9():6-3
When =3, the relatonshp between the mean error rates and CV s reversed (CV records hgher values). The maxmum error rate of 8.6% was recorded for sample sze 50 (rato :) and the CV results recorded a maxmum of 0.9% for sample sze 800 (rato :).The recorded error rates are however much lower than was earler recorded for lower values of (especally when vewed n comparson wth the four varable lnear models). When =7, the values of CV are much hgher than earler recorded as a result of the hgh reducton n mean error rate; although lttle mprovement was observed across sample szes, the functon recorded near zero msclassfcaton rates. When the number of varables s ncreased to eght (p=8), the observed pattern s smlar to the pattern observed for P=6 wth the mean error rates recordng hgher values than the CV for lower values of.the maxmum value of mean error rate observed s 36.67% for sample sze 50 (rato :) and reducton n CV s not as rapd as earler recorded and the reducton n mean error rate s much more gradual than earler observed. At =3, the values of CV are larger than the mean error rate except at the tal end of the curves. A maxmum error rate of 8.% s recorded for sample sze 50 (rato :) However, the asymptotc reducton n CV s no longer observed for =5. Here, an ntal reducton s observed after whch the values stablzes. The mean error rates are close to zero. ERROR RATES STABILITY OF THE HOMOSCEDASTIC... Improvement n the performance of the functon s more pronounced for the eght varable model than the four varable case. Maxmum error rate of 0.56% was observed for sample sze 00 (rato :3) and mnmum value of 0.0% for sample sze 50 (rato :) was observed for =6 for =7, the values were 0.044% for sample sze 50 (rato :4) and mnmum value of 0.04% for sample sze 50 (rato :). DISCUSSION There appears a turnng pont n the reducton n the mean error (or msclassfcaton) rates as well as mprovement n ther stablty beyond p = 6. Ths s suggestng that not more than sx varables should be ncluded n dscrmnant analyss even when the sample sze s as large as 5000. A reasonable corollary to ths fndng s the plausble concluson that the smaller your sample szes the fewer should be the number of varables. Ths declne n stablty of observed error rates s observable for hgher values of wth =7 recordng much hgher nstablty than =5. Also, ncreasng the sample sze wll cease to result n an mprovement n the performance of the functon once a thresh-hold s reached, beyond whch, there s nothng to be ganed by any further ncrease even to values that gve sample estmates that would equal the populaton parameters. J. Nat. Sc. Engr. Tech. 00, 9():6-3
A. ADEBANJI, S. NOKOE AND O. IYANIWURA 3 CONCLUSION These nconsstences n the behavour of average error rates are the observed deteroraton n the stablty of the error rates as p changes from 6 to 8 suggest that there s a turnng pont between p = 6 and p = 8 n the relatonshp between the number of varables and the magntude of error rates and ther varaton. Ths s most plausble at p = 7. Ths pattern was observed for the dfferent values of that we consdered. Joossens, K. 006. Robust Dscrmnant Analyss; Ph.D. Thess of Katholeke Unverstet, Leuven. Lachenbruch, P.A., Mckey, M.R. 968. Estmaton of error rates n dscrmnant analyss. Technometrcs, 0: -. Marks, S., Dunn, O.J. 974. Dscrmnant Functons when covarance matrces are unequal; Journal of the Amercan Statstc Assocaton, 69: 555-559. ACKNOWLEDGEMENT Ths study was supported by the Thrd World Organzaton for Women n Scences (TWOWS) fellowshp. REFERENCES Adebanj, A.O., Nokoe, S. 004. Evaluatng the Quadratc Classfer; Proceedngs of the Thrd Internatonal Workshop on contemporary problems n Mathematcal Physcs, P. 369-394. Gr N.C. 004. Multvarate Statstcal Analyss. DEKKER Seres 004 P. 435-477. He, X.M., Fung, W.K. 000. Hgh breakdown estmaton for multple populatons wth applcatons to dscrmnant analyss; Journal of Multvarate Analyss, 7: 5-6. McFarlan, R.H., Rchards, D. 00. Exact Msclassfcaton Problems for Plug-n Normal Dscrmnant Functons. Equal Mean Case. Journal of Multvarate Analyss, 77: - 53 McLachlan, G. 99 Dscrmnant Analyss and Statstcal Pattern Recognton. Wley Seres n Probablty and Mathematcal Statstcs. P. 4-64. Murray, G.D. 977. A Cautonary Note on Selecton of Varables n Dscrmnant Analyss. Appled Statstcs, 6(3): 46-50. Okomato, M. 963. An Asymptotc Expanson for the Dstrbuton of the Lnear Dscrmnant Functon. Annals of Math Stat., 34: 86-30. Johnson, R.A., Wchern, D.W. 998. Appled Multvarate Statstcal Analyss, 4 th Edton. Prentce Hall Inc. USA. P. 69 73. (Manuscrpt receved: 6th January, 00; accepted: 4th June, 00). J. Nat. Sc. Engr. Tech. 00, 9():6-3 3