Detection of AUC and Confidence Interval Using Normal- Mixture ROC Curve

Transcription

1 IOSR Joural of Matheatcs (IOSR-JM e-issn: p-issn: X. Volue Issue Ver. III (Mar. - Apr. 6 PP Detecto of AUC ad Cofdece Iterval Usg Noral- Mxture ROC Curve Sudesh Pudr Azharudd Departet of Statstcs Podcherry Uversty INDIA Abstract: Recever Operatg Characterstc (ROC Curve s oe of the reowed statstcal techques to assess the accuracy of a dagostc test to dscrate betwee healthy ad dseased subjects. I ths paper two copoet Noral-Mxture ROC curve s studed ad ts propertes are dscussed to kow the characterstcs of the ROC Curve. Area uder the Curve (AUC of Noral-Mxture ROC Curve ad Cofdece Iterval of AUC are also derved. The proposed odel s valdated by usg the sulato studes. Keywords : Noral-Mxture dstrbuto ROC Curve AUC Cofdece Iterval of Area Uder the ROC Curve MLE of AUC of ROC Curve va Expectato Maxzato (EM algorth ad Mote Carlo sulato. I. Itroducto The Mxture dstrbuto arses where a statstcal populato cotas two or ore sub populatos. It takes to accout the heterogeety of the populato. For exaple edcal dagoss f we have a set of patets sufferg fro cacer dsease ad we ca also do further sub-classfcato aga of the patets sufferg fro stoach cacer blood cacer throat cacer eck cacer etc. So ths case the xture dstrbuto s used for cosderg the heterogeety of the populato. The fte xture dstrbutos are wdely used real lfe stuatos as a statstcal odel. Frst te the Noral-Mxture dstrbuto troduced s by Newcob (886 who showed the applcato of oralxture dstrbutos as odels for outlers. Pearso (894 dscussed the estato of paraeters of two copoet oral-xture dstrbuto by the ethod of oets. I hs paper Pearso estate the fve paraeter of the two copoet oral-xture dstrbuto. The frst coprehesve oographs of the xture dstrbuto are Evertt ad Had (98 ad Tttergto et al. (985. They dscussed the fte xture dstrbutos ad the dfferet estato ethods of xture dstrbutos. Mclachla ad Basford (988 dscussed dfferet xture dstrbutos. Bohg et al. (998 also dscussed the applcato of xture odel edce. McLachla ad Peel ( dscussed the fte xture dstrbutos. Schlatta (9 dscussed the applcato of fte xture dstrbuto edce. Wrjato ad Dgha (9 dscussed the applcato of oral xture dstrbuto face. They dscussed the axu lkelhood estate oet geeratg fucto swtchg regresso odel ad stochastc regresso odel by usg the oral-xture dstrbuto. Patrck et al. ( dscussed estato of fte xture dstrbuto by the ethod of oets. Dass ad Seog ( dscussed estato of ultvarate b-oral xture ROC Curve. Lee ( showed the Gaussa xture odel pathologcal voces. Goe (3 dscussed oral xture of Recever Operatg Characterstc Curve. He also dscussed that the dseased populato has hgher varablty tha ca be laed by a sgle dstrbuto. Hughes ad Bhattacharya (3 dscussed the syetrc propertes of Boral ad B-Gaa ROC Curves usg the Kullback-Lebler Dvergeces. The Recever Operatg Characterstc (ROC Curve s dscovered by the radar egeers durg World war II sgal detecto theory. Gree ad Swets (966 dscussed the ROC curve ad Ega (975 dscussed the ROC Curve. Karzaowsk ad Had (9 also dscussed the ROC Curve for cotuous data. The Area uder the ROC Curve geerally deoted AUC ad dscussed by Gree ad Swets (966. Baber (975 dscussed the area above ad below the ROC curve. Haley ad McNel (98 dscussed use of the area uder the ROC Curve. Bradley (997 also dscussed the use of the Area uder the Curve. Pudr S ad Aala R (3 dscussed varous aspects of ROC Curve cotuous data. Dorfa ad Alf (969 dscussed the axu lkelhood estato of paraeters sgal detecto theory ad they also dscussed how to fd the cofdece terval. Wrjato ad Xu (9 dscussed the axu lkelhood estate of oral-xture dstrbuto. Kaaruzzaa et al. ( also dscussed the axu lkelhood estate the oral-xture dstrbuto. They also dscussed the Expectato Maxzato algorth for estatg the paraeters. For fdg the ML estate of oral-xture dstrbuto we also used the EM algorth because the oral-xture dstrbuto have ot closed for. Depster et al. (977 dscussed the Maxu Lkelhood Estate of coplete data va EM algorth. Joshua (3 dscussed the EM algorth ultvarate Gaussa xture odels. Suppose X s a rado varable whch follows ay cotuous dstrbuto havg scores of patets to healthy ad dseased group of dvduals. For classfyg the patets to healthy ad dseased cases let us DOI:.979/ Page

2 Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve take a referece stadard or a threshold value (t. Let the dseased ad healthy dvduals are deoted by ad. The dvduals are regarded as dseased cases f the score (s exceeds the threshold value (t else they are regarded as healthy cases. To kow the perforace of classfer we defe four probabltes rates vz. True Postve Rate (TPR False Postve Rate (FPR True Negatve Rate (TNR ad False Negatve Rate (FNR. These are defed as follows: True Postve Rate s the probablty that a dvdual fro dseased populato s correctly classfed.e. TPR= p (s > t /. False Postve Rate s the probablty that a dvdual fro healthy populato s sclassfed.e. FPR = p ( s> t /. True Negatve Rate s probablty that a dvdual fro healthy populato s correctly classfed.e. TNR = p ( s t /. False Negatve Rate s the probablty that a dvdual fro Dseased populato s sclassfed.e. FNR = p( s t /. Moreover the ROC Curve focuses oly o the probabltes such that s > t the two populatos the for all saller values of t we ust have TPR= whle FPR vares fro to ad for all larger values of t we ust have FPR= whle TPR vares fro to. The coordates of the area uder the curve lyg betwee [] [] ad [] correspod to the ROC space. The ROC curve that falls ear to [] get axu accuracy. The ROC Curve s a graph that shows True Postve Rate o the vertcal axs ad False Postve Rate o the horzotal axs as the classfcato threshold t vares. ROC Curve have bee extesvely used the evaluato of dagostc test. We have two groups of dvduals aely dseased cases ad healthy cotrols. ROC Curve s defed as - y( t = -G[ F ( - x( t ] x ( t (. where x (t deote the -specfcty or False Postve Rate (FPR ad y(t deote the sestvty or True Postve Rate (TPR. Specfcty s also defed as the proporto of patets wthout dsease who test egatve ad Sestvty s defed as the proporto of patets wth dsease who test postve. G ad F deote the dstrbuto fucto of the healthy cases ad dseased cases. The perforace of the curve s easured by the area uder the ROC Curve. The area uder a ROC Curve assesses the overall ablty of the test to dscrate betwee healthy cases ad dseased cases. The atheatcal defto of AUC s AUC Py x y(tdt (. A test havg a area of.5 s a worthless test ad a perfect test has a area of.. The ROC Curve ust satsfy the followg propertes.. ROC Curve ad AUC s ualtered wth respect to ootoe creasg trasforato of the test scores.. The test values of X are saller tha Y. 3. ROC Curve s ootocally creasg fucto.e. dy(t dx(t 4. ROC Curve s sad to be covex f d y(t dx (t ad cocave f d y(t dx (t. 5. The slope of ROC Curve at ay operatg pot s equal to the rato of PDF of dseased to PDF of healthy at a partcular cut-off pot t s gve by p( t slope = (.3 q( t 6. For syetrc ROC Curve f p(x ad q(x are the two cotuous probablty dstrbuto the Kullback- Lebler dvergece s a o-syetrc easure of two probablty dstrbutos of p(x ad q(x. Let p(x deote the healthy dstrbuto ad q(x deote the dseased dstrbuto the KL(pq deote the Kullback-Lebler (K- L dvergece betwee the dstrbutos of dseased ad healthy group wth p(x as the coparso dstrbuto ad q(x as the coparso referece. The KL p q px D p l q x x dx (.4 Slarly let KL(qp deote the K-L dvergece betwee the dstrbuto of healthy ad dseased group wth q(x as the coparso dstrbuto ad p(x as the referece dstrbuto the DOI:.979/ Page

3 Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve x x q KLq p qxl dx (.5 D p where D s the coo rage of p ad q. It should be oted that f KL(pq ad KL(qp are postve ad KL(pq = KL(qp = f ad oly f p(x = q(x. these two easure gves the asyetry of ROC Curve about the dagoal. If KL(pq < KL(qp the the ROC Curve s sad to be True Postve Rate asyetrc ad f KL(pq > KL(qp the the ROC Curve s sad to be True Negatve Rate asyetrc. I ths paper there are sx sectos. I Secto we dscussed the Noral Mxture ROC (NMROC Curve ad ts propertes are dscussed to kow the behavor of the oral-xture ROC Curve. To kow the accuracy of ROC Curve Area uder the oral-xture ROC curve s also foud. I Secto 3 we foud the axu lkelhood estator of AUC of oral-xture ROC curve. I Secto 4 cofdece terval of Area Uder the NMROC Curve s derved. We dscussed the sulato studes by takg a uercal exaple for the valdato of the odel Secto 5. Secto 6 cossts of the cocluso. II. Noral Mxture ROC Curve Let X be a rado varable whch follows oral-xture dstrbuto ad t coes fro healthy cotrols. It has probablty desty fucto f x pf x - p f x (. where μ μ σ ad σ are the eas ad varaces of the oral xture dstrbuto. Frst subscrpt paraeter shows that the t s cog fro st ad d oral desty ad secod subscrpt shows that t s cog fro healthy cotrols. p ad -p are the weghts of the xture dstrbuto. The su of weght should be equal to xture dstrbuto. Slarly Y be a rado varable whch follows oral-xture dstrbuto ad t coes fro dseased cases. It has probablty desty fucto g(y pf y - p f y (. where μ μ σ ad σ are the eas ad varaces of oral xture dstrbuto. Frst subscrpt shows that t s cog fro st ad d oral desty ad secod subscrpt eas that t s cog fro dseased case. p ad -p are the weghts of the xture dstrbuto. The two copoet oral-xture ROC (NMROC curve s defed as y t p xt p xt DOI:.979/ Page where s the cdf of oral xture dstrbuto ad - t ( t p t - p x - < t < - < < > = j=. - Before plottg ROC curve oe should check the followg assuptos. The ea of the healthy cotrols whch follows oral xture dstrbuto should be greater tha the ea of the dsease cases whch also follows oral xture dstrbuto oral-xture ROC odel.. If > > ad > > hgher the Area uder the NMROC curve. 3. All propertes of NMROC Curve should be satsfed. 4. Attach ore weghts to the desty fucto whch have hgh ea ad hgh varace. Propertes. NMROC Curve s ootocally creasg fucto. Proof : A fucto s sad to be ootocally creasg fucto f the frst dervatve of the fucto s postve. Dfferetatg (.3 wth respect to x we get dy dx t t p p x t x t - < x < - < ad > j j (.3 xt - p> < j j =. (.4

4 Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve where s the desty fucto ad Ф s the dstrbuto fucto of oral-xture dstrbuto. Hece NMROC Curve s ootocally creasg fucto.. NMROC Curve s cocave. Proof : A fucto s sad to be cocave f the secod dervatve s egatve. Dfferetatg (.3 wth respect to x.e. p - p - x t x t d yt dx t - xt - p - - p - - x t x t x t p > - < x < - < μj < ad j > =. 3. The slope of the NMROC Curve at the threshold t s gve by t - p - - p dyt dxt t - p - - p - - t - t - 4. NMROC curve s varat wth respect to ootocally creasg trasforato of the test scores. 5. The NMROC Curve s TNR asyetrc. Proof:The K-L dvergece betwee the dstrbuto of dseased ad healthy group wth p(x as the coparso dstrbuto ad q(x as the referece dstrbuto s gve as - - x x l l l p - - l - l - p ( p x p x KL p q - - x x l p - p (.5 (.6 (.7 Slarly the K-L dvergece betwee the dstrbuto of healthy ad dseased group wth q(x as the coparso dstrbuto ad p(x as the referece dstrbuto has bee gve as KL( q p l l l p x - l p - p x - - p x - x - - l - l - p x - - p x - (.8 It s foud that KL(qp > KL(pq. Hece NMROC Curve s TNR asyetrc. Optal cut-off value I edcal dagoss the optal cut-off value.e. t decdes about the codto of patet whether he s healthy or have dsease. Frst te t s troduced by the Fluss et al. (5. He appled the optal cut-off value the youde dex whch s obtaed by takg the axu dfferece betwee F(t ad G(t where F(t represets the dstrbuto fucto of healthy cotrols ad G(t represets dstrbuto fucto of dsease cases. The optal cut-off value of NMROC Curve s gve as t ax F t G t t l l p ax pax t t DOI:.979/ Page (.9

5 Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve The AUC of two copoet NMROC curve s derved as - - AUC p p. (. III. Maxu Lkelhood Estate of paraeters of NMROC curve va EM algorth For assessg the dagostc accuracy there are three approaches of estatg the ROC Curve- Paraetrc o-paraetrc ad se-paraetrc approaches. Here we dscuss the paraetrc approach for estatg the ROC Curve. The probablty desty fucto of the oral-xture dstrbuto s k f x p j N k ( j j p j = p j j j= ad the CDF of the oral-xture dstrbuto s F x k x j p j j j. For k= the pdf of two copoet oral-xture dstrbuto s gve as p x - - p x - f xp - -. (3. The lkelhood fucto of the two copoet oral-xture dstrbuto s p x - - p x - L - -. (3. The log-lkelhood fucto of the (3. s as follows p x - - p x - l L -. (3.3 For axzg the log-lkelhood fucto dfferetatg (3.3 wth respect to p μ μ σ σ ad equate the to zero we get L - - l p l L l L l L l L - p - - p - p - - x x - - x - x - x x x - p x x 3 p x p (3.4 (3.5 (3.6 (3.7 x x (3.8 DOI:.979/ Page

6 where p - Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve x - - p x - - We observe that the lkelhood equato of two copoet oral-xture dstrbuto s ot a closed for. If the equato s ot foud a closed for we wll use the Expectato-Maxzato (EM algorth to estate the paraeters. I ths ethod there are two steps to get the estates usg EM algorth. ( Take ectato of the lkelhood fucto ad ( Maxze the log lkelhood fucto. Suppose X s a xture data wth N observatos the lkelhood of the data assug that depedetly dstrbuted s gve as f X L X p f N K k k x k. x are (3.9 The proble of xture estato fro data X ca be forulated as to fd the set of paraeters that gves the axu lkelhood estate (MLE soluto * arg ax L X. (3. The suato sde the product (3.9 checks the possblty of aalytcal solutos. Oe alteratve s to axze the coplete lkelhood a ectato-axzato (EM algorth. So after axzato of the of (3.4-(3.8 we get the estates as follows - x x (- y í - (- í y - x - (x y - y - ad. - (3. For healthy cotrol we used weght p px ; x ; - p x ; ad p = ad for dseased cases the weght s p = where p py; ; - ;. y p y Substtutg axu lkelhood estates fro (3. (. we ca get the estate of AUC. IV. Cofdece Iterval of AUC of NMROC Curve The estated AUC of two copoet NMROC s gve as AUC p p p p DOI:.979/ Page (4.

7 where ad Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve (4. The cofdece terval of AUC of NMROC Curve caot be foud drectly because t s a xture of two copoet CDF fucto. Frst we wll fd the varace of ad ootocally creasg fuctos of ad. because ad are x y x y x y x V( V( V( V( V( ov( ov( C y x C ov( y y C y x y x y y Cov( ov(. x x C x y y x x x x y (4.3 y x y x y x y x V( V( V( V( V( Cov( y x Cov( y y Cov( y x y x y y Cov( x x Cov( y x y x x x y x Now o partally dfferetatg ad wth respect to y x y x a y x y x respectvely we get / y ( y x ( - y x - 3/ y ( y x / y ( y x ( - y x - 3/ y ( y x - / x ( y x ( - - y x 3/ x ( y x - / x ( y x ( - -. y x 3/ x ( y x (4.4 (4.5 The estated varace of y x y ad y V( y y y y V( x x V( y x V( x x x x are gve as y y V( y - y V( y - y V( x x x V( x x x - - (4.6 DOI:.979/ Page

8 Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve The covarace ters are all zero due to detcally depedetly dstrbuted rado varables. Substtutg all the ressos fro (4.5 ad (4.6 (4.3 ad (4.4 we get y ( x y - x y x V( (4.7 3 ( y x y x ( y x y - x - y ( x y - x y x V(. (4.8 3 ( y x y x ( y x y - x - Usg (4.7 ad (4.8 the cofdece terval of AUC s gve as follows p Z V p Z V Z V p Z V where Z s the crtcal value of Z for a two taled test ad α s the level of sgfcace. V. Sulato Studes Mote Carlo sulato s a estatg algorth that s based o repeated rado saplg to obta uercal results. Aderso (986 dscussed the Metropols algorth ad Mote Carlo Sulato. The followg steps are used Mote Carlo sulato. Defe a rage of possble puts. Geerate rado ubers fro a probablty dstrbuto over the rage. 3 Execute a deterstc coputato fro the puts. 4 Aggregate the results. I ths secto we dscuss a uercal exaple to observe how the cofdece terval of AUC behaves by usg Mote Carlo sulato. We check the behavor of Area uder the NMROC Curve (AUC. Let the saple szes for healthy ad dseased cases are = 3 3 ad the xture of healthy cases wth populato paraeters p=.7 μ = 6σ = μ = 3σ =. Slarly the xture of dsease cases wth populato paraeters p=.7 μ =σ = 3μ = 7σ =. The weghts are sae for healthy ad dsease cases. Usg these values of paraeters the axu lkelhood estates of paraeters estated AUC ad 95% cofdece terval are gve Table 5.. Let the fxed values of populato paraeters of healthy cotrols as ad fxed values of. ad ad 4 dsease cases are ad (4.9 for dsease cases for all saple szes. The ea of for all saple szes. The weght of healthy cotrols ad dsease cases are fx p=.7 for all. The saple szes =( for all healthy ad dsease cases. Usg these values of paraeters the axu lkelhood estates of paraeters estated AUC ad 95% cofdece terval are gve Table 5.. It s observed fro Tables 5. ad 5. that f we crease the saple sze the estated value of ea ad varace becoe closer to the populato paraeters. I table 5. we see that as the dfferece betwee the ea of dsease cases creases wth saple szes the AUC of NMROC Curve creases ad dfferece betwee the UCL ad LCL also decreases. VI. Cocluso Whe a populato has the heterogeety ad t s further dvded to subpopulatos the xture dstrbuto s used for cosderg the heterogeety of the populato. I ths case the two copoet oralxture ROC curve gves hgher accuracy of dagostc test as copared to the Boral ROC curve. Its propertes are dscussed ad t s foud that Noral xture ROC curve s ootocally creasg cocave TNR asyetrc ad s varat uder the ootocally creasg trasforato. The cofdece terval of AUC s also derved. Hece t s observed that wheever heterogeety s preset the populato oe should use two copoet oral-xture ROC curve stead of Boral ROC curve. DOI:.979/ Page

9 Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve Refereces []. Newcob S. (886 A geeralzed theory of the cobato of observatos so as to obta the best result Aer. J. Math []. Pearso K (894 Cotrbuto to the atheatcal theory of evoluto Phl. Tras. Roy. Soc. A [3]. Evertt B. ad Had D.J. (98 Fte Mxture Dstrbutos Chapa ad Hall. New York. [4]. Tttergto D.M. Sth A.F.M. ad Makov U.E. (985 Statstcal Aalyss of Fte Mxture Dstrbutos Joh Wley ad Sos Ltd. [5]. McLachla G.J. ad Basford K.E. (988 Mxture Models: Iferece ad Applcatos to Clusterg Marcel Dekker New York. [6]. Bohg D. Detz E. ad Schlatta P. (998 Recet Developets Coputer Asssted Aalyss of Mxtures Boetrcs [7]. McLachla G. ad Peel D. ( Fte Mxture Models Joh Wley New York. [8]. Schlatta P. (9 Medcal Applcato of Fte Mxture Models Sprger-Verlag Berl Hedelberg. [9]. Wrjato Toy S. ad Dgha Xu (9 The Applcato of Mxtures of Noral Dstrbutos Eprcal Face: A Selected SurveyUversty of Waterloo Departet of Statstcs 9. []. Patrck J.F. Ehsaes Saleh A.K.M. ad Zhag Z. ( Methods of Moets Estato Fte Mxtures Sakhya: The Ida Joural of Statstcs 73-A part 8-3. []. Dass S.C. ad Seog W.K. ( A Se-paraeterc Approach to Estato of ROC Curves for Multvarate Boral Mxtures. []. Lee J.Y. ( A two-stage approach usg Gaussa xture odels ad hgher-order statstcs for a classfcato of oral ad pathologcal voces Joural o Advaces Sgal Processg 5-8. [3]. Goe M. (3 Mxtures of Recever Operatg Characterstc Curves Meoral Sloa-Katter Cacer. http//works.bepress.co/that_goe/3. [4]. Hughes G. ad Bhattacharya B. (3 Syetry Propertes of B-Noral ad B-Gaa Recever Operatg Curves are Descrbed by Kullback-Lebler Dvergeces Etropy 5( [5]. Gree D.M. ad Swets J.A. (966 Sgal Detecto Theory ad Psychophyscs Wley New York. [6]. Ega J.P. (975 Sgal Detecto Theory ad ROC Aalyss Acadec Press New York. [7]. Krzaowsk W.J. ad Had D.J. ( ROC curves for cotuous data Moographs o Statstcs ad Appled Probablty CRC Press Taylor ad Fracs Group New York. [8]. Baber D. (975 The area above the ordal doace graph ad area below the recever operatg characterstc graph Joural of Matheatcal Psychology [9]. Haley J.A. ad McNel B.J. (98 The eag ad use of the area uder a ROC curve Radology []. Bradley A.P. (997 The use of the area uder the ROC curve the evaluato of ache learg algorths Patter Recogto []. Pudr S. ad Aala R. (3 Recever Operatg Characterstc odelg for cotuous data: A glace Model Asssted Statstcs ad Applcato 9( -35. []. Dorfa D.D. ad Alf E.J. (969 Maxu lkelhood estato of paraeters of sgal detecto theory ad deterato of cofdece terval ratg ethod data Joural of Matheatcal Psychology [3]. Kaaruzzaa Z.A. Zad I ad Isal M.T. ( Mxture of Noral Dstrbutos: Applcato to Bursa Malaysa Stock Market Idces World Appled Scece Joural 6( [4]. Depster A.P. Lard N.M. ad Rub D.B. (977 Maxu- Lkelhood fro coplete data va the EM algorth Joural of Royal Statstcs Socety Ser. B (Methodologcal 39 ( -38. [5]. Joshua H.P. (3 The EM Algorth Multvarate Gaussa Mxture Models usg Aderso Accelerato. A Project Report Worcester Polytechc Isttute. [6]. Fluss R. Faragg D. ad Reser B (5 Estato of the Youde Idex ad ts assocated cutoff pot Boetrcal Joural 47( [7]. Aderso H.L. (986 Metropols Mote Carlo ad the MANIAC Los Alaos Scece. DOI:.979/ Page

10 .7 (..85 ( Detecto of AUC Ad Cofdece Iterval Usg Noral-Mxture ROC Curve Table 5.: MLE p μ μ σ σ ( ( ( ( ( ( ( ( (.5.7 ( ( (.36 ( : bas of the estates.7 ( A ÛC ad 95% cofdece terval of ÛC p μ μ σ σ AU C ( ( Table 5. MLE p μ μ σ σ ( ( ( (..66 (.6.3 (.73.9 (.59.7 (..9 (..77 (.7.8 ( ( ( ( ( ( ( A for dfferet saple szes ( ( ( ( ( ( (.4 A ÛC ad 95% cofdece terval of ÛC p μ μ σ σ AU C ( ( ( ( (.9.7 (..989 ( ( (.46 A for dfferet saple szes.39 ( ( ( ( (.4.95 ( Cofde ce Iterval [ ] [.9.99] [ ] [ ] [.854.9] [.9.935] Cofdece Iterval [ ] [ ] [ ] [ ] [ ] DOI:.979/ Page