Fisher Information Test of Normality

Transcription

1 Fisher Iformatio Test of Normality by Yew-Haur Lee Dissertatio submitted to the Faculty of the Virgiia Polytechic Istitute ad State Uiversity i partial fulfillmet of the requiremets for the degree of Doctor of Philosophy i Statistics Approved by Dr George R. Terrell, Chairma Dr Clit W. Coakley Dr Klaus Hikelma Dr Eric P. Smith Dr Keyig Ye September 3, 998 Blacksburg, Virgiia Keywords: Normality testig, Fisher iformatio for locatio, Power study, Residuals, Noparametric desity estimatio, Calculus of variatio Copyright 998, Yew-Haur Lee

2 Fisher Iformatio Test of Normality by Yew-Haur Lee (ABSTRACT) A extremal property of ormal distributios is that they have the smallest Fisher Iformatio for locatio amog all distributios with the same variace. A ew test of ormality proposed by Terrell (995) utilizes the above property by fidig that desity of maximum likelihood costraied o havig the expected Fisher Iformatio uder ormality based o the sample variace. The test statistic is the costructed as a ratio of the resultig likelihood agaist that of ormality. Sice the asymptotic distributio of this test statistic is ot available, the critical values for = 3 to 00 have bee obtaied by simulatio ad smoothed usig polyomials. A extesive power study shows that the test has superior power agaist distributios that are symmetric ad leptokurtic (log-tailed). Aother advatage of the test over existig oes is the direct depictio of ay deviatio from ormality i the form of a desity estimate. This is evidet whe the test is applied to several real data sets. Testig of ormality i residuals is also ivestigated. Various approaches i dealig with residuals beig possibly heteroscedastic ad correlated suffer from a loss of power. The approach with the fewest udesirable features is to use the Ordiary Least Squares (OLS) residuals i place of idepedet observatios. From simulatios, it is show that oe has to be careful about the levels of the ormality tests ad also i geeralizig the results.

3 Ackowledgemets I would like to exted my sicere gratitude ad appreciatio to Dr George R. Terrell for his guidace throughout this period of research. He was always there whe I eeded his advice. I am ideed fortuate to have this opportuity to lear ad work with him. I would also like to thak Dr Klaus Hikelma, Dr Eric P. Smith, Dr Clit W. Coakley, ad Dr Keyig Ye for their time i carefully readig my dissertatio ad for the may helpful suggestios ad commets that greatly improve my dissertatio. I would also like to exted my thaks to the Statistics Departmet for the ivaluable learig experiece that I received. I would also like to thak Michele Marii ad Bill Sydor for resolvig the may computig obstacles that I ecoutered. I would also like to express my thaks to the may frieds that I have come to kow for their ecouragemet ad compaioship. I would also like to express special thaks to my parets, my sisters ad Poh Lig, for their love, ecouragemet ad support all this while. This would ot be possible without sacrifices they made so that I could pursue my goal. Fially, I would like to thak God for his grace, mercy ad stregth that has sustaied me throughout this time of my life. iii

4 Table of Cotets Ackowledgemets...iii List of Tables...vi Table of Figures...viii Chapter Itroductio ad Motivatio.... Statemet of the Problem.... Directio of Research... Chapter Existig Normality Tests...4. Momets Tests - Bowma-Sheto K...4. Distace/ECDF Tests - Aderso-Darlig A Regressio/Correlatio Tests - Shapiro-Wilk W...6 Chapter 3 Fisher Iformatio Test Normal Iformatio Iequality (for locatio) Normal Iformatio Statistic F Solutio to the Problem Computatioal Algorithm Geeratio of Critical Values Usig Simulatio...6 Chapter 4 Evaluatios ad Applicatios to Real Data Features of Data that Affect Desity Estimate, g Power Compariso Simulatio Set-up Results Summary Applicatios to Real Data Sets Male Weights Data Mississippi River Data PCB Data Buffalo Sowfall Data Mice Data iv

5 Chapter 5 Testig Normality of Residuals Backgroud Power Comparisos Simulatio Set-up Results Coclusio Chapter 6 Summary ad Discussio...46 Appedix A Critical Values of the Normal Iformatio Statistic, F...48 Appedix B Computatioal Details...5 B. Details of programmig for F...5 B. Details of power study...5 B.3 Program Listig for F...5 Appedix C Results from Power Study...56 Refereces...67 Vita...70 v

6 List of Tables Table 4- Properties of symmetric distributios used i simulatio study... Table 4- Properties of asymmetric distributios used i simulatio study... Table 4-3 Discrete distributio with ormal momets... Table 4-4 Power estimates of discrete distributio with ormal momets... Table 5- Power comparisos of ormality tests o iid observatios ad OLS residuals across differet values of based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α= Table 5- Measures of correspodece betwee true ad modified test statistics across differet values of based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α= Table 5-3 Power comparisos of ormality tests o iid observatios ad OLS residuals across differet values of k based o 000 samples usig X=data set from Weisberg (980) at α=0. ad = Table 5-4 Measures of correspodece betwee true ad modified test statistics across differet values of k based o 000 samples usig X=data set from Weisberg (980) at α=0. ad = Table B- List of critical values used i power study... 5 Table C- Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0.05 ad = Table C- Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0.05 ad = Table C-3 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0.05 ad = Table C-4 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0.05 ad = vi

7 Table C-5 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0.05 ad = Table C-6 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0. ad = Table C-7 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0. ad = Table C-8 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0. ad = Table C-9 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0. ad = Table C-0 Power comparisos of ormality tests o iid observatios ad OLS residuals based o 000 samples usig k=4 with X = ad X i, i=,3,4 draw from the uiform distributio at α=0. ad = vii

8 Table of Figures Figure 4- Estimated desity for {-, 0, }... 8 Figure 4- Estimated desity for {-, 0, 0, }... 8 Figure 4-3 Estimated desity for {-, 0, 0,, 5}... 9 Figure 4-4 Desity estimates of Male Weights Data (=)... 9 Figure 4-5 Desity estimates of Mississippi River Data (=49) Figure 4-6 Desity estimates of PCB Data (=65)... 3 Figure 4-7 Desity estimates of Buffalo Sowfall Data(=63)... 3 Figure 4-8 Desity estimates of Mice Data(=99) viii

9 Chapter Itroductio ad Motivatio The problem of ormality testig is well kow ad has geerated plety of attetio from researchers, see Mardia (980) ad D Agostio ad Stephes (986). This is because a lot of classical optimal procedures were developed based o the ormality assumptio. However, researchers soo realized that this assumptio was ot always satisfied. Three approaches ca be take to deal with o-ormality of data. The first approach is trasformig the data to ormality so that the classical procedures could still be used. The secod approach is the use of oparametric procedures. The third is to use robust procedures that are less sesitive to deviatio from ormality, especially tail behavior. Each of the three comes with stregths ad weakesses; there is o cosesus o which is the best approach. The role of ormality testig is ot just to see if the data are well approximated by the ormal distributio; but also to provide iformatio o the deviatio from ormality. This iformatio would the guide the researchers to the best approach to dealig with the o-ormality of their data.. Statemet of the Problem I this dissertatio, it will be assumed that oe is testig ormality because the user wishes to fit a locatio model. Suppose the data collected x, x,..., x represet a idepedet ad idetically distributed (iid) radom sample of size from a populatio with probability desity fuctio f(x) ad cumulative desity fuctio (cdf) F(x). Let Φ be the cdf of x that is ormally distributed with ukow mea ad variace. The ull hypothesis i this problem of testig for ormality is H 0 : F(x) = Φ(x) ad the alterative hypothesis simply states H 0 is false. Hece oly omibus tests will be cosidered i this dissertatio. Here, omibus refers to the ability of a test to detect ay deviatio from ormality with a adequate sample size.

10 I this problem, the focus is o failig to reject H 0 so that the coclusio is that the data come from a ormal distributio. As oted by D Agostio ad Stephes (986), this distiguishes ormality testig from most statistical tests. Also, with a vague alterative hypothesis, they commeted that the appropriate statistical test will ofte be by o meas clear ad o geeral Neyma-Pearso type (test) appears applicable. Hece, it will be ulikely to have a sigle test that will have power superior to their alteratives.. Directio of Research There are literally hudreds of ormality tests i the literature. Major power studies doe by Shapiro et al. (968) ad Pearso et al. (977) have ot arrived at a defiitive aswer; but a geeral cosesus has bee reached about which tests are powerful. Pearso s (900) chi-squared test, which is possibly the oldest, is ot very sesitive. Data are grouped ad compared to the expected couts uder ormality. Sice iformatio is lost i the groupig ad this test is ot specially tailored for the ormal distributio, the coclusio is ot surprisig. Bowma ad Sheto (975) proposed the use of joit cotour plots of the third ad fourth momets for their test. It proves powerful amog tests based o momets. For these tests, sample momets are compared to those which are expected from a ormal distributio. However, these tests are ot omibus sice a distributio could have skewess ad kurtosis close to that of a ormal but yet the distributio could be oormal. Aother class of ormality tests are based o the empirical cumulative distributio fuctio (ECDF). Deviatio from ormality is measured as a fuctio of the discrepacies betwee the empirical ad hypothesized distributio fuctios. Stephe s (974) versio to the Aderso-Darlig (954) test has proved to be the most powerful amog these tests. However, it is ot clear if this measure of deviatio is of primary importace i decidig what to do if the data are o-ormal. A ew directio was established i ormality testig whe Shapiro ad Wilk (965) formalized the evaluatio of ormality i probability plottig. Probability plottig

11 ivolves plottig of ordered observatios agaist their expected values uder ormality. Normality is judged by the liearity of the plot. This test ad its modificatios have proved to be popular amog researchers sice it is as powerful, if ot more so, agaist certai alterative distributios as the Aderso-Darlig ad the Bowma-Sheto tests. However, as with the ECDF tests, it is uclear if the measure of deviatio from ormality is what researchers are cocered about. A well kow property of the ormal distributio is that it has the least Fisher iformatio amog all other distributios with the same variace. A ew approach to ormality testig suggested by Terrell (995) essetially provides a oparametric desity estimate of the data costraied o the above property whe fidig the likelihood of the data. This costrait elimiates the eed for a smoothig parameter. The test is the based o the ratio of that likelihood to that uder ormality. Excess iformatio would be reflected i a poorer fit of the data to ormality sice the Fisher Iformatio is uderestimated. Hece, the test is omibus ad sesitive to departures from ormality that are maifested i the excess iformatio. The goal of this dissertatio is to gai a uderstadig ito the workigs of the Fisher Iformatio Test for ormality, provide a comprehesive table of critical values ad evaluate the power performace agaist existig tests. The Fisher Iformatio Test will also be modified to test for ormality i residuals. I the ext chapter, we will give a brief review of the backgroud ad details of the existig tests of ormality. Chapter 3 develops the theory ad operatioal details behid the Fisher Iformatio Test. Chapter 4 presets a evaluatio of the sesitivity of the Fisher Iformatio Test ad a power compariso is doe agaist existig tests. Chapter 5 ivestigates the testig of ormality i residuals ad aother power compariso is doe to see if the results differ from those usig idepedet observatios. Fially, Chapter 6 summarizes the fidigs ad also discusses directios for future research. 3

12 Chapter Existig Normality Tests For a detailed survey of the literature, see Mardia (980) ad D Agostio ad Stephes (986). I this chapter, the focus is o existig ormality tests which are metioed i Chapter that are powerful. Each of these tests belogs to a differet class of ormality tests. A brief backgroud to the geeral approach i each class is give before details are preseted for each test.. Momets Tests - Bowma-Sheto K Sice the cocepts of skewess ad kurtosis ca be used to differetiate betwee distributios, oe of the earliest classes of ormality tests is based o these momets. The stadardized coefficiets of skewess, β, ad kurtosis, β are defied as β µ 3 = 3 ad β σ = µ σ 4 4 where µ i is the ith cetral momet. Skewess refers to the symmetry of a distributio. For a symmetric distributio like the ormal, β = 0. A distributio that is skewed to the right has β > 0 while oe that is skewed to the left has β < 0. Kurtosis refers to the flatess or peakedess of a distributio. The ormal distributio has β = 3 ad is used as a referece for other distributios. A leptokurtic distributio is oe that is more peaked ad with heavier tails tha the ormal, resultig i β > 3. A platykurtic distributio has a flatter distributio with shorter tails tha the ormal, hece β < 3. The sample skewess, b, ad kurtosis, b, are defied as b m3 = ad ( s ) 3 b = m 4 ( s ) where m i is the ith sample momet. Sice the momets of b ad b are kow, their distributios have bee approximated usig Pearso curves. The critical values for the 4

13 ormality tests of skewess ad kurtosis are tabulated i Pearso ad Hartley (97) for selected values of 5 at α = 0.0 ad 0.0. Normalizig trasformatios have bee foud for b ad b by D Agostio (970) ad D Agostio ad Pearso (973) respectively. Z( b ) ad Z( b ) deote the resultig approximate stadardized ormal variables. D Agostio ad Pearso suggested combiig b ad b i the followig way: ( ) ( ) K = Z b + Z b where K is distributed as χ sice it is the sum of the squares of stadardized ormal equivalet deviates. However, they assumed that the squared stadardized ormal equivalet deviates are idepedet which is ot true especially for small sample sizes. Usig simulatio, Bowma ad Sheto (975) obtaied 90%, 95% ad 99% cotours for K for sample sizes betwee 0 ad 000. Carryig out this test would the oly require calculatig b ad b, selectig the appropriate cotour, ad determiig if ( b,b ) falls withi the cotours. If it does ot, the ormality is rejected.. Distace/ECDF Tests - Aderso-Darlig A ECDF or distace tests are aother broad class of ormality tests that are based o i a compariso betwee the ECDF, F ( x( i) ) =, ad the hypothesized distributio uder ormality, Z i, as defied by Z i = x i Φ ( ) x s x i where x = = i ad s = ( xi x) i= ECDF tests with ukow µ ad σ.. Stephes (974) provided versios of the 5

14 ECDF tests ca be further classified ito those ivolvig either the supremum or the square of the discrepacies, F ( x( i) ) Zi. The most well kow ECDF tests ivolvig the supremum is the Kolmogorov-Smirov statistic + K = max( D, D ) i i where D + = max Z i ad D = Zi max ECDF tests ivolvig the square of the discrepacies are kow as those from the Cramér-vo Mises family with the geeral form CvM i = [ Z Z dz ] ψ ( ) i i i where ψ( Z i ) is the weightig fuctio. If ψ( Z i ) =, that is the Cramér-vo Mises statistic itself, W. For the Aderso- Darlig statistic, A, ψ( Zi ) = Z ( Z ) values ad the computatioal form is give by A i i. This choice of ψ( Z i ) gives emphasis to tail [ [ i + i ]] = ( i ) l( Z ) + l( Z ) i= Stephes foud that A has the highest power amog all ECDF tests. The asymptotic distributio is kow ad it was foud that the critical values for fiite samples quickly coverge to their asymptotic values for 5..3 Regressio/Correlatio Tests - Shapiro-Wilk W The mai idea behid these tests is ormal probability plottig. Normal probability plottig is a graphical techique to determie the ormality of the data by lookig for liearity i a plot of the ordered observatios x (i) agaist the expected values of stadard ormal order statistics, m i. Formal determiatio of the liearity uses regressio or correlatio techiques, hece the ame of this group of tests. If x (i) is ideed ormal, the the slope would give the stadard deviatio of x i, σ, ad the itercept, the mea of the x i s, µ. Sice the ordered observatios are ot 6

15 idepedet, let V=(v ij ) be the x covariace matrix, x = (x, x,, x ) ad m = (m, m,, m ). The best liear ubiased estimators of the slope ad itercept usig geeralized least squares are m V x $σ = ad $µ = x m V m The usual symmetric estimate of the variace regardless of the distributio of x i is give by s. The Shapiro ad Wilk (965) W statistic is defied as where K $ a x W = = ( ) s ( ) s σ = i= i= a x i ( i ) ( x x) ( a, a,..., a ) = m V [( m V )( V )] a = m ' m V m K = m V V m i W compares the ratio of two estimates of variace, ˆσ ad s, apart from a ormalizig costat, K, ad (-). If the distributio of x i is ormal, the W will be close to. Otherwise, W is less tha. The critical values of W are tabulated up to sample sizes of 50. However, values for {a i } are also eeded to carry out this test. For larger sample sizes, Shapiro ad Fracia (97) oted that the ordered observatios, as icreases, may be treated as idepedet (i.e. v ij = 0 for i j). Treatig V as a idetity matrix, W ca be exteded for larger tha 50 by mi x( i) i= W = ( xi x) mi i= i= Values of {m i } are available from Harter (96) up to sample sizes of 400. However, two tables are still eeded to carry out this test. 7

16 A further modificatio was suggested by Weisberg ad Bigham (975) that uses this approximatio 3 i Φ m 8 i + 4 due to Blom (958). This approximatio was show to be close eve i small samples, ad the ull distributio of W was practically idetical to W. This simplifies the computatio of the test statistics sice separate values for m i eed ot be kept. Roysto (98) used aother approximatio suggested by Shapiro ad Wilk (965) for {a i } ad applied the followig ormalizig trasformatio to W: λ y = ( W ) ad z = ( y µ y ) / σ y where z is stadard ormal ad λ, µ y ad σ y are fuctios of. λ is estimated by maximizig the correlatio betwee certai empirical quatiles of W ad the correspodig stadard ormal equivalet with weights give accordig to the variace of a ormal quatile. The relatio betwee µ y ad σ y ad is the determied by applyig λ to simulated values W. The ormalizig trasformatio producig W* does away with ay special tables, besides the stadard oes, eeded to fid the critical values of W. 8

17 Chapter 3 Fisher Iformatio Test I this chapter, the theory behid the Fisher Iformatio Test as suggested by Terrell (995) will be explaied i detail. First, the Fisher Iformatio Iequality will be derived. Next, the Fisher Iformatio Test will be developed ad a implemetatio algorithm will be preseted. Lastly, details of the simulatio doe to geerate the critical values will be give. 3. Normal Iformatio Iequality (for locatio) The Fisher iformatio umber which is deoted by I F is give by I F = E[ (log f ) ] = E[{(log f ) } ] = ( f ) where f is ay desity. It measures, o average, how fast the log-likelihood chages as the mea moves away from the ceter of the distributio. Aother way of lookig at it would be the ease of usig a sample to locate the ceter of a distributio. A famous property of the ormal distributio is that its I F is the smallest amog all distributios with the same variace. The implicatio is that it is hardest to tell where the mea is i a ormal distributio. The proof give by Terrell (995) is preseted here sice it is itegral to the developmet of the Fisher Iformatio Test. The right had side of the familiar property of a desity give below = f is itegrated by parts for ay µ ad f that goes to zero at the limits of its support that results i f = (x µ ) f Itroducig f to the itegral by replacig f with f f f results i = ( x µ ) f f f 9

18 Applyig the Cauchy-Schwartz Iequality yields f f = [ ( x µ )] f [ ( x µ )] f f = ( x µ ) f f f f f f Choosig µ to miimize the first itegral makes ( x µ ) f = var( X ). Hece, the iequality after rearragig becomes I F = f ( ) f var( X ) f where equality is achieved whe ( x µ ) is proportioal to = (log f ) almost f everywhere. For a ormal distributio, d (log f ) = [ ( x µ ) ] = ( x µ ) ( x µ ) dµ πσ σ σ Therefore, for the ormal distributio, the equality is achieved ad I F = σ. This Fisher Iformatio Iequality is sort of a dual to the Cramér-Rao Iequality. The Fisher Iformatio Iequality gives the lower boud for the Fisher Iformatio for ay distributio i terms of its variace while the Cramér-Rao Iequality gives the lower boud for the variace of ay locatio estimator i terms of its Fisher Iformatio. 3. Normal Iformatio Statistic F For ay other distributio besides the ormal, the Fisher Iformatio umber would be i excess of the iverse of its variace. This excess would thus be a atural measure of o-ormality or deviatio from ormality. A atural test statistic for ormality would be to get a direct estimate of I F. However, that would require a estimate of the desity which relies o oparametric methods that are asymptotically iefficiet. Moreover, there is also the eed to specify a smoothig parameter. Terrell circumveted these two problems by formulatig the problem usig maximum likelihood i the followig way max log f ( xi ) subject to I F f i= = ad f s = (3-) 0

19 where the first costrait estimates I F usig the asymptotically efficiet statistic s uder ormality. The secod costrait gives the familiar property of a desity. The Normal Iformatio statistic, F, is the a log-ratio of this likelihood to that uder ormality. If the data are ormal, the F is close to sice I F has bee correctly estimated. Otherwise, I F is uderestimated ad F reflects a poorer fit of the data. Rewritig (3.) with Lagrage multipliers gives ( f ) f x i max log ( ) f i= λ γ f (3-) f where the values of λ ad γ are chose so that the costraits are met. If λ is treated as a parameter, the above form is similar to the first pealized maximum likelihood desity estimatio problem of Good ad Gaskis (97). This problem is a formal dual to theirs, ad the form of the solutios to both are similar. A solutio i terms of expoetial splies has bee foud by demotricher et al. (975). Hece, the Fisher Iformatio Test of Normality gives a graphical tool i the form of a oparametric desity estimate with the smoothig parameter specified by the costrait o I F. 3.3 Solutio to the Problem Sice there is a eed for the resultig desity estimate to be o-egative, the techique from Good-Gaskis of gettig the solutio i terms of a fuctio h = f will be used. This will elimiate the eed for a o-egativity costrait. With this modificatio, f = hh. The expressio for Fisher iformatio i terms of h is the I F = ( f ) ( hh ) = = 4 f h ( h ) The optimizatio ca be modified by maximizig the average log-likelihood ad (3-) becomes max h i= logh( x ) 4λ i ( h ) γ h (3-3)

20 where λ ad γ are chose so that 4 ( h ) = ad = s h. (3-3) is a particular case of the so-called isoperimetric problem i the calculus of variatio. Calculus of variatio is a techique usig classical calculus methods to solve maximizatio ad miimizatio problems where the solutio is a fuctio istead of a poit while isoperimetric problems are those that ivolve derivatives i the costraits. The mai idea is to write the objective fuctio ad costraits as a Lagragia fuctio with h beig replaced by g + εp. Here, g is assumed to be the solutio to the problem ad ε p is the perturbig fuctio with p beig a arbitrary fuctio that vaishes at - ad (same as g ) ad ε is a arbitrary costat. The Lagragia fuctio (V) is the a fuctio of ε. Note that as ε 0, h g. Therefore, the first-order ecessary coditio dv for the problem is the give by = 0 d ε ε= 0 ad the secod-order sufficiet coditio for d V maximizatio problems is < 0. For a referece to calculus of variatio, see Chiag dε (99). To get the Euler-Lagrage variatioal coditio for (3-3) from first priciples, (3-3) is re-writte as a fuctio of ε as follows: V ( ε) = log{ g( xi ) + εp( xi )} 4λ ( g + εp ) γ { g + εp} (3-4) i= Expadig (3-4) results i V ( ε) = log i i p i= { g( x ) + εp( x )} 4λ {( g ) + εg p + ε ( p ) } γ { g + εgp + ε } Differetiatig V(ε) with respect to ε, dv ( ε) = dε i= p( xi ) 4λ g( x ) + εp( x ) ad usig the first-order ecessary coditio gives i i { g p + ε( p ) } γ { gp + εp } dv ( ε) dε ε= 0 = p( xi ) 4λ g p γ gp = 0 g( x ) i= i

21 Usig the shiftig property of the Dirac delta fuctio, p x ) ca be writte as ad itegratig the secod term by parts, p x ) = p( x) δ( x x ) ( i i ( i g p = g p g p = g where use is made of the fact that p vaishes at - ad. Substitutig the above ito the first-order coditio gives p( x) δ( x x ) i + 4λ g p γ gp = i = g( x ) Factorig p ad collectig terms gives p i = Sice p is arbitrary, the above reduces to i = i δ( x xi ) + 4λg γg = 0 g( xi ) δ( x xi ) + 4λg γg = 0 g( x ) i As the Dirac delta fuctioal is zero except at zero, the fial form of the Euler-Lagrage variatioal coditio is give by = δ ( x xi ) For the secod-order ecessary coditio, g = γg 4λg p 0 (3-4) d V p( xi ) = 8λ dε i= { g( x ) + εp( x )} i i ( p ) γ p < 0 which esures that the solutio g gives a maximum solutio to the problem. Kloias (98) foud that g * h ( x) = i= ( i ) *( x ) K x x h g h i (3-5) characterized the solutios to (3-4) where K h = h K x ad K = h e x. Here, the priciple of suppositio is employed where (3-4) is solved for each ad the the 3

22 solutio is added together for the fial solutio to (3-4). Kerel fuctios, K, are used to solve the equatio K K = δ ad the scaled versio, K h, is a solutio to the equatio K h h K = δ (3-6) * which looks like a = versio of (3-4). Dividig (3-6) by gh ( xi ) from to gives * where g ( x) h = ( i ) ( x ) h ( i ) ( ) K x x K x x h h δ h = i = g i= g x i= i= * * * h i h i h * * g x h g x h ( ) ( ) ( i ) *( x ) K x x h g h i h δ = i= ( x xi ) * g ( x ) fuctioal is zero except at zero δ x x * * i= gh ( x) h gh ( x) = * g ( x) h i ( x xi ) g ( x ) i ad summig ad (3-5) are used for the substitutio. Sice the delta h ( ) i (3-7) Istead of solvig for λ ad γ, the solutio has bee reparameterized to oly oe * parameter, h. To tackle the secod costrait that g h is the square root of a desity * g where the square will itegrate to oe, let a = g * h ( x) h ( x) dx. The gh ( x) = where a * itegratig the square of g h gives oe, which verifies that g h root of a desity. Replacig (3-7) with g h results i * where g ( x) = ag ( x). h h δ i= a gh ( x) a h gh ( x) = g ( x) h ( x x ) i ideed results i a square (3-8) As for the first costrait o the Fisher Iformatio, a expressio for Fisher Iformatio is obtaied by multiplyig (3-8) by g h ( x) ad itegratig. The first term equals a sice gh ( x) dx = by defiitio. Itegratig the secod by parts 4

23 h ( ) h ( ) [ h ( ) h ( ) ( ) ( ) h ( ) ] h ( ) h a g x g x dx = h a g x g x g x dx = h a g x dx where use is made of the assumptio that the desity vaishes at the limits of its support. The third term equals sice δ( x xi ) dx = δ( x xi ) dx = = i= i= i= After the above simplificatio, (3-8) becomes Hece, I F ( gh ) a = 4 = 4 a h ( ) a + a h g h = where the Fisher Iformatio is a cotiuously decreasig fuctio of h. If the data have bee stadardized, solvig h for I F = would produce the required desity estimate. The Normal Iformatio Test statistic, F, is twice the log likelihood ratio that compares the estimated desity, g, to that of ormality, f 0 : ( g ) l F = log = log g log f0 l f (3-9) ( 0) i= i= Usig the maximum likelihood estimators, $ σ ad x, i f 0 yields F = log g log + log $ xi x i= σ$ i= ( π ) ( σ ) ( ) The third term vaishes sice σ$ = as the data have bee stadardized. I additio, the last term simplifies to usig the defiitio of $ σ. Fially, F simplifies to F = 4 log g + log( π ) + (3-0) i= 3.4 Computatioal Algorithm The algorithm to get F is as follows :. Stadardize the data to get variace oe. *. Solve the fixed poit equatio, g ( x) h = 5 i= ( i ) *( x ) K x x h g h i, by

24 a. usig a iitial estimate g ( 0) by takig the square root of a Laplace kerel desity estimate. ( i ) ( 0) ( xi ) K x x ( ) h b. computig a secod estimate by g ( x) = i= g ( ) c. suppressig oscillatios by g ( ) ( ) ( x) = [ g 0 ( x) + g ( x) ] ad iteratig to covergece. 3. Normalize the desity so that its square itegrates to oe. 4. Compute Fisher Iformatio i terms of h. 5. Fid h that gives Fisher Iformatio of oe usig the secat method, ad the calculate F. The details of implemetig this algorithm i FORTRAN are give i Appedices B. ad B Geeratio of Critical Values Usig Simulatio Sice the distributio of F is ukow, critical values have bee geerated via simulatio. Sets of ormal deviates are obtaied usig the subroutie ra from Press et al. (99). Te thousad values of F were geerated for each sample size, = 3()00(5)00. Differet sets of pseudo-radom umbers were used for each simulatio to avoid depedece betwee results. The critical values obtaied were the smoothed usig fifth degree polyomials. The resultig smoothed critical values are tabulated i Appedix A for α at 0.50, 0.5, 0.0, 0.5, 0.0, 0.05, 0.05, 0.0 ad 0.0 where bigger α values are available for those who are more iclied to acceptig o-ormality i their data. 6

25 Chapter 4 Evaluatios ad Applicatios to Real Data From the defiitio of F i (3-9), it ca be expected that the power of F is drive by the discrepacies betwee g ad f 0. The exact relatioship is give i (3-0) which shows that F depeds o the sample size,, ad the resultig square root of the desity, g. To evaluate the sesitivity of F to o-ormality, features of the data that affect g will be examied. The, based o those features, cojectures will be formed to see what aspects of o-ormality F will be sesitive to. These cojectures could the be cofirmed by a power compariso of F agaist existig tests. Fially, F is applied to some real data sets. 4. Features of Data that Affect Desity Estimate, g If the data are ormal, the theory behid F would idicate that g would give a good estimate of the desity. To get a rough bell-shaped desity, oe would expect clusterig of data poits i the ceter ad tail behavior to greatly affect the shape of g. Figure 4- shows the estimated desity plot for {-, 0, }. There are spikes at each data poit with the oe i the ceter receivig more weight tha the other two. For {-, 0, 0, } i Figure 4-, the middle spike has eve more weight with the additioal data poit i the ceter. With sparse data, the resultig desity estimate has to fill the spaces betwee ad aroud data poits to have a desity with area that sums to oe. Hece, oe would ot expect F to be powerful. As sample size icreases, the desity estimate is icreasigly drive by the locatio of data poits ad how they cluster together. As a result, the ability of F to detect o-ormality icreases. 7

26 Desity x Leged(desity):Solid-Estimated;Dotted-Normal Figure 4- Estimated desity for {-, 0, } Desity x Leged(desity):Solid-Estimated;Dotted-Normal Figure 4- Estimated desity for {-, 0, 0, } 8

27 Note that for Figure 4- ad Figure 4-, the resultig tails are both taperig getly dow at both eds. With o data i the tails, there is little discrepacy betwee g ad f 0. For {-, 0, 0,, 5} with a promiet outlier, the spike i the right tail i Figure 4-3 testifies to the sesitivity of g to tail behavior. Hece, tail behavior is aother feature i the data that affects g. Although g is ot a cosistet desity estimate of the uderlyig desity with o-ormal data, the discrepacies betwee g ad f 0 will have the potetial to iflate F sice the data might exhibit asymmetry ad/or sigificat tail misbehavior. Sice g is affected by clusterig of data ad tail behavior, oe would cojecture that F is most sesitive to leptokurtic, symmetric distributios sice the ability to iflate F exists i both tails. Next would be leptokurtic, asymmetric where the ability is ow cofied to oly oe tail. With short tails i platykurtic distributios, F should be less powerful. 0. Desity x Leged(desity):Solid-Estimated;Dotted-Normal Figure 4-3 Estimated desity for {-, 0, 0,, 5} 4. Power Compariso The power of F agaist existig ormality tests is compared through a simulatio study. For a review of other major power studies, see Shapiro et al. (968) ad Pearso 9

28 et al. (977). Refer to Sectio. for some geeral coclusios that have bee reached from these major power studies. 4.. Simulatio Set-up The simulatio study was carried out with = 0, 0, 50, 70 ad 00 with 000 samples draw from 3 o-ormal distributios specified i Table 4- ad Table 4- for symmetric ad asymmetric distributios, respectively. The distributios cosidered are classified accordig to the followig groups: I. symmetric, leptokurtic II. III. IV. symmetric, platykurtic asymmetric, leptokurtic asymmetric, platykurtic The distributios withi each group are arraged i order of icreasig departure from ormality as measured by the stadardized Fisher Iformatio, var(x)i F. This measure is chose so as to accout for differig variaces i distributios. Where var(x)i F does ot exist, the distributios are ordered o the basis of their stadardized coefficiet of kurtosis, β. I group I, the distributios iclude SC(ε,σ ε ) which is the scale-cotamiated ormal with 00ε% of N(0,σ ε ) beig the cotamiat. Similarly, LC(ε,µ ε ) is the locatio-cotamiated ormal with 00ε% of N( µ ε,) beig the cotamiat i group III. 0

29 Table 4- Properties of symmetric distributios used i simulatio study Distributios Var(X) β I F Var(X)I F I Symmetric, leptokurtic Normal 3 t Logistic SC(0.05, 9) *.4 SC(0.0, 9) *.4 t SC(0.05, 5) *.95 Laplace 6 SC(0.0, 5) *.7 t Cauchy II Symmetric, platykurtic U(0,) Beta(.5,.5) Beta(,) *usig umerical itegratio Table 4- Properties of asymmetric distributios used i simulatio study Distributios Var(X) β β I F Var(X)I F III Asymmetric, leptokurtic Weibull() LC(0.05,3) *. LC(0.0,3) *.40 LC(0.0,3) *.63 Chi-squared(0) LC(0.05,5) *.09 LC(0.0,5) * 3.03 LC(0.05,7) * 3.3 LC(0.0,5) * 4.54 LC(0.0,7) * 5.38 LC(0.0,7) * 8.78 Chi-squared(4) Chi-squared() Chi-squared() Weibull(0.5) Logormal(0, ) IV Asymmetric, platykurtic Beta(3,) Beta(,) *usig umerical itegratio

30 Table 4-3 Discrete distributio with ormal momets X P(X=x) Table 4-4 Power estimates of discrete distributio with ormal momets Sample size, K W W* A F α = α = The existig ormality tests cosidered i this study iclude W(W ), W* ad A. Recall that W is the Shapiro-Wilk (965) test ad A is Stephe s (974) versio to the Aderso-Darlig (954) test. Where the sample size exceeds 50, Shapiro-Fracia (97) W will be used i place of W sice it exteds the rage of W from 50 ad below to 400. W*, which is Roysto s (98) approximatio to W(W ), will be cosidered a separate test as it will be iformative to compare its power to W(W ). K is left out of the power study sice it is ot a omibus test. To illustrate this poit, a discrete distributio with ormal momets is added to this simulatio study. Table 4-3 gives the details of such a discrete distributio that has the same first to fourth momets as the ormal. The power of the ormality tests with this distributio is give i Table 4-4. Results are ot obtaied for K at = 0 sice the exact cotours are ot available. All the tests except K had estimated power above 0.60 eve for as low as 0. For sample sizes 0 or larger, these tests had estimated power of.00. The power for K is eve lower tha the omial α value especially for higher sample sizes. Hece, K,

31 i particular, ad momets tests, i geeral, are oly able to detect distributios with oormal momets ad are ot omibus tests. To differetiate betwee the tests to see if oe test is superior to aother, the practice i the literature has bee to determie which test has the highest power based o the same set of pseudo-radom umbers for each distributio. To geeralize the results across differet distributios, the averaged rak calculated for each test is sometimes used. The fact that a differet set of pseudo-radom umbers might give rise to a differet orderig of the power is usually igored. To accout for this variability, a formal statistical test o the equality of the power of the tests is coducted i this power study. As all the tests are subjected to the same set of pseudo-radom umbers, the powers of the idividual tests are correlated. Hece, Cochra s Q is used to accout for this correlatio. I cases where the equal power hypothesis is rejected, McNemar s test with correctio for cotiuity is used for pairwise comparisos to determie whether the test with the highest power is sigificatly differet from the rest. To maitai the overall type I error rate at 0.05 i the presece of multiple testigs, the idea from Fisher s Least Sigificace Differece is used here. This meas that multiple comparisos are carried out oly if the hypothesis of equal power usig Cochra s Q is rejected. I additio, the same type I error rate is used for both Cochra s Q ad McNemar s tests. For details of both tests, refer to Siegel ad Castella (988). The results from usig Cochra s Q ad McNemar s tests will be reflected as superscripts to the test with the highest power i this power study. The superscripts will deote the umber of tests, icludig the oe with the highest power, that are sigificatly better tha the rest. Hece, a would reflect that the test with the highest power has sigificatly higher power tha the rest while a 4 would mea that all the tests have the same power. The empirical level of each test is also give based o a ormal sample of % cofidece itervals o the empirical level of each test will be used to assess if they cotai the relevat omial levels. This iformatio is useful sice it acts as a check o possible iflatio/deflatio of the power estimates. 3

32 For programmig details ivolved i this simulatio study, please refer to Appedix B Results =0 Table C-(a) shows the results for α = The empirical level for each test is give by the power estimates for the ormal distributio. Here, all the cofidece itervals cotai the omial value of For group I, F is the most sesitive for most of the distributios, havig sigificatly higher power tha the other tests for SC(0.0, 9), SC(0.0, 5) ad t. For t 0 ad SC(0.05, 5) where F did ot have the highest power, all four tests have power that are ot sigificatly differet from oe aother. W is the least sesitive i distributios where ot all tests have the same power. For group II, A has the highest power i all three distributios. However, its power is ot sigificatly higher tha W* ad W while F proves to be the least sesitive. For asymmetric ad leptokurtic distributios i group III, there is o clear domiace of ay oe test. For locatio cotamiated ormals (LCs), A, W* ad F have the highest power for differet LCs, with A havig sigificatly higher power for LC(0.0,5). As for o-lcs, W* clearly is the most sesitive with sigificatly higher power for all distributios except Weibull(), Chi-squared(0) ad Weibull(0.5); F is the least sesitive especially for those with higher var(x)i F. As for distributios i group IV, W* has the highest power but it is ot sigificatly differet from W ad A while F agai proves to be the least sesitive. The results for α = 0.0 are give i Table C-6(a). Here, F has the highest power i most of the distributios i group I with the power beig sigificatly higher for the Laplace distributio. Agai, W is the least sesitive for distributios that are symmetric ad leptokurtic. As for group II, all tests except F are equally good at detectig oormality. For o-lcs i group III, W* is the most sesitive, with the power for Chisquared(4), Chi-squared() ad Logormal(0,) beig sigificatly higher. As for LCs, A ad F are more sesitive tha W ad W* i detectig o-ormality, with A havig 4

33 sigificatly higher power i LC(0.0,7). For group IV, both A ad W have the highest power but oe of them are sigificatly higher tha the rest. Oce agai, F is the least sesitive. =0 The results for α = 0.05 are give i Table C-(a). F is the most sesitive i detectig o-ormality i group I with sigificatly higher power i all distributios except t 0 ad SC(0.05, 9). For groups II, IV ad o-lcs, W is the most sesitive i most distributios, with sigificatly higher power i Chi-squared(0). W* proves to be equally good i most cases while F is the least sesitive. As for LCs, F is the most sesitive i six of the distributios with those for LC(0.05,3), LC(0.0,3) ad LC(0.05,7) beig sigificatly higher. W, W* ad A are equally sesitive i detectig o-ormality for the remaiig LCs but are ot as domiat as F. The results for α = 0.0 are give i Table C-7(a). For group I, F has sigificatly higher power i all distributios except i t 0 where all four tests are equally sesitive. O the whole, both W ad W* are most sesitive i detectig o-ormality i groups II ad IV as well as o-lcs. As for LCs, F has sigificatly higher power i LC(0.05,3), LC(0.0,3) ad LC(0.0,5). For the remaiig LCs, F, W ad W* are equally sesitive. =50 The results for α = 0.05 are give i Table C-3(a) with the empirical level for W beig much lower tha the omial value. Hece, the power for W is uderestimated ad it is ot surprisig that W* emerged with sigificatly higher power i groups II ad IV as well as i Weibull() ad Chi-squared(0) for the o-lcs. Further, F s positio is uchalleged i group I with sigificatly higher power i all distributios except for the Cauchy. F is also most sesitive for most LCs with sigificatly higher power for LC(0.05,3), LC(0.0,3), LC(0.05,5) ad LC(0.0,5). The other thig to ote is that certai distributios i group III with higher var(x)i F are begiig to be so extreme that all tests are equally adept at detectig them. 5

34 These distributios iclude LC(0.0,5), Chi-squared(), Weibull(0.5) ad Logormal(0, ). Table C-8(a) cotais the results for α =0.0. Here, the power for W is ot uderestimated. A fairer compariso ca the be made of the sesitivity of W* ad F. The results are similar for F i group I ad i LCs. I groups II ad III, W* still has sigificatly higher power i Beta(,), Beta(,) ad Beta(3,) but are equally sesitive for the remaiig distributios as W. The same applies to o-lcs. As for LCs, the oly aomaly is that A has sigificatly higher power for LC(0.0,3). Agai, distributios with high var(x)i F i group III are all detected by all of the ormality tests. =70 Table C-4(a) displays the results for α = The empirical level of W of is much higher tha the omial value of This cofirms the hesitace of Pearso et al. (977) i recommedig the use of W sice they poited out that the empirical critical values were overstated as a result of beig based oly o 000 samples. They wared that this ufairly ehaces the power of W. With this i mid, it is ot surprisig that for distributios i group I ad some LCs, W has sigificatly higher power tha the other tests. For these distributios, F cosistetly has the secod highest power for these distributios. I spite of the iflated power for W, both W* ad A maaged to have sigificatly higher power: W* i groups II ad IV as well as Weibull() i group III ad A i LC(0.0,3). I additio, the iflatio of power i W did ot affect those distributios i group III that are detected by all the ormality tests 00% of the time. The results for α = 0.0 i Table C-9(a) are very similar. =00 Sice the power of W is iflated for =70, the critical value used for W for =00 was the average empirical critical value of W obtaied by Pearso et al. (977) to adjust for the iflatio to get a fair compariso. This is reflected i Table C-5(a) ad 6

35 Table C-0(a) where the omial levels are cotaied i the 95% cofidece itervals for the empirical levels. I Table C-5(a) for α = 0.05, F ad W are sesitive i detectig o-ormality i group I with F havig sigificatly higher power for t 4 ad the Laplace distributio. As for groups II ad IV as well as o-lcs distributio like Weibull() ad Chi-squared(0), W* has sigificatly higher power. As for LCs, W has sigificatly higher power i LC(0.05,3) ad LC(0.0,3) while A excels i detectig LC(0.0,3). For the most of the remaiig distributios i group III, all the tests are able to detect o-ormality 00% of the time. A look at Table C-0(a) for α = 0.0 reveals similar fidigs. W* has sigificatly higher power i group II as well as for Weibull() i group III ad Beta(3,) i group IV. Agai, W ad A have sigificatly higher power i the same LCs ad slightly more tha half of the distributios i group III are detected 00% of the time by all the ormality tests. However, i group I, F ad W are ow equally sesitive i detectig o-ormality i group I Summary As expected, o oe test has sigificatly higher power tha all other tests for all the distributios. However, some broad patters have emerged regardig the sesitivity of each test to the differet types of distributio. The followig summarizes the results from the power study:. For distributios that are symmetric ad leptokurtic, F is superior to the other tests for detectig o-ormality.. For distributios that are platykurtic or asymmetric excludig LCs, W* is superior for larger sample sizes ( 50) while both W ad W* are equally sesitive for smaller oes. 3. LCs behaves like a cotiuum betwee leptokurtic distributios that are symmetric to those that are asymmetric as, p ad µ ε icreases. Hece, o oe test is superior. With p ad µ ε small, F is more sesitive for smaller ( 50) while W is better at larger. As p ad µ ε icrease, A is more sesitive. However, there comes a poit 7

36 whe p ad µ ε become so big that all the ormality tests easily detect o-ormality 00% of the time. 4. It is ot surprisig that whe sample sizes are small (<50), W* has power that is equal to W. However, at larger sample sizes, W* is preferred, sice its power is either iflated/deflated. I some cases, W* has sigificatly higher power tha W eve whe W s power estimates are iflated. Hece W* is preferred over W. 5. A examiatio of the power of F shows that besides icreasig with, it also varies directly with Var(X)I F, albeit the relatioship is ot a determiistic oe. 4.3 Applicatios to Real Data Sets I this sectio, F is applied to several real data sets. Here, the estimated desity is plotted agaist the ormal desity with the same sample mea ad variace as the data. This graphic best illustrates ay deviatio from ormality ad provides a ready explaatio whe ormality is rejected. 8

37 0.0 Desity F = P-value < X Leged(desity):solid-estimated;dotted-ormal Figure 4-4 Desity estimate of Male Weights Data (=) 4.3. Male Weights Data Shapiro ad Wilk (965) used their test o a data set of adult male weights take from Sedecor (946). These are, i pouds, 48, 54, 58, 60, 6, 6, 66, 70, 8, 95, ad 36. The resultig statistic F is 4.758, which is beyod the 99 th percetile. This is cosistet with the result give by W. The resultig desity estimate i Figure 4-4 shows a promiet outlier at 36 with a peak i the right tail that accouts for the rejectio of ormality for this data set. 9

38 Desity F = 5.63 P-value betwee 0.0 ad X Leged(desity):solid-estimated;dotted-ormal Figure 4-5 Desity estimate of Mississippi River Data (=49) 4.3. Mississippi River Data Aother example is take from Gumbel (943) which gives the maximum daily rates of discharge from the Mississippi River at Vicksburg i cubic feet per secod for 49 years startig from 890. Assumig that the data are idepedet, the resultig statistic F beig 5.63 is betwee the 50 th ad 90 th percetile. From Figure 4-5, it ca be see that the data do ot deviate much from ormality ad hece supports the cotetio that the data ca be approximated by the ormal distributio. 30

39 Desity F = P-value betwee 0.05 ad X Leged(desity):solid-estimated;dotted-ormal Figure 4-6 Desity estimate of PCB Data (=65) PCB Data A third example is take from Risebrough (97) who was studyig cocetratios of polychloriated bipheyl (PCB), a idustrial pollutat, i the yolk lipids of pelica eggs. He had a sample size of 65 ad the resultig F is 7.675, which is betwee the 95 th ad 97.5 th percetile. Figure 4-6 shows the resultig desity plot which is close to ormal except for two outlyig poits i the right tail. Rejectio of ormality usig α of 0.05 is therefore ot surprisig. 3