Note: for continuous distributions, the histogram is. Statistic Introduction with R - Gabriel Baud-Bovy. main="",xlab="dice side",ylab="counts")

Transcription

1 Emprcal dtrbuton 5 Dcrete probablty dtrbuton 4 3 # mulated data et for throw of a dce et.eed() x<-ample(:6,ze,replacetrue) Count # count outcome (count<-table(x)) Dce de # barplot barplot(count,la,xlab"dce de",ylab"count") ablne(h/6,lty) 3 # htogram ht(x,breakeq(.5,6.5,),la, man"",xlab"dce de",ylab"count") ablne(h/6,lty) Count Dce de ote: for dcrete dtrbuton, the htogram repreent the dtrbuton of the obervaton IIT Emprcal dtrbuton 8 Contnuou probablty dtrbuton 6 4 tern<-read.data("ternberg.dat") Count # Frequency or ount plot ht(tern$rt,xlab"rt",ylab"count", man"",la) RT # denty plot ht(tern$rt,breakeq(,4,5),freqfalse, xlab"rt",ylab"",man"",la) mtext("count",de,lne4) lne(denty(tern$rt)) ote: for contnuou dtrbuton, the htogram baed on arbtrary bn Count [R] quantle and CDF plot # quantle p<-c(.5,.5,.75) (q<-quantle(tern$rt,c(.5,.5,.75))) # cumulatve probablty plot plot(ort(tern$rt), cumum(rep(/length(tern$rt),length(tern$rt))), type"",xlab"rt",ylab"cdf",yaxt"n") ax(,ateq(,,.5),la) ablne(hp,vq,lty) RT # alternatve method plot(ecdf(tern$rt),xlab"rt",ylab"cdf",man"") IIT RT IIT Cumulatve denty functon. 4 Skewne, kurto ormal dtrbuton (top left) Example of devaton from the aumpton of normalty: modal dtrbuton (combnaton of two normal dtrbuton) Skewne a meaure of aymmetry (ymmetrc) Kurto a meaure of tal length. IIT

2 Probablty plot 5 Probablty plot (alo called rankt plot or theoretcal QQ plot) repreent the ordered data on the vertcal ax agant the correpondng normal core on the horzontal ax. For normally dtrbuted data, pont hould form a traght lne. See graph to ee how to nterpret devaton from the traght lne. IIT [R] qqnorm, qq.plot ormal quantle 6 Make a probablty plot wth the reacton tme of the Sternberg dataet. The hape of the curve ugget data reacton tme are kewed veru the left (th ndeed the cae, ee htogram n a prevou lde) qqnorm(tern$rt,man", xlab"ormal quantle", ylab"rt dtrbuton") or lbrary(car) qq.plot(tern$rt, xlab"ormal quantle", ylab"rt dtrbuton") Make the probablty plot of the log of the reacton tme. IIT RT dtrbuton ormal quantle log(rt) dtrbuton IIT IIT

3 9 Hypothe tetng IIT The ull Hypothe (H ) The frt tep n hypothe to formulate an hypothe about the varable of nteret. In general, the hypothe beng teted called the null hypothe H - counter to what we hope to demontrate. For example, the null hypothe mght be that there are no dfference between the mean µ and µ of two group,.e. H : µ µ, f you are ntereted to prove that there a dfference between the two group. The phloophcal argument ntroduced by Fher for th approach, that we can never prove omethng to be true but we can prove omethng to be fale. IIT Inferental tattc Inferental tattc: The branch of tattc concerned wth method that ue a mall et of data (ample) to make a decon (nference) about a larger et of data (populaton). Hypothe tetng:. Formulaton of the (null) hypothe. Realzaton of the experment 3. Computaton of the tattc of nteret from the ample 4. Tet of the hypothe by comparng the tattc to the theoretcal dtrbuton aumng that the null hypothe true. 5. Rejecton of the null hypothe f the tet tattcally gnfcant In hypothe tetng, the goal to ee f there uffcent tattcal evdence n the data to reject a preumed null hypothe IIT Hypothe tetng The econd tep to formulate a tattc a mathematcal ndcator - that wll allow u to tet the null hypothe. For example, the t-tattc or the F-tattc allow u to realze tet about mean. to compute the theoretcal dtrbuton aocated wth th tattc under the aumpton that the null hypothe true (e.g., the Student or t-dtrbuton for the t-tattc or the Fher dtrbuton for the F-tattc). Th theoretcal dtrbuton wll allow u to quantfy the probablty of obervng the computed value for the tattc of nteret under the aumpton that the null hypothe true. The maller th probablty, the le lkely the null hypothe. The null hypothe ad to be rejected f th probablty maller that ome arbtrary mall threhold value (typcally,.5 or.). Th value, referred to a the level of gnfcance and noted by the Greek letter α, correpond to the probablty of rejectng erroneouly the null hypothe (Type error). IIT

4 Type I and II Error 3 An hypothe tet a tattcal decon; the concluon wll ether be to reject the null hypothe n favor of the alternatve, or to fal to reject the null hypothe. The ultmate decon may be correct or may be n error. Hypothe tetng True tate H true H fale H true Correct Type error (β) Decon H fale Type error (α) Correct There are two type of error, dependng on whch of the hypothee actually true (ee Table): A type error rejectng the null hypothe when t true. The probablty of a Type I error degnated by the Greek letter alpha (α) and called the Type I error rate. A type error falng to reject the null hypothe when t fale. A Type error only an error n the ene that an opportunty to reject the null hypothe correctly wa lot. It not an error n the ene that an ncorrect concluon wa drawn nce no concluon drawn when the null hypothe not rejected. IIT Effect ze 5 A tattcally gnfcant tet gve the ndcaton that there a good probablty of obervng th effect agan f the tudy repeated. In other word, tattcal gnfcance tet ae only the relablty of the oberved effect. However, obervng a tattcally effect doe not mean necearly that th effect relevant or meanngful. Gven large enough ample, any effect (e.g., a dfference between the mean of two group), even very mall effect, can be made to be tattcally gnfcant. It therefore often uefull to gve an ndcaton of the effect. In ome cae, one mght know from the context the meanngfulne of a gnfcant effect. For example, let aume that a det followed by the mother ha the effect of decreang the weght of the babe by 5 g, th certanly a meanngful dfference but t would not be the cae f the tattcally gnfcant dfference wa of 5 g. A typcal way of meaurng the ze of an effect to gve the proporton of the total varance that accounted by the effect. Varou meaure of effect ze ext (e.g., R, η, ω ). IIT Rejectng the null hypothe 4 If the probablty of obervng the computed value of the tattc maller than α, we can reject the null hypothe wth le than α % of chance of makng an error. In th cae, we ay that the tet tattcally gnfcant at the α level. In the context of a tudy where dfferent treatment have been agned to the varou group, we alo ay the effect of the factor of nteret tattcally gnfcant (the factor of nteret the ndependent varable that manpulated by the expermenter, the effect the dfference between the mean value of the dfferent group). We wll ay more on th when we wll tudy lnear model. We can alo ay that the (mean value of dependent) varable depend on (the value of) the factor of nteret. There alway a rk of makng an error when rejectng an hypothe. However, f the tet well done, we can guaranty that th rk below ome arbtrary level. IIT Falng to reject the null hypothe 6 If the probablty of obervng the computed value of the tattc larger than α, we cannot reject the null hypothe. A null hypothe not proved becaue t not rejected. The only vald nterpretaton n th cae that there not enough evdence n the data to reject H wth le than α % of chance of makng an error. For example, the obervaton that ome data et not uffcent to how convncngly that a dfference between mean not zero doe not prove that the dfference zero: o experment can dtnguh between the cae of no dfference between mean and an extremely mall dfference between mean. A tated by Fher, we can never prove that an hypothe true: the fact that data are content wth an hypothe doe not exclude that t mght alo be content wth other other hypothee makng mlar predcton. We can only prove that an hypothe fale. The fnal concluon once the tet ha been carred out alway gven n term of the null hypothe. Even f we reject the null hypothe (H ), th doe not mean that the alternatve hypothe (H ) true. IIT

5 Alternatve Hypothe 7 The alternatve hypothe, H, a tatement of what a tattcal hypothe tet et up to etablh For example, n a clncal tral of a new drug, the alternatve hypothe mght be that the new drug ha a dfferent effect compared to that of the current drug. In that cae, the null hypothe H : µ µ and the alternatve hypothe H :µ µ. To tet th hypothe, need to ue a two-ded tet. The alternatve hypothe mght alo be that the new drug better than the current drug. In th cae, the null hypothe H :µ -µ and the alternatve hypothe H :µ >µ. To tet th hypothe, we need to ue a one-ded tet. ote that thee alternatve hypothe do not pecfy what the dfference between µ and µ. Rejectng the null hypothe rejected. IIT Power and ample ze 9 Once a prece alternatve hypothe ha been formulated, then t alo poble to compute: The probablty (β) of a type error,.e. the probablty of not rejectng H f H true (when H fale). the power of the tet (-β),.e. the probablty of rejectng H o f H true. The power of a tet ncreae wth ample ze. Once a mnmal meanngful dfference ha been fxed, t poble to found out the mnmum ample ze that wll allow a tet to reach the dered level of gnfcance and power. The ample ze mut be large enough to reveal the mnmal meanngful dfference but not too large to yeld a gnfcant reult wth a trval dfference- IIT The mnmal meanngful dfference 8 To compute the probablty of acceptng the null hypothe when t fale (Type error or β), we need to tate a more pecfc alternatve hypothe. For example, the alternatve hypothe mght be that the new drug brng a 5% mprovement of the condton. In practce, however, we often lack a theory to predct the ze of an effect, makng t dffcult to tate pecfc alternatve hypothe. One poble way to determne an alternatve hypothe would be to decded beforehand what would be the mnmal meanngful dfference δ (or effect) that relevant or nteretng for the expermenter: H : µ µ H : µ µ δ. For example, letì aume that the null hypothe that the mean and the mnmal meanng dfference, then t poble to calculate the probabolty of a Type II error β and the power of the tet - β. IIT Power (normal dtrbuton) What the probablty of rejectng the hull hypothe f the dfference between the null and alternatve hypothe δ? Power δ δ β Pr Z < z α Ψ z σ / σ / α where Ψ the cumulatve normal dtrbuton and z - α/ Ψ - (-α/). For unlateral tet, replace z - α/ wth z - α. In the prevou example, the dfference between the oberved mean and the null hypothe The probablty of rejectng the null hypothe aumng that th dfference true almot %:.6 Pr Z <.96 Pr Z 3./ 3 ( <.7 ) Exercce. What the power of the tet f there were only obervaton? Anwer:.8 (8% of chance to reject H H true). IIT / /

6 Parameter etmaton Power analy n R Sample ze plannng for the power analytc approach avalable for certan tet n R by default power.t.tet Power calculaton for one and two ample t-tet power.prop.tet Power calculaton two ample tet for proporton power.anova.tet Power calculaton for balanced one-way analy of varance tet Other pecalzed package pwr package for ome general lnear model tet aypow package for the aymptotc power of lkelhood tet ME package for behavoral and ocal cence IIT Hypothe tetng ummary 3 Expermental (Decrptve tattc) Theoretcal (Inferental tattc) Sample electon Treatment agnment Realzaton of the experment (meaure of the varable of nteret) Computaton of the tattc of nteret (ample tattc) from the meaure (e.g., mean, dfference between the two mean, or coeffcent of correlaton) Aumpton about the dtrbuton of the varable of nteret. ull hypothe: e.g. no dfference or no correlaton between the group. Computaton the theoretcal dtrbuton (or amplng dtrbuton) of the tattc of nteret under the aumpton that the null hypothe (H) true. Compare the ample tattc to the amplng dtrbuton and reject H f the probablty of obervng the ample tattc lower that ome value alpha (uually fxed at.5 or.) Addtonal ue: Effect ze, power IIT Hypothe tetng ummary Sample Group Populaton x x x 3 x 4 x tattc Dtrbuton of the parent populaton ull hypothe H Samplng dtrbuton under H Acceptaton or rejecton of H IIT Addtonal Reference 4 Cohen (994) The earth round (p<.5). Amercan Pychologt, 49():997-3 ckeron () ull Hypothe Sgnfcance Tetng: A Revew of an Old and Contnung Controvery. Pychologcal Method, 5():4-3. ruce Thompon (999) Why "Encouragng" Effect Sze Reportng I ot Workng: the Etology of Reearcher Retance to Changng Practce. Journal of Pychology, Vol. 33, 999 IIT

7 Tetng Mean 5 Parametrc tet Student t tet, one-way AOVA Multple comparon Checkng Aumpton on-parametrc tet IIT One ample tet 7 Objectve: Tetng whether the mean of a ample {x,,x } dfferent from ome theoretcal value. The varance of the parent populaton unknown. Aumpton: The ample normally dtrbuted and meaure are ndependent. ull hypothe: H : µµ > the theoretcal mean µ equal to the value µ. Stattc: The one ample t tattc m µ t / where m and are the ample mean and ample tandard devaton, and µ the theoretcal value of the mean ( x m ) Stattc theoretcal dtrbuton: If the null hypothe and our aumpton are true, then the t-tattc follow a t-dtrbuton wth - degree of freedom. IIT Tetng the Mean Standard devaton of the populaton known (> normal dtrbuton) Standard devaton of the populaton unknown (> Student or t-dtrbuton) One group One ample T tet Two group Independent ample T tet Pared ample T tet Three or more group One-way AOVA One veru two-taled t-tet Two-taled tet: H : µ µ >The alternatve hypothe that the populaton mean µ dfferent from the value µ. The lower and upper crtcal value are repectvely t α/ (-) and t -α/ (-) One-taled tet: H : µ>µ > the alternatve hypothe that the populaton mean µ larger than the value µ. The crtcal value t -α (-). One-taled tet: H : µ<µ > the alternatve hypothe that the populaton mean µ maller than the value µ. The crtcal value t α (-). 6 IIT 8 IIT

8 t.tet 9 t.tet(x, y ULL, alternatve c("two.ded", "le", "greater"), mu, pared FALSE, var.equal FALSE, conf.level.95,...) Decrpton Perform one and two ample t-tet on vector of data. Man argument x a (non-empty) numerc vector of data value.. y an optonal (non-empty) numerc vector of data value.. alternatve pecfy the alternatve hypothe: two-taled "two.ded" (default), one-taled "greater" or "le". mu a number ndcatng the true value of the mean (or dfference n mean f you are performng a two ample tet). pared a logcal ndcatng whether you want a pared t-tet var.equal a logcal varable ndcatng whether to treat the two varance a beng equal. If TRUE then the pooled varance ued to etmate the varance otherwe the Welch (or Satterthwate) approxmaton to the degree of freedom ued. conf.level confdence level of the nterval. IIT Independent-ample T Tet 3 Uage. The Independent-Sample T Tet procedure compare mean for two group of cae. Ideally, for th tet, the ubject hould be randomly agned to two group, o that any dfference n repone due to the treatment (or lack of treatment) and not to other factor. Th not the cae f you compare average ncome for male and female. A peron not randomly agned to be a male or female. In uch tuaton, you hould enure that dfference n other factor are not makng or enhancng a gnfcant dfference n mean. Dfference n average ncome may be nfluenced by factor uch a educaton and not by ex alone. Example. Patent wth hgh blood preure are randomly agned to a placebo group and a treatment group. The placebo ubject receve an nactve pll and the treatment ubject receve a new drug that expected to lower blood preure. After treatng the ubject for two month, the two-ample t tet ued to compare the average blood preure for the placebo group and the treatment group. Each patent meaured once and belong to one group. IIT One ample t-tet 3 > tern<-read.table("ternberg.dat",headertrue) > t.tet(tern$rt,mu5) One Sample t-tet data: tern$rt t , df 99, p-value <.e-6 alternatve hypothe: true mean not equal to 5 95 percent confdence nterval: ample etmate: mean of x 6.6 Manual computaton > (m<-mean(tern$rt)) [] 6.6 > (m<-d(tern$rt)/qrt(length(tern$rt))) [] > (mean(tern$rt)-5)/m [] > mc(-,)*qt(.975,99)*m Tet f the mean reacton tme d dfferent from 5 (.e., H : µ5, two-taled tet). Set µ 5. y default, the t.tet functon doe two-taled tet. value of the tattc t, number of degree of freedom of the theoretcal t-dtrbuton, and p value The P value extremely mall p-value Pr( t > 3.65) therefore the null hypothe can be rejected wth confdence. alternatve hypothe H µ 5, th mean two-taled tet. 95% confdence nterval around the mean m µ IIT Independent ample T tet 3 Objectve: Tetng the dfference between the mean of two ndependent ample {x,,x } and {x,,x } where and are the ze of the frt and econd ample. Aumpton: oth ample are normally dtrbuted and have approxmately the ame varance. ull hypothe: H : µ µ, the two mean are equal. Stattc: The two-ample t tattc t m p / where m and m are the mean of the frt and econd ample, and p the pooled etmate of the tandard devaton. ( x m ) ( x m / m ) ( ) ( ) Theoretcal tattc dtrbuton: If the null hypothe and our aumpton are true, then the t-tattc follow a t-dtrbuton wth - degree of freedom. IIT p t /

9 Welch tet 33 Welch tet a varant of Student tet for ndependent ample when the varance are not equal. In th cae, the t tattc follow a tudent dtrbuton wth ν degree of freedom: ν 4 4 ( ) ( ) where ν an approxmaton to the effectve degree of freedom gven by the Welch Satterthwate equaton. Welch, L (947) "The generalzaton of "Student'" problem when everal dfferent populaton varance are nvolved", ometrka 34 ( ): 8 35, IIT Independent ample T tet (SP) 34 Reacton tme Pared T tet 35 Uage. The Pared-Sample T Tet procedure compare the mean of two varable for a ngle group. It compute the dfference between value of the two varable for each cae and tet whether the average dffer from. Example. In a tudy on hgh blood preure, all patent are meaured at the begnnng of the tudy, gven a treatment, and meaured agan. Thu, each ubject ha two meaure, often called before and after meaure. An alternatve degn for whch th tet ued a matched-par or cae-control tudy. Here, each record n the data fle contan the repone for the patent and alo for h or her matched control ubject. In a blood preure tudy, patent and control mght be matched by age (a 75-year-old patent wth a 75-year-old control group member). IIT > boxplot(tern$rt[tern$tet], tern$rt[tern$tet], ylab"reacton tme",xlab"tet") > # ame but ung a formula > boxplot(rt~tet,tern,ylab"reacton tme",xlab"tet") The boxplot repreent the dtrbuton of the reacton tme when the tet number wa preent (TEST) or mng (TEST) among the prevouly preented et of number. The queton whether the dfference between the two mean tattcally gnfcant. > t.tet(tern$rt[tern$tet],tern$rt[tern$tet], var.equaltrue) Two Sample t-tet data: tern$rt[tern$tet ] and tern$rt[tern$tet ] t , df 98, p-value.3 alternatve hypothe: true dfference n mean not equal to 95 percent confdence nterval: ample etmate: mean of x mean of y m 49(3.79 /5 m 98 / Tet y default, t. tet aume that varance are not equal and ue Welch (or Satterthwate) approxmaton. Settng var.equaltrue wll force - degree of freedom. The null hypothe can be rejected wth confdence becaue the P value (Type error) extremely mall (P, two-taled T tet). IIT Pared T tet 36 Objectve: Tetng the dfference between the mean of two varable meaured on the ame expermental unt {(x, x ),, (x, x )} where the number of expermental unt. Aumpton: The dfference d x -x are ndependent and normally dtrbuted. ull hypothe: H : µ µ, the two mean are equal. ote that th equvalent to ay that the mean m d of the dfference d zero(h : md ) Stattc: Let compute the dfference d x -x for each par of ample (,,). The tattc m d t / where m d and d are the mean and tandard devaton of d repectvely. Theoretcal tattc dtrbuton: If the null hypothe and our aumpton are true, then the t-tattc follow a t-dtrbuton wth - degree of freedom. ote that the procedure for pared t-tet correpond to a one-ample t tet for the dfference d. d IIT t p d p d )

10 Ozone data et The ozone data et gve the maxmum ozone concentraton n Stamford and Yonker for each day of a fve month perod. Mng data are coded wth the value Stamford Yonker Day ozone<-read.table("ozone.dat",headertrue) # read data ozone[ozone-999]<-a # replace mng data wth A ozone$month<-ordered(ozone$month, # order factor month levelc("may","june","july","augut","september")) > table(tmf.na(ozone$tmf), # count mng data ykr.na(ozone$ykr)) ykr tmf FALSE TRUE FALSE 3 4 TRUE 6 x<-ozone[,c("tmf","ykr")] # plot tme ere matplot(row(x),x,type"l",xaxt"n",xlab"day",ylab"ozone", la,lty,colc("red","blue")) ax(,atwhch(ozone$day),labpate(ozone$month,ozone$day)[ozone$day]) legend("topleft",c("stamford","yonker"),lty,lwd,colc("red","blue"),bty"n") 37 IIT Ozone May June July Augut September Pared t-tet 38 5 Plot Stamford agan Yonker concentraton 5 > plot(ozone$tmf,ozone$ykr,la,ap xlab"stamford",ylab"yonker") 5 5 Stamford > t.tet(ozone$tmf,ozone$ykr, paredtrue) Pared t-tet data: ozone$tmf and ozone$ykr t 3.44, df 3, p-value <.e-6 alternatve hypothe: true dfference n mean not equal to 95 percent confdence nterval: ample etmate: mean of the dfference Cae (day) where there a mng value for one of the two cte are excluded from th analy. The number of degree of freedom (3) correpond to number of day where the ozone concentraton ha been meaured for both cte mnu. The tet hghly gnfcant (P, two-taled T tet), ndcatng that the mean m d not equal to zero and, therefore, that the two mean are dfferent. IIT Yonker 39 IIT Obervaton n Stamford and Yonker are correlated (whch hould be expected nce they were taken on the ame day). 4 IIT

11 Aanaly of varance (AOVA) One-way AOVA Effect ze Multple Comparon Checkng Aumpton Tranformaton on-parametrc tet One-way AOVA The am of the one-way AOVA to tet whether the mean of two or more group are equal or not. In the one-way AOVA, there a unque ndependent varable or experment factor (A) that manpulated by the expermenter. The dependent varable (Y) meaured n dfferent expermental condton whch correpond to dfferent value or level of the expermental factor. In other word, the value of the dependent varable (or the level of the expermental factor) determne the group to whch the obervaton belong. The expermental unt mut have been agned randomly to each treatment (fully randomzed degn). 4 IIT 43 IIT Expermental degn and the AOVA The Analy of Varance (AOVA) one of the mot common tattcal technque n pychology and medcal cence The AOVA can be ued to analyze the effect of manpulaton (or expermental factor) on the mean value of the obervaton n a large varety of tuaton (or expermental degn). Termnology: Completely randomzed, block randomzed or repeated-meaure degn refer to the way treatment have been agned to the expermental unt One-way, two-way, n-way AOVA refer to the number of expermental factor. Factoral AOVA refer to a partcular way of analyzng expermental degn wth two or more experment factor whch allow one to ae the poble nteracton effect of nteracton. One-way AOVA Objectve: Compare the mean of g ample {y,,y n },, {y g,,y gng } where the ample ze of the th ample (n n g ) Aumpton: Data are ndependent and normally dtrbuted. The varance of each group approxmately equal. ull hypothe: H : µ µ g, the mean of all group are equal. Stattc: The F-tattc the rato F E / / g g MS MS E where the treatment (or between-group) um of quare, E the error (or wthn-group) um of quare. Theoretcal tattc dtrbuton: If the null hypothe and our aumpton are true, then the F-tattc follow a Fher dtrbuton wth g- and -g degree of freedom. 4 IIT 44 IIT

12 Sum of quare 45 y Uncorrected um of quare T ( y y ) Y Total um of quare (alo called corrected um of quare for the y ) R ( y ˆ y ) Explaned um of quare R ( y y ˆ ) ε E Redual um of quare (or um of quare of the error, or unexplaned um of quare) X ( x x) Corrected um of quare for the x SXY ( x x )( y y ) Corrected um of quare of the product IIT Why the AOVA work? 46 It can be proved that the total um of quare (T) equal to the treatment um of quare plu the error um of quare T 3 5 g n ( y m ) j E MS an etmate of the varance of the mean: MS g n m g g ( m ) where m the mean of the th ample and m the general (or grand) mean. ote that MST f H true. The mean quare error MSE an etmate wthn-group varance MS E E g ote that we have aumed that the varance wa the ame for all group. F become larger f the dfference between the mean are mportant and, therefore, the varance between group larger. F become maller f there much uncertanty about the mean (that, a larger wthn group varance) g g n n ( y m ) j IIT j j Reacton tme (/) Example 47 Tet f the mean reacton tme (rt) depend on the number of dgt memorzed. > boxplot(rt~ndgt,tern,outlerfalse, ylab"reacton tme (/)", xlab"umber of dgt") 3 5 > ft<-aov(rt~ndgt,tern) > anova(ft) Analy of Varance Table The AOVA table ndcate that the null hypothe can be rejected wth hgh confdence (F(,97)33.4,P<.). Repone: rt Df Sum Sq Mean Sq F value Pr(>F) ndgt e-4 *** Redual Sgnf. code: ***. **. *.5.. F MS MS E IIT umber of dgt Reacton tme (/) umber of dgt Multple comparon 48 The F-tet n the AOVA tell u that there at leat one group that ha a mean that tattcally dfferent from the mean of the other group. Problem: How can we dentfy whch group have mean that are tattcally dfferent one from another? One approach would be to ue the t-tet to make everal two-by-two (or parwe) comparon. The problem that the probablty of makng a fale dcovery (.e., the rk of fndng a tattcally gnfcant dfference between two group when the null hypothe true) ncreae wth the number of tet. ote that the problem extremely general and occur not only when comparng mean but when one make multple tet. When makng multple tet to fnd out ome effect (data noopng), we alway ncreae the rk of makng a fale dcovery. IIT

13 onferron nequalty 49 Probablty of makng a fale dcovery n a ere of n tet n 3 5 onferron lmt Independent tet onferron nequalty: There a theorem that ay that the probablty α of makng a fale dcovery n a ere of n tet maller than the um of the probablte α of makng a Type I error n the ndvdual tet. α α L α n If all tet are ndependent, then we can compute the probablty of makng a fale dcovery n a ere of n tet: α ( α where α the rk of a Type I error n the ndvdual tet ) n IIT A pror or planned comparon 5 In general, th tuaton nvolve a lmted number of tet. In the cae of comparon between mean, the uual approach, baed on onferron nequalty, to ue t-tet wth an alpha level α for the ndvdual tet equal to the dered alpha level α of protecton dvded by the number of tet α α n where α the adjuted alpha level for th t tet. ote th th method extremely general and could be ued to adjut the alpha level each tme we make multple tet. A partcular type of hypothe that one mght want to tet, for example, whether the dfference between the mean of a ubet of group dfferent from the mean from another ubet of group. In that cae, one mght ue contrat (we wll not ee contrat n th cla). Another poblty to ue a t tet between the two ubet (remember to adjut the alpha accordng to the number of tet performed). IIT Comparng mean 5 There are everal technque to adjut the level of confdence of the ndvdual tet to control for the rk of a fale dcovery when comparng mean between dfferent group. We need however to dtnguh between the followng tuaton: A pror or planned-comparon: One ha dentfed n advance whch par of mean (or combnaton of mean) hould be teted. Pot-hoc comparon: In abence of pecfc hypothe, one want to make all parwe comparon between the mean of everal treatment. ote that the number of comparon (tet) n pot-hoc comparon uually much hgher than n planned-comparon. IIT Parwe comparon 5 A parwe comparon tet examne the dfference of every par of mean. ote that there are g(g-)/ par of mean for g group. Parwe comparon tet: Sheffe method onferron Sgnfcant Dfference (SD) Tukey Honet Sgnfcant Dfference (HSD) Ryan-Eno-Gabrel-Welh-Range (REGWR) Student-ewman-Keul (SK) Leat Sgnfcant Dfference (LSD) The mot ued tet the HSD tet. It more powerful than Sheffe method and SD when all par are condered. The other tet mght be more powerful but provde le protecton agant Type I error (ee defnton of error rate n Stattcal book). IIT

14 Tukey HSD 53 The functon TukeyHSD can be ued to compute Tukey Honet Sgnfcant Dfference > ft<-aov(rt~ndgt,tern) > TukeyHSD(ft, whch"ndgt") Tukey multple comparon of mean 95% famly-we confdence level Ft: aov(formula rt ~ ndgt, data tern) $ndgt dff lwr upr p adj The multcomp package offer a convenent nterface to perform multple comparon n a general context. Reference: retz F, Hothorn T, Wetfall P () Multple Comparon Ung R. Routledge. IIT Cohen d and f 55 One-ample t tet d µ µ m σ µ Two-ample t-tet µ d m µ σ m one-way AOVA σ f σ ( µ ) p µ p σ p MS MSE MSE p value of f. a mall effect ze, f.5 a medum effect ze, and f.4 or larger a large effect ze. IIT Effect ze 54 A tattcally gnfcant tet doe mean that the dfference acro goup are large, relevant or meanngful, n partcular f the ample ze large (ee Hypothe tetng). Several meaure of the trenght of aocaton of effect ze ext: Cohen d and f R, η, ω The computaton of the effect ze depend on the expermental degn. We conder only the mplet cae. IIT Effect ze (η ) 56 The mot common meaure of effect ze expre treatment magntude a a proporton of the total varablty that aocated wth the effect The mplet meaure of effect ze η (alo called R ): η R T For example, n the prevou example, the tmulu length explan 8% of the varance of the oberved reacton tme (η 957./ ). ote that t poble to compute confdence nterval around effect ze (Smthon, 3) but we won t cover t here (ee Tabachnck & Fddell, 7, for an ntroductory tattcal textbood that cover t) IIT

15 Effect ze (ω ) 57 The mplet meaure of effect η a decrptve tattc baed on the proporton of varance accoundted n a partcular ample and tend to overetmate the ze of the effect n the populaton. An alternatve meaure, called ω, an unbaed parameter etmate that generalze to the populaton ˆ ω T df MS MS E df df ( F ( F ) ) where the total number of ubject (both formulae are equvalent). ecaue t adjut the overtmaton of effect ze by η, t alway maller than η. In the prevou example, ˆ ω (33.4 (33.4 ) ) 3.77 IIT The aumpton of the AOVA 59 Independence of the obervaton. Techncally, th aumpton tate the the redual ε j are ndependent: cov(ε j, ε kl ). For any two obervaton wthn an expermental treatment, we aume that knowng how one of thee obervaton tand relatve to the treatment mean tell u nothng about the other obervaton. Th one of the reaon why ubject are randomly agned to group. Volaton of th aumpton can have erou conequence for an analy. Homogenety of the varance. The varance nde each group are equal (Var(ε j ) σ ). ormalty. Obervaton are dtrbuted normally around ther mean: y j ~(µ,σ) or, equvalently, ε j ~(,σ). The number of treatment (group) fxed. It n prncple poble to conder tuaton where the number of group condered n the analy conttute only a ubet of all poble group/treatment. In the followng, we hall however aume that all poble group/treatment have been condered. IIT Effect ze The object returned by anova a data.frame. The element of th table can be elected and manpulated. > x<-anova(aov(rt~ndgt,tern)) >.data.frame(x) [] TRUE Compute Cohen f > df<-x["ndgt","df"] > dfe<-x["redual","df"] > MS<-x["ndgt","Sum Sq"] > MSE<-x["Redual","Mean Sq"] > qrt(df/dfe*(ms-mse)/mse) [] Compute eta > x["ndgt","sum Sq"]/um(x[,"Sum Sq"]) [] Compute omega > df<-x["ndgt","df"] > F<-x["ndgt","F value"] # F rato > <-um(x[,"df"]) > df*(f-)/(df*(f-)) [].7693 Checkng the aumpton The qualty of our nference depend on the valdty of our aumpton. The man aumpton are:. data/error are ndependent. data/error are normally dtrbuted 3. data/error have contant varance (acro group) Independence the mot mportant of thee aumpton, and alo the mot dffcult to accommodate when t fal. ormalty the leat mportant aumpton (for t-tet or AOVA), partcularly for large ample ze. alanced degn (ame number of data n each group) are le uceptble to the effect of non-normalty and non-contant varance. There are two man way of dealng wth volaton of the aumpton: Tranformng the data Ung non parametrc tet 58 IIT 6 IIT

16 Independence 6 Independence the mot mportant of thee aumpton, and alo the mot dffcult to accommodate when t fal. Example of dependence eral dependence tme ere, learnng or fatgue effect drft n meaurng ntrument patal aocaton A graphcal method to dentfy eral dependence to plot the data (or the redual) veru ther tme or the order of acquton To deal wth poble order (learnng or fatgue) effect, t good practce to randomze the order of preentaton of tmul and vary th order acro partcpant. IIT Example. Seral dependence 6 Ozone data: plot maxmum ozone concentraton veru the day for Stamford. To plot all cae, Select Graph/Lne Select Value of ndvdual cae and Smple Select Lne Repreent (tmf) and Cae number n Category Labek 3 Auto correlaton plot: Select Graph/Tme Sere/ Autocorrelaton Select Varable (tmf) There ome evdence that conecutve data are potvely correlated at lag (bar above the horzontal lne) Cae umber Syntax IIT 9 Value STMF on-contant varance 63 Several tet ext to tet equalty of varance acro ample (homocedatcty): artlett tet Levene tet rown-forythe tet artlett' tet entve to departure from normalty. That, f the ample come from non-normal dtrbuton, then artlett' tet may mply be tetng for non-normalty. The Levene tet and rown Forythe tet are alternatve to the artlett tet that are le entve to departure from normalty. The mot common devaton from contant varance are thoe were the redual varaton depend on the mean. You can plot the tandard devaton agant the mean for each group to check f there a problem. To tablze the varance when t ncreae wth the mean, you can tranform the varable wth the quare root or logarthm functon. IIT Lag umber GRAPH /LIE(SIMPLE)VALUE( tmf ). ACF VARIALES tmf /OLOG /MXAUTO 6 /SERRORID. levene.tet (lbrary car) 64 levene.tet(y, data,...) levene.tet(y, group,...) Decrpton Tet for homeogenety of varance acro group. Man argument y repone varable for the default method, or formula object. If y a formula, the varable on the rght-hand-de of the model mut all be factor and mut be completely croed. group factor defnng group. data data frame contanng dependent and factor referred to n the formula. ote Th functon part of the lbrary car (J. Fox). IIT

17 Aeng non-normalty ormalty the leat mportant aumpton (for t-tet or AOVA), partcularly for large ample ze. Graphcal method Htogram boxplot (ueful for dentfyng quckly outler). probablty blot (alo named theoretcal quantle-quantle plot or rankt plot) Formal method Ch quare tet Kolmogorov-Smrnov tet Outler are an extreme form of non-normalty and mght have an mportant effect. An outler an obervaton dfferent from the bulk of the data. Revewng all the method to deal wth outler outde the cope of th coure. If ome data unequvocally an outler that need to be deal wth, t poble to replace the outler wth the mean value nde the group to remove the outler from the analy [R] tetng aumpton Tet normalty > k.tet(tern$rt,y"pnorm",meanmean(tern$rt),dd(tern$rt)) One-ample Kolmogorov-Smrnov tet data: tern$rt D.8, p-value.3465 alternatve hypothe: two-ded Warnng meage: In k.tet(tern$rt,y"pnorm", meanmean(tern$rt),d(tern$rt)): cannot compute correct p-value wth te The preence of te generate a warnng, nce contnuou dtrbuton do not generate them. Tet equalty of varance > lbrary(car) > levene.tet(tern$rt, a.factor(tern$ndgt)) Levene' Tet for Homogenety of Varance Df F value Pr(>F) group IIT 67 IIT Kolmogorov-Smrnov tet k.tet 66 k.tet(x, y,..., alternatve c("two.ded", "le", "greater"), exact ULL) Decrpton Perform one or two ample Kolmogorov-Smrnov tet. Man argument x a (non-empty) numerc vector of data value.. y ether a numerc vector of data value, or a character trng namng a cumulatve dtrbuton functon or an actual cumulatve dtrbtuton functon uch a pnorm alternatve pecfy the alternatve hypothe: two-taled "two.ded" (default), one-taled "greater" or "le". Detal If y numerc, a two-ample tet of the null hypothe that x and y were drawn from the ame contnuou dtrbuton performed. Alternatvely, y can be a character trng namng a contnuou (cumulatve) dtrbuton functon, or uch a functon. In th cae, a one-ample tet carred out of the null that the dtrbuton functon whch generated x dtrbuton y wth parameter pecfed by... IIT Tranformaton 68 Score mght be tranformed to better for the followng reaon:. to acheve homogenety of error varance. to acheve normalty of error effect 3. to obtane addtvty effect Poble tranformaton nclude The quare root tranformaton: y qrt(y) the logarthmc tranformaton: y log(y) or y log(y) the recprocal tranformaton: y /x or y /(y) etc. Gven the relatve robutne of the AOVA to devaton from the normalty or homocadetcty aumpton, t not clear whether tranformng thendata derable. Stll, t conventonal to tranfrom ome type of data. IIT

18 Data wth non-normal dtrbuton 69 Data Tranformaton Some data do not have a normal dtrbuton and a varance that depend on the mean. For example, Proporton (nomal) Coeffcent of correlaton arcn log ( p ) r r Proporton Coeffcent of correlaton Poon dtrbuted data (e.g. count, nterpke nterval) Count (Poon) y To perform a t-tet or an AOVA on data of th type, t neceary tranform the varable wth the functon ndcated n the table. For proporton and count, t alo poble to ue generalzed lnear model (logtc regreon, log-lnear model, etc.). IIT Example - Ren Data Set 7 The Ren Data Set: Th data et repreent the reult of a tet of the lfetme (n hour) of an encapulatng ren ued n the contructon of ntegrated chp. Thrty even unt were agn at random to one of fve dfferent temperature tre (temp varable, n Celu). The tme varable gve the tme (n hour) and the tme untl the unt faled. Count Tme (hour) Htogram (and probablty plot) how that data are not normally dtrbuted and boxplot how that varance vary wth the mean. The ame plot wth the log of the lfetme how that the data atfy better the aumpton of the AOVA (next lde). The AOVA on log of the lfetme how that the null hypothe can be rejected wth confdence. Tme (hour) Ren Data Set Tme log(hour) IIT AOVA LOGTIME Temperature (Celu) etween Group Wthn Group Total Sum of Square df Mean Square F Sg E IIT Count Tme log(hour) Count 5 5 Tme (hour) Vt duraton dtrbuton Tme (hour) ormal quantle Temperature (Celu) log(vt duraton dtrbuton) Temperature (Celu) - - ormal quantle on-parametrc tet 7 on-parametrc tet can be ued when aumpton of parametrc tet are not atfed. Parametrc on-parametrc* ndependent ample dependent ample ndependent ample T tet Pared T tet One-way AOVA Mann-Whtney U tet Wlconxon tet Sgn tet Krukal-Wall tet Medan tet on-parametrc tet are le powerful than parametrc tet. IIT

19 wlcox.tet (R) 73 wlcox.tet(x, y ULL, alternatve c("two.ded", "le", "greater"), mu, pared FALSE, exact ULL, correct TRUE, conf.nt FALSE, conf.level.95,...) Decrpton Perform one and two ample Wlcoxon tet on vector of data; the latter alo known a Mann-Whtney tet.. Man argument y an optonal (non-empty) numerc vector of data value.. alternatve pecfy the alternatve hypothe: two-taled "two.ded" (default), one-taled "greater" or "le". mu a number ndcatng the true value of the mean (or dfference n mean f you are performng a two ample tet). pared a logcal ndcatng whether you want a pared t-tet data frame contanng dependent and factor referred to n the formula. exact a logcal ndcatng whether an exact p-value hould be computed. correct a logcal ndcatng whether to apply contnuty correcton n the normal approxmaton for the p-value. IIT [R] on parametrc tet 75 > wlcox.tet(tern$rt[tern$tet],tern$rt[tern$tet]) Wlcoxon rank um tet wth contnuty correcton data: tern$rt[tern$tet ] and tern$rt[tern$tet ] W 898, p-value 4.83e-5 alternatve hypothe: true locaton hft not equal to > krukal.tet(rt~ndgt,tern) Krukal-Wall rank um tet data: rt by ndgt Krukal-Wall ch-quared , df, p-value <.e-6 IIT krukal.tet (R) 74 krukal.tet(x, g,...) krukal.tet(formula, data, ubet, na.acton,...) Decrpton Perform a Krukal-Wall rank um tet... Man argument x a numerc vector of data value, or a lt of numerc data vector. g a vector or factor object gvng the group for the correpondng element of x. Ignored f x a lt. formula a formula of the form lh ~ rh where lh gve the data value and rh the correpondng group. data an optonal matrx or data frame contanng the varable n the formula. IIT