Unit 5: Assumptions and Robustness of t-based Inference. Chapter 3 in the Text

Size: px

Start display at page:

Download "Unit 5: Assumptions and Robustness of t-based Inference. Chapter 3 in the Text"

George Warner
5 years ago
Views:

1 Uit 5: Assumptios ad Robustess of t-based Iferece Chapter 3 i the Text

2 Uit 5 Outlie Assumptios ad Robustess Idepedece Assumptio Normality Assumptio Equal Variace Assumptio Formal F-test of Equal Variace

3 t-based Assumptios I order to perform ifereces usig the t-distributio (as see i Uit 4), what assumptios were we makig o the geerative process of the data? ) Idepedece of Observatios ) Observatios come from a Normal Distributio For the -sample t-test (ad related C.I.), this ca be summarized as: i i. i. d. ~ N, The questio is: what happes to the properties of these procedures whe these assumptios break dow? 3

4 -sample t-based Assumptios For the -sample t ifereces, we have more assumptios: I the upooled t-procedures, the added assumptio is that the two groups are idepedet. How did we take advatage of this assumptio i doig calculatios? So we assume: i. i. d. i. i. d., i N, i,, ad idepedetly ~ N ~ I the pooled t-procedures, what is the oe glarig extra assumptio? The two groups have equal variace: 4

5 Robustess defied The performace of a iferetial procedure whe the assumptios fail is called its robustess. What should we measure i order to quatify robustess? Type I error ad Power/Type II error! Type I error: the probability of icorrectly rejectig the ull hypothesis whe the ull hypothesis is actually true. What is this typically set to (whe assumptios are correct)? Power: the probability of correctly rejectig the ull hypothesis whe the alterative hypothesis is actually true. What s difficult about calculatig power? It s usually a rage of values for the parameter so we have to pick a specific value (or set of values) 5

6 Uit 5 Outlie Assumptios ad Robustess Idepedece Assumptio Normality Assumptio Equal Variace Assumptio Formal F-test of Equal Variace 6

7 How the Idepedece Assumptio Ca Break Dow Subgroups of uits similar to each other (cluster effect). If they are cetered aroud differet averages, the regressio will allow us to cotrol for it. Depedece over time (serial effect). Iterferece across space (spatial effect). Check: thik carefully about how data were collected; sometimes graphics ca help. Whe assumptio is violated: Samplig variace calculatios are icorrect; more advaced calculatios eeded (e.g., S&R Ch.5); or redefie uits! 7

8 Graphical Checks for Idepedece Everybody Split by batch -axis is either ) Time ) Observatio # (chroologically) 8

9 Uit 5 Outlie Assumptios ad Robustess Idepedece Assumptio Normality Assumptio Equal Variace Assumptio Formal F-test of Equal Variace 9

10 Desity Desity Graphical Checks: Normality Public Private Tuitio, i $ Tuitio, i $000 Visual examiatio: Shapes Skewed (left, right) or symmetric (log or shorttailed), uimodal/multimodal. 0

11 Desity Desity Graphical Checks: Normality Public Private Tuitio, i $ Tuitio, i $000 Overlay the desity plot with the kerel desity ad the ormal curve (for example, fitted by the method of momets, MOM).

12 Graphical Checks: R-code x <- data$tuitio[data$public==] hist(x, col="light gray", mai="public", xlab="tuitio, i $000", prob=true, breaks = 0, cex.mai=,cex.lab=) # Kerel desity lies(desity(x, adjust = ), col = "blue", lwd=) # Approx. ormal curve, fitted usig MOM poits(seq(mi(x), max(x), legth.out=500), dorm(seq(mi(x), max(x), legth.out=500), mea(x), sd(x)), type="l", col="red", lwd=, lty=)

13 Other Graphical Checks for Normality Public Private Tuitio Tuitio Box-plot (media, lower/upper quartiles, outliers) I R: boxplot() 3

14 Sample Quatiles Sample Quatiles Other Graphical Checks for Normality Public Private Theoretical Quatiles Theoretical Quatiles Normal quatile-quatile (QQ) plot (has to follow a straight lie). I R: qqorm(), qqlie() 4

15 Sample Quatiles Sample Quatiles Sample Quatiles Sample Quatiles QQ-plots for samples from various distributios Normal Dist.( = 0) Normal Dist. ( = 00) Normal Theoretical Quatiles Normal Dist.( = 0) Theoretical Quatiles Normal Dist. ( = 00) Chi-squared, 3 Rightskewed Theoretical Quatiles Theoretical Quatiles 5

16 Sample Quatiles - 0 Sample Quatiles Sample Quatiles Sample Quatiles Sample Quatiles Sample Quatiles - 0 Sample Quatiles Sample Quatiles Sample Quatiles QQ-plots from may ormal samples Normal Q-Q Plot Normal Q-Q Plot Normal Q-Q Plot i ~ N(0,) for = 0 i each of 9 samples Theoretical Quatiles Normal Q-Q Plot Theoretical Quatiles Theoretical Quatiles Normal Q-Q Plot Theoretical Quatiles Theoretical Quatiles Normal Q-Q Plot Theoretical Quatiles Normal Q-Q Plot Normal Q-Q Plot Normal Q-Q Plot Theoretical Quatiles Theoretical Quatiles Theoretical Quatiles 6

17 Aalytical Tests for Normality. Aderso-Darlig Test (R: ad.test() i ortest package). Shapiro-Wilk Test (R: shapiro.test()) 3. Kolmogorov-Smirov Test (R: ks.test()) For small samples they have very low power may ot reject eve for a o-normal sample. For large samples will always reject. The o-normality should be assessed i a way that is relevat to the problem at had: i.e., the distributio does ot have to be exactly Normal for a t-test to still be valid. 7

18 Uiform Distributio ~ Uif ( a, b), a b, has a Uiform distributio, or if each value o the iterval [a,b] is equally probable. If a=0 ad b=, the PDF: CDF: 8

19 Normality Assumptio: Simulatio Let, be i.i.d. ad, be i.i.d. radom variables, i i. i. d. ~ N(, ), i. i. d. N(, ) Suppose we test H 0 : 0 agaist ay alterative (oe or two-sided). The resultig p-value is also a radom variable. It turs out that p-value ~ Uif(0,) uder the ull if the samplig distributio used is correct ad exact! i ~ 9

20 = 50 Normal 0 T S p ~ t = 50 Normal 0 If we pick α=0.05 (Type-I error) 0

21 = 50 Normal 0 T S p ~ t = 500 Normal 0

22 = 5 Normal 0 T S p ~ t = 5 Normal 0

23 = 50 Skewed Normal 0 T S p ~ t = 50 Skewed Normal 0 3

24 = 50 Skewed Normal 0 T S p ~ t = 50 Skewed Normal 0 4

25 = 50 Uiform 0 T S p ~ t = 50 Expoetial 0 5

26 = 0 Uiform 0 T S p ~ t = 30 Expoetial 0 6

27 Normality Assumptio t-tests are fairly robust to departures from ormality, especially i large samples (CLT). Whe the sample sizes are ot equal, t-tests are more sesitive to skewedess ad log-tailedess. For small samples, t-tests are somewhat sesitive to markedly differet skewedess i two groups. Watch out for outliers. Whe ormality assumptio is violated: t-test is usually still valid, or Use data trasformatio (later), or Use o-parametric test (Uit 6). 7

28 Uit 5 Outlie Assumptios ad Robustess Idepedece Assumptio Normality Assumptio Equal Variace Assumptio Formal F-test of Equal Variace 8

29 Equal Variace Assumptio Whe sample sizes are equal, the pooled t-test is fairly robust to uequal variaces. Whe sample sizes are uequal, the pooled t-test is typically ot valid for uequal variaces; the upooled t-test is a robust alterative. Whe assumptio violated: Use upooled t-test; or Use data trasformatio; or Maybe the populatios are ot comparable? 9

30 Maybe the populatios are ot comparable? 30

31 = 50 Normal 0 T S p ~ t = 50 Normal 0 9 3

32 = 50 Normal 0 T S p ~ t = 500 Normal 0 9 3

33 = 500 Normal 0 T S p ~ t = 50 Normal

34 = 500 Normal 0 T ~ t~ S S = 50 Normal

35 Upooled (Welch) Two-Sample t-test Let,, ad,, be i.i.d. r.v.s, H 0 : vs. H A : Test Statistic: Samplig distributio of T uder H 0 is approximately 35 ), ( ~ ),, ( ~ d i i i d i i i N N where ~ ~, t T 0 0 S S T 0) (

5 0 5 0 5 30 35 Equal Variaces : Graphical Check Adjust the rage. Look out for outliers! Box plots may also be used.

36 Equal Variaces : Graphical Check Adjust the rage. Look out for outliers! Box plots may also be used. > by(data$tuitio,data$public,sd) data$public: 0 [] data$public: [] Tuitio Split by =Public ad 0=Private 0 Coclusio? 36

37 Uit 5 Outlie Assumptios ad Robustess Idepedece Assumptio Normality Assumptio Equal Variace Assumptio Formal F-test of Equal Variace 37

38 Formal Tests for Equal Variace: F-test Start with graphical check ad look at the ratio of sample variaces! Other approach: F-test for compariso of two variaces. R: var.test() Same assumptios as for t-tests (i.i.d., Normality) H 0 : H A : Test Statistic: F S S / ~ F, F-distributio with - ad - degrees of freedom. Cautio: will reject for large samples, eve whe the ratio is very close to. 38

39 F-distributio ~ ~ Let ad, idepedet of each other. The the followig ratio has a F-distributio with ad d.f. / R ~ F, / Later: very hady for ANOVA & model selectio i Liear Regressio. 39

40 Two-sided p-values for F-test 40 Two-sided p-value for the F-test is defied as *mi(p,-p), where uder H 0. / P s s F p /

Comparing Two Populations. Topic 15 - Two Sample Inference I. Comparing Two Means. Comparing Two Pop Means. Background Reading

Comparing Two Populations. Topic 15 - Two Sample Inference I. Comparing Two Means. Comparing Two Pop Means. Background Reading Topic 15 - Two Sample Iferece I STAT 511 Professor Bruce Craig Comparig Two Populatios Research ofte ivolves the compariso of two or more samples from differet populatios Graphical summaries provide visual