Lecture Outline. Biost 517 Applied Biostatistics I. Comparing Independent Proportions. Summary Measures. Comparing Independent Proportions

Transcription

1 Bost 57 Aled Bostatstcs I Scott S. Emerso, M.D., Ph.D. Professor of Bostatstcs Uversty of Washgto Lecture 4: Two Samle Iferece About Ideedet Proortos Lecture Outle Comarg Ideedet Proortos Large Samles (Ucesored Ch Squared Test Small Samles (Ucesored Fsher s Exact Test Adjusted Ch Squared or Fsher s Exact Test ovember, 6, 3, 5 Scott S. Emerso, M.D., Ph.D. Comarg Ideedet Proortos Large Samles (Ucesored Summary Measures Comarg dstrbutos of bary varables across two grous Dfferece of roortos (most commo Most commo Iferece based o dfferece of estmated roortos Rato of roortos Of most relevace wth low robabltes Ofte actually use odds rato 3 4

2 5 Data: Cotgecy Tables The cross classfed couts Resose Yes o Tot Grou a b c d Total m m 6 Large Samle Dstrbuto Wth totally deedet data, we use the Cetral Lmt Theorem Proortos are meas Samle roortos are samle meas Stadard error estmates for each grou s estmated roorto based o the mea varace relatosh 7 Asymtotc Samlg Dst Comarg two bomal roortos ( ( ( ( + ( (, ~ (, ~ (, ~ We wat to make ferece about, ~, ~ ad, ~, ~ Suose deedet Y X B Y Y B Y B X X B X & & & 8 Asymtotc Cofdece Itervals Cofdece terval for dfferece betwee two bomal roortos ( ( ( ( / / ( ( Estmate, % cofdece terval s ( We wat to make ferece about se se z se z + + α α α

3 Asymtotc Hyothess Tests Test statstc for dfferece betwee two bomal roortos Suose we wat to test H Uder the ull hyothess, Uder H Estmate ( : ( the etre dstrbutos are equal so se ( ( + Z se ~ & (, X + Y wth + 9 Goodess of Ft Test A alteratve dervato of the asymtotc test of bomal roortos as a secal case of the goodess of ft test For cotgecy tables of arbtrary sze R by C table E.g., tumor grade by boe sca score PSA data set Goodess of Ft Test Gve categorcal radom varables, we ofte have scetfc questos that relate to the frequecy dstrbuto for the measuremets Examles: Checkg for Posso, ormal, etc. Dstrbuto of heotyes to agree wth geetc theory Comarg dstrbutos of categorcal data across oulatos Ideedece of radom varables. tabulate grade bss, row col cell bss grade 3 Total Total

4 Asde: Samlg omeclature We ofte characterze the samlg scheme accordg to the total couts that were fxed by desg Posso samlg: oe Multomal samlg: total couts Bomal samlg: ether row or colum totals Hyergeometrc samlg: both row ad colum totals Basc Idea We comare the observed couts each cell to the umber we mght have exected Observed Exected Couts, OT roortos 3 4 Samlg Dstrbuto Test Statstc The test to see whether the data fts the hyotheses (hece goodess of ft Observed couts each cell are assumed to be Posso radom varables Stadard error s the square root of the mea Test statstc based o sum of Z scores for each cell Actually doe as squared Z scores Sum of squared ormals has a ch squared dst 5 Pearso s ch-square goodess of ft statstc the geeral case Assumg oly oe costrat o the cells Gve K O categores, ad for the the observed cout ad K χ E H ( O E ~ χ K th cell E the exected cout (tycally E 6

5 Determg the Exected The scetfc ull hyothess usually secfes the robablty that a radom observato would belog each cell Most ofte: Testg deedece of varables Probabltes each cell are exected to be the roduct of the margal dstrbutos Other uses Testg goodess of ft to dstrbutos Testg agreemet wth geetc hyotheses 7 Test for Ideedece Ch-square test for deedece (or assocato R C table for two categorcal O E rc rc χ where the observed cout ad the exected cout R C ( Oj Ej j r r c ad E c j H ~ χ varables, the ( R ( C are the estmated roortos are for the row ad colum margs, resectvely ( r, c th cell 8. tabulate grade bss, row col cell bss grade 3 Total Total tabulate grade bss, row col ex bss grade 3 Total Total

6 I x Cotgecy Tables Elevator Statstcs The ch squared test ad the Z test comarg bomal roortos are exactly the same test The ch squared statstc s just the square of the Z statstc The ch squared test P value wll be the same as the two-sded P value for the Z test Most software ackages have a tedecy to tell you the value of the ch squared statstc Well, early elevator statstcs χ ( ad bc m m + a b c d m m Z ( ad bc m m eed for Large Samles ote that because the goodess of ft test s relyg o asymtotc roertes, t s oly vald large samles A commoly used rule of thumb f that the exected couts be greater tha 5 the vast majorty of the cells Other Large Samle Tests Lkelhood Rato Test Also has ch squared dstrbuto large samles But ot the same statstc as ch squared statstc Good large samle roertes Ofte most owerful Less commoly used x tables We wll use t more ofte logstc regresso 3 4

7 Stata Commads: CI Ex: Stata Commads cs resvar grouvar, level(# Both varables must be coded as ad Resose wll be called cases ad ocases CI ca be foud uder Rsk dfferece Ch squared statstc ad two-sded P value tabulate resvar grouvar, row col ch lr Row ad colum ercetages Examle: Heatomegaly by treatmet grou PBC data set. g tx treatmt. cs hemeg tx tx Exosed Uexosed Total Cases ocases Total Rsk Ch squared ad lkelhood rato P values 5 6 Ex: Stata Commads (cot. Iterretato Examle: Heatomegaly by treatmet grou PBC data set (cot. Pot estmate [95% Cof. Iterval] Rsk dff Rsk rato Prev frac ex Prev frac o.93 ch( 3.33 Pr>ch Wth 95% cofdece, the true dfferece the revalece of heatomegaly at basele s betwee.7 hgher treatmet grou ad.4 lower treatmet grou. (What are these oulatos? At basele, we oly have samles that are treated ad cotrol. Both grous were draw from the same oulato. Based o two sded P.679, we caot reject the ull hyothess of equalty 8

8 Caveat: Sad Fact of Lfe Comarso to t Test Dfferet varace estmates are tycally used for CI ad hyothess tests We ca see dsagreemet betwee the cocluso reached by CI ad P value The P value mght be less tha.5, but the CI cota The P value mght be greater tha.5, but the CI exclude 9 Heatomegaly by treatmet grou usg two samle t test wth uequal varaces Z test uses the stadard ormal dstrbuto ad does ot use the samle varace.ttest hemeg, by(tx uequal Two-samle t test; uequal varaces : obs 57 : obs 53 Varable Mea St Err t P> t [95% CI] dff Comarso to t Test (cot Yates Correcto Satterthwate's degrees of freedom: Ho: mea( - mea( dff Ha: dff < Ha: dff ~ Ha: dff > t -.83 t -.83 t -.83 P < t.34 P > t.68 P > t.9659 From stadard aalyss CI: -.43,.7; P.679 From ostadard t test based aalyss CI: -.5,.8; P.68 3 Hstorcally, a cotuty correcto to the ch squared test to try to avod ts atcoservatsm small samles All that was acheved was gettg a test that behaves as oorly as the Fsher s exact test I heartly dsrecommed use of the cotuty correcto whe comarg two samles (There s a cotuty correcto used oe samle Z tests that s useful, but exact dstrbutos are eve better 3

9 Comarg Ideedet Proortos Small Samles (Ucesored Small Samle Dstrbuto The exact dstrbuto for the dfferece two roortos ca ot be determed geeral, because of the mea varace relatosh We eed to kow the value of the two roortos beg comared order to fd the exact dstrbuto of the dfferece Small Samle CI We have o way of obtag exact CI for the dfferece roortos We could cosder all ossble values of the two roortos, ad see whether a test would reject each combato But the resultg jot cofdece terval would ot always gve the same decso for equal dffereces E.g, t mght reject. ad., but ot.4 ad.5 Small Samle Tests We ca, however, descrbe the exact dstrbuto of the data uder the ull hyothess codtoal o all the margs of a cotgecy table A ermutato dstrbuto We mage radomly assgg observatos betwee the grous 35 36

10 Permutato Idea Radomly ermute m ostves ad m egatves Call the frst grou ad the last grou Reeat may tmes ad see how ofte grou has a or more ostves Resose + a b Grou c d m m Permutato Tests I usually object to ermutato dstrbutos excet as a last resort They test equalty of dstrbutos, ot just equalty of the oulato arameter Usually they are ot, however, guarateed to detect arbtrary dffereces betwee dstrbutos eve fte samles Codto o values of m, m,, Permutato wth Bary Data However, wth bary data, dstrbutos are dfferet f ad oly f the roortos are dfferet Hece ermutato tests are okay for testg But stll, we have o cofdece tervals because we have ot quatfed alteratves Small Samle Tests Codtog o the margs Ofte oe marg s fxed by desg Cohort studes samle by exosure Case-cotrol studes samle by dsease I ay case, t ca be show that oe of the marg totals cotrbute formato about the dfferece roortos 39 4

11 Fsher s Exact Test Probablty of more extreme cotgecy tables wth the same margal totals Probabltes by hyergeometrc dstrbuto (Use a comuter Resose + a k b + k Grou c + k d k m Cosder all ossble values of m k 4 Stata Commads The Fsher s exact test P values are gve by several commads cs resvar grouvar, exact tabulate resvar grouvar, exact Oe-sded ad two-sded P values are rovded 4 Stata Examle Commets Examle: Heatomegaly by treatmet grou (cot.. cs hemeg tx, exact Pt. Est. [95% CI] Rsk dff Rsk rato Attr fr ex Attr fr o.994 _ -sded Fsher's exact P.433 -sded Fsher's exact P Fsher s exact test does ot tur out to be a exact test ractce A roblem s osed by the dscrete ature of the data To acheve the desred level.5 two-sded test, we would sometmes have to reject the ull whe both grous had successes Wth some results fl a based co to decde whether sgfcat Few eole are wllg to do ths 44

12 Problem We the face a dlemma The ch squared test (Z test for roortos may be at-coservatve small samles Geerally so log as all cell couts the cotgecy table are exected to be greater tha 5 uder the ull hyothess, we are OK The Fsher s exact test s too coservatve Alteratves Great mrovemets statstcal ower obtaed by modfyg ether of those tests to acheve as close to the omal tye I error wthout exceedg Several statstcal ackages rovde such modfed tests (e.g., StatExact Stata does ot Modfcatos True Tye I Error by Commo Basc dea Use the statstc Do t resume the classcal dstrbuto Do t assume ch squared statstc has ch square dstrbuto Do t assume Fsher s Exact P value has uform dstrbuto Cosder all ossble values of commo to both grous, ad use exact dstrbuto The take worst case 47 48

13 Gas Power Geeral Commets Power of uadjusted, adjusted level.5 tests 49 It s geerally mmateral whether the Fsher s exact test P value or the ch square statstc or lkelhood rato statstc s used as the bass for the exact test I ay case, the crtcal value s deedet uo the samle szes Usg ths aroach, substatal mrovemet ower s obtaed low samle szes I strogly recommed ts use whe cofroted wth small samles real lfe 5 Iferece About Odds Ratos Odds of Exceedg a Threshold Prevously: ferece based o the robablty of exceedg a threshold Sometmes t s more coveet to dscuss the odds of exceedg a threshold odds rob / ( rob 5 I oe ad two samle roblems, ferece about the odds s easly obtaed from ferece about the robablty (roorto Ad the roorto s more easly uderstood 5

14 Advatage of Usg Odds Avodg effect modfcato Whe adjustg for cofouders or recso varables, t s tutvely ulkely that dffereces roortos wll be the same across all subgrous Proortos must be betwee ad Odds ca be betwee ad fty (log odds ca be betwee egatve fty ad fty Case cotrol studes Case Cotrol Studes Whe outcome evet s rare Case cotrol samlg s effcet Odds s very close to the robablty ( rob s aroxmately equal to rob Hece, the odds rato s aroxmately the rsk rato Mathematcs Based Logc The odds rato s deedet of the codtoal robablty beg estmated Cohort studes: Pr (Dsease Exosed Case-cotrol studes: Pr (Exosed Dsease Ca cosder Odds Rato for exosure based o dsease from case-cotrol study Equal to Odds Rato for dsease by exosure For rare dsease, ths s aroxmately rato of dsease robablty 55 Odds of Exceedg a Threshold Iferece about the odds s usually made the cotext of the ch squared test I Stata, we ca obta estmates of the odds rato usg cc casevar exvar cc case cotrol Provdes odds rato ad ch squared statstc 56

15 Lookg to the Future I two samle tests, I thk usg dfferece roortos s best Whe multle samles or adjustg for covarates we ted to use logstc regresso Summary measures based o odds rato 57