Analyzing Frequencies

Transcription

1 Frquncy (# ndvduals) Frquncy (# ndvduals) /3/16 H o : No dffrnc n obsrvd sz frquncs and that prdctd by growth modl How would you analyz ths data? 15 Obsrvd Numbr 15 Expctd Numbr from growth modl Abalon Sz (mm) Abalon Sz (mm) Analyzng Frquncs Goodnss of Ft Tsts Basd on Ch Squar Kolmogorov-Smrnov/ Shapro Wlks Tsts 1

2 /3/16 Ch Squar Statstc A fundamntal way to analyz frquncy data Usd n both Goodnss of Ft and Basd on th dvaton btwn obsrvd (o) and xpctd frquncs () c Th tst statstc = n = 1 ( o - ) Ch Squar Statstc c = n = 1 o( - ) Approxmats a Ch Squar dstrbuton f th followng assumptons ar tru Obsrvatons ar classfd nto catgors ndpndntly. No mor than about % of clls hav obsrvatons (o) lss than 5. If ths s volatd thn probablts drvd from th Ch Squar Statstc can b msladng. Mal Fmal Undrgraduat dgr Cll Graduat Dgr 11 33

3 /3/16 Analyzng Frquncs Goodnss of Ft Tsts Basd on Ch Squar Is a sampl of data consstnt wth a gvn probablty dstrbuton??? Exampl: A survy s mad of lngths of abalon. Th goal s to dtrmn f th frquncy dstrbuton of szs s that prdctd by a growth modl. Bcaus masurmnts ar takn undrwatr, oftn n cracks thy ar catgorzd nto 1 mm ntrvals Goodnss of Ft Exampl - Abalon Sz Sz Catgory Numbr obsrvd Sum 557 Count 1. Indvdual ar arrangd nto catgors SIZE Proporton pr Bar 3

4 /3/16 Goodnss of Ft Exampl - Abalon Sz Sz Catgory Numbr obsrvd Expctd proportons 1. Indvdual ar arrangd nto catgors. Entr xpctd proportons Sum Goodnss of Ft Exampl - Abalon Sz Sz Catgory Numbr obsrvd Expctd proportons Expctd frquncs 1. Indvdual ar arrangd nto catgors. Entr xpctd proportons 3. Calculat xpctd frquncs 557 x.3 = Sum

5 /3/16 Goodnss of Ft Exampl - Abalon Sz Sz Catgory Numbr obsrvd Expctd proportons Expctd frquncs Indvdual ar arrangd nto catgors. Entr xpctd proportons 3. Calculat xpctd frquncs (xpctd proporton x sum of obsrvatons) Sum Goodnss of Ft Exampl - Abalon Compar th dstrbutons Thy appar dffrnt Dtrmn th probablty that thy ar ndd dffrnt 5

6 /3/16 Sz Catgory Numbr obsrvd Goodnss of Ft Exampl - Abalon Expctd proportons Expctd frquncs Ch Squar valu 1. Indvdual ar arrangd nto catgors. Entr xpctd proportons 3. Calculat xpctd frquncs (xpctd proporton x sum of obsrvatons) 4. Calculat ndvdual Ch squar valus - ( o ) ( 16 - ) =.3 Goodnss of Ft Exampl - Abalon Sz Catgory Numbr obsrvd Expctd proportons Expctd frquncs Ch Squar valu Indvdual ar arrangd nto catgors. Entr xpctd proportons 3. Calculat xpctd frquncs (xpctd proporton x sum of obsrvatons) 4. Calculat ndvdual Ch squar valus 5. Sum Valus c = n = 1 ( o - ) 6

7 /3/16 Dnsty c = n = 1 Goodnss of Ft Exampl Abalon ( o - ) Ch Squar dstrbuton wth 9 df = Ch Squar Us probablty calculator for dmonstraton df = (1-1) = 9 c Valu, crtcal (p=.5) = 16.9 P(36.95, 9) <.1 1. Indvdual ar arrangd nto catgors. Entr xpctd proportons 3. Calculat xpctd frquncs (xpctd proporton x sum of obsrvatons) 4. Calculat ndvdual Ch squar valus 5. Sum Valus 6. Dtrmn probablty that obsrvd dstrbuton quals th xpctd dstrbuton Us Ch Squar tabls or cumulatv dstrbuton functon wth df = k 1 (usually) If dsrd dtrmn crtcal valu for rjcton Goodnss of Ft Exampl - Abalon Concluson Null hypothss can b rjctd Th obsrvd dstrbuton dffrs from th xpctd dstrbuton at p<.1 7

8 /3/16 Analyzng Frquncs Goodnss of Ft Tsts Basd on Ch Squar Shapro Wlks / Kolmogorov-Smrnov Tsts Kolmogorov-Smrnov Tsts (KS) Th on-sampl Kolmogorov-Smrnov tst s usd to compar th shap and locaton of a sampl dstrbuton to a spcfd dstrbuton. Th Kolmogorov-Smrnov tst and ts gnralzatons ar among th handst of dstrbuton-fr tsts. Th tst statstc s basd on th maxmum dffrnc btwn two cumulatv dstrbuton functons (CDF). In th on-sampl tst, on of th CDF s s contnuous (typcally th xpctd dstrbuton) and th othr s dscrt (typcally th obsrvd). 8

9 /3/16 Goodnss of Ft Exampl (KS) - Abalon 1. Dtrmn th dstrbuton wth whch you want to compar to th obsrvd dstrbuton (rcall w ar comparng cumulatv dstrbutons). Assum w wsh to compar to a normal dstrbuton.5 Dnsty functon 1. Cumulatv functon Proporton Cummulatv Proporton Sz Sz Goodnss of Ft Exampl - Abalon 1. Dtrmn th dstrbuton wth whch you want to compar to th obsrvd dstrbuton (rcall w ar comparng cumulatv dstrbutons). Assum w wsh to compar to a normal dstrbuton Count Obsrvd data Man = 61 Standard Dvaton = Cumulatv Dnsty Expctd functon (undr assumpton of normalty) Cummulatv Proporton Man = 61 Standard Dvaton = Sz Sz 9

10 /3/16 Goodnss of Ft Exampl (Shapro- Wlks tst) - Abalon Count Man = 61 Standard Dvaton = Cumulatv Dnsty Sz Analyzng Frquncs Most common form of valuatng catgorcal data n th bologcal scncs Usd for counts of obsrvatons mad n two or mor layrs of catgors (varabls) say - sz of abalon nsd and outsd a rsrv 15 Do ths dstrbutons dffr? Abalon Numbr 1 5 Locaton Sz (mm) Outsd Rsrv Insd Rsrv 1

11 /3/16 1. Gnrally contngncy tabls ar analyzd so that nthr varabl s consdrd as a prdctor or rspons varabl. Thr ar occasons whr varabls can b dstngushd nto prdctor and rspons varabls 3. In practc th analyss of contngncy tabls s not affctd by whthr varabls can b charactrzd as rspons or prdctor th hypothss rman th sam Hypothss tstd s on of Indpndnc H : Th two varabls ar ndpndnt. Null Hypothss H 1 : Th two varabls ar assocatd. Altrnatv Hypothss 11

12 /3/16 W can us a Ch-squard tst (or Gnralzd Lnar Modl) to tst for an assocaton btwn ncdnc of tubrculoss and th ABO blood groups. H : Incdnc of Tubrculoss and ABO blood groups ar ndpndnt. H 1 : Incdnc of Tubrculoss and ABO blood groups ar assocatd. Blood Group Tubrculoss Incdnc Modrat/ Advancd O A AB B Mnmal Not prsnt What s mant by ndpndnc mathmatcally? 1. If varabls ar ndpndnt thn P(A+B) = P(A)*P(B) For xampl what s th probablty of gttng a 4 of damonds n a sngl draw from a dck of cards = P(4)*P(Damond) = (1/13)*(1/4) = 1/5 Thrfor th xpctd lklhood of any cll (undr th hypothss of ndpndnc) s gvn by Probablty of Blood Group * Probablty of Tubrculoss Incdnc [P(BG)*P(TI)] And th xpctd frquncy s N [P(BG)*P(TI)] Blood Group Tubrculoss Incdnc Modrat/ Advancd O A AB B Mnmal Not prsnt

13 /3/16 Calculaton of N [P(BG)*P(TI)] 1. Calculat margnal totals and N Tubrculoss Incdnc Modrat/ Advancd Blood Group O A AB B Total Mnmal Not prsnt Total N Calculaton of N [P(BG)*P(TI)] Calculat margnal totals and N Any cll s xpctd lklhood (f factors ar ndpndnt) s gvn by: [P(BG)*P(TI)] = (Column Margnal Total / N) * (Row Margnal Total/N) For xampl: (89/49)*(8/49) =.4 Blood Group Tubrculoss Incdnc Modrat/ Advancd O A AB B Total Mnmal Not prsnt Total N 13

14 /3/16 Calculaton of N [P(BG)*P(TI)] Calculat margnal totals and N Any cll s xpctd lklhood s gvn by: [P(BG)*P(TI)] = (Column Margnal Total / N) * (Row Margnal Total/N) Multply ach clls xpctd lklhood by N: For xampl.4*49 = 1. Blood Group Tubrculoss Incdnc Modrat/ Advancd O A AB B Total 7 (.4) Mnmal Not prsnt Total N Equvalntly N [P(BG)*P(TI)] = N(Column Margnal Total / N) * (Row Margnal Total/N) = ((Column Margnal Total) * (Row Margnal Total))/N Calculaton of N [P(BG)*P(TI)] Calculat xpctd frquncs for all clls not ths s don automatcally n th Stats program O A AB B Modrat/Advancd 49[(89/49)*(8/49)] 49[(87/49)*(8/49)] 49[(89/49)*(8/49)] 49[(55/49)*(8/49)] Mnmal 49[(89/49)*(85/49)] 49[(87/49)*(85/49)] 49[(89/49)*(8/49)] 49[(55/49)*(85/49)] Not prsnt 49[(89/49)*(136/49)] 49[(87/49)*(136/49)] 49[(89/49)*(8/49)] 49[(55/49)*(136/49)] Expctd Frquncs Tubrculoss Incdnc Modrat/ Advancd O A AB B Mnmal Not prsnt

15 /3/16 1. Rcall that. Hnc: c = n = 1 o( - ) c = (7-1) +..(4 3.1) = Wth dgrs of frdom (df) = (rows 1)(columns-1) = (3-1)(4-1) = 6 Modrat/ Advancd Modrat/ Advancd Obsrvd Frquncs O A AB B Mnmal Not prsnt Expctd Frquncs O A AB B Mnmal Not prsnt H : Incdnc of Tubrculoss and ABO blood groups ar ndpndnt. REJECTED H 1 : Incdnc of Tubrculoss and ABO blood groups ar assocatd. SUPPORTED Dnsty Ch squar dstrbuton (6 df) c calculatd P=.18 = Ch Squar valu Blood Group A AB B O 6 5 MA Mn NP MA Mn NP Tubrculoss Incdnc Tubrculoss Incdnc COUNT COUNT 15

16 /3/16 H : Incdnc of Tubrculoss and ABO blood groups ar ndpndnt. REJECTED H 1 : Incdnc of Tubrculoss and ABO blood groups ar assocatd. SUPPORTED Obsrvd Frquncs Expctd Frquncs (undr assumpton that Tubrculoss Incdnc and Blood Groups ar ndpndnt) Blood Groups A AB B O MA Mn NP MA Mn NP MA Mn NP MA Mn NP Tubrculoss Incdnc Extra slds 16

17 /3/16 H 1 : Incdnc of Tubrculoss and ABO blood groups ar assocatd. SUPPORTED Now w know th varabls ar not ndpndnt, but w do not know what catgory (s) drv th assocaton 1) Us squntal dlton of catgors If thr ar prdctor and rspons varabls thn work on th prdctor varabl Hr, t s unlkly that Tubrculoss causs Blood Groups but Blood Group mght affct Incdnc of Tubrculoss Slct th Catgory that sms most dffrnt from th rst Hr, th dstrbutons of Tubrculoss Incdnc for Blood Groups B and AB sm most dffrnt Dlt that catgory and rpat analyss Rmmbr that df wll chang Blood Group A AB B O 6 5 MA Mn NP MA Mn NP Tubrculoss Incdnc Tubrculoss Incdnc COUNT COUNT H 1 : Incdnc of Tubrculoss and ABO blood groups ar assocatd. SUPPORTED H a : Incdnc of Tubrculoss and ABO blood groups ar ndpndnt (aftr rmoval of group AB) H 1a : Incdnc of Tubrculoss and ABO blood groups ar assocatd (aftr rmoval of group AB) 1) Rmov Blood Group AB from analyss Calculat Ch Squar Valu and assocatd probablty for (3-1)(3-1) = 4 df ` c calculatd = P =.8 Tubrculoss Incdnc and Blood Groups ar not ndpndnt vn aftr rmoval of group AB Try Group B Blood Group A AB B O 6 5 MA Mn NP MA Mn NP Tubrculoss Incdnc Tubrculoss Incdnc COUNT COUNT 17

18 /3/16 H 1 : Incdnc of Tubrculoss and ABO blood groups ar assocatd. SUPPORTED H b : Incdnc of Tubrculoss and ABO blood groups ar ndpndnt (aftr rmoval of group B) H 1b : Incdnc of Tubrculoss and ABO blood groups ar assocatd (aftr rmoval of group B) 1) Rmov Blood Group B from analyss Calculat Ch Squar Valu and assocatd probablty for (3-1)(3-1) = 4 df c calculatd P =.31 = 4.76 Tubrculoss Incdnc and Blood Groups ar ndpndnt aftr rmoval of group B Hnc th assocaton of Tubrculoss Incdnc and Blood Groups s drvn by Blood Group B Blood Group A AB B O 6 5 MA Mn NP MA Mn NP Tubrculoss Incdnc Tubrculoss Incdnc COUNT COUNT 1) Subsqunt analyss could rsolv how Blood Group B s assocatd wth Tubrculoss Incdnc (g mor Modrat and advancd cass than xpctd) How???? A Blood Group AB COUNT B O MA Mn NP MA Mn NP Tubrculoss Incdnc Tubrculoss Incdnc COUNT 18

19 /3/16 Usng Goodnss of Ft Tsts to dtrmn contrbuton of catgors 1) How s Blood Group B assocatd wth Tubrculoss Incdnc 1) Dtrmn xpctd lklhood's of ncdnc basd on ndpndnt varabls (Blood Group x Tubrculoss Incdnc, lavng out group B) LIKELIHOOD MA Mn NP Tubrculoss Incdnc Usng Goodnss of Ft Tsts to dtrmn contrbuton of catgors 1) How s Blood Group B assocatd wth Tubrculoss Incdnc Dtrmn xpctd lklhood's of ncdnc basd on ndpndnt varabls (Blood Group x Tubrculoss Incdnc, lavng out group B) Dtrmn confdnc ntrvals for obsrvd data (for group B) Tubrculoss Incdnc Numbr obsrvd 95% Confdnc ntrval (lowr) 95% Confdnc ntrval (uppr) Modrat / Advancd 13 (.19) 5.95 Mnmal 18 (.179) 9.84 Not Prsnt 4 (.7) (.393) 1.61 (.49) 6.95 (.599)

20 Proporton /3/16 Usng Goodnss of Ft Tsts to dtrmn contrbuton of catgors 1) How s Blood Group B assocatd wth Tubrculoss Incdnc Dtrmn xpctd lklhood's of ncdnc basd on ndpndnt varabls (Blood Group x Tubrculoss Incdnc, lavng out group B) Dtrmn confdnc ntrvals for obsrvd data (for group B) Compar to xpctd lklhood's Tubrculoss Incdnc (Proporton) Numbr obsrvd 95% Confdnc ntrval (lowr) 95% Confdnc ntrval (uppr) Expctd (lklhood's) frquncs Modrat / Advancd (.36) 13 (.19) 5.95 (.393) 1.61 (.77) 4.3 Obsrvd > Expctd Mnmal (.37) 18 Not Prsnt (.436) 4 (.179) 9.84 (.7) (.49) 6.95 (.599) 3.94 (.345) (.577) Obsrvd ~ Expctd Usng Goodnss of Ft Tsts to dtrmn contrbuton of catgors Rsult: Blood Group B s assocatd wth gratr ncdnc of modrat / advancd Tubrculoss MA Mn NP Tubrculoss Incdnc Lklhood Blood Group B