Statistical Methods. Sample spaces and events. Axioms of Probability. UCLA M284 Principles of Neuroimaging A

Size: px

Start display at page:

Download "Statistical Methods. Sample spaces and events. Axioms of Probability. UCLA M284 Principles of Neuroimaging A"

Winfred Phillips
5 years ago
Views:

UCL M84 riciples of euroimagig Statistical Methods Istructor: : Ivo Diov, sst. rof. I Statistics ad eurology Uiversity of Califoria, Los geles, Witer 007 http://www.stat.ucla.edu/~diov/ http://www.

1 UCL M84 riciples of euroimagig Statistical Methods Istructor: : Ivo Diov, sst. rof. I Statistics ad eurology Uiversity of Califoria, Los geles, Witer Outlie robability Theory xioms Basic riciples for probability modelig ad computatio Law of Total robability & Bayesia Theorem Data Summaries ad ED Distributios ( Experimets & Demos ( Statistical Iferece Hypothesis Testig & Cofidece itervals arameter Estimatio arametric vs. o-parametric iferece ( CLT & LL Liear modelig Simple liear regressio, Multiple liear regressio OV & GLM Outlie robability Theory xioms Basic riciples for probability modelig ad computatio Law of Total robability & Bayesia Theorem Distributios ( Experimets & Demos ( Statistical Iferece Hypothesis Testig & Cofidece itervals arameter Estimatio arametric vs. o-parametric iferece ( Liear modelig Simple liear regressio, Multiple liear regressio OV & GLM Sample spaces ad evets sample space, S,, for a radom experimet is the set of all possible outcomes of the experimet. E.g., Roll a pair of fair Hexagoal dice, S? evet is a collectio of outcomes. E.g., E {a eve sum turs up} evet occurs if ay outcome maig up that evet occurs. E.g., E occurs if total sum is oe of: {, 4, 6, 8, 0 or } E? R.V.: X D + D : S R xioms of robability Let. be a fuctio, that has these 3 properties. for ay evet E, 0 E 0.. S, where S is the sample space. 3. For ay fiite (or ifiite collectio of mutually exclusive evets y fuctio that satisfies the above three axioms is a probability fuctio. U U ( (

2 The complemet of a evet,, deoted, c, `, occurs if ad oly if does ot occur. S The complemet of a evet (a Sample space cotaiig evet (b Evet shaded (c shaded roperties of robability Fuctios.E C E.If E ad E are logically equivalet, the EE. E: ot all cars are worth > $0K. E: Some cars are worth $0K. The EE. 3.E E mi (E,( E. robability ad Ve diagrams i + ( + ( r+ + i i < i i < i<... < ir ( i I i ( I I I i i... i + ( i I i I... I i r + Melaoma type of si cacer type of si cacer a example of laws of coditioal probabilities 400 Melaoma atiets by Type ad Site Site Head ad Row Type ec Tru Extremities Totals Hutchiso's melaomic frecle 0 34 Superficial odular Idetermiat Colum Totals Cotigecy table based o Melaoma histological type ad its locatio Coditioal robability The coditioal probability of occurrig give that B occurs is give by Remars I pr( B, how should the symbol is read give that. How do we iterpret the fact that: The evet always occurs whe B occurs?? What ca you say about pr( B? B pr( B pr( ad B pr(b B Suppose we select oe out of the 400 patiets i the study ad we wat to fid the probability that the cacer is o the extremities give that it is of type odular: 73/5 C. o Extremities odular #odular patiets with cacer o extremities #odular patiets Whe drawig a probability tree for a particular problem, how do you ow what evets to use for the first fa of braches ad which evets to use for the subsequet brachig? (at each brachig stage coditio o all the ifo available up to here. E.g., at first brachig use all simple evets, o prior r is available. t 3-rd brachig coditio of the previous evets, etc..

3 Statistical idepedece Evets ad B are statistically idepedet if owig whether B has occurred gives o ew iformatio about the chaces of occurrig, i.e. if pr( B pr( Similarly, B B, sice B B & / BB/ B If ad B are statistically idepedet,, the Ivertig Coditioal robabilities B B x B B x pr( ad B pr( pr( B Formula summary cot. Multiplicatio Rule uder idepedece: If ad B are idepedet evets, the B B If,,, are mutually idepedet,... If {{,,,, } are a partitio of the sample space (mutually exclusive ad U i S the for ay evet B Ex: B B + B B B B + B Law of Total robability S B B B Bayesia Rule If {{,,,, } are a o-trivial partitio of the sample space (mutually exclusive ad U i S, i >0 the for ay o-trivial evet ad B ( B>0 I i B i B / B [B i x i ] / B B i i B Bayesia Rule i B i i D the test perso has the disease. B T the test result is positive. Ex: (Laboratory blood test ssume: Fid: positive Test Disease 0.95 Disease positive Test? positive Test o Disease0.0 D T? Disease D IT T D D D T T c c T D D + T D D

4 Classes vs. Evidece Coditioig Classes: healthy(c, cacer Evidece: positive mammogram (pos, egative mammogram (eg If a woma has a positive mammogram result, what is the probability that she has breast cacer? evidece class class class evidece cacer evidece class class 0. 0 classes pos cacer 0. 8 ( pos healthy 0. cacer pos? C CxC/[ CxC+ HxH] C 0.8x0.0 / [0.8x x0.99]? Test Results Bayesia Rule (differet data/example! egative ositive Total True Disease State o Disease Disease OK ( False egative II ( False ositive I ( C ( C ( C T I D T D D OK ( Total ower of Test T C D / Sesitivity: T/(T+F /( Specificity: T/(T+F /( Factors affectig the power ED Larger: Causes: Sample size (positive Sample variace (egative Effect size (positive The chose level for α (positive robability Theory xioms Basic riciples for probability modelig ad computatio Law of Total robability & Bayesia Theorem Data Summaries ad ED Distributios ( Experimets & Demos ( Statistical Iferece Hypothesis Testig & Cofidece itervals arameter Estimatio arametric vs. o-parametric iferece ( Liear modelig Simple liear regressio, Multiple liear regressio OV & GLM The Big Three: Ceter,, Spread ad Shape! Ceter: Cetral tedecy, the middle, as a sigle umber value of parameter value of parameter value of parameter value of parameter (a Uimodal (b Bimodal (c Trimodal Mode: The most frequet score i the distributio. Media: The cetermost score if there are a odd umber of scores or the average of the two cetermost scores if there are a eve umber of scores. Mea: The sum of all (umeric observatios divided by the umber of scores (arithmetic average. (d Symmetric (e ositively sewed (log upper tail (f egatively sewed (log lower tail (g Symmetric (h Bimodal with gap (i Expoetial shape 4

5 Variability ot oly iterested i a distributio s middle. lso iterested i its spread (deviatio or variability. Fudametal characteristics of distributios (as models: - Cetral tedecy - Variability - Shape How ca we describe variability with a sigle umber? Shape: Sewess & Kurtosis What do we mea by symmetry ad positive ad egative sewess? Kurtosis? roperties?!? Sewess ( SD 3 ( Y Y ( Y Y 3 ; Kurtosis ( SD Sewess is liearly ivariat S(aX+b S(X Sewess is a measure of usymmetry Kurtosis is (also liearly ivariat a measure of flatess Both are used to quatify departures from Stdormal Sewess(Stdorm0; Kurtosis(Stdorm3 4 4 Distributios robability Theory xioms Basic riciples for probability modelig ad computatio Law of Total robability & Bayesia Theorem Distributios ( Experimets & Demos ( Statistical Iferece Hypothesis Testig & Cofidece itervals arameter Estimatio arametric vs. o-parametric iferece ( Liear modelig Simple liear regressio, Multiple liear regressio OV & GLM edu.ucla.stat.socr.distributios.beroullidistributio edu.ucla.stat.socr.distributios.biomialdistributio edu.ucla.stat.socr.distributios.birthdaydistributio edu.ucla.stat.socr.distributios.diedistributio edu.ucla.stat.socr.distributios.discretercsiedistributio edu.ucla.stat.socr.distributios.discreteuiformdistributio edu.ucla.stat.socr.distributios.geometricdistributio edu.ucla.stat.socr.distributios.hypergeometricdistributio edu.ucla.stat.socr.distributios.egativebiomialdistributio edu.ucla.stat.socr.distributios.oitmassdistributio edu.ucla.stat.socr.distributios.oissodistributio edu.ucla.stat.socr.distributios.oerdicedistributio edu.ucla.stat.socr.distributios.walmaxdistributio edu.ucla.stat.socr.distributios.walositiodistributio Discrete Distributio Models What, whe ad how to use? Examples? Example: Hypergeometric Distributio The three assumptios that lead to a hypergeometric distributio:. The populatio or set to be sampled cosists of idividuals, objects, or elemets (a fiite populatio.. Each idividual ca be characterized as a success (S or failure (F, ad there are M successes i the populatio. 3. sample of idividuals is selected without replacemet i such a way that each subset of size is equally liely to be chose. Hypergeometric Distributio If X is the umber of S s i a completely radom sample of size draw w/o replacemet from a populatio cosistig of M S s ad ( M F s, the the probability distributio of X, called the hypergeometric distributio, is give by M M max(0, + M x mi(, M x x X ( x hxm ( ;,, 5

Hypergeometric Mea ad Variace M M M EX ( VX ( Ball_ad_Ur_Experimet HyperGeometric Distributio & Biomial pproximatio to HyperGeometric http://www.socr.ucla.

html Computatio of a Biomial pmf x b( x;, p p x ( 0 x x p Hypergeometric Distributio & Biomial Biomial approximatio to Hypergeometric M is small (usually <

at most 7 favor the ew tax? http://socr.stat.ucla.edu/pplets.dir/ormal_t_chi_f_tables.

distributio is also sometimes referred to as the distributio of rare evets.

6 Hypergeometric Mea ad Variace M M M EX ( VX ( Ball_ad_Ur_Experimet HyperGeometric Distributio & Biomial pproximatio to HyperGeometric Computatio of a Biomial pmf x b( x;, p p x ( 0 x x p Hypergeometric Distributio & Biomial Biomial approximatio to Hypergeometric M is small (usually < 0., the p approaches HyperGeom( x;,, M Bi( x;, p M / p Ex: 4,000 out of 0,000 residets are agaist a ew tax. 5 residets are selected at radom ad surveyed. at most 7 favor the ew tax? HyperGeometric ad Biomial Experimet/Distributios oisso Distributio Defiitio Used to model couts umber of arrivals (( o a give iterval The oisso distributio is also sometimes referred to as the distributio of rare evets. Examples of oisso distributed variables are umber of accidets per perso, umber of sweepstaes wo per perso, or the umber of catastrophic defects foud i a productio process. Fuctioal Brai Imagig - ositro Emissio Tomography (ET Fuctioal Brai Imagig ositro Emissio Tomography (ET Isotope Eergy (MeV Rage(mm /-life ppl. C mi receptors 5O.7.5 mi stroe/activatio 8 F mi eurology 4I ~ days ocology 6

7 oisso Distributio Mea Used to model couts umber of arrivals (( o a give iterval e Y~oisso(, the Y, 0,,,! Mea of Y, μ Y,, sice e E( Y e! e 0 e (! 0 0 e! e! e 0,,, (! oisso Distributio - Variace Y~oisso(, the Y e, 0,,,! Variace of Y, σ Y ½, sice σ e Y Var( Y (... 0! For example, suppose that Y deotes the umber of bloced shots (arrivals i a radomly sampled game for the UCL Bruis me's basetball team. The a oisso distributio with h mea4 may be used to model Y. oisso as a approximatio to Biomial Suppose we have a sequece of Biomial(, p models, with lim( p, as ifiity. For each 0<y<, if Y ~ Biomial(, p, the Y y But this coverges to: p y y ( p y y y p p y ( WHY? y e y! p Thus, Biomial(, p oisso( oisso as a approximatio to Biomial Rule of thumb is that approximatio is good if: >00 p<0.0 p <0 The, Biomial(, p oisso( Validate usig: ml Biomial, HyperGeometric ad oisso Experimets Example usig oisso approx to Biomial Suppose Disease Fid the probability that a village of 5,000 people has > people havig the disease! Y~ Biomial(5,000, 0.000, fid Y>. ote that Z~oisso( p 5,000 x Z > Z 0.5 e 0! e!.5 z e! z.5 e z! Why bother discussig distributios? rovide a rich source of (aalytical Models. Geeral properties of processes may be studies without regard to the uderlyig molecular, physiological, geotypic or pheotypic properties or characteristics of the pheomeo. Easy to fit models to data ad mae iferece usig the model istead of limited data. Low computatioal costs (efficiecy( efficiecy What else? Example: 7

Cotiuous Distributio Models http://www.socr.ucla.edu/htmls/socr_distributios.html Beta Distributio, edu.ucla.stat.socr.distributios.betadistributio Beta (Geeralized Distributio, edu.ucla.stat.socr.distributios.betageeraldistributio CauchyDistributio, edu.

ucla.stat.socr.distributios.cotiuousuiformdistributio CotiuousUiformDistributio Expoetial Distributio, edu.ucla.stat.socr.distributios.expoetialdistributio Fisher's F Distributio, edu.ucla.stat.socr.distributios.fisherdistributio Gamma Distributio, edu.

8 Cotiuous Distributio Models Beta Distributio, edu.ucla.stat.socr.distributios.betadistributio Beta (Geeralized Distributio, edu.ucla.stat.socr.distributios.betageeraldistributio CauchyDistributio, edu.ucla.stat.socr.distributios.cauchydistributio CauchyDistributio Chi-Square Distributio, edu.ucla.stat.socr.distributios.chisquaredistributio ChiSquareDistributio Circle Distributio, edu.ucla.stat.socr.distributios.circledistributio CircleDistributio Cotiuous Uiform Distributio, edu.ucla.stat.socr.distributios.cotiuousuiformdistributio CotiuousUiformDistributio Expoetial Distributio, edu.ucla.stat.socr.distributios.expoetialdistributio Fisher's F Distributio, edu.ucla.stat.socr.distributios.fisherdistributio Gamma Distributio, edu.ucla.stat.socr.distributios.gammadistributio GeeralCauchyDistributio, edu.ucla.stat.socr.distributios.geeralcauchydistributio Gilbrats Distributio, edu.ucla.stat.socr.distributios.gilbratsdistributio GumbelDistributio, edu.ucla.stat.socr.distributios.gumbeldistributio Half-ormal Distributio, edu.ucla.stat.socr.distributios.halformaldistributio Laplace Distributio, edu.ucla.stat.socr.distributios.laplacedistributio Logistic Distributio, edu.ucla.stat.socr.distributios.logisticdistributio Log-ormal Distributio, edu.ucla.stat.socr.distributios.logormaldistributio Maxwell Distributio, edu.ucla.stat.socr.distributios.maxwelldistributio MixtureDistributio, edu.ucla.stat.socr.distributios.mixturedistributio ormal Distributio, edu.ucla.stat.socr.distributios.ormaldistributio areto Distributio, edu.ucla.stat.socr.distributios.aretodistributio Rayleigh Distributio, edu.ucla.stat.socr.distributios.rayleighdistributio RayleighDistributio Studet's T Distributio, edu.ucla.stat.socr.distributios.studetdistributio StudetDistributio Triagle Distributio, edu.ucla.stat.socr.distributios.triagledistributio Weibull Distributio, edu.ucla.stat.socr.distributios.weibulldistributio What, whe ad how to use? Examples? Logormal (Y μ,σ Relatio amog Distributios ormal (X μ,σ X ly X Y e α U X β α Beta α, β α β Uiform(X α, β μ Z X σ χ Uiform(U 0, i X ( β α U + α Z i ormal (Z 0, Chi-square ( χ α /, β Gamma α, β α X β lu df Weibull γ, β γ Expoetial(X β T df (0, df Cauchy (0, The ormal Distributio ormal desity curve ca be summarized with the followig formula: y μ σ f ( y e σ π Every ormal curve uses this formula, what maes them differet is what gets plugged i for μ ad σ Each ormal curve is cetered at μ ad the width depeds o σ ( small tall, large short/wide. d-dimesioal dimesioal Gaussia distributios with mea vector μ ad covariace matrix Σ : T ( x Exp ( x μ Σ ( x μ p (π d / Σ / The ormal Distributio Each ormal curve is characterized by it's μ ad σ Y μ 3σ μ σ μ σ μ μ + σ μ + σ μ + 3σ μ μ μ + σ μ σ μ 3σ σ μ + σ μ + 3σ If radom variable Y is ormal with mea μ ad stadard deviatio σ,, we write Y ~ ( μ, σ Y Defiitio of the expected value,, i geeral The expected value: E(X x all x x x x dx all X Sum of (value times probability of value Example I the at least oe of each or at most 3 childre example, where X {umber of Girls} we have: X 0 3 pr(x E( X x x x

9 The expected value ad populatio mea μ X E(X is called the mea of the distributio of X. μ X E(X is usually called the populatio mea. μx is the poit where the bar graph of X x balaces. opulatio stadard deviatio The populatio stadard deviatio is sd(x E[(X - μ ] ote that if X is a RV, the (X-μ is also a RV, ad so is (X-μ. Hece, the expectatio, E[(X-μ ], maes sese. opulatio mea & stadard deviatio Expected value: E ( X x X x Variace Stadard Deviatio x ( x E( x X Var ( X x x ( x E( x X SD ( X Var( X x x For ay radom variable X E(aX +b a E(X +b+ ad SD(aX +b a SD(X Chebyshev s Theorem pplies to all distributios where σ, μ < afuty Chebyshev (Пафнутий Чебышёв (8( K Chebyshov, Tchebycheff or Tschebyscheff. μ σ < X< μ+ σ for > Chebyshev s Theorem Gives a lower boud for the probability that a value of a radom variable, with fiite variace,, lies withi a certai distace from the variable's mea; equivaletly, the theorem provides a upper boud for the probability that values lie outside the same distace from the mea. The theorem applies eve to o "bell-shaped" distributios ad puts bouds o how much of the data is or is ot "i the middle". Let X be a radom variable with mea μ ad fiite variace σ. ow, for ay real umber > 0, X μ X μ < σ σ Oly the cases > provide useful iformatio. Why? 9

10 Marov s s & Chebyshev s Iequalities Marov's iequality: (Marov was a studet of (Marov was a studet of Chebyshev If Y 0 & d > 0 E( Y Y d d Sice, if d, if Y d X, 0, otherwise ote Y 0, X 0 The : E( Y E( X d Y { d} Let Y X E(X ad d with > 0 E ( ( ( X E(X Y d X E(X Var(X σ ( X - E(X ( X - E(X σ Let /σ σ Chebyshev s Theorem pplies to all distributios, where mea umber exists (σ,( ofμ< Stadard Distace from Deviatios the Mea μ± σ μ± 3σ Miimum roportio of Values Fallig Withi Distace K -/ 0.75 K 3 -/ K 4 μ± 4σ -/ Coefficiet of Variatio Ratio of the stadard deviatio to the mea, expressed as a percetage Measuremet of relative dispersio Coefficiet of Variatio a example μ σ C. V. ( 00 CV.. ( 00 CV.. ( 00 μ 46. ( ( σ σ μ μ 84 0 σ σ μ Outlie robability Theory xioms Basic riciples for probability modelig ad computatio Law of Total robability & Bayesia Theorem Data Summaries ad ED Distributios ( Experimets & Demos ( Statistical Iferece arameter Estimatio Hypothesis Testig & Cofidece itervals arametric vs. o-parametric iferece ( CLT Liear modelig Simple liear regressio, Multiple liear regressio OV & GLM arameters, Estimators, Estimates parameter is a characteristic of process, populatio or distributio E.g., mea, st quartile, SD, mi, max, rage, sewess,, 97 th percetile, etc. estimator is a abstract rule for calculatig a quatity (or parameter from sample data. estimate is the value obtaied whe real data are plugged-i the estimator rule. 0

Discrete Random Variables and Probability Distributions. Random Variables. Discrete Models

Discrete Random Variables and Probability Distributions. Random Variables. Discrete Models UCLA STAT 35 Applied Computatioal ad Iteractive Probability Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Chris Barr Uiversity of Califoria, Los Ageles, Witer 006 http://www.stat.ucla.edu/~diov/