UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig 4 http://www.stat.ucla.edu/~diov/ Chapter 5 Joit Probability Distributios ad Radom Samples Slide Slide 5. Joitly Distributed Radom Variables Slide Joit Probability Mass Fuctio Let ad Y be two discrete rv s defied o the sample space of a eperimet. The joit probability mass fuctio p(, is defied for each pair of umbers (, by p( y, ) = P ( = ad Y= Let A be the set cosistig of pairs of (, values, the P (, Y) A = p(, ( y, ) A Slide 4 Margial Probability Mass Fuctios The margial probability mass fuctios of ad Y, deoted p () ad p Y ( are give by p ( ) = p(, p ( = p(, y Slide 5 Y Joit Probability Desity Fuctio Let ad Y be cotiuous rv s. The f (, is a joit probability desity fuctio for ad Y if for ay two-dimesioal set A ( ) P, Y A = f(, ddy If A is the two-dimesioal rectagle ( y, ): a bc, y d, { } ( ) P, Y A = f(, dyd A bd ac Slide 6
(, ) P Y A f ( y, ) A = shaded rectagle = Volume uder desity surface above A Margial Probability Desity Fuctios The margial probability desity fuctios of ad Y, deoted f () ad f Y (, are give by f ( ) = f(, dy for < < f ( = f(, d for < y< Y Slide 7 Slide 8 Idepedet Radom Variables Two radom variables ad Y are said to be idepedet if for every pair of ad y values p( y, ) = p( ) py( whe ad Y are discrete or f ( y, ) = f( ) fy( whe ad Y are cotiuous. If the coditios are ot satisfied for all (, the ad Y are depedet. Slide 9 More Tha Two Radom Variables If,,, are all discrete radom variables, the joit pmf of the variables is the fuctio p(,..., ) = P( =,..., = ) If the variables are cotiuous, the joit pdf is the fuctio f such that for ay itervals [a,b ],,[a,b ], Pa ( b,..., a b) b a b =... f(,..., ) d... d a Slide Idepedece More Tha Two Radom Variables The radom variables,,, are idepedet if for every subset,,..., of the variables, the joit pmf or pdf of the subset is equal to the product of the margial pmf s or pdf s. i i i Coditioal Probability Fuctio Let ad Y be two cotiuous rv s with joit pdf f (, ad margial pdf f (). The for ay value for which f () >, the coditioal probability desity fuctio of Y give that = is f(, fy ( y ) = y f ( ) < < If ad Y are discrete, replacig pdf s by pmf s gives the coditioal probability mass fuctio of Y whe =. Slide Slide
Margial probability distributios (Cot.) If ad Y are discrete radom variables with joit probability mass fuctio f Y (,, the the margial probability mass fuctio of ad Y are f ( ) = P( = ) = f Y (, Y ) f Y ( = P( Y = = f Y (, Y ) where R deotes the set of all poits i the rage of (, Y) for which = ad Ry deotes the set of all poits i the rage of (, Y) for which Y = y R Ry Mea ad Variace If the margial probability distributio of has the probability fuctio f(), the E ( ) = µ = = f ( ) f Y (, = f Y (, R R V ( = = R ) = σ R f (, Y = ( µ ) f ( µ ) (, = R = Set of all poits i the rage of (,Y). Eample 5-4. Y f ( ) = R ( µ ) ( µ ) f Y R (, f Y (, Slide Slide 4 Joit probability mass fuctio eample The joit desity, P{,Y}, of the umber of miutes waitig to catch the first fish,, ad the umber of miutes waitig to catch the secod fish, Y, is give below. P { = i,y = k } k Row Sum P{ = i }...8. i...8..7.8.6.78 Colum Sum P {Y =k }.9..79. The (joit) chace of waitig miutes to catch the first fish ad miutes to catch the secod fish is: The (margial) chace of waitig miutes to catch the first fish is: The (margial) chace of waitig miutes to catch the first fish is (circle all that are correct): The chace of waitig at least two miutes to catch the first fish is (circle oe, oe or more): The chace of waitig at most two miutes to catch the first fish ad at most two miutes to catch the secod fish is (circle oe, oe or more): Slide 5 Coditioal probability Give discrete radom variables ad Y with joit probability mass fuctio f Y (,Y), the coditioal probability mass fuctio of Y give = is f Y (y ) = f Y ( = f Y (,/f () for f () > Slide 6 Coditioal probability (Cot.) Because a coditioal probability mass fuctio f Y ( is a probability mass fuctio for all y i R, the followig properties are satisfied: () f Y ( () f Y ( = R () P(Y=y =) = f Y ( Slide 7 Slide 8
Coditioal probability (Cot.) Let R deote the set of all poits i the rage of (,Y) for which =. The coditioal mea of Y give =, deoted as E(Y ) or µ Y, is E ( Y ) = yf Y ( R Ad the coditioal variace of Y give =, deoted as V(Y ) or σ Y is V ( Y ) = ( y µ ) f ( = y f ( y µ Y Y Y ) R R Y Idepedece For discrete radom variables ad Y, if ay oe of the followig properties is true, the others are also true, ad ad Y are idepedet. () f Y (, = f () f Y ( for all ad y () f Y ( = f Y ( for all ad y with f () > () f y ( = f () for all ad y with f Y ( > (4) P( A, Y B) = P( A)P(Y B) for ay sets A ad B i the rage of ad Y respectively. Slide 9 Slide Epected Value 5. Epected Values, Covariace, ad Correlatio Slide Let ad Y be joitly distributed rv s with pmf p(, or pdf f (, accordig to whether the variables are discrete or cotiuous. The the epected value of a fuctio h(, Y), deoted E[h(, Y)] or µ h (, Y ) is hy (, ) py (, ) discrete y = h(, f (, ddy cotiuous Slide Covariace The covariace betwee two rv s ad Y is ( Y) = E ( µ )( Y µ ) Cov, Y ( µ )( y µ Y) p(, discrete y = ( µ )( y µ Y) f(, ddy cotiuous Short-cut Formula for Covariace ( ) ( ) Cov Y, = E Y µ µ Y Slide Slide 4 4
Correlatio The correlatio coefficiet of ad Y, deoted by Corr(, Y), ρy,, or just ρ, is defied by Cov (, Y ) ρ Y, = σ σ Y Correlatio Propositio. If a ad c are either both positive or both egative, Corr(a + b, cy + d) = Corr(, Y). For ay two rv s ad Y, Corr( Y, ). Slide 5 Slide 6 Correlatio Propositio. If ad Y are idepedet, the ρ =, but ρ = does ot imply idepedece.. ρ = or iff Y = a + b for some umbers a ad b with a. 5. Statistics ad their Distributios Slide 7 Slide 8 Statistic A statistic is ay quatity whose value ca be calculated from sample data. Prior to obtaiig data, there is ucertaity as to what value of ay particular statistic will result. A statistic is a radom variable deoted by a uppercase letter; a lowercase letter is used to represet the calculated or observed value of the statistic. Radom Samples The rv s,, are said to form a (simple radom sample of size if. The i s are idepedet rv s.. Every i has the same probability distributio. Slide 9 Slide 5
Simulatio Eperimets The followig characteristics must be specified:. The statistic of iterest.. The populatio distributio.. The sample size. 4. The umber of replicatios k. 5.4 The Distributio of the Sample Mea Slide Slide Usig the Sample Mea Let,, be a radom sample from a distributio with mea value µ ad stadard deviatio σ. The ( ) ( ). E. V = µ = µ = σ = σ Normal Populatio Distributio Let,, be a radom sample from a ormal distributio with mea value µ ad stadard deviatio σ. The for ay, is ormally distributed, as is T o. I additio, with T o = + +, E T = µ, V T = σ,ad σ = σ. ( ) ( ) o o T o Slide Slide 4 The Cetral Limit Theorem Let,, be a radom sample from a distributio with mea value µ ad variace σ. The if sufficietly large, has approimately a ormal distributio with µ = µ ad σ = σ, ad T o also has approimately a ormal distributio with µ T = µ, σ. o T = σ The larger the value of o, the better the approimatio. Populatio distributio The Cetral Limit Theorem small to moderate µ large Slide 5 Slide 6 6
Rule of Thumb If >, the Cetral Limit Theorem ca be used. Approimate Logormal Distributio Let,, be a radom sample from a distributio for which oly positive values are possible [P(i > ) = ]. The if is sufficietly large, the product Y = has approimately a logormal distributio. Slide 7 Slide 8 5.5 The Distributio of a Liear Combiatio Liear Combiatio Give a collectio of radom variables,, ad umerical costats a,,a, the rv Y = a+... + a = aii i= is called a liear combiatio of the i s. Slide 9 Slide 4 Epected Value of a Liear Combiatio Let,, have mea values µ, µ,..., µ ad variaces of σ, σ,..., σ, respectively Whether or ot the i s are idepedet, ( +... + ) = ( ) +... + ( ) E a a ae ae = a µ + + a µ... Variace of a Liear Combiatio If,, are idepedet, ( +... + ) = ( ) +... + ( ) V a a a V a V ad σ... aσ = a + + a+... + a = a +... + a σ σ σ Slide 4 Slide 4 7
Variace of a Liear Combiatio For ay,,, V a... a aa Cov, ( + + ) = i j ( i j) i= j= Differece Betwee Two Radom Variables ( ) = ( ) ( ) E E E ad, if ad are idepedet, ( ) = ( ) + ( ) V V V Slide 4 Slide 44 Differece Betwee Normal Radom Variables If,, are idepedet, ormally distributed rv s, the ay liear combiatio of the i s also has a ormal distributio. The differece betwee two idepedet, ormally distributed variables is itself ormally distributed. Cetral Limit Theorem heuristic formulatio Cetral Limit Theorem: Whe samplig from almost ay distributio, is approimately Normally distributed i large samples. Show Samplig Distributio Simulatio Applet: file:///c:/ivo.dir/ucla_classes/witer/additioalistructoraids/ SampligDistributioApplet.html Slide 45 Slide 46 Idepedece For discrete radom variables ad Y, if ay oe of the followig properties is true, the others are also true, ad ad Y are idepedet. () f Y (, = f () f Y ( for all ad y () f Y ( = f Y ( for all ad y with f () > () f y ( = f () for all ad y with f Y ( > (4) P( A, Y B) = P( A)P(Y B) for ay sets A ad B i the rage of ad Y respectively. Slide 47 Slide 48 8
Recall we looked at the samplig distributio of Cetral Limit Effect Histograms of sample meas Triagular Distributio For the sample mea calculated from a radom sample, E( ) = µ ad SD( ) = σ, provided = ( + + + )/, ad k ~N(µ, σ). The σ ~ N(µ, ). Ad variability from sample to sample i the sample-meas is give by the variability of the idividual observatios divided by the square root of the sample-size. I a way, averagig decreases variability. Slide 49 = Y= Area =...4.6.8. =...4.6.8. Sample meas from sample size =, =, 5 samples...4.6.8. Slide 5 Cetral Limit Effect -- Histograms of sample meas = 4...4.6.8. Triagular Distributio Sample sizes =4, = Slide 5 =...4.6.8. Cetral Limit Effect Histograms of sample meas =...4.6.8. Sample meas from sample size =, =, 5 samples...4.6.8. Uiform Distributio = Y = Area =...4.6.8. Slide 5 Stat A, UCLA, Diov Ivo Cetral Limit Effect -- Histograms of sample meas = 4...4.6.8. Uiform Distributio Sample sizes =4, = 4 Slide 5 =...4.6.8...8.6.4. Cetral Limit Effect Histograms of sample meas =. 4 5 6 Sample meas from sample size =, =, 5 samples Slide 54..8.6.4.. 4 5 6.8.6.4. Epoetial Distributio e, [, ) =. 4 Area = 9
..8.6.4. Cetral Limit Effect -- Histograms of sample meas = 4 Epoetial Distributio Sample sizes =4, =...8.4 Slide 55 =. Cetral Limit Effect Histograms of sample meas =...4.6.8. Sample meas from sample size =, =, 5 samples Slide 56 Quadratic U Distributio...4.6.8. = ( ), [, ] Y = Area =...4.6.8. Cetral Limit Effect -- Histograms of sample meas = 4 Quadratic U Distributio Sample sizes =4, =...4.6.8. =...4.6.8. Cetral Limit Theorem heuristic formulatio Cetral Limit Theorem: Whe samplig from almost ay distributio, is approimately Normally distributed i large samples. Show Samplig Distributio Simulatio Applet: file:///c:/ivo.dir/ucla_classes/witer/additioalistructoraids/ SampligDistributioApplet.html Slide 57 Slide 58 Cetral Limit Theorem theoretical formulatio { } Let,,...,,... be a sequece of idepedet k observatios from oe specific radom process. Let ad E( ) = µ ad SD( ) = σ ad both be fiite ( < σ < ; µ < ). If =, sample-avg, k = k The has a distributio which approaches N(µ, σ /), as. Slide 59