Lecture 3: Probablty Dstrbutons Random Varables Let us begn by defnng a sample space as a set of outcomes from an experment. We denote ths by S. A random varable s a functon whch maps outcomes nto the real lne. It s gven by x: S R. It assgns to each element n a sample space a real number value. Each element n the sample space has an assocated probablty and such probabltes sum to one. Probablty Dstrbutons. Let A R and let Prob(x A) denote the probablty that x wll belong to A. 2. Def. The dstrbuton functon of a random varable x s a functon defned by F(x') Prob(x x'), x' R. 3. Key Propertes of a Dstrbuton Functon P. F s nondecreasng n x. P.2 lm F(x) = and lm F(x) = 0. x x P.3 For all x', Prob(x > x') = - F(x'). P.4 For all x' and x'' such that x'' > x', Prob(x' < x x'') = F(x'') - F(x'). P.5 For all x', Prob(x < x') = lm F(x). ' x x P.6 For all x', Prob(x=x') = lm F(x) - lm F(x). x x' + x x' Dscrete Random Varables. If the random varable can assume only a fnte number or a countable nfnte set of values, t s sad to be a dscrete random varable. The set of values assumed by the random varable has a one-to-one correspondence to a subset of the postve ntegers. In contrast, a contnuous random varable has a set of possble values whch conssts of an nterval of the reals. 2. Wth a dscrete random varable, x, x takes on values as {x,...,x k } (fnte) or {x,x 2,...} (countable but nfnte)
3. There are three key propertes of dscrete random varables. P. Prob(x = x') f(x') 0. (f s called the probablty mass functon or the probablty functon.) P.2 f( x ) = Pr ob( x= x ) =. = = P.3 Prob(x A) = f( x ). x A 3. Graphcally, the dstrbuton functon of a dscrete random varable s a step-dot graph wth jumps between ponts equal to f(x ). Example: # Consder the random varable assocated wth 2 tosses of a far con. The possble values for the #heads x are {0,, 2}. We have that f(0) = /4, f() = /2, and f(2) = /4. f(x) F(x) X /2 X 3/4 X /4 X X /4 X 0 2 0 2 #2 A sngle toss of a far de. f(x) = /6 f x =,2,3,4,5,6. F(x ) = x /6. Dscrete Jont Dstrbutons. Let the two random varables x and y have a jont probablty functon f(x ',y ') = Prob(x = x ' and y = y '). The set of values taken on by (x,y) s X Y = { (x,y ) : x X and y Y}. 2
The probablty functon satsfes P. f(x, y ) 0. P.2 Prob((x,y ) A) = f( x, y ). P.3 f( x, y ) =. ( x, y) ( x, y) A The dstrbuton functon s gven by F(x ', y ' ' ) = Prob( x x and y y ' ) = f( x, y ), where ' L = {(x, y ) : x x and y y ' }. ( x, y) L 2. Next we wsh to consder the margnal probablty functon and the margnal dstrbuton functon. a. The margnal probablty functon assocated wth x s gven by f (x j ) Prob(x = x j ) = fx (, y). Lkewse the margnal probablty functon assocated wth y s gven by f 2 (y j ) y j Prob(y = y j ) = f( x, y ). x j b. The margnal dstrbuton functon of x s gven by F (x j ) = Prob(x x j ) = lm Prob(x x j y j and y y j ) = lm F(x j,y j ). Lkewse for y, the margnal dstrbuton functon s F 2 (y j ) = y j lm x j F(x j,y j ). 3. An example. Let x and y represent random varables representng whether or not two dfferent stocks wll ncrease or decrease n prce. Each of x and y can take on the values 0 or, where a means that ts prce has ncreased and a 0 means that t has decreased. The probablty functon s descrbed by f(,) =.50 f(0,) =.35 f(,0) =.0 f(0,0) =.05. Answer each of the followng questons. 3
a. Fnd F(,0) and F(0,). F(,0) =. +.05 =.5. F(0,) =.35 +.05 =.40. b. Fnd F (0) = lm y F(0,y) = F(0,) =.4 c. Fnd F 2 () = lm F(x,) = F(,) =. x d. Fnd f (0) = f( 0, y) = f(0,) + f(0,0) =.4. y e. Fnd f () = f(, y) = f(,) +f(,0) =.6 y 4. Next, we consder condtonal dstrbutons. a. After a value of y has been observed, the probablty that a value of x wll be observed s gven by Prob(x = x y = y ) = b. The functon g (x y ) f ( x, y ) f ( y ). 2 Pr ob(x = x & y = y ) Pr ob(y = y ) = f ( x, y ) f ( y ). 2 s called the condtonal probablty functon of x, gven y. g 2 (y x ) s defned analogously. c. Propertes. () g (x y ) 0. () g (x y ) = x f(x,y ) / x (() and () hold for g 2 (y x )) f(x,y ) =. x () f(x,y ) = g (x y )f 2 (y ) = g 2 (y x )f (x ). 5. The condtonal dstrbuton functons are defned by the followng. G (x y ) = f( x, y)/ f2 ( y), x x G 2 (y x ) = f( x, y )/ f( x). y y 6. The stock prce example revsted. 4
a. Compute g ( 0) = f(,0)/f 2 (0). We have that f 2 (0) = f(0,0) + f(,0) =.05 +. =.5. Further f(,0) =.. Thus, g ( 0) =./.5 =.66. b. Fnd g 2 (0 0) = f(0,0)/f (0) =.05/.4 =.25. Here f (0) = f( 0, y ) = f(0,0) + f(0,) =.05 + y.35 =.4. Independent Random Varables. Def. The random varables (x,...,x n ) are sad to be ndependent f for any n sets of real numbers A, we have Prob(x A & x 2 A 2 &...& x n A n ) = Prob(x A )Prob(x 2 A 2 ) Prob(x n A n ). 2. Gven ths defnton, let F and f represent the jont margnal denstes and the margnal dstrbuton functons for the random varables x and y. Let F and f represent the jont dstrbuton and densty functons. The random varables x and y are ndependent ff F(x,y) = F (x)f 2 (y) or f(x,y) = f (x)f 2 (y). Further, f x and y are ndependent, then g (x y) = f(x,y)/f 2 (y) = f (x)f 2 (y)/ f 2 (y) = f (x). Summary Measures of Probablty Dstrbutons. Summary measures are scalars that convey some aspect of the dstrbuton. Because each s a scalar, all of the nformaton about the dstrbuton cannot be captured. In some cases t s of nterest to know multple summary measures of the same dstrbuton. 2. There are two key types of measures. a. Measures of central tendency: Expectaton, medan and mode b. measures of dsperson: Varance 5
Expectaton. The expectaton of a random varable x s gven by E(x) = Σ x f(x ) 2. Examples #. A lottery. A church holds a lottery by sellng 000 tckets at a dollar each. One wnner wns $750. You buy one tcket. What s your expected return? E(x) =.00(749) +.999(-) =.749 -.999 = -.25. The nterpretaton s that f you were to repeat ths game nfntely your long run return would be -.25. #2. You purchase 00 shares of a stock and sell them one year later. The net gan s x. The dstrbuton s gven by. (-500,.03), (-250,.07), (0,.), (250,.25),(500,.35), (750,.5), and (000,.05). E(x) = $367.50 3. E(x) s also called the mean of x. A common notaton for E(x) s µ. 4. Propertes of E(x) P. Let g(x) be a functon of x. Then E(g(x)) s gven by E(g(x)) = Σ g(x ) f(x ). In what follows, tems wth an astersk may be skpped. *P.2 If k s a constant, then E(k) = k. *P.3 Let a and b be two arbtrary constants. Then E(ax + b) = ae(x) + b. *P.4 Let x,...,x n be n random varables. Then E(Σx ) = ΣE(x ). 6
*P.5 If there exsts a constant k such that Prob(x k) =, then E(x) k. If there exsts a constant k such that Prob(x k) =, then E(x) k. *P.6 Let x,...,x n be n ndependent random varables. Then E( n x = n ) = Ex ( ). = Medan. Def. If Prob(x m).5 and Prob(x m).5, then m s called a medan. m need not be unque. Example: (x,f(x )) gven by (6,.), (8,.4), (0,.3), (5,.), (25,.05), (50,.05). In ths case, m = 8 or 0. One conventon s to average these two and take the result as the medan. Mode. Def. The mode s gven by m o = argmax f(x ). 2. A mode s a maxmzer of the densty functon. Obvously, t need not be unque. *Other Descrptve Termnology for Central Tendency. Symmetry. The dstrbuton can be dvded nto two mrror mage halves. 2. Skewed. Rght skewed means that the bulk of the probablty falls on the lower values of x. Left skewed means that the bulk of probablty falls on the hgher values. A Summary Measure of Dsperson: The Varance. In many cases the mean the mode or the medan are not nformatve. 7
2. In partcular, two dstrbutons wth the same mean can be very dfferent dstrbutons. One would lke to know how common or typcal s the mean. The varance measures ths noton by takng the expectaton of the squared devaton about the mean. Def. For a random varable x, the varance s gven by E[(x-µ) 2 ]. Remark: The varance s also wrtten as Var(x) or as σ 2. The square root of the varance s called the standard devaton of the dstrbuton. It s wrtten as σ. 3. Computaton of the varance. For the dscrete case, Var(x) = Σ (x -µ) 2 f(x ). As an example, f (x, f(x )) are gven by (0,.), (500,.8), and (000,.). We have that E(x) = 500. Var(x) = (0-500) 2 (.) + (500-500) 2 (.8) + (000-500) 2 (.) = 50,000. 4. Propertes of the Varance. *P. Var(x) = 0 ff there exsts a c such that Prob(x = c) =. *P.2 For any constants a and b, Var(ax +b) = a 2 Var(x). P.3 Var(x) = E(x 2 ) - [E(x)] 2. *P.4 If x, =,...,n, are ndependent, then Var(Σx ) = Σ Var(x ). *P.5 If x are ndependent, =,...,n, then Var(Σa x ) = Σ a 2 Var(x ). 5. The Standard Devaton and Standardzed Random Varables 8
a. Usng the standard devaton, we may transform any random varable x nto a random varable wth zero mean and untary varance. Such a varable s called the standardzed random varable assocated wth x. b. Gven x we would defne ts standardzed form as z = x µ σ. z tells us how many standard devatons x s from ts mean (.e., σz = (x-µ)). c. Propertes of z. P. E(z) = 0. P.2 Var (z) =. Proof: P.2. Var(z) = E(z-0) 2 = E(z 2 ) = E[(x-µ) 2 /σ 2 ] = (/σ 2 )E(x-µ) 2 = σ 2 /σ 2 =. *4. A remark on moments a. Var (x) s sometmes called the second moment about the mean, wth E(x-µ) = 0 beng the frst moment about the mean. b. Usng ths termnology, E(x-µ) 3 s the thrd moment about the mean. It can gve us nformaton about the skewedness of the dstrbuton. E(x-µ) 4 s the fourth moment about the mean and t can yeld nformaton about the modes of the dstrbuton or the peaks (kurtoss). *Moments of Condtonal and Jont Dstrbutons. Gven a jont probablty densty functon f(x,..., x n ), the expectaton of a functon of the n varables say g(x,..., x n ) s defned as 9
E(g(x,..., x n )) = Σ g(x ) f(x ). 2. Uncondtonal expectaton of a jont dstrbuton. Gven a jont densty f(x,y), E(x) s gven by E(x) = = x f(x ) = = = x f (x, y ). Lkewse, E(y) s E(y) = = y f 2 (y ) = = = x f (x, y ). 3. Condtonal Expectaton. defned by The condtonal expectaton of x gven that x and y are jontly dstrbuted as f(x,y) s E(x y) = = x g (x y j ). Further the condtonal expectaton of y gven x s defned analogously as E(y x) = = y g 2 (y x j ). Correlaton and Covarance. Covarance. a. Covarance s a moment reflectng drecton of movement of two varables. It s defned as Cov(x,y) = E[(x-µ x )(y-µ y )]. 0
When ths s large and postve, then x and y tend to be both much above or both much below ther respectve means at the same tme. Conversely when t s negatve. b. Computaton of the covarance. Frst compute (x-µ x )(y-µ y ) = xy - µ x y - µ y x +µ x µ y. Takng E, E(xy) - µ x µ y - µ x µ y + µ x µ y = E(xy) - µ x µ y. Thus, Cov(xy) = E(xy) - E(x)E(y). If x and y are ndependent, then E(xy) = E(x)E(y) and Cov(xy) = 0. 2. Correlaton Coeffcent. a. The covarance s a good measure when dealng wth just two varables. However, t has the flaw that ts sze depends on the unts n whch the varables x and y are measured. Ths s a problem when one wants to compare the relatve strengths of two covarance estmates, say between x and y and z and w. b. The correlaton coeffcent cures ths problem by standardzng the unts of the covarance. The correlaton coeffcent s defned by ρ = Cov(x,y)/σ x σ y. c. Generally, ρ [-,]. If y and x are perfectly lnearly related then ρ. The less lnearly related are x and y, the closer s ρ to zero.