Chapter 1 Probability Theory

Transcription

1 Review for the previous lecture Eample: how to calculate probabilities of events (especially for sampling with replacement) and the conditional probability Definition: conditional probability, statistically independent for two or more events Theorem: Bayes Rule Chapter Probability Theory 4 Random Variables Eample: If we decide to ask 50 people whether they agree or disagree with a certain issue If we record for 50 the agree and 0 for the disagree Then we have a sample space with the size of 2, each an ordered string of s and 0s of length 50 Actually, we may be only interested in the number of people who agree (or disagree) out of 50 In this situation, we can define =number of s recorded out of 50 Note now that the sample space is the set of integers {0,,2,,50}, which is easier to be dealt with Definition 4: A random variable is a function from a sample space S into the real numbers Definition and Theorem: Suppose we have a sample space S = { s, s2,, s n } with a probability function P and we define random variable with range = {,, m }, then we can define a function P on in the following way: P( = i) = P({ sj S: ( sj) = i}) Then P is a probability function defined on the sample space We

2 call it as the induced probability function on We can simply write P ( = i ) as P ( = i ) This definition can be etended to countable (infinite) sample spaces (but not to uncountable sample spaces) To generalize this definition, for any set A B where B is an appropriate sigma algebra defined on (in the general case, B is derived from the original sigma algebra defined on the sample space S ), we can define: P ( A) = P({ s S: ( s) A}) Notation: A random variable will be denoted by uppercase letter, say, and the realized values (or its range) will be denoted by the corresponding lower case letters, Eample: Roll two fair dice, =sum of face numbers Solution: s (,) (,2) (2,) (,3) (2,2) (3,3) (,4) (2,3) (3,2) (4,) (,5) (2,4) (3,3) (4,2) (5,) (,6) (2,5) (3,4) (4,3) (5,2) (6,) (2,6) (3,5) (4,4) (5,3) (6,2) (3,6) (4,5) (5,4) (6,3) (4,6) (5,5) (6,4) (5,6) (6,5) () s (6,6) Ps ( ( )) /36 2/36 3/36 4/36 5/36 6/36 5/36 4/36 3/36 2/36 /36 Eample 43: Toss a fair coin 3 times Define a random variable and obtain its corresponding distribution Solution: =number of heads; 2

3 s HHH HHT,HTH,THH TTH, THT, HTT TTT () s Ps ( ( )) /8 3/8 3/8 /8 Eample 44 (Distribution of a random variable): If we toss a fair coin n times, and define =number of heads, then what is its corresponding distribution? Solution: n P ( = k) = P(there are k heads in n coins)= /2 n k ( k = 0,,, n) 5 Distribution Functions Definition 5: The cumulative distribution function or cdf of a random variables, denoted by F ( ), is defined by F ( ) = P ( ), for all Eample 52: Tossing three coins =number of heads Then we have: 0 if < < 0 /8 if 0 < F ( ) = 4/8 if < 2 7/8 if 2 < 3 if 3 < 3

4 Theorem 53: The function F( ) is a cdf if and only if the following three conditions hold: () lim F( ) = 0 and lim F( ) = (2) F( ) is nondecreasing function of (3) F( ) is right-continuous; that is, for every number 0, Proof of necessity: lim F ( ) = F ( ) 0 0 (2) < y, define A = { s S: ( s) } and B = { s S: ( s) y}, then A B, therefore: F( ) = P({ s S: ( s) }) = P( A) P( B) = P({ s S: ( s) y}) = F( y) () First, F( ) = P({ s S: ( s) }), then 0 F( ) Define A = { s S: n < ( s) n} ( n =,,,0,,, ), then we have S = An, PA ( ) = Fn ( ) Fn ( ), and A ( n=,, ) are disjoint Therefore, = PS ( ) = PA ( ) = ( Fn ( ) Fn n ( )) = lim Fn n ( ) lim Fm m ( ) n= n= decreasing, we have lim F( ) = lim F( n) and lim F( ) = lim F( m) n= n Because ( ) n n m n F is non (3) To prove F( ) is right-continuous, we only need to prove for any > > n > 0 and lim n n = 0, such that lim F( ) = F( ) It is easy to see n n 0 F( ) F( ) = P({ s: < ( s) }) = P( { s: < ( s) n}) ( ( ) ( )) ( ) lim ( ) 0 0 n= n+ = F n n F n+ = F n F = n+ Proof of Sufficiency: The proof of sufficiency is harder, we need to construct a sample space ( S ), a sigma algebra ( B ) on the sample space, a probability function P, and a random variable 4

5 We define the sample space: S = [0,], the sigma algebra B as the minimum sigma algebra containing [ ab, ],[ ab, ),( ab, ],( ab, ), where 0 a b, P as the Lebesgue measure For any function F( ) on the real line that stratifies (), (2), and (3) in the theorem 53, we can define the inverse function of F( ) : F () s = inf{ : F() s} (0 s ) Then Y = F () s is the random variable on S = [0,] To calculate the cdf of Y, we have Therefore, Y is a random variable with the cdf of F P( Y y) = P( F ( s) y) = P( s F( y)) = F( y) Eample 54: Tossing a coin for a head Let p = probability of a head on any given toss, and = number of tosses required to get a head, then ( = ) = ( ) ( =,2, ) P p p To obtain the cdf of, we recall the results on sums of geometric series For any positive integer, i ( p) F( ) = P( ) = ( p) p= p= ( p) i= ( p) ( =,2, ) It is easy to see that () lim F( ) = 0 and lim F( ) = ; (2) F( ) is nondecreasing; and (3) F( ) is rightcontinuous Therefore, F( ) is cdf Eample 55 (Continuous cdf): Show that the following function is a cdf F( ) = + e 5

6 Definition 57: A random variable is continuous if F( ) is a continuous function of A random variable is discrete if F( ) is a step function of Eample 56: A random variable is neither continuous nor discrete Consider the following function: F ε if y < 0 y + e ( y) = for some 0< ε < ε ε + if y 0 y + e Definition 58: The random variables and Y are identically distributed if, for every set A B, P ( A) = PY ( A), where B is the smallest sigma algebra containing all the intervals of real numbers of the form (a, b), [a, b), (a, b], and [a, b] Theorem 50: The following two statements are equivalent: The random variables and Y are identically distributed 2 F ( ) = F ( ) for every Y Eample 59 (Identically distributed random variables): If a fair coin is tossed n times, define the following random variables: =number of heads observed, and Y =number of tails observed It is easy to prove that and Y have the same distribution but they are different Actually we have ( s) + Y( s) = n 6

7 6 Density and Mass Functions Definition 6 The probability mass function (pmf) of a discrete random variable us given by f ( ) = P( = ) for all Note: The pmf gives the point probabilities of a discrete random variable Eample 62: From Eample 54, the geometric distribution has pmf given by ( p) p for =, 2, f ( ) = P( = ) = 0 otherwise It follows then that for a b, we have and in particular, if a =, then Pa ( b b ) f b k ( k ) ( p = = ) p k= a k= a b P ( b) = f ( k) = F( b) k= Eample: Twenty telephones have just been received at an authorized service center 4 of these telephones are corded, 0 are cordless and 6 are cellular Suppose we select phones one by one until we get a corded phone Obtain the pmf of = the number of phones selected until a corded phone is selected Solution: The pmf of is: 7

8 6 k 4 if k =,2,7 P ( = k) = ( k ) k 0 Otherwise When is a continuous random variable with cdf F ( ), how to get its pdf? Since { = } { ε < } for any ε > 0, we have from Theorem 29(c) that P ( = ) P ( ε < ) = F( ) F( ε ) for any ε > 0, then we have P ( = ) = 0 due to its continuity Discrete: F( ) = f( a) a Continuous: F( ) = f( t) dt By the Fundamental Theorem of Calculus, we have d F ( ) = f ( ) d Definition 63 The probability density function or pdf, f ( ), of a continuous random variable is the function that satisfies F( ) = f( t) dt for all 8

9 Notations: ~ F ( ) has distribution F ( ); ~ f ( ) has pmf (or pdf) f ( ) ; ~ Y have the same distribution (identically distributed); For continuous random variables, we have Pa ( < < b) = Pa ( < b) = Pa ( < b) = Pa ( b) Eample 64: Obtain the pdf of a logistic random variable with cdf Solution: F ( ) = + e d e f( ) = F( ) = d ( + e ) 2 Theorem 65: A function f ( ) is a pdf (or pmf) of a random variables if and only if a f ( ) 0 for all b f ( ) = (pmf) or f ( ) d= (pdf) Proof 9

10 Eample: Let denote the amount of space occupied by an article placed in a -ft 3 packing container The pdf of is: k 8 ( ) if 0 ; f ( ) = 0 otherwise Find the value of k that will make this a valid density function 2 Obtain the cdf of Solution: 2 k 8 d k k 0 ( ) = / 9 /0 =, thus k = 90 0 if < 0; 8 9 k k F ( ) = if 0 < ; 9 0 if < < Remarks: We emphasize that the distribution of is uniquely determined by its cdf or pdf (pmf) So when we ask you to find the distribution of, it is sufficient to find either cdf or pdf (pmf) 2 We note if the sample space S is discrete, then on S is discrete if the sample space S contains an interval of (, ), then on S can be discrete, continuous, and mied 3 For continuous random variable, cdf and pdf have the following relationships: d P ( ) = F( ) = f( tdt ) and f ( ) ( ) = F d 0

11 4 In practice no real world eperiments can yield continuous random variables The assumption that is continuous type is a simplifying assumption leading to an idealized description of the model 5 We emphasize that cdf and pdf (pmf) are defined for all real number However, for simplicity, we only concern P ( = ) > 0 for discreet random variables and { : f ( ) > 0} for continuous random variables