3.1 Random variables

Transcription

1 3 Chapte III Random Vaiables 3 Random vaiables A sample space S may be difficult to descibe if the elements of S ae not numbes discuss how we can use a ule by which an element s of S may be associated with a numbe We shall Definition 3-: Given a andom epeiment with a sample space S, a function that assigns to each element s in S one and only one eal numbe ( s) is called a andom vaiable The space of is the set of eal numbes ( s), s S the element s belongs to the set S :, whee s S means Eample 3-: In eample 3-, we had the sample space S { H, T} Let be a function defined on S such that ( H ) and ( T ) 0 Thus is a eal-function that has the sample space S as its domain and the space of eal numbes : 0, as its ange Note that it may be that the set S has elements that ae themselves eal numbes In such instance we could wite ( s) s so that is the identity function and the space of is also S Eample 3-: In eample 3-, the sample space is S {,, 3, 4, 5, 6} Fo each s S, let ( s) s The space of the andom vaiable is then {,, 3, 4, 5, 6} If we want to find the pobabilities associated with events descibed in tems of, we use the pobabilities of those events in the oiginal space S if they ae known P( a b ) P { s : s S and a ( s) b } Fo instance, The pobabilities ae induced on the points of the space of by the pobabilities assigned to outcomes of the sample space S though the function Hence, the pobability P( ) is often called an induced pobability Eample 3-3: In eample 3-, we associate a pobability of / fo each outcome, then, fo eample, P ( ) P( 0) / Eample 3-4: containing 0 balls numbeed though 0 Thee balls ae to be andomly selected without eplacement fom an un If we bet that at least one of the dawn balls has a numbe as lage as o lage than 7, what is the pobability that we win the bet? Solution: Let denote the lagest numbe selected Then is a andom vaiable taking on one 0 of the values 3, 4,, 0 Futhemoe, if we suppose that each of the possible selections is 3 equally likely to occu, then i 0 P i i 3, 4,, 0 3

2 3 The above equation follows because the numbe of the selections that esult in the event i is just the numbe of selections that esult in ball numbeed i and two of the balls numbeed i though i being chosen As thee ae clealy such selections, we obtain the pobabilities epessed in above equation Fom this equation we see that P P P P Hence, as the event 7 is the union of the disjoint (mutually eclusive) events i, i = 7, 8, 9, 0, it follows that the pobability of ou winning the bet is given by P Thee ae two majo difficulties hee: () In many pactical situations the pobabilities assigned to the events A of the sample space S ae unknown () Since thee ae many ways of defining a function on S, which function do we want to use? In consideing (), we need, though epeated obsevations (called sampling), to estimate these pobabilities o pecentages fequency afte a numbe of obsevations in this couse, othe ways of estimating pobabilities mathematical statistics is concened One obvious way of estimating these is by use of the elative If additional assumptions can be made, we will study, It is this latte aspect with which That is, if we assume cetain models, we find that the theoy of statistics can eplain how best to daw conclusions o make pedictions Fo (), statisticians ty to detemine what measuement (o measuements) should be taken on an outcome; that is, how best do we mathematize the outcome? These measuement poblems ae most difficult and can only be answeed by getting involved in a pactical poject Nevetheless, in many instances it is clea eactly what function the epeimente wants to define on the sample space of the spot, say, which is up on the die Definition 3-: Fo eample, the die game in Eample 3- is concened about the numbe A andom vaiable is said to be discete if it can assume only a finite o countably infinite numbe of distinct values A set of elements is countably infinite if the elements in the set can be put into one-to-one coespondence with the

3 33 Definition 3-3: The pobability that takes on the value, P( ), is the defined as the sum of the pobabilities of all sample points in S that ae assigned the value Fo a andom vaiable of the discete type, the induced pobability P( ) is fequently denoted by f (), and this function f () Note that some authos efe to f () the pobability mass function (pmf) Eample 3-5: is called the discete pobability density function (pdf) as the pobability function (pf), the fequency function, o We will use the teminology pmf in this couse A supeviso in a manufactuing plant has thee men and thee women woking fo him He wants to choose two wokes fo a special job Not wishing to show any biases in his selection, he decides to select the two wokes at andom Let denote the numbe of women in his selection Find the pobability distibution fo 6 Solution: The supeviso can select two wokes fom si in 5 ways Hence S contains 5 sample points, which we assume to be equally likely because andom sampling was employed Thus P( ) 5, fo i =,,, 5 The value fo that have nonzeo pobability ae 0,, E i and The numbe of ways of selecting = 0 women is Thus thee ae sample points in the event = 0, and Similaly, f ( 0) P( 0) f ( ) P( ) f ( ) P( ) Table o histogam can epesent the above esults, but the most concise method of epesenting discete pobability distibutions is by means of a fomula The fomula fo f() can be witten as 3 3 f ( ), = 0,, 6 Notice that the pobabilities associated with all distinct value of a discete andom vaiable must sum to one positive integes

4 34 Theoem 3-: A function f () popeties fo at most a countably infinite set of eal values 0 f ( ) fo all i f ( ) all i Poof: 0 f ( ) i i i is a pmf if and only if it satisfies both of the following,, : follows fom the fact that the value of a discete pmf is a pobability and must be nonnegative Since,, epesent all possible values of, the events [ ],[ ], constitute an ehaustive patition of the sample space Thus, Definition 3-4: f i ) P[ ] ( all i all i i The cumulative distibution function (CDF), o simply efeed to as the distibution function, of a discete andom vaiable is defined fo any eal by F ( ) P f ( Eample 3-6: The pobability mass function of a andom vaiable is given by f ( ) c!, 0,,, whee is some positive value Find P{ 0} and P { } 0 Solution: Since f ( ), we have that implying, because 0 P { 0} e 0! e P{ } P{ t c! 0 e, that ce o c e Hence, 0! 0} P{ } P{ } e e e The cumulative distibution function F() can be epessed in tems of f () F( a) f ( ), a = 0,,, all a by Definition 3-5: A andom vaiable is called a continuous andom vaiable if thee is a function f (), called the pobability density function (pdf) of, such that the CDF can be epesented as F( ) f ( dt The above defining popety povides a way to deive the CDF when the pdf is given, and it follows by the Fundamental Theoem of Calculus that the pdf can be obtained fom the CDF by diffeentiation Specifically,

5 35 wheneve the deivative eists d f ( ) F( ) F ( ) d Theoem 3-: A function f () only if it satisfies the popeties f ( ) 0 fo all eal is a pdf fo some continuous andom vaiable if and f ( ) Popeties of a CDF F () : (a) 0 F ( ) since F() is a pobability (b) F() is a non-deceasing function of Fo instance, if a b, then b : a : a b : and P( b) P( a) P( a b) That is, F ( b) F ( a) P( a b) 0 (c) Fom the poof of (b), it is obseved that if a b, then P( a b) F ( b) F ( a) (d) lim F( ) 0 and lim F( ) one-dimensional space and the set : (e) F() is continuous to the ight at each point because the set : is the null set (f) If is a andom vaiable of the discete type, then F() a step at,, equals the pobability P( ) is the entie is a step function, and the height of (g) If is a continuous andom vaiable, then F() is a continuous function The pobability P( a b) is the aea bounded by the gaph of f (), the -ais, and the lines a and b Futhemoe, the pobability at any paticula point is zeo Eample 3-7: function is given by Suppose that is a continuous andom vaiable whose pobability density c(4 ) 0 f ( ) 0 othewise (a) What is the value c? (b) Find P { }? Solution: Since f is a pobability density function, we must have that f ( ) d, implying that c (4 ) d Hence, 0 3 c 3 0 c 3 8

6 (4 ) P { } d The cumulative distibution function F is given by 3 3 8(4 t t ) dt F( ) f ( dt, 0 < < 0 Eample 3-8: The distibution function of the andom vaiable Y is given by F( y) 0 y / / 3 / y 0 0 y y y 3 y 3 A gaph of F(y) is pesented in Figue 3- / / 3 F (y) / 3 Figue 3- Gaph of F (y) y Compute P { Y 3}, P { Y }, P { Y 05}, and P { Y 4} Solution: P{ Y 3} lim P Y 3 lim F3 n n n n P{ Y } P{ Y } P{ } F() lim F n n 3 6 P { Y 05} P Y 05 F (05) 0 75 P { Y 4} F (4) F () Eample 3-9: An Unbounded Density Function Let the andom vaiable have the distibution function F() given by:

7 37 F( ) 0 if 0 if 0 if Then has a density function f () given by if 0 f ( ) 0 othewise Note that f () is unbounded fo nea zeo In fact, as appoaches zeo by positive value, f () tends towad infinity slowly enough so that the density function still integates to one An altenative eample is the density of the chi-squaed distibution with one degee of feedom Definition 3-6: If 0 < p <, a 00 pth pecentile of the distibution of a andom vaiable is the smallest value, p, such that F( p ) p In essence, p is the value such that 00 p% of the population values ae less than o equal to p We can also think in tems of a popotion p athe than a pecentage 00 p of the population, and p is often efeed to as a pth quantile If is continuous, then p is a solution to the equation F( p ) p Eample 3-0: Conside the distibution of lifetimes, (in months), of a paticula type of component We will assume that the CDF has the fom F( ) ep and zeo othewise The median lifetime is m 3 3, 0 05 ln 05 3 ln 498 months It is desied to find the time t such that 0% of the components fail befoe t This is the 0% pecentile: 05 ln 0 3 ln(09) 0974 months 00 3 Thus, if the components ae guaanteed fo one month, slightly moe than 0% will need to be eplaced 3 Mathematical Epectation One of the most impotant concepts in pobability theoy is that of the mathematical epectation of a andom vaiable

8 38 Definition 3-: Let be a andom vaiable having a pdf (o pmf) f (), and let u( ) a function of Then the mathematical epectation of ( ) be Eu ( ) u( ) f ( ) d if is a continuous type of andom vaiable, o u ( if is a discete type of andom vaiable E ) u( ) f ( ) Remaks: The usual definition of Eu( ) absolutely That is, u ( ) f ( ) d (o u ( ) f ( ) ) u, denoted by E u( ) be, is defined to equies that the integal (o sum) convege Theoem 3-: Let be a andom vaiable having a pmf (o pdf) f () Mathematical epectation E (), if it eists, satisfies the following popeties: (a) If c us a constant, E( c) c (b) If c is a constant and u is a function, Ecu ( ) ceu ( ) (c) If c and c ae constants and u and u ae functions, then E c u ) c u ( ) c E u ( ) c E u ( ) ( Poof: Fist, we have fo the poof of (a) that E ( c) cf ( ) cf ( ) c Net, to pove (b), we see that Ecu ( ) cu ( ) f ( ) cu ( ) f ( ) ceu ( ) Finally, the poof of (c) is given by u ) c u ( ) cu ( ) c u ( ) f ( ) c u( ) f ( ) E c ( cu ( ) f ( ) By applying (b), we obtain Ec u ) c u ( ) c Eu ( ) c Eu ( ) have ( Popety (c) can be etended to moe than two tems by mathematical induction; that is, we k k ( c) E ciui ( ) c i Eu i ( ) i i Because of popety ( c), mathematical epectation E() is called a linea o distibutive opeato Cetain mathematical epectations, if they eist, have special names and symbols to epesent them Fist, let u( ), whee is a andom vaiable of the discete type having a pmf f () Then E f ( ) If the discete points of the space of positive pobabilities ae a, a,, then

9 39 E ) a f ( a) a f ( a ) a f ( a ) ( 3 3 This sum of poduct is seen to be a weighted aveage of the values a, a,, the weight associated with each a being f a ) This suggests that we call E( ) i ( i the mean value of (o the mean value of the distibution) The mean value of a andom vaiable is defined, when it eists, to be E( ) Anothe special mathematical epectation is obtained by taking u ( ) ( ) If is a andom vaiable of the discete type having a pmf f (), then E ) ( ) f ( ) ( a ) f ( a ) ( a ) f ( a ) ( whee a, a,, ae the discete points of the space of positive pobabilities This sum of poduct may be intepeted as a weighted aveage of the squaes of the deviations of the numbes a, a,, fom the mean value of those numbes whee the weight asociated with each ( a ) is f a ) This mean value of the squae of the deviation of fom its mean value is i ( i called the vaiance of (o the vaiance of the distibution) ) The vaiance of will be denoted by ) E E E E E (, if it eists The vaiance can be computed in anothe manne: E ( It fequently affods an easie way of computing the vaiance of It is customay to call (the positive squae oot of the vaiance) the standad deviation of (o the standad deviation of the distibution) The numbe is sometimes intepeted as a measue of the dispesion of the points of the space elative to the mean value the space contains only one point fo which f ( ) 0, then = 0, We note that if We net define a thid special mathematical epectation, called the moment geneating function of a andom vaiable Definition 3-: If is a andom vaiable, the epected value M ( E e t is called the moment geneating function (MGF) of if this epected value eists fo all values of t in some open inteval containing 0 of the fom h t h fo some h > 0 It is evident that if we set t = 0, we have M ( 0) The moment geneating function is unique and completely detemines the distibution of the andom vaiable; thus, if two andom vaiables have the same moment geneating function, they have the same distibution andom vaiable has a pmf () b,,, then f with suppot b If a discete

10 30 t tb tb M e f ( ) f ( b ) e f ( b ) e ( Hence, the coefficient of tbi e is f b ) P( b ) That is, if we wite a moment geneating ( i i function of a discete-type andom vaiable in the above fom, the pobability of any value of, say b i, is the coefficient of tbi e Eample 3-: Let the moment geneating function of be defined by t t 3 3 t 4 4 t 5 5 t M ( e e e e e t Then, fo eample, the coefficient of e is 5 Thus f ( ) P( ) 5 In geneal, we see that the pmf of is f ( ) 5, =,, 3, 4, 5 Definition 3-3: Let be a andom vaiable and let be a positive intege If E eists, it is called the th moment of the distibution about the oigin E b is called the th moment of the distibution about b Theoem 3-: E If the moment geneating function of eists, then M ( ) d M ( (0) dt t 0 fo al =,, 3, In addition, the epectation Poof: Fom the theoy of mathematical analysis, it can be shown that the eistence of M (, fo h t h, implies that deivatives of M ( of all odes eists at t = 0; moeove, it is pemissible to intechange of the diffeentiation and epectation opeato Thus, dm ( dt if is of the continuous type, o t M ( e f ( ) d, dm ( dt M ( e t f ( ) if is of the discete type Setting t = 0, we have in eithe case M ( 0) E( ) The second deivative of M ( is t M ( e f ( ) d so that M (0) E( ) o t M ( e f ( ), In geneal, if is a positive intege, we have, by epeated diffeentiation with espect to t,

11 3 ( ) Note that M ( 0) M (0 ) M (0) E( ) Since ( the moment geneating function M geneates the value of E When the moment geneating function eists, deivatives of all odes eist at t = 0 possible to epesent M ( as a Maclauin s seies, namely, 3 t t t M ( M (0) M (0) M (0) M (0)!! 3! That is, if the Maclauin s seies epansion of (, it is called Thus it is M can be found, the th moment of, E, is the coefficient of t! O, if M ( eists and the moments ae given, we can fequently sum the Maclauin s seies to obtain the closed fom of M ( These points ae illustated in the net two eamples Eample 3-: fo all eal t be Suppose that the andom vaiable has the moment geneating function t M t e e t e t ( ) e t 5 Using the seies epansion of u e, the Maclauin s seies of M ( 4 ( 4) t ( 6) t ( M ( 5! 5 4! 5 hee is even Since the coefficient of t! E 5 0,, ) t ;! is zeo when is odd, we have,3,5,, 4, 6, ( ) In paticula, E( ) 0 and 5 is easily found to Eample 3-3: Let the moment of be defined by E 08,,,3, The moment geneating function of is then t t t 0 t t M ( M (0) e 0 8e!! 0! Thus, P( 0) 0 and P ( ) 0 8 Result: Suppose that the moment geneating function of eists Let R( ln M ( and

12 3 (k ) (0) R denote the kth deivative of R( evaluated fo t = 0 Then () () R (0) E( ) and R ( 0) E dr( M ( M (0) Poof: E( ) dt M ( M (0) d R( dt t 0 t 0 t 0 M ( M ( M ( M ( M ( t 0 M M (0) (0) M (0) E( ) E ( ) Eample 3-4: Let have the density function A andom vaiable with infinite mean f ( ), 0, othewise Then the epected values of is d E( ) Eample 3-5: A andom vaiable whose mean does not eist Let the continuous andom vaiable have the Cauchy distibution centeed at the oigin with density given by f ( ), The mean of is then and this integal does not eist since E( ) d log( ) 0 0 f ( ) d f ( ) d Eample 3-6: A andom vaiable whose fist moment eists but no highe moments eist Let the andom vaiable have a density given by 3 5 f ( ), Then the epected value of is

13 E ( ) 3 3 d Howeve, fo intege values of k >, we find that k 3 k 5 3 k 3 E ( ) d k 3 In fact, fo this eample the moment of ode k dose eist, although it is infinite the eample slightly to achieve a case whee the highe moments would be of the fom theefoe would not eist about zeo: We may modify and To do this, let the density have the same basic fom but be symmetic 3 5 g( ), 4 Moe genealy, the Student s t-distibution with + degees of feedom has moments of ode 0,,,, but no highe moments eist Eample 3-7: A andom vaiable whose moment geneating function does not eist Suppose the andom vaiable has the Cauchy distibution with density given by The integal is infinite fo any 0 f ( ), t t E( e ) e t since e t d is positive fo and tends to as Thus the moment geneating function does not eist in this eample Eample 3-8: A andom vaiable, all of whose moment eist, but whose moment geneating function dose not eist Eistence (finiteness) of the moment geneating function fo some t > 0 implies that all moments eist and ae finite; howeve, the convese is false Conside the lognomal distibution, which is the distibution of Y e whee has a nomal distibution Suppose has mean zeo and vaiance one, so that Y has the standad lognomal distibution The moments of Y eist fo all odes k =,, 3, since k k k k E e e e d ep E Y k d ep k Howeve, the moment geneating function does not eists since if t > 0,

14 34 ty k e E e E e te e d 3 ep t d 0 6 since the eponential tem is a thid-degee polynomial in fo which the tem has a positive coefficient; this eponential must then tend to as Theefoe the moment geneating function of Y does not eist Remak In moe advanced couse, we would not wok with the moment geneating function since so many distibutions do not have moment geneating functions 3 Instead, we would let i it denote the imaginay unit, t an abitay eal, and we would define ( Ee This epectation eists fo evey distibution and it is called the chaacteistic function of the distibution To see why (t ) eists fo all eal t, we note, in the continuous case, that its absolute value Howeve, f ( ) f ( ) since f () it ( e f ( ) d e f ( ) d is nonnegative and e it cos t i sin t cos t sin t Thus ( e f ( ) d e f ( ) d f ( ) d it it Evey distibution has a unique chaacteistic function; and to each chaacteistic function thee coesponds a unique distibution of pobability it If has a distibution with chaacteistic function (, then fo instance, if E( ) and E( ) eist, they ae given, espectively, by ie( ) (0) and i E( ) (0) It may wite ( M ( i 33 Chebyshev s Inequality Theoem 33- (Makov s Inequality): If is a andom vaiable that takes only nonnegative values, then fo any value a > 0 P E( a a ) Poof: We give a poof fo the case whee is continuous with density f E( ) a f ( ) d f ( ) d f ( ) d 0 0 a

15 35 and the esult is poved f d a ( ) a af ( ) d a f ( ) d ap( a) a vaiance Theoem 33- (Chebyshev s Inequality): If is a andom vaiable with finite mean and, then fo any value b > 0 P b b Poof: Since ( ) is nonnegative andom vaiable, we can apply Makov s inequality (with a b ) to obtain P E b b b But since if and only if b, we have and the poof is complete P E b b b The impotance of Makov s and Chebyshev s inequalities is that they enable us to deivative bounds on pobabilities when only the mean, o both the mean and the vaiance, of the pobability distibution ae known Of couse, if the actual distibution wee known, then the desied pobabilities could be eactly computed and we would not need to esot to bounds Eample 33-: Suppose that it is known that the numbe of items poduced in a factoy duing a week is a andom vaiable with mean 50 (a) What can be said about the pobability that this week s poduction wil eceed 75? (b) If the vaiance of a week s poduction is known to equal 5, then what can be said about the pobability that this week s poduction wil be between 40 and 60? Solution: Let be the numbe of items that will be poduced in a week: E( ) 50 (a) By Makov s inequality, P ( 75) P (b) By Chebyshev s inequality 0

16 36 3 P Hence, 0 Note that although Chebyshev s inequality is valid fo all distibutions of the andom vaiable, we cannot epect the bound on the pobability to be vey close to the actual pobability in most case