1.4. Experiments, Outcome, Sample Space, Events, and Random Variables

Transcription

1 1.4. Experments, Outcome, Sample Space, Events, and Random Varables In Secton 1.2.5, we dscuss how to fnd probabltes based on countng. Whle the probablty of any complex event s bult on countng, brute force countng s neffcent when s complex. In fact, there s no way to count f takes up uncountably many values (e.g., the useful lfe of a car can be any real number from 0 to 14 years, say). Fortunately, there has already been a rgorous, formalzed approach to gude us to determne probabltes of events. The approach s bult on defnng an experment, an outcome, a sample space, an event, and a random varable step by step, wth the latter concepts bult on earler ones. An experment s what we do to get raw data (samples, observatons) n order to study a gven (random, stochastc) phenomenon. Example (Dfferent Types of Experments) Rollng a Dce Suppose that you want to study the statstcal characterstcs of the number that you get n rollng a partcular dce. Then the rollng of the dce s the experment that you carry out to get sample data, whch may be a sequence such as {4, 1, 1, 6, 3, 2,...}. In settng your experment, you certanly need to decde the number of rolls (amount of data) that you want to make. Usually, ths s determned by the amount of resources that you have. (b) Qualty Control To check whether a process s n control or not, you sample output from the process and measure the sampled output accordng to some chosen crtera. Based on these sampled data (the raw data), you derve statstcs from whch you make nference on whether the process s n control or not. In ths case, the experment s what you do to get the sampled data. (c) Input Parameters of a Model To help the unversty cafetera to ease the congeston problem at lunchtme, you need to collect data that reflect the arrval and servce patterns of the cafetera at lunchtmes. In ths case, the experment s the desgn and the process to get the data; the data collected may look lke: Batch Sze n Tme arrved, t a Tme to get a Table, t t Tme to order, t o Tme to get the fst dsh, t f Tme to fnsh the last dsh, t l Tme to get the bll, t b Tme to leave, t d The exact type and amount of data that you collect depend on your model. An outcome of an experment s a sample data that you get from your experment. An outcome can be a vector, not just a sngle element; also, t can be qualtatve rather than quanttatve. 1

2 Example (Outcomes of Experments) Rollng a Dce 4 s a possble outcome n rollng a dce once. Actually, any number from 1 to 6 s a possble outcome for a standard dce. If we are talkng about rollng a dce twce, the outcome wll be a vector (m, n) such that 1 m, n 6. (b) Qualty Control The outcome can ether be quanttatve or qualtatve. Suppose that the process s a drllng machne that drlls holes wth a dameter cm. If you measure the actual dameter of holes n your experment, an outcome could be a number such as 4.01 or However, f n our experment, we only get a measure aganst a standard template whch shows whether a hole s good (wthn lmts specfed by the standard) or not (outsde lmts), then the outcome s just one of the two possbltes (good, defectve) [or (wthn lmts, outsde ltmus), etc.]. (c) Input Parameters of a Model An outcome of the experment may be a vector (n, t a, t t, t o, t f, t l, t b, t d ), or ts equvalence. Clearly outcomes vary wth the desgn of an experment. The set of all possble outcomes of an experment, denoted by, s the sample space of the experment. The elements of arecompletely dctated by how an experment s conducted. Later we wll see that outcomes of a sample space (of a gven experment) may or may not appear equally lkely. Example (Sample Space) Rollng a dce = {1, 2, 3, 4, 5, 6}. (b) Qualty Control = {x 3.98 x 4.02}. Note that there are two ways to defne : () lstng out all elements as n part, () specfyng the necessary and suffcent condtons for an outcome to be an element as n part (b). Usually () s used for smple, fnte, whle () s used for complex or nfnte. (c) Input Parameters of a Model 2

3 In ths case, we may not know the real, even after we have fnshed our experment. Who can know for sure the pattern of students and staff arrvng at the cafetera? In practce, we may defne = {(n, t a, t t, t o, t f, t l, t b, t d ) n {0}, a t a < t t < t o < t f < t l < t b < t d b}, where s the set of natural numbers, and a (say, = 11:30 am) and b (say, = 2:30 pm) are the two parameter values that we choose to stand for the begnnng and the end of the lunch tme. (d) Rollng two dce = {( 1, 2 ) and 1 6, = 1, 2}. (e) Flppng cons ndefntely = {( 1, 2,...) {head, tal}}. A sample space tells us what outcomes are possble. Frequently, we are nterested n a collecton of outcomes, rather than ndvdual ones. For example, when we bet n a casno, we may be nterested n the occurrence of any of {10, 11,, 18}, rather than n any partcular number (outcome). Such a collecton of outcome s called an event. Mathematcally, an event s a subset of the sample space. Later, we wll see that probabltes are defned on events rather than on outcomes. There are events of a sngle outcome n a fnte. Then what are the dfferences between an outcome and an event for such cases? Bascally, an outcome s an element of, whle an event s a subset. Of course there are subsets of sngle elements, but an element and the correspondng sngle-element subset are two dfferent objects. Example (Events) In the followng, A, B, C, and D are events defned n ther own sample space. Rollng a dce A = the event of gettng a 5 n a roll = {5}. B = the event of gettng an even number n a roll = {2, 4, 6}. C = the event of gettng a number less than 3 or no more than 5 n a roll = {1, 2, 5, 6}. (b) Qualty Control A = the event that the dameter of a hole s less than 4.00 cm = {x x < 4}. B = the event that the dameter of a hole s defectve = {x x < 3.98 or x > 4.02}. C = the event that the dameter of a hole les wthn the 1st to 3rd quadrants of tolerance range = { x 3.99 x 4.01}. 3

4 (c) Input Parameters of a Model A = the event that a batch of customers comes after 1:00 pm = { t a > 13}. B = the event that a batch of customers wats more than 15 mnutes before gettng a table. = { t t -t a 0.25}. C = the event that a batch s of three or more customers = { n 3}. (d) Rollng two dce A = the event that the frst dce s even and the second dce s less than the frst = {( 1, 2 ) 1 {2, 4, 6} and 1 > 2 }. B = the event that the sum of the two dce s greater than or equal to 5 = {( 1, 2 ) }. C = the event that the dfference between the two dce s greater than 7 = {( 1, 2 ) max( 1, 2 ) - mn( 1, 2 ) > 7} =. Note that C s an mpossble event. (e) Flppng cons ndefntely A B = the event that the number of heads s no less than the number of tals = {( 1, 2,...) number of such that s a head number of such that s a tal}. = the event that the frst two flps are heads and the fourth flp s a tal = {( 1, 2,...) 1 = 2 = head, 4 = tal }. We can generate events by set operatons unon (), ntersecton (), and complement [() c ]. Let A and B be events of. Then AB, AB, A c, and any sets formed by countable set operatons of A and B are subsets, and hence events, of. We have sad n the last part of Secton 1.1 that OR II s a course to study (sequences of) random varables. Here we wll gve more detals. Remember varables are nothng more than the x, y, or z that we use to denote unknowns n problems of our hgh school algebra. Random varables X, Y, or Z are of smlar nature. The dfference s that a real varable x can only take up one value, whle before we know the exact value of a random varable X, t may take up dfferent values (wth possbly dfferent probabltes). For example, there s only one result n rollng a far dce. However, before we observe the outcome, the face that shows up s wth probablty 1/6, = 1 to 6. How do we defne a functon f when we work wth real varables? One common defnton s that f s a mappng from. The defnton of a random varable s an analogy of real-valued functons: A random varable s a functon mapped from the sample space to the real lne. It mples that to defne random varables, we must frst have (an experment and ts). A random varable X s defned by the functon that takes up the real value X() for every. 4

5 For notatonal smplcty, usually we do not spell out when ts content s clear or when explctly spellng out s not necessary n solvng a problem. Smlarly, we hde as long as the actual content of s clear n the problem. Example (Random Varables Con t of Example 1.4.3) space. In the followng, N, X, Y, and Z are random varables defned n ther own sample Rollng a dce Let N be the number shown up n a roll. Remember that = {1, 2, 3, 4, 5, 6}. Note that N(1) = 1, N(2) = 2, N(3) = 3, N(4) = 4, N(5) = 5, and N(6) = 6. Clearly, N: s a functon mapped from to. Let X = 2N X s a functon mapped from to, snce t s defned through N, whch s a functon mapped from to. Check that X: such that X(1) = -1.5, X(2) = 0.5, X(3) = 2.5, X(4) = 4.5, X(5) = 6.5, X(6) = 8.5. It s clear that X s a functon mapped from to the real lne. 1, the roll gves an odd number, Let Y = Then Y: such that Y(1) = Y(3) = Y(5) 0, o. w. = 1 and Y(2) = Y(4) = Y(6) = 0. Y s an ndcator varable (functon) that ndcates whether an event occurs or not. Let Z = the square root of the number shown up n a roll. Then Z: such that Z() =, for. (b) Qualty Control Let X be the dameter of a drlled hole. Then X: such that X() =, for. Let Y be the cost spent n reworkng a defectve hole, gven that each rework costs $3. Then Y: such that Y() = 3, 0, 3.98 or 4.02, o. w.,. Note that s a dummy varable that can be replaced by any symbol as long as the symbol s defned to be an element of. (c) Input Parameters of a Model Let X be number of customers n a batch f the batch spends no more than an hour n the cafetera, and s equal to otherwse. 5

6 Then X: such that X() = n( ), f td( ) ta( ) 1, 489., ow..,. (d). Rollng two dce Let X be the sum of the two throws. Then X: such that X(( 1, 2 )) = 1 + 2, Let Y be the dfference between the larger and the smaller numbers. Then Y: such that Y(( 1, 2 )) = max( 1, 2 ) - mn( 1, 2 ),. (e) Flppng cons ndefntely Let X be the total number of heads n the second through the ffth flps; I j be the ndcator functon such that t s equal to 1 f the jth flp s a head, and s equal to 0 otherwse. (Could you wrte out the functon form of I j?) Then X: such that X() = 5 I j j2 ( ),. As we sad above, when the underlyng sample space s clear, we can save our effort by not explctly lstng out, nor. For example, the random varables X s n (c) and (e) of n, f td ta 1, 5 Example can be lsted, respectvely, as X = and X = I 489., ow.., j. In j2 fact, most appled probablty books prefer to use ths form, rather than the lengthy form wth explct lstng of and. Events are generated by random varables n ther correspondng by operators >,, <,, and =. For example, n part (c) of Example 1.4.5, {: X() > 0} {: t d () - t a () 1}; n part (e) of the same example, {X = 1} {there s 1 head n the second to the ffth flps}. Snce random varables and events are related ntmately, they wll be studed n parallel. You wll see that n latter chapters the dscusson of random varables and events comes hand n hand: (condtonal) expectaton of a random varable comes together wth the (condtonal) probablty of an event. Agan, for notatonal smplcty, we usually hde the dependence of events on. For example, we wrte {X > 0} for {: X() > 0} and {t d - t a 1} for {: t d () - t a () 1}. Don t you feel uncomfortable that up to now we have not used nor dscussed probablty? 1.5. Probablty of Events As we mentoned above, t s mpossble to compute probabltes by purely countng. For example, there may be uncountably many outcomes n. The followng s a standard, mathematcally consstent way to defne probabltes. 6

7 The probablty of events s a (set) functon [.e., a functon that defnes on a set] P that has the followng propertes: P(A) 0 for any A. (b) If A s are mutually exclusve subsets of,.e., A and A A j = for j, then P(A 1 A 2...) = P(A 1 ) + P(A 2 ) (c) P() = 1. Exercse Further Propertes of Probablty Let A and B be subsets of. Show that P(A c ) = 1 - P(A). (b) P() = 0. (c) f A B, then P(A) P(B). (d) 0 P(A) 1. (e) P(AB) = P(A) + P(B) - P(AB). The probabltes of events on a sample space are dependent on how the events are generated by the random mechansm of the sample space. For smple problems, we just use the ntutve dea of farness among dfferent outcomes. Example Roll a dce There are sx faces on a (standard) dce. Provdng that we roll randomly, there s no reason for us to beleve that one face has more chance than another one. So t suffces to determne the probablty of an event by total # of outcomes relevant to the event. P(an even number shown up) = 3/6 = 0.5. total # of possble outcomes (b) Qualty Control Suppose that a hole drlled s equally lkely to be somewhere n (3.97, 4.02), whle the acceptable range s (3.98, 4.02). Snce the length of a lne segment reflects the number of possble outcomes, we may ntutvely put the probablty of an event as length of the range of outcome relevant to the event length of the possble range ( ) For example, P(a hole s defectve) = 02.. Remark We wll soon see that n complex problems the probabltes of events are not found from the ntutve farness dea. In fact, there s no farness at all; smlar events can have substantally dfferent probabltes. We work wth random varables. How can we compute the probablty of a random varable? To do so, we fnd the probablty of the event that s generated by a random 7

8 varable (through the equalty and nequalty operators). For example, to fnd the probablty that X s equal to 3, we determne the probablty of the set { X() = 3}; smlarly, to fnd the probablty that X s greater than t, we determne the probablty of the set { X() > t}. To smplfy exposton, we usually hde n the above expressons and wrte as P(X = 3) and P(X > t). How to fnd the probabltes of complex events? Obvously t s not by countng as n Example Real-lfe problems have so many varables that you cannot fnsh countng a small porton of a problem before you graduate. We may count wth the help of computers, whch s what smulaton for. To prepare you for smulaton, you need basc knowledge n stochastc models, whch s one motvaton of OR II. The process to fnd probabltes of complex events can be smplfed by notng the structure of the underlyng problem and by manpulatng rules n probablty theory and analyss. A lot of the calculaton for probabltes (and expected values) wll be smplfed by condtonng. See Chapter Dscrete and Contnuous Random Varables, Probablty Mass Functons, Densty Functons, and Cumulatve Dstrbuton Functons Random varables that you encounter n undergraduate textbooks generally can be classfed as ether dscrete or contnuous. A dscrete random varable has non-zero probablty on countable number of tems whle a contnuous random varable has non-zero probablty on uncountable number of tems. (Strctly speakng, a contnuous random varable has non-zero densty, rather than probablty, on uncountable number of tems,.) As a result, the sample space of a dscrete random varable has a countable number of elements [for example, of Example 1.4.3, (d), and (e)], and the sample space a contnuous random varable s uncountable [for example, of Example (b) and (c)]. How to fnd the probabltes of events {X = 3} or {X 4.69}? Ths s determned by a probablty mass functon for a dscrete random varable and by a densty functon for a contnuous random varable. The probablty mass functon (p.m.f.) p: of a dscrete random varable X s the functon satsfyng p() = P(X = ), for all. (b) p () 1. By defnton, p() 0. For smplcty, we may use p to denote p(). Example Dscrete Random Varables Let = {1, 2, 3, 4, 5, 6}. The followng p s are p.m.f. s on. 8

9 p 1 = p 2 = p 3 = p 4 = p 5 = p 6 = 1/6 and all other p s are equal to 0. Ths s a (dscrete) unform dstrbuton on. (b) p 2 = 0.5, p 3 = 0.05, p 4 = 0.3, p 5 = 0.05, p 6 = 0.1, and all other p s are equal to 0. (c) p 4 = 1 and all other p s are equal to 0. The random varable has a pont mass at 4. Example shows that for the same, there can be many (n fact, uncountably many) dfferent probablty densty functons that can be defned on t. The densty functon f: of a contnuous random varable X s the functon satsfyng f(x) 0, for all x. (b) f ( x) dx 1. (b) P ( X t) f ( x) dx, for all t. { x t} Example Contnuous Random Varables Let = (0, 1), a lne segment from 0 to 1. The followng f s are densty functons on. f(x) = 1 for 0 < x < 1, and f(x) = 0 otherwse. Ths s a unform dstrbuton on. (b) f(x) = 4x for 0 < x < 0.5, and f(x) = 4-4x for 0.5 x < 1. Ths s a trangular dstrbuton on. The cumulatve dstrbuton functon F: of a random varable, no matter dscrete or contnuous, s defned as p, t for dscrete random varables, F(t) = P(X t) = f ( x) dx, for contnuous random varables. xt Exercse Cumulatve Dstrbuton Functons Fnd the cumulatve dstrbuton functons of the p.m.f. s and densty functons n Examples and It can be shown that gven the cumulatve dstrbuton functons F (of a random varable X), ts correspondng p.m.f. (of X for a dscrete X) or densty functon (of X for a contnuous X) s completely determned. Smlarly, gven a p.m.f. or a densty functon, the correspondng F s also completely determned. From here onwards, when we refer to the 9

10 dstrbuton of a random varable, we may talk about ts p.m.f., densty functon, or ts cumulatve dstrbuton functon, whchever exsts and whchever s dscussed under the gven context Mean, Moments, and Varance of Random Varables The mean (expectaton) of a random varable X s = E(X) = p, xf ( x) dx, for dscrete random varables, for contnuous random varables. It s a measure of the central tendency of the random varable. The rth moment, r > 0, a random varable X s E(X r ) = r x r p, f ( x) dx, for dscrete random varables, for contnuous random varables. The frst moment s the mean of a random varable. The varance 2 of a random varable X s 2 = V(X) = E[(X-) 2 ] = ( ) ( x ) 2 2 p, f ( x) dx, for dscrete random varables, for contnuous random varables. It s a measure of the dsperson of the random varable. Expectaton s a lnear operaton: Let x and y be the mean of X and Y, respectvely; a and b be two real numbers; then E(aX + by) = a x + b y. The expected value of a functon of a random varable can be found easly. Let h(x) be a real-valued functon. Then E[h(X)] = h( ) p, h( x) f ( x) dx, f X s dscrete, f X s contnuous. Exercse Propertes of Mean, Moments, Varance, and Covarance Lnear Transform of Varances Let Y = ax + b, where a and b are two real numbers. Show that V(Y) = a 2 V(X). (b) Relatonshp between the Second Moment and Varance 10

11 E(X 2 ) = V(X) + E 2 (X). (c) Covarance V(X+Y) = V(X) + V(Y) + 2 Cov(X, Y), where Cov(X, Y) = E[(X- x )(Y- y )] s the covarance of X and Y Condtonal Probablty, Bayes formula, and Independence of Events Let A and B be two events defned on the same such that P(B) > 0. Then the probablty of A condtonng on (the occurrence of) B s P(A B) = P(AB)/ P(B). Example Rollng two dce Two dce are rolled. Let A = {the sum s less than 5} and B = {the frst number s 2}. We can fnd P(A B) ether by countng or by condtonal probablty. Countng = {(m, n): 1 m, n 6} has 36 elements. A = {(1, 1), (1, 2), (1,3), (2, 1), (2, 2), (3, 1)}; B = {(2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6)}. AB = {(2, 1), (2, 2)} has two elements (outcomes). Gven the occurrence of B, there s no preference of any of ts element over other ones. So, the outcome belongs to A wth probablty 2/6 = 1/3, whch s P(A B). (b) Mathematcal Defnton Clearly P(B) = 1/6 (why?). AB = {(2, 1), (2, 2)} whch means that P(AB) = 2/36 = 1/18. P(A B) = P(AB)/ P(B) = (1/18)/(1/6) = 1/3. In the above smple example, there s not much dfference between countng and condtonal probablty. In countng, P(A B) s found from 2/6; n condtonal probablty, P(A B) s found from (1/18)/(1/6) = (2/36)/(6/36),.e., the condtonal probablty s smply got by dvdng the numerator and denomnator n countng by the total number of elements n. In a way ths s true for all condtonal probabltes. However, the power of condtonng to smplfy the calculaton of probablty and expectaton s tremendous. See Chapter 3. Let B s be mutually exhaustve sets on ;.e., B = and B B j = for j. Assume that P(B ) > 0. The Bayes formula shows how to get P(B A) from P(A B ): P(B A) = PB ( A ) P( A) Example P( B A) = P( B A) j j P( B ) P( A B ) =. P( B ) P( A B ) j j j 11

12 A gadget s produced from two producton lnes L 1 and L 2. The producton rates of L 1 and L 2 are 200 peces / hour and 300 peces / hour, respectvely. The defectve rates of L 1 and L 2 are 0.01 and 0.02, respectvely. Gven that a gadget s defectve, fnd the probablty that t s produced by L 1. Sol. Let A = {the gadget s defectve}; B = {the gadget s produced by L }, = 1, 2. P(A B 1 ) = 0.01, P(A B 2 ) = 0.02, P(B 1 ) = 0.4, P(B 2 ) = 0.6. The probablty that the gadget s produced by L 1 s P(B 1 A) = P( B1 A) PB ( A) = P( B1) PAB ( 1) PB ( ) PAB ( ) = ( 0.4)( 001. ) ( 0.4)( 001. ) ( 06. )( 002. ) = Thnk twce about ths deceptvely smple example. What s the bg deal of t? The calculaton s so smple that even a kd of 12 years old can plug numbers nto the formulae PB ( 1) PAB ( 1) to get Gettng a correct number s not the man pont of such a problem PB ( ) PAB ( ) (n fact, not the man pont of studyng). Rather, the essence s to buld up the ablty to formulate a problem, whch, n ths example, s to defne A and B s correctly to make the calculaton trval. Bear n mnd that modelng s the process to defne varables and represent ther nteractons logcally and/or analytcally. Just knowng how to use CPLEX (ARENA) to solve a lnear program (to smulate) does not necessarly mean that you know lnear programmng (smulaton). Now we are gong to ntroduce another mportant concept n probablty: ndependence. Vaguely speakng, two random objects are ndependent from each f ganng knowledge on one object does not affect your assessment on any probablstc evaluaton of the other object. Ths dea can be formalzed. We frst study the ndependence of events. Later, we wll generalze nto the ndependence of random varables. Two events A and B are ndependent from each other f P(AB) = P(A)P(B). Such a relaton among events (and generalzed to random varables) smplfes calculaton. Example (Independent Events) Sx dce are rolled ndependently. Let A = {face shows up n rollng the th dce}, = 1 to 6. Fnd P( 6 1 A ). Sol. Snce the rolls are ndependent, P( 6 1 A 6 ) = PA ( ) 1 = See how ndependence smplfes the calculaton. We do not need to construct for rollng the sx dce and count the number of elements. We get the correct probablty of an event by followng standard rules n ndependence. An equvalent defnton of ndependence of two events A and B s that P(A B) = P(A) (when P(B) > 0). Intutvely, ths defnton says that events A and B are ndependent f the 12

13 occurrence of B does not affect the probablty of occurrence of A. Note that the above relaton s symmetrc wth respect to A and B,.e., P(A B) = P(A) s equvalent to P(B A) = P(B) (when P(B) > 0) Jont Dstrbutons, Margnal Dstrbutons, and Independence of Random Varables The dstrbuton of a dscrete (resp. contnuous) random varable can be specfed by the dstrbuton functon and the p.m.f. (resp. densty functon) of the random varable. In real-lfe stochastc models, usually there are many random varables and we need to consder a jont dstrbuton: Let X and Y be two random varables defned on the same. Ther jont dstrbuton functon F(x, y) = P(X x, Y y), for all real x and y. If X and Y are dscrete random varables, they have a jont p.m.f. p a,b = P(X = a, Y = b). If X and Y are contnuous 2 F ( x, y) random varables, they have a jont densty functon f(x, y) =. By defnton, for all x xy and y, F(x, y) = y j y xx P( X { ty} { sx} x, Y y f ( s, t) dsdt, j ), for dscrete X and Y, for contnuous X and Y. For a par of contnuous random varables, the jont densty functon f(x, y) = 2 2 F( x, y) P( X x, Y y). xy xy The margnal dstrbuton of X can be found from the jont dstrbuton by settng the lmt for Y to : P(X x) = P(X x, Y < ) = F(x, ). Smlarly, the margnal dstrbuton of Y s found from P(Y y) = P(X <, Y y) = F(, b). The margnal dstrbuton can also be defned from jont p.m.f. or densty functon. Check that f f(x, y) s the jont densty of X and Y and f X (x) s the margnal densty of X, then f X (x) = f ( x, t) dt. Smlarly, the margnal densty of Y f Y (y) = f ( s, y) ds. Example (Jont and Margnal Dstrbutons of Dscrete Random Varables) A bn contans three whte and two black balls. The whte balls are numbered 1, 2, and 3, whle the black balls are numbered 4 and 5. Let X be the number of a ball randomly pcked from bn; Y = 1 f the ball s black; Y = 0 o.w. Fnd the jont dstrbuton of (X, Y). (b) Fnd the margnal dstrbuton of X. (c) Fnd the margnal dstrbuton of Y. 13

14 Sol. In terms of the outcomes, the probablty dstrbuton s: W1 W2 W3 B4 B5 (X, Y) () (1, 0) (2, 0) (3, 0) (4, 1) (5, 1) P{} In terms of the random varables, the probablty dstrbuton s: Y X (b) P(X = 1) = y P( X 1, Y y) = 0.2. Smlarly, P(X = 2) = P(X = 3) = P(X = 4) = P(X = 5) = 0.2. (c) P(Y = 0) = x P( X x, Y 0 ) = 0.6, and P(Y = 1) = x P( X x, Y 1 ) = 0.4. Example (Jont and Margnal Dstrbutons of Contnuous Random Varables) The jont densty functon of X and Y s f(x, y) = 3 x y, 0 < x, y < 1. 2 Fnd the margnal dstrbuton of X. (b) Fnd the margnal dstrbuton of Y. Sol. f X (x) = x y dy = 5 x, 0 < x < (b) f Y (y) = x y dx = 5 y, 0 < y < We have seen n Secton 1.8 how ndependence of events smplfes calculatng ther probabltes. Ths can be generalzed to random varables, snce events are generated by random varables wth operators,, <, >, and =. Two random varables X and Y are ndependent f ther jont dstrbuton F(x, y) = P(X x, Y y) = P(X x)p(y y) for all x and y. If X and Y are dscrete, the ndependence s equvalent to P(X = x, Y = y) = P(X = x)p(y = y) for all x and y. If X and Y are contnuous, the ndependence s equvalent to the jont densty functon f(x, y) = f(x) f(y) for all x and y. Ths s a drect generalzaton of ndependent of events. A term such as {X x, Y y} ({X = x, Y = y}, etc.) s ndeed the ntersecton of {X x}{y y} ({X = x}{y = y}, etc.). The defnton bascally says that for random varables X and Y to be ndependent, any par of 14

15 events {X x} and {Y y} ({X = x} and {Y = y}, etc.) generated by X and Y must be ndependent (events). The ndependence of random varables can also be expressed by the more ntutvely appealng concept that nformaton of one random varable does not affect the dstrbuton of the other varable. For example, when both X and Y are dscrete, they are ndependent f P(X = x Y = y) = P(X = x) for all x, y. Agan, the above expresson s symmetrc. It s equvalence to P(Y = y X = x) = P(Y = y). Example (Independent Random Varables) Two cons are flpped (ndependently). Let T be the number of tals and H be the number of heads n the two flps; H be the number of head n the th flp, = 1, 2. We are gong to check whch pars of random varables are ndependent from each other. T and H Snce H + T = 2, they cannot be ndependent from each other. For example, P(T = 1, H = 0) = 0 P(T = 1) P(H = 0). (b) H and H 1 They cannot be ndependent from each other. Gven H = 0, H 1 must be 0, whch means that P(H = 0, H 1 = 1) = 0 P(H = 0) P(H 1 = 1). By a smlar reasonng, H and H 2 are dependent; T and H 1 are dependent; T and H 2 are dependent. (c) H 1 and H 2 Snce the flps are ndependent, H 1 and H 2 must be ndependent random varables. To be rgorous, we need to go through the mathematcs. = {(head, head), (head, tal), (tal, head), (tal, tal)}, and each element has equal chance of occurrence. P(H 1 = 1, H 2 = 1) = 0.25 = P(H 1 = 1) P(H 2 = 1), P(H 1 = 1, H 2 = 0) = 0.25 = P(H 1 = 1) P(H 2 = 0), P(H 1 = 0, H 2 = 1) = 0.25 = P(H 1 = 0) P(H 2 = 1), P(H 1 = 0, H 2 = 0) = 0.25 = P(H 1 = 0) P(H 2 = 0), whch confrms that H 1 and H 2 are ndependent. Note that n Example the dependence and ndependence of random varables are more or less argued by notng the nteracton among random varables. Mathematcs s only used to confrm our ntuton. For smple problems as ths one, our qualtatve argument s 15

16 suffcent. Mathematcs s reserved for or complex problems. In fact, notng the nteracton among random varables s so essental for modelng that we better sharpen our ablty n t. One may ask where the boundary between smple and complex problems s. The answer s that there s no well-defned boundary at all. Experts can use ntuton to shorten a fve-page mathematcal proof to half a page [though sometme, some experts may force a (wrong) result by (wrong) ntuton]. In any case, one should equp hmself wth both ntuton and mathematcs. Example (Cont n of Example Independence of random varables) The two random varables X and Y n Example are dependent because f(x, y) = 3 x y 5 x 5 y ( )( ) = f X (x) f Y (y) for at least some x and y We say that X and Y are ndependent and dentcally dstrbuted (..d.) f they are ndependent and have the same dstrbuton. Example (A Property of Independent Random Varables) Let X and Y be ndependent random varables. Show that E(XY) = E(X)E(Y). Sol. We gve a partal proof that s suffcent for our purpose. Consder the case that X and Y are dscrete random varables. E(XY) = y x xyp X x, Y y) ( = yp ( Y y) xp( X ) = yp ( Y y) E( X ) = E ( X ) yp( Y y) = E(X)E(Y). y y y x x In fact, f X and Y are ndependent, for any real-valued functons g and h, E[g(X)h(Y)] = E[g(X)]E[h(Y)]. Example (Covarance of ndependent random varables) If X and Y are ndependent, show that ther covarance s equal to zero. Sol. Cov(X, Y) = E[(X- x )(Y- y )] = E(X- x ) E(Y- y ) = 0. The second equalty makes use of the ndependence of X and Y as stated n Example Exercse Let X and Y be two ndependent random varables; G and H are two realvalued functon. Show that G(X) and H(Y) are ndependent Convoluton 16

17 Let X and Y be two ndependent random varables and Z = X + Y. Then the dstrbuton of Z s called the convoluton of X and Y. Ths can be expressed as p.m.f., densty functons, or cumulatve dstrbuton functons, whchever approprate. We wll dscuss how to fnd Z n later chapters. 17

18 Fnal Remark of Chapter 1. The materal from 1.4 to 1.10 can be found from Ross [2000] and Walpole and Myers [1989]. Sectons n Notes Sectons n Ross [2000] Sectons n Walpole & Myers [1989] , , 1.5, (pp 23-27, 33-34) 2.2, (pp 37-38) ,