Continuous Random Variables: Conditioning, Expectation and Independence

Size: px

Start display at page:

Download "Continuous Random Variables: Conditioning, Expectation and Independence"

Clifford Melton
6 years ago
Views:

1 Cotiuous Radom Variables: Coditioig, Expectatio ad Idepedece Berli Che Departmet o Computer ciece & Iormatio Egieerig Natioal Taiwa Normal Uiversit Reerece: - D.. Bertsekas, J. N. Tsitsiklis, Itroductio to robabilit, ectios

2 Coditioig DF Give a Evet (/3) The coditioal DF o a cotiuous radom variable, give a evet A A I caot be described i terms o, the coditioal DF is deied as a oegative uctio satisig ( B A) ( x)dx B A A ( x) Normalizatio propert A ( x ) dx robabilit-berli Che

3 Coditioig DF Give a Evet (/3) A I ca be described i terms o ( is a subset o the real lie with ( A) > 0 ), the coditioal DF is deied as a oegative uctio x satisig A ( x ) 0, The coditioal DF is zero outside the coditioig evet ad or a subset Normalizatio ropert ( B A ) ( x ) ( A ) B A ( ) A, i x A otherwise ( B, A ) ( A ) I B ( x ) dx ( A ) A ( x ) dx ( x ) dx ( x ) dx A B A A A A the vertical axis remais the same shape as except that it is scaled alog robabilit-berli Che 3

4 Coditioig DF Give a Evet (3/3) A, A,, i A I K are disjoit evets with ( A i ) > 0 or each, that orm a partitio o the sample space, the ( x ) ( A ) ( x ) i i A i Veriicatio o the above total probabilit theorem ( x) ( A ) ( x A ) x Takig the i x () t dt ( A ) () t ( x) ( A ) ( x) i i i derivative i i A i i A i dt o both sides with respective to x robabilit-berli Che 4

5 A Illustrative Example () t Example 3.9. The expoetial radom variable is memorless. T is expoetial T λt ( > t) e T The time T util a ew light bulb burs out is expoetial distributio. Joh turs the light o, leave the room, ad whe he returs, t time uits later, id that the light bulb is still o, which correspods to the evet A{T>t} Let be the additioal time util the light bulb burs out. What is the coditioal DF o give A? T t, A { T > t} λeλt, t > 0 0, otherwise The coditioal CDF o ( > x A) ( T t > x T > t) (where x ( ) ( T > t + x ad T > t) T > t + xt > t e e e ( > t + x) ( T > t) T λ λx ( t+ x) λt give A is deied b ( > t) T 0) The coditioal DFo theevet A is also expoetial with parameter λ. give robabilit-berli Che 5

6 Coditioal Expectatio Give a Evet The coditioal expectatio o a cotiuous radom variable, give a evet ( A > ), is deied b E The coditioal expectatio o a uctio also has the orm E[ g( ) A] g( x) A( x) dx A [ A ] x ( x ) A dx ( ) 0 g ( ) Total Expectatio Theorem ad E E [ ] ( A ) E[ ] i A, A,, A i A i [ g( )] ( A ) E g( ) i [ ] i A i Where K are disjoit evets with ( A i ) > 0 each i, that orm a partitio o the sample space or robabilit-berli Che 6

Illustrative Examples (/) Example 3.0. Mea ad Variace o a iecewise Costat DF. uppose that the radom variable has the piecewise costat DF Deie ( x) / 3, / 3, 0, lies i 0 x, i x, otherwise.

7 Illustrative Examples (/) Example 3.0. Mea ad Variace o a iecewise Costat DF. uppose that the radom variable has the piecewise costat DF Deie ( x) / 3, / 3, 0, lies i 0 x, i x, otherwise. evet A { i the irst iterval [0,] } evet A { lies i the secod iterval [,] } ( A ) 0 / 3dx / 3, ( A ) / 3dx / 3 ( x) ( x), 0 x ( x) ( A ) ( x) ( A ) Recall that the mea ad secod momet o a uiorm radom variable over a iterval ( a + b) / ad ( a + ab b )/ 3 [ a, b ] is + E E A [ A ] /, E[ A ] / 3 [ A ] 3 /, E[ A ] 7 / 3 0, A otherwise E E var 0, [ ] ( A ) E[ A ] + ( A ) E[ A ] / 3 / + / 3 3 / [ ] ( A ) E[ A ] + ( A ) E[ A ] / 3 / 3 + / 3 7 / 3 5 / 9 ( ) 5 / 9 ( 7 / 6) / 36, x otherwise 7 / 6 robabilit-berli Che 7

8 Illustrative Examples (/) /0 Example 3.. The metro trai arrives at the statio ear our home ever quarter hour startig at 6:00 AM. ou walk ito the statio ever morig betwee 7:0 ad 7:30 AM, with the time i this iterval beig a uiorm radom variable. What is the DF o the time ou have to wait or the irst trai to arrive? - The arrival time, deoted b, is a uiorm radom variable over the iterval 7 :0 to 7 : 30 - Let radom varible model - Let A be a evet A B - Let - Let { 7 :0 7 :5} - Let B be a evet { 7 :5 < 7 : 30} the waitig time (ou board the 7 :5 trai) (ou board the 7 : 30 trai) be uiorm coditioe d o A be uiorm coditioe d o B ( ) ( A) ( ) ( B) ( ) A + B robabilit-berli Che 8

9 Multiple Cotiuous Radom Variables (/) Two cotiuous radom variables ad associated with a commo experimet are joitl cotiuous ad ca be described i terms o a joit DF satisig ((, ) B) ( x ) ) B, is a oegative uctio Normalizatio robabilit, ) dxd, ( a, c) imilarl, ca be viewed as the probabilit per uit area i the viciit o ( a, c) Where δ is a small positive umber,,, dxd ( a a + δ, c c + δ ) a+ δ c+ δ ) dxd ( a c) δ a c,,, robabilit-berli Che 9

10 Multiple Cotiuous Radom Variables (/) Margial robabilit ( A) ( A ad (, ) ) We have alread deied that We thus have the margial DF imilarl A, ) ( A ) ( x ) A ( x) ( x ),, ( ) ( x ),, d dx ddx dx robabilit-berli Che 0

11 A Illustrative Example Example 3.3. Two-Dimesioal Uiorm DF. We are told that the joit DF o the radom variables ad is a costat c o a area ad is zero outside. Fid the value o c ad the margial DFs o ad. or The a area, correspod ), x ig is deied uiorm ize o area 0, 4 to be (c. Example, joit ) or ) 4 ( x) ) 4 d 4 or x 3 3 ( x) ),, d 4 4 d d i DF ) o otherwise 3.) or or ( ) ) dx ( ) ),, 4 3 dx 4 dx dx or 3 ( ) ) 4, dx 4 4 dx robabilit-berli Che

12 Coditioig oe Radom Variable o Aother Two cotiuous radom variables ad have a joit DF. For a with ( ) > 0, the coditioal DF o give that is deied b ( x ) Normalizatio ropert, ) ( ) ( x ) dx The margial, joit ad coditioal DFs are related to each other b the ollowig ormulas ) ( ) ( x ),, ( x) ) d., margializatio robabilit-berli Che

13 Illustrative Examples (/) ( ) Notice that the coditioal DF x has the same shape as the joit DF, ), because the ormalizig actor ( ) does ot deped o x ( x 3.5), 3.5) ( 3.5).5) (.5),.5 ) (.5) ( ) Figure 3.7: Visualizatio o the coditioal DF x. Let, have a joit DF which is uiorm o the set. For each ixed, we cosider the joit DF alog the slice ad ormalize it so that it itegrates to ( x 3.5) ( x 3.5), c. example 3.3 / 4 / 4 / 4 / / / 4 / 4 robabilit-berli Che 3

14 Illustrative Examples (/) Example 3.5. Circular Uiorm DF. Be throws a dart at a circular target o radius r. We assume that he alwas hits the target, ad that all poits o impact ) are equall likel, so that the joit DF, ) o the radom variables x ad is uiorm What is the margial DF ( ), ), area o the circle 0,, x + r πr 0, otherwise ( ) ), dx x + x + r dx πr πr r, i r πr (Notice here that DF r i ) otherwise πr ( ) is ot uiorm) r r dx is i the circle dx ( x ), ) ( ) πr r πr, i x r For each value, x is uiorm robabilit-berli Che 4 ( ) + r

15 Expectatio o a Fuctio o Radom Variables I ad are joitl cotiuous radom variables, ad g is some uctio, the Z g(, ) is also a radom variable (ca be cotiuous or discrete) The expectatio o Z ca be calculated b E [ Z ] E[ g( )] g ) ( x ),,, dxd Z b I is a liear uctio o ad, e.g., Z a +, the [ Z ] E[ a + b ] ae[ ] be[ ] E + a b Where ad are scalars robabilit-berli Che 5

16 robabilit-berli Che 6 Coditioal Expectatio The properties o ucoditioal expectatio carr though, with the obvious modiicatios, to coditioal expectatio [ ] ( ) dx x x E ( ) [ ] ( ) ( ) dx x x g g E ( ) [ ] ( ) ( ) dx x x g g,, E

17 Total robabilit/expectatio Theorems Total robabilit Theorem For a evet A ad a cotiuous radom variable ( A) ( A ) ( ) d Total Expectatio Theorem For a cotiuous radom variables ad E E E [ ] E[ ] ( ) [ g ( )] E g ( ) d [ ] ( ) [ g (, )] E g (, ) d [ ] ( ) d robabilit-berli Che 7

18 Idepedece Two cotiuous radom variables ad are idepedet i ice that ) ( x) ( ), or all x,, ) ( ) ( x ) ( x) ( x), We thereore have ( x ) ( x) or all x ad all with ( ), > 0 Or ( x) ( ) or all ad all x with ( x), > 0 robabilit-berli Che 8

19 More Factors about Idepedece (/) I two cotiuous radom variables ad are idepedet, the A two evets o the orms A ad B are idepedet { } { } ( A, B) x A B, ( x ) x A B ( x) ( ) x A ( x) dx B ( A)( B), ddx ddx [ ] d [ ] ( ) The coverse statemet is also true (ee the ed-o-chapter problem 8) robabilit-berli Che 9

20 More Factors about Idepedece (/) I two cotiuous radom variables ad are idepedet, the [ ] E[ ] E[ ] E ( + ) var ( ) var ( ) var + The radom variables ad h are idepedet or a uctios g ad h Thereore, g ( ) ( ) [ g( ) h( )] E[ g( )] E[ h( )] E robabilit-berli Che 0

21 Joit CDFs I ad are two (either cotiuous or discrete) radom variables, their joit cumulative distributio uctio (CDF) is deied b F I ad urther have a joit DF, the Ad ) ( x, ),,, ( x ) ( s t) F,,,, ) x F, x ) I ca be dieretiated at the poit dsdt F, ) robabilit-berli Che

22 A Illustrative Example Example 3.0. Veri that i ad are described b a uiorm DF o the uit square, the the joit CDF is give b F ) ( x, ) x, or 0 x,, ( 0,) (, ) ( 0,0) (,0) F, x ), ), or all ) i the uit square robabilit-berli Che

23 Recall: the Discrete Baes Rule Let A, A, K, A be disjoit evets that orm a partitio o the sample space, ad assume that ( A i ) 0, or all i. The, or a evet such that we have B ( B) > 0 ( A B) i ( A ) ( B A ) i ( B) i ( A ) ( ) i B Ai ( ) ( ) k Ak B Ak ( A ) ( ) i B Ai ( A ) ( B A ) + L + ( A ) ( B A ) Multiplicatio rule Total probabilit theorem robabilit-berli Che 3

24 Ierece ad the Cotiuous Baes Rule (/) As we have a model o a uderlig but uobserved pheomeo, represeted b a radom variable with DF, ad we make a ois measuremet, which is modeled i terms o a coditioal DF. Oce the experimetal value o is measured, what iormatio does this provide o the ukow value o? ( x) Measuremet Ierece ( ) x ( x ) ( x ), ) ( ) ( x) ( ) x () t ( t) dt Note that, robabilit-berli Che 4

25 robabilit-berli Che 5 Ierece ad the Cotiuous Baes Rule (/) I the uobserved pheomeo is iheretl discrete Let is a discrete radom variable o the orm that represets the dieret discrete probabilities or the uobserved pheomeo o iterest, ad be the MF o N { } N N p N ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) () ( ) i N N N N N N i i p p p N N N N δ δ δ δ δ Total probabilit theorem

26 Illustrative Examples (/) Example 3.8. A lightbulb produced b the Geeral Illumiatio Compa is kow to have a expoetiall distributed lietime. However, the compa has bee experiecig qualit cotrol problems. O a give da, the parameter Λ λ o the DF o is actuall a radom variable, uiorml distributed i the iterval [, 3 / ]. I we test a lightbulb ad record its lietime ( ), what ca we sa about the uderlig parameter λ? Λ Λ Λ λ ( λ ) λe, 0, λ > 0 ( λ ), or λ 3 / 0, otherwise ( λ ) 3 / Λ ( λ ) Λ ( λ ) () t ( t) Λ Λ dt Coditioed o Λ λ, has a expoetial distributio with parameter λ 3 / λe te λ t, or λ 3/ dt robabilit-berli Che 6

27 robabilit-berli Che 7 Illustrative Examples (/) Example 3.9. igal Detectio. A biar sigal is trasmitted, ad we are give that ad. The received sigal is, where ormal oise with zero mea ad uit variace, idepedet o. What is the probabilit that, as a uctio o the observed value o? ( ) p ( ) p N + N ( ) ( ) - - s e s s ad, ad or, / π σ Coditioed o, has a ormal distributio with mea ad uit variace s s ( ) ( ) ( ) ( ) () ( ) () ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) e p pe pe e p e pe e pe e e p e p e p p p p p / / / / / / π π π

28 Recitatio ECTION 3.4 Coditioig o a Evet roblems 4, 7, 8 ECTION 3.5 Multiple Cotiuous Radom Variables roblems 9, 4, 5, 6, 8 robabilit-berli Che 8

Quick Review of Probability

Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig