Eperiments, Outcomes, Events nd Rndom Vriles: A Revisit Berlin Chen Deprtment o Computer Science & Inormtion Engineering Ntionl Tiwn Norml University Reerence: - D. P. Bertseks, J. N. Tsitsiklis, Introduction to Proility
Eperiments, Outcomes nd Event An eperiment Produces ectly one out o severl possile outcomes The set o ll possile outcomes is clled the smple spce o the eperiment, denoted y A suset o the smple spce ( collection o possile outcomes) is clled n event Emples o the eperiment A single toss o coin (inite outcomes) Three tosses o two dice (inite outcomes) An ininite sequences o tosses o coin (ininite outcomes) Throwing drt on squre (ininite outcomes), etc. Proility-Berlin Chen
Proilistic Models A proilistic model is mthemticl description o n uncertinty sitution or n eperiment Elements o proilistic model The smple spce The set o ll possile outcomes o n eperiment The proility lw Assign to set A o possile outcomes (lso clled n event) nonnegtive numer P ( A )(clled the proility o A) tht encodes our knowledge or elie out the collective likelihood o the elements o A Proility-Berlin Chen 3
Three Proility Aioms Nonnegtivity 0 P A, or every event A Additivity I A nd B re two disjoint events, then the proility o their union stisies ( A U B ) P ( A ) P ( B ) P + Normliztion The proility o the entire smple spce Ω is equl to 1, tht is, P Ω 1 Proility-Berlin Chen 4
Rndom Vriles Given n eperiment nd the corresponding set o possile outcomes (the smple spce), rndom vrile ssocites prticulr numer with ech outcome This numer is reerred to s the (numericl) vlue o the rndom vrile We cn sy rndom vrile is rel-vlued unction o the eperimentl outcome Proility-Berlin Chen 5
Discrete/Continuous Rndom Vriles (1/) A rndom vrile is clled discrete i its rnge (the set o vlues tht it cn tke) is inite or t most countly ininite inite : { 1,, 3, 4}, countly ininite : { 1,,L} A rndom vrile is clled continuous (not discrete) i its rnge (the set o vlues tht it cn tke) is uncountly ininite E.g., the eperiment o choosing point rom the intervl [1, 1] A rndom vrile tht ssocites the numericl vlue to the outcome is not discrete Proility-Berlin Chen 6
Discrete/Continuous Rndom Vriles (/) A discrete rndom vrile hs n ssocited proility mss unction (PMF), p, which gives the proility o ech numericl vlue tht the rndom vrile cn tke A continuous rndom vrile cn e descried in terms o nonnegtive unction ( ) ( ( ) 0 ), clled the proility density unction (PDF) o, which stisies P ( B ) ( ) B d or every suset B o the rel line Proility-Berlin Chen 7
Cumultive Distriution Functions (1/4) The cumultive distriution unction (CDF) o rndom vrile is denoted y F nd provides the proility P F P( ) k p ( k ) () t i is discrete dt, i is continuous The CDF F ccumultes proility up to The CDF F provides uniied wy to descrie ll kinds o rndom vriles mthemticlly, Proility-Berlin Chen 8
Cumultive Distriution Functions (/4) The CDF is monotoniclly non-decresing F i i, then j F i F j The CDF tends to 0 s, nd to 1 s F I is discrete, then F is piecewise constnt unction o Proility-Berlin Chen 9
Proility-Berlin Chen 10 Cumultive Distriution Functions (3/4) I is continuous, then is continuous unction o F c c d c or c 1, dt t dt t F or, 1 dt dt t F 1
Cumultive Distriution Functions (4/4) I is discrete nd tkes integer vlues, the PMF nd the CDF cn e otined rom ech other y summing or dierencing F p ( k ) P( k ) p ( i) k i ( k ) P( k ) P( k 1) F ( k ) F ( k 1) I is continuous, the PDF nd the CDF cn e otined rom ech other y integrtion or dierentition F p P( ) ( t ) df d The second equlity is vlid or those or which the CDF hs derivtive (e.g., the piecewise constnt rndom vrile), dt, Proility-Berlin Chen 11
Conditioning Let nd e two rndom vriles ssocited with the sme eperiment I nd re discrete, the conditionl PMF o is deined s ( where ) p p ( y) ( y) P( y) P ( y) P( y) p,, p (, y) ( y) I nd re continuous, the conditionl PDF o is deined s ( where y > ) ( y), 0 (, y) ( y) Proility-Berlin Chen 1
Independence Two rndom vriles nd re independent i p (, y) p p ( y), or ll y,, (I nd re discrete) (, y) ( y), or ll,y, (I nd re continuous) I two rndom vriles nd re independent p ( y) p, or ll y, (I nd re discrete) ( y), or ll,y (I nd re continuous) Proility-Berlin Chen 13
Epecttion nd Moments The epecttion o rndom vrile is deined y or E E [ ] p [ ] ( ) The n-th moment o rndom vrile is the n epected vlue o rndom vrile (or the rndom vrile n n E p d [ ] (I is discrete) (I is continuous) (I is discrete) or E [ n ] n ( ) d (I is continuous) The 1st moment o rndom vrile is just its men Proility-Berlin Chen 14
Vrince The vrince o rndom vrile is the epected vlue o rndom vrile ( E) vr [ ] [ ] [ ] ( [ ] E E ) E ( E ) The stndrd derivtion is nother mesure o dispersion, which is deined s ( squre root o vrince) σ vr ( ) Esier to interpret, ecuse it hs the sme units s Proility-Berlin Chen 15
More Fctors out Men nd Vrince Let e rndom vrile nd let E vr [ ] E[ ] + vr ( ) + I nd re independent rndom vriles [ ] E[ ] E[ ] E ( + ) vr ( ) vr ( ) vr + [ g( ) h( )] E[ g( )] E[ h( )] E nd re unctions g h o nd,respectively Proility-Berlin Chen 16