ST2004 Week 8 Probability Distributions

Transcription

1 ST004 Week 8 Probability Distributions Mathematical models for distributions The language of distributions Expected Values and Variances Law of Large Numbers ST004 0 Week 8

2 Probability Distributions Random Variable Y a name Two lists: (i) poss values y (ii) ass. probs Tabulated, or Defined by a Formula Discrete or Continuous Expected Value of Y (or a function g( ) of Y) Weighted Avg of poss values y (or g(y)) Summation or Integral ST004 0 Week 8

3 Prob Dists; cdf, pmf, pdf Two lists: possibilities and probabilities Probs can be via cdf cum prob dist functions Pr(Y y) OR if discrete (ie step function cdf) by pmf prob mass functions Pr(Y=y) if continuous, (smooth cdf) by pdf prob density functions ( Pr(Y y) ) ST004 0 Week 8 3

4 Why Math Models? Infinitely long run Thought expts Insight/Reasoning Conditional prob Approximations MC algorithms Transforms Properties Efficiency Approximations Systems neither Uniformly random Indep Methods Counting Equally likely Calculus Continuous Probability Rules Events Conditional prob ST004 0 Week 8 4

5 Random variable Y: Random Variables and Probability Distributions Outcome of an expt transformation of one of more calls to RAND() Often - numeric - pair (or a set ) of numbers Probability Dist: Two lists: possibilities and probabilities Lists and functions can be explicitly tabulated or defined by some mathematical formula ST004 0 Week 8 Distribution? Replicate sims, transform and summarise OR Directly from transform via probability Alt, by approximation 5

6 Discrete Random Variables and Probability Distributions Poss y Pr(Y =y ) Pr(Y y ) Formal math statement for cdf int( y) y 6 6 FY ( y) 0 y y Pr(Y=y). 0.8 Pr(Yy). 0.8 Pr(Yy) ST004 0 Week 8 6

7 Sum of k dice via Recursion Numbers of combinations Sk Density f ( y) 7k 35k e y Adding stuff Adding Indep Stuff Normal distribution Central Limit Theorem ST004 0 Week 8 7

8 Sum k Dice To simulate S with one cl alto RAND() Exact Transform Recursion Probs p, 7 ; VarS e k ; Var k Sk Y NOR 6k LOOKUP RAND() with cum probs Approximate Transform E S Not E S k 35 from sim or pref theory 7 35k M. S. INV RAND() S ROUND 7k 35k Y k k p To compute: eg Probs, Long run avg, based on S k Skip simulation stage if poss Adding stuff Similar Electoral College votes for Obama Hard-drives available for Google Class Pension Age, given 0+ Random Walk Convergence rate LLN (avgs) ST004 0 Week 8 8

9 Mean k Dice: Density Plot k= probx k=0 probx Scaling Area = e x 7 35 k Averaging stuff Mean of 0 less variable than mean of St Dev of Sampling Dist of Means /k Normal distribution, CLT Density plot Scaled/ Smoothed Rel Freqs Theory later ST004 0 Week 8 9

10 Density Estimate Freq table: bins x 0,, x i, x & corresponding relative frequencies th k x k x k Start Start bin corresp to ( ), ( ) Rel Freq Start Start End Smoothed Rel Fr estimate of eq pdf f x ST004 0 Week 8 0

11 Cumulative Probs Simulation Direct k calls to RAND, k LOOKUPs Via exact cdf; call to RAND, LOOKUP Via app cdf + NORM.S.INV(RAND()) Cts Normal Approx ; Pr 7k 35k S k s x, variable of integration Alt : y, v, t, z,... xs e x dx NORM.S.DIST s, TRUE Theory later Inverting x y = norm.s.dist norm.s.inv Simpler but equivalent case x y=x^3 y^(/3) ST004 0 Week 8

12 Max k Dice Cum probs Pr(Mk i) 3 4 i i Pr(Mk=i) E Mk E (Mk)^ Var SD Event Identity M i All k scores i k k Hence prob calculation Pr Pr k i M i k 6 k i i M i 6 6 k Not Normal dist > rolls before "score i" k Closely related to Counting stuff Time to ST004 0 Week 8

13 Max k Dice As k, E M 6, SD M 0 k Why? Dice score constrained 6 Not Law of Large Numbers k ST004 0 Week 8 3

14 cdf If Y is numeric and scalar Most general form cum prob dist function, cdf Poss values y eg y =,, K, eg 0 y, eg y > 0 Probs Pr(Yy)=F(y) eg Y = Dice score F(y)= y/6; poss Y are y =,..6 (equiv S = Dice score F(s)= s/6; poss S are s =,..6) Pr(Yy) eg Y = RAND() F Y (y)= y; poss Y are 0 y X = 4Y + 0 F X (x)= 0.5(x-0) poss X are 0 x 4 ST004 0 Week 8 4

15 cdf and prob mass function, pmf Pr y Y y Pr Y y Pr Y y F y F y Y Dice Score Pr Y 5 PrY 5 PrY Pr Y Pr Y Pr Y Pr Y 6 p ( y) Prob mass function for Y Y Lists Pr( Y y) p ( y)for each poss value y of Y or eg ( ) Prob mass function for S S X p s S. x, s index for X, S; could equally use i, j... If no ambiguity, use simpler notation p( x), p( s) X Pr(Yy) pmf: risers ST004 0 Week 8 5

16 Linear Functions and Expected Values Special Case S g( Y ) ay b; ( a, b) constants ( )Pr Pr E S g y Y y ay b Y y all y all y Pr a y Pr Y y b Y y all y ae Y b all y S g( Y ) ay b ( a, b) constants Pr Var S ay b E ay b Y y all y Pr all y a Var Y [ ] Pr ay E ay Y y a Y E Y Y y all y Could use, instead of Pr( Y y), Pr( Y i), p ( y), p ( i), or even p( y) ST004 0 Week 8 6 Y Y

17 Linear Functions and Expected Values Special Case S g( Y ) ay b g( y)pry y ay bpry y ES all y all y Pr a y Pr Y y b Y y all y ae Y b all y X score single dice E[ X ] E[X 3] 0 7 Var[ X ] Var[X 3] o o C Temp C, F Temp F EF 9 5 SDF 9 E C SD C S g( Y ) ay b ( a, b) constants Var S ay b E ay b PrY y all y ay EaY PrY y a Y E all y a Var [ Y ] SD[ as] a SD[ S] Y PrY y ST004 0 Week 8 7 all y

18 Variance: Useful Formula S Y EY (const); py PrY y ES Special Case; ; Var Y all y all y y y p y y p... EY Compare y y py all y Sample Var = Sample Avg y - Sample Avg y y Confirm for S 3, M 3 Exp Val & Var Sample Avg & Var ST004 0 Week 8 8

19 Law of Large Numbers, Sample Size Note 7k 35k k ; VarS E S k Linear Function E S ; Var S ; SD S k k k k k k k k Law of Large Numbers,Convergence n n 7 35 E X ; SD X i i n n n Avg of n const at rate n ST004 0 Week 8 9

20 cdf F X (x) ;X=cts transf(rand()) Some simple cases t( ) non-decreasing function X t Y Y X Y X Y) eg ( ) 4 0; ; ln( A B C F Pr X Pr 4Y 0 Pr Y A A F x Pr X x Pr Y x 0 x 0 A A 4 4 F x Pr X x Y x Y x x 0 x B B F x Pr X x ln Y x Y e x ex; x0 C C 0x 4 ; ; ST004 0 Week 8 0

21 Prob density function, pdf Some simple cdf/pdf Pr X Pr X Pr X A A A df x Pr x X x B d x ; B f x dx dx x B f x ; 0 x B df Pr x X x C C dx x x d ex C x; 0 f x e x e x dx f C x Cts: cdf is smooth pdf is slope cdf is area under pdf ST004 0 Week 8

22 Continuous Probability Distributions Defined by math functions cdf F Y (y)= Pr(Y y) F(y) OK if not ambiguous continuous increasing pdf f(y) = rate of change of F(y) F(y) = indefinite integral of f(y) Prob is area under f(y) ST004 0 Week 8

23 0 Uniform Probability Dist F(y) Y ~ U(0,) pdf f ( y) probability density function rateof changeof cdf F( y) cdf F( y) areaunder pdf toleft of y Y equally likely to have any value in (0,) cdf eg Pr( Y 0.6) y 0 general F( y) Pr( Y y) y 0 y y 0 y 0 df( y) pdf f ( y) 0 y dy 0 y ST004 0 Week 8 3

24 F(y) Height 0.6 f(y) Uniform(0,) Probability Dist Area Slope Y ~ U(0,) Y equally likely to have any value in (0,) Flat Symmetric cdf 0 y 0 F( y) Pr( Y y) y 0 y y pdf 0 y 0 df( y) f ( y) 0 y dy 0 y ST004 0 Week 8 4

25 Generating Y~U(a,b) U = RAND() To generate Y uniform in (-, +3) U RAND() Y - 4U Y - 4 U g( U ) U ( Y ) g ( U ) Pr( Y y) Pr U ( y ) ( y ) 4 4 cdf F( y) ( y ) y 3 d pdf f ( y) ( y ) 4 dy y 3 ST004 0 Week 8 5

26 F(y) Ht = 0.86 Exponential Prob Dist Y ~ Exp( para ) Y 0 likely to be near 0 pdf f ( y) e y cdf F( y) e y f(y) 0 Area = 0.86 Skew Asymmetric Show by transform Y ln RAND() 0 ST004 0 Week 8 6

27 Normal Probability Dist Y ~N(0,) F(y)=Pr(Yy) Ht 0.84=Pr(Y) f(y) Bell-shaped Symmetric Area 0.84 Y ~ N(0,) Y likely to be "near" 0 pdf f ( y) e ST004 0 Week cdf F( y) Pr( Y y) f ( u) du transform Y y y NORM. S. DIST y, TRUE NORM. S. INV RAND() y = norm.s.dist norm.s.inv x

28 Simulating Cts Random Variables Defined System explicitly defined output var Y implicitly defined dist for output, eg F(y) RAND() t(rand()) System Output Defined Dist F(y) seek transformation Y = t(rand()) easy if know (code for) F - (y) See Tijms, 3 rd ed Sec 0.5 can be challenging esp if y is multivariate ST004 0 Week 8 8

29 Expected Vals for Cts Random Vars E[ Y ] y Pr Y y E[ Y ] y PrY y all y Pr( Y y) Pr Y in( y, y dy) f ( y) dy; f ( y) E[ Y ] yf ( y) dy all y df( y) dy Discrete case Continuous case As previously E[ ay b] ae[ Y ] b Var ay b a Var Y [ ] [ ] Var Y E Y E Y [ ] [ ] [ ] ST004 0 Week 8 9

30 Expected Vals for Cts Random Vars Var E X X ~ Uniform(0,); pdf f ( x) ; x in (0,) E X xf ( x) dx xdx 3X k E X Ee X EX ; k ? x f ( x) dx Continuous case Long Run Avg Var X=RAND() 3X+ X^ X^3 exp(x) ST004 0 Week

31 Expected Vals for Cts Random Vars x x x x X x X ~ Exp; pdf f ( x) e ; x Continuous case E X xf ( x) dx xe dx x e dx -xe e dx E X Var X Y x x e dx Y EY [ ] Var Y pdf f ( y) 3X X Long Run Average Variance rand() Y= -LN(- RAND()) 3Y Y= - LN(RAND()) ST004 0 Week

32 Examples / Homework Simulation AND Thought Expt Discrete State prob dist cdf/pmf: avg of scores dice 3 + min(scores dice) # 6 s when roll 3 dice # rolls of single die before first 6 ST004 0 Week 8 3

33 Examples / Homework Simulation AND Thought Expt Continuous Y = RAND() ~ U(0,) Give cdf and pdf, and sketch both X = 3Y + 4 X = 3Y + 4 X = e Y X = 3e Y + 4 Also Simulate and form/plot ecdf by using ranks ST004 0 Week 8 33

34 Examples / Homework Simulation AND Thought Expt The Rayleigh density and cdf are as below. Sketch. Use the sketches to show how these functions can be used to compute probabilities. Propose a transform of Y = RAND(). Use calculus to determine its expected value, and confirm by simulation. pdf f r re F r e r ( ) r r ; ( ) ; 0 This Rayleigh model arises as distance from A to B when the North/South and East/West distances are each indep Normally distributed ie NORM.S.INV(RAND()). This suggests an alternative method for generating values from this distribution. Confirm. ST004 0 Week 8 34

35 Additional Material ST004 0 Week 8 35

36 Aside: Inverting transform X=t(RAND()) t( ) non-decreasing eg X t( Y) 4Y 0; X Y ; X ln( Y) A B C X x Y y where t( y) x X 4Y 0 Y 0.5 A X x Y x Y x B B X x ln Y x Y ex ST004 0 Week 8 36

37 #Rolls before score > i Infinitely Long Run E[K] E[K^] Var[K] Prob > k Roll until score Prob k Roll until score rolls > > >5 rolls > > >5 0 k k Geometric Dist Poss values k,.3.. Probs Pr( K k) p( p) E[ K] k Pr( K k) p k k ST004 0 Week 8 37

38 #Rolls before score > 5; cts approx Exp approx Prob > k 5 Prob > k 5 rolls rolls 5 before >5 before >5 exact app k k Exponential Approx e k k k Pr K k e e ST004 0 Week 8 38

39 #Rolls before score > 5; cts approx cdf Pr K k e k 6 Generate U RAND() Solve U e K 6 3 Return K 6 LN( U ) inv cdf 4 Here Return ROUND.DOWN(K) Inverting Simulation Exact Approx Form exact cdf Call RAND(); LOOKUP via log transform of RAND() x y = - exp(-x) ``-LN(-y) But Stat Properties U U Simpler Stat Prop's of Return K 6 LN( U ) ST004 0 Week 8 39 U

40 Alt: Using ranks instead of bins Rep 0 Summarising System Life without Bins System Life prop <= this ascending To join the dots in EXCEL, data must be sorted System ascend-proing <= Life this rank value rank value rank/n ecdf.xlxs Continuous case ST004 0 Week 8 40

41 Empirical cdf Summarising System Life without Bins n=0000 System ecdf = Rep Life rank/n ecdf=prop( <= this value) lines and thus sorting not needed when n is large ecdf=prop( <= this value) Continuous case ST004 0 Week 8 4

42 Law of Large Numbers; counter-example running X inv X avg /X running avg /X 0 E X x dx ln( x), ill defined. 0 Alt, the dist of X E[X - ]; X~U(0,) is such that it can generate very large values with non-trivial probability. The Law of Large Numbers does not always apply ST004 0 Week 8 4