Review of Elementary Probability Lecture I Hamid R. Rabiee

Transcription

1 Stochastic Processes Review o Elementar Probabilit Lecture I Hamid R. Rabiee

2 Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important Theorems

3 Histor & Philosoph Started b gamblers dispute Probabilit as a game analzer! Formulated b B. Pascal and P. Fermet First Problem (654) : Double Si during 24 throws First Book (657) : Christian Hugens, De Ratiociniis in Ludo Aleae, In German, 657.

4 Histor & Philosoph (Cont d) Rapid development during 8 th Centur Major Contributions: J. Bernoulli ( ) A. De Moivre ( )

5 Histor & Philosoph (Cont d) A renaissance: Generalizing the concepts rom mathematical analsis o games to analzing scientiic and practical problems: P. Laplace ( ) New approach irst book: P. Laplace, Théorie Analtique des Probabilités, In France, 82.

6 Histor & Philosoph (Cont d) 9 th centur s developments: Theor o errors Actuarial mathematics Statistical mechanics Other giants in the ield: Chebshev, Markov and Kolmogorov

7 Histor & Philosoph (Cont d) Modern theor o probabilit (20 th ) : A. Kolmogorov : Aiomatic approach First modern book: A. Kolmogorov, Foundations o Probabilit Theor, Chelsea, New ork, 950 Nowadas, Probabilit theor as a part o a theor called Measure theor!

8 Histor & Philosoph (Cont d) Two major philosophies: Frequentist Philosoph Observation is enough Baesian Philosoph Observation is NOT enough Prior knowledge is essential Both are useul

9 Histor & Philosoph (Cont d) Frequentist philosoph There eist ied parameters like mean,. There is an underling distribution rom which samples are drawn Likelihood unctions(l()) maimize parameter/data For Gaussian distribution the L() or the mean happens to be /N i i or the average. Baesian philosoph Parameters are variable Variation o the parameter deined b the prior probabilit This is combined with sample data p(/) to update the posterior distribution p(/). Mean o the posterior, p(/),can be considered a point estimate o.

10 Histor & Philosoph (Cont d) An Eample: A coin is tossed 000 times, ielding 800 heads and 200 tails. Let p = P(heads) be the bias o the coin. What is p? Baesian Analsis Our prior knowledge (belie) : p (Uniorm(0,)) Our posterior knowledge : pobservation p 800 p Frequentist Analsis Answer is an estimator such that 8 Mean : E pˆ 0. Conidence Interval : P pˆ pˆ

11 Histor & Philosoph (Cont d) Further reading: hist/stathist.htm stat/histor/indehistor.shtml istprob.pd

12 Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important Theorems

13 Random Variables Probabilit Space A triple o represents a nonempt set, whose elements are sometimes known as outcomes or states o nature represents a set, whose elements are called events. The events are subsets o. should be a Borel Field. F P, F, P represents the probabilit measure. F Fact: P

14 Random Variables (Cont d) Random variable is a unction ( mapping ) rom a set o possible outcomes o the eperiment to an interval o real (comple) numbers. In other words : Outcomes F P : F I : I R r Real Line

15 Random Variables (Cont d) Eample I : Mapping aces o a dice to the irst si natural numbers. Eample II : Mapping height o a man to the real interval (0,3] (meter or something else). Eample III : Mapping success in an eam to the discrete interval [0,20] b quantum 0..

16 Random Variables (Cont d) Random Variables Discrete Dice, Coin, Grade o a course, etc. Continuous Temperature, Humidit, Length, etc. Random Variables Real Comple

17 Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important Theorems

18 Densit/Distribution Functions Probabilit Mass Function (PMF) Discrete random variables Summation o impulses The magnitude o each impulse represents the probabilit o occurrence o the outcome P Eample I: Rolling a air dice PMF 6 6 i i

19 Densit/Distribution Functions (Cont d) Eample II: Summation o two air dices P Note : Summation o all probabilities should be equal to ONE. (Wh?)

20 Densit/Distribution Functions (Cont d) Probabilit Densit Function (PDF) Continuous random variables The probabilit o occurrence o will be P d. 0 d, 2 d 2 P P

21 Densit/Distribution Functions (Cont d) Some amous masses and densities Uniorm Densit P a. U end Ubegin a Gaussian (Normal) Densit. e 2 2. N 2 2,. 2 P a

22 Densit/Distribution Functions (Cont d) Binomial Densit Poisson Densit! : Note e n n p N p n N n.. 0 N.p n n N Important Fact:!... :. n p N e p p n N For Suicientl large N n p N n n N

23 Densit/Distribution Functions (Cont d) Cauch Densit P 2 2 Weibull Densit k k e k

24 Densit/Distribution Functions (Cont d) Eponential Densit Raleigh Densit e U e e

25 Densit/Distribution Functions (Cont d) Epected Value The most likelihood value Linear Operator Function o a random variable Epectation E E E. d a. b a. E b g g. d

26 Densit/Distribution Functions (Cont d) PDF o a unction o random variables Assume RV such that g The inverse equation g ma have more than one solution called,...,, 2 PDF o can be obtained rom PDF o as ollows n n i absolute value( g ) i d d i

27 Densit/Distribution Functions (Cont d) Cumulative Distribution Function (CDF) Both Continuous and Discrete Could be deined as the integration o PDF PDF CDF F F P. d CDF()

28 Densit/Distribution Functions (Cont d) Some CDF properties Non-decreasing Right Continuous F(-ininit) = 0 F(ininit) =

29 Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important Theorems

30 Joint/Conditional Distributions Joint Probabilit Functions Densit Distribution Eample I In a rolling air dice eperiment represent the outcome as a 3-bit digital number z... 0 ; 6 0 ; 3 0; 3 0 0; 6,, O W z dd and P F,,,,

31 Joint/Conditional Distributions (Cont d) Eample II Two normal random variables What is r? Independent Events (Strong Aiom) r r e r , ,.,,

32 Joint/Conditional Distributions (Cont d) Obtaining one variable densit unctions,,,, Distribution unctions can be obtained just rom the densit unctions. (How?) d d

33 Joint/Conditional Distributions (Cont d) Conditional Densit Function Probabilit o occurrence o an event i another event is observed (we know what is). Baes Rule,, d..

34 Joint/Conditional Distributions (Cont d) Eample I Rolling a air dice : the outcome is an even number : the outcome is a prime number Eample II Joint normal (Gaussian) random variables 3 2 6, P P P r r e r

35 Joint/Conditional Distributions (Cont d) Conditional Distribution Function Note that is a constant during the integration. dt t dt t d while P F,,,,

36 Joint/Conditional Distributions (Cont d) Independent Random Variables Remember! Independenc is NOT heuristic..,,

37 Joint/Conditional Distributions (Cont d) PDF o a unctions o joint random variables Assume that ( U, V) g, The inverse equation set (, ) g U, V has a set o solutions,, 2, 2,..., n, n Deine Jacobean matri as ollows The joint PDF will be U, V u, v n,, J. i, i i i U U absolute determinant J, i V V

38 Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important Theorems

39 Correlation Knowing about a random variable, how much inormation will we gain about the other random variable? Shows linear similarit More ormal:, E Crr. Covariance is normalized correlation. E.. Cov(, ) E

40 Correlation (cont d) Variance Covariance o a random variable with itsel Var 2 E Relation between correlation and covariance Standard Deviation Square root o variance E

41 Correlation (cont d) Moments n th order moment o a random variable is the epected value o n M E n n Normalized orm M n E n Mean is irst moment Variance is second moment added b the square o the mean

42 Outline Histor/Philosoph Random Variables Densit/Distribution Functions Joint/Conditional Distributions Correlation Important Theorems

43 Important Theorems Central limit theorem Suppose i.i.d. (Independent Identicall Distributed) RVs k with inite variances Let S n n i a n. n PDF o S n converges to a normal distribution as n increases, regardless to the densit o RVs. Eception : Cauch Distribution (Wh?)

44 Important Theorems (cont d) Law o Large Numbers (Weak) For i.i.d. RVs k 0 Pr lim 0 n i i n n

45 Important Theorems (cont d) Law o Large Numbers (Strong) For i.i.d. RVs k Wh this deinition is stronger than beore? lim Pr n i i n n

46 Important Theorems (cont d) Chebshev s Inequalit Let be a nonnegative RV Let c be a positive number Another orm: Pr Pr c c E 2 2 It could be rewritten or negative RVs. (How?)

47 Important Theorems (cont d) Schwarz Inequalit For two RVs and with inite second moments E E.E Equalit holds in case o linear dependenc.

48 Acknowledgement Thanks to Mr. Jalali or preparing slides

49 Net Lecture Elements o Stochastic Processes