V-0. Review of Probability

Transcription

1 V-0. Review of Probabilit

2 Random Variable! Definition Numerical characterization of outcome of a random event!eamles Number on a rolled die or dice Temerature at secified time of da 3 Stock Market at close 4 Height of wheel going over a rock road

3 Random Variable! Non-eamles Heads or Tails on coin Red or Black ball from urn But we can make these into RV s! Basic Idea don t know how to comletel determine what value will occur Can onl secif robabilities of RV values occurring. 3

4 Two tes of Random Variables Random Variable Discrete RV Die Stocks Continuous RV Temerature Wheel height 4

5 Given CRV, What is the robabilit that 0? Oddit : P 0 0 Otherwise the Prob. Sums to infinit Need to think of Prob. Densit Function PDF The Probabilit densit function of RV o o + P 0 < < 0 + area shown d 5

6 Most Commonl Used PDF: Gaussian PDF e σ π m / σ A RV with this df is called a Gaussian RV m & σ are arameters describing one of the man Gaussian df, where m mean of RV σ std. Deviation of RV Note: σ > 0 σ Variance of RV 6

7 Three views of Guassian PDF s σ σ Area i.e. 68.3% of area is within σ of mean m Small σ Small variabilit/uncertaint Large σ Large variabilit/uncertaint 7

8 Wh Is Gaussian Used? "Central Limit theorem CLT The sum of N indeendent RV s has a df that tends to be Gaussian as N "So What! Here is what : Electronic sstems generate internal noise due to random motion of atoms in electronic comonents. The noise is the result of summing the random effects of lots of atoms. CLT alies Guassian Noise 8

9 " Generall: take the noise to be Zero Mean σ e π σ 9

10 Joint PDF of and Y: Y, Describes robabilities of joint events concerning and Y. For eamle, the robabilit that lies in interval [a,b] and Y lies in interval [a,b] is given b: Pr b d { < < b and c < Y < d } a, dd a c Y This grah shows the Joint PDF Grah from B. P. Lathi s book: Modern Digital & Analog Communication Sstems 0

11 Conditional PDF When ou have two RVs it is often necessar to ask questions like: What is the PDF of Y if is constrained to take on a secific value. In other words: What is the PDF of Y conditioned on the fact is constrained to take on a secific value. As an eamle consider the husband/wife salaries above: What is the PDF of the husband salar conditioned on the wife salar is $00K? So ou first find all wives who make EACTLY $00K and look at how that set of husband salaries are distributed. Clearl the result deends on the joint PDF since that catures all the robabilistic details of how and Y interact and clearl it should onl deend on the slice of the joint PDF at the value of Y$00K. Now we have to adjust this to account for the fact that the joint PDF even its slice reflects how likel it is that $00K will occur e.g., if is unlikel then Y 00000, will be small; so if we divide b we adjust for this.

12 Conditional PDF cont. Thus, the conditional PDFs are defined as slice and normalize : Y 0, Y,, 0 otherwise Y 0, Y Y,, Y 0 otherwise is held fied is held fied slice and normalize is held fied This grah shows the Conditional PDF Grah from B. P. Lathi s book: Modern Digital & Analog Communication Sstems

13 3 Indeendent RV s Indeendence should be thought of as saing that: neither RV imacts the other statisticall thus, the values that one will likel take should be irrelevant to the value that the other has taken. In other words: conditioning doesn t change the PDF!!!,, Y Y Y Y Y Y

14 Eamle: Indeendent Gaussian RVs Contours,. Indeendent zero mean If & Y are indeendent, then the contour ellises are aligned with either the or ais Indeendent non-zero mean Different slices give same normalized curves Deendent Different slices give different normalized curves 4

15 5 An Indeendent RV Result RV s & Y are indeendent if:, Y Y, Y Y Y Y Here s wh:

16 Characterizing RVs! PDFs tell everthing about RVs but sometimes the are more than we need/know! So we make due with a few Characteristics Mean of an RV Describes the centroid of PDF Variance of an RV Describes the sread of PDF Correlation of RVs Describes tilt of joint PDF 6

17 Mean of RV Mean Average Eected Value Call it E{} Motivation First w/ Data Analsis View Consider RV Score on a test Data:,, N Possible values of : V 0 V V... V Test Average N 00 i i NV + NV +... NnV00 N N i V i N N i N i # of scores of value i n N N i Total # of scores i PV i This is called Data Analsis or Emirical View Statistics 7

18 Theoretical View of Mean Data Analsis View leads to Probabilit Theor: " For Discrete random Variables : Ε{ } n n " This Motivates form for Continuous RV: i P Ε{ } d PDF i Probabilit Notation: Ε { } 8

19 Aside: Probabilit vs. Statistics Probabilit Theor» Given a PDF Model» Predict how the data will behave Statistics» Given a set of data» Determine how the data did behave Ε{ } d PDF Dumm Variable There is no DATA here!!! The PDF models how data will behave Law of Large Numbers Avg N n i i Data There is no PDF here!!! The Statistic measures how the data did behave 9

20 Variance of RV There are similar Data vs. Theor Views here Let s go to the theor Variance measures etent of Deviation Around the Mean Variance: σ E{ m } m d Note : If zero mean σ E{ } d 0

21 Correlation Between RV s Motivation First w/ Data Analsis View Consider a random eeriment with two outcomes RVs and Y of height and weight resectivel Positivel Correlated m m

22 Three main Categories of Correlation Y Y Y Y Y Y Positive correlation Best Friends Height & Weight Zero Correlation i.e. uncorrelated Comlete Strangers Height & $ in Pocket Negative Correlation Worst Enemies Student Loans & Parents Salar

23 Now the Theor To cature this, define Covariance : σ Y E{ Y Y } If the RVs are both Zero-mean : If Y: σ σ σ Y σ Y Ε{Y} If & Y are indeendent, then: σ Y 0 Y 3

24 If E{ Y Y } 0 σ Y Sa that and Y are uncorrelated If E{ Y Y } 0 σ Y Then E { Y} Y Called Correlation of &Y 4

25 Indeendence vs. Uncorrelated & Y are Indeendent f Y, f f Y PDFs Searate Imlies Uncorrelated Indeendence & Y are Uncorrelated E{ Y} E{ } E{ Y} Means Searate INDEPENDENCE IS A STRONGER CONDITION!!!! 5

26 Confusing Terminolog Covariance : σ Y E{ Y Y } Correlation : E{Y} Same if zero mean Correlation Coefficient : ρ Y σ σ Y σ Y ρ Y 6

27 7 Correlation Matri : { } { } { } { } { } { } { } { } { } N N N N N N T E E E E E E E E E E! " # " "!! } { R For Random Vectors T N ] [! Covariance Matri : } { T E C