Introduction to Computational Finance and Financial Econometrics Probability Review - Part 2
|
|
- Spencer Whitehead
- 6 years ago
- Views:
Transcription
1 You can t see this text! Introduction to Computational Finance and Financial Econometrics Probability Review - Part 2 Eric Zivot Spring 2015 Eric Zivot (Copyright 2015) Probability Review - Part 2 1 / 46
2 Outline 1 Bivariate Probability Distribution - Discrete rv s Marginal distributions Conditional distributions 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Eric Zivot (Copyright 2015) Probability Review - Part 2 2 / 46
3 Example Example: Two discrete rv s X and Y Bivariate pdf Y % 0 1 Pr(X) 0 1/8 0 1/8 X 1 2/8 1/8 3/8 2 1/8 2/8 3/ /8 1/8 Pr(Y ) 4/8 4/8 1 p(x, y) = Pr(X = x, Y = y) = values in table e.g., p(0, 0) = Pr(X = 0, Y = 0) = 1/8 Eric Zivot (Copyright 2015) Probability Review - Part 2 3 / 46
4 Properties of joint pdf p(x, y) S XY = {(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1), (3, 0), (3, 1)} p(x, y) 0 for x, y S XY p(x, y) = 1 x,y S XY Eric Zivot (Copyright 2015) Probability Review - Part 2 4 / 46
5 Outline 1 Bivariate Probability Distribution - Discrete rv s Marginal distributions Conditional distributions 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Eric Zivot (Copyright 2015) Probability Review - Part 2 5 / 46
6 Marginal pdfs p(x) = Pr(X = x) = p(x, y) y S Y = sum over columns in joint table p(y) = Pr(Y = y) = p(x, y) x S X = sum over rows in joint table Eric Zivot (Copyright 2015) Probability Review - Part 2 6 / 46
7 Outline 1 Bivariate Probability Distribution - Discrete rv s Marginal distributions Conditional distributions 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Eric Zivot (Copyright 2015) Probability Review - Part 2 7 / 46
8 Conditional Probability Suppose we know Y = 0. How does this knowledge affect the probability that X = 0, 1, 2, or 3? The answer involves conditional probability. Example: Pr(X = 0 Y = 0) = Pr(X = 0, Y = 0) Pr(Y = 0) = joint probability marginal probability = 1/8 4/8 = 1/4 Remark: Pr(X = 0 Y = 0) = 1/4 Pr(X = 0) = 1/8 = X depends on Y The marginal probability, Pr(X = 0), ignores information about Y. Eric Zivot (Copyright 2015) Probability Review - Part 2 8 / 46
9 Definition - Conditional Probability The conditional pdf of X given Y = y is, for all x S X, p(x y) = Pr(X = x Y = y) = Pr(X = x, Y = y) Pr(Y = y) The conditional pdf of Y given X = x is, for all values of y S Y p(y x) = Pr(Y = y X = x) = Pr(X = x, Y = y) Pr(X = x) Eric Zivot (Copyright 2015) Probability Review - Part 2 9 / 46
10 Conditional Mean and Variance µ X Y =y = E[X Y = y] = x Pr(X = x Y = y), x S X µ Y X=x = E[Y X = x] = y Pr(Y = y X = x). y S Y σx Y 2 =y σy 2 X=x = var(x Y = y) = = var(y X = x) = x S X (x µ X Y =y ) 2 Pr(X = x Y = y), y S Y (y µ Y X=x ) 2 Pr(Y = y X = x). Eric Zivot (Copyright 2015) Probability Review - Part 2 10 / 46
11 Example E[X] = 0 1/ / / /8 = 3/2 E[X Y = 0] = 0 1/ / / = 1, E[X Y = 1] = / / /4 = 2, var(x) = (0 3/2) 2 1/8 + (1 3/2) 2 3/8 +(2 3/2) 2 3/8 + (3 3/2) 2 1/8 = 3/4, var(x Y = 0) = (0 1) 2 1/4 + (1 1) 2 1/2 +(2 1) 2 1/2 + (3 1) 2 0 = 1/2, var(x Y = 1) = (0 2) (1 2) 2 1/4 +(2 2) 2 1/2 + (3 2) 2 1/4 = 1/2. Eric Zivot (Copyright 2015) Probability Review - Part 2 11 / 46
12 Independence Let X and Y be discrete rvs with pdfs p(x), p(y), sample spaces S X, S Y and joint pdf p(x, y). Then X and Y are independent rv s if and only if: p(x, y) = p(x) p(y) for all values of x S X and y S Y Result: If X and Y are independent rv s then, p(x y) = p(x) for all x S X, y S Y p(y x) = p(y) for all x S X, y S Y Intuition: Knowledge of X does not influence probabilities associated with Y Knowledge of Y does not influence probablities associated with X Eric Zivot (Copyright 2015) Probability Review - Part 2 12 / 46
13 Outline 1 Bivariate Probability Distribution - Discrete rv s 2 Bivariate Probability Distribution - Continuous rv s Marginal and conditional distributions 3 Independence 4 Covariance and Correlation Eric Zivot (Copyright 2015) Probability Review - Part 2 13 / 46
14 Bivariate Distributions - Continuous rv s The joint pdf of X and Y is a non-negative function f(x, y) such that: f(x, y)dxdy = 1 Let [x 1, x 2 ] and [y 1, y 2 ] be intervals on the real line. Then, Pr(x 1 X x 2, y 1 Y y 2 ) = x2 y2 x 1 y 1 f(x, y)dxdy = volume under probability surface over the intersection of the intervals [x 1, x 2 ] and [y 1, y 2 ] Eric Zivot (Copyright 2015) Probability Review - Part 2 14 / 46
15 Outline 1 Bivariate Probability Distribution - Discrete rv s 2 Bivariate Probability Distribution - Continuous rv s Marginal and conditional distributions 3 Independence 4 Covariance and Correlation Eric Zivot (Copyright 2015) Probability Review - Part 2 15 / 46
16 Marginal and conditional distributions The marginal pdf of X is found by integrating y out of the joint pdf f(x, y) and the marginal pdf of Y is found by integrating x out of the joint pdf: f(x) = f(y) = f(x, y)dy, f(x, y)dx. The conditional pdf of X given that Y = y, denoted f(x y), is computed as, f(x y) = f(x, y) f(y), and the conditional pdf of Y given that X = x is computed as, f(y x) = f(x, y) f(x). Eric Zivot (Copyright 2015) Probability Review - Part 2 16 / 46
17 Conditional mean and variance The conditional means are computed as: µ X Y =y = E[X Y = y] = x p(x y)dx, µ Y X=x = E[Y X = x] = y p(y x)dy and the conditional variances are computed as, σ 2 X Y =y = var(x Y = y) = (x µ X Y =y ) 2 p(x y)dx, σ 2 Y X=x = var(y X = x) = (y µ Y X=x ) 2 p(y x)dy. Eric Zivot (Copyright 2015) Probability Review - Part 2 17 / 46
18 Outline 1 Bivariate Probability Distribution - Discrete rv s 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Eric Zivot (Copyright 2015) Probability Review - Part 2 18 / 46
19 Independence Let X and Y be continuous random variables. X and Y are independent iff: f(x y) = f(x), for < x, y <, f(y x) = f(y), for < x, y <. Result: Let X and Y be continuous random variables. X and Y are independent iff: f(x, y) = f(x)f(y) The result in the above proposition is extremely useful in practice because it gives us an easy way to compute the joint pdf for two independent random variables: we simple compute the product of the marginal distributions. Eric Zivot (Copyright 2015) Probability Review - Part 2 19 / 46
20 Example Example: Bivariate standard normal distribution Let X N(0, 1), Y N(0, 1) and let X and Y be independent. Then, f(x, y) = f(x)f(y) = 1 2π e 1 2 x2 1 2π e 1 2 y2 = 1 2π e 1 2 (x2 +y 2). To find Pr( 1 < X < 1, 1 < Y < 1) we must solve, π e 1 2 (x2 +y 2) dxdy which, unfortunately, does not have an analytical solution. Numerical approximation methods are required to evaluate the above integral. See R package mvtnorm. Eric Zivot (Copyright 2015) Probability Review - Part 2 20 / 46
21 Independence cont. Result: If the random variables X and Y (discrete or continuous) are independent then the random variables g(x) and h(y ) are independent for any functions g( ) and h( ). For example, if X and Y are independent then X 2 and Y 2 are also independent. Eric Zivot (Copyright 2015) Probability Review - Part 2 21 / 46
22 Outline 1 Bivariate Probability Distribution - Discrete rv s 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Bivariate normal distribution Linear Combination of rv s Eric Zivot (Copyright 2015) Probability Review - Part 2 22 / 46
23 Covariance and Correlation - Measuring linear dependence between two rv s Covariance: Measures direction but not strength of linear relationship between 2 rv s σ XY = E[(X µ X )(Y µ Y )] = = x.y S XY (x µ X )(y µ Y ) p(x, y) (discrete) (x µ X )(y µ Y )f(x, y)dxdy (cts) Eric Zivot (Copyright 2015) Probability Review - Part 2 23 / 46
24 Example Example: For the data in Table 2, we have σ XY = Cov(X, Y ) = (0 3/2)(0 1/2) 1/8 +(0 3/2)(1 1/2) 0 + +(3 3/2)(1 1/2) 1/8 = 1/4 Eric Zivot (Copyright 2015) Probability Review - Part 2 24 / 46
25 Properties of Covariance Cov(X, Y ) = Cov(Y, X) Cov(aX, by ) = a b Cov(X, Y ) = a b σ XY Cov(X, X) = Var(X) X, Y independent = Cov(X, Y ) = 0 Cov(X, Y ) = 0 X and Y are independent Cov(X, Y ) = E[XY ] E[X]E[Y ] Eric Zivot (Copyright 2015) Probability Review - Part 2 25 / 46
26 Correlation Correlation: Measures direction and strength of linear relationship between 2 rv s ρ XY = Cor(X, Y ) = Cov(X, Y ) SD(X) SD(Y ) = σ XY σ X σ Y = scaled covariance Eric Zivot (Copyright 2015) Probability Review - Part 2 26 / 46
27 Example Example: For the Data in Table 2 ρ XY = Cor(X, Y ) = 1/4 (3/4) (1/2) = Eric Zivot (Copyright 2015) Probability Review - Part 2 27 / 46
28 Properties of Correlation 1 ρ XY 1 ρ XY = 1 if Y = ax + b and a > 0 ρ XY = 1 if Y = ax + b and a < 0 ρ XY = 0 if and only if σ XY = 0 ρ XY = 0 X and Y are independent in general ρ XY = 0 = independence if X and Y are normal Eric Zivot (Copyright 2015) Probability Review - Part 2 28 / 46
29 Outline 1 Bivariate Probability Distribution - Discrete rv s 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Bivariate normal distribution Linear Combination of rv s Eric Zivot (Copyright 2015) Probability Review - Part 2 29 / 46
30 Bivariate normal distribution Let X and Y be distributed bivariate normal. The joint pdf is given by: f(x, y) = { exp 1 2(1 ρ 2 ) 1 2πσ X σ Y 1 ρ 2 [ ( x µx σ X ) 2 ( ) ]} + y µy 2 σ Y 2ρ(x µ X )(y µ Y ) σ X σ Y where E[X] = µ X, E[Y ] = µ Y, SD(X) = σ X, SD(Y ) = σ Y, and ρ = cor(x, Y ). Eric Zivot (Copyright 2015) Probability Review - Part 2 30 / 46
31 Outline 1 Bivariate Probability Distribution - Discrete rv s 2 Bivariate Probability Distribution - Continuous rv s 3 Independence 4 Covariance and Correlation Bivariate normal distribution Linear Combination of rv s Eric Zivot (Copyright 2015) Probability Review - Part 2 31 / 46
32 Linear Combination of 2 rv s Let X and Y be rv s. Define a new rv Z that is a linear combination of X and Y : Z = ax + by where a and b are constants. Then, µ Z = E[Z] = E[aX + by ] = ae[x] + be[y ] = a µ X + b µ Y Eric Zivot (Copyright 2015) Probability Review - Part 2 32 / 46
33 Linear Combination of 2 rv s cont. and, σ 2 Z = Var(Z) = Var(a X + b Y ) = a 2 Var(X) + b 2 Var(Y ) + 2a b Cov(X, Y ) = a 2 σ 2 X + b 2 σ 2 Y + 2a b σ XY If X N(µ X, σ 2 X ) and Y N(µ Y, σ 2 Y ) then Z N(µ Z, σ 2 Z ). Eric Zivot (Copyright 2015) Probability Review - Part 2 33 / 46
34 Example Example: Portfolio returns R A = return on asset A with E[R A ] = µ A and Var(R A ) = σ 2 A R B = return on asset B with E[R B ] = µ B and Var(R B ) = σ 2 B Cov(R A, R B ) = σ AB and Cor(R A, R B ) = ρ AB = σ AB σ A σ B Portfolio: x A = share of wealth invested in asset A, x B = share of wealth invested in asset B x A + x B = 1 (exhaust all wealth in 2 assets) R P = x A R A + x B R B = portfolio return Eric Zivot (Copyright 2015) Probability Review - Part 2 34 / 46
35 Problem Portfolio Problem: How much wealth should be invested in assets A and B? Portfolio expected return (gain from investing): E[R P ] = µ P = E[x A R A + x B R B ] = x A E[R A ] + x B E[R B ] = x A µ A + x B µ B Eric Zivot (Copyright 2015) Probability Review - Part 2 35 / 46
36 Problem cont. Portfolio variance (risk from investing): Var(R P ) = σp 2 = Var(x A R A + x B R B ) = x 2 AVar(R A ) + x 2 BVar(R B )+ 2 x A x B Cov(R A, R B ) SD(R P ) = = x 2 Aσ 2 A + x 2 Bσ 2 B + 2x A x B σ AB Var(R P ) = σ P ) 1/2 = (x 2 AσA 2 + x 2 BσB 2 + 2x A x B σ AB Eric Zivot (Copyright 2015) Probability Review - Part 2 36 / 46
37 Linear Combination of N rv s Let X 1, X 2,, X N be rvs and let a 1, a 2,..., a N be constants. Define, N Z = a 1 X 1 + a 2 X a N X N = a i X i i=1 Then, µ Z = E[Z] = a 1 E[X 1 ] + a 2 E[X 2 ] + + a N E[X N ] N N = a i E[X i ] = a i µ i i=1 i=1 Eric Zivot (Copyright 2015) Probability Review - Part 2 37 / 46
38 Linear Combination of N rv s cont. For the variance, σ 2 Z = Var(Z) = a 2 1Var(X 1 ) + + a 2 NVar(X N ) + 2a 1 a 2 Cov(X 1, X 2 ) + 2a 1 a 3 Cov(X 1, X 3 ) + + 2a 2 a 3 Cov(X 2, X 3 ) + 2a 2 a 4 Cov(X 2, X 4 ) + + 2a N 1 a N Cov(X N 1, X N ) Note: N variance terms and N(N 1) = N 2 N covariance terms. If N = 100, there are = 9900 covariance terms! Result: If X 1, X 2,, X N variables then, are each normally distributed random N Z = a i X i N(µ Z, σz) 2 i=1 Eric Zivot (Copyright 2015) Probability Review - Part 2 38 / 46
39 Example Example: Portfolio variance with three assets R A, R B, R C are simple returns on assets A, B and C x A, x B, x C are portfolio shares such that x A + x B + x C = 1 R p = x A R A + x B R B + x C R C Portfolio variance, σ 2 P = x 2 Aσ 2 A + x 2 Bσ 2 B + x 2 Cσ 2 C + 2x A x B σ AB + 2x A x C σ AC + 2x B x C σ BC Note: Portfolio variance calculation may be simplified using matrix layout, x A x B x C x A σa 2 σ AB σ AC x B σ AB σb 2 σ BC x C σ AC σ BC σc 2 Eric Zivot (Copyright 2015) Probability Review - Part 2 39 / 46
40 Example Example: Multi-period continuously compounded returns and the square-root-of-time rule r t = ln(1 + R t ) = monthly cc return r t N(µ, σ 2 ) for all t Cov(r t, r s ) = 0 for all t s Annual return, r t (12) = 11 r t j j=0 = r t + r t r t 11 Eric Zivot (Copyright 2015) Probability Review - Part 2 40 / 46
41 Example cont. Then, E[r t (12)] = = 11 j=0 11 j=0 E[r t j ] µ (E[r t ] = µ for all t) = 12µ (µ = mean of monthly return) Eric Zivot (Copyright 2015) Probability Review - Part 2 41 / 46
42 Example cont. And, Var(r t (12)) = Var = 11 r t j j= Var(r t j ) = σ 2 j=0 j=0 = 12 σ 2 (σ 2 = monthly variance) SD(r t (12)) = 12 σ (square root of time rule) Then, r t (12) N(12µ, 12σ 2 ) Eric Zivot (Copyright 2015) Probability Review - Part 2 42 / 46
43 Example cont. For example, suppose: r t N(0.01, (0.10) 2 ) Then, E[r t (12)] = 12 (0.01) = 0.12 Var(r t (12)) = 12 (0.10) 2 = 0.12 SD(r t (12)) = 0.12 = r t (12) N(0.12, (0.346) 2 ) Eric Zivot (Copyright 2015) Probability Review - Part 2 43 / 46
44 Example cont. And, (qα) r A = 12 µ + 12 σ z α = z α (qα R ) A = e (qr α )A 1 = e zα 1 Eric Zivot (Copyright 2015) Probability Review - Part 2 44 / 46
45 Covariance between two linear combinations of random variables Consider two linear combinations of two random variables: Then, X = X 1 + X 2 Y = Y 1 + Y 2 cov(x, Y ) = cov(x 1 + X 2, Y 1 + Y 2 ) = cov(x 1, Y 1 ) + cov(x 1, Y 2 ) + cov(x 2, Y 1 ) + cov(x 2, Y 2 ) The result generalizes to linear combinations of N random variables in the obvious way. Eric Zivot (Copyright 2015) Probability Review - Part 2 45 / 46
46 You can t see this text! faculty.washington.edu/ezivot/ Eric Zivot (Copyright 2015) Probability Review - Part 2 46 / 46
Introduction to Computational Finance and Financial Econometrics Matrix Algebra Review
You can t see this text! Introduction to Computational Finance and Financial Econometrics Matrix Algebra Review Eric Zivot Spring 2015 Eric Zivot (Copyright 2015) Matrix Algebra Review 1 / 54 Outline 1
More informationIntroduction to Computational Finance and Financial Econometrics Probability Theory Review: Part 2
Introduction to Computational Finance and Financial Econometrics Probability Theory Review: Part 2 Eric Zivot July 7, 2014 Bivariate Probability Distribution Example - Two discrete rv s and Bivariate pdf
More informationElements of Probability Theory
Short Guides to Microeconometrics Fall 2016 Kurt Schmidheiny Unversität Basel Elements of Probability Theory Contents 1 Random Variables and Distributions 2 1.1 Univariate Random Variables and Distributions......
More informationENGG2430A-Homework 2
ENGG3A-Homework Due on Feb 9th,. Independence vs correlation a For each of the following cases, compute the marginal pmfs from the joint pmfs. Explain whether the random variables X and Y are independent,
More informationMAS113 Introduction to Probability and Statistics. Proofs of theorems
MAS113 Introduction to Probability and Statistics Proofs of theorems Theorem 1 De Morgan s Laws) See MAS110 Theorem 2 M1 By definition, B and A \ B are disjoint, and their union is A So, because m is a
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationCovariance. Lecture 20: Covariance / Correlation & General Bivariate Normal. Covariance, cont. Properties of Covariance
Covariance Lecture 0: Covariance / Correlation & General Bivariate Normal Sta30 / Mth 30 We have previously discussed Covariance in relation to the variance of the sum of two random variables Review Lecture
More informationmatrix-free Elements of Probability Theory 1 Random Variables and Distributions Contents Elements of Probability Theory 2
Short Guides to Microeconometrics Fall 2018 Kurt Schmidheiny Unversität Basel Elements of Probability Theory 2 1 Random Variables and Distributions Contents Elements of Probability Theory matrix-free 1
More informationSTA 256: Statistics and Probability I
Al Nosedal. University of Toronto. Fall 2017 My momma always said: Life was like a box of chocolates. You never know what you re gonna get. Forrest Gump. There are situations where one might be interested
More information5 Operations on Multiple Random Variables
EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y
More informationCHAPTER 4 MATHEMATICAL EXPECTATION. 4.1 Mean of a Random Variable
CHAPTER 4 MATHEMATICAL EXPECTATION 4.1 Mean of a Random Variable The expected value, or mathematical expectation E(X) of a random variable X is the long-run average value of X that would emerge after a
More informationBASICS OF PROBABILITY
October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,
More informationLecture 11. Multivariate Normal theory
10. Lecture 11. Multivariate Normal theory Lecture 11. Multivariate Normal theory 1 (1 1) 11. Multivariate Normal theory 11.1. Properties of means and covariances of vectors Properties of means and covariances
More informationLecture 2: Review of Probability
Lecture 2: Review of Probability Zheng Tian Contents 1 Random Variables and Probability Distributions 2 1.1 Defining probabilities and random variables..................... 2 1.2 Probability distributions................................
More informationECE Lecture #9 Part 2 Overview
ECE 450 - Lecture #9 Part Overview Bivariate Moments Mean or Expected Value of Z = g(x, Y) Correlation and Covariance of RV s Functions of RV s: Z = g(x, Y); finding f Z (z) Method : First find F(z), by
More informationP (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n
JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationChp 4. Expectation and Variance
Chp 4. Expectation and Variance 1 Expectation In this chapter, we will introduce two objectives to directly reflect the properties of a random variable or vector, which are the Expectation and Variance.
More informationEcon 424 Review of Matrix Algebra
Econ 424 Review of Matrix Algebra Eric Zivot January 22, 205 Matrices and Vectors Matrix 2 A = 2 22 2 ( )...... 2 = of rows, = of columns Square matrix : = Vector x ( ) = 2. Remarks R is a matrix oriented
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationLecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationJoint Distributions. (a) Scalar multiplication: k = c d. (b) Product of two matrices: c d. (c) The transpose of a matrix:
Joint Distributions Joint Distributions A bivariate normal distribution generalizes the concept of normal distribution to bivariate random variables It requires a matrix formulation of quadratic forms,
More informationf X, Y (x, y)dx (x), where f(x,y) is the joint pdf of X and Y. (x) dx
INDEPENDENCE, COVARIANCE AND CORRELATION Independence: Intuitive idea of "Y is independent of X": The distribution of Y doesn't depend on the value of X. In terms of the conditional pdf's: "f(y x doesn't
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Linear Combinations of Variables Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More informationRandom Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay
1 / 13 Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 8, 2013 2 / 13 Random Variable Definition A real-valued
More informationSTAT 430/510: Lecture 16
STAT 430/510: Lecture 16 James Piette June 24, 2010 Updates HW4 is up on my website. It is due next Mon. (June 28th). Starting today back at section 6.7 and will begin Ch. 7. Joint Distribution of Functions
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationStatistics STAT:5100 (22S:193), Fall Sample Final Exam B
Statistics STAT:5 (22S:93), Fall 25 Sample Final Exam B Please write your answers in the exam books provided.. Let X, Y, and Y 2 be independent random variables with X N(µ X, σ 2 X ) and Y i N(µ Y, σ 2
More informationChapter 4 continued. Chapter 4 sections
Chapter 4 sections Chapter 4 continued 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP:
More information11. Regression and Least Squares
11. Regression and Least Squares Prof. Tesler Math 186 Winter 2016 Prof. Tesler Ch. 11: Linear Regression Math 186 / Winter 2016 1 / 23 Regression Given n points ( 1, 1 ), ( 2, 2 ),..., we want to determine
More informationThe Multivariate Normal Distribution. In this case according to our theorem
The Multivariate Normal Distribution Defn: Z R 1 N(0, 1) iff f Z (z) = 1 2π e z2 /2. Defn: Z R p MV N p (0, I) if and only if Z = (Z 1,..., Z p ) T with the Z i independent and each Z i N(0, 1). In this
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 17-27 Review Scott Sheffield MIT 1 Outline Continuous random variables Problems motivated by coin tossing Random variable properties 2 Outline Continuous random variables Problems
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Problem Set 8 Fall 2007
UC Berkeley Department of Electrical Engineering and Computer Science EE 6: Probablity and Random Processes Problem Set 8 Fall 007 Issued: Thursday, October 5, 007 Due: Friday, November, 007 Reading: Bertsekas
More information3-1. all x all y. [Figure 3.1]
- Chapter. Multivariate Distributions. All of the most interesting problems in statistics involve looking at more than a single measurement at a time, at relationships among measurements and comparisons
More informationApplied Econometrics - QEM Theme 1: Introduction to Econometrics Chapter 1 + Probability Primer + Appendix B in PoE
Applied Econometrics - QEM Theme 1: Introduction to Econometrics Chapter 1 + Probability Primer + Appendix B in PoE Warsaw School of Economics Outline 1. Introduction to econometrics 2. Denition of econometrics
More informationMore than one variable
Chapter More than one variable.1 Bivariate discrete distributions Suppose that the r.v. s X and Y are discrete and take on the values x j and y j, j 1, respectively. Then the joint p.d.f. of X and Y, to
More informationRandom Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.
Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example
More informationLet X and Y denote two random variables. The joint distribution of these random
EE385 Class Notes 9/7/0 John Stensby Chapter 3: Multiple Random Variables Let X and Y denote two random variables. The joint distribution of these random variables is defined as F XY(x,y) = [X x,y y] P.
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More informationRandom Variables and Expectations
Inside ECOOMICS Random Variables Introduction to Econometrics Random Variables and Expectations A random variable has an outcome that is determined by an experiment and takes on a numerical value. A procedure
More informationECON Fundamentals of Probability
ECON 351 - Fundamentals of Probability Maggie Jones 1 / 32 Random Variables A random variable is one that takes on numerical values, i.e. numerical summary of a random outcome e.g., prices, total GDP,
More informationChapter 2. Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Chapter 2 Some Basic Probability Concepts 2.1 Experiments, Outcomes and Random Variables A random variable is a variable whose value is unknown until it is observed. The value of a random variable results
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationPractice Examination # 3
Practice Examination # 3 Sta 23: Probability December 13, 212 This is a closed-book exam so do not refer to your notes, the text, or any other books (please put them on the floor). You may use a single
More informationACM 116: Lectures 3 4
1 ACM 116: Lectures 3 4 Joint distributions The multivariate normal distribution Conditional distributions Independent random variables Conditional distributions and Monte Carlo: Rejection sampling Variance
More informationNotes on Random Vectors and Multivariate Normal
MATH 590 Spring 06 Notes on Random Vectors and Multivariate Normal Properties of Random Vectors If X,, X n are random variables, then X = X,, X n ) is a random vector, with the cumulative distribution
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationLIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
More informationProbability and Statistics Notes
Probability and Statistics Notes Chapter Five Jesse Crawford Department of Mathematics Tarleton State University Spring 2011 (Tarleton State University) Chapter Five Notes Spring 2011 1 / 37 Outline 1
More informationMAS113 Introduction to Probability and Statistics. Proofs of theorems
MAS113 Introduction to Probability and Statistics Proofs of theorems Theorem 1 De Morgan s Laws) See MAS110 Theorem 2 M1 By definition, B and A \ B are disjoint, and their union is A So, because m is a
More information18 Bivariate normal distribution I
8 Bivariate normal distribution I 8 Example Imagine firing arrows at a target Hopefully they will fall close to the target centre As we fire more arrows we find a high density near the centre and fewer
More informationHomework 5 Solutions
126/DCP126 Probability, Fall 214 Instructor: Prof. Wen-Guey Tzeng Homework 5 Solutions 5-Jan-215 1. Let the joint probability mass function of discrete random variables X and Y be given by { c(x + y) ifx
More informationProbability- the good parts version. I. Random variables and their distributions; continuous random variables.
Probability- the good arts version I. Random variables and their distributions; continuous random variables. A random variable (r.v) X is continuous if its distribution is given by a robability density
More informationProblem Solving. Correlation and Covariance. Yi Lu. Problem Solving. Yi Lu ECE 313 2/51
Yi Lu Correlation and Covariance Yi Lu ECE 313 2/51 Definition Let X and Y be random variables with finite second moments. the correlation: E[XY ] Yi Lu ECE 313 3/51 Definition Let X and Y be random variables
More informationStat 5101 Notes: Algorithms (thru 2nd midterm)
Stat 5101 Notes: Algorithms (thru 2nd midterm) Charles J. Geyer October 18, 2012 Contents 1 Calculating an Expectation or a Probability 2 1.1 From a PMF........................... 2 1.2 From a PDF...........................
More informationUCSD ECE153 Handout #34 Prof. Young-Han Kim Tuesday, May 27, Solutions to Homework Set #6 (Prepared by TA Fatemeh Arbabjolfaei)
UCSD ECE53 Handout #34 Prof Young-Han Kim Tuesday, May 7, 04 Solutions to Homework Set #6 (Prepared by TA Fatemeh Arbabjolfaei) Linear estimator Consider a channel with the observation Y XZ, where the
More informationThe Multivariate Normal Distribution 1
The Multivariate Normal Distribution 1 STA 302 Fall 2017 1 See last slide for copyright information. 1 / 40 Overview 1 Moment-generating Functions 2 Definition 3 Properties 4 χ 2 and t distributions 2
More information8 - Continuous random vectors
8-1 Continuous random vectors S. Lall, Stanford 2011.01.25.01 8 - Continuous random vectors Mean-square deviation Mean-variance decomposition Gaussian random vectors The Gamma function The χ 2 distribution
More information2 (Statistics) Random variables
2 (Statistics) Random variables References: DeGroot and Schervish, chapters 3, 4 and 5; Stirzaker, chapters 4, 5 and 6 We will now study the main tools use for modeling experiments with unknown outcomes
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More informationProblem Y is an exponential random variable with parameter λ = 0.2. Given the event A = {Y < 2},
ECE32 Spring 25 HW Solutions April 6, 25 Solutions to HW Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where
More informationFinal Exam # 3. Sta 230: Probability. December 16, 2012
Final Exam # 3 Sta 230: Probability December 16, 2012 This is a closed-book exam so do not refer to your notes, the text, or any other books (please put them on the floor). You may use the extra sheets
More informationIntroduction to Normal Distribution
Introduction to Normal Distribution Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 17-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Introduction
More informationA Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
More informationJoint Distribution of Two or More Random Variables
Joint Distribution of Two or More Random Variables Sometimes more than one measurement in the form of random variable is taken on each member of the sample space. In cases like this there will be a few
More informationProbability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.
Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel
More informationInteresting Probability Problems
Interesting Probability Problems Jonathan Mostovoy - 4665 University of Toronto August 9, 6 Contents Chapter Questions a).8.7............................................ b)..............................................
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More information18.440: Lecture 26 Conditional expectation
18.440: Lecture 26 Conditional expectation Scott Sheffield MIT 1 Outline Conditional probability distributions Conditional expectation Interpretation and examples 2 Outline Conditional probability distributions
More informationSolutions and Proofs: Optimizing Portfolios
Solutions and Proofs: Optimizing Portfolios An Undergraduate Introduction to Financial Mathematics J. Robert Buchanan Covariance Proof: Cov(X, Y) = E [XY Y E [X] XE [Y] + E [X] E [Y]] = E [XY] E [Y] E
More informationRandom Vectors 1. STA442/2101 Fall See last slide for copyright information. 1 / 30
Random Vectors 1 STA442/2101 Fall 2017 1 See last slide for copyright information. 1 / 30 Background Reading: Renscher and Schaalje s Linear models in statistics Chapter 3 on Random Vectors and Matrices
More informationLecture 4: Conditional expectation and independence
Lecture 4: Conditional expectation and independence In elementry probability, conditional probability P(B A) is defined as P(B A) P(A B)/P(A) for events A and B with P(A) > 0. For two random variables,
More information01 Probability Theory and Statistics Review
NAVARCH/EECS 568, ROB 530 - Winter 2018 01 Probability Theory and Statistics Review Maani Ghaffari January 08, 2018 Last Time: Bayes Filters Given: Stream of observations z 1:t and action data u 1:t Sensor/measurement
More informationPreliminary Statistics Lecture 3: Probability Models and Distributions (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 3: Probability Models and Distributions (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics,
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationECE 4400:693 - Information Theory
ECE 4400:693 - Information Theory Dr. Nghi Tran Lecture 8: Differential Entropy Dr. Nghi Tran (ECE-University of Akron) ECE 4400:693 Lecture 1 / 43 Outline 1 Review: Entropy of discrete RVs 2 Differential
More informationEE4601 Communication Systems
EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two
More informationSolutions to Homework Set #5 (Prepared by Lele Wang) MSE = E [ (sgn(x) g(y)) 2],, where f X (x) = 1 2 2π e. e (x y)2 2 dx 2π
Solutions to Homework Set #5 (Prepared by Lele Wang). Neural net. Let Y X + Z, where the signal X U[,] and noise Z N(,) are independent. (a) Find the function g(y) that minimizes MSE E [ (sgn(x) g(y))
More informationMATHEMATICS 154, SPRING 2009 PROBABILITY THEORY Outline #11 (Tail-Sum Theorem, Conditional distribution and expectation)
MATHEMATICS 154, SPRING 2009 PROBABILITY THEORY Outline #11 (Tail-Sum Theorem, Conditional distribution and expectation) Last modified: March 7, 2009 Reference: PRP, Sections 3.6 and 3.7. 1. Tail-Sum Theorem
More informationPreliminary Statistics. Lecture 3: Probability Models and Distributions
Preliminary Statistics Lecture 3: Probability Models and Distributions Rory Macqueen (rm43@soas.ac.uk), September 2015 Outline Revision of Lecture 2 Probability Density Functions Cumulative Distribution
More information3 Multiple Discrete Random Variables
3 Multiple Discrete Random Variables 3.1 Joint densities Suppose we have a probability space (Ω, F,P) and now we have two discrete random variables X and Y on it. They have probability mass functions f
More informationChapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationMATH 38061/MATH48061/MATH68061: MULTIVARIATE STATISTICS Solutions to Problems on Random Vectors and Random Sampling. 1+ x2 +y 2 ) (n+2)/2
MATH 3806/MATH4806/MATH6806: MULTIVARIATE STATISTICS Solutions to Problems on Rom Vectors Rom Sampling Let X Y have the joint pdf: fx,y) + x +y ) n+)/ π n for < x < < y < this is particular case of the
More informationBivariate Distributions. Discrete Bivariate Distribution Example
Spring 7 Geog C: Phaedon C. Kyriakidis Bivariate Distributions Definition: class of multivariate probability distributions describing joint variation of outcomes of two random variables (discrete or continuous),
More informationStochastic Models of Manufacturing Systems
Stochastic Models of Manufacturing Systems Ivo Adan Organization 2/47 7 lectures (lecture of May 12 is canceled) Studyguide available (with notes, slides, assignments, references), see http://www.win.tue.nl/
More information14.30 Introduction to Statistical Methods in Economics Spring 2009
MIT OpenCourseWare http://ocw.mit.edu 14.30 Introduction to Statistical Methods in Economics Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationEEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 18-27 Review Scott Sheffield MIT Outline Outline It s the coins, stupid Much of what we have done in this course can be motivated by the i.i.d. sequence X i where each X i is
More informationMore on Distribution Function
More on Distribution Function The distribution of a random variable X can be determined directly from its cumulative distribution function F X. Theorem: Let X be any random variable, with cumulative distribution
More informationOutline Properties of Covariance Quantifying Dependence Models for Joint Distributions Lab 4. Week 8 Jointly Distributed Random Variables Part II
Week 8 Jointly Distributed Random Variables Part II Week 8 Objectives 1 The connection between the covariance of two variables and the nature of their dependence is given. 2 Pearson s correlation coefficient
More informationCovariance and Correlation Class 7, Jeremy Orloff and Jonathan Bloom
1 Learning Goals Covariance and Correlation Class 7, 18.05 Jerem Orloff and Jonathan Bloom 1. Understand the meaning of covariance and correlation. 2. Be able to compute the covariance and correlation
More informationProbability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3
Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................
More information5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More informationRaquel Prado. Name: Department of Applied Mathematics and Statistics AMS-131. Spring 2010
Raquel Prado Name: Department of Applied Mathematics and Statistics AMS-131. Spring 2010 Final Exam (Type B) The midterm is closed-book, you are only allowed to use one page of notes and a calculator.
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More information