The Multivariate Normal Distribution. In this case according to our theorem
|
|
- Ashley Simon
- 5 years ago
- Views:
Transcription
1 The Multivariate Normal Distribution Defn: Z R 1 N(0, 1) iff f Z (z) = 1 2π e z2 /2. Defn: Z R p MV N p (0, I) if and only if Z = (Z 1,..., Z p ) T with the Z i independent and each Z i N(0, 1). In this case according to our theorem f Z (z 1,..., z p ) = 1 2π e z2 i /2 = (2π) p/2 exp{ z T z/2} ; superscript t denotes matrix transpose. Defn: X R p has a multivariate normal distribution if it has the same distribution as AZ+µ for some µ R p, some p q matrix of constants A and Z MV N q (0, I). 35
2 p = q, A singular: X does not have a density. A invertible: derive multivariate normal density by change of variables: X = AZ + µ Z = A 1 (X µ) So X Z = A Z X = A 1. f X (x) = f Z (A 1 (x µ)) det(a 1 ) = exp{ (x µ)t (A 1 ) T A 1 (x µ)/2} (2π) p/2 det A Now define Σ = AA T and notice that and Thus f X is Σ 1 = (A T ) 1 A 1 = (A 1 ) T A 1 det Σ = det A det A T = (det A) 2.. exp{ (x µ) T Σ 1 (x µ)/2} (2π) p/2 (det Σ) 1/2 ; the M V N(µ, Σ) density. Note density is the same for all A such that AA T = Σ. This justifies the notation MV N(µ, Σ). 36
3 For which µ, Σ is this a density? Any µ but if x R p then x T Σx = x T AA T x = (A T x) T (A T x) = p 1 y 2 i 0 where y = A T x. Inequality strict except for y = 0 which is equivalent to x = 0. Thus Σ is a positive definite symmetric matrix. Conversely, if Σ is a positive definite symmetric matrix then there is a square invertible matrix A such that AA T = Σ so that there is a MV N(µ, Σ) distribution. (A can be found via the Cholesky decomposition, e.g.) When A is singular X will not have a density: a such that P (a T X = a T µ) = 1; X is confined to a hyperplane. Still true: distribution of X depends only on Σ = AA T : if AA T = BB T then AZ + µ and BZ + µ have the same distribution. 37
4 Expectation, moments Defn: If X R p has density f then E(g(X)) = any g from R p to R. g(x)f(x) dx. FACT: if Y = g(x) for a smooth g (mapping R R) E(Y ) = = yf Y (y) dy g(x)f Y (g(x))g (x) dx = E(g(X)) by change of variables formula for integration. This is good because otherwise we might have two different values for E(e X ). Linearity: E(aX + by ) = ae(x) + be(y ) for real X and Y. 38
5 Defn: The r th moment (about the origin) of a real rv X is µ r = E(X r ) (provided it exists). We generally use µ for E(X). Defn: The r th central moment is µ r = E[(X µ) r ] We call σ 2 = µ 2 the variance. Defn: For an R p valued random vector X µ X = E(X) is the vector whose i th entry is E(X i ) (provided all entries exist). Fact: same idea used for random matrices. Defn: The (p p) variance covariance matrix of X is Var(X) = E [ (X µ)(x µ) T ] which exists provided each component X i has a finite second moment. 39
6 Example moments: If Z N(0, 1) then E(Z) = ze z2 /2 dz/ 2π = e z2 /2 2π = 0 and (integrating by parts) E(Z r ) = so that zr e z2 /2 dz/ 2π = zr 1 e z2 /2 2π + (r 1) zr 2 e z2 /2 dz/ 2π µ r = (r 1)µ r 2 for r 2. Remembering that µ 1 = 0 and µ 0 = z0 e z2 /2 dz/ 2π = 1 we find that { 0 r odd µ r = (r 1)(r 3) 1 r even. 40
7 If now X N(µ, σ 2 ), that is, X σz + µ, then E(X) = σe(z) + µ = µ and µ r (X) = E[(X µ) r ] = σ r E(Z r ) In particular, we see that our choice of notation N(µ, σ 2 ) for the distribution of σz + µ is justified; σ is indeed the variance. Similarly for X M V N(µ, Σ) we have X = AZ + µ with Z MV N(0, I) and E(X) = µ and Var(X) = E { (X µ)(x µ) T } = E { AZ(AZ) T } = AE(ZZ T )A T = AIA T = Σ. Note use of easy calculation: E(Z) = 0 and Var(Z) = E(ZZ T ) = I. 41
8 Moments and independence Theorem: If X 1,..., X p are independent and each X i is integrable then X = X 1 X p is integrable and E(X 1 X p ) = E(X 1 ) E(X p ). Moment Generating Functions Defn: The moment generating function of a real valued X is M X (t) = E(e tx ) defined for those real t for which the expected value is finite. Defn: The moment generating function of X R p is M X (u) = E[e ut X ] defined for those vectors u for which the expected value is finite. 42
9 Example: If Z N(0, 1) then M Z (t) = 1 2π = 1 2π = 1 2π = e t2 /2 etz z2 /2 dz e (z t)2 /2+t 2 /2 dz e u2 /2+t 2 /2 du Theorem: (p = 1) If M is finite for all t in a neighbourhood of 0 then 1. Every moment of X is finite. 2. M is C (in fact M is analytic). 3. µ k = dk dt km X(0). Note: C means has continuous derivatives of all orders. Analytic means has convergent power series expansion in neighbourhood of each t ( ɛ, ɛ). The proof, and many other facts about mgfs, rely on techniques of complex variables. 43
10 Characterization & MGFs Theorem: Suppose X and Y are R p valued random vectors such that M X (u) = M Y (u) for u in some open neighbourhood of 0 in R p. Then X and Y have the same distribution. The proof relies on techniques of complex variables. 44
11 MGFs and Sums If X 1,..., X p are independent and Y = X i then mgf of Y is product mgfs of individual X i : E(e ty ) = i E(e tx i) or M Y = M Xi. (Also for multivariate X i.) Example: If Z 1,..., Z p are independent N(0, 1) then E(e ai Z i ) = E(e a iz i ) i = i e a2 i /2 = exp( a 2 i /2) Conclusion: If Z MNV p (0, I) then M Z (u) = exp( u 2 i /2) = exp(ut u/2). Example: If X N(µ, σ 2 ) then X = σz + µ and M X (t) = E(e t(σz+µ) ) = e tµ e σ2 t 2 /2. 45
12 Theorem: Suppose X = AZ + µ and Y = A Z + µ where Z MV N p (0, I) and Z MV N q (0, I). Then X and Y have the same distribution if and only iff the following two conditions hold: 1. µ = µ. 2. AA T = A (A ) T. Alternatively: if X, Y each MVN then E(X) = E(Y) and Var(X) = Var(Y) imply that X and Y have the same distribution. Proof: If 1 and 2 hold the mgf of X is ( E e tt X ) = E (e tt (AZ+µ ) ( = e tt µ E e (AT t) T Z ) = e tt µ+(a T t) T (A T t) = e tt µ+t T Σt Thus M X = M Y. Conversely if X and Y have the same distribution then they have the same mean and variance. 46
13 Thus mgf is determined by µ and Σ. Theorem: If X MV N p (µ, Σ) then there is A a p p matrix such that X has same distribution as AZ + µ for Z MV N p (0, I). We may assume that A is symmetric and nonnegative definite, or that A is upper triangular, or that Ba is lower triangular. Proof: Pick any A such that AA T = Σ such as PD 1/2 P T from the spectral decomposition. Then AZ + µ MV N p (µ, Σ). From the symmetric square root can produce an upper triangular square root by the Gram Schmidt process: if A has rows a T 1,..., at p then let v p be a p / a T p a p. Choose v p 1 proportional to a p 1 bv p where b = a T p 1 v p so that v p 1 has unit length. Continue in this way; you automatically get a T j v k = 0 if j < k. If P has columns v 1,..., v p then P is orthogonal and AP is an upper triangular square root of Σ. 47
14 Variances, Covariances, Correlations Defn: The covariance between X and Y is Cov(X, Y) = E { (X µ X )(Y µ Y ) T } This is a matrix. Properties: Cov(X, X) = Var(X). Cov is bilinear: Cov(AX + BW, Y) =ACov(X, Y) + BCov(W, Y) and Cov(X, CY + DZ) =Cov(X, Y)C T + Cov(X, Z)D T 48
15 Properties of the M V N distribution 1: All margins are multivariate normal: if and Σ = X = µ = [ X1 X 2 [ µ1 µ 2 ] ] [ Σ11 Σ 12 Σ 21 Σ 22 then X MV N(µ, Σ) X 1 MV N(µ 1, Σ 11 ). 2: MX + ν MV N(Mµ + ν, MΣM T ): affine transformation of MVN is normal. 3: If Σ 12 = Cov(X 1, X 2 ) = 0 then X 1 and X 2 are independent. 4: All conditionals are normal: the conditional distribution of X 1 given X 2 = x 2 is MV N(µ 1 + Σ 12 Σ 1 22 (x 2 µ 2 ), Σ 11 Σ 12 Σ 1 22 Σ 21) ] 49
16 Proof of (1): If X = AZ + µ then X 1 = [I 0] X for I the identity matrix of correct dimension. So X 1 = ([I 0] A) Z + [I 0] µ Compute mean and variance to check rest. Proof of (2): If X = AZ + µ then MX + ν = MAZ + ν + Mµ Proof of (3): If then u = [ u1 u 2 ] M X (u) = M X1 (u 1 )M X2 (u 2 ) 50
17 Proof of (4): first case: assume Σ 22 has an inverse. Define Then [ W X 2 W = X 1 Σ 12 Σ 1 22 X 2 ] = Thus (W, X 2 ) T where Σ = [ I Σ12 Σ I ] [ X1 X 2 is MV N(µ 1 Σ 12 Σ 1 22 µ 2, Σ ) [ Σ11 Σ 12 Σ 1 22 Σ Σ 22 ] ] 51
18 Now joint density of W and X factors f W,X2 (w, x 2 ) = f W (w)f X2 (x 2 ) By change of variables joint density of X is f X1,X 2 (x 1, x 2 ) = cf W (x 1 Mx 2 )f X2 (x 2 ) where c = 1 is the constant Jacobian of the linear transformation from (W, X 2 ) to (X 1, X 2 ) and M = Σ 12 Σ 1 22 Thus conditional density of X 1 given X 2 = x 2 is f W (x 1 Mx 2 )f X2 (x 2 ) f X2 (x 2 ) = f W (x 1 Mx 2 ) As a function of x 1 this density has the form of the advertised multivariate normal density. 52
19 Specialization to bivariate case: Write Σ = where we define Note that [ σ 2 1 ρσ 1 σ 2 ρσ 1 σ 2 σ 2 2 ρ = Cov(X 1, X 2 ) Var(X 1 )Var(X 2 ) σ 2 i = Var(X i) ] Then W = X 1 ρ σ 1 σ 2 X 2 is independent of X 2. The marginal distribution of W is N(µ 1 ρσ 1 µ 2 /σ 2, τ 2 ) where τ 2 =Var(X 1 ) 2ρ σ 1 Cov(X 1, X 2 ) σ 2 ( + ρ σ ) 2 1 Var(X 2 ) σ 2 53
20 This simplifies to σ 2 1 (1 ρ2 ) Notice that it follows that 1 ρ 1 More generally: any X and Y : 0 Var(X λy ) = Var(X) 2λCov(X, Y ) + λ 2 Var(Y ) RHS is minimized at Minimum value is λ = Cov(X, Y ) Var(Y ) 0 Var(X)(1 ρ 2 XY ) where ρ XY = Cov(X, Y ) Var(X)Var(Y ) defines the correlation between X and Y. 54
21 Multiple Correlation Now suppose X 2 is scalar but X 1 is vector. Defn: Multiple correlation between X 1 and X 2 over all a 0. R 2 (X 1, X 2 ) = max ρ a T X 1,X 2 2 Thus: maximize Cov 2 (a T X 1, X 2 ) Var(a T X 1 )Var(X 2 ) = at Σ 12 Σ 21 a ( a T Σ 11 a ) Σ 22 Put b = Σ 1/2 11 a. For Σ 11 invertible problem is equivalent to maximizing where b T Qb b T b Q = Σ 1/2 11 Σ 12 Σ 21 Σ 1/2 11 Solution: find largest eigenvalue of Q. 55
22 Note where is a vector. Set Q = vv T v = Σ 1/2 11 Σ 12 vv T x = λx and multiply by v T to get v T x = 0 or λ = v T v If v T x = 0 then we see λ = 0 so largest eigenvalue is v T v. Summary: maximum squared correlation is R 2 (X 1, X 2 ) = vt v = Σ 21Σ 1 11 Σ 12 Σ 22 Σ 22 Achieved when eigenvector is x = v = b so a = Σ 1/2 11 Σ 1/2 11 Σ 12 = Σ 1 11 Σ 12 Notice: since R 2 is squared correlation between two scalars (a t X 1 and X 2 ) we have 0 R 2 1 Equals 1 iff X 2 is linear combination of X 1. 56
23 Correlation matrices, partial correlations: Correlation between two scalars X and Y is ρ XY = Cov(X, Y ) Var(X)Var(Y ) If X has variance Σ then the correlation matrix of X is R X with entries R ij = Cov(X i, X j ) Var(Xi )Var(X j ) = Σ ij Σii Σ jj If X 1, X 2 are MVN with the usual partitioned variance covariance matrix then the conditional variance of X 1 given X 2 is Σ 11 2 = Σ 11 Σ 12 Σ 1 22 Σ 21 From this define partial correlation matrix R 11 2 = (Σ 11 2 ) ij Σ11 2 ) ii Σ 11 2 ) jj Note: these are used even when X 1, X 2 are NOT MVN 57
STAT 450. Moment Generating Functions
STAT 450 Moment Generating Functions There are many uses of generating functions in mathematics. We often study the properties of a sequence a n of numbers by creating the function a n s n n0 In statistics
More informationNext tool is Partial ACF; mathematical tools first. The Multivariate Normal Distribution. e z2 /2. f Z (z) = 1 2π. e z2 i /2
Next tool is Partial ACF; mathematical tools first. The Multivariate Normal Distribution Defn: Z R 1 N(0,1) iff f Z (z) = 1 2π e z2 /2 Defn: Z R p MV N p (0, I) if and only if Z = (Z 1,..., Z p ) (a column
More informationSTAT 830 The Multivariate Normal Distribution
STAT 830 The Multivariate Normal Distribution Richard Lockhart Simon Fraser University STAT 830 Fall 2013 Richard Lockhart (Simon Fraser University)STAT 830 The Multivariate Normal Distribution STAT 830
More informationThe Multivariate Normal Distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36
The Multivariate Normal Distribution Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 36 The Moment Generating Function (MGF) of a random vector X is given by M X (t) = E(e t X
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationMoment Generating Function. STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution
Moment Generating Function STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution T. Linder Queen s University Winter 07 Definition Let X (X,...,X n ) T be a random vector and
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationLecture 11. Multivariate Normal theory
10. Lecture 11. Multivariate Normal theory Lecture 11. Multivariate Normal theory 1 (1 1) 11. Multivariate Normal theory 11.1. Properties of means and covariances of vectors Properties of means and covariances
More informationNotes on Random Vectors and Multivariate Normal
MATH 590 Spring 06 Notes on Random Vectors and Multivariate Normal Properties of Random Vectors If X,, X n are random variables, then X = X,, X n ) is a random vector, with the cumulative distribution
More informationRandom Vectors and Multivariate Normal Distributions
Chapter 3 Random Vectors and Multivariate Normal Distributions 3.1 Random vectors Definition 3.1.1. Random vector. Random vectors are vectors of random 75 variables. For instance, X = X 1 X 2., where each
More informationBASICS OF PROBABILITY
October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,
More informationThe Multivariate Normal Distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36
The Multivariate Normal Distribution Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 36 The Moment Generating Function (MGF) of a random vector X is given by M X (t) = E(e t X
More informationProbability Background
Probability Background Namrata Vaswani, Iowa State University August 24, 2015 Probability recap 1: EE 322 notes Quick test of concepts: Given random variables X 1, X 2,... X n. Compute the PDF of the second
More informationMultivariate Gaussian Distribution. Auxiliary notes for Time Series Analysis SF2943. Spring 2013
Multivariate Gaussian Distribution Auxiliary notes for Time Series Analysis SF2943 Spring 203 Timo Koski Department of Mathematics KTH Royal Institute of Technology, Stockholm 2 Chapter Gaussian Vectors.
More informationBasic Concepts in Matrix Algebra
Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1
More information5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More informationBIOS 2083 Linear Models Abdus S. Wahed. Chapter 2 84
Chapter 2 84 Chapter 3 Random Vectors and Multivariate Normal Distributions 3.1 Random vectors Definition 3.1.1. Random vector. Random vectors are vectors of random variables. For instance, X = X 1 X 2.
More informationChapter 4. Chapter 4 sections
Chapter 4 sections 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP: 4.8 Utility Expectation
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More information2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
More informationIn this course we: do distribution theory when ǫ i N(0, σ 2 ) discuss what if the errors, ǫ i are not normal? omit proofs.
Distribution Theory Question: What is distribution theory? Answer: How to compute the distribution of an estimator, test or other statistic, T : Find P(T t), the Cumulative Distribution Function (CDF)
More informationProbability Lecture III (August, 2006)
robability Lecture III (August, 2006) 1 Some roperties of Random Vectors and Matrices We generalize univariate notions in this section. Definition 1 Let U = U ij k l, a matrix of random variables. Suppose
More informationCovariance. Lecture 20: Covariance / Correlation & General Bivariate Normal. Covariance, cont. Properties of Covariance
Covariance Lecture 0: Covariance / Correlation & General Bivariate Normal Sta30 / Mth 30 We have previously discussed Covariance in relation to the variance of the sum of two random variables Review Lecture
More informationACM 116: Lectures 3 4
1 ACM 116: Lectures 3 4 Joint distributions The multivariate normal distribution Conditional distributions Independent random variables Conditional distributions and Monte Carlo: Rejection sampling Variance
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 17-27 Review Scott Sheffield MIT 1 Outline Continuous random variables Problems motivated by coin tossing Random variable properties 2 Outline Continuous random variables Problems
More informationIntroduction to Computational Finance and Financial Econometrics Probability Review - Part 2
You can t see this text! Introduction to Computational Finance and Financial Econometrics Probability Review - Part 2 Eric Zivot Spring 2015 Eric Zivot (Copyright 2015) Probability Review - Part 2 1 /
More informationIII - MULTIVARIATE RANDOM VARIABLES
Computational Methods and advanced Statistics Tools III - MULTIVARIATE RANDOM VARIABLES A random vector, or multivariate random variable, is a vector of n scalar random variables. The random vector is
More informationJoint Distributions. (a) Scalar multiplication: k = c d. (b) Product of two matrices: c d. (c) The transpose of a matrix:
Joint Distributions Joint Distributions A bivariate normal distribution generalizes the concept of normal distribution to bivariate random variables It requires a matrix formulation of quadratic forms,
More informationSection 8.1. Vector Notation
Section 8.1 Vector Notation Definition 8.1 Random Vector A random vector is a column vector X = [ X 1 ]. X n Each Xi is a random variable. Definition 8.2 Vector Sample Value A sample value of a random
More informationThe Multivariate Normal Distribution 1
The Multivariate Normal Distribution 1 STA 302 Fall 2017 1 See last slide for copyright information. 1 / 40 Overview 1 Moment-generating Functions 2 Definition 3 Properties 4 χ 2 and t distributions 2
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationECE Lecture #9 Part 2 Overview
ECE 450 - Lecture #9 Part Overview Bivariate Moments Mean or Expected Value of Z = g(x, Y) Correlation and Covariance of RV s Functions of RV s: Z = g(x, Y); finding f Z (z) Method : First find F(z), by
More informationMLES & Multivariate Normal Theory
Merlise Clyde September 6, 2016 Outline Expectations of Quadratic Forms Distribution Linear Transformations Distribution of estimates under normality Properties of MLE s Recap Ŷ = ˆµ is an unbiased estimate
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationThe Multivariate Gaussian Distribution
The Multivariate Gaussian Distribution Chuong B. Do October, 8 A vector-valued random variable X = T X X n is said to have a multivariate normal or Gaussian) distribution with mean µ R n and covariance
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationLecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
More information5 Operations on Multiple Random Variables
EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y
More informationMAS113 Introduction to Probability and Statistics. Proofs of theorems
MAS113 Introduction to Probability and Statistics Proofs of theorems Theorem 1 De Morgan s Laws) See MAS110 Theorem 2 M1 By definition, B and A \ B are disjoint, and their union is A So, because m is a
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
More informationGaussian random variables inr n
Gaussian vectors Lecture 5 Gaussian random variables inr n One-dimensional case One-dimensional Gaussian density with mean and standard deviation (called N, ): fx x exp. Proposition If X N,, then ax b
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More information8 - Continuous random vectors
8-1 Continuous random vectors S. Lall, Stanford 2011.01.25.01 8 - Continuous random vectors Mean-square deviation Mean-variance decomposition Gaussian random vectors The Gamma function The χ 2 distribution
More informationP (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n
JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are
More informationJoint Distribution of Two or More Random Variables
Joint Distribution of Two or More Random Variables Sometimes more than one measurement in the form of random variable is taken on each member of the sample space. In cases like this there will be a few
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 18-27 Review Scott Sheffield MIT Outline Outline It s the coins, stupid Much of what we have done in this course can be motivated by the i.i.d. sequence X i where each X i is
More informationLecture 14: Multivariate mgf s and chf s
Lecture 14: Multivariate mgf s and chf s Multivariate mgf and chf For an n-dimensional random vector X, its mgf is defined as M X (t) = E(e t X ), t R n and its chf is defined as φ X (t) = E(e ıt X ),
More informationStatistics 351 Probability I Fall 2006 (200630) Final Exam Solutions. θ α β Γ(α)Γ(β) (uv)α 1 (v uv) β 1 exp v }
Statistics 35 Probability I Fall 6 (63 Final Exam Solutions Instructor: Michael Kozdron (a Solving for X and Y gives X UV and Y V UV, so that the Jacobian of this transformation is x x u v J y y v u v
More informationA Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationVectors and Matrices Statistics with Vectors and Matrices
Vectors and Matrices Statistics with Vectors and Matrices Lecture 3 September 7, 005 Analysis Lecture #3-9/7/005 Slide 1 of 55 Today s Lecture Vectors and Matrices (Supplement A - augmented with SAS proc
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationProblem Solving. Correlation and Covariance. Yi Lu. Problem Solving. Yi Lu ECE 313 2/51
Yi Lu Correlation and Covariance Yi Lu ECE 313 2/51 Definition Let X and Y be random variables with finite second moments. the correlation: E[XY ] Yi Lu ECE 313 3/51 Definition Let X and Y be random variables
More informationGaussian vectors and central limit theorem
Gaussian vectors and central limit theorem Samy Tindel Purdue University Probability Theory 2 - MA 539 Samy T. Gaussian vectors & CLT Probability Theory 1 / 86 Outline 1 Real Gaussian random variables
More information4. Distributions of Functions of Random Variables
4. Distributions of Functions of Random Variables Setup: Consider as given the joint distribution of X 1,..., X n (i.e. consider as given f X1,...,X n and F X1,...,X n ) Consider k functions g 1 : R n
More informationEEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
More informationPrincipal Components Theory Notes
Principal Components Theory Notes Charles J. Geyer August 29, 2007 1 Introduction These are class notes for Stat 5601 (nonparametrics) taught at the University of Minnesota, Spring 2006. This not a theory
More information18 Bivariate normal distribution I
8 Bivariate normal distribution I 8 Example Imagine firing arrows at a target Hopefully they will fall close to the target centre As we fire more arrows we find a high density near the centre and fewer
More informationRandom Vectors 1. STA442/2101 Fall See last slide for copyright information. 1 / 30
Random Vectors 1 STA442/2101 Fall 2017 1 See last slide for copyright information. 1 / 30 Background Reading: Renscher and Schaalje s Linear models in statistics Chapter 3 on Random Vectors and Matrices
More information1.12 Multivariate Random Variables
112 MULTIVARIATE RANDOM VARIABLES 59 112 Multivariate Random Variables We will be using matrix notation to denote multivariate rvs and their distributions Denote by X (X 1,,X n ) T an n-dimensional random
More informationThe Multivariate Gaussian Distribution [DRAFT]
The Multivariate Gaussian Distribution DRAFT David S. Rosenberg Abstract This is a collection of a few key and standard results about multivariate Gaussian distributions. I have not included many proofs,
More informationRandom Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.
Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example
More informationMathematical foundations - linear algebra
Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationElements of Probability Theory
Short Guides to Microeconometrics Fall 2016 Kurt Schmidheiny Unversität Basel Elements of Probability Theory Contents 1 Random Variables and Distributions 2 1.1 Univariate Random Variables and Distributions......
More informationChp 4. Expectation and Variance
Chp 4. Expectation and Variance 1 Expectation In this chapter, we will introduce two objectives to directly reflect the properties of a random variable or vector, which are the Expectation and Variance.
More informationLIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
More informationTAMS39 Lecture 2 Multivariate normal distribution
TAMS39 Lecture 2 Multivariate normal distribution Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content Lecture Random vectors Multivariate normal distribution
More information1 Solution to Problem 2.1
Solution to Problem 2. I incorrectly worked this exercise instead of 2.2, so I decided to include the solution anyway. a) We have X Y /3, which is a - function. It maps the interval, ) where X lives) onto
More informationMultivariate Distributions
IEOR E4602: Quantitative Risk Management Spring 2016 c 2016 by Martin Haugh Multivariate Distributions We will study multivariate distributions in these notes, focusing 1 in particular on multivariate
More informationChapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationSometimes can find power series expansion of M X and read off the moments of X from the coefficients of t k /k!.
Moment Generating Functions Defn: The moment generating function of a real valued X is M X (t) = E(e tx ) defined for those real t for which the expected value is finite. Defn: The moment generating function
More informationMultivariate Distributions (Hogg Chapter Two)
Multivariate Distributions (Hogg Chapter Two) STAT 45-1: Mathematical Statistics I Fall Semester 15 Contents 1 Multivariate Distributions 1 11 Random Vectors 111 Two Discrete Random Variables 11 Two Continuous
More informationPCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities
PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More information2 Functions of random variables
2 Functions of random variables A basic statistical model for sample data is a collection of random variables X 1,..., X n. The data are summarised in terms of certain sample statistics, calculated as
More informationBivariate Distributions. Discrete Bivariate Distribution Example
Spring 7 Geog C: Phaedon C. Kyriakidis Bivariate Distributions Definition: class of multivariate probability distributions describing joint variation of outcomes of two random variables (discrete or continuous),
More informationChapter 5. The multivariate normal distribution. Probability Theory. Linear transformations. The mean vector and the covariance matrix
Probability Theory Linear transformations A transformation is said to be linear if every single function in the transformation is a linear combination. Chapter 5 The multivariate normal distribution When
More informationn! (k 1)!(n k)! = F (X) U(0, 1). (x, y) = n(n 1) ( F (y) F (x) ) n 2
Order statistics Ex. 4.1 (*. Let independent variables X 1,..., X n have U(0, 1 distribution. Show that for every x (0, 1, we have P ( X (1 < x 1 and P ( X (n > x 1 as n. Ex. 4.2 (**. By using induction
More information3d scatterplots. You can also make 3d scatterplots, although these are less common than scatterplot matrices.
3d scatterplots You can also make 3d scatterplots, although these are less common than scatterplot matrices. > library(scatterplot3d) > y par(mfrow=c(2,2)) > scatterplot3d(y,highlight.3d=t,angle=20)
More informationTwo hours. Statistical Tables to be provided THE UNIVERSITY OF MANCHESTER. 14 January :45 11:45
Two hours Statistical Tables to be provided THE UNIVERSITY OF MANCHESTER PROBABILITY 2 14 January 2015 09:45 11:45 Answer ALL four questions in Section A (40 marks in total) and TWO of the THREE questions
More informationDef. The euclidian distance between two points x = (x 1,...,x p ) t and y = (y 1,...,y p ) t in the p-dimensional space R p is defined as
MAHALANOBIS DISTANCE Def. The euclidian distance between two points x = (x 1,...,x p ) t and y = (y 1,...,y p ) t in the p-dimensional space R p is defined as d E (x, y) = (x 1 y 1 ) 2 + +(x p y p ) 2
More informationStat 5101 Notes: Algorithms (thru 2nd midterm)
Stat 5101 Notes: Algorithms (thru 2nd midterm) Charles J. Geyer October 18, 2012 Contents 1 Calculating an Expectation or a Probability 2 1.1 From a PMF........................... 2 1.2 From a PDF...........................
More informationBMIR Lecture Series on Probability and Statistics Fall 2015 Discrete RVs
Lecture #7 BMIR Lecture Series on Probability and Statistics Fall 2015 Department of Biomedical Engineering and Environmental Sciences National Tsing Hua University 7.1 Function of Single Variable Theorem
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationLecture 6: Special probability distributions. Summarizing probability distributions. Let X be a random variable with probability distribution
Econ 514: Probability and Statistics Lecture 6: Special probability distributions Summarizing probability distributions Let X be a random variable with probability distribution P X. We consider two types
More informationRecall : Eigenvalues and Eigenvectors
Recall : Eigenvalues and Eigenvectors Let A be an n n matrix. If a nonzero vector x in R n satisfies Ax λx for a scalar λ, then : The scalar λ is called an eigenvalue of A. The vector x is called an eigenvector
More informationn! (k 1)!(n k)! = F (X) U(0, 1). (x, y) = n(n 1) ( F (y) F (x) ) n 2
Order statistics Ex. 4. (*. Let independent variables X,..., X n have U(0, distribution. Show that for every x (0,, we have P ( X ( < x and P ( X (n > x as n. Ex. 4.2 (**. By using induction or otherwise,
More informationREVIEW OF MAIN CONCEPTS AND FORMULAS A B = Ā B. Pr(A B C) = Pr(A) Pr(A B C) =Pr(A) Pr(B A) Pr(C A B)
REVIEW OF MAIN CONCEPTS AND FORMULAS Boolean algebra of events (subsets of a sample space) DeMorgan s formula: A B = Ā B A B = Ā B The notion of conditional probability, and of mutual independence of two
More informationBivariate Transformations
Bivariate Transformations October 29, 29 Let X Y be jointly continuous rom variables with density function f X,Y let g be a one to one transformation. Write (U, V ) = g(x, Y ). The goal is to find the
More information1 Last time: least-squares problems
MATH Linear algebra (Fall 07) Lecture Last time: least-squares problems Definition. If A is an m n matrix and b R m, then a least-squares solution to the linear system Ax = b is a vector x R n such that
More information01 Probability Theory and Statistics Review
NAVARCH/EECS 568, ROB 530 - Winter 2018 01 Probability Theory and Statistics Review Maani Ghaffari January 08, 2018 Last Time: Bayes Filters Given: Stream of observations z 1:t and action data u 1:t Sensor/measurement
More informationMULTIVARIATE DISTRIBUTIONS
Chapter 9 MULTIVARIATE DISTRIBUTIONS John Wishart (1898-1956) British statistician. Wishart was an assistant to Pearson at University College and to Fisher at Rothamsted. In 1928 he derived the distribution
More informationExam 2. Jeremy Morris. March 23, 2006
Exam Jeremy Morris March 3, 006 4. Consider a bivariate normal population with µ 0, µ, σ, σ and ρ.5. a Write out the bivariate normal density. The multivariate normal density is defined by the following
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationStatistics 3657 : Moment Generating Functions
Statistics 3657 : Moment Generating Functions A useful tool for studying sums of independent random variables is generating functions. course we consider moment generating functions. In this Definition
More informationStat 206: Linear algebra
Stat 206: Linear algebra James Johndrow (adapted from Iain Johnstone s notes) 2016-11-02 Vectors We have already been working with vectors, but let s review a few more concepts. The inner product of two
More informationStatistical Pattern Recognition
Statistical Pattern Recognition A Brief Mathematical Review Hamid R. Rabiee Jafar Muhammadi, Ali Jalali, Alireza Ghasemi Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda Probability theory
More informationCHAPTER 4 MATHEMATICAL EXPECTATION. 4.1 Mean of a Random Variable
CHAPTER 4 MATHEMATICAL EXPECTATION 4.1 Mean of a Random Variable The expected value, or mathematical expectation E(X) of a random variable X is the long-run average value of X that would emerge after a
More informationMATH4210 Financial Mathematics ( ) Tutorial 7
MATH40 Financial Mathematics (05-06) Tutorial 7 Review of some basic Probability: The triple (Ω, F, P) is called a probability space, where Ω denotes the sample space and F is the set of event (σ algebra
More information