Joint Gaussian Graphical Model Review Series I
|
|
|
- Augustine Sharp
- 5 years ago
- Views:
Transcription
1 Joint Gaussian Graphical Model Review Series I Probability Foundations Beilun Wang Advisor: Yanjun Qi 1 Department of Computer Science, University of Virginia June 23rd, 2017 Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
2 Outline 1 Notation 2 Probability 3 Dependence and Correlation 4 Conditional Dependence and Partial Correlation Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
3 Notation Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
4 Notation P The probability measure. Ω The sample space. F The event set. X, Y, Z The random variables. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
5 Probability Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
6 Probability Space Probability Space Let (Ω, F, P) be the probability space. Ω be an arbitrary non-empty set. F 2 Ω is a set of events. P is the probability measure. In another word, a function : F [0, 1]. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
7 Events F contains Ω. F is closed under complements. F is closed under countable unions. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
8 Probability Measure Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
9 Random Variable Random Variable Let X : Ω R be a random variable. X is a measurable function. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
10 Random Variable Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
11 Probability Distribution Probability Distribution function Let F (x) : R [0, 1] = P[X < x] where x R. X = Y, they follow same distribution? F X = F Y, then X = Y? Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
12 Joint Probability Joint Probability The probability distribution of random vector (X, Y ). Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
13 Joint Probability Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
14 Marginal Probability Marginal Probability A pair of random variable (X, Y ), the probability distribution of X. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
15 Joint Probability Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
16 Conditional Distribution Conditional Distribution Given the information of Y, the probability distribution of X. Notation X Y. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
17 Joint Probability Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
18 Relationship Relationship P(X = x, Y = y) = P(Y = y)p(x = x Y = y) Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
19 Dependence and Correlation Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
20 Independence Independence X and Y are independent if and only if p X,Y (x, y) = p X (x)p Y (y), where p is the probability density function. Independence Y X = Y Filp coin example Causal relationship Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
21 Correlation Covariance Cov(X, Y ) = E[(X µ X )(Y µ Y )], where µ X, µ Y is the mean vector. Correlation ρ(x, Y ) = Cov(X,Y ) σ X σ Y Linear relationship Linear dependency between X and Y. ρ(x, Y ) = 1 means that X and Y are in the same linear direction while ρ(x, Y ) = 1 means that X and Y are in the reverse linear direction. ρ(x, Y ) = 1 means that when X increase, Y increase with all the points lying on the same line. ρ(x, Y ) = 0 means that X and Y are perpendicular with each other. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
22 Correlation Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
23 Dependence and Correlation Correlation is easy to estimate the value while independence is a relationship to infer. Dependence is stronger relationship than correlation. In another word, if X and Y are independent, ρ(x, Y ) = 0. However, the reverse doesn t hold. For example, suppose the random variable X is symmetrically distributed about zero and Y = X 2. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
24 Gaussian Example The distribution of bivariate Gaussian is: ( ( 1 f (x, y) = 2πσ X σ exp 1 (x Y 1 ρ 2 2(1 ρ 2 ) µx ) 2 σ 2 X + (y µ Y ) 2 σ 2 Y (3.1) Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
25 Gaussian Example Suppose (X, Y ) are uncorrelated. i.e.,(x, Y ) N(0, diag(σ 2 X, σ2 Y )). f (x, y) = = 1 exp( 1 2πσ X σ Y 2 ((x µ X ) 2 σx 2 (x µ X ) 2 1 2πσX exp( 1 2 = f (x)f (y) σ 2 X + (y µ Y ) 2 σy 2 )) 1 ) exp( 1 (y µ Y ) 2 ) 2πσY 2 σ 2 Y (3.2) Therefore, if (X, Y ) follows bivariate Gaussian, (X, Y ) are uncorrelated if and only if (X, Y ) are independent. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
26 Summary Correlation is easy to estimate the value while independence is a relationship to infer. In the Gaussian Case, they are equivalent. From the structure learning angle, dependence is about the causal relationship, while correlation is, more specifically, the linear relationship. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
27 Conditional Dependence and Partial Correlation Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
28 Conditional Dependence Let s consider a more complicated case. There is another third random variable Z. There are two ways to view the conditional dependence. X and Y are independent conditional on Z X Z and Y Z are independent Conditional Dependence X and Y are independent on Z if and only if p X,Y Z (x, y) = p X Z (x)p Y Z (y), where p is the probability density function. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
29 Partial Correlation Partial Correlation Formally, the partial correlation between X and Y given random variable Z, written ρ XY Z, is the correlation between the residuals R X and R Y resulting from the linear regression of X with Z and of Y with Z, respectively. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
30 Partial Correlation Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
31 Partial Correlation Partial Correlation Calculation Suppose P = Σ 1 (Σ is covariance matrix or Correlation matrix) ρ Xi X j V\{X i,x j } = p ij pii p jj. The value is exactly related to the precision matrix (the inverse of covariance matrix)! Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
32 Conditional Dependence and Partial Correlation Similarly, in the Gaussian Case, they are equivalent. A detailed derivation is in the next talk. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
33 Gaussian Case Partial Correlation is easy to estimate the value while conditional independence is a relationship to infer. Conditional Dependence is stronger relationship than partial correlation. In another word, if X Z and Y Z are independent, ρ(x, Y Z) = 0. However, the reverse doesn t hold. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
34 Summary Partial correlation is easy to estimate the value while conditional independence is a relationship to infer. In the Gaussian Case, they are equivalent. From the structure learning angle, conditional dependence is about the causal relationship, while partial correlation is, more specifically, the linear relationship. Beilun Wang, Advisor: Yanjun Qi (UniversityJoint of Virginia) Gaussian Graphical Model Review Series I June 23rd, / 34
Joint Gaussian Graphical Model Review Series I
Joint Gaussian Graphical Model Review Series I Probability Foundations Beilun Wang Advisor: Yanjun Qi 1 Department of Computer Science, University of Virginia http://jointggm.org/ June 23rd, 2017 Beilun
Introduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
Introduction to Normal Distribution
Introduction to Normal Distribution Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 17-Jan-2017 Nathaniel E. Helwig (U of Minnesota) Introduction
Multivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
Covariance and Correlation
Covariance and Correlation ST 370 The probability distribution of a random variable gives complete information about its behavior, but its mean and variance are useful summaries. Similarly, the joint probability
Algorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
Lecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
Multiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
01 Probability Theory and Statistics Review
NAVARCH/EECS 568, ROB 530 - Winter 2018 01 Probability Theory and Statistics Review Maani Ghaffari January 08, 2018 Last Time: Bayes Filters Given: Stream of observations z 1:t and action data u 1:t Sensor/measurement
More than one variable
Chapter More than one variable.1 Bivariate discrete distributions Suppose that the r.v. s X and Y are discrete and take on the values x j and y j, j 1, respectively. Then the joint p.d.f. of X and Y, to
5 Operations on Multiple Random Variables
EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y
Covariance. Lecture 20: Covariance / Correlation & General Bivariate Normal. Covariance, cont. Properties of Covariance
Covariance Lecture 0: Covariance / Correlation & General Bivariate Normal Sta30 / Mth 30 We have previously discussed Covariance in relation to the variance of the sum of two random variables Review Lecture
EE4601 Communication Systems
EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two
P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n
JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are
Correlation analysis. Contents
Correlation analysis Contents 1 Correlation analysis 2 1.1 Distribution function and independence of random variables.......... 2 1.2 Measures of statistical links between two random variables...........
conditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
ENGG2430A-Homework 2
ENGG3A-Homework Due on Feb 9th,. Independence vs correlation a For each of the following cases, compute the marginal pmfs from the joint pmfs. Explain whether the random variables X and Y are independent,
Let X and Y denote two random variables. The joint distribution of these random
EE385 Class Notes 9/7/0 John Stensby Chapter 3: Multiple Random Variables Let X and Y denote two random variables. The joint distribution of these random variables is defined as F XY(x,y) = [X x,y y] P.
18 Bivariate normal distribution I
8 Bivariate normal distribution I 8 Example Imagine firing arrows at a target Hopefully they will fall close to the target centre As we fire more arrows we find a high density near the centre and fewer
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
![EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as](/thumbs/85/92409620.jpg "EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as")
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
18.440: Lecture 26 Conditional expectation
18.440: Lecture 26 Conditional expectation Scott Sheffield MIT 1 Outline Conditional probability distributions Conditional expectation Interpretation and examples 2 Outline Conditional probability distributions
Introduction to Computational Finance and Financial Econometrics Probability Review - Part 2
You can t see this text! Introduction to Computational Finance and Financial Econometrics Probability Review - Part 2 Eric Zivot Spring 2015 Eric Zivot (Copyright 2015) Probability Review - Part 2 1 /
x. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
Review of Statistics
Review of Statistics Topics Descriptive Statistics Mean, Variance Probability Union event, joint event Random Variables Discrete and Continuous Distributions, Moments Two Random Variables Covariance and
Lecture 11. Probability Theory: an Overveiw
Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the
Introduction to Computational Finance and Financial Econometrics Matrix Algebra Review
You can t see this text! Introduction to Computational Finance and Financial Econometrics Matrix Algebra Review Eric Zivot Spring 2015 Eric Zivot (Copyright 2015) Matrix Algebra Review 1 / 54 Outline 1
STAT/MATH 395 PROBABILITY II
STAT/MATH 395 PROBABILITY II Bivariate Distributions Néhémy Lim University of Washington Winter 2017 Outline Distributions of Two Random Variables Distributions of Two Discrete Random Variables Distributions
LIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
Bivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
Review of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities
PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets
Definition of a Stochastic Process
Definition of a Stochastic Process Balu Santhanam Dept. of E.C.E., University of New Mexico Fax: 505 277 8298 bsanthan@unm.edu August 26, 2018 Balu Santhanam (UNM) August 26, 2018 1 / 20 Overview 1 Stochastic
EE 302: Probabilistic Methods in Electrical Engineering
EE : Probabilistic Methods in Electrical Engineering Print Name: Solution (//6 --sk) Test II : Chapters.5 4 //98, : PM Write down your name on each paper. Read every question carefully and solve each problem
Two hours. Statistical Tables to be provided THE UNIVERSITY OF MANCHESTER. 14 January :45 11:45
Two hours Statistical Tables to be provided THE UNIVERSITY OF MANCHESTER PROBABILITY 2 14 January 2015 09:45 11:45 Answer ALL four questions in Section A (40 marks in total) and TWO of the THREE questions
Lecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
Joint probability distributions: Discrete Variables. Two Discrete Random Variables. Example 1. Example 1
Joint probability distributions: Discrete Variables Two Discrete Random Variables Probability mass function (pmf) of a single discrete random variable X specifies how much probability mass is placed on
Lecture 16 : Independence, Covariance and Correlation of Discrete Random Variables
Lecture 6 : Independence, Covariance and Correlation of Discrete Random Variables 0/ 3 Definition Two discrete random variables X and Y defined on the same sample space are said to be independent if for
Continuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
Joint Distributions. (a) Scalar multiplication: k = c d. (b) Product of two matrices: c d. (c) The transpose of a matrix:
Joint Distributions Joint Distributions A bivariate normal distribution generalizes the concept of normal distribution to bivariate random variables It requires a matrix formulation of quadratic forms,
Appendix A : Introduction to Probability and stochastic processes
A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of
STOR Lecture 16. Properties of Expectation - I
STOR 435.001 Lecture 16 Properties of Expectation - I Jan Hannig UNC Chapel Hill 1 / 22 Motivation Recall we found joint distributions to be pretty complicated objects. Need various tools from combinatorics
Multivariate probability distributions and linear regression
Multivariate probability distributions and linear regression Patrik Hoyer 1 Contents: Random variable, probability distribution Joint distribution Marginal distribution Conditional distribution Independence,
[POLS 8500] Review of Linear Algebra, Probability and Information Theory
![[POLS 8500] Review of Linear Algebra, Probability and Information Theory](/thumbs/72/67586683.jpg "[POLS 8500] Review of Linear Algebra, Probability and Information Theory")
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
Bivariate Paired Numerical Data
Bivariate Paired Numerical Data Pearson s correlation, Spearman s ρ and Kendall s τ, tests of independence University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html
Math Review Sheet, Fall 2008
1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the
Chapter 4. Multivariate Distributions. Obviously, the marginal distributions may be obtained easily from the joint distribution:
4.1 Bivariate Distributions. Chapter 4. Multivariate Distributions For a pair r.v.s (X,Y ), the Joint CDF is defined as F X,Y (x, y ) = P (X x,y y ). Obviously, the marginal distributions may be obtained
matrix-free Elements of Probability Theory 1 Random Variables and Distributions Contents Elements of Probability Theory 2
Short Guides to Microeconometrics Fall 2018 Kurt Schmidheiny Unversität Basel Elements of Probability Theory 2 1 Random Variables and Distributions Contents Elements of Probability Theory matrix-free 1
Statistics, Data Analysis, and Simulation SS 2015
Statistics, Data Analysis, and Simulation SS 2015 08.128.730 Statistik, Datenanalyse und Simulation Dr. Michael O. Distler Mainz, 27. April 2015 Dr. Michael O. Distler
Review: mostly probability and some statistics
Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random
Joint Probability Distributions and Random Samples (Devore Chapter Five)
Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 1 Joint Probability Distributions 2 1.1 Two Discrete
Statistics 351 Probability I Fall 2006 (200630) Final Exam Solutions. θ α β Γ(α)Γ(β) (uv)α 1 (v uv) β 1 exp v }
Statistics 35 Probability I Fall 6 (63 Final Exam Solutions Instructor: Michael Kozdron (a Solving for X and Y gives X UV and Y V UV, so that the Jacobian of this transformation is x x u v J y y v u v
Basics on Probability. Jingrui He 09/11/2007
Basics on Probability Jingrui He 09/11/2007 Coin Flips You flip a coin Head with probability 0.5 You flip 100 coins How many heads would you expect Coin Flips cont. You flip a coin Head with probability
UCSD ECE153 Handout #34 Prof. Young-Han Kim Tuesday, May 27, Solutions to Homework Set #6 (Prepared by TA Fatemeh Arbabjolfaei)
UCSD ECE53 Handout #34 Prof Young-Han Kim Tuesday, May 7, 04 Solutions to Homework Set #6 (Prepared by TA Fatemeh Arbabjolfaei) Linear estimator Consider a channel with the observation Y XZ, where the
18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 18-27 Review Scott Sheffield MIT Outline Outline It s the coins, stupid Much of what we have done in this course can be motivated by the i.i.d. sequence X i where each X i is
2. Variance and Covariance: We will now derive some classic properties of variance and covariance. Assume real-valued random variables X and Y.
CS450 Final Review Problems Fall 08 Solutions or worked answers provided Problems -6 are based on the midterm review Identical problems are marked recap] Please consult previous recitations and textbook
Chapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
Solutions to Homework Set #6 (Prepared by Lele Wang)
Solutions to Homework Set #6 (Prepared by Lele Wang) Gaussian random vector Given a Gaussian random vector X N (µ, Σ), where µ ( 5 ) T and 0 Σ 4 0 0 0 9 (a) Find the pdfs of i X, ii X + X 3, iii X + X
18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 17-27 Review Scott Sheffield MIT 1 Outline Continuous random variables Problems motivated by coin tossing Random variable properties 2 Outline Continuous random variables Problems
Preliminary statistics
1 Preliminary statistics The solution of a geophysical inverse problem can be obtained by a combination of information from observed data, the theoretical relation between data and earth parameters (models),
ECON 3150/4150, Spring term Lecture 6
ECON 3150/4150, Spring term 2013. Lecture 6 Review of theoretical statistics for econometric modelling (II) Ragnar Nymoen University of Oslo 31 January 2013 1 / 25 References to Lecture 3 and 6 Lecture
Lecture 14: Multivariate mgf s and chf s
Lecture 14: Multivariate mgf s and chf s Multivariate mgf and chf For an n-dimensional random vector X, its mgf is defined as M X (t) = E(e t X ), t R n and its chf is defined as φ X (t) = E(e ıt X ),
Probability Review. Chao Lan
Probability Review Chao Lan Let s start with a single random variable Random Experiment A random experiment has three elements 1. sample space Ω: set of all possible outcomes e.g.,ω={1,2,3,4,5,6} 2. event
Joint p.d.f. and Independent Random Variables
1 Joint p.d.f. and Independent Random Variables Let X and Y be two discrete r.v. s and let R be the corresponding space of X and Y. The joint p.d.f. of X = x and Y = y, denoted by f(x, y) = P(X = x, Y
Elements of Probability Theory
Short Guides to Microeconometrics Fall 2016 Kurt Schmidheiny Unversität Basel Elements of Probability Theory Contents 1 Random Variables and Distributions 2 1.1 Univariate Random Variables and Distributions......
1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
Simulation - Lectures - Part III Markov chain Monte Carlo
Simulation - Lectures - Part III Markov chain Monte Carlo Julien Berestycki Part A Simulation and Statistical Programming Hilary Term 2018 Part A Simulation. HT 2018. J. Berestycki. 1 / 50 Outline Markov
E. Vitali Lecture 5. Conditional Expectation
So far we have focussed on independent random variables, which are well suitable to deal with statistical inference, allowing to exploit the very powerful central limit theorem. However, for the purpose
Lecture Note 1: Probability Theory and Statistics
Univ. of Michigan - NAME 568/EECS 568/ROB 530 Winter 2018 Lecture Note 1: Probability Theory and Statistics Lecturer: Maani Ghaffari Jadidi Date: April 6, 2018 For this and all future notes, if you would
B4 Estimation and Inference
B4 Estimation and Inference 6 Lectures Hilary Term 27 2 Tutorial Sheets A. Zisserman Overview Lectures 1 & 2: Introduction sensors, and basics of probability density functions for representing sensor error
4 Pairs of Random Variables
B.Sc./Cert./M.Sc. Qualif. - Statistical Theory 4 Pairs of Random Variables 4.1 Introduction In this section, we consider a pair of r.v. s X, Y on (Ω, F, P), i.e. X, Y : Ω R. More precisely, we define a
a b = a T b = a i b i (1) i=1 (Geometric definition) The dot product of two Euclidean vectors a and b is defined by a b = a b cos(θ a,b ) (2)
This is my preperation notes for teaching in sections during the winter 2018 quarter for course CSE 446. Useful for myself to review the concepts as well. More Linear Algebra Definition 1.1 (Dot Product).
(3) Review of Probability. ST440/540: Applied Bayesian Statistics
Review of probability The crux of Bayesian statistics is to compute the posterior distribution, i.e., the uncertainty distribution of the parameters (θ) after observing the data (Y) This is the conditional
A Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
MATH c UNIVERSITY OF LEEDS Examination for the Module MATH2715 (January 2015) STATISTICAL METHODS. Time allowed: 2 hours
MATH2750 This question paper consists of 8 printed pages, each of which is identified by the reference MATH275. All calculators must carry an approval sticker issued by the School of Mathematics. c UNIVERSITY
1 Random variables and distributions
Random variables and distributions In this chapter we consider real valued functions, called random variables, defined on the sample space. X : S R X The set of possible values of X is denoted by the set
CSC 411: Lecture 09: Naive Bayes
CSC 411: Lecture 09: Naive Bayes Class based on Raquel Urtasun & Rich Zemel s lectures Sanja Fidler University of Toronto Feb 8, 2015 Urtasun, Zemel, Fidler (UofT) CSC 411: 09-Naive Bayes Feb 8, 2015 1
Recitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
FE 5204 Stochastic Differential Equations
Instructor: Jim Zhu e-mail:zhu@wmich.edu http://homepages.wmich.edu/ zhu/ January 20, 2009 Preliminaries for dealing with continuous random processes. Brownian motions. Our main reference for this lecture
UQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables
UQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables To be provided to students with STAT2201 or CIVIL-2530 (Probability and Statistics) Exam Main exam date: Tuesday, 20 June 1
MATH 829: Introduction to Data Mining and Analysis Graphical Models II - Gaussian Graphical Models
1/13 MATH 829: Introduction to Data Mining and Analysis Graphical Models II - Gaussian Graphical Models Dominique Guillot Departments of Mathematical Sciences University of Delaware May 4, 2016 Recall
Stochastic process for macro
Stochastic process for macro Tianxiao Zheng SAIF 1. Stochastic process The state of a system {X t } evolves probabilistically in time. The joint probability distribution is given by Pr(X t1, t 1 ; X t2,
Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
Error propagation. Alexander Khanov. October 4, PHYS6260: Experimental Methods is HEP Oklahoma State University
Error propagation Alexander Khanov PHYS660: Experimental Methods is HEP Oklahoma State University October 4, 017 Why error propagation? In many cases we measure one thing and want to know something else
Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
Data Fitting - Lecture 6
1 Central limit theorem Data Fitting - Lecture 6 Recall that for a sequence of independent, random variables, X i, one can define a mean, µ i, and a variance σi. then the sum of the means also forms a
Lecture Notes on the Gaussian Distribution
Lecture Notes on the Gaussian Distribution Hairong Qi The Gaussian distribution is also referred to as the normal distribution or the bell curve distribution for its bell-shaped density curve. There s
Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016
8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying
3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
TAMS39 Lecture 2 Multivariate normal distribution
TAMS39 Lecture 2 Multivariate normal distribution Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content Lecture Random vectors Multivariate normal distribution
Discrete Probability Refresher
ECE 1502 Information Theory Discrete Probability Refresher F. R. Kschischang Dept. of Electrical and Computer Engineering University of Toronto January 13, 1999 revised January 11, 2006 Probability theory
Chapter 4 continued. Chapter 4 sections
Chapter 4 sections Chapter 4 continued 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP:
Exercises with solutions (Set D)
Exercises with solutions Set D. A fair die is rolled at the same time as a fair coin is tossed. Let A be the number on the upper surface of the die and let B describe the outcome of the coin toss, where
Random vectors X 1 X 2. Recall that a random vector X = is made up of, say, k. X k. random variables.
Random vectors Recall that a random vector X = X X 2 is made up of, say, k random variables X k A random vector has a joint distribution, eg a density f(x), that gives probabilities P(X A) = f(x)dx Just
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics)
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming
Jointly Distributed Random Variables
Jointly Distributed Random Variables CE 311S What if there is more than one random variable we are interested in? How should you invest the extra money from your summer internship? To simplify matters,
Formulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms