Multivariate probability distributions and linear regression
|
|
- Kathleen Wheeler
- 5 years ago
- Views:
Transcription
1 Multivariate probability distributions and linear regression Patrik Hoyer 1
2 Contents: Random variable, probability distribution Joint distribution Marginal distribution Conditional distribution Independence, conditional independence Generating data Multivariate Gaussian distribution Multivariate linear regression Expectation, variance, covariance, correlation Estimating a distribution from sample data Patrik Hoyer 2
3 Random variable - sample space (set of possible elementary outcomes) - probability distribution over sample space Examples: - The throw of a die x P (x) /6 1/6 1/6 1/6 1/6 1/6 - The sum of two dice x P (x) /36 1/18 1/12 1/9 5/36 1/6 5/36 1/9 1/12 1/18 1/36 - Two separate dice (red, blue) x P (x) (1,1) (1,2) (1,3) (1,4) (1,5) (1,6) (2,1) (2,2) (2,3)... (6,6) 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 1/36 Patrik Hoyer 3
4 Discrete variables: - Finite number of states (e.g. dice examples) - Infinite number of states (e.g. how many heads before one tales in a sequence of coin tosses?) Continuous variables: Each particular state has a probability of zero, so we need the concept of a probability density: P (X x) = x p(t) dt (e.g. how long time until next bus arrives? what will be the price of oil a year from now?) Patrik Hoyer 4
5 A probability distribution satisfies Probabilities are non-negative: P (X = x) = P X (x) = P (x) 0 2. Sum to one: P (x) = 1 x p(x) dx = 1 (discrete) (continuous) [Note that in the discrete case this means that there exists no value of x such that P (x) > 1. However this does not in general hold for a continuous density p(x)!] Patrik Hoyer 5
6 The joint distribution of two random variables: - Let X and Y be random variables. Their joint distribution is P (x, y) = P (X = x and Y = y) - Example: Two coin tosses, X denotes first throw, Y denotes second (note: independence!) P (x, y) : X H T H Y T Example: X: Rain today? Y : Rain tomorrow? P (x, y) : X Y N Y Y N Patrik Hoyer 6
7 Marginal distribution: - Interested in or observing only one of the two variables - The distribution is obtained by summing (or integrating) over the other variable: P (x) = y P (x, y) p(x) = - Example (continued): What is the probability of rain tomorrow? That is, what is P (y)? p(x, y) dy X Y N Y Y N In the same fashion, we can calculate that the chance of rain today is 0.7. P (y) : Patrik Hoyer 7
8 Conditional distribution: - If we observe X = x how does that affect our belief about the value of Y? - Obtained by selecting the appropriate row/column of the joint distribution, and renormalizing it to sum to one: P (x, y) p(x, y) P (y X = x) = P (y x) = p(y x) = P (x) p(x) - Example (continued): What is the probability that tomorrow rains, given that today does not rain? i.e. what is P (y X = no rain )? Y Y N P (y X = no rain ) X Y N / ( ) Y N Patrik Hoyer 8
9 Chain rule: P (x, y) = P (x)p (y x) = P (y)p (x y) p(x, y) = p(x)p(y x) = p(y)p(x y) So the joint distribution can be specified directly, or using the marginal and conditional distribution (can even choose, which way one specifies it) Patrik Hoyer 9
10 Independence: Two random variables are independent, if and only if knowing the value of one does not change our belief about the second: x : P (y x) = P (y) y : P (x y) = P (x) This is equivalent to being able to write the joint distribution as the product of the marginals: P (x, y) = P (x)p (y) We write this as: X Y or, if we want to explicitly specify the distribution: (X Y ) P Example: Two coin tosses... Patrik Hoyer 10
11 Three or more variables: - joint distribution: ( multidimensional array/function ) - marginal distributions: (e.g.) P (x) = P (x, y) = v,w,y,z,... v,w,z,... P (v, w, x, y, z,...) P (v, w, x, y, z,...) P (v, w, x, y, z,...) - conditional distributions: (e.g.) P (x v, w, y, z,...) = P (v, w, x, y, z,...)/p (v, w, y, z,...) P (x, y v, w, z,...) = P (v, w, x, y, z,...)/p (v, w, z,...) P (v, w, y, z,... x) = P (v, w, x, y, z,...)/p (x) P (x y) = P (v, w, x, z,... y) v,w,z,... marginal and conditional Patrik Hoyer 11
12 - Chain rule P (v, w, x, y, z,...) = P (v)p (w v)p (x v, w)p (y v, w, x) P (z v, w, x, y)p (... v, w, x, y, z) - Complete independence between all variables if and only if: P (v, w, x, y, z,...) = P (v)p (w)p (x)p (y)p (z)p (...) - Conditional independence (e.g: if we know the value of z then does not give any additional information about y ): P (x, y z) = P (x z)p (y z) x This is also written: X Y Z or explicitly noting the distribution: (X Y Z) P Patrik Hoyer 12
13 - In general we can say that marginal distributions are conditional on not knowing the value of other variables: P (x) = P (x ) and (marginal) independence is independence conditional on not observing other variables: P (x, y ) = P (x )P (y ) - Example of conditional independence: Drownings and ice-cream sales. These are mutually dependent (both happen during warm weather) but are, at least approximately, conditionally independent given the weather Patrik Hoyer 13
14 Example: conditional dependence: Two coin tosses and a bell that rings whenever they get the same result. The coins are marginally independent but conditionally dependent given the bell! X : Y : Z : First coin toss Second coin toss Bell H Y T P (x, y) = X H T (independent) H Y T P (x, y Z = bell rang ) = X H T (dependent!) Patrik Hoyer 14
15 Data generation, sampling - Given some P (x), how can we draw samples (generate data) from that distribution? Answer: Divide the unit interval [0,1] into parts corresponding to the probabilities, draw a uniformly distributed number in the interval, and select the state into which we fell: X := x 2 P (x 1 ) P (x 2 ) P (x 4 ) P (x 6 ) P (x 3 ) P (x 5 ) Patrik Hoyer 15
16 Given a joint distribution P (x, y, z) (generate data)? - We could list all joint states, then proceed as above, or... - Draw data sequentially from conditional distributions: 1. First draw x from 2. Next y from 3. Finally from P (x) P (y x) z P (z x, y), how can we draw samples Note: We can freely choose any ordering of the variables! Patrik Hoyer 16
17 Example (continued): Two coin tosses and a bell that rings if and only if the two tosses give the same result - can draw all the variables simultaneously by listing all the joint states, calculating their probabilities, placing them on the unit interval, and then draw the joint state - can first independently generate the coin tosses, then assign the bell - can first draw one coin toss and the bell, and then assign the second coin toss Patrik Hoyer 17
18 Numerical random variables - Expectation: E{X} = x xp (x) (discrete) E{X} = x p(x) dx (continuous) - Variance: Var(X) = σx 2 = σ XX = E{(X E{X}) 2 } - Covariance: Cov(X, Y ) = σ XY = E{(X E{X})(Y E{Y })} - Correlation coefficient: ρ XY = σ XY σ 2 X σ 2 Y Patrik Hoyer 18
19 - Multivariate numerical random variables... (random vectors) Expectation: E{V} = E{V 1 } E{V 2 }. E{V N } Covariance matrix ( variance-covariance matrix ) C V = Σ V = E{(V E{V})(V E{V}) T } = Var(V 1 ) Cov(V 1, V N )... Cov(V N, V 1 ) Var(V 2 ) = σ V1 V 1 σ VN V 1... σ V1 V N σ VN V N Patrik Hoyer 19
20 Conditional expectation, variance, covariance, correlation - Conditional expectation (note: function of y!) E{X Y = y} = x xp (x y) (discrete) E{X Y = y} = x p(x y) dx (continuous) - Conditional variance (note: function of y!) Var(X Y = y) = σx y 2 = σ XX y = E{(X E{X}) 2 } P (X Y =y) - Conditional covariance (note: function of z!) Cov(X, Y z) = σ XY z = E{(X E{X})(Y E{Y })} P (X,Y Z=z) - Conditional correlation coefficient (note: function of z!) ρ XY z = σ XY z σ 2 X z σ2 Y z Patrik Hoyer 20
21 - Multivariate Gaussian ( normal ) density: p(x) = N (µ, Σ) = (2π) d/2 Σ 1/2 exp 1 2 (x µ)t Σ 1 (x µ) has the following properties: mean vector (µ, and covariance matrix Σ) as the only parameters x 2 x 1 Patrik Hoyer 21
22 all marginal and conditional distributions are also Gaussian, and the conditional (co)variances do not depend on the values of the conditioning variables: Let x and y be random vectors whose dimensions are n and m. If they are joined together into one random vector z = (x T, y T ), with dimension n + m, then its mean m z and covariance matrix C z are mx Cx C m z =, C z = xy, (1) C yx C y m y where m x and m y are the means of x and y, and C x and C y are the covariance matrices of x and y respectively, and C xy contains the cross covariances. If z is multivariate Gaussian then x and y are also Gaussian. Additionally the conditional distributions p(x y) and p(y x) are Gaussian. The latter s mean and covariance matrix are m y x = m y + C yx C 1 x (x m x ) (2) C y x = C y C yx C 1 x C xy (3) Patrik Hoyer 22
23 The conditional variance, conditional covariance, and conditional correlation coefficient, for the Gaussian distribution, are known as partial variance σx Z 2, partial covariance σ XY Z, and partial correlation coefficient ρ XY Z (respectively) These can of course always be computed directly from the covariance matrix (regardless of whether the distribution actually is Gaussian!)......but they can only be safely interpreted as conditional variance, conditional covariance, and conditional correlation coefficient (respectively) for the Gaussian distribution. Patrik Hoyer 23
24 for Gaussian: zero (partial) covariance zero (conditional) covariance (conditional) independence i.e. (σ XY Z = 0) ( z : σ XY z = 0) (X Y Z) in general: we only have one-way implication: zero (conditional) covariance (conditional) independence i.e. ( z : σ XY z = 0) (X Y Z) Note, however, that conditional independence does not imply zero partial covariance in the completely general case! Patrik Hoyer 24
25 Linear regression: ŷ = r yx x + y y Fit a line through the data, explaining how y varies with x. Minimize sum of squares error between ŷ and y. r yx = σ XY σx 2 Probabilistic interpretation: ŷ E{Y X = x} (note that this is true only for roughly linear relationships) x Patrik Hoyer 25
26 Note the symmetry: We could equally well regress x on y! ˆx = r xy y + x x y Patrik Hoyer 26
27 Multivariate linear regression: ẑ = ax + by + z a = r zx y = σ ZX Y σ 2 X Y x z y Note that the partial regression coefficient r zx y is NOT the same, in general, as one gets from regressing z on x, ignoring y : Note also that r zx y is derived from the partial (co)variances. This holds regardless of the form of the underlying distribution. r zx Patrik Hoyer 27
CS37300 Class Notes. Jennifer Neville, Sebastian Moreno, Bruno Ribeiro
CS37300 Class Notes Jennifer Neville, Sebastian Moreno, Bruno Ribeiro 2 Background on Probability and Statistics These are basic definitions, concepts, and equations that should have been covered in your
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationIntroduction to Machine Learning
What does this mean? Outline Contents Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola December 26, 2017 1 Introduction to Probability 1 2 Random Variables 3 3 Bayes
More informationSome Concepts of Probability (Review) Volker Tresp Summer 2018
Some Concepts of Probability (Review) Volker Tresp Summer 2018 1 Definition There are different way to define what a probability stands for Mathematically, the most rigorous definition is based on Kolmogorov
More informationMore than one variable
Chapter More than one variable.1 Bivariate discrete distributions Suppose that the r.v. s X and Y are discrete and take on the values x j and y j, j 1, respectively. Then the joint p.d.f. of X and Y, to
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More information1 Random variables and distributions
Random variables and distributions In this chapter we consider real valued functions, called random variables, defined on the sample space. X : S R X The set of possible values of X is denoted by the set
More informationReview of Probability. CS1538: Introduction to Simulations
Review of Probability CS1538: Introduction to Simulations Probability and Statistics in Simulation Why do we need probability and statistics in simulation? Needed to validate the simulation model Needed
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationBrief Review of Probability
Brief Review of Probability Nuno Vasconcelos (Ken Kreutz-Delgado) ECE Department, UCSD Probability Probability theory is a mathematical language to deal with processes or experiments that are non-deterministic
More informationSolutions to Homework Set #6 (Prepared by Lele Wang)
Solutions to Homework Set #6 (Prepared by Lele Wang) Gaussian random vector Given a Gaussian random vector X N (µ, Σ), where µ ( 5 ) T and 0 Σ 4 0 0 0 9 (a) Find the pdfs of i X, ii X + X 3, iii X + X
More information18.440: Lecture 19 Normal random variables
18.440 Lecture 19 18.440: Lecture 19 Normal random variables Scott Sheffield MIT Outline Tossing coins Normal random variables Special case of central limit theorem Outline Tossing coins Normal random
More informationIntroduction to Machine Learning
Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB
More information18.440: Lecture 26 Conditional expectation
18.440: Lecture 26 Conditional expectation Scott Sheffield MIT 1 Outline Conditional probability distributions Conditional expectation Interpretation and examples 2 Outline Conditional probability distributions
More informationENGG2430A-Homework 2
ENGG3A-Homework Due on Feb 9th,. Independence vs correlation a For each of the following cases, compute the marginal pmfs from the joint pmfs. Explain whether the random variables X and Y are independent,
More informationReview of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
More informationJointly Distributed Random Variables
Jointly Distributed Random Variables CE 311S What if there is more than one random variable we are interested in? How should you invest the extra money from your summer internship? To simplify matters,
More informationJoint Probability Distributions and Random Samples (Devore Chapter Five)
Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 1 Joint Probability Distributions 2 1.1 Two Discrete
More informationNotes for Math 324, Part 19
48 Notes for Math 324, Part 9 Chapter 9 Multivariate distributions, covariance Often, we need to consider several random variables at the same time. We have a sample space S and r.v. s X, Y,..., which
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More information01 Probability Theory and Statistics Review
NAVARCH/EECS 568, ROB 530 - Winter 2018 01 Probability Theory and Statistics Review Maani Ghaffari January 08, 2018 Last Time: Bayes Filters Given: Stream of observations z 1:t and action data u 1:t Sensor/measurement
More informationME 597: AUTONOMOUS MOBILE ROBOTICS SECTION 2 PROBABILITY. Prof. Steven Waslander
ME 597: AUTONOMOUS MOBILE ROBOTICS SECTION 2 Prof. Steven Waslander p(a): Probability that A is true 0 pa ( ) 1 p( True) 1, p( False) 0 p( A B) p( A) p( B) p( A B) A A B B 2 Discrete Random Variable X
More information2. Suppose (X, Y ) is a pair of random variables uniformly distributed over the triangle with vertices (0, 0), (2, 0), (2, 1).
Name M362K Final Exam Instructions: Show all of your work. You do not have to simplify your answers. No calculators allowed. There is a table of formulae on the last page. 1. Suppose X 1,..., X 1 are independent
More informationChapter 4. Chapter 4 sections
Chapter 4 sections 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP: 4.8 Utility Expectation
More informationLecture 4: Least Squares (LS) Estimation
ME 233, UC Berkeley, Spring 2014 Xu Chen Lecture 4: Least Squares (LS) Estimation Background and general solution Solution in the Gaussian case Properties Example Big picture general least squares estimation:
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationMULTIVARIATE PROBABILITY DISTRIBUTIONS
MULTIVARIATE PROBABILITY DISTRIBUTIONS. PRELIMINARIES.. Example. Consider an experiment that consists of tossing a die and a coin at the same time. We can consider a number of random variables defined
More informationLecture 2: Review of Probability
Lecture 2: Review of Probability Zheng Tian Contents 1 Random Variables and Probability Distributions 2 1.1 Defining probabilities and random variables..................... 2 1.2 Probability distributions................................
More informationProbability. Machine Learning and Pattern Recognition. Chris Williams. School of Informatics, University of Edinburgh. August 2014
Probability Machine Learning and Pattern Recognition Chris Williams School of Informatics, University of Edinburgh August 2014 (All of the slides in this course have been adapted from previous versions
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More informationMAS223 Statistical Inference and Modelling Exercises
MAS223 Statistical Inference and Modelling Exercises The exercises are grouped into sections, corresponding to chapters of the lecture notes Within each section exercises are divided into warm-up questions,
More informationChapter 5 continued. Chapter 5 sections
Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationReview of Basic Probability Theory
Review of Basic Probability Theory James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) 1 / 35 Review of Basic Probability Theory
More informationExpectation and Variance
Expectation and Variance August 22, 2017 STAT 151 Class 3 Slide 1 Outline of Topics 1 Motivation 2 Expectation - discrete 3 Transformations 4 Variance - discrete 5 Continuous variables 6 Covariance STAT
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationRandom Variables and Expectations
Inside ECOOMICS Random Variables Introduction to Econometrics Random Variables and Expectations A random variable has an outcome that is determined by an experiment and takes on a numerical value. A procedure
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Financial Econometrics / 49
State-space Model Eduardo Rossi University of Pavia November 2013 Rossi State-space Model Financial Econometrics - 2013 1 / 49 Outline 1 Introduction 2 The Kalman filter 3 Forecast errors 4 State smoothing
More informationACM 116: Lectures 3 4
1 ACM 116: Lectures 3 4 Joint distributions The multivariate normal distribution Conditional distributions Independent random variables Conditional distributions and Monte Carlo: Rejection sampling Variance
More informationDept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 34 To start out the course, we need to know something about statistics and This is only an introduction; for a fuller understanding, you would
More informationCovariance and Correlation
Covariance and Correlation ST 370 The probability distribution of a random variable gives complete information about its behavior, but its mean and variance are useful summaries. Similarly, the joint probability
More informationLecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
More informationHW5 Solutions. (a) (8 pts.) Show that if two random variables X and Y are independent, then E[XY ] = E[X]E[Y ] xy p X,Y (x, y)
HW5 Solutions 1. (50 pts.) Random homeworks again (a) (8 pts.) Show that if two random variables X and Y are independent, then E[XY ] = E[X]E[Y ] Answer: Applying the definition of expectation we have
More informationV7 Foundations of Probability Theory
V7 Foundations of Probability Theory Probability : degree of confidence that an event of an uncertain nature will occur. Events : we will assume that there is an agreed upon space of possible outcomes
More informationFINAL EXAM: Monday 8-10am
ECE 30: Probabilistic Methods in Electrical and Computer Engineering Fall 016 Instructor: Prof. A. R. Reibman FINAL EXAM: Monday 8-10am Fall 016, TTh 3-4:15pm (December 1, 016) This is a closed book exam.
More informationNotes on Mathematics Groups
EPGY Singapore Quantum Mechanics: 2007 Notes on Mathematics Groups A group, G, is defined is a set of elements G and a binary operation on G; one of the elements of G has particularly special properties
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationM378K In-Class Assignment #1
The following problems are a review of M6K. M7K In-Class Assignment # Problem.. Complete the definition of mutual exclusivity of events below: Events A, B Ω are said to be mutually exclusive if A B =.
More informationReview of probability. Nuno Vasconcelos UCSD
Review of probability Nuno Vasconcelos UCSD robability probability is the language to deal with processes that are non-deterministic examples: if I flip a coin 00 times how many can I expect to see heads?
More informationRandom Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.
Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationSummary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016
8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else
ECE 450 Homework #3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3 4 5
More informationAppendix A : Introduction to Probability and stochastic processes
A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of
More informationProperties of Random Variables
Properties of Random Variables 1 Definitions A discrete random variable is defined by a probability distribution that lists each possible outcome and the probability of obtaining that outcome If the random
More informationECE 541 Stochastic Signals and Systems Problem Set 9 Solutions
ECE 541 Stochastic Signals and Systems Problem Set 9 Solutions Problem Solutions : Yates and Goodman, 9.5.3 9.1.4 9.2.2 9.2.6 9.3.2 9.4.2 9.4.6 9.4.7 and Problem 9.1.4 Solution The joint PDF of X and Y
More informationStatistics for Managers Using Microsoft Excel/SPSS Chapter 4 Basic Probability And Discrete Probability Distributions
Statistics for Managers Using Microsoft Excel/SPSS Chapter 4 Basic Probability And Discrete Probability Distributions 1999 Prentice-Hall, Inc. Chap. 4-1 Chapter Topics Basic Probability Concepts: Sample
More information6.041/6.431 Fall 2010 Quiz 2 Solutions
6.04/6.43: Probabilistic Systems Analysis (Fall 200) 6.04/6.43 Fall 200 Quiz 2 Solutions Problem. (80 points) In this problem: (i) X is a (continuous) uniform random variable on [0, 4]. (ii) Y is an exponential
More information1 Basic continuous random variable problems
Name M362K Final Here are problems concerning material from Chapters 5 and 6. To review the other chapters, look over previous practice sheets for the two exams, previous quizzes, previous homeworks and
More information. Find E(V ) and var(v ).
Math 6382/6383: Probability Models and Mathematical Statistics Sample Preliminary Exam Questions 1. A person tosses a fair coin until she obtains 2 heads in a row. She then tosses a fair die the same number
More informationEcon 371 Problem Set #1 Answer Sheet
Econ 371 Problem Set #1 Answer Sheet 2.1 In this question, you are asked to consider the random variable Y, which denotes the number of heads that occur when two coins are tossed. a. The first part of
More informationReview of probability
Review of probability Computer Sciences 760 Spring 2014 http://pages.cs.wisc.edu/~dpage/cs760/ Goals for the lecture you should understand the following concepts definition of probability random variables
More informationE X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.
E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,
More informationRandom variables (discrete)
Random variables (discrete) Saad Mneimneh 1 Introducing random variables A random variable is a mapping from the sample space to the real line. We usually denote the random variable by X, and a value that
More informationNotes 12 Autumn 2005
MAS 08 Probability I Notes Autumn 005 Conditional random variables Remember that the conditional probability of event A given event B is P(A B) P(A B)/P(B). Suppose that X is a discrete random variable.
More informationJoint Distribution of Two or More Random Variables
Joint Distribution of Two or More Random Variables Sometimes more than one measurement in the form of random variable is taken on each member of the sample space. In cases like this there will be a few
More informationSTAT 430/510 Probability Lecture 7: Random Variable and Expectation
STAT 430/510 Probability Lecture 7: Random Variable and Expectation Pengyuan (Penelope) Wang June 2, 2011 Review Properties of Probability Conditional Probability The Law of Total Probability Bayes Formula
More informationMath 416 Lecture 3. The average or mean or expected value of x 1, x 2, x 3,..., x n is
Math 416 Lecture 3 Expected values The average or mean or expected value of x 1, x 2, x 3,..., x n is x 1 x 2... x n n x 1 1 n x 2 1 n... x n 1 n 1 n x i p x i where p x i 1 n is the probability of x i
More informationProbability Review. Chao Lan
Probability Review Chao Lan Let s start with a single random variable Random Experiment A random experiment has three elements 1. sample space Ω: set of all possible outcomes e.g.,ω={1,2,3,4,5,6} 2. event
More informationDiscrete Random Variables
CPSC 53 Systems Modeling and Simulation Discrete Random Variables Dr. Anirban Mahanti Department of Computer Science University of Calgary mahanti@cpsc.ucalgary.ca Random Variables A random variable is
More informationChapter 5. Chapter 5 sections
1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationSTAT 516 Midterm Exam 3 Friday, April 18, 2008
STAT 56 Midterm Exam 3 Friday, April 8, 2008 Name Purdue student ID (0 digits). The testing booklet contains 8 questions. 2. Permitted Texas Instruments calculators: BA-35 BA II Plus BA II Plus Professional
More informationIAM 530 ELEMENTS OF PROBABILITY AND STATISTICS LECTURE 3-RANDOM VARIABLES
IAM 530 ELEMENTS OF PROBABILITY AND STATISTICS LECTURE 3-RANDOM VARIABLES VARIABLE Studying the behavior of random variables, and more importantly functions of random variables is essential for both the
More information2. Variance and Covariance: We will now derive some classic properties of variance and covariance. Assume real-valued random variables X and Y.
CS450 Final Review Problems Fall 08 Solutions or worked answers provided Problems -6 are based on the midterm review Identical problems are marked recap] Please consult previous recitations and textbook
More informationExercises with solutions (Set D)
Exercises with solutions Set D. A fair die is rolled at the same time as a fair coin is tossed. Let A be the number on the upper surface of the die and let B describe the outcome of the coin toss, where
More informationLecture 1: Bayesian Framework Basics
Lecture 1: Bayesian Framework Basics Melih Kandemir melih.kandemir@iwr.uni-heidelberg.de April 21, 2014 What is this course about? Building Bayesian machine learning models Performing the inference of
More informationBASICS OF PROBABILITY
October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,
More informationHomework 5 Solutions
126/DCP126 Probability, Fall 214 Instructor: Prof. Wen-Guey Tzeng Homework 5 Solutions 5-Jan-215 1. Let the joint probability mass function of discrete random variables X and Y be given by { c(x + y) ifx
More information1.1 Review of Probability Theory
1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationECE Homework Set 3
ECE 450 1 Homework Set 3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3
More informationChapter 2. Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Chapter 2 Some Basic Probability Concepts 2.1 Experiments, Outcomes and Random Variables A random variable is a variable whose value is unknown until it is observed. The value of a random variable results
More informationCS 630 Basic Probability and Information Theory. Tim Campbell
CS 630 Basic Probability and Information Theory Tim Campbell 21 January 2003 Probability Theory Probability Theory is the study of how best to predict outcomes of events. An experiment (or trial or event)
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationECON Fundamentals of Probability
ECON 351 - Fundamentals of Probability Maggie Jones 1 / 32 Random Variables A random variable is one that takes on numerical values, i.e. numerical summary of a random outcome e.g., prices, total GDP,
More informationMATH 151, FINAL EXAM Winter Quarter, 21 March, 2014
Time: 3 hours, 8:3-11:3 Instructions: MATH 151, FINAL EXAM Winter Quarter, 21 March, 214 (1) Write your name in blue-book provided and sign that you agree to abide by the honor code. (2) The exam consists
More informationCME 106: Review Probability theory
: Probability theory Sven Schmit April 3, 2015 1 Overview In the first half of the course, we covered topics from probability theory. The difference between statistics and probability theory is the following:
More informationBivariate Distributions. Discrete Bivariate Distribution Example
Spring 7 Geog C: Phaedon C. Kyriakidis Bivariate Distributions Definition: class of multivariate probability distributions describing joint variation of outcomes of two random variables (discrete or continuous),
More informationMachine Learning Lecture Notes
Machine Learning Lecture Notes Predrag Radivojac January 3, 25 Random Variables Until now we operated on relatively simple sample spaces and produced measure functions over sets of outcomes. In many situations,
More informationChapter 4 continued. Chapter 4 sections
Chapter 4 sections Chapter 4 continued 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP:
More informationGaussian random variables inr n
Gaussian vectors Lecture 5 Gaussian random variables inr n One-dimensional case One-dimensional Gaussian density with mean and standard deviation (called N, ): fx x exp. Proposition If X N,, then ax b
More informationECE531: Principles of Detection and Estimation Course Introduction
ECE531: Principles of Detection and Estimation Course Introduction D. Richard Brown III WPI 22-January-2009 WPI D. Richard Brown III 22-January-2009 1 / 37 Lecture 1 Major Topics 1. Web page. 2. Syllabus
More informationEEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
More informationLecture 16 : Independence, Covariance and Correlation of Discrete Random Variables
Lecture 6 : Independence, Covariance and Correlation of Discrete Random Variables 0/ 3 Definition Two discrete random variables X and Y defined on the same sample space are said to be independent if for
More information18.440: Lecture 28 Lectures Review
18.440: Lecture 28 Lectures 18-27 Review Scott Sheffield MIT Outline Outline It s the coins, stupid Much of what we have done in this course can be motivated by the i.i.d. sequence X i where each X i is
More informationSTAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
More informationReview of Probabilities and Basic Statistics
Alex Smola Barnabas Poczos TA: Ina Fiterau 4 th year PhD student MLD Review of Probabilities and Basic Statistics 10-701 Recitations 1/25/2013 Recitation 1: Statistics Intro 1 Overview Introduction to
More informationState-space Model. Eduardo Rossi University of Pavia. November Rossi State-space Model Fin. Econometrics / 53
State-space Model Eduardo Rossi University of Pavia November 2014 Rossi State-space Model Fin. Econometrics - 2014 1 / 53 Outline 1 Motivation 2 Introduction 3 The Kalman filter 4 Forecast errors 5 State
More information