Lecture 2: Repetition of probability theory and statistics

Size: px
Start display at page:

Download "Lecture 2: Repetition of probability theory and statistics"

Transcription

1 Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites: none Time: 90 minutes Content Univariate concepts (continuous & discrete): RVs, CDF, PDF (PMF), examples Multivariate concepts: random vectors, covariance & correlation, independency, example bivariate normal distribution Concept of Building Block: Prerequisites: none Time: 90 minutes Content Univariate concepts (continuous & discrete): RVs, CDF, PDF (PMF), examples Multivariate concepts: random vectors, covariance & correlation, independency, example bivariate normal distribution Expected Learning Outcomes The participants can define a probability space and explain it for the coin flipping example. They are able to explain the concepts of discrete and continuous random variables in univariate settings with corresponding cumulative distribution functions, point mass distributions, and probability density functions and can compute basic statistical moments such as expectation and variance. They can define, explain and compute covariances and correlations for multivariate random variables and can test them w.r.t. independency. The participants are able to list examples for different univariate and multivariate distributions. T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat : Coin Flipping given a fair coin (heads and tails) flip it once head/tail? T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat (cont d) given a fair coin (heads and tails) flip it once head/tail? Experiment 2 given two fair coins flip them twice {(head, head), (tail, head)}? What about {(head, head), (head, tail)}? Answer possible outcomes: head, tail P(head) = P(tail) = /2 T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

2 (cont d) Experiment 2 given two fair coins flip them twice {(head, head), (tail, head)}? What about {(head, head), (head, tail)}? Formal of Probability Answer possible outcomes: (head, head), (tail, head), (head, tail), (tail, tail) P({(head, head), (tail, head)}) = /2 P({(head, head), (head, tail)}) = /2 T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Two Perspectives on Probability Two Perspectives on Probability Frequentist probability = frequency with which an event occurs if the experiment is repeated a large number of times ( ) Frequentist probability = frequency with which an event occurs if the experiment is repeated a large number of times ( ) Bayesian probability = distribution of subjective values constructed or updated as data is observed T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Probability Space Coin Flipping (revised) Let h := head, t := tail A probability space is a triple (Ω, F, P), where Ω: sample space; set of all possible outcomes F: σ algebra; set of events s.t. each event is a set containing zero or more outcomes P : F [0, ] probability measure that satisfies:. P( ) = 0 2. P(Ω) = 3. if A i F and A i A j =, then P( A i) = P(A i) sample space Ω = {h, t} σ algebra F = {, {h}, {t}, {h, t}} events of interest A = {h}, B = {t} P(A) = P(B) = /2, P( ) = 0, P({h, t}) = T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Coin Flipping (revised) Let h := head, t := tail sample space Ω = {h, t} σ algebra F = {, {h}, {t}, {h, t}} events of interest A = {h}, B = {t} P(A) = P(B) = /2, P( ) = 0, P({h, t}) = Univariate Concepts Experiment 2 sample space Ω = {(h, h), (t, h), (h, t), (t, t)} σ algebra F = {, (h, h), (h, t), (t, h), (t, t), {(h, h), (h, t)},..., Ω} events of interest A = {(h, h), (t, h)}, B = {(h, h), (h, t)} P(A) = P(B) = /2 T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

3 Random Variables A random variable is a function X : Ω R s.t. {ω Ω X(ω) x} F If, in Experiment 2 (two coins flip), X(ω) counts the number of tails, 0, ω = (h, h), ω = (h, t) X(ω) =, ω = (t, h) 2, ω = (t, t) Random Variables A random variable is a function X : Ω R s.t. {ω Ω X(ω) x} F If, in Experiment 2 (two coins flip), X(ω) counts the number of tails, 0, ω = (h, h), ω = (h, t) X(ω) =, ω = (t, h) 2, ω = (t, t) X is said to be discrete if it takes values in a countable subset {x, x 2,...} R; otherwise, it is said to be continuous T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Cumulative Function (CDF) Cumulative Function (CDF) Every random variable X has an associated cumulative distribution function (CDF) F X : R [0, ] F X = P(ω Ω X(ω) x) Often, the CDF is expressed as F X (x) = P{X x} Every random variable X has an associated cumulative distribution function (CDF) F X : R [0, ] F X = P(ω Ω X(ω) x) Often, the CDF is expressed as F X (x) = P{X x} 0, ω = (h, h) 0, x < 0, ω = (h, t) /4, 0 x < If X(ω) =, then F X =, ω = (t, h) 3/4, x < 2 2, ω = (t, t), x 2 T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Probability Density Function (PDF) Probability Density Function (PDF) A random variable X is continuous if its CDF is absolutely continuous, i.e. x F X (x) = f X (s)ds, x R () A random variable X is continuous if its CDF is absolutely continuous, i.e. x F X (x) = f X (s)ds, x R () From Equation, the derivative f X (x) = df X (x), f X : R [0, ) dx is called the probability density function (PDF) of X T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Properties of the PDF Properties of the PDF f X (x) 0 x supp(f X ) f X (x) 0 x supp(f X ) supp(fx ) f X (x)dx = T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

4 Expectation, Variance The expectation (mean value, first statistical moment) of a continuous random variable X with PDF f X is defined as µ := E[X] = xf X (x)dx, supp(fx ) Expectation, Variance The expectation (mean value, first statistical moment) of a continuous random variable X with PDF f X is defined as µ := E[X] = xf X (x)dx, supp(fx ) The variance (density s variability, second central statistical moment) of a continuous random variable X with PDF f X is defined as σ 2 := Var(X) = (x E[X]) 2 f X (x)dx = E[X 2 ] E[X] 2 supp(fx ) T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Discrete Random Variables Discrete Random Variables The probability mass function (PMF) of a discrete random variable X is given by f X (x) = P(X = x) The probability mass function (PMF) of a discrete random variable X is given by f X (x) = P(X = x) The (sample) mean ˆX and variance S 2 of a discrete random variable X with equiprobable realizations X,..., X n are ˆX = n X i, S 2 = n (X i ˆX) 2 T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat s of s Discrete distributions Binomial Poisson Bernoulli geometric... s of s Discrete distributions Binomial Poisson Bernoulli geometric... Continuous distributions T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Normal (Gaussian) Uniform Beta Gamma... T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat of Continuous Distrib.: Normal The PDF of the normal distribution is f X : R [0, ), σ 2π exp ( (x µ)2 /2σ 2 ) of Continuous Distrib.: Normal The PDF of the normal distribution is f X : R [0, ), σ 2π exp ( (x µ)2 /2σ 2 ) The notation X N (µ, σ 2 ) means that X is normally distributed with mean µ and variance σ T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

5 of Continuous Distrib.: Normal One-dimensional Normal The PDF of the normal distribution is f X : R [0, ), σ 2π exp ( (x µ)2 /2σ 2 ) The notation X N (µ, σ 2 ) means that X is normally distributed with mean µ and variance σ When µ = 0, σ =, X N (0, ) is a standard normal random variable T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat of Continuous Distrib.: Uniform The PDF of the uniform distribution is f X : [a, b] {0, b a }, where I [a,b] (x) = b a I [a,b](x), {, x [a, b] 0, otherwise of Continuous Distrib.: Uniform The PDF of the uniform distribution is f X : [a, b] {0, b a }, where I [a,b] (x) = b a I [a,b](x), {, x [a, b] 0, otherwise The notation X U(a, b) means X is uniformly distributed on [a, b] T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat of Continuous Distrib.: Uniform One-dimensional Uniform The PDF of the uniform distribution is f X : [a, b] {0, b a }, where I [a,b] (x) = b a I [a,b](x), {, x [a, b] 0, otherwise The notation X U(a, b) means X is uniformly distributed on [a, b] Expectation and variance of the uniform distribution If X U(a, b), E[X] = a+b (b a)2 2, Var(X) = 2 T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Random Vectors, Covariance, Correlation Multivariate Concepts X : Ω R n, X = (X, X 2,..., X n ) T is called an n-dimensional random vector, where X, X 2,..., X n are random variables T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

6 Random Vectors, Covariance, Correlation X : Ω R n, X = (X, X 2,..., X n ) T is called an n-dimensional random vector, where X, X 2,..., X n are random variables The covariance of two random variables X and Y is cov(x, Y ) = E[(X E[X])(Y E[Y ])] = E[XY ] E[X]E[Y ] Random Vectors, Covariance, Correlation X : Ω R n, X = (X, X 2,..., X n ) T is called an n-dimensional random vector, where X, X 2,..., X n are random variables The covariance of two random variables X and Y is cov(x, Y ) = E[(X E[X])(Y E[Y ])] = E[XY ] E[X]E[Y ] The Pearson correlation coefficient of two random variables X and Y is ρ XY = cov(x, Y) Var(X)Var(Y ), ρ XY [, ] T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Expectation & Variance of Sums of RVs Let a, a 2,..., a n R and X, X 2,..., X n be random variables E [ n ] a i X i = a i E[X i ] Expectation & Variance of Sums of RVs Let a, a 2,..., a n R and X, X 2,..., X n be random variables E [ n ] a i X i = a i E[X i ] Var ( n ) a i X i = ai 2 Var(X i ) + 2 a i a j cov(x i, X j ) i<j If X i, X i+ are uncorrelated, i.e. ρ Xi = 0, i n, Xi+ T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Var ( n ) a i X i = ai 2 Var(X i ) T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Independence Let A, B denote two events. The joint probability of A and B, P(A, B), is defined as P(A, B) = P(A B) = P(A B)P(B), where P(A B) is the probability of A given that B already happened Independence Let A, B denote two events. The joint probability of A and B, P(A, B), is defined as P(A, B) = P(A B) = P(A B)P(B), where P(A B) is the probability of A given that B already happened A and B are independent if P(A, B) = P(A)P(B), i. e. P(A B) = P(A) T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Independence Let A, B denote two events. The joint probability of A and B, P(A, B), is defined as P(A, B) = P(A B) = P(A B)P(B), where P(A B) is the probability of A given that B already happened A and B are independent if P(A, B) = P(A)P(B), i. e. P(A B) = P(A) The random variables X and Y are independent if their joint PDF (PMF) is f XY (x, y) = f X (x)f Y (y) T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat i.i.d. Random Variables Random variables X, X 2,..., X n are called independent and identically distributed (i.i.d.) with PDF f X if they are mutually independent and, if f Xi is the PDF of X i, i n, f X = f X2 =... = f Xn := f X and n f X (x, x 2,... x n ) = f X (x i ) T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

7 of Multivariate Distrib.: Multivariate Normal : Standard Bivariate Normal The n-dimensional random vector X is normally distributed with mean vector µ = (µ, µ 2,..., µ n ) T and covariance matrix V, V ij = cov(x i, X j ), written X N (µ, V ), if f X (x) = [ (2π)n V exp ] 2 (x µ)t V (x µ) where V is the determinant of V Standard multivariate normal: µ = (0, 0,..., 0) T, V = I n T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat Summary Summary Coin flipping Probability space (Ω, F, P) Univariate concepts Random variables: discrete and continuous Cumulative distribution function (CDF) Continuous random variables & PDF Discrete random variables & PMF s: normal & uniform distributions Multivariate concepts Random vectors Covariance & correlation Independent and identically distributed random variables : bivariate normal distribution T. Neckel Algorithms for Uncertainty Quantification L2: Repetition of prob. theory and stat

Algorithms for Uncertainty Quantification

Algorithms for Uncertainty Quantification Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example

More information

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

Probability Theory for Machine Learning. Chris Cremer September 2015

Probability Theory for Machine Learning. Chris Cremer September 2015 Probability Theory for Machine Learning Chris Cremer September 2015 Outline Motivation Probability Definitions and Rules Probability Distributions MLE for Gaussian Parameter Estimation MLE and Least Squares

More information

Random Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay

Random Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay 1 / 13 Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 8, 2013 2 / 13 Random Variable Definition A real-valued

More information

Appendix A : Introduction to Probability and stochastic processes

Appendix A : Introduction to Probability and stochastic processes A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of

More information

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions

More information

Chapter 5 continued. Chapter 5 sections

Chapter 5 continued. Chapter 5 sections Chapter 5 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions

More information

Quick Tour of Basic Probability Theory and Linear Algebra

Quick Tour of Basic Probability Theory and Linear Algebra Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions

More information

Lecture 1: August 28

Lecture 1: August 28 36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random

More information

3 Multiple Discrete Random Variables

3 Multiple Discrete Random Variables 3 Multiple Discrete Random Variables 3.1 Joint densities Suppose we have a probability space (Ω, F,P) and now we have two discrete random variables X and Y on it. They have probability mass functions f

More information

1 Presessional Probability

1 Presessional Probability 1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional

More information

1.1 Review of Probability Theory

1.1 Review of Probability Theory 1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,

More information

Probability and Distributions

Probability and Distributions Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Lecture 3: Probability, Bayes Theorem, and Bayes Classification Peter Belhumeur Computer Science Columbia University Probability Should you play this game? Game: A fair

More information

Random Variables and Their Distributions

Random Variables and Their Distributions Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital

More information

STAT Chapter 5 Continuous Distributions

STAT Chapter 5 Continuous Distributions STAT 270 - Chapter 5 Continuous Distributions June 27, 2012 Shirin Golchi () STAT270 June 27, 2012 1 / 59 Continuous rv s Definition: X is a continuous rv if it takes values in an interval, i.e., range

More information

Review of Probability. CS1538: Introduction to Simulations

Review of Probability. CS1538: Introduction to Simulations Review of Probability CS1538: Introduction to Simulations Probability and Statistics in Simulation Why do we need probability and statistics in simulation? Needed to validate the simulation model Needed

More information

Fundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner

Fundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner Fundamentals CS 281A: Statistical Learning Theory Yangqing Jia Based on tutorial slides by Lester Mackey and Ariel Kleiner August, 2011 Outline 1 Probability 2 Statistics 3 Linear Algebra 4 Optimization

More information

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample

More information

Bayesian statistics, simulation and software

Bayesian statistics, simulation and software Module 1: Course intro and probability brush-up Department of Mathematical Sciences Aalborg University 1/22 Bayesian Statistics, Simulations and Software Course outline Course consists of 12 half-days

More information

Lecture 11. Probability Theory: an Overveiw

Lecture 11. Probability Theory: an Overveiw Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the

More information

MA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2

MA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 MA 575 Linear Models: Cedric E Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 1 Revision: Probability Theory 11 Random Variables A real-valued random variable is

More information

Lecture 1: Review on Probability and Statistics

Lecture 1: Review on Probability and Statistics STAT 516: Stochastic Modeling of Scientific Data Autumn 2018 Instructor: Yen-Chi Chen Lecture 1: Review on Probability and Statistics These notes are partially based on those of Mathias Drton. 1.1 Motivating

More information

Chapter 2: Random Variables

Chapter 2: Random Variables ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:

More information

STAT2201. Analysis of Engineering & Scientific Data. Unit 3

STAT2201. Analysis of Engineering & Scientific Data. Unit 3 STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random

More information

Basics on Probability. Jingrui He 09/11/2007

Basics on Probability. Jingrui He 09/11/2007 Basics on Probability Jingrui He 09/11/2007 Coin Flips You flip a coin Head with probability 0.5 You flip 100 coins How many heads would you expect Coin Flips cont. You flip a coin Head with probability

More information

Review of Probability Theory

Review of Probability Theory Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving

More information

Multivariate Random Variable

Multivariate Random Variable Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate

More information

Northwestern University Department of Electrical Engineering and Computer Science

Northwestern University Department of Electrical Engineering and Computer Science Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability

More information

STAT 430/510: Lecture 16

STAT 430/510: Lecture 16 STAT 430/510: Lecture 16 James Piette June 24, 2010 Updates HW4 is up on my website. It is due next Mon. (June 28th). Starting today back at section 6.7 and will begin Ch. 7. Joint Distribution of Functions

More information

Probability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3

Probability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3 Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................

More information

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables. Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel

More information

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.

Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows. Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage

More information

EE514A Information Theory I Fall 2013

EE514A Information Theory I Fall 2013 EE514A Information Theory I Fall 2013 K. Mohan, Prof. J. Bilmes University of Washington, Seattle Department of Electrical Engineering Fall Quarter, 2013 http://j.ee.washington.edu/~bilmes/classes/ee514a_fall_2013/

More information

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed

More information

Chapter 5. Chapter 5 sections

Chapter 5. Chapter 5 sections 1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions

More information

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline. Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example

More information

3. Probability and Statistics

3. Probability and Statistics FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important

More information

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed

More information

1: PROBABILITY REVIEW

1: PROBABILITY REVIEW 1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following

More information

Introduction to Stochastic Processes

Introduction to Stochastic Processes Stat251/551 (Spring 2017) Stochastic Processes Lecture: 1 Introduction to Stochastic Processes Lecturer: Sahand Negahban Scribe: Sahand Negahban 1 Organization Issues We will use canvas as the course webpage.

More information

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are

More information

EE4601 Communication Systems

EE4601 Communication Systems EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two

More information

Recitation 2: Probability

Recitation 2: Probability Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions

More information

Lecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN

Lecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and

More information

BASICS OF PROBABILITY

BASICS OF PROBABILITY October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,

More information

Continuous Random Variables

Continuous Random Variables 1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables

More information

Lecture 3. Discrete Random Variables

Lecture 3. Discrete Random Variables Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition

More information

Introduction to Statistical Inference Self-study

Introduction to Statistical Inference Self-study Introduction to Statistical Inference Self-study Contents Definition, sample space The fundamental object in probability is a nonempty sample space Ω. An event is a subset A Ω. Definition, σ-algebra A

More information

CME 106: Review Probability theory

CME 106: Review Probability theory : Probability theory Sven Schmit April 3, 2015 1 Overview In the first half of the course, we covered topics from probability theory. The difference between statistics and probability theory is the following:

More information

Introduction to Machine Learning

Introduction to Machine Learning Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB

More information

M378K In-Class Assignment #1

M378K In-Class Assignment #1 The following problems are a review of M6K. M7K In-Class Assignment # Problem.. Complete the definition of mutual exclusivity of events below: Events A, B Ω are said to be mutually exclusive if A B =.

More information

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section

More information

Probability Review. Chao Lan

Probability Review. Chao Lan Probability Review Chao Lan Let s start with a single random variable Random Experiment A random experiment has three elements 1. sample space Ω: set of all possible outcomes e.g.,ω={1,2,3,4,5,6} 2. event

More information

Lecture 6 Basic Probability

Lecture 6 Basic Probability Lecture 6: Basic Probability 1 of 17 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 6 Basic Probability Probability spaces A mathematical setup behind a probabilistic

More information

Expectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda

Expectation. DS GA 1002 Statistical and Mathematical Models.   Carlos Fernandez-Granda Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,

More information

Formulas for probability theory and linear models SF2941

Formulas for probability theory and linear models SF2941 Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms

More information

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019

Lecture 10: Probability distributions TUESDAY, FEBRUARY 19, 2019 Lecture 10: Probability distributions DANIEL WELLER TUESDAY, FEBRUARY 19, 2019 Agenda What is probability? (again) Describing probabilities (distributions) Understanding probabilities (expectation) Partial

More information

Review: mostly probability and some statistics

Review: mostly probability and some statistics Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random

More information

ECON Fundamentals of Probability

ECON Fundamentals of Probability ECON 351 - Fundamentals of Probability Maggie Jones 1 / 32 Random Variables A random variable is one that takes on numerical values, i.e. numerical summary of a random outcome e.g., prices, total GDP,

More information

Lecture 2: Review of Probability

Lecture 2: Review of Probability Lecture 2: Review of Probability Zheng Tian Contents 1 Random Variables and Probability Distributions 2 1.1 Defining probabilities and random variables..................... 2 1.2 Probability distributions................................

More information

1 Review of Probability and Distributions

1 Review of Probability and Distributions Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote

More information

Probability theory for Networks (Part 1) CS 249B: Science of Networks Week 02: Monday, 02/04/08 Daniel Bilar Wellesley College Spring 2008

Probability theory for Networks (Part 1) CS 249B: Science of Networks Week 02: Monday, 02/04/08 Daniel Bilar Wellesley College Spring 2008 Probability theory for Networks (Part 1) CS 249B: Science of Networks Week 02: Monday, 02/04/08 Daniel Bilar Wellesley College Spring 2008 1 Review We saw some basic metrics that helped us characterize

More information

Random variables. DS GA 1002 Probability and Statistics for Data Science.

Random variables. DS GA 1002 Probability and Statistics for Data Science. Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities

More information

Fundamental Tools - Probability Theory II

Fundamental Tools - Probability Theory II Fundamental Tools - Probability Theory II MSc Financial Mathematics The University of Warwick September 29, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory II 1 / 22 Measurable random

More information

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets

More information

Random Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping

Random Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the

More information

Probability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27

Probability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27 Probability Review Yutian Li Stanford University January 18, 2018 Yutian Li (Stanford University) Probability Review January 18, 2018 1 / 27 Outline 1 Elements of probability 2 Random variables 3 Multiple

More information

EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as

EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace

More information

Expectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda

Expectation. DS GA 1002 Probability and Statistics for Data Science.   Carlos Fernandez-Granda Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,

More information

1 Random Variable: Topics

1 Random Variable: Topics Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?

More information

[POLS 8500] Review of Linear Algebra, Probability and Information Theory

[POLS 8500] Review of Linear Algebra, Probability and Information Theory [POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming

More information

ECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else

ECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else ECE 450 Homework #3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3 4 5

More information

STAT/MATH 395 A - PROBABILITY II UW Winter Quarter Moment functions. x r p X (x) (1) E[X r ] = x r f X (x) dx (2) (x E[X]) r p X (x) (3)

STAT/MATH 395 A - PROBABILITY II UW Winter Quarter Moment functions. x r p X (x) (1) E[X r ] = x r f X (x) dx (2) (x E[X]) r p X (x) (3) STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 07 Néhémy Lim Moment functions Moments of a random variable Definition.. Let X be a rrv on probability space (Ω, A, P). For a given r N, E[X r ], if it

More information

Probability theory basics

Probability theory basics Probability theory basics Michael Franke Basics of probability theory: axiomatic definition, interpretation, joint distributions, marginalization, conditional probability & Bayes rule. Random variables:

More information

Preliminary Statistics. Lecture 3: Probability Models and Distributions

Preliminary Statistics. Lecture 3: Probability Models and Distributions Preliminary Statistics Lecture 3: Probability Models and Distributions Rory Macqueen (rm43@soas.ac.uk), September 2015 Outline Revision of Lecture 2 Probability Density Functions Cumulative Distribution

More information

Data Analysis and Monte Carlo Methods

Data Analysis and Monte Carlo Methods Lecturer: Allen Caldwell, Max Planck Institute for Physics & TUM Recitation Instructor: Oleksander (Alex) Volynets, MPP & TUM General Information: - Lectures will be held in English, Mondays 16-18:00 -

More information

1 Probability theory. 2 Random variables and probability theory.

1 Probability theory. 2 Random variables and probability theory. Probability theory Here we summarize some of the probability theory we need. If this is totally unfamiliar to you, you should look at one of the sources given in the readings. In essence, for the major

More information

Review (Probability & Linear Algebra)

Review (Probability & Linear Algebra) Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint

More information

STAT 430/510: Lecture 15

STAT 430/510: Lecture 15 STAT 430/510: Lecture 15 James Piette June 23, 2010 Updates HW4 is up on my website. It is due next Mon. (June 28th). Starting today back at section 6.4... Conditional Distribution: Discrete Def: The conditional

More information

Statistics, Data Analysis, and Simulation SS 2015

Statistics, Data Analysis, and Simulation SS 2015 Statistics, Data Analysis, and Simulation SS 2015 08.128.730 Statistik, Datenanalyse und Simulation Dr. Michael O. Distler Mainz, 27. April 2015 Dr. Michael O. Distler

More information

Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016

Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying

More information

Chapter 4 Multiple Random Variables

Chapter 4 Multiple Random Variables Review for the previous lecture Theorems and Examples: How to obtain the pmf (pdf) of U = g ( X Y 1 ) and V = g ( X Y) Chapter 4 Multiple Random Variables Chapter 43 Bivariate Transformations Continuous

More information

conditional cdf, conditional pdf, total probability theorem?

conditional cdf, conditional pdf, total probability theorem? 6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random

More information

The Binomial distribution. Probability theory 2. Example. The Binomial distribution

The Binomial distribution. Probability theory 2. Example. The Binomial distribution Probability theory Tron Anders Moger September th 7 The Binomial distribution Bernoulli distribution: One experiment X i with two possible outcomes, probability of success P. If the experiment is repeated

More information

Chapter 2. Continuous random variables

Chapter 2. Continuous random variables Chapter 2 Continuous random variables Outline Review of probability: events and probability Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction

More information

Relationship between probability set function and random variable - 2 -

Relationship between probability set function and random variable - 2 - 2.0 Random Variables A rat is selected at random from a cage and its sex is determined. The set of possible outcomes is female and male. Thus outcome space is S = {female, male} = {F, M}. If we let X be

More information

Introduction to Probability Theory

Introduction to Probability Theory Introduction to Probability Theory Ping Yu Department of Economics University of Hong Kong Ping Yu (HKU) Probability 1 / 39 Foundations 1 Foundations 2 Random Variables 3 Expectation 4 Multivariate Random

More information

CDA5530: Performance Models of Computers and Networks. Chapter 2: Review of Practical Random Variables

CDA5530: Performance Models of Computers and Networks. Chapter 2: Review of Practical Random Variables CDA5530: Performance Models of Computers and Networks Chapter 2: Review of Practical Random Variables Definition Random variable (R.V.) X: A function on sample space X: S R Cumulative distribution function

More information

E X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.

E X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl. E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,

More information

Probability Theory. Patrick Lam

Probability Theory. Patrick Lam Probability Theory Patrick Lam Outline Probability Random Variables Simulation Important Distributions Discrete Distributions Continuous Distributions Most Basic Definition of Probability: number of successes

More information

Bivariate distributions

Bivariate distributions Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient

More information

Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology

Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna

More information

Random Variables. Lecture 6: E(X ), Var(X ), & Cov(X, Y ) Random Variables - Vocabulary. Random Variables, cont.

Random Variables. Lecture 6: E(X ), Var(X ), & Cov(X, Y ) Random Variables - Vocabulary. Random Variables, cont. Lecture 6: E(X ), Var(X ), & Cov(X, Y ) Sta230/Mth230 Colin Rundel February 5, 2014 We have been using them for a while now in a variety of forms but it is good to explicitly define what we mean Random

More information

Bivariate Distributions

Bivariate Distributions STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 17 Néhémy Lim Bivariate Distributions 1 Distributions of Two Random Variables Definition 1.1. Let X and Y be two rrvs on probability space (Ω, A, P).

More information

ELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables

ELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Random Variable Discrete Random

More information

STAT/MATH 395 PROBABILITY II

STAT/MATH 395 PROBABILITY II STAT/MATH 395 PROBABILITY II Bivariate Distributions Néhémy Lim University of Washington Winter 2017 Outline Distributions of Two Random Variables Distributions of Two Discrete Random Variables Distributions

More information

Stochastic processes Lecture 1: Multiple Random Variables Ch. 5

Stochastic processes Lecture 1: Multiple Random Variables Ch. 5 Stochastic processes Lecture : Multiple Random Variables Ch. 5 Dr. Ir. Richard C. Hendriks 26/04/8 Delft University of Technology Challenge the future Organization Plenary Lectures Book: R.D. Yates and

More information

Introduction to Machine Learning

Introduction to Machine Learning What does this mean? Outline Contents Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola December 26, 2017 1 Introduction to Probability 1 2 Random Variables 3 3 Bayes

More information