Probability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27
|
|
- Samuel Alexander
- 5 years ago
- Views:
Transcription
1 Probability Review Yutian Li Stanford University January 18, 2018 Yutian Li (Stanford University) Probability Review January 18, / 27
2 Outline 1 Elements of probability 2 Random variables 3 Multiple random variables 4 Common inequalities Yutian Li (Stanford University) Probability Review January 18, / 27
3 Outline 1 Elements of probability 2 Random variables 3 Multiple random variables 4 Common inequalities Yutian Li (Stanford University) Probability Review January 18, / 27
4 Elements of probability Definition (Sample space Ω) The set of all the outcomes of a random experiment. Yutian Li (Stanford University) Probability Review January 18, / 27
5 Elements of probability Definition (Event space F) A set whose elements A F (called events) are subsets of Ω. Yutian Li (Stanford University) Probability Review January 18, / 27
6 Elements of probability Definition (Probability measure) A function P : F R that satisfies the following properties. P(A) 0, for all A F. P(Ω) = 1. If A 1, A 2,... are disjoint events, then P( i A i ) = i P(A i ). These three properties are called the the Axioms of Probability. Yutian Li (Stanford University) Probability Review January 18, / 27
7 Elements of probability Definition (Probability measure) A function P : F R that satisfies the following properties. P(A) 0, for all A F. P(Ω) = 1. If A 1, A 2,... are disjoint events, then P( i A i ) = i P(A i ). These three properties are called the the Axioms of Probability. Yutian Li (Stanford University) Probability Review January 18, / 27
8 Elements of probability Properties: 1 If A B = P(A) P(B). 2 P(A B) min(p(a), P(B)). 3 Union bound: P(A B) P(A) + P(B). 4 P(Ω \ A) = 1 P(A). 5 Law of total probability: If A 1,..., A k are a set of disjoint events such that k i=1 A i = Ω, then k P(A i ) = 1. i=1 Yutian Li (Stanford University) Probability Review January 18, / 27
9 Elements of probability Properties: 1 If A B = P(A) P(B). 2 P(A B) min(p(a), P(B)). 3 Union bound: P(A B) P(A) + P(B). 4 P(Ω \ A) = 1 P(A). 5 Law of total probability: If A 1,..., A k are a set of disjoint events such that k i=1 A i = Ω, then k P(A i ) = 1. i=1 Yutian Li (Stanford University) Probability Review January 18, / 27
10 Elements of probability Properties: 1 If A B = P(A) P(B). 2 P(A B) min(p(a), P(B)). 3 Union bound: P(A B) P(A) + P(B). 4 P(Ω \ A) = 1 P(A). 5 Law of total probability: If A 1,..., A k are a set of disjoint events such that k i=1 A i = Ω, then k P(A i ) = 1. i=1 Yutian Li (Stanford University) Probability Review January 18, / 27
11 Elements of probability Properties: 1 If A B = P(A) P(B). 2 P(A B) min(p(a), P(B)). 3 Union bound: P(A B) P(A) + P(B). 4 P(Ω \ A) = 1 P(A). 5 Law of total probability: If A 1,..., A k are a set of disjoint events such that k i=1 A i = Ω, then k P(A i ) = 1. i=1 Yutian Li (Stanford University) Probability Review January 18, / 27
12 Elements of probability Properties: 1 If A B = P(A) P(B). 2 P(A B) min(p(a), P(B)). 3 Union bound: P(A B) P(A) + P(B). 4 P(Ω \ A) = 1 P(A). 5 Law of total probability: If A 1,..., A k are a set of disjoint events such that k i=1 A i = Ω, then k P(A i ) = 1. i=1 Yutian Li (Stanford University) Probability Review January 18, / 27
13 Elements of probability Definition (Conditional probability) P(A B) = P(A B). P(B) Yutian Li (Stanford University) Probability Review January 18, / 27
14 Elements of probability Theorem (Chain rule) Let S 1,..., S k be events, P(S i ) > 0. Then P(S 1 S 2 S k ) =P(S 1 )P(S 2 S 1 )P(S 3 S 2 S 1 ) P(S k S 1 S 2 S k 1 ). Yutian Li (Stanford University) Probability Review January 18, / 27
15 Elements of probability Definition (Independence) Two events are called independent if and only if P(A B) = P(A)P(B). Yutian Li (Stanford University) Probability Review January 18, / 27
16 Outline 1 Elements of probability 2 Random variables 3 Multiple random variables 4 Common inequalities Yutian Li (Stanford University) Probability Review January 18, / 27
17 Random variables Definition (Random variable) A random variable X is a function X : Ω R. Typically we denote random variables using upper case letters X (ω) or more simply X. We denote the value that a random variable may take on using lower case letters x. Yutian Li (Stanford University) Probability Review January 18, / 27
18 Random variables Definition (Cumulative distribution function) A cumulative distribution function (CDF) is a function F X : R [0, 1] defined as F X (x) = P(X x). Properties: 1 0 F X (x) 1. 2 lim x F X (x) = 0. 3 lim x F X (x) = 1. 4 x y = F X (x) F X (y). 5 Right-continuous: lim x +a F X (x) = F x (a). Yutian Li (Stanford University) Probability Review January 18, / 27
19 Random variables Definition (Cumulative distribution function) A cumulative distribution function (CDF) is a function F X : R [0, 1] defined as F X (x) = P(X x). Properties: 1 0 F X (x) 1. 2 lim x F X (x) = 0. 3 lim x F X (x) = 1. 4 x y = F X (x) F X (y). 5 Right-continuous: lim x +a F X (x) = F x (a). Yutian Li (Stanford University) Probability Review January 18, / 27
20 Random variables Definition (Cumulative distribution function) A cumulative distribution function (CDF) is a function F X : R [0, 1] defined as F X (x) = P(X x). Properties: 1 0 F X (x) 1. 2 lim x F X (x) = 0. 3 lim x F X (x) = 1. 4 x y = F X (x) F X (y). 5 Right-continuous: lim x +a F X (x) = F x (a). Yutian Li (Stanford University) Probability Review January 18, / 27
21 Random variables Definition (Cumulative distribution function) A cumulative distribution function (CDF) is a function F X : R [0, 1] defined as F X (x) = P(X x). Properties: 1 0 F X (x) 1. 2 lim x F X (x) = 0. 3 lim x F X (x) = 1. 4 x y = F X (x) F X (y). 5 Right-continuous: lim x +a F X (x) = F x (a). Yutian Li (Stanford University) Probability Review January 18, / 27
22 Random variables Definition (Cumulative distribution function) A cumulative distribution function (CDF) is a function F X : R [0, 1] defined as F X (x) = P(X x). Properties: 1 0 F X (x) 1. 2 lim x F X (x) = 0. 3 lim x F X (x) = 1. 4 x y = F X (x) F X (y). 5 Right-continuous: lim x +a F X (x) = F x (a). Yutian Li (Stanford University) Probability Review January 18, / 27
23 Random variables Definition (Cumulative distribution function) A cumulative distribution function (CDF) is a function F X : R [0, 1] defined as F X (x) = P(X x). Properties: 1 0 F X (x) 1. 2 lim x F X (x) = 0. 3 lim x F X (x) = 1. 4 x y = F X (x) F X (y). 5 Right-continuous: lim x +a F X (x) = F x (a). Yutian Li (Stanford University) Probability Review January 18, / 27
24 Random variables Definition (Probability density function) If the cumulative distribution function F X is differentiable everywhere, we define the probability density function (PDF) as the derivative of the CDF, f X (x) = df X (x). dx Note that the PDF for a continuous random variable may not always exist. Properties: 1 f X (x) 0. 2 f X (x)dx = 1. 3 x A f X (x)dx = P(X A). Yutian Li (Stanford University) Probability Review January 18, / 27
25 Random variables Definition (Probability density function) If the cumulative distribution function F X is differentiable everywhere, we define the probability density function (PDF) as the derivative of the CDF, f X (x) = df X (x). dx Note that the PDF for a continuous random variable may not always exist. Properties: 1 f X (x) 0. 2 f X (x)dx = 1. 3 x A f X (x)dx = P(X A). Yutian Li (Stanford University) Probability Review January 18, / 27
26 Random variables Definition (Probability density function) If the cumulative distribution function F X is differentiable everywhere, we define the probability density function (PDF) as the derivative of the CDF, f X (x) = df X (x). dx Note that the PDF for a continuous random variable may not always exist. Properties: 1 f X (x) 0. 2 f X (x)dx = 1. 3 x A f X (x)dx = P(X A). Yutian Li (Stanford University) Probability Review January 18, / 27
27 Random variables Definition (Probability density function) If the cumulative distribution function F X is differentiable everywhere, we define the probability density function (PDF) as the derivative of the CDF, f X (x) = df X (x). dx Note that the PDF for a continuous random variable may not always exist. Properties: 1 f X (x) 0. 2 f X (x)dx = 1. 3 x A f X (x)dx = P(X A). Yutian Li (Stanford University) Probability Review January 18, / 27
28 Random variables Definition (Expectation) If X is a continuous random variable with PDF f X (x) and g : R R is an arbitrary function. The expectation or expected value of g(x ) is defined as E[g(X )] = Properties: g(x)f X (x)dx. 1 E[a] = a for any constant a R. 2 E[ag(X )] = ae[g(x )] for any constant a R. 3 Linearity: E[f (X ) + g(x )] = E[f (X )] + E[g(X )]. 4 For a discrete random variable X, E[1{X = k}] = P(X = k). Yutian Li (Stanford University) Probability Review January 18, / 27
29 Random variables Definition (Expectation) If X is a continuous random variable with PDF f X (x) and g : R R is an arbitrary function. The expectation or expected value of g(x ) is defined as E[g(X )] = Properties: g(x)f X (x)dx. 1 E[a] = a for any constant a R. 2 E[ag(X )] = ae[g(x )] for any constant a R. 3 Linearity: E[f (X ) + g(x )] = E[f (X )] + E[g(X )]. 4 For a discrete random variable X, E[1{X = k}] = P(X = k). Yutian Li (Stanford University) Probability Review January 18, / 27
30 Random variables Definition (Expectation) If X is a continuous random variable with PDF f X (x) and g : R R is an arbitrary function. The expectation or expected value of g(x ) is defined as E[g(X )] = Properties: g(x)f X (x)dx. 1 E[a] = a for any constant a R. 2 E[ag(X )] = ae[g(x )] for any constant a R. 3 Linearity: E[f (X ) + g(x )] = E[f (X )] + E[g(X )]. 4 For a discrete random variable X, E[1{X = k}] = P(X = k). Yutian Li (Stanford University) Probability Review January 18, / 27
31 Random variables Definition (Expectation) If X is a continuous random variable with PDF f X (x) and g : R R is an arbitrary function. The expectation or expected value of g(x ) is defined as E[g(X )] = Properties: g(x)f X (x)dx. 1 E[a] = a for any constant a R. 2 E[ag(X )] = ae[g(x )] for any constant a R. 3 Linearity: E[f (X ) + g(x )] = E[f (X )] + E[g(X )]. 4 For a discrete random variable X, E[1{X = k}] = P(X = k). Yutian Li (Stanford University) Probability Review January 18, / 27
32 Random variables Definition (Expectation) If X is a continuous random variable with PDF f X (x) and g : R R is an arbitrary function. The expectation or expected value of g(x ) is defined as E[g(X )] = Properties: g(x)f X (x)dx. 1 E[a] = a for any constant a R. 2 E[ag(X )] = ae[g(x )] for any constant a R. 3 Linearity: E[f (X ) + g(x )] = E[f (X )] + E[g(X )]. 4 For a discrete random variable X, E[1{X = k}] = P(X = k). Yutian Li (Stanford University) Probability Review January 18, / 27
33 Random variables Definition (Variance) The variance of a random variable X is a measure of how concentrated the distribution of a random variable X is around its mean. Formally, the variance of a random variable X is defined as Var[X ] = E[(X E[X ]) 2 ]. Alternatively, Var[X ] = E[X 2 ] E[X ] 2. Properties: 1 Var[a] = 0 for any constant a R. 2 Var[af (X )] = a 2 Var[f (X )] for any constant a R. Yutian Li (Stanford University) Probability Review January 18, / 27
34 Random variables Definition (Variance) The variance of a random variable X is a measure of how concentrated the distribution of a random variable X is around its mean. Formally, the variance of a random variable X is defined as Var[X ] = E[(X E[X ]) 2 ]. Alternatively, Var[X ] = E[X 2 ] E[X ] 2. Properties: 1 Var[a] = 0 for any constant a R. 2 Var[af (X )] = a 2 Var[f (X )] for any constant a R. Yutian Li (Stanford University) Probability Review January 18, / 27
35 Random variables Definition (Variance) The variance of a random variable X is a measure of how concentrated the distribution of a random variable X is around its mean. Formally, the variance of a random variable X is defined as Var[X ] = E[(X E[X ]) 2 ]. Alternatively, Var[X ] = E[X 2 ] E[X ] 2. Properties: 1 Var[a] = 0 for any constant a R. 2 Var[af (X )] = a 2 Var[f (X )] for any constant a R. Yutian Li (Stanford University) Probability Review January 18, / 27
36 Random variables Definition (Variance) The variance of a random variable X is a measure of how concentrated the distribution of a random variable X is around its mean. Formally, the variance of a random variable X is defined as Var[X ] = E[(X E[X ]) 2 ]. Alternatively, Var[X ] = E[X 2 ] E[X ] 2. Properties: 1 Var[a] = 0 for any constant a R. 2 Var[af (X )] = a 2 Var[f (X )] for any constant a R. Yutian Li (Stanford University) Probability Review January 18, / 27
37 Random variables Discrete random variables: X Bernoulli(p) (where 0 p 1): one if a coin with heads probability p comes up heads, zero otherwise. { p if p = 1 p(x) = 1 p if p = 0 X Binomial(n, p) (where 0 p 1): the number of heads in n independent flips of a coin with heads probability p. ( ) n p(x) = p x (1 p) n x x Yutian Li (Stanford University) Probability Review January 18, / 27
38 Random variables Discrete random variables: X Geometric(p) (where p > 0): the number of flips of a coin with heads probability p until the first heads. p(x) = p(1 p) x 1 X Poisson(λ) (where λ > 0): a probability distribution over the nonnegative integers used for modeling the frequency of rare events. λ λx p(x) = e x! Yutian Li (Stanford University) Probability Review January 18, / 27
39 Random variables Continuous random variables: X Uniform(a, b) (where a < b): equal probability density to every value between a and b on the real line. f (x) = { 1 b a if a x b 0 otherwise X Exponential(λ) (where λ > 0): decaying probability density over the negative reals. { λe λx if x 0 f (x) = 0 otherwise Yutian Li (Stanford University) Probability Review January 18, / 27
40 Random variables Continuous random variables: X Normal(µ, σ 2 ): also known as the Gaussian distribution. f (x) = 1 2πσ 2 e 1 2σ 2 (x µ)2 Yutian Li (Stanford University) Probability Review January 18, / 27
41 Outline 1 Elements of probability 2 Random variables 3 Multiple random variables 4 Common inequalities Yutian Li (Stanford University) Probability Review January 18, / 27
42 Multiple random variables Definition (Joint cumulative distribution function) Suppose that we have two random variables X and Y, F XY (x, y) = P(X x, Y y). The joint CDF F XY (x, y) and the marginal cumulative distribution functions F X (x) and F Y (y) of each variable separately are related by F X (x) = lim y F XY (x, y), F Y (y) = lim x F XY (x, y). Yutian Li (Stanford University) Probability Review January 18, / 27
43 Multiple random variables Definition (Joint cumulative distribution function) Suppose that we have two random variables X and Y, F XY (x, y) = P(X x, Y y). The joint CDF F XY (x, y) and the marginal cumulative distribution functions F X (x) and F Y (y) of each variable separately are related by F X (x) = lim y F XY (x, y), F Y (y) = lim x F XY (x, y). Yutian Li (Stanford University) Probability Review January 18, / 27
44 Multiple random variables In the discrete case, the conditional probability mass function of X given Y is simply p Y X (y x) = p XY (x, y), p X (x) assuming that p X (x) 0. A useful formula that often arises when trying to derive expression for the conditional probability of one variable given another, is Bayes rule: p Y X (y x) = p XY (x, y) p X (x) = p X Y (x y)p Y (y) y p X Y (x y )p Y (y ). Yutian Li (Stanford University) Probability Review January 18, / 27
45 Multiple random variables In the discrete case, the conditional probability mass function of X given Y is simply p Y X (y x) = p XY (x, y), p X (x) assuming that p X (x) 0. A useful formula that often arises when trying to derive expression for the conditional probability of one variable given another, is Bayes rule: p Y X (y x) = p XY (x, y) p X (x) = p X Y (x y)p Y (y) y p X Y (x y )p Y (y ). Yutian Li (Stanford University) Probability Review January 18, / 27
46 Outline 1 Elements of probability 2 Random variables 3 Multiple random variables 4 Common inequalities Yutian Li (Stanford University) Probability Review January 18, / 27
47 Common inequalities Theorem (Markov inequality) Let X be a non-negative random variable, then P(X a) E[X ] a. Yutian Li (Stanford University) Probability Review January 18, / 27
48 Common inequalities Theorem (Chebyshev inequality) Let X be a random variable, then P( X E[X ] a) Var(x) a 2. Yutian Li (Stanford University) Probability Review January 18, / 27
49 Common inequalities Theorem (Jensen inequality) Let φ be a convex function and X be a random variable, then E[φ(X )] φ(e[x ]). Yutian Li (Stanford University) Probability Review January 18, / 27
Quick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationReview of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
More informationMore on Distribution Function
More on Distribution Function The distribution of a random variable X can be determined directly from its cumulative distribution function F X. Theorem: Let X be any random variable, with cumulative distribution
More information1 Random Variable: Topics
Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?
More informationRandom Variables. Will Perkins. January 11, 2013
Random Variables Will Perkins January 11, 2013 Random Variables If a probability model describes an experiment, a random variable is a measurement - a number associated with each outcome of the experiment.
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More information2 Continuous Random Variables and their Distributions
Name: Discussion-5 1 Introduction - Continuous random variables have a range in the form of Interval on the real number line. Union of non-overlapping intervals on real line. - We also know that for any
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationLecture 11. Probability Theory: an Overveiw
Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the
More information2. Suppose (X, Y ) is a pair of random variables uniformly distributed over the triangle with vertices (0, 0), (2, 0), (2, 1).
Name M362K Final Exam Instructions: Show all of your work. You do not have to simplify your answers. No calculators allowed. There is a table of formulae on the last page. 1. Suppose X 1,..., X 1 are independent
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 14: Continuous random variables Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More informationSTAT Chapter 5 Continuous Distributions
STAT 270 - Chapter 5 Continuous Distributions June 27, 2012 Shirin Golchi () STAT270 June 27, 2012 1 / 59 Continuous rv s Definition: X is a continuous rv if it takes values in an interval, i.e., range
More informationProbability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.
Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel
More informationProbability Review. Chao Lan
Probability Review Chao Lan Let s start with a single random variable Random Experiment A random experiment has three elements 1. sample space Ω: set of all possible outcomes e.g.,ω={1,2,3,4,5,6} 2. event
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationProbability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3
Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................
More informationSTAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationEE514A Information Theory I Fall 2013
EE514A Information Theory I Fall 2013 K. Mohan, Prof. J. Bilmes University of Washington, Seattle Department of Electrical Engineering Fall Quarter, 2013 http://j.ee.washington.edu/~bilmes/classes/ee514a_fall_2013/
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationSTT 441 Final Exam Fall 2013
STT 441 Final Exam Fall 2013 (12:45-2:45pm, Thursday, Dec. 12, 2013) NAME: ID: 1. No textbooks or class notes are allowed in this exam. 2. Be sure to show all of your work to receive credit. Credits are
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationRandom variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationChapter 4. Continuous Random Variables 4.1 PDF
Chapter 4 Continuous Random Variables In this chapter we study continuous random variables. The linkage between continuous and discrete random variables is the cumulative distribution (CDF) which we will
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationJoint Distribution of Two or More Random Variables
Joint Distribution of Two or More Random Variables Sometimes more than one measurement in the form of random variable is taken on each member of the sample space. In cases like this there will be a few
More informationDeep Learning for Computer Vision
Deep Learning for Computer Vision Lecture 3: Probability, Bayes Theorem, and Bayes Classification Peter Belhumeur Computer Science Columbia University Probability Should you play this game? Game: A fair
More informationProbability Theory. Patrick Lam
Probability Theory Patrick Lam Outline Probability Random Variables Simulation Important Distributions Discrete Distributions Continuous Distributions Most Basic Definition of Probability: number of successes
More informationChapter 4. Chapter 4 sections
Chapter 4 sections 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP: 4.8 Utility Expectation
More informationNorthwestern University Department of Electrical Engineering and Computer Science
Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability
More informationRandom Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay
1 / 13 Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 8, 2013 2 / 13 Random Variable Definition A real-valued
More informationProbability and Distributions
Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated
More informationMotivation and Applications: Why Should I Study Probability?
Motivation and Applications: Why Should I Study Probability? As stated by Laplace, Probability is common sense reduced to calculation. You need to first learn the theory required to correctly do these
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationLecture 1. ABC of Probability
Math 408 - Mathematical Statistics Lecture 1. ABC of Probability January 16, 2013 Konstantin Zuev (USC) Math 408, Lecture 1 January 16, 2013 1 / 9 Agenda Sample Spaces Realizations, Events Axioms of Probability
More informationFundamental Tools - Probability Theory II
Fundamental Tools - Probability Theory II MSc Financial Mathematics The University of Warwick September 29, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory II 1 / 22 Measurable random
More informationLecture 1: Review on Probability and Statistics
STAT 516: Stochastic Modeling of Scientific Data Autumn 2018 Instructor: Yen-Chi Chen Lecture 1: Review on Probability and Statistics These notes are partially based on those of Mathias Drton. 1.1 Motivating
More informationDefinition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R
Random Variables Definition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R As such, a random variable summarizes the outcome of an experiment
More informationChapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University
Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real
More informationProbability reminders
CS246 Winter 204 Mining Massive Data Sets Probability reminders Sammy El Ghazzal selghazz@stanfordedu Disclaimer These notes may contain typos, mistakes or confusing points Please contact the author so
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationCS145: Probability & Computing
CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,
More informationOrder Statistics. The order statistics of a set of random variables X 1, X 2,, X n are the same random variables arranged in increasing order.
Order Statistics The order statistics of a set of random variables 1, 2,, n are the same random variables arranged in increasing order. Denote by (1) = smallest of 1, 2,, n (2) = 2 nd smallest of 1, 2,,
More informationMath Review Sheet, Fall 2008
1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the
More informationContinuous Probability Spaces
Continuous Probability Spaces Ω is not countable. Outcomes can be any real number or part of an interval of R, e.g. heights, weights and lifetimes. Can not assign probabilities to each outcome and add
More informationChapter 2. Continuous random variables
Chapter 2 Continuous random variables Outline Review of probability: events and probability Random variable Probability and Cumulative distribution function Review of discrete random variable Introduction
More informationA Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.
A Probability Primer A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. Are you holding all the cards?? Random Events A random event, E,
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More information1.1 Review of Probability Theory
1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More informationFundamentals. CS 281A: Statistical Learning Theory. Yangqing Jia. August, Based on tutorial slides by Lester Mackey and Ariel Kleiner
Fundamentals CS 281A: Statistical Learning Theory Yangqing Jia Based on tutorial slides by Lester Mackey and Ariel Kleiner August, 2011 Outline 1 Probability 2 Statistics 3 Linear Algebra 4 Optimization
More information1 Review of Probability and Distributions
Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote
More informationIntroduction to Probability
Introduction to Probability SECOND EDITION Dimitri P. Bertsekas and John N. Tsitsiklis Massachusetts Institute of Technology Selected Summary Material All Rights Reserved WWW site for book information
More informationReview: mostly probability and some statistics
Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random
More information6.1 Moment Generating and Characteristic Functions
Chapter 6 Limit Theorems The power statistics can mostly be seen when there is a large collection of data points and we are interested in understanding the macro state of the system, e.g., the average,
More informationV7 Foundations of Probability Theory
V7 Foundations of Probability Theory Probability : degree of confidence that an event of an uncertain nature will occur. Events : we will assume that there is an agreed upon space of possible outcomes
More informationIntroduction to Machine Learning
Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola Computer Science & Engineering State University of New York at Buffalo Buffalo, NY, USA chandola@buffalo.edu Chandola@UB
More informationIntroduction to Probability Theory
Introduction to Probability Theory Ping Yu Department of Economics University of Hong Kong Ping Yu (HKU) Probability 1 / 39 Foundations 1 Foundations 2 Random Variables 3 Expectation 4 Multivariate Random
More informationSet Theory Digression
1 Introduction to Probability 1.1 Basic Rules of Probability Set Theory Digression A set is defined as any collection of objects, which are called points or elements. The biggest possible collection of
More informationChapter 3. Chapter 3 sections
sections 3.1 Random Variables and Discrete Distributions 3.2 Continuous Distributions 3.4 Bivariate Distributions 3.5 Marginal Distributions 3.6 Conditional Distributions 3.7 Multivariate Distributions
More informationMathematical Preliminaries
Mathematical Preliminaries Economics 3307 - Intermediate Macroeconomics Aaron Hedlund Baylor University Fall 2013 Econ 3307 (Baylor University) Mathematical Preliminaries Fall 2013 1 / 25 Outline I: Sequences
More informationContinuous Random Variables and Continuous Distributions
Continuous Random Variables and Continuous Distributions Continuous Random Variables and Continuous Distributions Expectation & Variance of Continuous Random Variables ( 5.2) The Uniform Random Variable
More informationCME 106: Review Probability theory
: Probability theory Sven Schmit April 3, 2015 1 Overview In the first half of the course, we covered topics from probability theory. The difference between statistics and probability theory is the following:
More informationLecture 1. ABC of Probability
Math 408 - Mathematical Statistics Lecture 1. ABC of Probability January 16, 2013 Konstantin Zuev (USC) Math 408, Lecture 1 January 16, 2013 1 / 9 Agenda Sample Spaces Realizations, Events Axioms of Probability
More informationSingle Maths B: Introduction to Probability
Single Maths B: Introduction to Probability Overview Lecturer Email Office Homework Webpage Dr Jonathan Cumming j.a.cumming@durham.ac.uk CM233 None! http://maths.dur.ac.uk/stats/people/jac/singleb/ 1 Introduction
More informationProbability and Statistics
Probability and Statistics Jane Bae Stanford University hjbae@stanford.edu September 16, 2014 Jane Bae (Stanford) Probability and Statistics September 16, 2014 1 / 35 Overview 1 Probability Concepts Probability
More informationAppendix A : Introduction to Probability and stochastic processes
A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More informationMathematical Statistics 1 Math A 6330
Mathematical Statistics 1 Math A 6330 Chapter 2 Transformations and Expectations Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 14, 2015 Outline 1 Distributions of Functions
More information1 Variance of a Random Variable
Indian Institute of Technology Bombay Department of Electrical Engineering Handout 14 EE 325 Probability and Random Processes Lecture Notes 9 August 28, 2014 1 Variance of a Random Variable The expectation
More informationPCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities
PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets
More informationReview of Probability. CS1538: Introduction to Simulations
Review of Probability CS1538: Introduction to Simulations Probability and Statistics in Simulation Why do we need probability and statistics in simulation? Needed to validate the simulation model Needed
More informationCS 246 Review of Proof Techniques and Probability 01/14/19
Note: This document has been adapted from a similar review session for CS224W (Autumn 2018). It was originally compiled by Jessica Su, with minor edits by Jayadev Bhaskaran. 1 Proof techniques Here we
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 17: Continuous random variables: conditional PDF Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin
More informationSome Basic Concepts of Probability and Information Theory: Pt. 2
Some Basic Concepts of Probability and Information Theory: Pt. 2 PHYS 476Q - Southern Illinois University January 22, 2018 PHYS 476Q - Southern Illinois University Some Basic Concepts of Probability and
More informationIntroduction to Stochastic Processes
Stat251/551 (Spring 2017) Stochastic Processes Lecture: 1 Introduction to Stochastic Processes Lecturer: Sahand Negahban Scribe: Sahand Negahban 1 Organization Issues We will use canvas as the course webpage.
More informationStatistics STAT:5100 (22S:193), Fall Sample Final Exam B
Statistics STAT:5 (22S:93), Fall 25 Sample Final Exam B Please write your answers in the exam books provided.. Let X, Y, and Y 2 be independent random variables with X N(µ X, σ 2 X ) and Y i N(µ Y, σ 2
More informationReview for Exam Spring 2018
Review for Exam 1 18.05 Spring 2018 Extra office hours Tuesday: David 3 5 in 2-355 Watch web site for more Friday, Saturday, Sunday March 9 11: no office hours March 2, 2018 2 / 23 Exam 1 Designed to be
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More informationContinuous Distributions
Continuous Distributions 1.8-1.9: Continuous Random Variables 1.10.1: Uniform Distribution (Continuous) 1.10.4-5 Exponential and Gamma Distributions: Distance between crossovers Prof. Tesler Math 283 Fall
More information1 Basic continuous random variable problems
Name M362K Final Here are problems concerning material from Chapters 5 and 6. To review the other chapters, look over previous practice sheets for the two exams, previous quizzes, previous homeworks and
More informationGeneral Random Variables
1/65 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Probability A general random variable is discrete, continuous, or mixed. A discrete random variable
More informationRandom Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.
Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example
More informationLecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
More informationProbability Models. 4. What is the definition of the expectation of a discrete random variable?
1 Probability Models The list of questions below is provided in order to help you to prepare for the test and exam. It reflects only the theoretical part of the course. You should expect the questions
More informationPart IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 1 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationF X (x) = P [X x] = x f X (t)dt. 42 Lebesgue-a.e, to be exact 43 More specifically, if g = f Lebesgue-a.e., then g is also a pdf for X.
10.2 Properties of PDF and CDF for Continuous Random Variables 10.18. The pdf f X is determined only almost everywhere 42. That is, given a pdf f for a random variable X, if we construct a function g by
More informationTwo hours. Statistical Tables to be provided THE UNIVERSITY OF MANCHESTER. 14 January :45 11:45
Two hours Statistical Tables to be provided THE UNIVERSITY OF MANCHESTER PROBABILITY 2 14 January 2015 09:45 11:45 Answer ALL four questions in Section A (40 marks in total) and TWO of the THREE questions
More informationMath-Stat-491-Fall2014-Notes-I
Math-Stat-491-Fall2014-Notes-I Hariharan Narayanan October 2, 2014 1 Introduction This writeup is intended to supplement material in the prescribed texts: Introduction to Probability Models, 10th Edition,
More informationData Modeling & Analysis Techniques. Probability & Statistics. Manfred Huber
Data Modeling & Analysis Techniques Probability & Statistics Manfred Huber 2017 1 Probability and Statistics Probability and statistics are often used interchangeably but are different, related fields
More informationDept. of Linguistics, Indiana University Fall 2015
L645 Dept. of Linguistics, Indiana University Fall 2015 1 / 34 To start out the course, we need to know something about statistics and This is only an introduction; for a fuller understanding, you would
More informationChapter 3 sections. SKIP: 3.10 Markov Chains. SKIP: pages Chapter 3 - continued
Chapter 3 sections Chapter 3 - continued 3.1 Random Variables and Discrete Distributions 3.2 Continuous Distributions 3.3 The Cumulative Distribution Function 3.4 Bivariate Distributions 3.5 Marginal Distributions
More informationCMPSCI 240: Reasoning about Uncertainty
CMPSCI 240: Reasoning about Uncertainty Lecture 2: Sets and Events Andrew McGregor University of Massachusetts Last Compiled: January 27, 2017 Outline 1 Recap 2 Experiments and Events 3 Probabilistic Models
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 8 10/1/2008 CONTINUOUS RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 8 10/1/2008 CONTINUOUS RANDOM VARIABLES Contents 1. Continuous random variables 2. Examples 3. Expected values 4. Joint distributions
More informationMath Bootcamp 2012 Miscellaneous
Math Bootcamp 202 Miscellaneous Factorial, combination and permutation The factorial of a positive integer n denoted by n!, is the product of all positive integers less than or equal to n. Define 0! =.
More information