Random variables. DS GA 1002 Probability and Statistics for Data Science.
|
|
- Lynne Patterson
- 6 years ago
- Views:
Transcription
1 Random variables DS GA 1002 Probability and Statistics for Data Science Carlos Fernandez-Granda
2 Motivation Random variables model numerical quantities that are uncertain They allow us to structure the information we have about these quantities in a principled way
3 Definition Given a probability space (Ω, F, P), a random variable X is a function from the sample space Ω to the real numbers R We use uppercase letters to denote random variables: X, Y,... Once the outcome ω Ω is revealed, X (ω) is the realization of X We use lowercase letters to denote numerical values: x, y,...
4 Characterization Given a probability space (Ω, F, P), for any set S P (X S) = P ({ω X (ω) S}). We will almost never construct probabilistic models like this!
5
6 Discrete random variables Discrete random variables take values on a finite or countably infinite subset of R such as the integers The probability mass function (pmf) of X is defined as p X (x) := P ({ω X (ω) = x}) In words, p X (x) is the probability that X equals x The pmf completely specifies a random variable
7 Probability mass function If D is the range of X, then ( D, 2 D ), p X is a valid probability space Any pmf satisfies p X (x) 0 for any x D, p X (x) = 1 x D P (X S) = x S p X (x) for any S D
8 Probability mass function px (x) x
9 Example P (X {1, 4}) P (X > 3)
10 Example P (X {1, 4}) = p X (1) + p X (4) = 0.5 P (X > 3)
11 Example P (X {1, 4}) = p X (1) + p X (4) = 0.5 P (X > 3) = p X (4) + p X (5) = 0.6
12 Defining a discrete random variable To define a discrete random variable X we just need A discrete range D A nonnegative function p X satisfying p X (x) = 1 x D
13 Bernoulli random variable Experiment with two possible outcomes (coin flip with bias p) p X (0) = 1 p p X (1) = p Special case: Indicator random variable of an event S { 1, if ω S, 1 S (ω) = 0, otherwise Bernoulli with parameter P (S) Allows to represent an event by a random variable
14 Example: Coin flips You flip a coin with bias p until you obtain heads (flips are independent) If you model the number of flips as a random variable X, what is p X?
15 Example: Coin flips p X (k)
16 Example: Coin flips p X (k) = P (k flips)
17 Example: Coin flips p X (k) = P (k flips) = P (1st flip = tails,..., k 1th flip = tails, kth flip = heads)
18 Example: Coin flips p X (k) = P (k flips) = P (1st flip = tails,..., k 1th flip = tails, kth flip = heads) = P (1st flip = tails) P (k 1th flip = tails) P (kth flip = heads)
19 Example: Coin flips p X (k) = P (k flips) = P (1st flip = tails,..., k 1th flip = tails, kth flip = heads) = P (1st flip = tails) P (k 1th flip = tails) P (kth flip = heads) = (1 p) k 1 p
20 Geometric random variable The pmf of a geometric random variable with parameter p is p X (k) = (1 p) k 1 p k = 1, 2,...
21 Geometric random variable p = px (k) k
22 Geometric random variable p = px (k) k
23 Geometric random variable p = px (k) k
24 Example: Coin flips You flip a coin with bias p n times (flips are independent) If you model the number of heads as a random variable X, what is p X?
25 Example: Coin flips What is the probability of getting k heads and then n k tails? P (k heads, then n k tails)
26 Example: Coin flips What is the probability of getting k heads and then n k tails? P (k heads, then n k tails) = P (1st = heads,..., kth = heads, k + 1th = tails,..., nth = tails)
27 Example: Coin flips What is the probability of getting k heads and then n k tails? P (k heads, then n k tails) = P (1st = heads,..., kth = heads, k + 1th = tails,..., nth = tails) = P (1st = heads) P (kth = heads) P (k + 1th = tails) P (nth = tails)
28 Example: Coin flips What is the probability of getting k heads and then n k tails? P (k heads, then n k tails) = P (1st = heads,..., kth = heads, k + 1th = tails,..., nth = tails) = P (1st = heads) P (kth = heads) P (k + 1th = tails) P (nth = tails) = p k (1 p) n k
29 Example: Coin flips Any fixed order of k heads and n k tails has the same probability
30 Example: Coin flips Any fixed order of k heads and n k tails has the same probability We are interested in the union of these events
31 Example: Coin flips Any fixed order of k heads and n k tails has the same probability We are interested in the union of these events Can we just add their probabilities?
32 Example: Coin flips Any fixed order of k heads and n k tails has the same probability We are interested in the union of these events Can we just add their probabilities? How many possible orders are there?
33 Example: Coin flips Any fixed order of k heads and n k tails has the same probability We are interested in the union of these events Can we just add their probabilities? How many possible orders are there? ( ) n := k n! k! (n k)!
34 Example: Coin flips Any fixed order of k heads and n k tails has the same probability We are interested in the union of these events Can we just add their probabilities? How many possible orders are there? ( ) n := k p X (k) = n! k! (n k)! ( ) n p k (1 p) n k k
35 Binomial random variable The pmf of a binomial random variable with parameters n and p is p X (k) = ( ) n p k (1 p) n k, k k = 0, 1, 2,..., n
36 Binomial random variable n = 20, p = px (k) k
37 Binomial random variable n = 20, p = px (k) k
38 Binomial random variable n = 20, p = px (k) k
39 Example: Call center Model the number of calls received per day Assumptions: 1. Each call occurs independently from every other call 2. A given call has the same probability of occurring at any given time of the day 3. Calls occur at a rate of λ calls per day
40 Example: Call center Discretize day into n slots
41 Example: Call center Discretize day into n slots Probability of receiving m calls in one slot?
42 Example: Call center Discretize day into n slots Probability of receiving m calls in one slot? (λ/n) m
43 Example: Call center Discretize day into n slots Probability of receiving m calls in one slot? (λ/n) m If n is large enough λ/n >> (λ/n) m for all m > 1
44 Example: Call center Discretize day into n slots Probability of receiving m calls in one slot? (λ/n) m If n is large enough λ/n >> (λ/n) m for all m > 1 Assume that in each slot we either receive one call or none at all. What is probability of k calls in a day?
45 Example: Call center Discretize day into n slots Probability of receiving m calls in one slot? (λ/n) m If n is large enough λ/n >> (λ/n) m for all m > 1 Assume that in each slot we either receive one call or none at all. What is probability of k calls in a day? Binomial with parameters n and p := λ/n!
46 Example: Call center P (k calls during the day )
47 Example: Call center P (k calls during the day ) = lim P (k calls in n small intervals) n
48 Example: Call center P (k calls during the day ) = lim P (k calls in n small intervals) n ( ) n = lim p k (1 p) (n k) n k
49 Example: Call center P (k calls during the day ) = lim n = lim n = lim n P (k calls in n small intervals) ( ) n p k (1 p) (n k) k ( n k ) ( λ n ) k ( 1 λ ) (n k) n
50 Example: Call center P (k calls during the day ) = lim n = lim n = lim n = lim n P (k calls in n small intervals) ( ) n p k (1 p) (n k) k ( n k ) ( λ n ) k ( 1 λ ) (n k) n n! λ k k! (n k)! (n λ) k ( 1 λ ) n n
51 Example: Call center P (k calls during the day ) = lim n = lim n = lim n = lim n P (k calls in n small intervals) ( ) n p k (1 p) (n k) k ( n k = λk e λ k! ) ( λ n ) k ( 1 λ ) (n k) n n! λ k k! (n k)! (n λ) k ( 1 λ ) n n Identity proved in the notes lim n n! (n k)! (n λ) k ( 1 λ ) n = e λ n
52 Poisson random variable The pmf of a Poisson random variable with parameter λ is p X (k) = λk e λ k! k = 0, 1, 2,...
53 Poisson random variable λ = px (k) k
54 Poisson random variable λ = px (k) k
55 Poisson random variable λ = px (k) k
56 Example: Call center Pmf of binomial with parameters n and p = λ n Poisson with parameter λ converges to pmf of This is an example of convergence in distribution
57 Binomial random variable n = 40, p = k
58 Binomial random variable n = 80, p = k
59 Binomial random variable n = 400, p = k
60 Poisson random variable λ = k
61 Call-center data Assumptions do not hold over the whole day (why?) They do hold (approximately) for intervals of time Example: Data from a call center in Israel We compare the histogram of the number of calls received in an interval of 4 hours over 2 months and the pmf of a Poisson random variable fitted to the data
62 Call-center data Real data Poisson distribution Number of calls
63
64 Continuous random variables Useful to model continuous quantities without discretizing Assigning nonzero probabilities to events of the form {X = x} for x R doesn t work! Instead, we only consider events of the form {X S} where S is a union of intervals (formally a Borel set) We cannot consider every possible subset of R for technical reasons
65 Cumulative distribution function The cumulative distribution function (cdf) of X is defined as F X (x) := P ({X (ω) x : ω Ω}) = P (X x) In words, F X (x) is the probability of X being smaller than x The cdf can be defined for both continuous and discrete random variables
66 Cumulative distribution function The cdf completely specifies the distribution of the random variable The probability of any interval (a, b] is given by P (a < X b) = P (X b) P (X a) = F X (b) F X (a) To define a continuous random variable we just need a valid cdf! A valid underlying probability space exists, but we don t need to worry about it
67 Properties of the cdf lim F X (x) = 0, x lim F X (x) = 1, x F X (b) F X (a) if b > a, i.e. F X is nondecreasing
68 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3
69 Example 1 FX (x) x
70 Example P (0.5 < X 2.5) =
71 Example P (0.5 < X 2.5) = F X (2.5) F X (0.5) = 0.375
72 Example 1 FX (x) 0.5 P(X (0.5, 2.5]) x
73 Probability density function When the cdf is differentiable, its derivative can be interpreted as a density Probability density function f X (x) := df X (x) d x The pdf is not a probability measure! (It can be greater than 1)
74 Probability density function By the fundamental theorem of calculus Intuitively, P (a < X b) = F X (b) F X (a) = b a f X (x) dx lim P (X (x, x + )) = f X (x) 0
75 Properties of the pdf For any union of intervals (any Borel set) S P (X S) = f X (x) dx In particular, S f X (x) dx = 1 From the monotonicity of the cdf f X (x) 0
76 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3 f X (x) = for x < 0, for 0 x 1 for 1 x 2 for 2 x 3 for x > 3.
77 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3 f X (x) = 0 for x < 0, for 0 x 1 for 1 x 2 for 2 x 3 for x > 3.
78 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3 0 for x < 0, 0.5 for 0 x 1 f X (x) = for 1 x 2 for 2 x 3 for x > 3.
79 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3 0 for x < 0, 0.5 for 0 x 1 f X (x) = 0 for 1 x 2 for 2 x 3 for x > 3.
80 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3 0 for x < 0, 0.5 for 0 x 1 f X (x) = 0 for 1 x 2 x 2 for 2 x 3 for x > 3.
81 Example 0 for x < x for 0 x 1 F X (x) := 0.5 ( for 1 x (x 2) 2) for 2 x 3 1 for x > 3 0 for x < 0, 0.5 for 0 x 1 f X (x) = 0 for 1 x 2 x 2 for 2 x 3 0 for x > 3.
82 Example FX (x) x fx (x) x
83 Example P (0.5 < X 2.5) =
84 Example P (0.5 < X 2.5) = f X (x) dx
85 Example P (0.5 < X 2.5) = = f X (x) dx 0.5 dx x 2 dx = 0.375
86 Example 1 P (X (0.5, 2.5]) fx (x) x
87 Uniform random variable Pdf of a uniform random variable with domain [a, b]: { 1 f X (x) = b a, if a x b, 0, otherwise
88 Uniform random variable in [a, b] 1 b a 1 fx (x) FX (x) 0 0 a x b a x b
89 Exponential random variable Used to model waiting times (time until a certain event occurs) Examples: decay of a radioactive particle, telephone call, mechanical failure of a device Pdf of an exponential random variable with parameter λ: { λe λx, if x 0, f X (x) = 0, otherwise
90 Exponential random variables 1.5 λ = 0.5 λ = 1.0 λ = 1.5 fx (x) x
91 Call-center data Example: Data from a call center in Israel We compare the histogram of the inter-arrival times between calls occurring between 8 pm and midnight over two days and the pdf of an exponential random variable fitted to the data
92 Call center Exponential distribution Real data Interarrival times (s)
93 Gaussian or normal random variable Extremely popular in probabilistic models and statistics Sums of independent random variables converge to Gaussian distributions under certain assumptions Pdf of a Gaussian random variable with mean µ and standard deviation σ f X (x) = 1 e (x µ)2 2σ 2 2πσ
94 Gaussian random variables µ = 2 σ = 1 µ = 0 σ = 2 µ = 0 σ = 4 fx (x) x
95 Height data Example: Data from a population of people We compare the histogram of the heights and the pdf of a Gaussian random variable fitted to the data
96 Height data Gaussian distribution Real data Height (inches)
97 Problem The Gaussian cdf does not have a closed form solution This complicates computing the probability that a Gaussian belongs to a set
98 Standard Gaussian If X is Gaussian with mean µ and standard deviation σ, then U := X µ σ is a standard Gaussian, with mean zero and unit standard deviation ( [ X µ a µ P (X [a, b]) = P σ σ, b µ ]) σ ( ) ( ) b µ a µ = Φ Φ σ σ Φ is the cdf of a standard Gaussian
99 Beta random variable Useful in Bayesian statistics Unimodal continuous distribution in the unit interval The pdf of a beta distribution with parameters a and b is defined as f β (θ; a, b) := { θ a 1 (1 θ) b 1 β(a,b), if 0 θ 1, 0 otherwise β (a, b) := u a 1 (1 u) b 1 du u
100 Beta random variables fx (x) a = 1 b = 1 a = 1 b = 2 a = 3 b = 3 a = 6 b = 2 a = 3 b = x
101
102 Conditioning on an event We usually define random variables using their pmf, cdf or pdf How can we incorporate the information that X S for some set S?
103 Conditional pmf If X has pmf p X, the conditional pmf of X given X S is p X X S (x) := P (X = x X S) { px (x) = s S p X (s) if x S 0 otherwise. Valid pmf in the new probability space restricted to the event {X S}
104 Conditional cdf If X has pdf f X, the conditional cdf of X given X S is F X X S (x) := P (X x X S) P (X x, X S) = P (X S) u x,u S = f X (u) du u S f X (u) du Valid cdf in the new probability space restricted to the event {X S}
105 Example: Geometric random variables are memoryless We flip a coin repeatedly until we obtain heads, but pause after k 0 flips (which were tails) What is the probability of obtaining heads in k more flips?
106 Example: Geometric random variables are memoryless P (k more flips)
107 Example: Geometric random variables are memoryless P (k more flips) = p X X >k0 (k)
108 Example: Geometric random variables are memoryless P (k more flips) = p X X >k0 (k) = p X (k) m=k 0 +1 p X (m)
109 Example: Geometric random variables are memoryless P (k more flips) = p X X >k0 (k) = = p X (k) m=k 0 +1 p X (m) (1 p) k 1 p m=k 0 +1 (1 p)m 1 p
110 Example: Geometric random variables are memoryless P (k more flips) = p X X >k0 (k) = = p X (k) m=k 0 +1 p X (m) (1 p) k 1 p m=k 0 +1 (1 p)m 1 p = (1 p) k k 0 1 p for k > k 0 Geometric series: m=k 0 +1 α m 1 = αk 0 1 α for any α < 1
111 Example: Exponential random variables are memoryless Assume inter-arrival times are exponential with parameter λ You get an , then no for t 0 minutes How is the waiting time until the next distributed now?
112 Example: Exponential random variables are memoryless F T T >t0 (t)
113 Example: Exponential random variables are memoryless F T T >t0 (t) = t t 0 f T (u) du f T (u) du t 0
114 Example: Exponential random variables are memoryless F T T >t0 (t) = = t t 0 f T (u) du f T (u) du t 0 t t 0 λe λu du λe λu du t 0
115 Example: Exponential random variables are memoryless F T T >t0 (t) = = t t 0 f T (u) du f T (u) du t 0 t t 0 λe λu du λe λu du t 0 = e λt e λt 0 e λt 0
116 Example: Exponential random variables are memoryless F T T >t0 (t) = = t t 0 f T (u) du f T (u) du t 0 t t 0 λe λu du λe λu du t 0 = e λt e λt 0 e λt 0 = 1 e λ(t t 0) for t > t 0
117 Example: Exponential random variables are memoryless F T T >t0 (t) = = t t 0 f T (u) du f T (u) du t 0 t t 0 λe λu du λe λu du t 0 = e λt e λt 0 e λt 0 = 1 e λ(t t 0) for t > t 0 Differentiating with respect to t f T T >t0 (t) = λe λ(t t 0) for t > t 0
118
119 Functions of random variables For any deterministic function g and r.v. X, Y := g (X ) is a random variable Formally, X maps elements of Ω to R, so Y does too since Y (ω) = g (X (ω))
120 Discrete random variables If X is discrete p Y (y) = P (Y = y) = P (g (X ) = y) = p X (x) {x g(x)=y}
121 Continuous random variables If X is continuous F Y (y) = P (Y y) = P (g (X ) y) = f X (x) dx, {x g(x) y} Then we can differentiate to obtain the pdf f Y
122 Gaussian random variable If X is a Gaussian random variable with mean µ and standard deviation σ, derive the distribution of U := X µ σ
123 Gaussian random variable F U (u)
124 Gaussian random variable ( ) X µ F U (u) = P u σ
125 Gaussian random variable ( X µ F U (u) = P σ = (x µ)/σ u ) u 1 2πσ e (x µ)2 2σ 2 dx
126 Gaussian random variable ( X µ F U (u) = P σ = = (x µ)/σ u u ) u 1 e (x µ)2 2σ 2 dx 2πσ 1 2π e w2 2 dw by the change of variables w = x µ σ
127 Gaussian random variable ( X µ F U (u) = P σ = = (x µ)/σ u u ) u 1 e (x µ)2 2σ 2 dx 2πσ 1 2π e w2 2 dw by the change of variables w = x µ σ To obtain the pdf we differentiate with respect to u
128 Gaussian random variable ( X µ F U (u) = P σ = = (x µ)/σ u u ) u 1 e (x µ)2 2σ 2 dx 2πσ 1 2π e w2 2 dw by the change of variables w = x µ σ To obtain the pdf we differentiate with respect to u f U (u) = 1 2π e u2 2
129 Gaussian random variable ( X µ F U (u) = P σ = = (x µ)/σ u u ) u 1 e (x µ)2 2σ 2 dx 2πσ 1 2π e w2 2 dw by the change of variables w = x µ σ To obtain the pdf we differentiate with respect to u f U (u) = 1 2π e u2 2 U is a standard Gaussian random variable
130
131 Generating random variables Simulation is crucial to leverage probabilistic models effectively (life is not a homework problem!) It requires being able to sample from arbitrary distributions General approach: 1. Generate samples uniformly from the unit interval [0, 1] 2. Transform the samples so that they have the desired distribution
132 Sampling from a discrete distribution Aim: Generate a discrete random variable X with pmf p X using samples from a uniform random variable U Possible values of X : x 1 < x 2 <... How can we assign samples u 1, u 2,... from U to x 1, x 2,...?
133 Sampling from a discrete distribution x 3 x 2 x 1 0 u 1 u 2 u 3 u 4 u 5 1
134 Sampling from a discrete distribution Idea: Assign samples in an interval of length p X (x i ) to x i
135 Sampling from a discrete distribution x 3 x 2 x 1 0 u 1 u 2 u 3 u 4 u 5 1
136 Sampling from a discrete distribution x 3 x 2 x 1 0 u 1 u 2 u 3 u 4 u 5 1
137 Sampling from a discrete distribution x 1 if 0 U p X (x 1 ) x 2 if p X (x 1 ) U p X (x 1 ) + p X (x 2 ) X =... x i if i 1 j=1 p X (x j ) U i j=1 p X (x j )...
138 Sampling from a discrete distribution x 1 if 0 U F X (x 1 ) x 2 if F X (x 1 ) U F X (x 2 ) X =... x i if F X (x i 1 ) U F X (x i )...
139 Sampling from a discrete distribution x 3 x 2 x 1 0 u 1 F X (x 1 ) u 2 u 3 u 4 F X (x 2 ) u 5 1
140 Inverse-transform sampling Aim: Generate a continuous random variable X with cdf F X using samples from a uniform random variable U Algorithm: 1. Obtain a sample u of U 2. Set x := F 1 X (u)
141 Inverse-transform sampling F Y (y)
142 Inverse-transform sampling F Y (y) = P (Y y)
143 Inverse-transform sampling F Y (y) = P (Y y) = P ( F 1 X (U) y)
144 Inverse-transform sampling F Y (y) = P (Y y) = P ( F 1 X (U) y) = P (U F X (y))
145 Inverse-transform sampling F Y (y) = P (Y y) = P ( F 1 X (U) y) = P (U F X (y)) = FX (y) u=0 du
146 Inverse-transform sampling F Y (y) = P (Y y) = P ( F 1 X (U) y) = P (U F X (y)) = FX (y) u=0 = F X (y) du
147 Generating an exponential random variable Aim: Generate an exponential random variable X with parameter λ F X (x) := 1 e λx F 1 X (u) = 1 ( ) 1 λ log 1 u F 1 X (U) is an exponential random variable with parameter λ
148 Generating an exponential random variable F 1 X (u 5) F 1 X (u 4) F 1 X (u 3) F 1 F 1 X (u 2) X (u 1) 0 u 1 u 2 u 3 u 4 u 51
DS-GA 1002 Lecture notes 2 Fall Random variables
DS-GA 12 Lecture notes 2 Fall 216 1 Introduction Random variables Random variables are a fundamental tool in probabilistic modeling. They allow us to model numerical quantities that are uncertain: the
More informationProbability and Statistics for Data Science. Carlos Fernandez-Granda
Probability and Statistics for Data Science Carlos Fernandez-Granda Preface These notes were developed for the course Probability and Statistics for Data Science at the Center for Data Science in NYU.
More informationRandom Processes. DS GA 1002 Probability and Statistics for Data Science.
Random Processes DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Modeling quantities that evolve in time (or space)
More informationNorthwestern University Department of Electrical Engineering and Computer Science
Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationCMPSCI 240: Reasoning Under Uncertainty
CMPSCI 240: Reasoning Under Uncertainty Lecture 5 Prof. Hanna Wallach wallach@cs.umass.edu February 7, 2012 Reminders Pick up a copy of B&T Check the course website: http://www.cs.umass.edu/ ~wallach/courses/s12/cmpsci240/
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationBrief Review of Probability
Maura Department of Economics and Finance Università Tor Vergata Outline 1 Distribution Functions Quantiles and Modes of a Distribution 2 Example 3 Example 4 Distributions Outline Distribution Functions
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationRandom variable X is a mapping that maps each outcome s in the sample space to a unique real number x, x. X s. Real Line
Random Variable Random variable is a mapping that maps each outcome s in the sample space to a unique real number,. s s : outcome Sample Space Real Line Eamples Toss a coin. Define the random variable
More informationLecture Notes 2 Random Variables. Random Variable
Lecture Notes 2 Random Variables Definition Discrete Random Variables: Probability mass function (pmf) Continuous Random Variables: Probability density function (pdf) Mean and Variance Cumulative Distribution
More informationLecture Notes 2 Random Variables. Discrete Random Variables: Probability mass function (pmf)
Lecture Notes 2 Random Variables Definition Discrete Random Variables: Probability mass function (pmf) Continuous Random Variables: Probability density function (pdf) Mean and Variance Cumulative Distribution
More informationSample Spaces, Random Variables
Sample Spaces, Random Variables Moulinath Banerjee University of Michigan August 3, 22 Probabilities In talking about probabilities, the fundamental object is Ω, the sample space. (elements) in Ω are denoted
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 8 10/1/2008 CONTINUOUS RANDOM VARIABLES
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 8 10/1/2008 CONTINUOUS RANDOM VARIABLES Contents 1. Continuous random variables 2. Examples 3. Expected values 4. Joint distributions
More informationEE/CpE 345. Modeling and Simulation. Fall Class 5 September 30, 2002
EE/CpE 345 Modeling and Simulation Class 5 September 30, 2002 Statistical Models in Simulation Real World phenomena of interest Sample phenomena select distribution Probabilistic, not deterministic Model
More informationSlides 8: Statistical Models in Simulation
Slides 8: Statistical Models in Simulation Purpose and Overview The world the model-builder sees is probabilistic rather than deterministic: Some statistical model might well describe the variations. An
More informationMultivariate random variables
Multivariate random variables DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Joint distributions Tool to characterize several
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 20, 2012 Introduction Introduction The world of the model-builder
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationStatistics and Econometrics I
Statistics and Econometrics I Random Variables Shiu-Sheng Chen Department of Economics National Taiwan University October 5, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 5, 2016
More information(Ch 3.4.1, 3.4.2, 4.1, 4.2, 4.3)
3 Probability Distributions (Ch 3.4.1, 3.4.2, 4.1, 4.2, 4.3) Probability Distribution Functions Probability distribution function (pdf): Function for mapping random variables to real numbers. Discrete
More informationGuidelines for Solving Probability Problems
Guidelines for Solving Probability Problems CS 1538: Introduction to Simulation 1 Steps for Problem Solving Suggested steps for approaching a problem: 1. Identify the distribution What distribution does
More informationMultivariate random variables
DS-GA 002 Lecture notes 3 Fall 206 Introduction Multivariate random variables Probabilistic models usually include multiple uncertain numerical quantities. In this section we develop tools to characterize
More informationProbability and Distributions
Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated
More information(Ch 3.4.1, 3.4.2, 4.1, 4.2, 4.3)
3 Probability Distributions (Ch 3.4.1, 3.4.2, 4.1, 4.2, 4.3) Probability Distribution Functions Probability distribution function (pdf): Function for mapping random variables to real numbers. Discrete
More informationThe random variable 1
The random variable 1 Contents 1. Definition 2. Distribution and density function 3. Specific random variables 4. Functions of one random variable 5. Mean and variance 2 The random variable A random variable
More informationX 1 ((, a]) = {ω Ω : X(ω) a} F, which leads us to the following definition:
nna Janicka Probability Calculus 08/09 Lecture 4. Real-valued Random Variables We already know how to describe the results of a random experiment in terms of a formal mathematical construction, i.e. the
More informationThe exponential distribution and the Poisson process
The exponential distribution and the Poisson process 1-1 Exponential Distribution: Basic Facts PDF f(t) = { λe λt, t 0 0, t < 0 CDF Pr{T t) = 0 t λe λu du = 1 e λt (t 0) Mean E[T] = 1 λ Variance Var[T]
More informationChapter 2: Random Variables
ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:
More informationFundamental Tools - Probability Theory II
Fundamental Tools - Probability Theory II MSc Financial Mathematics The University of Warwick September 29, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory II 1 / 22 Measurable random
More informationProbability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27
Probability Review Yutian Li Stanford University January 18, 2018 Yutian Li (Stanford University) Probability Review January 18, 2018 1 / 27 Outline 1 Elements of probability 2 Random variables 3 Multiple
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationReview of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
More informationRandom Variables. Definition: A random variable (r.v.) X on the probability space (Ω, F, P) is a mapping
Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the
More informationChapter 1. Sets and probability. 1.3 Probability space
Random processes - Chapter 1. Sets and probability 1 Random processes Chapter 1. Sets and probability 1.3 Probability space 1.3 Probability space Random processes - Chapter 1. Sets and probability 2 Probability
More informationProbability Density Functions and the Normal Distribution. Quantitative Understanding in Biology, 1.2
Probability Density Functions and the Normal Distribution Quantitative Understanding in Biology, 1.2 1. Discrete Probability Distributions 1.1. The Binomial Distribution Question: You ve decided to flip
More informationChapter 1 Statistical Reasoning Why statistics? Section 1.1 Basics of Probability Theory
Chapter 1 Statistical Reasoning Why statistics? Uncertainty of nature (weather, earth movement, etc. ) Uncertainty in observation/sampling/measurement Variability of human operation/error imperfection
More information2 Continuous Random Variables and their Distributions
Name: Discussion-5 1 Introduction - Continuous random variables have a range in the form of Interval on the real number line. Union of non-overlapping intervals on real line. - We also know that for any
More informationProbability (continued)
DS-GA 1002 Lecture notes 2 September 21, 15 Probability (continued) 1 Random variables (continued) 1.1 Conditioning on an event Given a random variable X with a certain distribution, imagine that it is
More informationChapter 2. Random Variable. Define single random variables in terms of their PDF and CDF, and calculate moments such as the mean and variance.
Chapter 2 Random Variable CLO2 Define single random variables in terms of their PDF and CDF, and calculate moments such as the mean and variance. 1 1. Introduction In Chapter 1, we introduced the concept
More informationChapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University
Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationECE 302: Probabilistic Methods in Electrical Engineering
ECE 302: Probabilistic Methods in Electrical Engineering Test I : Chapters 1 3 3/22/04, 7:30 PM Print Name: Read every question carefully and solve each problem in a legible and ordered manner. Make sure
More informationSTAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
More informationContinuous Probability Spaces
Continuous Probability Spaces Ω is not countable. Outcomes can be any real number or part of an interval of R, e.g. heights, weights and lifetimes. Can not assign probabilities to each outcome and add
More informationStat 512 Homework key 2
Stat 51 Homework key October 4, 015 REGULAR PROBLEMS 1 Suppose continuous random variable X belongs to the family of all distributions having a linear probability density function (pdf) over the interval
More informationRandom variable X is a mapping that maps each outcome s in the sample space to a unique real number x, < x <. ( ) X s. Real Line
Random Variable Random variable is a mapping that maps each outcome s in the sample space to a unique real number, <
More informationMoments. Raw moment: February 25, 2014 Normalized / Standardized moment:
Moments Lecture 10: Central Limit Theorem and CDFs Sta230 / Mth 230 Colin Rundel Raw moment: Central moment: µ n = EX n ) µ n = E[X µ) 2 ] February 25, 2014 Normalized / Standardized moment: µ n σ n Sta230
More informationStochastic Models in Computer Science A Tutorial
Stochastic Models in Computer Science A Tutorial Dr. Snehanshu Saha Department of Computer Science PESIT BSC, Bengaluru WCI 2015 - August 10 to August 13 1 Introduction 2 Random Variable 3 Introduction
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More informationDS-GA 1002 Lecture notes 11 Fall Bayesian statistics
DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian
More informationComputer Science, Informatik 4 Communication and Distributed Systems. Simulation. Discrete-Event System Simulation. Dr.
Simulation Discrete-Event System Simulation Chapter 4 Statistical Models in Simulation Purpose & Overview The world the model-builder sees is probabilistic rather than deterministic. Some statistical model
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More informationContinuous Distributions
Continuous Distributions 1.8-1.9: Continuous Random Variables 1.10.1: Uniform Distribution (Continuous) 1.10.4-5 Exponential and Gamma Distributions: Distance between crossovers Prof. Tesler Math 283 Fall
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationNotes on Continuous Random Variables
Notes on Continuous Random Variables Continuous random variables are random quantities that are measured on a continuous scale. They can usually take on any value over some interval, which distinguishes
More informationMath Review Sheet, Fall 2008
1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the
More informationChapter 2 Random Variables
Stochastic Processes Chapter 2 Random Variables Prof. Jernan Juang Dept. of Engineering Science National Cheng Kung University Prof. Chun-Hung Liu Dept. of Electrical and Computer Eng. National Chiao Tung
More informationCommon ontinuous random variables
Common ontinuous random variables CE 311S Earlier, we saw a number of distribution families Binomial Negative binomial Hypergeometric Poisson These were useful because they represented common situations:
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 29, 2014 Introduction Introduction The world of the model-builder
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationChapter 5. Statistical Models in Simulations 5.1. Prof. Dr. Mesut Güneş Ch. 5 Statistical Models in Simulations
Chapter 5 Statistical Models in Simulations 5.1 Contents Basic Probability Theory Concepts Discrete Distributions Continuous Distributions Poisson Process Empirical Distributions Useful Statistical Models
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 14: Continuous random variables Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/
More information1 Random Variable: Topics
Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationDiscrete Random Variables
CPSC 53 Systems Modeling and Simulation Discrete Random Variables Dr. Anirban Mahanti Department of Computer Science University of Calgary mahanti@cpsc.ucalgary.ca Random Variables A random variable is
More informationExample A. Define X = number of heads in ten tosses of a coin. What are the values that X may assume?
Stat 400, section.1-.2 Random Variables & Probability Distributions notes by Tim Pilachowski For a given situation, or experiment, observations are made and data is recorded. A sample space S must contain
More informationMFM Practitioner Module: Quantitative Risk Management. John Dodson. September 23, 2015
MFM Practitioner Module: Quantitative Risk Management September 23, 2015 Mixtures Mixtures Mixtures Definitions For our purposes, A random variable is a quantity whose value is not known to us right now
More informationLecture 8 : The Geometric Distribution
0/ 24 The geometric distribution is a special case of negative binomial, it is the case r = 1. It is so important we give it special treatment. Motivating example Suppose a couple decides to have children
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationStat 426 : Homework 1.
Stat 426 : Homework 1. Moulinath Banerjee University of Michigan Announcement: The homework carries 120 points and contributes 10 points to the total grade. (1) A geometric random variable W takes values
More informationProbability Review. Gonzalo Mateos
Probability Review Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ September 11, 2018 Introduction
More informationRandom Variables Example:
Random Variables Example: We roll a fair die 6 times. Suppose we are interested in the number of 5 s in the 6 rolls. Let X = number of 5 s. Then X could be 0, 1, 2, 3, 4, 5, 6. X = 0 corresponds to the
More information1 Review of Probability
1 Review of Probability Random variables are denoted by X, Y, Z, etc. The cumulative distribution function (c.d.f.) of a random variable X is denoted by F (x) = P (X x), < x
More informationBINOMIAL DISTRIBUTION
BINOMIAL DISTRIBUTION The binomial distribution is a particular type of discrete pmf. It describes random variables which satisfy the following conditions: 1 You perform n identical experiments (called
More informationBasics of Stochastic Modeling: Part II
Basics of Stochastic Modeling: Part II Continuous Random Variables 1 Sandip Chakraborty Department of Computer Science and Engineering, INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR August 10, 2016 1 Reference
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationBayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics
More informationContinuous-Valued Probability Review
CS 6323 Continuous-Valued Probability Review Prof. Gregory Provan Department of Computer Science University College Cork 2 Overview Review of discrete distributions Continuous distributions 3 Discrete
More informationTopic 3: The Expectation of a Random Variable
Topic 3: The Expectation of a Random Variable Course 003, 2017 Page 0 Expectation of a discrete random variable Definition (Expectation of a discrete r.v.): The expected value (also called the expectation
More informationBMIR Lecture Series on Probability and Statistics Fall, 2015 Uniform Distribution
Lecture #5 BMIR Lecture Series on Probability and Statistics Fall, 2015 Department of Biomedical Engineering and Environmental Sciences National Tsing Hua University s 5.1 Definition ( ) A continuous random
More informationTransform Techniques - CF
Transform Techniques - CF [eview] Moment Generating Function For a real t, the MGF of the random variable is t t M () t E[ e ] e Characteristic Function (CF) k t k For a real ω, the characteristic function
More informationProbability, Random Processes and Inference
INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx
More informationRandom Variable. Pr(X = a) = Pr(s)
Random Variable Definition A random variable X on a sample space Ω is a real-valued function on Ω; that is, X : Ω R. A discrete random variable is a random variable that takes on only a finite or countably
More informationfunctions Poisson distribution Normal distribution Arbitrary functions
Physics 433: Computational Physics Lecture 6 Random number distributions Generation of random numbers of various distribuition functions Normal distribution Poisson distribution Arbitrary functions Random
More informationContinuous Random Variables
Continuous Random Variables Recall: For discrete random variables, only a finite or countably infinite number of possible values with positive probability. Often, there is interest in random variables
More informationProbability distributions. Probability Distribution Functions. Probability distributions (contd.) Binomial distribution
Probability distributions Probability Distribution Functions G. Jogesh Babu Department of Statistics Penn State University September 27, 2011 http://en.wikipedia.org/wiki/probability_distribution We discuss
More informationLecture 2. Distributions and Random Variables
Lecture 2. Distributions and Random Variables Igor Rychlik Chalmers Department of Mathematical Sciences Probability, Statistics and Risk, MVE300 Chalmers March 2013. Click on red text for extra material.
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 17: Continuous random variables: conditional PDF Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin
More informationRelationship between probability set function and random variable - 2 -
2.0 Random Variables A rat is selected at random from a cage and its sex is determined. The set of possible outcomes is female and male. Thus outcome space is S = {female, male} = {F, M}. If we let X be
More informationName: Firas Rassoul-Agha
Midterm 1 - Math 5010 - Spring 016 Name: Firas Rassoul-Agha Solve the following 4 problems. You have to clearly explain your solution. The answer carries no points. Only the work does. CALCULATORS ARE
More informationChapter 4: Continuous Probability Distributions
Chapter 4: Continuous Probability Distributions Seungchul Baek Department of Statistics, University of South Carolina STAT 509: Statistics for Engineers 1 / 57 Continuous Random Variable A continuous random
More informationPart I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationHW7 Solutions. f(x) = 0 otherwise. 0 otherwise. The density function looks like this: = 20 if x [10, 90) if x [90, 100]
HW7 Solutions. 5 pts.) James Bond James Bond, my favorite hero, has again jumped off a plane. The plane is traveling from from base A to base B, distance km apart. Now suppose the plane takes off from
More informationThings to remember when learning probability distributions:
SPECIAL DISTRIBUTIONS Some distributions are special because they are useful They include: Poisson, exponential, Normal (Gaussian), Gamma, geometric, negative binomial, Binomial and hypergeometric distributions
More information3 Continuous Random Variables
Jinguo Lian Math437 Notes January 15, 016 3 Continuous Random Variables Remember that discrete random variables can take only a countable number of possible values. On the other hand, a continuous random
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More information