Random Processes. DS GA 1002 Probability and Statistics for Data Science.
|
|
- Kathleen Montgomery
- 5 years ago
- Views:
Transcription
1 Random Processes DS GA 1002 Probability and Statistics for Data Science Carlos Fernandez-Granda
2 Aim Modeling quantities that evolve in time (or space) Trajectory of a particle, price of oil, temperature in New York...
3 Definition Mean and autocovariance functions Important random processes
4 Notation We denote random processes using a tilde over an upper case letter X
5 Formal definition Given a probability space (Ω, F, P), a random process X is a function that maps each ω Ω to a function X (ω, ) : T R There are two interpretations for X (ω, t) 1. If we fix ω, then X (ω, t) is a deterministic function or realization of t 2. If we fix t then X (ω, t) is a random variable, usually denoted by X (t)
6 Continuous and discrete random processes We can classify a random process depending on the indexing variable t If t is defined on a continuous interval, the process is continuous If t is defined on a discrete set, the process is discrete
7 State space Set of possible values of the random variable X (t) for any t It can be continuous or discrete (also finite) There are continuous-state discrete-time random processes and discrete-state continuous-time random processes
8 Puddle Initial amount of water is uniform between 0 and 1 gallon After a time interval t there is t times less water Continuous-state continuous-time random process C C (ω, t) := ω t t [1, )
9 Puddle ω = 0.62 ω = 0.91 ω = 0.12 C (ω, t) t
10 Puddle We only care about how much water there is on day i Continuous-state discrete-time random process D D (ω, i) := ω, i = 1, 2,... i
11 Puddle 0.8 ω = 0.31 ω = 0.89 ω = 0.52 D (ω, i) i
12 nth-order distribution Joint distribution of X (t 1 ), X (t 2 ),..., X (t n ) for any {t 1, t 2,..., t n } If X (t 1 ), X (t 2 ),..., X (t n ) have the same joint distribution as X (t 1 + τ), X (t 2 + τ),..., X (t n + τ) for any τ the process is strictly/strongly stationary
13 Puddle F C(t) (x)
14 Puddle ( ) F C(t) (x) := P C (t) x
15 Puddle ( ) F C(t) (x) := P C (t) x = P (ω t x)
16 Puddle ( ) F C(t) (x) := P C (t) x = P (ω t x) t x u=0 du = t x if 0 x 1 t = 1 if x > 1 t 0 if x < 0
17 Puddle ( ) F C(t) (x) := P C (t) x = P (ω t x) t x u=0 du = t x if 0 x 1 t = 1 if x > 1 t 0 if x < 0 { t if 0 x 1 f C(t) (x) = t 0 otherwise
18 How to specify a random process Three options:
19 How to specify a random process Three options: 1. Define probability space and a function from Ω to a set of functions
20 How to specify a random process Three options: 1. Define probability space and a function from Ω to a set of functions 2. Define all nth-order distributions for all n 0
21 How to specify a random process Three options: 1. Define probability space and a function from Ω to a set of functions 2. Define all nth-order distributions for all n 0 3. Express it as a function of another random process
22 Definition Mean and autocovariance functions Important random processes
23 Mean The mean of a random process is the function ( ) µ X (t) := E X (t) It is a deterministic function of t
24 Autocovariance The autocovariance of a random process is the function ( R X (t 1, t 2 ) := Cov X (t1 ), X ) (t 2 ) In particular, ( ) R X (t, t) := Var X (t)
25 Wide-sense/weakly stationary process A process is stationary in a wide or weak sense if its mean is constant µ X (t) := µ and its autocovariance function is shift invariant, i.e. R X (t 1, t 2 ) := R X (t 1 + τ, t 2 + τ) for any t 1 and t 2 and any shift τ Common notation R X (s) := R X (t, t + s)
26 Autocovariance function R(s) s
27 Realization i
28 Realization i
29 Realization i
30 Autocovariance function R(s) s
31 Realization i
32 Realization i
33 Realization i
34 Autocovariance function R(s) s
35 Realization i
36 Realization i
37 Realization i
38 Definition Mean and autocovariance functions Important random processes
39 Independent identically-distributed sequences A discrete random process is iid if X (i) has the same distribution for any fixed i X (i1 ), X (i 2 ),..., X (i n ) are mutually independent for any i 1,..., i n and any n 2
40 Independent identically-distributed sequences Valid definition If the state is discrete p X (i1 ), X (i 2 ),..., X (i (x n) i 1, x i2,..., x in ) = If the state is continuous f X (i1 ), X (i 2 ),..., X (i (x n) i 1, x i2,..., x in ) = The process is strictly stationary n p X (x i ) i=1 n f X (x i ) i=1
41 Uniform distribution in [0, 1] i
42 Uniform distribution in [0, 1] i
43 Uniform distribution in [0, 1] i
44 Geometric distribution (p = 0.4) i
45 Geometric distribution (p = 0.4) i
46 Geometric distribution (p = 0.4) i
47 Independent identically-distributed sequences If the distribution at each time has mean µ and variance σ 2 µ X (i) R X (i, j)
48 Independent identically-distributed sequences If the distribution at each time has mean µ and variance σ 2 ( ) µ X (i) := E X (i) R X (i, j)
49 Independent identically-distributed sequences If the distribution at each time has mean µ and variance σ 2 ( ) µ X (i) := E X (i) = µ R X (i, j)
50 Independent identically-distributed sequences If the distribution at each time has mean µ and variance σ 2 ( ) µ X (i) := E X (i) = µ ( ) R X (i, j) := E X (i) X (j) = { σ 2 if i = j 0 if i j ( ) ( ) E X (i) E X (j)
51 Gaussian random process Any set of samples is a Gaussian random vector Fully characterized by mean function µ X and autocovariance function R X X (t 1 ) X (t X := 2 ) X (t n ) is a Gaussian random vector with mean and covariance µ X (t 1 ) R X (t 1, t 1 ) R X (t 1, t 2 ) R X (t 1, t n ) µ X := µ X (t 2 ) R Σ X := X (t 1, t 2 ) R X (t 2, t 2 ) R X (t 2, t n ) µ X (t n ) R X (t 2, t n ) R X (t 2, t n ) R X (t n, t n )
52 Generating a Gaussian random process Boils down to sampling a Gaussian random vector with the appropriate mean and covariance matrix 1. Compute the mean vector µ X and the covariance matrix Σ X 2. Generate n independent samples from a standard Gaussian 3. Color the samples according to Σ X and center them around µ X
53 Poisson process Sequential events occurring on [0, ) 1. Each event occurs independently from every other event 2. Events occur uniformly 3. Events occur at a rate of λ events per time interval Ñ (t) is the number of events between 0 and t
54 Poisson process For any t 1 < t 2 < t 3 < t 4 1. Ñ (t 2 ) Ñ (t 1) is Poisson with parameter λ (t 2 t 1 ) 2. Ñ (t 2 ) Ñ (t 1) and Ñ (t 4) Ñ (t 3) are independent A random process satisfying these two conditions is a Poisson process
55 Poisson process nth order distribution can be expressed in terms of p ( λ, x ) := λ x e λ x!
56 Poisson process pñ(t1 ),...,Ñ(tn) (x1,..., xn)
57 Poisson process pñ(t1 ),...,Ñ(tn) (x1,..., xn) ) = P (Ñ (t1) = x 1,..., Ñ (tn) = xn
58 Poisson process pñ(t1 ),...,Ñ(tn) (x1,..., xn) ) = P (Ñ (t1) = x 1,..., Ñ (tn) = xn ) = P (Ñ (t1) = x 1, Ñ (t2) Ñ (t1) = x2 x1,..., Ñ (tn) Ñ (tn 1) = xn xn 1
59 Poisson process pñ(t1 ),...,Ñ(tn) (x1,..., xn) ) = P (Ñ (t1) = x 1,..., Ñ (tn) = xn ) = P (Ñ (t1) = x 1, Ñ (t2) Ñ (t1) = x2 x1,..., Ñ (tn) Ñ (tn 1) = xn xn 1 ) ) ) = P (Ñ (t1) = x 1 P (Ñ (t2) Ñ (t1) = x2 x1... P (Ñ (tn) Ñ (tn 1) = xn xn 1
60 Poisson process pñ(t1 ),...,Ñ(tn) (x1,..., xn) ) = P (Ñ (t1) = x 1,..., Ñ (tn) = xn ) = P (Ñ (t1) = x 1, Ñ (t2) Ñ (t1) = x2 x1,..., Ñ (tn) Ñ (tn 1) = xn xn 1 ) ) ) = P (Ñ (t1) = x 1 P (Ñ (t2) Ñ (t1) = x2 x1... P (Ñ (tn) Ñ (tn 1) = xn xn 1 = p (λt 1, x 1) p (λ (t 2 t 1), x 2 x 1)... p (λ (t n t n 1), x n x n 1)
61 Poisson process (λ = 0.2) t
62 Poisson process (λ = 0.2) t
63 Poisson process (λ = 0.2) t
64 Poisson process (λ = 1) t
65 Poisson process (λ = 1) t
66 Poisson process (λ = 1) t
67 Poisson process (λ = 2) t
68 Poisson process (λ = 2) t
69 Poisson process (λ = 2) t
70 Call-center data Example: Data from a call center in Israel We compare the histogram of the number of calls received in an interval of 4 hours over 2 months and the pmf of a Poisson random variable fitted to the data
71 Call-center data Real data Poisson distribution Number of calls
72 Poisson process Distribution of interarrival times? F T (t)
73 Poisson process Distribution of interarrival times? F T (t) := P (T t)
74 Poisson process Distribution of interarrival times? F T (t) := P (T t) = 1 P (T > t)
75 Poisson process Distribution of interarrival times? F T (t) := P (T t) = 1 P (T > t) = 1 P (no events in an interval of length t)
76 Poisson process Distribution of interarrival times? F T (t) := P (T t) = 1 P (T > t) = 1 P (no events in an interval of length t) = 1 e λ t
77 Poisson process Distribution of interarrival times? F T (t) := P (T t) = 1 P (T > t) = 1 P (no events in an interval of length t) = 1 e λ t f T (t) = λe λ t Iid exponential sequence (allows to simulate Poisson process!)
78 Call-center data Example: Data from a call center in Israel We compare the histogram of the inter-arrival times between calls occurring between 8 pm and midnight over two days and the pdf of an exponential random variable fitted to the data
79 Call center Exponential distribution Real data
80 Generating a Poisson process To sample from a Poisson random process with parameter λ we: 1. Generate independent samples from an exponential random variable with parameter λ t 1, t 2, t 3, Set the events of the Poisson process to occur at t 1, t 2, t 3,...
81 Mean and autocovariance ( ) E X (t)
82 Mean and autocovariance ( ) E X (t) = λ t
83 Mean and autocovariance ( ) E X (t) = λ t R X (t 1, t 2 ) = λ min {t 1, t 2 }
84 Mean and autocovariance ( ) E X (t) = λ t R X (t 1, t 2 ) = λ min {t 1, t 2 } The process is neither strictly nor wide-sense stationary
85 Earthquakes Earthquakes in San Francisco follow a Poisson process with parameter 0.3 earthquakes/year Probability of no earthquakes in the next 10 years and then at least 1 over the following 20 years?
86 Earthquakes ( ) P X (10) = 0, X (30) 1
87 Earthquakes ( ) ( ) P X (10) = 0, X (30) 1 = P X (10) = 0, X (30) X (10) 1
88 Earthquakes ( ) P X (10) = 0, X (30) 1 ( ) = P X (10) = 0, X (30) X (10) 1 = P ( ) ( ) X (10) = 0 P X (30) X (10) 1
89 Earthquakes ( ) P X (10) = 0, X (30) 1 ( ) = P X (10) = 0, X (30) X (10) 1 ( ) X (10) = 0 = P = P ( X (10) = 0 ) ( 1 P ( ) P X (30) X (10) 1 ( X (30) X (10) = 0 ))
90 Earthquakes ( ) P X (10) = 0, X (30) 1 ( ) = P X (10) = 0, X (30) X (10) 1 ( ) X (10) = 0 = P = P ( X (10) = 0 ) ( 1 P = e 3 ( 1 e 6) = ( ) P X (30) X (10) 1 ( X (30) X (10) = 0 ))
91 Random walk Process that evolves by taking steps in random directions Step sequence Z is iid S (i) = { +1 with probability with probability 1 2 We define a random walk X as { 0 for i = 0 X (i) := i S j=1 (j) for i = 1, 2,...
92 Random walk i
93 Random walk i
94 Random walk i
95 First-order pmf p X (i) (x)? Distribution of number of positive steps S +? Negative steps: S = i S + p X (i) (x)
96 First-order pmf p X (i) (x)? Distribution of number of positive steps S +? Negative steps: S = i S + i p X (i) (x) = P S (i) = x j=0
97 First-order pmf p X (i) (x)? Distribution of number of positive steps S +? Negative steps: S = i S + i p X (i) (x) = P S (i) = x j=0 = P (S + S = x)
98 First-order pmf p X (i) (x)? Distribution of number of positive steps S +? Negative steps: S = i S + i p X (i) (x) = P S (i) = x j=0 = P (S + S = x) = P (2 S + i = x)
99 First-order pmf p X (i) (x)? Distribution of number of positive steps S +? Negative steps: S = i S + i p X (i) (x) = P S (i) = x j=0 = P (S + S = x) = P (2 S + i = x) ( = P S + = i + x ) 2
100 First-order pmf p X (i) (x)? Distribution of number of positive steps S +? Negative steps: S = i S + i p X (i) (x) = P S (i) = x j=0 = P (S + S = x) = P (2 S + i = x) ( = P S + = i + x ) 2 ( ) i 1 = i+x 2 2 i if i + x is an integer between 0 and i 2
101 Mean and autocovariance µ X (i)
102 Mean and autocovariance ( ) µ X (i) := E X (i)
103 Mean and autocovariance ( ) µ X (i) := E X (i) i = E S (j) j=1
104 Mean and autocovariance ( ) µ X (i) := E X (i) i = E S (j) = i j=1 j=1 ( ) E S (j) by linearity of expectation
105 Mean and autocovariance ( ) µ X (i) := E X (i) i = E S (j) = = 0 i j=1 j=1 ( ) E S (j) by linearity of expectation
106 Mean and autocovariance ( ) µ X (i) := E X (i) i = E S (j) = = 0 i j=1 j=1 ( ) E S (j) by linearity of expectation R X (i, j) = min {i, j}
107 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain?
108 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain? 0
109 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain? 0 Probability that the gambler is up 6 dollars or more after first 10 flips?
110 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain? 0 Probability that the gambler is up 6 dollars or more after first 10 flips? P (gambler is up $6 or more)
111 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain? 0 Probability that the gambler is up 6 dollars or more after first 10 flips? P (gambler is up $6 or more) = p X (10) (6) + p X (10) (8) + p X (10) (10)
112 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain? 0 Probability that the gambler is up 6 dollars or more after first 10 flips? P (gambler is up $6 or more) = p X (10) (6) + p X (10) (8) + p X (10) (10) ( ) ( ) =
113 Gambler A fair coin is flipped sequentially Heads: +$1 Tails: -$1 Expected gain? 0 Probability that the gambler is up 6 dollars or more after first 10 flips? P (gambler is up $6 or more) = p X (10) (6) + p X (10) (8) + p X (10) (10) ( ) ( ) = =
Random variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationConvergence of Random Processes
Convergence of Random Processes DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Define convergence for random
More informationDS-GA 1002 Lecture notes 2 Fall Random variables
DS-GA 12 Lecture notes 2 Fall 216 1 Introduction Random variables Random variables are a fundamental tool in probabilistic modeling. They allow us to model numerical quantities that are uncertain: the
More informationEcon 424 Time Series Concepts
Econ 424 Time Series Concepts Eric Zivot January 20 2015 Time Series Processes Stochastic (Random) Process { 1 2 +1 } = { } = sequence of random variables indexed by time Observed time series of length
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationProbability and Statistics
Probability and Statistics 1 Contents some stochastic processes Stationary Stochastic Processes 2 4. Some Stochastic Processes 4.1 Bernoulli process 4.2 Binomial process 4.3 Sine wave process 4.4 Random-telegraph
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationEE/CpE 345. Modeling and Simulation. Fall Class 5 September 30, 2002
EE/CpE 345 Modeling and Simulation Class 5 September 30, 2002 Statistical Models in Simulation Real World phenomena of interest Sample phenomena select distribution Probabilistic, not deterministic Model
More informationThe exponential distribution and the Poisson process
The exponential distribution and the Poisson process 1-1 Exponential Distribution: Basic Facts PDF f(t) = { λe λt, t 0 0, t < 0 CDF Pr{T t) = 0 t λe λu du = 1 e λt (t 0) Mean E[T] = 1 λ Variance Var[T]
More informationProbability and Statistics for Data Science. Carlos Fernandez-Granda
Probability and Statistics for Data Science Carlos Fernandez-Granda Preface These notes were developed for the course Probability and Statistics for Data Science at the Center for Data Science in NYU.
More informationExpectation. DS GA 1002 Probability and Statistics for Data Science. Carlos Fernandez-Granda
Expectation DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean,
More informationFundamentals of Applied Probability and Random Processes
Fundamentals of Applied Probability and Random Processes,nd 2 na Edition Oliver C. Ibe University of Massachusetts, LoweLL, Massachusetts ip^ W >!^ AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS
More informationELEMENTS OF PROBABILITY THEORY
ELEMENTS OF PROBABILITY THEORY Elements of Probability Theory A collection of subsets of a set Ω is called a σ algebra if it contains Ω and is closed under the operations of taking complements and countable
More informationExpectation. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Expectation DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Aim Describe random variables with a few numbers: mean, variance,
More informationStationary independent increments. 1. Random changes of the form X t+h X t fixed h > 0 are called increments of the process.
Stationary independent increments 1. Random changes of the form X t+h X t fixed h > 0 are called increments of the process. 2. If each set of increments, corresponding to non-overlapping collection of
More informationChapter 6: Random Processes 1
Chapter 6: Random Processes 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationRandom Processes Why we Care
Random Processes Why we Care I Random processes describe signals that change randomly over time. I Compare: deterministic signals can be described by a mathematical expression that describes the signal
More information. Find E(V ) and var(v ).
Math 6382/6383: Probability Models and Mathematical Statistics Sample Preliminary Exam Questions 1. A person tosses a fair coin until she obtains 2 heads in a row. She then tosses a fair die the same number
More informationExercises with solutions (Set D)
Exercises with solutions Set D. A fair die is rolled at the same time as a fair coin is tossed. Let A be the number on the upper surface of the die and let B describe the outcome of the coin toss, where
More informationChapter 2 Random Processes
Chapter 2 Random Processes 21 Introduction We saw in Section 111 on page 10 that many systems are best studied using the concept of random variables where the outcome of a random experiment was associated
More informationLecture Notes 2 Random Variables. Discrete Random Variables: Probability mass function (pmf)
Lecture Notes 2 Random Variables Definition Discrete Random Variables: Probability mass function (pmf) Continuous Random Variables: Probability density function (pdf) Mean and Variance Cumulative Distribution
More informationLecture Notes 2 Random Variables. Random Variable
Lecture Notes 2 Random Variables Definition Discrete Random Variables: Probability mass function (pmf) Continuous Random Variables: Probability density function (pdf) Mean and Variance Cumulative Distribution
More informationExponential Distribution and Poisson Process
Exponential Distribution and Poisson Process Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 215 Outline Introduction Exponential
More informationPart I Stochastic variables and Markov chains
Part I Stochastic variables and Markov chains Random variables describe the behaviour of a phenomenon independent of any specific sample space Distribution function (cdf, cumulative distribution function)
More informationIEOR 3106: Introduction to Operations Research: Stochastic Models. Professor Whitt. SOLUTIONS to Homework Assignment 2
IEOR 316: Introduction to Operations Research: Stochastic Models Professor Whitt SOLUTIONS to Homework Assignment 2 More Probability Review: In the Ross textbook, Introduction to Probability Models, read
More informationMultivariate random variables
Multivariate random variables DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Joint distributions Tool to characterize several
More informationSimulation. Where real stuff starts
1 Simulation Where real stuff starts ToC 1. What is a simulation? 2. Accuracy of output 3. Random Number Generators 4. How to sample 5. Monte Carlo 6. Bootstrap 2 1. What is a simulation? 3 What is a simulation?
More informationProbability Density Functions and the Normal Distribution. Quantitative Understanding in Biology, 1.2
Probability Density Functions and the Normal Distribution Quantitative Understanding in Biology, 1.2 1. Discrete Probability Distributions 1.1. The Binomial Distribution Question: You ve decided to flip
More informationChapter 1 Statistical Reasoning Why statistics? Section 1.1 Basics of Probability Theory
Chapter 1 Statistical Reasoning Why statistics? Uncertainty of nature (weather, earth movement, etc. ) Uncertainty in observation/sampling/measurement Variability of human operation/error imperfection
More informationProf. Dr. Roland Füss Lecture Series in Applied Econometrics Summer Term Introduction to Time Series Analysis
Introduction to Time Series Analysis 1 Contents: I. Basics of Time Series Analysis... 4 I.1 Stationarity... 5 I.2 Autocorrelation Function... 9 I.3 Partial Autocorrelation Function (PACF)... 14 I.4 Transformation
More informationOn 1.9, you will need to use the facts that, for any x and y, sin(x+y) = sin(x) cos(y) + cos(x) sin(y). cos(x+y) = cos(x) cos(y) - sin(x) sin(y).
On 1.9, you will need to use the facts that, for any x and y, sin(x+y) = sin(x) cos(y) + cos(x) sin(y). cos(x+y) = cos(x) cos(y) - sin(x) sin(y). (sin(x)) 2 + (cos(x)) 2 = 1. 28 1 Characteristics of Time
More informationENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University
ENSC327 Communications Systems 19: Random Processes Jie Liang School of Engineering Science Simon Fraser University 1 Outline Random processes Stationary random processes Autocorrelation of random processes
More informationNotes on Continuous Random Variables
Notes on Continuous Random Variables Continuous random variables are random quantities that are measured on a continuous scale. They can usually take on any value over some interval, which distinguishes
More informationEleventh Problem Assignment
EECS April, 27 PROBLEM (2 points) The outcomes of successive flips of a particular coin are dependent and are found to be described fully by the conditional probabilities P(H n+ H n ) = P(T n+ T n ) =
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationNorthwestern University Department of Electrical Engineering and Computer Science
Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability
More informationModule 9: Stationary Processes
Module 9: Stationary Processes Lecture 1 Stationary Processes 1 Introduction A stationary process is a stochastic process whose joint probability distribution does not change when shifted in time or space.
More informationLinear regression. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Linear regression DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Linear models Least-squares estimation Overfitting Example:
More informationMATH Notebook 5 Fall 2018/2019
MATH442601 2 Notebook 5 Fall 2018/2019 prepared by Professor Jenny Baglivo c Copyright 2004-2019 by Jenny A. Baglivo. All Rights Reserved. 5 MATH442601 2 Notebook 5 3 5.1 Sequences of IID Random Variables.............................
More information2. Variance and Covariance: We will now derive some classic properties of variance and covariance. Assume real-valued random variables X and Y.
CS450 Final Review Problems Fall 08 Solutions or worked answers provided Problems -6 are based on the midterm review Identical problems are marked recap] Please consult previous recitations and textbook
More informationProbability Models. 4. What is the definition of the expectation of a discrete random variable?
1 Probability Models The list of questions below is provided in order to help you to prepare for the test and exam. It reflects only the theoretical part of the course. You should expect the questions
More informationMath489/889 Stochastic Processes and Advanced Mathematical Finance Solutions for Homework 7
Math489/889 Stochastic Processes and Advanced Mathematical Finance Solutions for Homework 7 Steve Dunbar Due Mon, November 2, 2009. Time to review all of the information we have about coin-tossing fortunes
More informationDescriptive Statistics
Descriptive Statistics DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Descriptive statistics Techniques to visualize
More informationLecture Notes 7 Random Processes. Markov Processes Markov Chains. Random Processes
Lecture Notes 7 Random Processes Definition IID Processes Bernoulli Process Binomial Counting Process Interarrival Time Process Markov Processes Markov Chains Classification of States Steady State Probabilities
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More informationECE534, Spring 2018: Solutions for Problem Set #4 Due Friday April 6, 2018
ECE534, Spring 2018: s for Problem Set #4 Due Friday April 6, 2018 1. MMSE Estimation, Data Processing and Innovations The random variables X, Y, Z on a common probability space (Ω, F, P ) are said to
More informationPhysics 403 Probability Distributions II: More Properties of PDFs and PMFs
Physics 403 Probability Distributions II: More Properties of PDFs and PMFs Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Last Time: Common Probability Distributions
More informationS n = x + X 1 + X X n.
0 Lecture 0 0. Gambler Ruin Problem Let X be a payoff if a coin toss game such that P(X = ) = P(X = ) = /2. Suppose you start with x dollars and play the game n times. Let X,X 2,...,X n be payoffs in each
More informationECE 313: Conflict Final Exam Tuesday, May 13, 2014, 7:00 p.m. 10:00 p.m. Room 241 Everitt Lab
University of Illinois Spring 1 ECE 313: Conflict Final Exam Tuesday, May 13, 1, 7: p.m. 1: p.m. Room 1 Everitt Lab 1. [18 points] Consider an experiment in which a fair coin is repeatedly tossed every
More informationMultiple Random Variables
Multiple Random Variables Joint Probability Density Let X and Y be two random variables. Their joint distribution function is F ( XY x, y) P X x Y y. F XY ( ) 1, < x
More informationHomework 2. Spring 2019 (Due Thursday February 7)
ECE 302: Probabilistic Methods in Electrical and Computer Engineering Spring 2019 Instructor: Prof. A. R. Reibman Homework 2 Spring 2019 (Due Thursday February 7) Homework is due on Thursday February 7
More informationSTAT 520: Forecasting and Time Series. David B. Hitchcock University of South Carolina Department of Statistics
David B. University of South Carolina Department of Statistics What are Time Series Data? Time series data are collected sequentially over time. Some common examples include: 1. Meteorological data (temperatures,
More informationSTAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
More informationRandom Models. Tusheng Zhang. February 14, 2013
Random Models Tusheng Zhang February 14, 013 1 Introduction In this module, we will introduce some random models which have many real life applications. The course consists of four parts. 1. A brief review
More informationAppendix A : Introduction to Probability and stochastic processes
A-1 Mathematical methods in communication July 5th, 2009 Appendix A : Introduction to Probability and stochastic processes Lecturer: Haim Permuter Scribe: Shai Shapira and Uri Livnat The probability of
More informationSample Spaces, Random Variables
Sample Spaces, Random Variables Moulinath Banerjee University of Michigan August 3, 22 Probabilities In talking about probabilities, the fundamental object is Ω, the sample space. (elements) in Ω are denoted
More informationfor valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I
Code: 15A04304 R15 B.Tech II Year I Semester (R15) Regular Examinations November/December 016 PROBABILITY THEY & STOCHASTIC PROCESSES (Electronics and Communication Engineering) Time: 3 hours Max. Marks:
More informationE X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.
E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,
More informationSlides 8: Statistical Models in Simulation
Slides 8: Statistical Models in Simulation Purpose and Overview The world the model-builder sees is probabilistic rather than deterministic: Some statistical model might well describe the variations. An
More informationChapter 5. Statistical Models in Simulations 5.1. Prof. Dr. Mesut Güneş Ch. 5 Statistical Models in Simulations
Chapter 5 Statistical Models in Simulations 5.1 Contents Basic Probability Theory Concepts Discrete Distributions Continuous Distributions Poisson Process Empirical Distributions Useful Statistical Models
More information6.041/6.431 Fall 2010 Quiz 2 Solutions
6.04/6.43: Probabilistic Systems Analysis (Fall 200) 6.04/6.43 Fall 200 Quiz 2 Solutions Problem. (80 points) In this problem: (i) X is a (continuous) uniform random variable on [0, 4]. (ii) Y is an exponential
More informationChapter 2: Random Variables
ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:
More informationLecture 4: Sampling, Tail Inequalities
Lecture 4: Sampling, Tail Inequalities Variance and Covariance Moment and Deviation Concentration and Tail Inequalities Sampling and Estimation c Hung Q. Ngo (SUNY at Buffalo) CSE 694 A Fun Course 1 /
More informationChapter 5. Means and Variances
1 Chapter 5 Means and Variances Our discussion of probability has taken us from a simple classical view of counting successes relative to total outcomes and has brought us to the idea of a probability
More informationECE353: Probability and Random Processes. Lecture 18 - Stochastic Processes
ECE353: Probability and Random Processes Lecture 18 - Stochastic Processes Xiao Fu School of Electrical Engineering and Computer Science Oregon State University E-mail: xiao.fu@oregonstate.edu From RV
More informationECE 302, Final 3:20-5:20pm Mon. May 1, WTHR 160 or WTHR 172.
ECE 302, Final 3:20-5:20pm Mon. May 1, WTHR 160 or WTHR 172. 1. Enter your name, student ID number, e-mail address, and signature in the space provided on this page, NOW! 2. This is a closed book exam.
More informationMAT 271E Probability and Statistics
MAT 271E Probability and Statistics Spring 2011 Instructor : Class Meets : Office Hours : Textbook : Supp. Text : İlker Bayram EEB 1103 ibayram@itu.edu.tr 13.30 16.30, Wednesday EEB? 10.00 12.00, Wednesday
More informationLearning Objectives for Stat 225
Learning Objectives for Stat 225 08/20/12 Introduction to Probability: Get some general ideas about probability, and learn how to use sample space to compute the probability of a specific event. Set Theory:
More informationGaussian, Markov and stationary processes
Gaussian, Markov and stationary processes Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November
More informationStatistics and Econometrics I
Statistics and Econometrics I Random Variables Shiu-Sheng Chen Department of Economics National Taiwan University October 5, 2016 Shiu-Sheng Chen (NTU Econ) Statistics and Econometrics I October 5, 2016
More informationThings to remember when learning probability distributions:
SPECIAL DISTRIBUTIONS Some distributions are special because they are useful They include: Poisson, exponential, Normal (Gaussian), Gamma, geometric, negative binomial, Binomial and hypergeometric distributions
More informationFundamental Tools - Probability Theory II
Fundamental Tools - Probability Theory II MSc Financial Mathematics The University of Warwick September 29, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory II 1 / 22 Measurable random
More informationMarkov Chains. Chapter 16. Markov Chains - 1
Markov Chains Chapter 16 Markov Chains - 1 Why Study Markov Chains? Decision Analysis focuses on decision making in the face of uncertainty about one future event. However, many decisions need to consider
More informationName of the Student:
SUBJECT NAME : Probability & Queueing Theory SUBJECT CODE : MA 6453 MATERIAL NAME : Part A questions REGULATION : R2013 UPDATED ON : November 2017 (Upto N/D 2017 QP) (Scan the above QR code for the direct
More informationStochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno
Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.
More informationWe introduce methods that are useful in:
Instructor: Shengyu Zhang Content Derived Distributions Covariance and Correlation Conditional Expectation and Variance Revisited Transforms Sum of a Random Number of Independent Random Variables more
More informationProbability Models in Electrical and Computer Engineering Mathematical models as tools in analysis and design Deterministic models Probability models
Probability Models in Electrical and Computer Engineering Mathematical models as tools in analysis and design Deterministic models Probability models Statistical regularity Properties of relative frequency
More informationECE302 Spring 2006 Practice Final Exam Solution May 4, Name: Score: /100
ECE302 Spring 2006 Practice Final Exam Solution May 4, 2006 1 Name: Score: /100 You must show ALL of your work for full credit. This exam is open-book. Calculators may NOT be used. 1. As a function of
More information2. Suppose (X, Y ) is a pair of random variables uniformly distributed over the triangle with vertices (0, 0), (2, 0), (2, 1).
Name M362K Final Exam Instructions: Show all of your work. You do not have to simplify your answers. No calculators allowed. There is a table of formulae on the last page. 1. Suppose X 1,..., X 1 are independent
More informationELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 2 Discrete Random Variables Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Random Variable Discrete Random
More informationPh.D. Qualifying Exam Monday Tuesday, January 4 5, 2016
Ph.D. Qualifying Exam Monday Tuesday, January 4 5, 2016 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Find the maximum likelihood estimate of θ where θ is a parameter
More informationIntroduction to Probability and Statistics (Continued)
Introduction to Probability and Statistics (Continued) Prof. icholas Zabaras Center for Informatics and Computational Science https://cics.nd.edu/ University of otre Dame otre Dame, Indiana, USA Email:
More information1.1 Review of Probability Theory
1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More informationQueueing Theory and Simulation. Introduction
Queueing Theory and Simulation Based on the slides of Dr. Dharma P. Agrawal, University of Cincinnati and Dr. Hiroyuki Ohsaki Graduate School of Information Science & Technology, Osaka University, Japan
More informationReview of Probability. CS1538: Introduction to Simulations
Review of Probability CS1538: Introduction to Simulations Probability and Statistics in Simulation Why do we need probability and statistics in simulation? Needed to validate the simulation model Needed
More informationMath 151. Rumbos Fall Solutions to Review Problems for Exam 2. Pr(X = 1) = ) = Pr(X = 2) = Pr(X = 3) = p X. (k) =
Math 5. Rumbos Fall 07 Solutions to Review Problems for Exam. A bowl contains 5 chips of the same size and shape. Two chips are red and the other three are blue. Draw three chips from the bowl at random,
More informationCourse: ESO-209 Home Work: 1 Instructor: Debasis Kundu
Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear
More informationPreliminary statistics
1 Preliminary statistics The solution of a geophysical inverse problem can be obtained by a combination of information from observed data, the theoretical relation between data and earth parameters (models),
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationEECS 126 Probability and Random Processes University of California, Berkeley: Spring 2015 Abhay Parekh February 17, 2015.
EECS 126 Probability and Random Processes University of California, Berkeley: Spring 2015 Abhay Parekh February 17, 2015 Midterm Exam Last name First name SID Rules. You have 80 mins (5:10pm - 6:30pm)
More information18.05 Exam 1. Table of normal probabilities: The last page of the exam contains a table of standard normal cdf values.
Name 18.05 Exam 1 No books or calculators. You may have one 4 6 notecard with any information you like on it. 6 problems, 8 pages Use the back side of each page if you need more space. Simplifying expressions:
More informationChapter 4 Random process. 4.1 Random process
Random processes - Chapter 4 Random process 1 Random processes Chapter 4 Random process 4.1 Random process 4.1 Random process Random processes - Chapter 4 Random process 2 Random process Random process,
More informationContinuous-time Markov Chains
Continuous-time Markov Chains Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ October 23, 2017
More informationECE 302: Probabilistic Methods in Electrical Engineering
ECE 302: Probabilistic Methods in Electrical Engineering Test I : Chapters 1 3 3/22/04, 7:30 PM Print Name: Read every question carefully and solve each problem in a legible and ordered manner. Make sure
More information1. Fundamental concepts
. Fundamental concepts A time series is a sequence of data points, measured typically at successive times spaced at uniform intervals. Time series are used in such fields as statistics, signal processing
More informationEE126: Probability and Random Processes
EE126: Probability and Random Processes Lecture 18: Poisson Process Abhay Parekh UC Berkeley March 17, 2011 1 1 Review 2 Poisson Process 2 Bernoulli Process An arrival process comprised of a sequence of
More informationELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random
More informationBayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics
More information