Chapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University
|
|
- Beatrix McBride
- 6 years ago
- Views:
Transcription
1 Chapter 3, 4 Random Variables ENCS Probability and Stochastic Processes Concordia University
2 ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real number X(ω) to each outcome ω in the sample space of a random experiment. S is the domain, and S X = {X(ω) : ω S} is the range of r.v. X Example: toss a coin S = {H,T } X(H) = 0,X(T) = 1,S X = {0, 1} P(H) = P(X = 0) = 0.5,P(T) = P(X = 1) = 0.5 measure the temperature, S = {ω 10 < ω < 30} X(ω) = ω, then S X = {x 10 < x < 30} What is P(X = 25)?
3 ENCS6161 p.2/47 Cumulative Distribution Function F X (x) = P(X x) = P(ω : X(ω) x) < x < Properties of F X (x) 1. 0 F X (x) 1 2. lim x F X (x) = 1 3. lim x F X (x) = 0 4. F X (x) is nondecreasing, i.e., if a < b then F X (a) F X (b) 5. F X (x) is continuous from right, i.e., for h > 0 F X (b) = lim h 0 F X (b + h) = F X (b + ) (see examples below)
4 ENCS6161 p.3/47 Cumulative Distribution Function Properties of F X (x) 6. P(a < X b) = F X (b) F X (a) for a b Proof: {X a} {a < X b} = {X b} {X a} {a < X b} = so P {X a} + P {a < X b} = P {X b} P {a < X b} = F X (b) F X (a) 7. P {X = b} = F X (b) F X (b ) Proof: P {X = b} = lim P {b ε < X b} ε 0 = F X (b) lim F X (b ɛ) ε 0 = F X (b) F X (b )
5 ENCS6161 p.4/47 Cumulative Distribution Function Example: roll a dice S = {1, 2, 3, 4, 5, 6}, X(ω) = ω 0 x < x < x < 3 F X (x) = x < 6 1 x 6 check properties 5,6, F X (x) x
6 ENCS6161 p.5/47 Cumulative Distribution Function Example: pick a real number between 0 and 1 uniformly F X (x) F X (x) = 0 x < 0 x/1 0 x 1 1 x > x P {X = 0.5} =?
7 ENCS6161 p.6/47 Three Types of Random Variables 1. Discrete random variables S X = {x 0,x 1, } finite or countable Probability mass function (pmf ) P X (x k ) = P {X = x k } F X (x) = k P X(x k )u(x x k ) where u(x) = { 0 x < 0 1 x 0 see the example of rolling a dice F X (x) = 1 6 u(x 1) u(x 2) u(x 6)
8 ENCS6161 p.7/47 Three Types of Random Variables 2. Continuous random variables F x (x) is continuous everywhere P {X = x} = 0 for all x 3. Random variables of mixed type F X (x) = pf 1 (x) + (1 p)f 2 (x), where 0 < p < 1 F 1 (x) CDF of a discrete R.V F 2 (x) CDF of a continuous R.V Example: toss a coin if H generate a discrete r.v. if T generate a continuous r.v.
9 ENCS6161 p.8/47 Probability density function PDF, if it exists is defined as: f X (x) = df X(x) dx f X (x) F X(x+ x) F X (x) x = P {x<x x+ x} x Properties of pdf (assume continuous r.v.) 1. f X (x) 0 2. P {a X b} = b a f X(x) dx 3. F X (x) = x f X(t) dt 4. + f X(t) dt = 1 "density"
10 ENCS6161 p.9/47 Probability density function Example: uniform R.V f X (x) f X (x) = { 1 b a a x b 0 elsewhere 1 b a x a b F X (x) F X (x) = 0 x < a x a b a a x b 1 x > b 1 a b x
11 ENCS6161 p.10/47 Probability density function Example: if f X (x) = ce α x, < x < + 1. Find constant c + f X (x) dx = ce αx dx = 2c α = 1 c = α 2 2. Find P { X < v} P { X < v} = α 2 v v e α x dx = 2 α 2 v 0 e αx dx = 1 e αv
12 ENCS6161 p.11/47 PDF for a discrete r.v. For a discrete r.v. F X (x) = k P X(x k )u(x x k ) We define the delta function δ(x) s.t. u(x) = x du(x) δ(t) dt and δ(x) = dx So pdf for a discrete r.v. f X (x) = k P X(x k )δ(x x k ) Example: f X (x) x
13 ENCS6161 p.12/47 Conditional CDF and PDF Conditional CDF of X given event A F X (x A) = P {X x A} = P {X x A}, P(A) > 0 P(A) The conditional pdf f X (x A) = df X(x A) dx
14 ENCS6161 p.13/47 Conditional CDF and PDF Example: the life time X of a machine has CDF F X (x), find the conditional CDF & PDF given the event A = {X > t} F X (x X > t) = P {X x X > t} = = P {X x X > t} P {X > t} { 0 x t F X (x) F X (t) 1 F X (t) x > t f X (x X > t) = { 0 x t f X (x) 1 F X (t) X > t
15 ENCS6161 p.14/47 Important Random Variables Discrete r.v. 1. Bernoulli r.v. 2. Binomial r.v. 3. Geometric r.v. 4. Poisson r.v. Continuous r.v. 1. Uniform r.v. 2. Exponential r.v. 3. Gaussian (Normal) r.v.
16 ENCS6161 p.15/47 Bernoulli r.v. S X = {0, 1} P X (0) = 1 p P X (1) = p e.g toss a coin Let S be a a sample space and A S be an event with P(A) = p. The indicator { function of A 0 if ω A I A (ω) = 1 if ω A I A is a r.v. since it assigns a number to each outcome of S I A is a Bernoulli r.v. P IA (0) = P {ω A} = 1 P(A) = 1 p P IA (1) = P {ω A} = P(A) = p
17 ENCS6161 p.16/47 Binomial r.v. Repeat a random experiment n times independently and let X be the number of times that event A with P(A) = p occurs S X = {0, 1,,n} Let I j be the indicator function of the event A in jth trial X = I 1 + I 2 + I 3 + I n so X is a sum of Bernoulli r.v.s, where I 1,I 2,,I n are i.i.d r.v.s (independent identical distribution) X is called Binomial ( ) r.v. n P {X = k} = p k (1 p) n k, k = 0, 1,,n k
18 ENCS6161 p.17/47 Geometric r.v. Count the number X of independent Bernoulli trials until the first occurrence of a success. X is called the geometric r.v. S X = {1, 2, 3, } Let p = P(A) be the prob of "success" in each Bernoulli trial P {X = k} = (1 p) k 1 p, k = 1, 2, The geometric r.v. has the memoryless property: P {X k + j X > j} = P {X k} for all j,k 1 prove it by yourself
19 ENCS6161 p.18/47 Poisson r.v. Poisson r.v. has the pmf: P {N = k} = αk k! e α k = 0, 1, 2, A good model for the number of occurrences of an event in a certain time period. α is the avg number of event occurrences in the given time period. Example: number of phone calls in 1 hour, number of data packets arrived to a router in 10 mins. Poisson prob can be used to approximate a Binomial prob. when n is large and p is small. Let α = np ( n P k = )p k (1 p n k αk k k! e α k = 0, 1,,n
20 ENCS6161 p.19/47 Uniform r.v. Read on your own.
21 ENCS6161 p.20/47 Exponential r.v. f X (x) = { 0 x < 0 λe λx x 0 F X (x) = { 0 x < 0 1 e λx x 0 Good model for the lifetime of a device. Memoryless: for t,h > 0 P {X > t + h X > t} = = P {X > t + h X > t} P {X > t} P {X > t + h} P {X > t} = e λ(t+h) e λt = P {X > h} = e λh
22 ENCS6161 p.21/47 Gaussian (Normal) r.v. f X (x) = 1 2πσ e (x m)2 2σ 2 for < x < m, σ are two parameter (will be discussed later) CDF: F X (x) = P {X x} 1 x = 2πσ = x m 1 σ 2π m) 2 e (x 2σ 2 dx e t2 2 dt If we define Φ(x) = 1 x 2π e t2 2 dt Then ( ) x m F X = Φ σ x m (t = ) σ
23 ENCS6161 p.22/47 Gaussian (Normal) r.v. We usually use Q(x) = 1 Φ(x) = 1 2π x e t2 2 dt. f X (x) Φ(z) Q(z) z x Φ( x) = Q(x) = 1 Φ(x) Q( x) = Φ(x) = 1 Q(x)
24 ENCS6161 p.23/47 Functions of a random variable If X is a r.v., Y = g(x) will also be a r.v. The CDF of Y F Y (y) = P {Y y} = P {g(x) y} When x = g 1 (y) exists and is unique, the PDF of Y f Y (y) = f X(x) dy/dx = f X (x) dx x=g 1 (y) dy Note: f X (x) x f Y (y) y f Y (y) f X (x) x y f X(x) dx dy In general, if y = g(x) has n solutions x 1,x 2,,x n n f X (x) n f Y (y) = dy/dx = f X (x) dx x=xk dy k=1 k=1 x=xk
25 ENCS6161 p.24/47 Functions of a random variable Example: Find the pdf of Y = ax + b, in terms of pdf of X. Assume a 0 and X is continuous. F Y (y) = P {Y y} = P {ax + b y} { P {X y b = a } = F X( y b a ) if a > 0 P {X y b a } = 1 F X( y b a { ) if a < 0 f Y (y) = df Y (y) 1a f X ( y b = a ) if a > 0 dy a 1f X( y b a ) if a < 0 = 1 ( ) y b a f X a Example: Let X N(m,σ 2 ) i.e. f X (x) = 1 2πσ e (x m)2 2σ 2 if Y = ax + b (a 0) then 1 f Y (y) = e (y b am) 2(aσ) 2 2πσ a 2
26 ENCS6161 p.25/47 Functions of a random variable Example: Y = X 2 where X is a continuous r.v. F Y (y) = P {Y y} = P {X 2 0 y < 0 y} = P { y X y} y 0 0 y < 0 = F X ( y) F X ( y) y 0 So f Y (y) = If X N(0, 1) then 0 y < 0; f X ( y) 2 y + f X( y) 2 y y 0 f X (x) = 1 2π e x2 2 fy (y) = e y 2 2πy for y 0 ch-square r.v. with one degree of freedom
27 ENCS6161 p.26/47 Expected Value E[X] = xf X(x)dx (may not exist) for continuous r.v. E[X] = k x kp X (x k ) for discrete r.v. Example: uniform r.v. E[X] = b a x 1 b a dx = b + a 2 When f X (x) is symmetric about x = m then E[X] = m. (e.g., Gaussian r.v.) m x: odd symmetric about x = m (m x)f X (x): odd symmetric x = m (m x)f X(x)dx = 0 m = m f X(x)dx = xf X(x)dx = E[X]
28 ENCS6161 p.27/47 Expected Value Example: the arrival time of packets to a queue has exponential pdf f X (x) = λe λx for x 0 E[X] = 0 = 1 λ e λx xλe λx dx = xe λx 0 = 1 λ e λx dx
29 ENCS6161 p.28/47 Expected Value For a function Y = g(x) of r.v. X E[Y ] = g(x)f X (x)dx Example: Y = a cos(ωt + φ) where φ is uniformly distributed in [0, 2π]. Find E[Y ] and E[Y 2 ]. E[Y ] = 2π 0 1 2π a cos(ωt+φ)dφ = a 2π sin(ωt + φ) 2π 0 = 0 E[Y 2 ] = E[a 2 cos 2 (ωt+φ)] = E [ a a2 2 ] cos(2ωt + 2φ) = a2 2
30 ENCS6161 p.29/47 Variance of a random variable Var[X] = E[(X E[X]) 2 ] = E[X 2 ] E[X] 2 Example: uniform r.v. Var(X) = Let y = x a+b 2 b a 1 b a ( x a + b ) 2 dx 2 Var(X) = 1 b a b a 2 b a 2 y 2 dy = (b a)2 12
31 ENCS6161 p.30/47 Variance of a random variable Example: For a Gaussian r.v. X N(m,σ 2 ) Var(X) = = = (x m) 2 f X (x)dx (x m) 2 1 e (x m)2 2σ 2 dx 2πσ (x m) σ 2π d(e (x m)2 2σ 2 ) = (x m) σ e (x m)2 2σ 2 2π = σ πσ e (x m)2 2σ 2 dx = σ 2 σ e (x m)2 2σ 2 2π dx
32 ENCS6161 p.31/47 Variance of a random variable Some properties of variance Var[C] = 0 Var[X + C] = Var[X] Var[CX] = C 2 Var[X] nth moment of a random variable E[X n ] = x n f X (x)dx
33 ENCS6161 p.32/47 Markov Inequality CDF,PDF u,σ 2, how to u,σ 2? CDF PDF The Markov inequality: For non-negative r.v. X P[X a] E[X] for a > 0 a Proof: E[X] = 0 a xf X (x)dx = xf X (x)dx a 0 xf X (x)dx + P[X a] E[X] a a a xf X (x)dx af X (x)dx = ap[x a]
34 ENCS6161 p.33/47 Chebyshev Inequality Chebyshev inequality Proof: P { x m a} σ2 a 2 P { x m a} = P {(x m) 2 a 2 } E[(x m)2 ] a 2 = σ2 a 2 Markov inequality
35 ENCS6161 p.34/47 Transform Methods The characteristic function Φ X (ω) = E[e jωx ] = f X (x) e jωx dx j = 1 Fourier transform of f X (x) (with a reversal in the sign of exponent) f X (x) = 1 Φ X (ω) e jωx dx 2π Example: Exponential r.v., f X (x) = λ e λx,x 0 Φ X (ω) = 0 λ e λx e jωx dx = 0 λ e (λ jω)x dx = λ λ jω
36 ENCS6161 p.35/47 Characteristic Function If X is a discrete r.v., Φ X (ω) = k P X (x k ) e jωx k If x k are integer-valued, Φ X (ω) = k= P X (k) e jωk P X (k) = 1 2π 2π 0 Φ X (ω) e jωk dω
37 ENCS6161 p.36/47 Characteristic Function Example: For a geometric r.v., P X (k) = p(1 p) k, k = 0, 1, Φ X (ω) = = p = p(1 p) k e jωk k=0 [(1 p)e jω ] k k=0 p 1 (1 p)e jω
38 ENCS6161 p.37/47 Characteristic Function Moment Throrem: Proof: Φ X (ω) = so, E[X n ] = 1 d n j n dω Φ X(ω) n f X (x) e jωx dx = = 1 + jωe[x] + (jω)2 2! E[X 2 ] + ω=0 f X (x)(1 + jωx + (jωx)2 2! Φ X (ω) ω=0 = 1 (note: Φ X (0) = E[e 0 ] = E[1] = 1) d dω Φ X(ω) ω=0 = je[x] = E[X] = 1 d j dω Φ X(ω) ω=0. d n dω n Φ X(ω) ω=0 = j n E[X n ] = E[X n ] = 1 j n d n dω n Φ X(ω) ω=0 + )dx
39 ENCS6161 p.38/47 Characteristic Function Example: For an exponential r.v., so, Φ X (ω) = λ λ jω d dω Φ X(ω) = λj (λ jω) 2 = E[X] = 1 j d dω Φ X(ω) ω=0 = 1 λ Find E[X 2 ] and σ 2 by yourself.
40 ENCS6161 p.39/47 Probability Generating Function For non-negative, integer-valued r.v. N, G N (Z) = E[Z N ] = P N (k)z k k=0 which is the Z-transform of the pmf. P N (k) = 1 d k k! dz k G N(Z) Z=0 To find the first two moment of N : d dz G N(Z) Z=1 = d 2 dz 2 G N(Z) Z=1 = N (k)kz k=0p k 1 Z=1 = k k=0p N (k)k(k 1)Z k 2 Z=1 = k = E[N 2 ] E[N] kp N (k) = E[N] k(k 1)P N (k)
41 ENCS6161 p.40/47 Probability Generating Function Example: For a Poisson r.v., P N (k) = αk k! e α G N (Z) = k=0 α k k! e α Z k = e α k=0 (αz) k k! = e α e αz = e α(z 1) G N (Z) = αeα(z 1) G N (Z) = α2 e α(z 1) = E[N] = G N (1) = α E[N2 ] E[N] = G N (1) = α2 E[N 2 ] = α 2 + E[N] = α 2 + α V ar[n] = E[N 2 ] E[N] 2 = α 2 + α α 2 = α
42 ENCS6161 p.41/47 Entropy Let X be a r.v. with S X = {1, 2,,k} Uncertainty or information of X = k I(X = k) = log 1 P X (k) = log P X(k) Properties: 1. P X (k) small, I(X = k) large, more information 2. Additivity, if X, Y independent, P(X = k,y = m) = P(X = k)p(y = m) I(X = k,y = m) = log P(X = k,y = m) = log P(X = k) log P(Y = m) = I(X = k) + I(Y = m)
43 ENCS6161 p.42/47 Entropy Example: toss coins I(X = H) = log = 1 (1 bit of info) I(X 1 = H,X 2 = T) = log = 2 (bits) If the base of log is 2, we call the unit "bits" If the base of log is e, we call the unit "nats"
44 ENCS6161 p.43/47 Entropy Entropy: H(X) = E [ log ] 1 P X (k) = k P X (k) log P X (k) For any r.v. X with S X = {1, 2,,K} H(X) log K with equality iff P k = K 1, k = 1, 2,,K Read the proof by yourself.
45 ENCS6161 p.44/47 Entropy Example: r.v. X with S X = {1, 2, 3, 4} and The entropy: P H(X) = 1 2 log log log log 1 8 = 7 4
46 ENCS6161 p.45/47 Entropy Source Coding Theorem: the minimum average number of bits required to encode a source X is H(X). Example: Average number of bits = = 7 4 = H(X) Average number of bits = 2 > H(X).
47 ENCS6161 p.46/47 Differential Entropy For a continuous r.v., since P {X = x} = 0 the entropy is infinite. Differential entropy for continuous r.v. H(X) = f X (x) log f X (x)dx = E[ log f X (x)] Example: X is uniform in [a,b] [ ] 1 H X = E log b a = log(b a)
48 ENCS6161 p.47/47 Differential Entropy Example: X N(m,σ 2 ) f X (x) = 1 e (x m)2 2σ 2 2πσ H X = E[log f X (x)] = E [ log 1 2πσ (x m)2 2σ 2 ] = 1 2 log(2πσ2 ) = 1 2 log(2πeσ2 )
Chapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationNorthwestern University Department of Electrical Engineering and Computer Science
Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability
More informationChap 2.1 : Random Variables
Chap 2.1 : Random Variables Let Ω be sample space of a probability model, and X a function that maps every ξ Ω, toa unique point x R, the set of real numbers. Since the outcome ξ is not certain, so is
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationLecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable
Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed
More informationSTAT2201. Analysis of Engineering & Scientific Data. Unit 3
STAT2201 Analysis of Engineering & Scientific Data Unit 3 Slava Vaisman The University of Queensland School of Mathematics and Physics What we learned in Unit 2 (1) We defined a sample space of a random
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationChapter 2. Random Variable. Define single random variables in terms of their PDF and CDF, and calculate moments such as the mean and variance.
Chapter 2 Random Variable CLO2 Define single random variables in terms of their PDF and CDF, and calculate moments such as the mean and variance. 1 1. Introduction In Chapter 1, we introduced the concept
More informationLecture Notes 2 Random Variables. Discrete Random Variables: Probability mass function (pmf)
Lecture Notes 2 Random Variables Definition Discrete Random Variables: Probability mass function (pmf) Continuous Random Variables: Probability density function (pdf) Mean and Variance Cumulative Distribution
More informationAnalysis of Engineering and Scientific Data. Semester
Analysis of Engineering and Scientific Data Semester 1 2019 Sabrina Streipert s.streipert@uq.edu.au Example: Draw a random number from the interval of real numbers [1, 3]. Let X represent the number. Each
More informationSlides 8: Statistical Models in Simulation
Slides 8: Statistical Models in Simulation Purpose and Overview The world the model-builder sees is probabilistic rather than deterministic: Some statistical model might well describe the variations. An
More information1.1 Review of Probability Theory
1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More informationChapter 2: Random Variables
ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:
More information1 Random Variable: Topics
Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?
More informationFundamental Tools - Probability Theory II
Fundamental Tools - Probability Theory II MSc Financial Mathematics The University of Warwick September 29, 2015 MSc Financial Mathematics Fundamental Tools - Probability Theory II 1 / 22 Measurable random
More informationThings to remember when learning probability distributions:
SPECIAL DISTRIBUTIONS Some distributions are special because they are useful They include: Poisson, exponential, Normal (Gaussian), Gamma, geometric, negative binomial, Binomial and hypergeometric distributions
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More information3 Operations on One Random Variable - Expectation
3 Operations on One Random Variable - Expectation 3.0 INTRODUCTION operations on a random variable Most of these operations are based on a single concept expectation. Even a probability of an event can
More informationProbability and Distributions
Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated
More informationBrief Review of Probability
Maura Department of Economics and Finance Università Tor Vergata Outline 1 Distribution Functions Quantiles and Modes of a Distribution 2 Example 3 Example 4 Distributions Outline Distribution Functions
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationRandom variables. DS GA 1002 Probability and Statistics for Data Science.
Random variables DS GA 1002 Probability and Statistics for Data Science http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall17 Carlos Fernandez-Granda Motivation Random variables model numerical quantities
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 20, 2012 Introduction Introduction The world of the model-builder
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More information2 Continuous Random Variables and their Distributions
Name: Discussion-5 1 Introduction - Continuous random variables have a range in the form of Interval on the real number line. Union of non-overlapping intervals on real line. - We also know that for any
More informationChapter 4. Continuous Random Variables 4.1 PDF
Chapter 4 Continuous Random Variables In this chapter we study continuous random variables. The linkage between continuous and discrete random variables is the cumulative distribution (CDF) which we will
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationECE353: Probability and Random Processes. Lecture 7 -Continuous Random Variable
ECE353: Probability and Random Processes Lecture 7 -Continuous Random Variable Xiao Fu School of Electrical Engineering and Computer Science Oregon State University E-mail: xiao.fu@oregonstate.edu Continuous
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationContinuous Random Variables
Continuous Random Variables Recall: For discrete random variables, only a finite or countably infinite number of possible values with positive probability. Often, there is interest in random variables
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 14: Continuous random variables Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/
More informationCDA5530: Performance Models of Computers and Networks. Chapter 2: Review of Practical Random Variables
CDA5530: Performance Models of Computers and Networks Chapter 2: Review of Practical Random Variables Definition Random variable (R.V.) X: A function on sample space X: S R Cumulative distribution function
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More informationECE 302 Division 2 Exam 2 Solutions, 11/4/2009.
NAME: ECE 32 Division 2 Exam 2 Solutions, /4/29. You will be required to show your student ID during the exam. This is a closed-book exam. A formula sheet is provided. No calculators are allowed. Total
More informationLecture Notes 2 Random Variables. Random Variable
Lecture Notes 2 Random Variables Definition Discrete Random Variables: Probability mass function (pmf) Continuous Random Variables: Probability density function (pdf) Mean and Variance Cumulative Distribution
More informationRelationship between probability set function and random variable - 2 -
2.0 Random Variables A rat is selected at random from a cage and its sex is determined. The set of possible outcomes is female and male. Thus outcome space is S = {female, male} = {F, M}. If we let X be
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More information6.1 Moment Generating and Characteristic Functions
Chapter 6 Limit Theorems The power statistics can mostly be seen when there is a large collection of data points and we are interested in understanding the macro state of the system, e.g., the average,
More informationProbability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27
Probability Review Yutian Li Stanford University January 18, 2018 Yutian Li (Stanford University) Probability Review January 18, 2018 1 / 27 Outline 1 Elements of probability 2 Random variables 3 Multiple
More informationContinuous Random Variables and Continuous Distributions
Continuous Random Variables and Continuous Distributions Continuous Random Variables and Continuous Distributions Expectation & Variance of Continuous Random Variables ( 5.2) The Uniform Random Variable
More information3. Review of Probability and Statistics
3. Review of Probability and Statistics ECE 830, Spring 2014 Probabilistic models will be used throughout the course to represent noise, errors, and uncertainty in signal processing problems. This lecture
More informationStochastic Models of Manufacturing Systems
Stochastic Models of Manufacturing Systems Ivo Adan Organization 2/47 7 lectures (lecture of May 12 is canceled) Studyguide available (with notes, slides, assignments, references), see http://www.win.tue.nl/
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationLecture 3: Random variables, distributions, and transformations
Lecture 3: Random variables, distributions, and transformations Definition 1.4.1. A random variable X is a function from S into a subset of R such that for any Borel set B R {X B} = {ω S : X(ω) B} is an
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More informationDiscrete Random Variables
CPSC 53 Systems Modeling and Simulation Discrete Random Variables Dr. Anirban Mahanti Department of Computer Science University of Calgary mahanti@cpsc.ucalgary.ca Random Variables A random variable is
More informationCDA6530: Performance Models of Computers and Networks. Chapter 2: Review of Practical Random
CDA6530: Performance Models of Computers and Networks Chapter 2: Review of Practical Random Variables Definition Random variable (RV)X (R.V.) X: A function on sample space X: S R Cumulative distribution
More informationThe random variable 1
The random variable 1 Contents 1. Definition 2. Distribution and density function 3. Specific random variables 4. Functions of one random variable 5. Mean and variance 2 The random variable A random variable
More informationEE4601 Communication Systems
EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two
More informationProbability reminders
CS246 Winter 204 Mining Massive Data Sets Probability reminders Sammy El Ghazzal selghazz@stanfordedu Disclaimer These notes may contain typos, mistakes or confusing points Please contact the author so
More informationF X (x) = P [X x] = x f X (t)dt. 42 Lebesgue-a.e, to be exact 43 More specifically, if g = f Lebesgue-a.e., then g is also a pdf for X.
10.2 Properties of PDF and CDF for Continuous Random Variables 10.18. The pdf f X is determined only almost everywhere 42. That is, given a pdf f for a random variable X, if we construct a function g by
More informationCDA6530: Performance Models of Computers and Networks. Chapter 2: Review of Practical Random Variables
CDA6530: Performance Models of Computers and Networks Chapter 2: Review of Practical Random Variables Two Classes of R.V. Discrete R.V. Bernoulli Binomial Geometric Poisson Continuous R.V. Uniform Exponential,
More informationReview of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
More informationRandom Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay
1 / 13 Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 8, 2013 2 / 13 Random Variable Definition A real-valued
More informationComputer Science, Informatik 4 Communication and Distributed Systems. Simulation. Discrete-Event System Simulation. Dr.
Simulation Discrete-Event System Simulation Chapter 4 Statistical Models in Simulation Purpose & Overview The world the model-builder sees is probabilistic rather than deterministic. Some statistical model
More informationChapter 5. Statistical Models in Simulations 5.1. Prof. Dr. Mesut Güneş Ch. 5 Statistical Models in Simulations
Chapter 5 Statistical Models in Simulations 5.1 Contents Basic Probability Theory Concepts Discrete Distributions Continuous Distributions Poisson Process Empirical Distributions Useful Statistical Models
More informationLecture 3. Discrete Random Variables
Math 408 - Mathematical Statistics Lecture 3. Discrete Random Variables January 23, 2013 Konstantin Zuev (USC) Math 408, Lecture 3 January 23, 2013 1 / 14 Agenda Random Variable: Motivation and Definition
More informationIntroduction to Machine Learning
What does this mean? Outline Contents Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola December 26, 2017 1 Introduction to Probability 1 2 Random Variables 3 3 Bayes
More informationRandom variable X is a mapping that maps each outcome s in the sample space to a unique real number x, < x <. ( ) X s. Real Line
Random Variable Random variable is a mapping that maps each outcome s in the sample space to a unique real number, <
More informationExpectation of Random Variables
1 / 19 Expectation of Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 13, 2015 2 / 19 Expectation of Discrete
More informationChapter 6: Random Processes 1
Chapter 6: Random Processes 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More informationChapter 5. Chapter 5 sections
1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More information1 Review of Probability and Distributions
Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote
More informationMoments. Raw moment: February 25, 2014 Normalized / Standardized moment:
Moments Lecture 10: Central Limit Theorem and CDFs Sta230 / Mth 230 Colin Rundel Raw moment: Central moment: µ n = EX n ) µ n = E[X µ) 2 ] February 25, 2014 Normalized / Standardized moment: µ n σ n Sta230
More informationSTAT/MATH 395 A - PROBABILITY II UW Winter Quarter Moment functions. x r p X (x) (1) E[X r ] = x r f X (x) dx (2) (x E[X]) r p X (x) (3)
STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 07 Néhémy Lim Moment functions Moments of a random variable Definition.. Let X be a rrv on probability space (Ω, A, P). For a given r N, E[X r ], if it
More informationExponential Distribution and Poisson Process
Exponential Distribution and Poisson Process Stochastic Processes - Lecture Notes Fatih Cavdur to accompany Introduction to Probability Models by Sheldon M. Ross Fall 215 Outline Introduction Exponential
More informationProbability Models. 4. What is the definition of the expectation of a discrete random variable?
1 Probability Models The list of questions below is provided in order to help you to prepare for the test and exam. It reflects only the theoretical part of the course. You should expect the questions
More informationProbability Review. Gonzalo Mateos
Probability Review Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ September 11, 2018 Introduction
More informationCourse: ESO-209 Home Work: 1 Instructor: Debasis Kundu
Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear
More informationBasics of Stochastic Modeling: Part II
Basics of Stochastic Modeling: Part II Continuous Random Variables 1 Sandip Chakraborty Department of Computer Science and Engineering, INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR August 10, 2016 1 Reference
More informationChapter 1 Statistical Reasoning Why statistics? Section 1.1 Basics of Probability Theory
Chapter 1 Statistical Reasoning Why statistics? Uncertainty of nature (weather, earth movement, etc. ) Uncertainty in observation/sampling/measurement Variability of human operation/error imperfection
More informationS n = x + X 1 + X X n.
0 Lecture 0 0. Gambler Ruin Problem Let X be a payoff if a coin toss game such that P(X = ) = P(X = ) = /2. Suppose you start with x dollars and play the game n times. Let X,X 2,...,X n be payoffs in each
More informationECE 4400:693 - Information Theory
ECE 4400:693 - Information Theory Dr. Nghi Tran Lecture 8: Differential Entropy Dr. Nghi Tran (ECE-University of Akron) ECE 4400:693 Lecture 1 / 43 Outline 1 Review: Entropy of discrete RVs 2 Differential
More information3 Continuous Random Variables
Jinguo Lian Math437 Notes January 15, 016 3 Continuous Random Variables Remember that discrete random variables can take only a countable number of possible values. On the other hand, a continuous random
More informationTwelfth Problem Assignment
EECS 401 Not Graded PROBLEM 1 Let X 1, X 2,... be a sequence of independent random variables that are uniformly distributed between 0 and 1. Consider a sequence defined by (a) Y n = max(x 1, X 2,..., X
More information2 (Statistics) Random variables
2 (Statistics) Random variables References: DeGroot and Schervish, chapters 3, 4 and 5; Stirzaker, chapters 4, 5 and 6 We will now study the main tools use for modeling experiments with unknown outcomes
More information3 Operations on One R.V. - Expectation
0402344 Engineering Dept.-JUST. EE360 Signal Analysis-Electrical - Electrical Engineering Department. EE360-Random Signal Analysis-Electrical Engineering Dept.-JUST. EE360-Random Signal Analysis-Electrical
More informationStochastic Models in Computer Science A Tutorial
Stochastic Models in Computer Science A Tutorial Dr. Snehanshu Saha Department of Computer Science PESIT BSC, Bengaluru WCI 2015 - August 10 to August 13 1 Introduction 2 Random Variable 3 Introduction
More informationBMIR Lecture Series on Probability and Statistics Fall, 2015 Uniform Distribution
Lecture #5 BMIR Lecture Series on Probability and Statistics Fall, 2015 Department of Biomedical Engineering and Environmental Sciences National Tsing Hua University s 5.1 Definition ( ) A continuous random
More informationPart IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Theorems Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationEE514A Information Theory I Fall 2013
EE514A Information Theory I Fall 2013 K. Mohan, Prof. J. Bilmes University of Washington, Seattle Department of Electrical Engineering Fall Quarter, 2013 http://j.ee.washington.edu/~bilmes/classes/ee514a_fall_2013/
More informationStatistics for scientists and engineers
Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3
More informationWeek 12-13: Discrete Probability
Week 12-13: Discrete Probability November 21, 2018 1 Probability Space There are many problems about chances or possibilities, called probability in mathematics. When we roll two dice there are possible
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationProbability and Statistics Concepts
University of Central Florida Computer Science Division COT 5611 - Operating Systems. Spring 014 - dcm Probability and Statistics Concepts Random Variable: a rule that assigns a numerical value to each
More informationLecture 1: Review on Probability and Statistics
STAT 516: Stochastic Modeling of Scientific Data Autumn 2018 Instructor: Yen-Chi Chen Lecture 1: Review on Probability and Statistics These notes are partially based on those of Mathias Drton. 1.1 Motivating
More informationContinuous distributions
CHAPTER 7 Continuous distributions 7.. Introduction A r.v. X is said to have a continuous distribution if there exists a nonnegative function f such that P(a X b) = ˆ b a f(x)dx for every a and b. distribution.)
More information1 Review of Probability
1 Review of Probability Random variables are denoted by X, Y, Z, etc. The cumulative distribution function (c.d.f.) of a random variable X is denoted by F (x) = P (X x), < x
More informationBASICS OF PROBABILITY
October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,
More informationRandom Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.
Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example
More informationRandom Variables. P(x) = P[X(e)] = P(e). (1)
Random Variables Random variable (discrete or continuous) is used to derive the output statistical properties of a system whose input is a random variable or random in nature. Definition Consider an experiment
More informationGeneral Random Variables
1/65 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Probability A general random variable is discrete, continuous, or mixed. A discrete random variable
More informationMeasure-theoretic probability
Measure-theoretic probability Koltay L. VEGTMAM144B November 28, 2012 (VEGTMAM144B) Measure-theoretic probability November 28, 2012 1 / 27 The probability space De nition The (Ω, A, P) measure space is
More informationA Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.
A Probability Primer A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. Are you holding all the cards?? Random Events A random event, E,
More informationp. 6-1 Continuous Random Variables p. 6-2
Continuous Random Variables Recall: For discrete random variables, only a finite or countably infinite number of possible values with positive probability (>). Often, there is interest in random variables
More informationSystem Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models
System Simulation Part II: Mathematical and Statistical Models Chapter 5: Statistical Models Fatih Cavdur fatihcavdur@uludag.edu.tr March 29, 2014 Introduction Introduction The world of the model-builder
More information