ECE 650 Lecture #10 (was Part 1 & 2) D. van Alphen D. van Alphen 1
Lecture 10 Overview Part 1 Review of Lecture 9 Continuing: Systems with Random Inputs More about Poisson RV s Intro. to Poisson Processes More on Poisson Processes Introduction to Marov Processes D. van Alphen 2
Reviewing Lecture 9 0-mean RP V(t) is white if C V (t) = d(t) ( V(t i ) and V(t j ) are uncorrelated RV s if t i t j ) RP X(t) is normal (Gaussian) if RV s X(t 1 ),, X(t n ) are jointly normal for any n and any t 1, t 2,, t n RP X(t) is strict-sense stationary (SSS) if (all of) its statistical properties are invariant to shifts of the time origin. RP X(t) is wide-sense stationary (WSS) if E{X(t)} = h(t) = h, for all t (constant mean) R X (t 1, t 2 ) = R X (t) = E{ X(t + t) X(t) }, t = t 1 t 2 Note: E{ X 2 (t) } = R(0), the total average power of the RP D. van Alphen 3
Power Spectral Densities (PSD s) for WSS RP s Concept: The PSD, S X (w), of RP X(t) is a function which measures the distribution of power (on the average, of the RP) in the frequency domain. Wiener-Khinchin Theorem: For WSS processes, S X (w) = F { R X (t) } F R X (t) S X (w) (F : Fourier Transform) Integrate the PSD to find the total average power in the random process: P X 1 2 S ( w)dw X S X (f )df R X (0) E(X 2 (t)) D. van Alphen 4
White Noise, Re-visited White noise can be defined by either its autocorrelation function (delta-correlated) or its PSD (flat or constant for all w)): R X (t) (N 0 /2) t F S X (w) N 0 /2 w 1. E{ X 2 (t) } 1 2 S ( w)dw X R X (0) 2. E{X(t)} = (no constant term in R X (t)) 3. White noise has power, mean. D. van Alphen 5
Ergodic RP s A RP is ergodic if almost every member of the ensemble is typical of the ensemble, with the same statistical properties. For ergodic processes, time-averaging of a single sample function can be used to replace ensemble averaging. X(t) : notation for time-averaging of X(t) Ergodic stationary, but not conversely D. van Alphen 6
Continuing: Systems with Random Inputs Meaning: sample function X(t, z i ) yields output Y(t, z i ) Mean: E{Y(t)} = X(t) g Y(t) = g[x(t)] g(x) f (x; t)dx Autocorrelation: R Y (t 1, t 2 ) = E{ Y(t 1 ) Y(t 2 ) } x fix t; handle one RV at a time g(x1)g(x2) fx(x1,x2;t1,t2 )dx D. van Alphen 7
LTI Systems with Random Inputs X(t, x) = x(t) Inputting one sample function at a time h, H Y(t, x) = y(t) = x(t) * h(t) Goal: to describe the output RP, statistically Now consider LTI system with impulse response h(t), and frequency response H(f). From Lecture 9: S Y (f) = H(f) 2 S X (f) R YY (t) = R XX (t) * h(t) * h(-t) Also, we can find the mean of the output RP Y(t): h Y (t) = E[Y(t)] = E[h(t) * X(t, x)] = h(t) * h x (t) If the input RP is WSS, then h x (t) = h x, so h Y (t) = h X h(t t)dt h X H(0) D. van Alphen 8
Example Consider the RC LP Filter, with impulse response: h(t) = (1/t) exp(-t/t) u(t), where t = RC Describe the output RP if the input process is white Gaussian noise N(t), with PSD N 0 /2. Describe : give pdf, mean and Solution: R Y (t) or S Y (f). 1. Since the system is linear, the output RP is Gaussian. 2. 1 1 From a FT Table: H(f) =, so H(f) 2 = 1 j2ft 1 (2ft) 3. N0 / 2 Hence, S Y (f) = 2 1 (2ft) Is this output white? 2 4. h Y = h X H(0) = 0 D. van Alphen 9
Summary: Systems with Random Inputs X(t) g, h, H Y(t) = g[x(t)] If the system is LTI, h Y (t) = E[Y(t)] = E[h(t) * X(t, x)] = h(t) * h x (t) If the system is LTI and X(t) is WSS S Y (f) = H(f) 2 S X (f) R YY (t) = R XX (t) * h(t) * h(-t) h Y = h X H(0), the input mean times the system dc gain D. van Alphen 10
Repeating: Lecture 1, p. 23 Poisson Random Variables (another ind of counting RV) RV X is Poisson with parameter a iff P X (m) Pr{ X m} m a m! e a m 0, 1, 2, 3 PDF setch for the case: a = 3 Pr{ X m} m! 0 3 3 3 PX (0) Pr{ X 0} e e.04979 0! P X (1) Pr{ X 1} 1 3 e 1! 3 3e 3.14936 m e 3.. P X (m).224.224 (rounding).15.168.05. m 0 1 2 3 4 ECE 650 D. van Alphen 11
More on Poisson RV s - Say X is Poisson with mean & var. a Introduced in Lect. 1, pp. 23-26 a a Repeating: PX (m) Pr{ X m} e m 0, 1, 2, m! a 1 e a Ratio of adjacent terms in the pdf: Pr{X 1} ( 1)! Pr{X } a a e a! Consider 3 cases for the relationship between mean a and : 1. < a Pr{X = -1} < Pr{X = } pdf increases with 2. > a Pr{X = -1} > Pr{X = } pdf decreases as increases 3. = a (only possible for integer values of a) P{X = -1} = Pr{X = } 2 equal (adjacent) values of pdf at X = -1 and X = (See previous page for example) ECE 650 D. van Alphen 12 m
Poisson Distributions, continued If a Poisson RV counts the number of occurrences of some event occurring with rate q, over a time period of t seconds, then the RV is Poisson with parameter a= qt: Pr{X } (qt)! e qt 2, Example: The number of phone calls to be handled in a cellular system during a particular hour of the day is a Poisson RV, with average rate 80 calls/minute. Let RV X count the number of calls to be handled in an interval of length 6 seconds. Pr{X } (qt)! e (qt), (80.1)! e 0, 1, (80.1) (8)! e (8) ECE 650 D. van Alphen 13
Poisson Approximation for Binomial Distribution (large n and small p (or n, p unnown but np nown)) If n is large, and p is small (so np is medium), and << n, then n n a a Pr( successes) p q e where a np! Note: a = np is the expected number of successes in n trials Example (Carol Ash): Consider a typist who maes (on the average) 3 mistaes per page. Find the probability that the typist maes 10 mistaes on p. 392. Note: p = prob. of error on typing a single character n = number of characters typed per page unnown But a = np = 3 = exp. number of errors per page n 10 10 n 10 3 Pr(X 10) p q e 3 8.1x10 4 10 10! ECE 650 D. van Alphen 14
Queueing Theory: Poisson Arrivals As customers arrive (say to mae a ban transaction), write down the arrival time of each person Let a be the average # of customers arriving per hour Divide the hour into a large # (say n) of subintervals, so that at most 1 ( 0 or 1) person can arrive in a subinterval. Each subinterval is a Bernoulli trial, where success a customer arriving n, p unnown np = a, nown Use Poisson approximation for Binomial Poisson Arrivals: x 10:00 11:00 x x x x AM 1 2 3 4 5 6 n AM ECE 650 D. van Alphen 15
Queueing Theory: Poisson Arrivals 10:00 AM Poisson Arrivals: x x x x x 1 2 3 4 5 6 n 11:00 AM For independent arrivals, the # of arrivals in a time period is Poisson-distributed: Pr( arrivals in a time period) a! a where a: arrival rate (average # of arrivals/time period) e ECE 650 D. van Alphen 16
Example: Poisson Arrivals Suppose meteor bursts occur at a particular location in the upper atmosphere at a rate of 4 per minute. (a) Find the probability 2 bursts occur in the next minute. (b) Find the probability that at most 2 bursts occur in the next 3 minutes. 4 2 4 (a) Pr(2 bursts in 1 minute) e.1465 2! (b) Pr(at most 2 bursts in the next 3 minutes) = P(0, 1, or 2 bursts in the next 3 minutes). Note that for a period of 3 minutes, we have a = 12 bursts, on the average. Hence P(0, 1, or 2 bursts in the next 3 minutes) 0 1 2 = 12 12 12 12 12 12 e e e = 5.2226 x 10-4 0! 1! 2! ECE 650 D. van Alphen 17
Poisson Points Experiment: Place n points at random on (-T/2, T/2). Find the probability that of the n points are in sub-interval (t 1, t 2 ), of length t a, as shown below: -T/2 t 1 t 2 T/2 t a Note: Each placement of a point is a Bernoulli trial, where success means that the point landed in (t 1, t 2 ), and failure meaning that it didn t. Thus, Pr{ pts. in t a } = Pr( successes) where p = (t 2 t 1 )/T = t a /T n p q n () ECE 650 D. van Alphen 18
Poisson Points, continued Now suppose that n >> 1 and t a << T ( n large, p = t a /T small) Poisson Approximation to Binomial Distribution O.K. Pr{ pts. in t a } = n p q n a! e a (nt a / T)! e (nt a / T) for near np = nt a /T. Now, let n, and T, with the constraint that n/t = a 1, a constant representing the density of points. Then Pr{ pts. in t a } ( a 1t a) a e 1t a! ( ) where a 1 = n/t, the density of points The # of Poisson Points in an Interval is a Poisson RV. ECE 650 D. van Alphen 19
Waiting Time for Poisson Points or Arrivals If events (e.g., arrivals) are Poisson distributed with a = n/t, then the waiting time W from a given event to the next event is exponentially distributed: f W (w) = a e -aw, w > 0 Proof: Suppose that an arrival occurs at time 0: 0 w t a = w F w (w) = Pr{W w} = Pr(at least 1 other event occurs at or before time w) = 1 Pr(0 events occur at or before time w) ( a w) 0 1 e 0! 1 e d f w (w) = F (w) ae aw w, dw w 0 ( aw) aw ECE 650 D. van Alphen 20
Poisson Points: Summary 1. Pr{ points in t a } at ) a! e ( at ) ( a, a n/ T (density of points) 2. If 2 intervals (t 1, t 2 ) and (t 3, t 4 ) are non-overlapping, then the events { a in (t 1, t 2 )} and { b in (t 3, t 4 )} are independent. 3. If events (e.g., arrivals) are Poisson distributed with a = n/t, then the waiting time W from a given event to the next event is exponentially distributed (with parameter a, mean 1/a): f w (w) = a e -aw, w > 0 ECE 650 D. van Alphen 21
Poisson Arrivals: Example Suppose that raindrops land on a weather-station collection grid at the rate of 100 drops/sec. Find the probability that the time interval between 2 successive drops is greater than 1 ms. Raindrop arrival Poisson points with a = 100 drops/sec. Waiting time W Exponential with a = 100 Pr{W >.001} = 1 Pr(W.001) = 1 F W (.001) =e -100(.001) = e -.1 =.905 ECE 650 D. van Alphen 22
Poisson RP s Consider a RP which counts the number of occurrences of some event in the time interval [0, t). Each occurrence of the event is said to be an arrival. The RP is a Poisson Process if it has the following properties: 1. (Independent Increments) The number of arrivals in any 2 non-overlapping intervals are independent of each other. 2. (Stationary Increments) The number of arrivals in an interval [t, t+t) depends only of the length of the interval, t. 3. (Distribution of Infinitesimal Increments) For an interval of infinitesimal length, [t, t+dt), the probability of a single arrival is proportional to Dt, and the probability of more than one arrival is negligible compared to Dt. D. van Alphen 23
Poisson Processes, continued X: counts arrivals on interval (0, t) If we let l be the constant of proportionality in property 3: i.e., Pr(1 arrival in [t, t + Dt) ) = l dt l: arrival rate It can be shown (see text, pp. 361-2) that Pr(X = 0 for interval (0, t) = P X (0; t) = e -lt u(t) ( lt) Pr(X = for interval (0, t)) = P X (; t) = e lt! u(t) Note: average # of arrivals in [0, t) is lt. PMF Function Note: process is not stationary in the mean. for RV at time t D. van Alphen 24
Poisson Processes, continued X: counts arrivals on interval (0, t) For a Poisson RP with arrival rate l, ( lt) Pr(X = for interval (0, t)) = P X (; t) = e lt u(t) (*)! Example (Miller & Childers, 8.37) Suppose that the arrival of calls at a switchboard is a Poisson process with rate l =.1 calls/min. Find the probability that the number of calls arriving in a 10- minute interval is less than 10. 9 (1) 1 From (*): Pr(X < 10) = e u(t) (since lt = 1) 0! In MATLAB: >> poisscdf(9,1) ans = 0.999999888574521 D. van Alphen 25
Example, continued Since we are fixing time to t = 10, the Poisson Process (at t = 10) becomes a Poisson RV with parameter lt = 1, with PMF: 0.4 0.35 0.3 0.25 PMF for Poisson RV (from process at t = 10) MATLAB Code: x = 0:12; poisspdf(x, 1) stem(x, ans) 0.2 0.15 0.1 0.05 0 0 2 4 6 8 10 12 Checing prev. ans. >> x = 0:9; >> y = poisspdf(x,1); >> z = cumsum(y); >> z(10) ans = 0.999999888574521 D. van Alphen 26
Poisson Process Sample Functions Now suppose that we want to setch a sample function from a (different) Poisson Process. Model the time between arrivals as exponential, mean 2: >> wt_time = exprnd(2,7,1); >> time_to_arrival = cumsum(wt_time); >> [wt_time time_to_arrival] ans = 1.20 1.20 3.86 5.06 0.32 5.38 0.95 6.33 2.09 8.42 1.33 9.75 1.82 11.58 Arrival times 1.2 5.1 5.4 6.3 8.4 9.8 D. van Alphen 11.6 27
Poisson Process Sample Function, Example 7 6 5 Sample Function from Poisson Process MATLAB Code: >> x = [0 1.2 11.6]; >> y = [0:7]; >> stairs(x, y) 4 3 2 1 0 0 2 4 6 8 10 12 D. van Alphen 28
Marov Process Random process X(t) is a Marov process if, for any time points t 1 < t 2 < < t n < t n+1, the process satisfies the condition concerning the conditional pdf: f X (x n x n-1, x n-2,, x 1 ; t n,, t 1 ) = f X (x n x n-1 ; t n, t n-1 ) The Poisson counting process is an example of a Marov Process (due to the independent increments property). A discrete-valued Marov process satisfies the condition concerning the conditional PMF: 1 st order Marov P X (x n x n-1, x n-2,, x 1 ; t n,, t 1 ) = P X (x n x n-1 ; t n, t n-1 ) D. van Alphen 29