Communication Theory II Lecture 8: Stochastic Processes Ahmed Elnakib, PhD Assistant Professor, Mansoura University, Egypt March 5 th, 2015 1
o Stochastic processes What is a stochastic process? Types: Lecture Outlines o Stationary vs. nonstationary processes o Strictly stationary processes vs. weakly stationary processes (e.g., Ergodic processes) Parameters: o Mean, correlation, Covariance o Power spectral density Transmission of a weakly stationary process in LTI system Poisson and Gaussian Processes 2
Introduction to Stochastic Processes o Received signal at the wireless channel output varies randomly with time Processes of this kind are said to be random or stochastic Signal with random changes Source Encoder Channel Decoder o It is not possible to predict the exact value of a signal drawn from a stochastic process However, it is possible to characterize the process in terms of statistical parameters such as average power, correlation, and power spectra 3
Stochastic Processes o A stochastic process may be presented as the sample space or ensemble composed of functions of time o Each sample point of the sample space pertaining to a stochastic process is a function of time o The totality of sample points corresponding to the aggregate of all possible realizations of the stochastic process is called the sample space o A single realization of a process is a random waveform that evolves across time Each realization of a process is associated with a sample point Set (ensemble) of sample functions 4
Stochastic Processes (cont d) o Consider a stochastic process specified by: outcomes s observed from some sample space S events defined on the sample space S probabilities of these events o Suppose, we assign each sample point s a function in time in accordance with the rule: X(t,s j ), -T t T 2T: total observation interval o For a fixed sample point s j the graph of the function X(t,s j ) is called a realization or sample function of the stochastic process To simplify we denote this sample function X j (t)= X(t,s j ), -T t T Set (ensemble) of sample functions 5
Stochastic Processes and R.Vs o A realization or sample function of the stochastic process: X j (t)= X(t,s j ), -T t T o At particular instant of time, we deal with a R.V sampled (observed) at that instant of time o A random variable is constituted as the set of numbers observed at a fixed time t k inside the observation interval A stochastic process X(t,s) or X(t) is represented by the time indexed ensemble (family) of random variables {X(t k,s)} or {X(t k )} Set (ensemble) of sample functions 6
Stochastic Processes and R.Vs (cont d) o Stochastic process the outcomes of stochastic experiment is mapped into a waveform (function of time) o R.V. the outcomes of a stochastic process is mapped to a number A stochastic process X(t) is an ensemble of time functions, which, together with a probability rule, assigns a probability to any meaningful event associated with an observation of one of the sample functions of the stochastic process Set (ensemble) of sample functions 7
Important Types of Stochastic Processes o Stationary and nonstationary o Strictly stationary and weakly stationary (wide-sense stationary) Ergodic processes (subsets of weakly stationary processes) 8
Stationary Processes o In dealing with stochastic processes encountered in the real world: We often find that the statistical characterization of a process is independent of the time at which observation of the process is initiate That is, if such a process is divided into a number of time intervals, the various sections of the process exhibit essentially the same statistical properties This process is said to be stationary. Otherwise, it is said to be nonstationary o A stationary process arises from a stable phenomenon that has evolved into a steady-state mode of behavior, whereas a nonstationary process arises from an unstable phenomenon 9
Strictly stationary 10
Strictly stationary (cont d) oa stochastic process X(t), initiated at time t = -, is strictly stationary if the joint distribution of any set of random variables obtained by observing the process X(t) is invariant with respect to the location of the origin t = 0 ohow such a process is random? The finite-dimensional distributions depend on the relative time separation between random variables, not on their absolute time However, the stochastic process has the same probabilistic behavior throughout the global time t 11
Jointly Strictly Stationary Processes 12
Properties of Strictly Stationary Processes 1. 2. 13
Example (Multiple Spatial Windows) 14
Weakly (Wide-sense) stationary Processes o A stochastic process X(t) is said to be weakly stationary if its secondorder moments satisfy the following two conditions: 1. The mean of the process X(t) is constant for all time t 2. The autocorrelation function of the process X(t) depends solely (alone) on the difference between any two times at which the process is sampled auto in autocorrelation refers to the correlation of the process with itself 15
Statistical Parameters of Stochastic Processes o Mean o Correlation o Cross-correlation o Covariance o Power Spectral Density o Cross Spectral Density 16
Mean of Stochastic Processes othe mean of a real-valued stochastic process X(t) is the expectation of the random variable obtained by sampling the process at some time t Where is the first-order probability density function of the process X(t), observed at time t Note also that the use of single X as subscript in is intended to emphasize the fact that is a first-order moment 17
Mean of Weakly stationary Processes o The mean of a process is given by: o For a process X(t), to satisfy the first condition of weak stationary, The mean is a constant for all time (independent of time t) 18
Autocorrelation of A Stochastic Process oautocorrelation function of the stochastic process X(t) is the expectation of the product of two random variables, X(t 1 ) and X(t 2 ), obtained by sampling the process X(t) at times t 1 and t 2 : Where is the second-order probability density function of the process X(t), observed at times t 1 and t 2 19
Autocorrelation of A Weakly Stationary Process The autocorrelation function of the process X(t) depends solely (alone) on the difference between any two times at which the process is sampled Equivalently, if t 1 = t and t 2 = t 1 + τ : 20
Autocovariance of A Weakly Stationary Process 21
Mean and Autocorrelation of Weakly stationary Processes for the mean and the autocorrelation 22
Properties of the Autocorrelation Function o Mean Square value o Symmetry Thus the auto correlation may be defined also as: o Maximum magnitude at zero shift: or o Normalized autocorrelation function: 0 ρ XX 1 23
Autocorrelation Function and Decorrelation Time o The autocorrelation function R XX τ provides a means of describing the interdependence of two random variables obtained by sampling the stochastic process X(t) at times τ seconds apart o The more rapidly the stochastic process X(t) changes with time, the more rapidly will the autocorrelation function R XX τ decrease from its maximum R XX 0 as increases o At a decorrelation time τ dec, such that, for τ > τ dec : The magnitude of the autocorrelation function R XX τ remains below some prescribed value The one percent decorrelation time τ dec of a weakly stationary process of X(t) zero mean is the time taken for the magnitude of the autocorrelation function R XX τ decrease, for example, to 1% of its maximum value R XX 0 24
Example 1: Sinusoidal Wave with Random Phase o Consider a sinusoidal signal with random phase, defined by o The random variable is equally likely to have any value in the interval [ ] o Each value of corresponds to a point in the sample space S of the stochastic process X(t) o The process X(t) represents a locally generated carrier in the receiver of a communication system, which is used in the demodulation of a received signal o Find and plot the autocorrelation function 25
Homework o Write a Matlab program showing that by increasing the number of samples from a uniformly distributed R.V., the normalized histogram of these samples is closer to a uniformly distributed mass function frequency of X=x normalized histogram at (X=x i )= i summation of all frequencies = frequency of X=x i Number of samples o Group is allowed (3-5) using the oral grouping o Post your solution by Email to eng.nakib@gmail.com o You can be asked individually about your solution 26
Solution 27
Questions 28