Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes

Size: px
Start display at page:

Download "Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes"

Transcription

1 Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes Klaus Witrisal Signal Processing and Speech Communication Laboratory Graz University of Technology November 6, 5 Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36 Outline 4- Probability (brief review) 4- Random Variables 4-3 Random Processes 4-4 Noise Example for a Random Process Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36

2 4- Probability Fundamental concept: random experiment with an uncertain outcome Definitions: ω... outcomes: ω Ω Ω... sample space: set of all outcomes Events: subsets of sample space Ω for which propability can be defined Probability: functional mapping of events onto reals: P (E) for all events E B B... collection of (all possible) events (Ω, B,P)... probability space Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 3/36 Probability (cont d) Probability is: relative frequency of an event ocurring in n trials A, B... events P (A) = lim n n A n n A... number of occurancies of A n... number of trials definitions sure event E: P (E) = null event A =: P (A) = Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 4/36

3 Probability (cont d) Events can be represented as sets (Venn diagram) product of events A and B (intersection) C = A B = AB mutually exclusive events A B =,i.e.p (AB) = sum of events A or B (union) C = A B = A + B P (A + B) =P (A)+P(B) P (AB) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 5/36 Probability (cont d) conditional probability P (A B) = lim n n AB n B = P (AB) P (B) probability that an event A occurs, given that an event B has also occured product theorem P (AB) =P (A B)P (B) =P (B A)P (A) leads to the Bayes-Theorem P (A B) = P (B A)P (A) P (B) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 6/36

4 Probability (cont d) total probability: event B occurs jointly with one of the mutually exclusive events A,A,..., A K, where K P (A k )= k= Find the probability P (B), givenp (B A k ) and P (A k ): P (B) = P (B A )P (A )+P(B A )P (A )+... +P (B A K )P (A K ) = K K P (B A k )P (A k )= P (BA k ) k= k= Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 7/36 Probability (cont d) Def.: a-priori probabilities: P (A k ) Def.: a-posteriori probability: P (A k B) Assume: event B occurs jointly with one of the mutually exclusive events A,A,..., A K : Find the probability for A k, when B has already occured. (B could be an observation of A k ) can be solved using the Bayes-theorem and the total probability P (A k B) = P (B A k)p (A k ) P (B) = P (B A k)p (A k ) K i= P (B A i)p (A i ) given P (A k ) and P (B A k ) (likelihoods) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 8/36

5 4- Random Variables sample space Ω (with outcomes ω i,ν j (and events)) is mapped onto the set of reals R the random variable is a function of the outcomes notation: X = X(ω i ),Y = Y (ν j ), short X, Y upper-case symbols are used to distinguish from defined variables discrete random variables (RV) e.g. dice X {,,..., 6} (number of eyes); P (X = x m )= 6 continuous RV e.g. voltage of a battery X = U Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 9/36 Characterization of RVs cumulative distribution function CDF (dt: Verteilungsfunktion) F X (a) =P (X a) = lim n dimensionless probability density function PDF (dt: Wahrscheinlichkeitsdichtefunktion) unit is /[x] f X (x) = df X(x) dx = lim n,δx n (X a) n [ Δx ( nδx n ) ] Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36

6 Characterization of RVs (cont d) properties of the CDF F X (x), nondecreasing F X () = F X ( ) = following the def.: discrete points at X = x are included probability that X lies in some interval: P (x <X x ) = P (X x ) P (X x ) = F X (x ) F X (x ) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36 Characterization of RVs (cont d) properties of the PDF PDF is nonnegative f X (x) area under the PDF f X(x)dx = computation of the CDF from the PDF F X (x) = x+ɛ f X (λ)dλ = P (X x) probability that X lies in some interval: P (x <X x )= x +ɛ x +ɛ f X (λ)dλ Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36

7 Ensemble Averages and Moments mean value, expected value, first-order moment, ensemble average E{X} = m X = (center of gravity of the PDF) k-th order moment E{X k } = xf X (x)dx x k f X (x)dx centralized k-th-order moment (moment about mean) E{(X m X ) k } = (x m X ) k f X (x)dx Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 3/36 Ensemble Averages and Moments (cont d) variance (centralized second moment) σ X =var{x} =E{(X m X) } = (x m X ) f X (x)dx standard deviation σ X = var{x} generalized: mean of a function g(x) of an RV E{g(X)} = g(x)f X (x)dx Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 4/36

8 Ensemble Averages and Moments (cont d) definition through events (for lim n ) E{X} = lim n E{g(X)} = lim n n n n X(A i ) i= n g[x(a i )] i= canbeusedtoestimate moments will be an approximation for a limited sample n< Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 5/36 Extension to two Random Variables bivariate statistics: joint characterization of two RVs for > variables: multivariate statistics CDF F XY (x, y) =P (X x, Y y) PDF f XY (x, y) = F XY (x, y) x y Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 6/36

9 4 Scatter plot of samples of, Y 3 Scatter plot of samples of, Y 3 samples of Y samples of Y samples of samples of joint PDF of and Y joint PDF of and Y Y Y - Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 7/36 correlation Extension to two Random Variables (cont d) R XY =E{XY } = uncorrelated RVs xyf XY (x, y)dx dy R XY =E{XY } =E{X}E{Y } = m X m Y orthogonal RVs R XY =E{XY } = covariance K XY =E{(X m X )(Y m Y )} = R XY m X m Y Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 8/36

10 4-3 Random processes X(A i,t)... function of two variables: A i... event, realization t... time (usually) for an event A i : sample function x i (t) ensemble of sample functions: {x i (t)}, i random process: X(t) (also in lower case) illustration of an ensemble of sample functions: (blackboard) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 9/36 Characterization of random processes ensemble averages mean (at time t) m X (t) =E{X(t)} = xf X(t) (x) dx can be time-variant (e.g. sine-wave plus noise) autocorrelation(function ACF) R X (t,t )=E{X(t )X(t )} = x x f X(t )X(t )(x,x ) dx dx f X(t )X(t )(x,x )... joint PDF for X(t ) and X(t ) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36

11 relizations till 5 of an ensemble of trials 4 Mean value; first moment estimate of E x(t) Second moment (= autocorrelation) estimate of E x(5) x(t) sample number, time t sample number, time Power Spectrum Magnitude (db) Fundamentals.of Digital.4Commun.Ch..6 4:.8 Random Variables. and Random.4 Processes.6 p..8 /36. Frequency 4 Scatter plot of two samples of (t) 3 Scatter plot of two samples of (t) 3 (t ) for t = sample 3 - (t ) for t = sample (t ) for t = sample (t ) for t = sample joint PDF of (t ) and (t ) joint PDF of (t ) and (t ) x x - x x Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. /36

12 relizations till 5 of an ensemble of trials 5 GRAZ UNIVERSITY Mean value; OFfirstTECHNOLOGY moment estimate of E x(t) estimate of E x(5) x(t) Second moment (= autocorrelation) sample number, time t sample number, time Power Spectrum Magnitude (db) Frequency Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 3/36 Characterization of random processes (cont d) stationarity (time-invariance) first-order stationarity (stationary w.r.t. mean) E{X(t)} = m X =const. second-order stationarity (stationary w.r.t. ACF) R X (t,t )=R X (τ) =E{X(t)X(t + τ)} where τ = t t i.e.: no variability w.r.t. time shift exact definition: Stationarity (of n-th order): PDF (of n-th order) is invariant w.r.t. time shifts Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 4/36

13 Characterization of random processes (cont d) Important definitions: Strict-Sense-Stationary (SSS) (Stationär im strengen Sinn) PDFs of any order are stationary Wide-Sense-Stationary (WSS) (Stationär im weiteren Sinn) Mean and autocorrelation function (ACF) are stationary (i.e. first and second-order moments) for Gaussian processes: WSS implies SSS (hence the importance of the first two moments) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 5/36 Characterization (cont d); ACF of WSS Random Processes Properties R X (τ) =R X ( τ)... even symmetry R X () R X (τ) R X () = E{X (t)}... second moment; power Power Spectral Density PSD (Leistungsdichtespektrum) X(t)... power signal, if stationary PSD is Fourier transform of the ACF Def. : R X (τ) F S X (jω) PSD is power content in frequency domain Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 6/36

14 Power Spectral Density (PSD) (cont d) definition using the Fourier transform of X(t): needs time limitation of the sample functions: X T (A i,jω)= T/ T/ x(a i,t)e jωt dt Wiener-Kinchin theorem S X (jω) = lim = T Proof at blackboard! T E A i { X T (A i,jω} } R X (τ)e jωτ dτ = F[R X (τ)] Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 7/36 relizations till 5 of an ensemble of trials FT x (t) FT x (t) FT x 3 (t) FT x 4 (t) FT x 5 (t) E FT (t) sample number, time Fundamentals.of Digital.4Commun.Ch..6 4:.8 Random Variables. and Random.4Processes.6 p..8 8/36. normalized frequency

15 relizations till 5 of an ensemble of trials 4 Mean value; first moment estimate of E x(t) Second moment (= autocorrelation) estimate of E x(5) x(t) sample number, time t sample number, time Power Spectrum Magnitude (db) Fundamentals.of Digital.4Commun.Ch..6 4:.8 Random Variables. and Random.4 Processes.6 p..8 9/36. Frequency relizations till 5 of an ensemble of trials 5 GRAZ UNIVERSITY Mean value; OFfirstTECHNOLOGY moment estimate of E x(t) estimate of E x(5) x(t) Second moment (= autocorrelation) sample number, time t sample number, time Power Spectrum Magnitude (db) Frequency Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 3/36

16 Characterization (cont d); Ergodic Random Proc. Definition: (all) time and ensemble averages are identical necessary condition: stationarity (otherwise, ensemble average would be time-variant) example: ensemble average and DC-value it must hold: x DC m X, where x DC = x i (t) = lim T m X =E{X(t)} = T T/ T/ x i (t) dt xf X(t) (x) dx = lim n n n x i (t) i= Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 3/36 Characterization (cont d); Ergodic Random Proc. equivalence must hold for all averages hard to proof Example: ACF R X (τ) =E{X (t)x(t + τ)} x i (t)x i (t + τ) consequence of ergodicity: ensemble averages (moments) can be estimated from a single sample function! Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 3/36

17 4-4 Noise Example for a random process Physics: freely moving electrons e PSD, ACF S X (jω)=s X (f) = N N /... double-sided noise PSD thermal noise N = kt F R X (τ) = N δ(τ) PDF: Gaussian PDF = normal distribution Power: P N = N B = σ ; B... equivalent bandwidth f X(t) (x) = ( ) x exp πσ σ Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 33/36 Why is noise Gaussian distributed? Central Limit Theorem (Zentraler Grenzwertsatz): a sum of (many) random processes has Gaussian PDF illustration: PDF of one dice: uniform (discrete) two dice: triangular PDF many dice: PDF approaches Gaussian PDF generally (without proof): PDF of the sum of two (independent) random variables corresponds to the convolution of their PDFs Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 34/36

18 . Ein Wuerfel Wahrscheinlichk Zwei Wuerfel Wahrscheinlichk Wahrscheinlichk Drei Wuerfel Normalverteilung 5 5 Augenzahl (discrete PDFs) Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 35/36 Simulation: Ein Wuerfel, Wuerfe x4 Simulation: Ein Wuerfel, Wuerfe Ereignisse Simulation: Zwei Wuerfel, Wuerfe 5 5 x4 Simulation: Zwei Wuerfel, Wuerfe Ereignisse Simulation: Drei Wuerfel, Wuerfe 5 5 x4 Simulation: Drei Wuerfel, Wuerfe Ereignisse Augenzahl 5 5 Augenzahl Fundamentals of Digital Commun.Ch. 4: Random Variables and Random Processes p. 36/36

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process

ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random

More information

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno

Stochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.

More information

16.584: Random (Stochastic) Processes

16.584: Random (Stochastic) Processes 1 16.584: Random (Stochastic) Processes X(t): X : RV : Continuous function of the independent variable t (time, space etc.) Random process : Collection of X(t, ζ) : Indexed on another independent variable

More information

Stochastic Processes

Stochastic Processes Elements of Lecture II Hamid R. Rabiee with thanks to Ali Jalali Overview Reading Assignment Chapter 9 of textbook Further Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A First Course in Stochastic

More information

3F1 Random Processes Examples Paper (for all 6 lectures)

3F1 Random Processes Examples Paper (for all 6 lectures) 3F Random Processes Examples Paper (for all 6 lectures). Three factories make the same electrical component. Factory A supplies half of the total number of components to the central depot, while factories

More information

conditional cdf, conditional pdf, total probability theorem?

conditional cdf, conditional pdf, total probability theorem? 6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random

More information

EAS 305 Random Processes Viewgraph 1 of 10. Random Processes

EAS 305 Random Processes Viewgraph 1 of 10. Random Processes EAS 305 Random Processes Viewgraph 1 of 10 Definitions: Random Processes A random process is a family of random variables indexed by a parameter t T, where T is called the index set λ i Experiment outcome

More information

Continuous Random Variables

Continuous Random Variables 1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables

More information

for valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I

for valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I Code: 15A04304 R15 B.Tech II Year I Semester (R15) Regular Examinations November/December 016 PROBABILITY THEY & STOCHASTIC PROCESSES (Electronics and Communication Engineering) Time: 3 hours Max. Marks:

More information

Stochastic Processes. A stochastic process is a function of two variables:

Stochastic Processes. A stochastic process is a function of two variables: Stochastic Processes Stochastic: from Greek stochastikos, proceeding by guesswork, literally, skillful in aiming. A stochastic process is simply a collection of random variables labelled by some parameter:

More information

E X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.

E X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl. E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,

More information

3 Operations on One Random Variable - Expectation

3 Operations on One Random Variable - Expectation 3 Operations on One Random Variable - Expectation 3.0 INTRODUCTION operations on a random variable Most of these operations are based on a single concept expectation. Even a probability of an event can

More information

Fundamentals of Noise

Fundamentals of Noise Fundamentals of Noise V.Vasudevan, Department of Electrical Engineering, Indian Institute of Technology Madras Noise in resistors Random voltage fluctuations across a resistor Mean square value in a frequency

More information

where r n = dn+1 x(t)

where r n = dn+1 x(t) Random Variables Overview Probability Random variables Transforms of pdfs Moments and cumulants Useful distributions Random vectors Linear transformations of random vectors The multivariate normal distribution

More information

Lecture 2: Repetition of probability theory and statistics

Lecture 2: Repetition of probability theory and statistics Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:

More information

Statistical signal processing

Statistical signal processing Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable

More information

PROBABILITY AND RANDOM PROCESSESS

PROBABILITY AND RANDOM PROCESSESS PROBABILITY AND RANDOM PROCESSESS SOLUTIONS TO UNIVERSITY QUESTION PAPER YEAR : JUNE 2014 CODE NO : 6074 /M PREPARED BY: D.B.V.RAVISANKAR ASSOCIATE PROFESSOR IT DEPARTMENT MVSR ENGINEERING COLLEGE, NADERGUL

More information

Deep Learning for Computer Vision

Deep Learning for Computer Vision Deep Learning for Computer Vision Lecture 3: Probability, Bayes Theorem, and Bayes Classification Peter Belhumeur Computer Science Columbia University Probability Should you play this game? Game: A fair

More information

1 Random Variable: Topics

1 Random Variable: Topics Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?

More information

Chapter 6. Random Processes

Chapter 6. Random Processes Chapter 6 Random Processes Random Process A random process is a time-varying function that assigns the outcome of a random experiment to each time instant: X(t). For a fixed (sample path): a random process

More information

Algorithms for Uncertainty Quantification

Algorithms for Uncertainty Quantification Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example

More information

Random Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay

Random Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay 1 / 13 Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 8, 2013 2 / 13 Random Variable Definition A real-valued

More information

ENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University

ENSC327 Communications Systems 19: Random Processes. Jie Liang School of Engineering Science Simon Fraser University ENSC327 Communications Systems 19: Random Processes Jie Liang School of Engineering Science Simon Fraser University 1 Outline Random processes Stationary random processes Autocorrelation of random processes

More information

Review of Probability Theory

Review of Probability Theory Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving

More information

Deterministic. Deterministic data are those can be described by an explicit mathematical relationship

Deterministic. Deterministic data are those can be described by an explicit mathematical relationship Random data Deterministic Deterministic data are those can be described by an explicit mathematical relationship Deterministic x(t) =X cos r! k m t Non deterministic There is no way to predict an exact

More information

Introduction to Probability and Stochastic Processes I

Introduction to Probability and Stochastic Processes I Introduction to Probability and Stochastic Processes I Lecture 3 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark Slides

More information

BASICS OF PROBABILITY

BASICS OF PROBABILITY October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,

More information

Formulas for probability theory and linear models SF2941

Formulas for probability theory and linear models SF2941 Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms

More information

Random Processes Why we Care

Random Processes Why we Care Random Processes Why we Care I Random processes describe signals that change randomly over time. I Compare: deterministic signals can be described by a mathematical expression that describes the signal

More information

Lecture Notes 7 Stationary Random Processes. Strict-Sense and Wide-Sense Stationarity. Autocorrelation Function of a Stationary Process

Lecture Notes 7 Stationary Random Processes. Strict-Sense and Wide-Sense Stationarity. Autocorrelation Function of a Stationary Process Lecture Notes 7 Stationary Random Processes Strict-Sense and Wide-Sense Stationarity Autocorrelation Function of a Stationary Process Power Spectral Density Continuity and Integration of Random Processes

More information

Properties of the Autocorrelation Function

Properties of the Autocorrelation Function Properties of the Autocorrelation Function I The autocorrelation function of a (real-valued) random process satisfies the following properties: 1. R X (t, t) 0 2. R X (t, u) =R X (u, t) (symmetry) 3. R

More information

Quick Tour of Basic Probability Theory and Linear Algebra

Quick Tour of Basic Probability Theory and Linear Algebra Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions

More information

Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.

Perhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows. Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage

More information

EE4601 Communication Systems

EE4601 Communication Systems EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two

More information

A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011

A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 Reading Chapter 5 (continued) Lecture 8 Key points in probability CLT CLT examples Prior vs Likelihood Box & Tiao

More information

2 (Statistics) Random variables

2 (Statistics) Random variables 2 (Statistics) Random variables References: DeGroot and Schervish, chapters 3, 4 and 5; Stirzaker, chapters 4, 5 and 6 We will now study the main tools use for modeling experiments with unknown outcomes

More information

Signals and Spectra - Review

Signals and Spectra - Review Signals and Spectra - Review SIGNALS DETERMINISTIC No uncertainty w.r.t. the value of a signal at any time Modeled by mathematical epressions RANDOM some degree of uncertainty before the signal occurs

More information

Probability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27

Probability Review. Yutian Li. January 18, Stanford University. Yutian Li (Stanford University) Probability Review January 18, / 27 Probability Review Yutian Li Stanford University January 18, 2018 Yutian Li (Stanford University) Probability Review January 18, 2018 1 / 27 Outline 1 Elements of probability 2 Random variables 3 Multiple

More information

Chapter 5 Random Variables and Processes

Chapter 5 Random Variables and Processes Chapter 5 Random Variables and Processes Wireless Information Transmission System Lab. Institute of Communications Engineering National Sun Yat-sen University Table of Contents 5.1 Introduction 5. Probability

More information

Random Variables. P(x) = P[X(e)] = P(e). (1)

Random Variables. P(x) = P[X(e)] = P(e). (1) Random Variables Random variable (discrete or continuous) is used to derive the output statistical properties of a system whose input is a random variable or random in nature. Definition Consider an experiment

More information

Probability, CLT, CLT counterexamples, Bayes. The PDF file of this lecture contains a full reference document on probability and random variables.

Probability, CLT, CLT counterexamples, Bayes. The PDF file of this lecture contains a full reference document on probability and random variables. Lecture 5 A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2015 http://www.astro.cornell.edu/~cordes/a6523 Probability, CLT, CLT counterexamples, Bayes The PDF file of

More information

Chapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University

Chapter 3, 4 Random Variables ENCS Probability and Stochastic Processes. Concordia University Chapter 3, 4 Random Variables ENCS6161 - Probability and Stochastic Processes Concordia University ENCS6161 p.1/47 The Notion of a Random Variable A random variable X is a function that assigns a real

More information

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are

More information

Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016

Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying

More information

Recitation 2: Probability

Recitation 2: Probability Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions

More information

Random Variables and Their Distributions

Random Variables and Their Distributions Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital

More information

SRI VIDYA COLLEGE OF ENGINEERING AND TECHNOLOGY UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS

SRI VIDYA COLLEGE OF ENGINEERING AND TECHNOLOGY UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS 1. Define random process? The sample space composed of functions of time is called a random process. 2. Define Stationary process? If a random process is divided

More information

L2: Review of probability and statistics

L2: Review of probability and statistics Probability L2: Review of probability and statistics Definition of probability Axioms and properties Conditional probability Bayes theorem Random variables Definition of a random variable Cumulative distribution

More information

3. Probability and Statistics

3. Probability and Statistics FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important

More information

Stochastic Processes. Chapter Definitions

Stochastic Processes. Chapter Definitions Chapter 4 Stochastic Processes Clearly data assimilation schemes such as Optimal Interpolation are crucially dependent on the estimates of background and observation error statistics. Yet, we don t know

More information

ELEMENTS OF PROBABILITY THEORY

ELEMENTS OF PROBABILITY THEORY ELEMENTS OF PROBABILITY THEORY Elements of Probability Theory A collection of subsets of a set Ω is called a σ algebra if it contains Ω and is closed under the operations of taking complements and countable

More information

Module 9: Stationary Processes

Module 9: Stationary Processes Module 9: Stationary Processes Lecture 1 Stationary Processes 1 Introduction A stationary process is a stochastic process whose joint probability distribution does not change when shifted in time or space.

More information

Chapter 4. Chapter 4 sections

Chapter 4. Chapter 4 sections Chapter 4 sections 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP: 4.8 Utility Expectation

More information

ECE353: Probability and Random Processes. Lecture 7 -Continuous Random Variable

ECE353: Probability and Random Processes. Lecture 7 -Continuous Random Variable ECE353: Probability and Random Processes Lecture 7 -Continuous Random Variable Xiao Fu School of Electrical Engineering and Computer Science Oregon State University E-mail: xiao.fu@oregonstate.edu Continuous

More information

Introduction to Probability and Stocastic Processes - Part I

Introduction to Probability and Stocastic Processes - Part I Introduction to Probability and Stocastic Processes - Part I Lecture 1 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark

More information

Lecture 25: Review. Statistics 104. April 23, Colin Rundel

Lecture 25: Review. Statistics 104. April 23, Colin Rundel Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April

More information

Multivariate Random Variable

Multivariate Random Variable Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate

More information

Northwestern University Department of Electrical Engineering and Computer Science

Northwestern University Department of Electrical Engineering and Computer Science Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability

More information

ECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else

ECE 450 Homework #3. 1. Given the joint density function f XY (x,y) = 0.5 1<x<2, 2<y< <x<4, 2<y<3 0 else ECE 450 Homework #3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3 4 5

More information

Chapter 3: Random Variables 1

Chapter 3: Random Variables 1 Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.

More information

Definition of a Stochastic Process

Definition of a Stochastic Process Definition of a Stochastic Process Balu Santhanam Dept. of E.C.E., University of New Mexico Fax: 505 277 8298 bsanthan@unm.edu August 26, 2018 Balu Santhanam (UNM) August 26, 2018 1 / 20 Overview 1 Stochastic

More information

Signals and Spectra (1A) Young Won Lim 11/26/12

Signals and Spectra (1A) Young Won Lim 11/26/12 Signals and Spectra (A) Copyright (c) 202 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version.2 or any later

More information

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample

More information

TSKS01 Digital Communication Lecture 1

TSKS01 Digital Communication Lecture 1 TSKS01 Digital Communication Lecture 1 Introduction, Repetition, and Noise Modeling Emil Björnson Department of Electrical Engineering (ISY) Division of Communication Systems Emil Björnson Course Director

More information

Spectral Analysis of Random Processes

Spectral Analysis of Random Processes Spectral Analysis of Random Processes Spectral Analysis of Random Processes Generally, all properties of a random process should be defined by averaging over the ensemble of realizations. Generally, all

More information

Math-Stat-491-Fall2014-Notes-I

Math-Stat-491-Fall2014-Notes-I Math-Stat-491-Fall2014-Notes-I Hariharan Narayanan October 2, 2014 1 Introduction This writeup is intended to supplement material in the prescribed texts: Introduction to Probability Models, 10th Edition,

More information

5 Operations on Multiple Random Variables

5 Operations on Multiple Random Variables EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y

More information

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions

More information

Multiple Random Variables

Multiple Random Variables Multiple Random Variables Joint Probability Density Let X and Y be two random variables. Their joint distribution function is F ( XY x, y) P X x Y y. F XY ( ) 1, < x

More information

ECE 636: Systems identification

ECE 636: Systems identification ECE 636: Systems identification Lectures 3 4 Random variables/signals (continued) Random/stochastic vectors Random signals and linear systems Random signals in the frequency domain υ ε x S z + y Experimental

More information

1.1 Review of Probability Theory

1.1 Review of Probability Theory 1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,

More information

Lecture 11. Probability Theory: an Overveiw

Lecture 11. Probability Theory: an Overveiw Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the

More information

Review: mostly probability and some statistics

Review: mostly probability and some statistics Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random

More information

Estimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators

Estimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators Estimation theory Parametric estimation Properties of estimators Minimum variance estimator Cramer-Rao bound Maximum likelihood estimators Confidence intervals Bayesian estimation 1 Random Variables Let

More information

ENGG2430A-Homework 2

ENGG2430A-Homework 2 ENGG3A-Homework Due on Feb 9th,. Independence vs correlation a For each of the following cases, compute the marginal pmfs from the joint pmfs. Explain whether the random variables X and Y are independent,

More information

Bivariate Distributions

Bivariate Distributions STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 17 Néhémy Lim Bivariate Distributions 1 Distributions of Two Random Variables Definition 1.1. Let X and Y be two rrvs on probability space (Ω, A, P).

More information

1 Joint and marginal distributions

1 Joint and marginal distributions DECEMBER 7, 204 LECTURE 2 JOINT (BIVARIATE) DISTRIBUTIONS, MARGINAL DISTRIBUTIONS, INDEPENDENCE So far we have considered one random variable at a time. However, in economics we are typically interested

More information

Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides

Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering Stochastic Processes and Linear Algebra Recap Slides Stochastic processes and variables XX tt 0 = XX xx nn (tt) xx 2 (tt) XX tt XX

More information

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline. Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example

More information

1 Presessional Probability

1 Presessional Probability 1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional

More information

A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011

A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 Reading Chapter 5 of Gregory (Frequentist Statistical Inference) Lecture 7 Examples of FT applications Simulating

More information

Chapter 3: Random Variables 1

Chapter 3: Random Variables 1 Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.

More information

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables

Why study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section

More information

SDS 321: Introduction to Probability and Statistics

SDS 321: Introduction to Probability and Statistics SDS 321: Introduction to Probability and Statistics Lecture 14: Continuous random variables Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/

More information

EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as

EEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace

More information

Fig 1: Stationary and Non Stationary Time Series

Fig 1: Stationary and Non Stationary Time Series Module 23 Independence and Stationarity Objective: To introduce the concepts of Statistical Independence, Stationarity and its types w.r.to random processes. This module also presents the concept of Ergodicity.

More information

Probability and Statistics for Final Year Engineering Students

Probability and Statistics for Final Year Engineering Students Probability and Statistics for Final Year Engineering Students By Yoni Nazarathy, Last Updated: May 24, 2011. Lecture 6p: Spectral Density, Passing Random Processes through LTI Systems, Filtering Terms

More information

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets

More information

Set Theory Digression

Set Theory Digression 1 Introduction to Probability 1.1 Basic Rules of Probability Set Theory Digression A set is defined as any collection of objects, which are called points or elements. The biggest possible collection of

More information

SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416)

SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416) SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416) D. ARAPURA This is a summary of the essential material covered so far. The final will be cumulative. I ve also included some review problems

More information

ECE Homework Set 3

ECE Homework Set 3 ECE 450 1 Homework Set 3 0. Consider the random variables X and Y, whose values are a function of the number showing when a single die is tossed, as show below: Exp. Outcome 1 3 4 5 6 X 3 3 4 4 Y 0 1 3

More information

Introduction to Probability and Stocastic Processes - Part I

Introduction to Probability and Stocastic Processes - Part I Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark

More information

The Multivariate Normal Distribution. In this case according to our theorem

The Multivariate Normal Distribution. In this case according to our theorem The Multivariate Normal Distribution Defn: Z R 1 N(0, 1) iff f Z (z) = 1 2π e z2 /2. Defn: Z R p MV N p (0, I) if and only if Z = (Z 1,..., Z p ) T with the Z i independent and each Z i N(0, 1). In this

More information

Chap 2.1 : Random Variables

Chap 2.1 : Random Variables Chap 2.1 : Random Variables Let Ω be sample space of a probability model, and X a function that maps every ξ Ω, toa unique point x R, the set of real numbers. Since the outcome ξ is not certain, so is

More information

Probability theory. References:

Probability theory. References: Reasoning Under Uncertainty References: Probability theory Mathematical methods in artificial intelligence, Bender, Chapter 7. Expert systems: Principles and programming, g, Giarratano and Riley, pag.

More information

Stochastic Processes: I. consider bowl of worms model for oscilloscope experiment:

Stochastic Processes: I. consider bowl of worms model for oscilloscope experiment: Stochastic Processes: I consider bowl of worms model for oscilloscope experiment: SAPAscope 2.0 / 0 1 RESET SAPA2e 22, 23 II 1 stochastic process is: Stochastic Processes: II informally: bowl + drawing

More information

Chapter 7. Basic Probability Theory

Chapter 7. Basic Probability Theory Chapter 7. Basic Probability Theory I-Liang Chern October 20, 2016 1 / 49 What s kind of matrices satisfying RIP Random matrices with iid Gaussian entries iid Bernoulli entries (+/ 1) iid subgaussian entries

More information

STAT 430/510: Lecture 16

STAT 430/510: Lecture 16 STAT 430/510: Lecture 16 James Piette June 24, 2010 Updates HW4 is up on my website. It is due next Mon. (June 28th). Starting today back at section 6.7 and will begin Ch. 7. Joint Distribution of Functions

More information

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables. Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel

More information

Probability Background

Probability Background Probability Background Namrata Vaswani, Iowa State University August 24, 2015 Probability recap 1: EE 322 notes Quick test of concepts: Given random variables X 1, X 2,... X n. Compute the PDF of the second

More information