ECE 3800 Probabilistic Methods of Signal and System Analysis

Transcription

1 ECE 3800 Probabilistic Methods of Signal and System Analysis Dr. Bradley J. Bazuin Western Michigan University College of Engineering and Applied Sciences Department of Electrical and Computer Engineering 1903 W. Michigan Ave. Kalamazoo MI,

2 Course/Lecture Overview Syllabus Personal Intro. Textbook/Materials Used Additional Reading ID and Acknowledgment of Policies Chapter 1 ECE

3 Syllabus Everything useful for this class can be found on Dr. Bazuin s web site! The class web site is at The syllabus ECE

4 Dr. Bradley J. Bazuin Born and raised in Michigan, Grand Rapids Forest Hills Northern Education Undergraduate BS in Engineering and Applied Sciences, Extensive Electrical Engineering from Yale University in 1980 Graduate MS and PhD in Electrical Engineering from Stanford University in 1982 and 1989, respectively. Industrial Employment Part-time ARGOSystems, Inc., Sunnyvale, CA, Full-time ARGOSystems, Inc., Sunnyvale, CA, Full-time Radix Technologies, Mountain View, CA, Academics Term-appointed Faculty, WMU ECE Dept Tenure track Assistant Professor, WMU ECE Dept Interim Department Chair, WMU ECE 2017 Tenured Associate Professor, WMU ECE Dept Department Chair, WMU ECE ECE

5 Research and Technical Interests Wireless Communications Physical Layer signal and system implementation Software Defined Radios (SDR) - USRP & GNU radio Xilinx with VHDL coding and Graphic processing units (GPU) Advanced Digital Signal Processing Algorithmic techniques for processing detecting, estimating and exploiting signals (communications, electronics, and sensors). Multirate signal processing, estimation theory, adaptive signal processing CAPE & CASSS Center for the Advancement of Printed Electronics Center for Advanced Smart Sensors and Structures Sunseeker Solar Team Adviser & WMU Educational Solar Garden Technical Director Embedded processing systems (TI MSP430 based) Embedded software (control, monitoring, safety, telemetry) Energy conversion (solar cells, batteries, super capacitors) Collaborative Engineering Supporting other WMU research activities where I can contribute ECE

6 Textbook/Materials Required Textbook Charles Boncelet, Probability, Statistics, and Random Signals," Oxford University Press, 2016, ISBN: Recommended Material The MATH Works, MATLAB Student Version ($99) or CAE Center Learn MATLAB for free ECE

7 Other Books and Materials H. Stark and J.W. Woods, Probability, Statistics and Random Processes for Engineers, 4th ed.," Prentice-Hall, Inc., Upper Saddle River, NJ, 2012, ISBN: Previous text used for ECE 3800 George R. Cooper and Clare D. McGillem, Probabilistic Methods of Signal and System Analysis, 3rd ed., Oxford University Press Inc., ISBN: Previous text used for ECE 3800 Alberto Leon-Garcia, Probability, Statistics, and Random Processes For Electrical Engineering, 3rd ed., Pearson Prentice Hall, Upper Saddle River, NJ, 2008, ISBN: Graduate text used for ECE 5820 A. Papoulis, "Probability, Random Variables, and Stochastic Processes," McGraw-Hill, ISBM: My course textbook in graduate school ECE

8 Identification and Acknowledgement Identification for Grade Posting, Acknowledgment of completing prerequisites, Reminder of Course and University Policies, and Acknowledgement and Signature Block Please read, provide unique identification, sign and date, and return to Dr. Bazuin. ECE

9 Course/Text Overview 1 PROBABILITY BASICS 1.1 What Is Probability? 1.2 Experiments, Outcomes, and Events 1.3 Venn Diagrams 1.4 Random Variables 1.5 Basic Probability Rules 1.6 Probability Formalized 1.7 Simple Theorems 1.8 Compound Experiments 1.9 Independence 1.10 Example: Can S Communicate With D? 1.11 Example: Now Can S Communicate With D? 1.12 Computational Procedures 2 CONDITIONAL PROBABILITY 2.1 Definitions of Conditional Probability 2.2 Law of Total Probability and Bayes Theorem 2.3 Example: Urn Models 2.4 Example: A Binary Channel 2.5 Example: Drug Testing 2.6 Example: A Diamond Network 3 A LITTLE COMBINATORICS 3.1 Basics of Counting 3.2 Notes on Computation 3.3 Combinations and the Binomial Coefficients 3.4 The Binomial Theorem 3.5 Multinomial Coefficient and Theorem 3.6 The Birthday Paradox and Message Authentication 3.7 Hypergeometric Probabilities and Card Games 4 DISCRETE PROBABILITIES AND RANDOM VARIABLES 4.1 Discrete Random Variable and Probability Mass Functions 4.2 Cumulative Distribution Functions 4.3 Expected Values 4.4 Moment Generating Functions 4.5 Several Important Discrete PMFs 4.6 Gambling and Financial Decision Making Tentative Schedule for Exam #1 ECE 3800 Based on materials in the course textbook: Charles Boncelet, Probability, Statistics, and Random Signals," Oxford University Press,

10 Course/Text Overview (2) 5 MULTIPLE DISCRETE RANDOM VARIABLES 5.1 Multiple Random Variables and PMFs 5.2 Independence 5.3 Moments and Expected Values 5.4 Example: Two Discrete Random Variables 5.5 Sums of Independent Random Variables 5.6 Sample Probabilities, Mean, and Variance 5.7 Histograms 5.8 Entropy and Data Compression 6 BINOMIAL PROBABILITIES 6.1 Basics of the Binomial Distribution 6.2 Computing Binomial Probabilities 6.3 Moments of the Binomial Distribution 6.4 Sums of Independent Binomial Random Variables 6.5 Distributions Related to the Binomial 6.6 Parameter Estimation for Binomial and Multinomial Distributions 6.7 Alohanet 6.8 Error Control Codes 7 A CONTINUOUS RANDOM VARIABLE 7.1 A Continuous Random Variable and Its Density, Distribution Function, and Expected Values 7.2 Example Calculations for a Single Random Variable 7.3 Selected Continuous Distributions 7.4 Conditional Probabilities for a Continuous Random Variable 7.5 Discrete PMFs and Delta Functions 7.6 Quantization 7.7 A Final Word 8 MULTIPLE CONTINUOUS RANDOM VARIABLES 8.1 Joint Densities and Distribution Functions 8.2 Expected Values and Moments 8.3 Independence 8.4 Conditional Probabilities for Multiple Random Variables 8.5 Extended Example: Two Continuous Random Variables 8.6 Sums of Independent Random Variables 8.7 Random Sums 8.8 General Transformations and the Jacobian 8.9 Parameter Estimation for the Exponential Distribution 8.10 Comparison of Discrete and Continuous Distributions Tentative Schedule for Exam #2 ECE 3800 Based on materials in the course textbook: Charles Boncelet, Probability, Statistics, and Random Signals," Oxford University Press,

11 Course/Text Overview (3) 9 THE GAUSSIAN AND RELATED DISTRIBUTIONS 9.1 The Gaussian Distribution and Density 9.2 Quantile Function 9.3 Moments of the Gaussian Distribution 9.4 The Central Limit Theorem 9.5 Related Distributions 9.6 Multiple Gaussian Random Variables 9.7 Example: Digital Communications Using QAM 10 ELEMENTS OF STATISTICS 10.1 A Simple Election Poll 10.2 Estimating the Mean and Variance 10.3 Recursive Calculation of the Sample Mean 10.4 Exponential Weighting 10.5 Order Statistics and Robust Estimates 10.6 Estimating the Distribution Function 10.7 PMF and Density Estimates 10.8 Confidence Intervals 10.9 Significance Tests and p-values Introduction to Estimation Theory Minimum Mean Squared Error Estimation Bayesian Estimation 11 GAUSSIAN RANDOM VECTORS AND LINEAR REGRESSION 11.1 Gaussian Random Vectors 11.2 Linear Operations on Gaussian Random Vectors 11.3 Linear Regression 12 HYPOTHESIS TESTING 12.1 Hypothesis Testing: Basic Principles 12.2 Example: Radar Detection 12.3 Hypothesis Tests and Likelihood Ratios 12.4 MAP Tests Tentative Schedule for Exam #3 ECE 3800 Based on materials in the course textbook: Charles Boncelet, Probability, Statistics, and Random Signals," Oxford University Press,

12 Course/Text Overview (4) 13 RANDOM SIGNALS AND NOISE 13.1 Introduction to Random Signals 13.2 A Simple Random Process 13.3 Fourier Transforms 13.4 WSS Random Processes 13.5 WSS Signals and Linear Filters 13.6 Noise 13.7 Example: Amplitude Modulation 13.8 Example: Discrete Time Wiener Filter 13.9 The Sampling Theorem for WSS Random Processes 14 SELECTED RANDOM PROCESSES 14.1 The Lightbulb Process 14.2 The Poisson Process 14.3 Markov Chains 14.4 Kalman Filter Final Exam ECE 3800 Based on materials in the course textbook: Charles Boncelet, Probability, Statistics, and Random Signals," Oxford University Press,

13 Chapter 1: PROBABILITY BASICS 1.1 What Is Probability? 1.2 Experiments, Outcomes, and Events 1.3 Venn Diagrams 1.4 Random Variables 1.5 Basic Probability Rules 1.6 Probability Formalized 1.7 Simple Theorems 1.8 Compound Experiments 1.9 Independence 1.10 Example: Can S Communicate With D? 1.11 Example: Now Can S Communicate With D? 1.12 Computational Procedures On to the Course Material ECE 3800 Based on materials in the course textbook: Charles Boncelet, Probability, Statistics, and Random Signals," Oxford University Press,