SIMON FRASER UNIVERSITY School of Engineering Science

Transcription

1 SIMON FRASER UNIVERSITY School of Engineering Science Course Outline ENSC Digital Signal Processing Calendar Description This course covers advanced digital signal processing techniques. The main topics are as follows: transform representations of signals: fast transforms (FFT, DCT); signal processing of band-pass signals and the Hilbert transform; random signals; the response of LTI systems to random signals; quantization noise in DSP; power spectrum estimation; an introduction to adaptive filters; linear prediction in DSP; and an introduction to hardware implementations of DSP algorithms. Prerequisite: ENSC 802 and a previous course in DSP at the undergraduate level. 1. OVERVIEW 1.1 Entrance Requirements This is a graduate course, so you should have completed at least one digital signal processing course at the undergraduate level. In addition, the course makes extensive use of the theory of random processis for modelling signals, so you need a solid background in this area. Hence ENSC 802 or its equivalent is a prerequisite. 1.2 Course Objective In a wide variety of engineering areas, we must design processing methods when the statistics of the signals are either unknown or time-varying. Because of the importance of this problem, we will be highlighting "power spectrum estimation; an introduction to adaptive filters; linear prediction in DSP " and related areas. Essentially, the course is linked to statistical signal processing; that is, estimation and adaptive filtering. Several approaches, each with suites of algorithms, have become widely used in the past decade or two. The objective in this course is to give you a basic understanding of the methods, so they become part of your "kit" for research or product development. You should also gain the understanding required to master more sophisticated techniques for advanced work. 1

2 1.3 Text and References The text is Simon Haykin, Adaptive Filter Theory, Third Edition, Prentice-Hall, It covers the topics of this course, and supplies instructional material for advanced readings. Other good references: Louis L. Scharf, Statistical Signal Processing: Detection, Estimation and Time Series Analysis, Addison-Wesley, John G. Proakis, Digital Communications, Third Edition, McGraw-Hill, 1995 The lectures will be linked closely to the text, and will provide interpretation, perspective and examples. 2. TOPICS This is the intended list of topics, with number of weeks indicated. There may be variations from the plan, depending on the class preparedness and the time available. Examples will be drawn largely from the field of telecommunications. 1. Background and Review (1 1/2) Complex Gaussian vectors; 1- and 2-sided z-transforms, causality and stablility; characterization of discrete-time random processes by second order statistics, AR, MA and ARMA models; orthogonalization by Karhunen-Loeve and Gram-Schmidt. 2. Basic Estimation Theory (1 1/2) Bayes parameter estimation, minimum cost, MAP and maximum likelihood estimates; linear estimation, geometric view, Gram matrix, normal equations, projections; Cramér- Rao lower bound. 3. Optimum Linear Filters (2 1/2) Types of filters and optimality criteria; orthogonality principle; Wiener filters; constrained optimization; prediction: forward and backward forms, Levinson-Durbin equations, AR modeling, lattice filters, Cholesky decomposition. 4. Least Squares Estimation (2) 2

3 Estimation of parameter vectors by least squares; pseudo-inverse; projections; singular value decomposition; reduced rank estimation. 5. Adaptation by Least Mean Squares (1 1/2) Problem formulation; gradient and stochastic gradient; convergence and stability; excess mean squared error. 6. Adaptation by Recursive Least Squares (2) Problem formulation; matrix inversion lemma; recursive solution; convergence. 7. Kalman Filters (2) State space model; innovations process; recursive estimation of process state; filtering. 3. ASSESSMENT AND HOUSE-KEEPING There will be four assignments, collectively worth half of your mark. The other half will come from a final examination. Your instructor's contact points: tel: fax: cavers@sfu.ca 4. REALLY DETAILED COURSE OUTLINE 0. What is This Course About? 1. Background and Review 1.1 One- and Two-Sided z Transforms 1.2 Random Processes - Time Domain Characterization Justification for Complex Signals Means, Variances, Etc Random Vectors: pdfs, Characteristic Functions Random Processes Models of Stochastic Processes 1.3 Decorrelating Sets of Random Variables 1.4 Miscellaneous Covariance and Correlation Matrix Properties 1.5 Signal Spaces 3

4 2. Basic Estimation Theory 2.1 A Few Examples 2.2 A Model of What We re Doing 2.3 Properties of Estimators 2.4 Bayes Parameter Estimation 2.5 Linear MMSE Estimation Start With Scalars Multidimensional x Multidimensional θ and x Example: Pilot Symbol Assisted Modulation Partial Correlation General View of Linear Estimation 2.6 Maximum Likelihood Estimation The Maximum Likelihood Principle Properties of Maximum Likelihood Estimators Special Case: Least Squares 2.7 Estimation Error Variance and the Cramer-Rao Bound 2.8 Summary of Section 3. Optimum Linear Filters 3.1 General Model 3.2 Special Forms of Equations 3.3 Example: Source Estimation 3.4 Examples Equalizer Generalization of the Equalizer Improving SNR - Continuous Time, Infinite Length Maximum SNR - Finite Length 3.5 Linearly Constrained Minimum Variance Filter Deterministic Example Back to Stochastic 3.6 Summary 4. Linear Prediction 4.1 What s the Fuss About? 4.2 Wiener-Hopf Equations for Predictors 4.3 Levinson s Recursion 4.4 Orthogonality, Gram-Schmidt and Cholesky 4.5 Lattice Filters 4.6 Joint Process Estimation 4.7 Additional Properties of Prediction Error Filters 4.8 Block Estimation of Predictor Coefficients 5. Adaptation in the Mean - Steepest Descent 5.1 Why are We Studying This? 4

5 5.2 The Error Surface and Its Gradient 5.3 Steepest Descent: Algorithm and Convergence 5.4 Convergence of the MSE 6. The LMS Algorithm 6.1 About LMS 6.2 The Algorithm 6.3 OK - Let s Use It! 6.4 Convergence and Excess Mean Squared Error 6.5 Beating the Eigenvalue Spread - Decorrelation 6.6 The Gradient Adaptive Lattice 6.7 Summary and Perspective 7. Estimation by Least Squares 7.1 About This Section 7.2 Formulation of the LS Problem 7.3 Projections 7.4 Properties of LS Estimates 7.5 Using LS for Spectrum Estimation 7.6 How Many Solutions? 7.7 Singular Value Decomposition 7.8 The Pseudoinverse 7.9 Optimal Rank Reduction 7.10 Numerical Accuracy and SVD 8. Recursive Least Squares 8.1 Why We Are Interested in RLS 8.2 A Basic Recursive Algorithm 8.3 Matrix Inversion Lemma 8.4 Recursive Least Squares 8.5 Recursion for the Sum of Squared Errors 8.6 Convergence Behaviour of RLS 9. Kalman Filtering 9.1 Some Perspective on Filtering and Estimating 9.2 Recursive MMSE Estimation and Innovations - Scalar Case 9.3 The State Space Model and the Kalman Problem 9.4 Toolkit and Strategy 9.5 Vector Innovations and State Estimation 9.6 The Update Stage 9.7 The Extrapolation Stage 9.8 The Gain and Covariance Calculation 9.9 Putting It All Together 9.10 Kalman Roots of RLS 5