Adaptive Filtering. Squares. Alexander D. Poularikas. Fundamentals of. Least Mean. with MATLABR. University of Alabama, Huntsville, AL.

Size: px
Start display at page:

Download "Adaptive Filtering. Squares. Alexander D. Poularikas. Fundamentals of. Least Mean. with MATLABR. University of Alabama, Huntsville, AL."

Transcription

1 Adaptive Filtering Fundamentals of Least Mean Squares with MATLABR Alexander D. Poularikas University of Alabama, Huntsville, AL CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Croup, an informa business

2 Contents Preface Author Abbreviations MATLAB Functions X1 xiii xv xvii Chapter 1 Vectors Introduction Multiplication by a Constant and Addition and Subtraction Multiplication by a Constant Addition and Subtraction Unit Coordinate Vectors Inner Product Distance between Two Vectors Mean Value of a Vector Direction Cosines The Projection of a Vector Linear Transformations Linear Independence, Vector Spaces, and Basis Vectors Orthogonal Basis Vectors 13 Problems 13 Hints-Suggestions-Solutions 14 Chapter 2 Matrices Introduction General Types of Matrices Diagonal, Identity, and Scalar Matrices Upper and Lower Triangular Matrices Symmetric and Exchange Matrices Toeplitz Matrix Hankel and Hermitian Matrix Operations Determinant of a Matrix Definition and Expansion of a Matrix Trace of a Matrix Inverse of a Matrix Linear Equations Square Matrices (n x ri) Rectangular Matrices (n < m) Rectangular Matrices (m < n) 27

3 yj Contents Quadratic and Hermitian Forms Eigenvalues and Eigenvectors Eigenvectors Properties of Eigenvalues and Eigenvectors 33 Problems 36 Hints-Suggestions-Solutions 37 Chapter 3 Processing of Discrete Deterministic Signals: Discrete Systems Discrete-Time Signals Time-Domain Representation of Basic Continuous and Discrete Signals Transform-Domain Representation of Discrete Signals Discrete-Time Fourier Transform The Discrete FT Properties of DFT The z-transform Discrete-Time Systems Linearity and Shift Invariant Causality Stability Transform-Domain Representation 57 Problems 60 Hints-Suggestions-Solutions 61 Chapter 4 Discrete-Time Random Processes Discrete Random Signals, Probability Distributions, and Averages of Random Variables Stationary and Ergodic Processes Averages of RV Mean Value Correlation Covariance Stationary Processes Autocorrelation Matrix Purely Random Process (White Noise) Random Walk Special Random Signals and pdf's White Noise Gaussian Distribution (Normal Distribution) Exponential Distribution Lognormal Distribution Chi-Square Distribution Wiener-Khinchin Relations Filtering Random Processes 83

4 Contents vii 4.6 Special Types of Random Processes Autoregressive Process Nonparametric Spectra Estimation Periodogram Correlogram Computation of Periodogram and Correlogram Using FFT General Remarks on the Periodogram Windowed Periodogram Proposed Book Modified Method for Better Frequency Resolution Using Transformation of the rv's Blackman-Tukey Method Bartlett Periodogram The Welch Method Proposed Modified Welch Methods Modified Method Using Different Types of Overlapping Modified Welch Method Using Transformation of rv's Ill Problems 113 Hints-Solutions-Suggestions 114 Chapter 5 The Wiener Filter Introduction The LS Technique Linear LS LS Formulation Statistical Properties of LSEs The LS Approach Orthogonality Principle Corollary Projection Operator LS Finite Impulse Response Filter The Mean-Square Error The FIR Wiener Filter The Wiener Solution Orthogonality Condition Normalized Performance Equation Canonical Form of the Error-Performance Surface Wiener Filtering Examples Minimum MSE Optimum Filter (w ) Linear Prediction 161 Problems 162

5 viii Contents Additional Problems 164 Hints-Solutions-Suggestions 164 Additional Problems 166 Chapter 6 Eigenvalues of Rx: Properties of the Error Surface The Eigenvalues of the Correlation Matrix Karhunen-Loeve Transformation Geometrical Properties of the Error Surface 174 Problems 178 Hints-Solutions-Suggestions 178 Chapter 7 Newton's and Steepest Descent Methods One-Dimensional Gradient Search Method Gradient Search Algorithm Newton's Method in Gradient Search Steepest Descent Algorithm Steepest Descent Algorithm Applied to Wiener Filter Stability (Convergence) of the Algorithm Transient Behavior of MSE Learning Curve Newton's Method Solution of the Vector Difference Equation 194 Problems 197 Edition Problems 197 Hints-Solutions-Suggestions 198 Additional Problems 200 Chapter 8 The Least Mean- Square Algorithm Introduction The LMS Algorithm Examples Using the LMS Algorithm Performance Analysis of the LMS Algorithm Learning Curve The Coefficient-Error or Weighted-Error Correlation Matrix Excess MSE and Misadjustment Stability The LMS and Steepest Descent Methods Complex Representation of the LMS Algorithm 228 Problems 231 Hints-Solutions-Suggestions 232 Chapter 9 Variants of Least Mean-Square Algorithm The Normalized Least Mean-Square Algorithm Power Normalized LMS 244

6 Contents ix 9.3 Self-Correcting LMS Filter The Sign-Error LMS Algorithm The NLMS Sign-Error Algorithm The Sign-Regressor LMS Algorithm Self-Correcting Sign-Regressor LMS Algorithm The Normalized Sign-Regressor LMS Algorithm The Sign-Sign LMS Algorithm The Normalized Sign-Sign LMS Algorithm Variable Step-Size LMS The Leaky LMS Algorithm The Linearly Constrained LMS Algorithm The Least Mean Fourth Algorithm The Least Mean Mixed Norm LMS Algorithm Short-Length Signal of the LMS Algorithm The Transform Domain LMS Algorithm Convergence The Error Normalized Step-Size LMS Algorithm The Robust Variable Step-Size LMS Algorithm The Modified LMS Algorithm Momentum LMS The Block LMS Algorithm The Complex LMS Algorithm The Affine LMS Algorithm The Complex Affine LMS Algorithm 290 Problems 291 Hints-Solutions-Suggestions 293 Appendix 1: Suggestions and Explanations for MATLAB Use 301 Al. 1 Suggestions and Explanations for MATLAB Use 301 Al.1.1 Creating a Directory 301 Al.1.2 Help 301 A Save and Load 302 A MATLAB as Calculator 302 A1.1.5 Variable Names 302 Al.1.6 Complex Numbers 302 Al.1.7 Array Indexing 302 A1.1.8 Extracting and Inserting Numbers in Arrays 303 Al.1.9 Vectorization 303 A Windowing 304 A Matrices 304 A Producing a Periodic Function 305 Al.1.13 Script Files 305 A Functions 305 A Complex Expressions 306 A Axes 306 Al D Graphics 306

7 x Contents Al D Plots 308 Al Mesh-Type Figures 308 A1.2 General Purpose Commands 309 A1.2.1 Managing Commands and Functions 309 A Managing Variables and Workplace 309 A Operators and Special Characters 309 A Control Flow 310 A 1.3 Elementary Matrices and Matrix Manipulation 311 Al.3.1 Elementary Matrices and Arrays 311 A1.3.2 Matrix Manipulation 311 A1.4 Elementary Mathematical Functions 312 Al.4.1 Elementary Functions 312 A1.5 Numerical Linear Algebra 313 Al.5.1 Matrix Analysis 313 A1.6 Data Analysis 313 Al.6.1 Basic Operations 313 Al.6.2 Filtering and Convolution 313 Al.6.3 Fourier Transforms 314 A 1.7 2D Plotting 314 Al.7.1 2D Plots 314 Appendix 2: Matrix Analysis 317 A2.1 Definitions 317 A2.2 Special Matrices 319 A2.3 Matrix Operation and Formulas 322 A2.4 Eigendecomposition of Matrices 325 A2.5 Matrix Expectations 326 A2.6 Differentiation of a Scalar Function with respect to a Vector 327 Appendix 3: Mathematical Formulas 329 A3.1 Trigonometric Identities 329 A3.2 Orthogonality 330 A3.3 Summation of Trigonometric Forms 331 A3.4 Summation Formulas 331 A3.4.1 Finite Summation Formulas 331 A3.4.2 Infinite Summation Formulas 331 A3.5 Series Expansions 332 A3.6 Logarithms 332 A3.7 Some Definite Integrals 332 Appendix 4: Lagrange Multiplier Method 335 Bibliography 337 Index 339

Statistical and Adaptive Signal Processing

Statistical and Adaptive Signal Processing r Statistical and Adaptive Signal Processing Spectral Estimation, Signal Modeling, Adaptive Filtering and Array Processing Dimitris G. Manolakis Massachusetts Institute of Technology Lincoln Laboratory

More information

Signals and Systems Laboratory with MATLAB

Signals and Systems Laboratory with MATLAB Signals and Systems Laboratory with MATLAB Alex Palamides Anastasia Veloni @ CRC Press Taylor &. Francis Group Boca Raton London NewYork CRC Press is an imprint of the Taylor & Francis Group, an informa

More information

Advanced Digital Signal Processing -Introduction

Advanced Digital Signal Processing -Introduction Advanced Digital Signal Processing -Introduction LECTURE-2 1 AP9211- ADVANCED DIGITAL SIGNAL PROCESSING UNIT I DISCRETE RANDOM SIGNAL PROCESSING Discrete Random Processes- Ensemble Averages, Stationary

More information

FINITE-DIMENSIONAL LINEAR ALGEBRA

FINITE-DIMENSIONAL LINEAR ALGEBRA DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITE-DIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup

More information

NUMERICAL METHODS. lor CHEMICAL ENGINEERS. Using Excel', VBA, and MATLAB* VICTOR J. LAW. CRC Press. Taylor & Francis Group

NUMERICAL METHODS. lor CHEMICAL ENGINEERS. Using Excel', VBA, and MATLAB* VICTOR J. LAW. CRC Press. Taylor & Francis Group NUMERICAL METHODS lor CHEMICAL ENGINEERS Using Excel', VBA, and MATLAB* VICTOR J. LAW CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Croup,

More information

ADAPTIVE FILTER THEORY

ADAPTIVE FILTER THEORY ADAPTIVE FILTER THEORY Fourth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada Front ice Hall PRENTICE HALL Upper Saddle River, New Jersey 07458 Preface

More information

Linear Algebra and Probability

Linear Algebra and Probability Linear Algebra and Probability for Computer Science Applications Ernest Davis CRC Press Taylor!* Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor Sc Francis Croup, an informa

More information

ADAPTIVE FILTER THEORY

ADAPTIVE FILTER THEORY ADAPTIVE FILTER THEORY Fifth Edition Simon Haykin Communications Research Laboratory McMaster University Hamilton, Ontario, Canada International Edition contributions by Telagarapu Prabhakar Department

More information

Using MATLAB and. Abaqus. Finite Element Analysis. Introduction to. Amar Khennane. Taylor & Francis Croup. Taylor & Francis Croup,

Using MATLAB and. Abaqus. Finite Element Analysis. Introduction to. Amar Khennane. Taylor & Francis Croup. Taylor & Francis Croup, Introduction to Finite Element Analysis Using MATLAB and Abaqus Amar Khennane Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Croup, an informa business

More information

Contents. Acknowledgments

Contents. Acknowledgments Table of Preface Acknowledgments Notation page xii xx xxi 1 Signals and systems 1 1.1 Continuous and discrete signals 1 1.2 Unit step and nascent delta functions 4 1.3 Relationship between complex exponentials

More information

AND NONLINEAR SCIENCE SERIES. Partial Differential. Equations with MATLAB. Matthew P. Coleman. CRC Press J Taylor & Francis Croup

AND NONLINEAR SCIENCE SERIES. Partial Differential. Equations with MATLAB. Matthew P. Coleman. CRC Press J Taylor & Francis Croup CHAPMAN & HALL/CRC APPLIED MATHEMATICS AND NONLINEAR SCIENCE SERIES An Introduction to Partial Differential Equations with MATLAB Second Edition Matthew P Coleman Fairfield University Connecticut, USA»C)

More information

covariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of

covariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of Index* The Statistical Analysis of Time Series by T. W. Anderson Copyright 1971 John Wiley & Sons, Inc. Aliasing, 387-388 Autoregressive {continued) Amplitude, 4, 94 case of first-order, 174 Associated

More information

MATHEMATICAL OBJECTS in

MATHEMATICAL OBJECTS in MATHEMATICAL OBJECTS in Computational Tools in a Unified Object-Oriented Approach Yair Shapira @ CRC Press Taylor & Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor & Francis

More information

Preface to Second Edition... vii. Preface to First Edition...

Preface to Second Edition... vii. Preface to First Edition... Contents Preface to Second Edition..................................... vii Preface to First Edition....................................... ix Part I Linear Algebra 1 Basic Vector/Matrix Structure and

More information

An Introduction to Probability Theory and Its Applications

An Introduction to Probability Theory and Its Applications An Introduction to Probability Theory and Its Applications WILLIAM FELLER (1906-1970) Eugene Higgins Professor of Mathematics Princeton University VOLUME II SECOND EDITION JOHN WILEY & SONS Contents I

More information

Syllabus for IMGS-616 Fourier Methods in Imaging (RIT #11857) Week 1: 8/26, 8/28 Week 2: 9/2, 9/4

Syllabus for IMGS-616 Fourier Methods in Imaging (RIT #11857)  Week 1: 8/26, 8/28 Week 2: 9/2, 9/4 IMGS 616-20141 p.1 Syllabus for IMGS-616 Fourier Methods in Imaging (RIT #11857) 3 July 2014 (TENTATIVE and subject to change) Note that I expect to be in Europe twice during the term: in Paris the week

More information

Introduction to the Mathematics of Medical Imaging

Introduction to the Mathematics of Medical Imaging Introduction to the Mathematics of Medical Imaging Second Edition Charles L. Epstein University of Pennsylvania Philadelphia, Pennsylvania EiaJTL Society for Industrial and Applied Mathematics Philadelphia

More information

Hands-on Matrix Algebra Using R

Hands-on Matrix Algebra Using R Preface vii 1. R Preliminaries 1 1.1 Matrix Defined, Deeper Understanding Using Software.. 1 1.2 Introduction, Why R?.................... 2 1.3 Obtaining R.......................... 4 1.4 Reference Manuals

More information

Numerical Methods for Engineers. and Scientists. Applications using MATLAB. An Introduction with. Vish- Subramaniam. Third Edition. Amos Gilat.

Numerical Methods for Engineers. and Scientists. Applications using MATLAB. An Introduction with. Vish- Subramaniam. Third Edition. Amos Gilat. Numerical Methods for Engineers An Introduction with and Scientists Applications using MATLAB Third Edition Amos Gilat Vish- Subramaniam Department of Mechanical Engineering The Ohio State University Wiley

More information

New Introduction to Multiple Time Series Analysis

New Introduction to Multiple Time Series Analysis Helmut Lütkepohl New Introduction to Multiple Time Series Analysis With 49 Figures and 36 Tables Springer Contents 1 Introduction 1 1.1 Objectives of Analyzing Multiple Time Series 1 1.2 Some Basics 2

More information

BASIC MATRIX ALGEBRA WITH ALGORITHMS AND APPLICATIONS ROBERT A. LIEBLER CHAPMAN & HALL/CRC

BASIC MATRIX ALGEBRA WITH ALGORITHMS AND APPLICATIONS ROBERT A. LIEBLER CHAPMAN & HALL/CRC BASIC MATRIX ALGEBRA WITH ALGORITHMS AND APPLICATIONS ROBERT A. LIEBLER CHAPMAN & HALL/CRC A CRC Press Company Boca Raton London New York Washington, D.C. Contents Preface Examples Major results/proofs

More information

Control Systems. LMIs in. Guang-Ren Duan. Analysis, Design and Applications. Hai-Hua Yu. CRC Press. Taylor & Francis Croup

Control Systems. LMIs in. Guang-Ren Duan. Analysis, Design and Applications. Hai-Hua Yu. CRC Press. Taylor & Francis Croup LMIs in Control Systems Analysis, Design and Applications Guang-Ren Duan Hai-Hua Yu CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Croup, an

More information

CONTENTS NOTATIONAL CONVENTIONS GLOSSARY OF KEY SYMBOLS 1 INTRODUCTION 1

CONTENTS NOTATIONAL CONVENTIONS GLOSSARY OF KEY SYMBOLS 1 INTRODUCTION 1 DIGITAL SPECTRAL ANALYSIS WITH APPLICATIONS S.LAWRENCE MARPLE, JR. SUMMARY This new book provides a broad perspective of spectral estimation techniques and their implementation. It concerned with spectral

More information

Lesson 1. Optimal signalbehandling LTH. September Statistical Digital Signal Processing and Modeling, Hayes, M:

Lesson 1. Optimal signalbehandling LTH. September Statistical Digital Signal Processing and Modeling, Hayes, M: Lesson 1 Optimal Signal Processing Optimal signalbehandling LTH September 2013 Statistical Digital Signal Processing and Modeling, Hayes, M: John Wiley & Sons, 1996. ISBN 0471594318 Nedelko Grbic Mtrl

More information

Introduction. Finite and Spectral Element Methods Using MATLAB. Second Edition. C. Pozrikidis. University of Massachusetts Amherst, USA

Introduction. Finite and Spectral Element Methods Using MATLAB. Second Edition. C. Pozrikidis. University of Massachusetts Amherst, USA Introduction to Finite and Spectral Element Methods Using MATLAB Second Edition C. Pozrikidis University of Massachusetts Amherst, USA (g) CRC Press Taylor & Francis Group Boca Raton London New York CRC

More information

The Fractional Fourier Transform with Applications in Optics and Signal Processing

The Fractional Fourier Transform with Applications in Optics and Signal Processing * The Fractional Fourier Transform with Applications in Optics and Signal Processing Haldun M. Ozaktas Bilkent University, Ankara, Turkey Zeev Zalevsky Tel Aviv University, Tel Aviv, Israel M. Alper Kutay

More information

Mathematics for Economics and Finance

Mathematics for Economics and Finance Mathematics for Economics and Finance Michael Harrison and Patrick Waldron B 375482 Routledge Taylor & Francis Croup LONDON AND NEW YORK Contents List of figures ix List of tables xi Foreword xiii Preface

More information

Math 307 Learning Goals. March 23, 2010

Math 307 Learning Goals. March 23, 2010 Math 307 Learning Goals March 23, 2010 Course Description The course presents core concepts of linear algebra by focusing on applications in Science and Engineering. Examples of applications from recent

More information

SIGNALS AND SYSTEMS I. RAVI KUMAR

SIGNALS AND SYSTEMS I. RAVI KUMAR Signals and Systems SIGNALS AND SYSTEMS I. RAVI KUMAR Head Department of Electronics and Communication Engineering Sree Visvesvaraya Institute of Technology and Science Mahabubnagar, Andhra Pradesh New

More information

MATHEMATICS FOR ECONOMISTS. An Introductory Textbook. Third Edition. Malcolm Pemberton and Nicholas Rau. UNIVERSITY OF TORONTO PRESS Toronto Buffalo

MATHEMATICS FOR ECONOMISTS. An Introductory Textbook. Third Edition. Malcolm Pemberton and Nicholas Rau. UNIVERSITY OF TORONTO PRESS Toronto Buffalo MATHEMATICS FOR ECONOMISTS An Introductory Textbook Third Edition Malcolm Pemberton and Nicholas Rau UNIVERSITY OF TORONTO PRESS Toronto Buffalo Contents Preface Dependence of Chapters Answers and Solutions

More information

SIMON FRASER UNIVERSITY School of Engineering Science

SIMON FRASER UNIVERSITY School of Engineering Science SIMON FRASER UNIVERSITY School of Engineering Science Course Outline ENSC 810-3 Digital Signal Processing Calendar Description This course covers advanced digital signal processing techniques. The main

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Continuous and Discrete Time Signals and Systems

Continuous and Discrete Time Signals and Systems Continuous and Discrete Time Signals and Systems Mrinal Mandal University of Alberta, Edmonton, Canada and Amir Asif York University, Toronto, Canada CAMBRIDGE UNIVERSITY PRESS Contents Preface Parti Introduction

More information

Contents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2

Contents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2 Contents Preface for the Instructor xi Preface for the Student xv Acknowledgments xvii 1 Vector Spaces 1 1.A R n and C n 2 Complex Numbers 2 Lists 5 F n 6 Digression on Fields 10 Exercises 1.A 11 1.B Definition

More information

Quiz ) Locate your 1 st order neighbors. 1) Simplify. Name Hometown. Name Hometown. Name Hometown.

Quiz ) Locate your 1 st order neighbors. 1) Simplify. Name Hometown. Name Hometown. Name Hometown. Quiz 1) Simplify 9999 999 9999 998 9999 998 2) Locate your 1 st order neighbors Name Hometown Me Name Hometown Name Hometown Name Hometown Solving Linear Algebraic Equa3ons Basic Concepts Here only real

More information

Applied Time. Series Analysis. Wayne A. Woodward. Henry L. Gray. Alan C. Elliott. Dallas, Texas, USA

Applied Time. Series Analysis. Wayne A. Woodward. Henry L. Gray. Alan C. Elliott. Dallas, Texas, USA Applied Time Series Analysis Wayne A. Woodward Southern Methodist University Dallas, Texas, USA Henry L. Gray Southern Methodist University Dallas, Texas, USA Alan C. Elliott University of Texas Southwestern

More information

Ch4: Method of Steepest Descent

Ch4: Method of Steepest Descent Ch4: Method of Steepest Descent The method of steepest descent is recursive in the sense that starting from some initial (arbitrary) value for the tap-weight vector, it improves with the increased number

More information

3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE

3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE 3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE 3.0 INTRODUCTION The purpose of this chapter is to introduce estimators shortly. More elaborated courses on System Identification, which are given

More information

Lessons in Estimation Theory for Signal Processing, Communications, and Control

Lessons in Estimation Theory for Signal Processing, Communications, and Control Lessons in Estimation Theory for Signal Processing, Communications, and Control Jerry M. Mendel Department of Electrical Engineering University of Southern California Los Angeles, California PRENTICE HALL

More information

(Linear equations) Applied Linear Algebra in Geoscience Using MATLAB

(Linear equations) Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB (Linear equations) Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots

More information

Centre for Mathematical Sciences HT 2017 Mathematical Statistics

Centre for Mathematical Sciences HT 2017 Mathematical Statistics Lund University Stationary stochastic processes Centre for Mathematical Sciences HT 2017 Mathematical Statistics Computer exercise 3 in Stationary stochastic processes, HT 17. The purpose of this exercise

More information

Independent Component Analysis. Contents

Independent Component Analysis. Contents Contents Preface xvii 1 Introduction 1 1.1 Linear representation of multivariate data 1 1.1.1 The general statistical setting 1 1.1.2 Dimension reduction methods 2 1.1.3 Independence as a guiding principle

More information

INTEGRAL TRANSFORMS and THEIR APPLICATIONS

INTEGRAL TRANSFORMS and THEIR APPLICATIONS INTEGRAL TRANSFORMS and THEIR APPLICATIONS Lokenath Debnath Professor and Chair of Mathematics and Professor of Mechanical and Aerospace Engineering University of Central Florida Orlando, Florida CRC Press

More information

Mathematics for Engineers and Scientists

Mathematics for Engineers and Scientists Mathematics for Engineers and Scientists Fourth edition ALAN JEFFREY University of Newcastle-upon-Tyne B CHAPMAN & HALL University and Professional Division London New York Tokyo Melbourne Madras Contents

More information

MATLAB for Engineers

MATLAB for Engineers MATLAB for Engineers Adrian Biran Moshe Breiner ADDISON-WESLEY PUBLISHING COMPANY Wokingham, England Reading, Massachusetts Menlo Park, California New York Don Mills, Ontario Amsterdam Bonn Sydney Singapore

More information

Linear Algebra in Numerical Methods. Lecture on linear algebra MATLAB/Octave works well with linear algebra

Linear Algebra in Numerical Methods. Lecture on linear algebra MATLAB/Octave works well with linear algebra Linear Algebra in Numerical Methods Lecture on linear algebra MATLAB/Octave works well with linear algebra Linear Algebra A pseudo-algebra that deals with a system of equations and the transformations

More information

CONTENTS. Preface Preliminaries 1

CONTENTS. Preface Preliminaries 1 Preface xi Preliminaries 1 1 TOOLS FOR ANALYSIS 5 1.1 The Completeness Axiom and Some of Its Consequences 5 1.2 The Distribution of the Integers and the Rational Numbers 12 1.3 Inequalities and Identities

More information

A THEORETICAL INTRODUCTION TO NUMERICAL ANALYSIS

A THEORETICAL INTRODUCTION TO NUMERICAL ANALYSIS A THEORETICAL INTRODUCTION TO NUMERICAL ANALYSIS Victor S. Ryaben'kii Semyon V. Tsynkov Chapman &. Hall/CRC Taylor & Francis Group Boca Raton London New York Chapman & Hall/CRC is an imprint of the Taylor

More information

Math 307 Learning Goals

Math 307 Learning Goals Math 307 Learning Goals May 14, 2018 Chapter 1 Linear Equations 1.1 Solving Linear Equations Write a system of linear equations using matrix notation. Use Gaussian elimination to bring a system of linear

More information

Time Series: Theory and Methods

Time Series: Theory and Methods Peter J. Brockwell Richard A. Davis Time Series: Theory and Methods Second Edition With 124 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition vn ix CHAPTER 1 Stationary

More information

Mathematics for Chemists

Mathematics for Chemists Mathematics for Chemists MATHEMATICS FOR CHEMISTS D. M. Hirst Department of Molecular Sciences, university of Warwick, Coventry M D. M. Hirst 1976 All rights reserved. No part of this publication may be

More information

Analytical Mechanics for Relativity and Quantum Mechanics

Analytical Mechanics for Relativity and Quantum Mechanics Analytical Mechanics for Relativity and Quantum Mechanics Oliver Davis Johns San Francisco State University OXPORD UNIVERSITY PRESS CONTENTS Dedication Preface Acknowledgments v vii ix PART I INTRODUCTION:

More information

OPTIMAL CONTROL AND ESTIMATION

OPTIMAL CONTROL AND ESTIMATION OPTIMAL CONTROL AND ESTIMATION Robert F. Stengel Department of Mechanical and Aerospace Engineering Princeton University, Princeton, New Jersey DOVER PUBLICATIONS, INC. New York CONTENTS 1. INTRODUCTION

More information

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY

Time Series Analysis. James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY Time Series Analysis James D. Hamilton PRINCETON UNIVERSITY PRESS PRINCETON, NEW JERSEY & Contents PREFACE xiii 1 1.1. 1.2. Difference Equations First-Order Difference Equations 1 /?th-order Difference

More information

EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

More information

MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS

MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS T H I R D E D I T I O N MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS STANLEY I. GROSSMAN University of Montana and University College London SAUNDERS COLLEGE PUBLISHING HARCOURT BRACE

More information

MA3025 Course Prerequisites

MA3025 Course Prerequisites MA3025 Course Prerequisites MA 3025 (4-1) MA3025 (4-1) Logic and Discrete Mathematics: Provides a rigorous foundation in logic and elementary discrete mathematics. Topics from logic include modeling English

More information

Sample ECE275A Midterm Exam Questions

Sample ECE275A Midterm Exam Questions Sample ECE275A Midterm Exam Questions The questions given below are actual problems taken from exams given in in the past few years. Solutions to these problems will NOT be provided. These problems and

More information

Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides

Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering Stochastic Processes and Linear Algebra Recap Slides Stochastic processes and variables XX tt 0 = XX xx nn (tt) xx 2 (tt) XX tt XX

More information

Dynamic Systems. Modeling and Analysis. Hung V. Vu. Ramin S. Esfandiari. THE McGRAW-HILL COMPANIES, INC. California State University, Long Beach

Dynamic Systems. Modeling and Analysis. Hung V. Vu. Ramin S. Esfandiari. THE McGRAW-HILL COMPANIES, INC. California State University, Long Beach Dynamic Systems Modeling and Analysis Hung V. Vu California State University, Long Beach Ramin S. Esfandiari California State University, Long Beach THE McGRAW-HILL COMPANIES, INC. New York St. Louis San

More information

Probability Models in Electrical and Computer Engineering Mathematical models as tools in analysis and design Deterministic models Probability models

Probability Models in Electrical and Computer Engineering Mathematical models as tools in analysis and design Deterministic models Probability models Probability Models in Electrical and Computer Engineering Mathematical models as tools in analysis and design Deterministic models Probability models Statistical regularity Properties of relative frequency

More information

MATRIX AND LINEAR ALGEBR A Aided with MATLAB

MATRIX AND LINEAR ALGEBR A Aided with MATLAB Second Edition (Revised) MATRIX AND LINEAR ALGEBR A Aided with MATLAB Kanti Bhushan Datta Matrix and Linear Algebra Aided with MATLAB Second Edition KANTI BHUSHAN DATTA Former Professor Department of Electrical

More information

Acoustic MIMO Signal Processing

Acoustic MIMO Signal Processing Yiteng Huang Jacob Benesty Jingdong Chen Acoustic MIMO Signal Processing With 71 Figures Ö Springer Contents 1 Introduction 1 1.1 Acoustic MIMO Signal Processing 1 1.2 Organization of the Book 4 Part I

More information

Math 302 Outcome Statements Winter 2013

Math 302 Outcome Statements Winter 2013 Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the Three-Dimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a

More information

xvi xxiii xxvi Construction of the Real Line 2 Is Every Real Number Rational? 3 Problems Algebra of the Real Numbers 7

xvi xxiii xxvi Construction of the Real Line 2 Is Every Real Number Rational? 3 Problems Algebra of the Real Numbers 7 About the Author v Preface to the Instructor xvi WileyPLUS xxii Acknowledgments xxiii Preface to the Student xxvi 1 The Real Numbers 1 1.1 The Real Line 2 Construction of the Real Line 2 Is Every Real

More information

Mobile Robotics 1. A Compact Course on Linear Algebra. Giorgio Grisetti

Mobile Robotics 1. A Compact Course on Linear Algebra. Giorgio Grisetti Mobile Robotics 1 A Compact Course on Linear Algebra Giorgio Grisetti SA-1 Vectors Arrays of numbers They represent a point in a n dimensional space 2 Vectors: Scalar Product Scalar-Vector Product Changes

More information

(Mathematical Operations with Arrays) Applied Linear Algebra in Geoscience Using MATLAB

(Mathematical Operations with Arrays) Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB (Mathematical Operations with Arrays) Contents Getting Started Matrices Creating Arrays Linear equations Mathematical Operations with Arrays Using Script

More information

Mathematical Methods for Engineers and Scientists 1

Mathematical Methods for Engineers and Scientists 1 K.T. Tang Mathematical Methods for Engineers and Scientists 1 Complex Analysis, Determinants and Matrices With 49 Figures and 2 Tables fyj Springer Part I Complex Analysis 1 Complex Numbers 3 1.1 Our Number

More information

Advanced. Engineering Mathematics

Advanced. Engineering Mathematics Advanced Engineering Mathematics A new edition of Further Engineering Mathematics K. A. Stroud Formerly Principal Lecturer Department of Mathematics, Coventry University with additions by Dexter j. Booth

More information

L3: Review of linear algebra and MATLAB

L3: Review of linear algebra and MATLAB L3: Review of linear algebra and MATLAB Vector and matrix notation Vectors Matrices Vector spaces Linear transformations Eigenvalues and eigenvectors MATLAB primer CSCE 666 Pattern Analysis Ricardo Gutierrez-Osuna

More information

TIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M.

TIME SERIES ANALYSIS. Forecasting and Control. Wiley. Fifth Edition GWILYM M. JENKINS GEORGE E. P. BOX GREGORY C. REINSEL GRETA M. TIME SERIES ANALYSIS Forecasting and Control Fifth Edition GEORGE E. P. BOX GWILYM M. JENKINS GREGORY C. REINSEL GRETA M. LJUNG Wiley CONTENTS PREFACE TO THE FIFTH EDITION PREFACE TO THE FOURTH EDITION

More information

Orbital and Celestial Mechanics

Orbital and Celestial Mechanics Orbital and Celestial Mechanics John P. Vinti Edited by Gim J. Der TRW Los Angeles, California Nino L. Bonavito NASA Goddard Space Flight Center Greenbelt, Maryland Volume 177 PROGRESS IN ASTRONAUTICS

More information

Mathematical Theory of Control Systems Design

Mathematical Theory of Control Systems Design Mathematical Theory of Control Systems Design by V. N. Afarias'ev, V. B. Kolmanovskii and V. R. Nosov Moscow University of Electronics and Mathematics, Moscow, Russia KLUWER ACADEMIC PUBLISHERS DORDRECHT

More information

Elements of Multivariate Time Series Analysis

Elements of Multivariate Time Series Analysis Gregory C. Reinsel Elements of Multivariate Time Series Analysis Second Edition With 14 Figures Springer Contents Preface to the Second Edition Preface to the First Edition vii ix 1. Vector Time Series

More information

Biomedical Signal Processing and Signal Modeling

Biomedical Signal Processing and Signal Modeling Biomedical Signal Processing and Signal Modeling Eugene N. Bruce University of Kentucky A Wiley-lnterscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto

More information

Review of some mathematical tools

Review of some mathematical tools MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical

More information

Course Name: Digital Signal Processing Course Code: EE 605A Credit: 3

Course Name: Digital Signal Processing Course Code: EE 605A Credit: 3 Course Name: Digital Signal Processing Course Code: EE 605A Credit: 3 Prerequisites: Sl. No. Subject Description Level of Study 01 Mathematics Fourier Transform, Laplace Transform 1 st Sem, 2 nd Sem 02

More information

Chapter 2 Wiener Filtering

Chapter 2 Wiener Filtering Chapter 2 Wiener Filtering Abstract Before moving to the actual adaptive filtering problem, we need to solve the optimum linear filtering problem (particularly, in the mean-square-error sense). We start

More information

Partial Differential Equations with MATLAB

Partial Differential Equations with MATLAB CHAPMAN & HALL/CRC APPLIED MATHEMATICS AND NONLINEAR SCIENCE SERIES An Introduction to Partial Differential Equations with MATLAB Second Edition Matthew P. Coleman CHAPMAN & HALL/CRC APPLIED MATHEMATICS

More information

Theory and Problems of Signals and Systems

Theory and Problems of Signals and Systems SCHAUM'S OUTLINES OF Theory and Problems of Signals and Systems HWEI P. HSU is Professor of Electrical Engineering at Fairleigh Dickinson University. He received his B.S. from National Taiwan University

More information

ENGINEERING MATHEMATICS I. CODE: 10 MAT 11 IA Marks: 25 Hrs/Week: 04 Exam Hrs: 03 PART-A

ENGINEERING MATHEMATICS I. CODE: 10 MAT 11 IA Marks: 25 Hrs/Week: 04 Exam Hrs: 03 PART-A ENGINEERING MATHEMATICS I CODE: 10 MAT 11 IA Marks: 25 Hrs/Week: 04 Exam Hrs: 03 Total Hrs: 52 Exam Marks:100 PART-A Unit-I: DIFFERENTIAL CALCULUS - 1 Determination of n th derivative of standard functions-illustrative

More information

Adaptive Systems Homework Assignment 1

Adaptive Systems Homework Assignment 1 Signal Processing and Speech Communication Lab. Graz University of Technology Adaptive Systems Homework Assignment 1 Name(s) Matr.No(s). The analytical part of your homework (your calculation sheets) as

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Contents. 1 Preliminaries 3. Martingales

Contents. 1 Preliminaries 3. Martingales Table of Preface PART I THE FUNDAMENTAL PRINCIPLES page xv 1 Preliminaries 3 2 Martingales 9 2.1 Martingales and examples 9 2.2 Stopping times 12 2.3 The maximum inequality 13 2.4 Doob s inequality 14

More information

component risk analysis

component risk analysis 273: Urban Systems Modeling Lec. 3 component risk analysis instructor: Matteo Pozzi 273: Urban Systems Modeling Lec. 3 component reliability outline risk analysis for components uncertain demand and uncertain

More information

The Way of Analysis. Robert S. Strichartz. Jones and Bartlett Publishers. Mathematics Department Cornell University Ithaca, New York

The Way of Analysis. Robert S. Strichartz. Jones and Bartlett Publishers. Mathematics Department Cornell University Ithaca, New York The Way of Analysis Robert S. Strichartz Mathematics Department Cornell University Ithaca, New York Jones and Bartlett Publishers Boston London Contents Preface xiii 1 Preliminaries 1 1.1 The Logic of

More information

Practical Spectral Estimation

Practical Spectral Estimation Digital Signal Processing/F.G. Meyer Lecture 4 Copyright 2015 François G. Meyer. All Rights Reserved. Practical Spectral Estimation 1 Introduction The goal of spectral estimation is to estimate how the

More information

Contents. Preface to the Third Edition (2007) Preface to the Second Edition (1992) Preface to the First Edition (1985) License and Legal Information

Contents. Preface to the Third Edition (2007) Preface to the Second Edition (1992) Preface to the First Edition (1985) License and Legal Information Contents Preface to the Third Edition (2007) Preface to the Second Edition (1992) Preface to the First Edition (1985) License and Legal Information xi xiv xvii xix 1 Preliminaries 1 1.0 Introduction.............................

More information

The Essentials of Linear State-Space Systems

The Essentials of Linear State-Space Systems :or-' The Essentials of Linear State-Space Systems J. Dwight Aplevich GIFT OF THE ASIA FOUNDATION NOT FOR RE-SALE John Wiley & Sons, Inc New York Chichester Weinheim OAI HOC OUOC GIA HA N^l TRUNGTAMTHANCTINTHUVIIN

More information

Performance Analysis and Enhancements of Adaptive Algorithms and Their Applications

Performance Analysis and Enhancements of Adaptive Algorithms and Their Applications Performance Analysis and Enhancements of Adaptive Algorithms and Their Applications SHENGKUI ZHAO School of Computer Engineering A thesis submitted to the Nanyang Technological University in partial fulfillment

More information

Here represents the impulse (or delta) function. is an diagonal matrix of intensities, and is an diagonal matrix of intensities.

Here represents the impulse (or delta) function. is an diagonal matrix of intensities, and is an diagonal matrix of intensities. 19 KALMAN FILTER 19.1 Introduction In the previous section, we derived the linear quadratic regulator as an optimal solution for the fullstate feedback control problem. The inherent assumption was that

More information

Introduction to Mobile Robotics Compact Course on Linear Algebra. Wolfram Burgard, Bastian Steder

Introduction to Mobile Robotics Compact Course on Linear Algebra. Wolfram Burgard, Bastian Steder Introduction to Mobile Robotics Compact Course on Linear Algebra Wolfram Burgard, Bastian Steder Reference Book Thrun, Burgard, and Fox: Probabilistic Robotics Vectors Arrays of numbers Vectors represent

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

DISCRETE-TIME SIGNAL PROCESSING

DISCRETE-TIME SIGNAL PROCESSING THIRD EDITION DISCRETE-TIME SIGNAL PROCESSING ALAN V. OPPENHEIM MASSACHUSETTS INSTITUTE OF TECHNOLOGY RONALD W. SCHÄFER HEWLETT-PACKARD LABORATORIES Upper Saddle River Boston Columbus San Francisco New

More information

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.

APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2. APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product

More information

Numerical Methods with MATLAB

Numerical Methods with MATLAB Numerical Methods with MATLAB A Resource for Scientists and Engineers G. J. BÖRSE Lehigh University PWS Publishing Company I(T)P AN!NTERNATIONAL THOMSON PUBLISHING COMPANY Boston Albany Bonn Cincinnati

More information

Cryptography. Number Theory with AN INTRODUCTION TO. James S. Kraft. Lawrence C. Washington. CRC Press

Cryptography. Number Theory with AN INTRODUCTION TO. James S. Kraft. Lawrence C. Washington. CRC Press AN INTRODUCTION TO Number Theory with Cryptography James S Kraft Gilman School Baltimore, Maryland, USA Lawrence C Washington University of Maryland College Park, Maryland, USA CRC Press Taylor & Francis

More information

ELEMENTARY MATRIX ALGEBRA

ELEMENTARY MATRIX ALGEBRA ELEMENTARY MATRIX ALGEBRA Third Edition FRANZ E. HOHN DOVER PUBLICATIONS, INC. Mineola, New York CONTENTS CHAPTER \ Introduction to Matrix Algebra 1.1 Matrices 1 1.2 Equality of Matrices 2 13 Addition

More information

GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS

GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Methods in Geochemistry and Geophysics, 36 GEOPHYSICAL INVERSE THEORY AND REGULARIZATION PROBLEMS Michael S. ZHDANOV University of Utah Salt Lake City UTAH, U.S.A. 2OO2 ELSEVIER Amsterdam - Boston - London

More information

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS

Stochastic Processes. Theory for Applications. Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Stochastic Processes Theory for Applications Robert G. Gallager CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv Swgg&sfzoMj ybr zmjfr%cforj owf fmdy xix Acknowledgements xxi 1 Introduction and review

More information