Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering. Stochastic Processes and Linear Algebra Recap Slides
|
|
- Kristian Fisher
- 5 years ago
- Views:
Transcription
1 Prof. Dr.-Ing. Armin Dekorsy Department of Communications Engineering Stochastic Processes and Linear Algebra Recap Slides
2 Stochastic processes and variables XX tt 0 = XX xx nn (tt) xx 2 (tt) XX tt XX tt 0 xx ii tt xx ii tt 0 = XX = xx random process random variable realization of random process realization of random variable xx 1 (tt) tt 0 tt XX tt /tt continuous discrete continuous Continuous-state continuous-time process Continuous-state discretetime process discrete-time process = sequence discrete Discrete-state continuoustime process Discrete-state discrete-time process 2
3 Continuous-state discrete-time process process XX(kk) current realization of XX(kk): realization xx(kk) A stochastic process is said to be strict sense stationary (SSS), if the statistics are invariant to any translation of the time axis A stochastic process is said to be wide sense stationary (WSS), if its mean is constant and its autocorrelation depends on a time difference ττ only Here: we simply call WSS as stationary If expected values (averaging multiple realizations) can be calculated by time averaging of one realization, the process is said to be ergodic. Ergodic processes are always strict sense stationary; but not all strict sense stationary processes have to be ergodic We presume XX(kk) to be ergodic Moment calculation via averaging in time 3
4 Continuous-state discrete-time process Probability density function: 1 pp XX xx = lim Pr{xx < XX xx + xx} xx 0 xx Joint probability density function: 1 pp XX,YY xx, yy = lim Pr{xx < XX xx 0, yy 0 xx yy xx + xx, yy < YY yy + yy} Normal distribution: pp XX xx = 1 2πππππ ee xx μμ XX ² 2σσσ Moments E{ } st order: E XX = xx pp XX xx dddd = μμ XX 2 nd order: E XX² = xx² pp XX xx dddd Variance: E XX μμ XX ² = E XXX E XX 2 = σσ XX 2 = + xx μμ XX ²pp XX xx dddd 4
5 Correlation series of discrete-time processes auto correlation series (not necessarily stationary); complex valued process XX(kk) rr XXXX κκ 1, κκ 2 = E XX κκ 1 XX κκ 2 = E (XX RR κκ 1 jjxx II κκ 1 ) (XX RR κκ 2 + jjxx II κκ 2 ) stationary processes: κκ 1 kk, κκ 2 kk + κκ; rr XXXX κκ = E XX kk XX kk + κκ auto covariance series: cc XXXX (κκ) = E (XX kk μμ XX )(XX kk + κκ μμ XX ) = rr XXXX κκ μμ XX ² zero mean process: cc XXXX (κκ) = rr XXXX κκ cross correlation series of two processes XX kk, YY kk rr XXYY κκ 1, κκ 2 = E XX κκ 1 YY κκ 2 stationary rr XXYY κκ = E XX kk YY kk + κκ 5
6 Correlation series of discrete-time processes Properties of the ACS rr XXXX κκ = rr XXXX κκ real valued processes: rr XXXX κκ = rr XXXX κκ even ACF max rr XXXX κκ = rr XXXX 0 κκ rr XXXX 0 = E XX 2 kk XX kk = E XX kk ² zero mean: rr XXXX 0 = σσ XX Properties of the CCS rr XXYY κκ = rr YYXX κκ real valued processes: rr XXYY κκ = rr YYXX κκ cc XXYY κκ = rr XXXX κκ μμ XX μμ YY cross covariance sequence 6
7 Random Variables (RVs): Covariance/Uncorrelatedness/Orthogonality The covariance C of two RVs XX and YY is C = E (XX μμ XX ) (YY μμ YY ) =E XXXX - E XX E YY Uncorrelatedness: Two RVs are called uncorrelated if their covariance equals zero. C =0 E XXXX = E XX E YY Orthogonality: Two RVs are called orthogonal if E XXXX =0 7
8 Processes: Correlateness, Orthogonality, White noise Two WSS processes XX(kk) and YY(kk) are called uncorrelated if cc XXXX κκ = 0 κκ rr XXXX κκ = μμ XX μμ YY zero mean processes: rr XXXX κκ = 0 κκ Two WSS processes XX(kk) and YY(kk) are called (mutually) orthogonal if rr XXXX κκ = 0 κκ White noise: White noise is a stationary process with E{XX(kk)}=0 and rr XXXX κκ = σσ XX 2 δδ(κκ) 8
9 Power Spectral Density Definition (Wiener-Khintchine Theorem): SS XXXX ee jjω = DTFT rr XXXX κκ = rr XXXX κκ ee jjωκκ Because of conjugate even ACF Power Spectral Density always real valued Total power of the process (zero mean): Var XX kk ππ = σσ 2 XX = SS XXXX ee jjω ππ ddω = rr XXXX 0 White noise: PSD constant (total power limited, because of band-limited system) SS XXXX ee jjω rr XXXX κκ = IDTFT σσ XX 2 κκ= = σσ XX 2 for ππ < Ω < ππ = σσ XX 2 δδ(κκ) 9
10 ACF for bandlimited noise 10
11 Influence of a linear system rand. process at the input of the system: XX(kk) System impulse response: h kk rand. process at the output of the system: YY(kk) System-(energy-) autocorrelation sequence: rr hh ACS output: rr YYYY κκ = rr XXXX κκ rr EE hh CCS output: rr XXYY κκ = rr XXXX κκ h κκ EE κκ = h kk h(kk + κκ) kk= κκ = rr XXXX κκ h κκ h kk = h κκ h kk Power density output: SS YYYY ee jjω = SS XXXX ee jjω HH ee jjω ² phase blind Cross power density; in-output: SS XXYY ee jjω = SS XXXX ee jjω HH ee jjω White noise at the input of a system: rr YYYY κκ = σσ 2 XX δδ κκ rr EE hh κκ = σσ 2 XX rr EE hh κκ SS YYYY ee jjω = σσ 2 XX HH ee jjω 2 rr XXYY κκ = σσ 2 XX δδ κκ h κκ = σσ 2 XX h κκ SS XXYY ee jjω = σσ 2 XX HH ee jjω 11
12 Complex Gaussian noise PDF of a single real valued Gaussian random variable pp nn nn = 1 σσ NN 2ππ ee nn 2 2σσ 2 NN PDF of a complex valued random variable nn = nn + jjjjjj is given by the joint pdf of two real-valued (real and imaginary part) random variables pp nn (nn + jjjjjj) pp nn,nn (nn, nnnn) If we assume that real and imaginary part are statistically independent then pp nn (nn + jjjjjj) pp nn (nn ) pp nn (nn ) PDF of a single complex Gaussian random variable pp nn nn = 1 nn 2 +nn 2 σσ 2 NN ππ ee σσ2 NN = 1 nn 2 σσ 2 NN ππ ee σσ2 NN pp nn nn pp nn nn + jjnn nn nn nn 12
13 New nomenclature In the following we use small letters for both random variable and particular realization. Random variable: XX xx Scalar random variable: xx Vector-valued random variable: xx (column vector) Matrix-valued random variable: XX 13
14 Autocorrelation matrix Vector-valued random variable: xx = xx(0) xx(1) xx(nn 1) C NN 1 : column vector; expectation: E xx = NN 1 E xx 2 = E xx H xx = E xx ii 2 ii=0 Autocorrelation matrix: E xx 0 E xx 1 E xx NN 1 = E xx xx xx NN 1 2 Note: 14
15 Convolution as inner product Digital signals and linear time invariant system: Assume: h kk causal FIR with order mm impulse response of length mm + 1 Timely infinite input sequence xx(kk); < kk < Define: hh = xx(kk) yy kk = xx kk h kk = h(0) h(1) h(mm) h(kk) C mm+1 ; xx(kk) = mm υυ=0 yy(kk) h υυ xx(kk υυ) xx(kk) xx(kk 1) xx(kk mm) past values of xx(kk), non-causal input C mm+1 15
16 Convolution as inner product Output signal yy(kk) of filter as inner product: yy(kk) = hh T xx kk = xx kk T hh Assume: XX(kk) is stationary discrete-time process xx(kk) is vector of random variables yy(kk) is scalar random variable Power of output signal: E yy kk 2 = E yy kk yy kk = E hh T xx(kk)xx H (kk)hh = hh T E xx(kk)xx H (kk) hh = hh T RR xxxx hh 16
17 Convolution as matrix multiplication Causal input: xx kk = [xx 0, xx 1,, xx LL 1 ] T Finite impulse response: hh kk = [h 0, h 1,, h mm ] T Full equation system: 17
18 Convolution as matrix multiplication Example of convolution as matrix multiplication with mm = 2, LL = 4: transient phase Matrix HH has Toeplitz structure steady-state: complete impulse response in rows filter filled up with input samples decay phase 18
19 Convolution as matrix multiplication Convolutional matrix in general: Toeplitz structure mm transient phase LL + mm LL m steady state decay phase mm LL 19
20 Correlation as convolution Define correlation of two signals (at least one is deterministic) as: with LL LL 20
21 Correlation as scalar product Define correlation of two signals (at least one is deterministic) as: with Note: of convolution defined as causal input in contrast to anti-causal definition for formulation 21
22 Singular Value Decomposition (SVD) Every mm nn matrix AA of rank r can be written as Singular values σσ ii of AA = square roots of nonzero eigenvalues of A H A or AA H Unitary mm mm matrix UU contains left singular vectors of A = eigenvectors of AA H Unitary nn nn matrix VV contains right singular vectors of A = eigenvectors of A H A Verification with eigenvalue decomposition with the matrix of singular values Four fundamental subspaces: the vectors u 1,...,u r span the column space of A u r+1,...,u m span the left nullspace of A v 1,...,v r span the row space of A v r+1,...,v n span the right nullspace of A 22
23 Singular Value Decomposition (SVD) (2) Illustration of the fundamental subspaces x! Ax Consider linear mapping with orthogonal decomposition x = x r + x n x r Ax = Ax r 0 x Ax n = 0 x n 23
24 Moore-Penrose Pseudoinverse Inverse AA 1 exists only for square matrices with full rank Assume any mm nn matrix AA Definition: (Moore-Penrose) pseudo inverse A + ) Special cases for full rank matrices: It can be verified that if and only if AA has full rank 24
25 QR decomposition Every mm nn matrix A can be written as where Q is an m n matrix with orthonormal columns, R is an upper triangular nn nn matrix Columns of A are represented in the orthonormal base defined by Q Illustration for the mm 2 case q 2 r 2;2 q 2 a 2 = r 1;2 q 1 + r 2;2 q 2 q 1 r 1;2 q 1 a 1 = r 1;1 q 1 25
26 Matrix inversion lemma Matrix Inversion Lemma (A R m x m, B R m x n, C R n x n, D R n x m ) Inverse of block matrix E: with A R m x m, B R m x n, C R n x m, D R n x n Schur complement of A w.r.t E Schur complement of D w.r.t E 26
27 Wirtinger calculus since Derivative w.r.t. a vector derivative w.r.t. column-vector row-vector derivative w.r.t. row-vector column-vector 27
ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process
Department of Electrical Engineering University of Arkansas ELEG 3143 Probability & Stochastic Process Ch. 6 Stochastic Process Dr. Jingxian Wu wuj@uark.edu OUTLINE 2 Definition of stochastic process (random
More informationMATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.
MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation
More informationVariations. ECE 6540, Lecture 02 Multivariate Random Variables & Linear Algebra
Variations ECE 6540, Lecture 02 Multivariate Random Variables & Linear Algebra Last Time Probability Density Functions Normal Distribution Expectation / Expectation of a function Independence Uncorrelated
More informationLarge Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Linear Algebra U Kang Seoul National University U Kang 1 In This Lecture Overview of linear algebra (but, not a comprehensive survey) Focused on the subset
More informationParametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes
Parametric Signal Modeling and Linear Prediction Theory 1. Discrete-time Stochastic Processes Electrical & Computer Engineering North Carolina State University Acknowledgment: ECE792-41 slides were adapted
More information3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE
3. ESTIMATION OF SIGNALS USING A LEAST SQUARES TECHNIQUE 3.0 INTRODUCTION The purpose of this chapter is to introduce estimators shortly. More elaborated courses on System Identification, which are given
More informationLecture Notes 7 Stationary Random Processes. Strict-Sense and Wide-Sense Stationarity. Autocorrelation Function of a Stationary Process
Lecture Notes 7 Stationary Random Processes Strict-Sense and Wide-Sense Stationarity Autocorrelation Function of a Stationary Process Power Spectral Density Continuity and Integration of Random Processes
More informationChapter 6. Random Processes
Chapter 6 Random Processes Random Process A random process is a time-varying function that assigns the outcome of a random experiment to each time instant: X(t). For a fixed (sample path): a random process
More informationStochastic Processes
Elements of Lecture II Hamid R. Rabiee with thanks to Ali Jalali Overview Reading Assignment Chapter 9 of textbook Further Resources MIT Open Course Ware S. Karlin and H. M. Taylor, A First Course in Stochastic
More informationP 1.5 X 4.5 / X 2 and (iii) The smallest value of n for
DHANALAKSHMI COLLEGE OF ENEINEERING, CHENNAI DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING MA645 PROBABILITY AND RANDOM PROCESS UNIT I : RANDOM VARIABLES PART B (6 MARKS). A random variable X
More informationSingular Value Decomposition (SVD)
School of Computing National University of Singapore CS CS524 Theoretical Foundations of Multimedia More Linear Algebra Singular Value Decomposition (SVD) The highpoint of linear algebra Gilbert Strang
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationAdvanced Digital Signal Processing -Introduction
Advanced Digital Signal Processing -Introduction LECTURE-2 1 AP9211- ADVANCED DIGITAL SIGNAL PROCESSING UNIT I DISCRETE RANDOM SIGNAL PROCESSING Discrete Random Processes- Ensemble Averages, Stationary
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More information16.584: Random (Stochastic) Processes
1 16.584: Random (Stochastic) Processes X(t): X : RV : Continuous function of the independent variable t (time, space etc.) Random process : Collection of X(t, ζ) : Indexed on another independent variable
More informationfor valid PSD. PART B (Answer all five units, 5 X 10 = 50 Marks) UNIT I
Code: 15A04304 R15 B.Tech II Year I Semester (R15) Regular Examinations November/December 016 PROBABILITY THEY & STOCHASTIC PROCESSES (Electronics and Communication Engineering) Time: 3 hours Max. Marks:
More informationEAS 305 Random Processes Viewgraph 1 of 10. Random Processes
EAS 305 Random Processes Viewgraph 1 of 10 Definitions: Random Processes A random process is a family of random variables indexed by a parameter t T, where T is called the index set λ i Experiment outcome
More informationProbability Space. J. McNames Portland State University ECE 538/638 Stochastic Signals Ver
Stochastic Signals Overview Definitions Second order statistics Stationarity and ergodicity Random signal variability Power spectral density Linear systems with stationary inputs Random signal memory Correlation
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More informationEigenvalues and Eigenvectors
/88 Chia-Ping Chen Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Eigenvalue Problem /88 Eigenvalue Equation By definition, the eigenvalue equation for matrix
More informationA Review of Linear Algebra
A Review of Linear Algebra Mohammad Emtiyaz Khan CS,UBC A Review of Linear Algebra p.1/13 Basics Column vector x R n, Row vector x T, Matrix A R m n. Matrix Multiplication, (m n)(n k) m k, AB BA. Transpose
More information2. LINEAR ALGEBRA. 1. Definitions. 2. Linear least squares problem. 3. QR factorization. 4. Singular value decomposition (SVD) 5.
2. LINEAR ALGEBRA Outline 1. Definitions 2. Linear least squares problem 3. QR factorization 4. Singular value decomposition (SVD) 5. Pseudo-inverse 6. Eigenvalue decomposition (EVD) 1 Definitions Vector
More informationProblems on Discrete & Continuous R.Vs
013 SUBJECT NAME SUBJECT CODE MATERIAL NAME MATERIAL CODE : Probability & Random Process : MA 61 : University Questions : SKMA1004 Name of the Student: Branch: Unit I (Random Variables) Problems on Discrete
More information7.3 The Jacobi and Gauss-Seidel Iterative Methods
7.3 The Jacobi and Gauss-Seidel Iterative Methods 1 The Jacobi Method Two assumptions made on Jacobi Method: 1.The system given by aa 11 xx 1 + aa 12 xx 2 + aa 1nn xx nn = bb 1 aa 21 xx 1 + aa 22 xx 2
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More information7. Symmetric Matrices and Quadratic Forms
Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value
More informationSignals and Spectra - Review
Signals and Spectra - Review SIGNALS DETERMINISTIC No uncertainty w.r.t. the value of a signal at any time Modeled by mathematical epressions RANDOM some degree of uncertainty before the signal occurs
More informationStatistical signal processing
Statistical signal processing Short overview of the fundamentals Outline Random variables Random processes Stationarity Ergodicity Spectral analysis Random variable and processes Intuition: A random variable
More informationSingular Value Decomposition
Singular Value Decomposition Motivatation The diagonalization theorem play a part in many interesting applications. Unfortunately not all matrices can be factored as A = PDP However a factorization A =
More informationLinear Algebra, part 3 QR and SVD
Linear Algebra, part 3 QR and SVD Anna-Karin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2012 Going back to least squares (Section 1.4 from Strang, now also see section 5.2). We
More informationTime Domain Analysis of Linear Systems Ch2. University of Central Oklahoma Dr. Mohamed Bingabr
Time Domain Analysis of Linear Systems Ch2 University of Central Oklahoma Dr. Mohamed Bingabr Outline Zero-input Response Impulse Response h(t) Convolution Zero-State Response System Stability System Response
More informationName of the Student: Problems on Discrete & Continuous R.Vs
Engineering Mathematics 05 SUBJECT NAME : Probability & Random Process SUBJECT CODE : MA6 MATERIAL NAME : University Questions MATERIAL CODE : JM08AM004 REGULATION : R008 UPDATED ON : Nov-Dec 04 (Scan
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationUNIT 6: The singular value decomposition.
UNIT 6: The singular value decomposition. María Barbero Liñán Universidad Carlos III de Madrid Bachelor in Statistics and Business Mathematical methods II 2011-2012 A square matrix is symmetric if A T
More informationStochastic Processes. A stochastic process is a function of two variables:
Stochastic Processes Stochastic: from Greek stochastikos, proceeding by guesswork, literally, skillful in aiming. A stochastic process is simply a collection of random variables labelled by some parameter:
More informationProbability and Statistics for Final Year Engineering Students
Probability and Statistics for Final Year Engineering Students By Yoni Nazarathy, Last Updated: May 24, 2011. Lecture 6p: Spectral Density, Passing Random Processes through LTI Systems, Filtering Terms
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationMath 108b: Notes on the Spectral Theorem
Math 108b: Notes on the Spectral Theorem From section 6.3, we know that every linear operator T on a finite dimensional inner product space V has an adjoint. (T is defined as the unique linear operator
More informationStochastic Processes. M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno
Stochastic Processes M. Sami Fadali Professor of Electrical Engineering University of Nevada, Reno 1 Outline Stochastic (random) processes. Autocorrelation. Crosscorrelation. Spectral density function.
More informationSRI VIDYA COLLEGE OF ENGINEERING AND TECHNOLOGY UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS
UNIT 3 RANDOM PROCESS TWO MARK QUESTIONS 1. Define random process? The sample space composed of functions of time is called a random process. 2. Define Stationary process? If a random process is divided
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationECE 636: Systems identification
ECE 636: Systems identification Lectures 3 4 Random variables/signals (continued) Random/stochastic vectors Random signals and linear systems Random signals in the frequency domain υ ε x S z + y Experimental
More informationName of the Student: Problems on Discrete & Continuous R.Vs
Engineering Mathematics 08 SUBJECT NAME : Probability & Random Processes SUBJECT CODE : MA645 MATERIAL NAME : University Questions REGULATION : R03 UPDATED ON : November 07 (Upto N/D 07 Q.P) (Scan the
More informationFundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes
Fundamentals of Digital Commun. Ch. 4: Random Variables and Random Processes Klaus Witrisal witrisal@tugraz.at Signal Processing and Speech Communication Laboratory www.spsc.tugraz.at Graz University of
More informationPseudoinverse & Moore-Penrose Conditions
ECE 275AB Lecture 7 Fall 2008 V1.0 c K. Kreutz-Delgado, UC San Diego p. 1/1 Lecture 7 ECE 275A Pseudoinverse & Moore-Penrose Conditions ECE 275AB Lecture 7 Fall 2008 V1.0 c K. Kreutz-Delgado, UC San Diego
More informationData Preprocessing. Jilles Vreeken IRDM 15/ Oct 2015
Data Preprocessing Jilles Vreeken 22 Oct 2015 So, how do you pronounce Jilles Yill-less Vreeken Fray-can Okay, now we can talk. Questions of the day How do we preprocess data before we can extract anything
More informationStatistical and Adaptive Signal Processing
r Statistical and Adaptive Signal Processing Spectral Estimation, Signal Modeling, Adaptive Filtering and Array Processing Dimitris G. Manolakis Massachusetts Institute of Technology Lincoln Laboratory
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationA Posteriori Error Estimates For Discontinuous Galerkin Methods Using Non-polynomial Basis Functions
Lin Lin A Posteriori DG using Non-Polynomial Basis 1 A Posteriori Error Estimates For Discontinuous Galerkin Methods Using Non-polynomial Basis Functions Lin Lin Department of Mathematics, UC Berkeley;
More informationAdaptive Filtering. Squares. Alexander D. Poularikas. Fundamentals of. Least Mean. with MATLABR. University of Alabama, Huntsville, AL.
Adaptive Filtering Fundamentals of Least Mean Squares with MATLABR Alexander D. Poularikas University of Alabama, Huntsville, AL CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is
More informationContents. Preface for the Instructor. Preface for the Student. xvii. Acknowledgments. 1 Vector Spaces 1 1.A R n and C n 2
Contents Preface for the Instructor xi Preface for the Student xv Acknowledgments xvii 1 Vector Spaces 1 1.A R n and C n 2 Complex Numbers 2 Lists 5 F n 6 Digression on Fields 10 Exercises 1.A 11 1.B Definition
More informationLecture Notes to Big Data Management and Analytics Winter Term 2017/2018 Text Processing and High-Dimensional Data
Lecture Notes to Winter Term 2017/2018 Text Processing and High-Dimensional Data Matthias Schubert, Matthias Renz, Felix Borutta, Evgeniy Faerman, Christian Frey, Klaus Arthur Schmid, Daniyal Kazempour,
More informationDigital Image Processing
Digital Image Processing 2D SYSTEMS & PRELIMINARIES Hamid R. Rabiee Fall 2015 Outline 2 Two Dimensional Fourier & Z-transform Toeplitz & Circulant Matrices Orthogonal & Unitary Matrices Block Matrices
More information5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More informationlinearly indepedent eigenvectors as the multiplicity of the root, but in general there may be no more than one. For further discussion, assume matrice
3. Eigenvalues and Eigenvectors, Spectral Representation 3.. Eigenvalues and Eigenvectors A vector ' is eigenvector of a matrix K, if K' is parallel to ' and ' 6, i.e., K' k' k is the eigenvalue. If is
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationIndex. book 2009/5/27 page 121. (Page numbers set in bold type indicate the definition of an entry.)
page 121 Index (Page numbers set in bold type indicate the definition of an entry.) A absolute error...26 componentwise...31 in subtraction...27 normwise...31 angle in least squares problem...98,99 approximation
More informationI. Multiple Choice Questions (Answer any eight)
Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam - Course Instructor : Prashanth L.A. Date : Sep-24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY
More informationDS-GA 1002 Lecture notes 10 November 23, Linear models
DS-GA 2 Lecture notes November 23, 2 Linear functions Linear models A linear model encodes the assumption that two quantities are linearly related. Mathematically, this is characterized using linear functions.
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations
More informationLecture 2. Linear Systems
Lecture 2. Linear Systems Ivan Papusha CDS270 2: Mathematical Methods in Control and System Engineering April 6, 2015 1 / 31 Logistics hw1 due this Wed, Apr 8 hw due every Wed in class, or my mailbox on
More informationLesson 1. Optimal signalbehandling LTH. September Statistical Digital Signal Processing and Modeling, Hayes, M:
Lesson 1 Optimal Signal Processing Optimal signalbehandling LTH September 2013 Statistical Digital Signal Processing and Modeling, Hayes, M: John Wiley & Sons, 1996. ISBN 0471594318 Nedelko Grbic Mtrl
More informationReview of Some Concepts from Linear Algebra: Part 2
Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
More informationMAT Linear Algebra Collection of sample exams
MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationMath 3191 Applied Linear Algebra
Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have
More informationThe Singular Value Decomposition and Least Squares Problems
The Singular Value Decomposition and Least Squares Problems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 27, 2009 Applications of SVD solving
More informationThe Singular Value Decomposition
The Singular Value Decomposition An Important topic in NLA Radu Tiberiu Trîmbiţaş Babeş-Bolyai University February 23, 2009 Radu Tiberiu Trîmbiţaş ( Babeş-Bolyai University)The Singular Value Decomposition
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationModule 7 (Lecture 27) RETAINING WALLS
Module 7 (Lecture 27) RETAINING WALLS Topics 1.1 RETAINING WALLS WITH METALLIC STRIP REINFORCEMENT Calculation of Active Horizontal and vertical Pressure Tie Force Factor of Safety Against Tie Failure
More informationIntroduction to time series econometrics and VARs. Tom Holden PhD Macroeconomics, Semester 2
Introduction to time series econometrics and VARs Tom Holden http://www.tholden.org/ PhD Macroeconomics, Semester 2 Outline of today s talk Discussion of the structure of the course. Some basics: Vector
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationStat 159/259: Linear Algebra Notes
Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University October 17, 005 Lecture 3 3 he Singular Value Decomposition
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationWiener Filter for Deterministic Blur Model
Wiener Filter for Deterministic Blur Model Based on Ch. 5 of Gonzalez & Woods, Digital Image Processing, nd Ed., Addison-Wesley, 00 One common application of the Wiener filter has been in the area of simultaneous
More informationNotes on Eigenvalues, Singular Values and QR
Notes on Eigenvalues, Singular Values and QR Michael Overton, Numerical Computing, Spring 2017 March 30, 2017 1 Eigenvalues Everyone who has studied linear algebra knows the definition: given a square
More informationProposition 42. Let M be an m n matrix. Then (32) N (M M)=N (M) (33) R(MM )=R(M)
RODICA D. COSTIN. Singular Value Decomposition.1. Rectangular matrices. For rectangular matrices M the notions of eigenvalue/vector cannot be defined. However, the products MM and/or M M (which are square,
More informationTutorial on Principal Component Analysis
Tutorial on Principal Component Analysis Copyright c 1997, 2003 Javier R. Movellan. This is an open source document. Permission is granted to copy, distribute and/or modify this document under the terms
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More information[3] (b) Find a reduced row-echelon matrix row-equivalent to ,1 2 2
MATH Key for sample nal exam, August 998 []. (a) Dene the term \reduced row-echelon matrix". A matrix is reduced row-echelon if the following conditions are satised. every zero row lies below every nonzero
More informationThe Singular Value Decomposition
The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will
More informationMatrix Factorizations
1 Stat 540, Matrix Factorizations Matrix Factorizations LU Factorization Definition... Given a square k k matrix S, the LU factorization (or decomposition) represents S as the product of two triangular
More informationChapter 1. Matrix Algebra
ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface
More informationHands-on Matrix Algebra Using R
Preface vii 1. R Preliminaries 1 1.1 Matrix Defined, Deeper Understanding Using Software.. 1 1.2 Introduction, Why R?.................... 2 1.3 Obtaining R.......................... 4 1.4 Reference Manuals
More informationMatrix Theory, Math6304 Lecture Notes from September 27, 2012 taken by Tasadduk Chowdhury
Matrix Theory, Math634 Lecture Notes from September 27, 212 taken by Tasadduk Chowdhury Last Time (9/25/12): QR factorization: any matrix A M n has a QR factorization: A = QR, whereq is unitary and R is
More informationPseudoinverse & Orthogonal Projection Operators
Pseudoinverse & Orthogonal Projection Operators ECE 174 Linear & Nonlinear Optimization Ken Kreutz-Delgado ECE Department, UC San Diego Ken Kreutz-Delgado (UC San Diego) ECE 174 Fall 2016 1 / 48 The Four
More informationLecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed in the
More informationImage Registration Lecture 2: Vectors and Matrices
Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in
More informationLinear Algebra (Review) Volker Tresp 2017
Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.
More informationLinGloss. A glossary of linear algebra
LinGloss A glossary of linear algebra Contents: Decompositions Types of Matrices Theorems Other objects? Quasi-triangular A matrix A is quasi-triangular iff it is a triangular matrix except its diagonal
More informationThe QR Factorization
The QR Factorization How to Make Matrices Nicer Radu Trîmbiţaş Babeş-Bolyai University March 11, 2009 Radu Trîmbiţaş ( Babeş-Bolyai University) The QR Factorization March 11, 2009 1 / 25 Projectors A projector
More informationStatistical Signal Processing Detection, Estimation, and Time Series Analysis
Statistical Signal Processing Detection, Estimation, and Time Series Analysis Louis L. Scharf University of Colorado at Boulder with Cedric Demeure collaborating on Chapters 10 and 11 A TT ADDISON-WESLEY
More informationSample ECE275A Midterm Exam Questions
Sample ECE275A Midterm Exam Questions The questions given below are actual problems taken from exams given in in the past few years. Solutions to these problems will NOT be provided. These problems and
More informationThe Cooper Union Department of Electrical Engineering ECE111 Signal Processing & Systems Analysis Final May 4, 2012
The Cooper Union Department of Electrical Engineering ECE111 Signal Processing & Systems Analysis Final May 4, 2012 Time: 3 hours. Close book, closed notes. No calculators. Part I: ANSWER ALL PARTS. WRITE
More informationVAR Model. (k-variate) VAR(p) model (in the Reduced Form): Y t-2. Y t-1 = A + B 1. Y t + B 2. Y t-p. + ε t. + + B p. where:
VAR Model (k-variate VAR(p model (in the Reduced Form: where: Y t = A + B 1 Y t-1 + B 2 Y t-2 + + B p Y t-p + ε t Y t = (y 1t, y 2t,, y kt : a (k x 1 vector of time series variables A: a (k x 1 vector
More informationECE 6540, Lecture 06 Sufficient Statistics & Complete Statistics Variations
ECE 6540, Lecture 06 Sufficient Statistics & Complete Statistics Variations Last Time Minimum Variance Unbiased Estimators Sufficient Statistics Proving t = T(x) is sufficient Neyman-Fischer Factorization
More information