Data-driven signal processing
|
|
- Lambert Allison
- 6 years ago
- Views:
Transcription
1 1 / 35 Data-driven signal processing Ivan Markovsky
2 2 / 35 Modern signal processing is model-based 1. system identification prior information model structure 2. model-based design identification data parameter vector filter filtered signal design objective to-be-filtered data signal
3 3 / 35 Data-driven methods avoid modeling model filter design filter filtered signal system identification data-driven filter design data-driven filtering data data collection system
4 4 / 35 Combined modeling+design has benefits identification ignores the design objective the two-step approach is suboptimal we define and solve a direct problem observed data + filtering objective filtered signal
5 5 / 35 Plan Example: data-driven Kalman smoothing Generalization: missing data estimation Solution approach: matrix completion
6 6 / 35 Plan Example: data-driven Kalman smoothing Generalization: missing data estimation Solution approach: matrix completion
7 7 / 35 Dynamical system B is set of signals w w B the signal w is trajectory of the system B B is an exact model for w B is unfalsified by w we consider linear time-invariant (LTI) systems L LTI model class
8 8 / 35 Initial conditions past of the signal w w = w p w f w p wf t
9 9 / 35 Estimation in terms of trajectories observer: given model B and exact trajectory w f find w p, such that w p w f B smoother: given model B and noisy trajectory w f minimize w f ŵ f subject to ŵ p ŵ f B (MBS)
10 10 / 35 When does trajectory w d B specify B? identifiability conditions 1. w d is persistently exciting of "sufficiently high order" 2. B is controllable how to obtain B back from w d? w d B by choosing the simplest exact model for w d
11 The most powerful unfalsified model of w d, B mpum (w d ) is the data generating system 11 / 35 complexity # inputs m and # states n c(b) = (m,n) the most powerful unfalsified model B mpum (w d ) := arg min c( B) B L }{{} most powerful subject to w d B }{{} unfalsified model L m,n set of models with complexity bounded by (m,n)
12 Data-driven state estimation replaces the model B by trajectory w d B 12 / 35 observer: given trajectories w d and w f of B find w p, such that w p w f B mpum (w d ) smoother: given noisy traj. w d and w f of B and (m,l) minimize w f ŵ f 2 }{{ 2} estimation error + w d ŵ d 2 }{{ 2} identification error subject to ŵ p ŵ f B mpum (ŵ d ) L m,l
13 13 / 35 Classical approach: divide and conquer 1. identification: given w d and (m,l) minimize w d ŵ d subject to B mpum (ŵ d ) L m,l 2. model-based filtering: given w f and B := B mpum (ŵ d ) minimize w f ŵ f subject to ŵ p ŵ f B
14 Numerical example with Kalman smoothing simulation setup B L1,2 2nd order LTI system wf = w f + noise, w f B step response wd = w d + noise, w d B smoothing with known model state space solution solution of (MBS) smoothing with unknown model identification + model-based design solution of (DDS) 14 / 35
15 15 / 35 Smoothing with known model state space solution [ ] [ ] u minimize f 0 I ][ x ini O T (A,C) T T (H) y f û f (SSS) representation free solution (MBS) is a generalized least squares approximation error e := ( w f ŵ f )/ w f method (MBS) (SSS) error e
16 16 / 35 Smoothing with unknown model classical approach identification + (SSS) data-driven approach solution of (DDS) with local optimization simulation result method (MBS) (DDS) classical error e
17 17 / 35 Plan Example: data-driven Kalman smoothing Generalization: missing data estimation Solution approach: matrix completion
18 18 / 35 We aim to find missing part of trajectory missing data interpolated from w B exact data kept fixed inexact / "noisy" data approximated by min error 2
19 19 / 35 Other examples fit in the same setting? missing, E exact, N noisy w = Π [ u y ], u input, y output example w p u f y f state estimation? E E EIV Kalman smoothing? N N classical Kalman smoothing? E N simulation E E? partial realization E E E/? noisy realization E E N/? output tracking E? N
20 20 / 35 classical Kalman filter minimize y ŷ subject to w p (u,ŷ) B past future input? u output? y output tracking control minimize subject to y ref ŷ }{{} tracking error w p (û,ŷ) B past future input u p? output y p y ref
21 Weighted approximation criterion accounts for exact, missing, and noisy data 21 / 35 error vector: e := w ŵ e v := t i v i (t)e 2 i (t) weight used for to by v i (t) = w i (t) exact interpolate w i (t) e i (t) = 0 v i (t) (0, ) w i (t) noisy approx. w i (t) min e i (t) v i (t) = 0 w i (t) missing fill in w i (t) ŵ B
22 Data-driven signal processing can be posed as missing data estimation problem 22 / 35 minimize w d ŵ d w ŵ 2 v (DD-SP) subject to ŵ B mpum (ŵ d ) L m,l the recovered missing values of ŵ are the desired result
23 23 / 35 Plan Example: data-driven Kalman smoothing Generalization: missing data estimation Solution approach: matrix completion
24 24 / 35 w B Hankel matrix is low-rank exact trajectory w B L m,l R 0 w(t) + R 1 w(t + 1) + + R l w(t + l) = 0 rank deficient w(1) w(2) w(t l) w(2) w(3) w(t l + 1) H (w) := w(3) w(4) w(t l + 2)... w(l + 1) w(l + 2) w(t )
25 25 / 35 relation at time t = 1 R 0 w(1) + R 1 w(2) + + R l w(l + 1) = 0 in matrix form: w(1) ] w(2) [R 0 R 1 R l. = 0 w(l + 1)
26 25 / 35 relation at time t = 2 R 0 w(2) + R 1 w(3) + + R l w(l + 2) = 0 in matrix form: w(2) ] w(3) [R 0 R 1 R l. = 0 w(l + 2)
27 25 / 35 relation at time t = T l R 0 w(t l) + R 1 w(t l + 1) + + R l w(t ) = 0 in matrix form: w(t l) ] w(t l + 1) [R 0 R 1 R l w(t l + 2) = 0. w(t )
28 26 / 35 relation for t = 1,...,T l R 0 w(t) + R 1 w(t + 1) + + R l w(t + l) = 0 in matrix form: w(1) w(2) w(t l) ] w(2) w(3) w(t l + 1) [R 0 R 1 R l w(3) w(4) w(t l + 2) = 0 }{{} R... w(l + 1) } w(l + 2) {{ w(t ) } H (w)
29 27 / 35 w B L m,l there is R R (q m) q(l+1) full row rank, such that RH (w) = 0 rank ( H (w) ) ql + m q # of variables
30 ŵ B mpum (ŵ d ) mosaic-hankel matrix is rank deficient 28 / 35 ŵ B mpum (ŵ d ) L m,l ŵ d B L m,l and ŵ B ([ ]) rank H (ŵ d ) H (ŵ) ql + m }{{} H (ŵ d,ŵ)
31 Data-driven signal processing structured low-rank approximation 29 / 35 minimize w d ŵ d w ŵ 2 v subject to ŵ B mpum (ŵ d ) L m,l minimize w ŵ v subject to rank ( H (ŵ ) ) r
32 30 / 35 Three main classes of solution methods local optimization nuclear norm relaxation subspace methods considerations generality user defined hyper parameters availability of efficient algorithms/software
33 31 / 35 Local optimization using variable projections kernel representation ( ) min min w ŵ subject to RH (ŵ) = 0 R f.r.r. ŵ variable projection (VARPRO): elimination of ŵ leads to minimize f (R) subject to R full row rank
34 32 / 35 Dealing with the "R full row rank" constraint 1. impose a quadratic equality constraint RR = I 2. using specialized methods for optimization on a manifold 3. R full row rank RΠ = [ X ] I with Π permutation Π fixed total least-squares Π can be changed during the optimization
35 33 / 35 Summary of the VARPRO approach kernel representation parameter opt. problem min ŵ,r f.r.r. w ŵ subject to RH (ŵ) = 0 elimination of ŵ optimization on a manifold min f (R) R f.r.r. in case of mosaic-hankel H, f can be evaluated fast
36 34 / 35 Conclusion combined modeling and design problem (DD-SP) we aim to find is a missing part of w B reformulation as weighted structured low-rank approx.
37 35 / 35 Future work statistical analysis computational efficiency / recursive computation other methods: subspace, convex relaxation,...
ELEC system identification workshop. Subspace methods
1 / 33 ELEC system identification workshop Subspace methods Ivan Markovsky 2 / 33 Plan 1. Behavioral approach 2. Subspace methods 3. Optimization methods 3 / 33 Outline Exact modeling Algorithms 4 / 33
More informationA structured low-rank approximation approach to system identification. Ivan Markovsky
1 / 35 A structured low-rank approximation approach to system identification Ivan Markovsky Main message: system identification SLRA minimize over B dist(a,b) subject to rank(b) r and B structured (SLRA)
More informationSparsity in system identification and data-driven control
1 / 40 Sparsity in system identification and data-driven control Ivan Markovsky This signal is not sparse in the "time domain" 2 / 40 But it is sparse in the "frequency domain" (it is weighted sum of six
More informationA missing data approach to data-driven filtering and control
1 A missing data approach to data-driven filtering and control Ivan Markovsky Abstract In filtering, control, and other mathematical engineering areas it is common to use a model-based approach, which
More informationA software package for system identification in the behavioral setting
1 / 20 A software package for system identification in the behavioral setting Ivan Markovsky Vrije Universiteit Brussel 2 / 20 Outline Introduction: system identification in the behavioral setting Solution
More informationELEC system identification workshop. Behavioral approach
1 / 33 ELEC system identification workshop Behavioral approach Ivan Markovsky 2 / 33 The course consists of lectures and exercises Session 1: behavioral approach to data modeling Session 2: subspace identification
More informationDynamic measurement: application of system identification in metrology
1 / 25 Dynamic measurement: application of system identification in metrology Ivan Markovsky Dynamic measurement takes into account the dynamical properties of the sensor 2 / 25 model of sensor as dynamical
More informationTwo well known examples. Applications of structured low-rank approximation. Approximate realisation = Model reduction. System realisation
Two well known examples Applications of structured low-rank approximation Ivan Markovsky System realisation Discrete deconvolution School of Electronics and Computer Science University of Southampton The
More informationOptimization on the Grassmann manifold: a case study
Optimization on the Grassmann manifold: a case study Konstantin Usevich and Ivan Markovsky Department ELEC, Vrije Universiteit Brussel 28 March 2013 32nd Benelux Meeting on Systems and Control, Houffalize,
More informationA comparison between structured low-rank approximation and correlation approach for data-driven output tracking
A comparison between structured low-rank approximation and correlation approach for data-driven output tracking Simone Formentin a and Ivan Markovsky b a Dipartimento di Elettronica, Informazione e Bioingegneria,
More informationUsing Hankel structured low-rank approximation for sparse signal recovery
Using Hankel structured low-rank approximation for sparse signal recovery Ivan Markovsky 1 and Pier Luigi Dragotti 2 Department ELEC Vrije Universiteit Brussel (VUB) Pleinlaan 2, Building K, B-1050 Brussels,
More informationLow-rank approximation and its applications for data fitting
Low-rank approximation and its applications for data fitting Ivan Markovsky K.U.Leuven, ESAT-SISTA A line fitting example b 6 4 2 0 data points Classical problem: Fit the points d 1 = [ 0 6 ], d2 = [ 1
More informationStructured low-rank approximation with missing data
Structured low-rank approximation with missing data Ivan Markovsky and Konstantin Usevich Department ELEC Vrije Universiteit Brussel {imarkovs,kusevich}@vub.ac.be Abstract We consider low-rank approximation
More informationImproved initial approximation for errors-in-variables system identification
Improved initial approximation for errors-in-variables system identification Konstantin Usevich Abstract Errors-in-variables system identification can be posed and solved as a Hankel structured low-rank
More informationAN IDENTIFICATION ALGORITHM FOR ARMAX SYSTEMS
AN IDENTIFICATION ALGORITHM FOR ARMAX SYSTEMS First the X, then the AR, finally the MA Jan C. Willems, K.U. Leuven Workshop on Observation and Estimation Ben Gurion University, July 3, 2004 p./2 Joint
More informationRECURSIVE SUBSPACE IDENTIFICATION IN THE LEAST SQUARES FRAMEWORK
RECURSIVE SUBSPACE IDENTIFICATION IN THE LEAST SQUARES FRAMEWORK TRNKA PAVEL AND HAVLENA VLADIMÍR Dept of Control Engineering, Czech Technical University, Technická 2, 166 27 Praha, Czech Republic mail:
More informationChapter 3. Tohru Katayama
Subspace Methods for System Identification Chapter 3 Tohru Katayama Subspace Methods Reading Group UofA, Edmonton Barnabás Póczos May 14, 2009 Preliminaries before Linear Dynamical Systems Hidden Markov
More informationLeast Squares with Examples in Signal Processing 1. 2 Overdetermined equations. 1 Notation. The sum of squares of x is denoted by x 2 2, i.e.
Least Squares with Eamples in Signal Processing Ivan Selesnick March 7, 3 NYU-Poly These notes address (approimate) solutions to linear equations by least squares We deal with the easy case wherein the
More informationIdentification of MIMO linear models: introduction to subspace methods
Identification of MIMO linear models: introduction to subspace methods Marco Lovera Dipartimento di Scienze e Tecnologie Aerospaziali Politecnico di Milano marco.lovera@polimi.it State space identification
More informationhomogeneous 71 hyperplane 10 hyperplane 34 hyperplane 69 identity map 171 identity map 186 identity map 206 identity matrix 110 identity matrix 45
address 12 adjoint matrix 118 alternating 112 alternating 203 angle 159 angle 33 angle 60 area 120 associative 180 augmented matrix 11 axes 5 Axiom of Choice 153 basis 178 basis 210 basis 74 basis test
More informationEfficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter
Efficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter arxiv:physics/0511236 v1 28 Nov 2005 Brian R. Hunt Institute for Physical Science and Technology and Department
More informationStochastic gradient descent on Riemannian manifolds
Stochastic gradient descent on Riemannian manifolds Silvère Bonnabel 1 Robotics lab - Mathématiques et systèmes Mines ParisTech Gipsa-lab, Grenoble June 20th, 2013 1 silvere.bonnabel@mines-paristech Introduction
More informationA Behavioral Approach to GNSS Positioning and DOP Determination
A Behavioral Approach to GNSS Positioning and DOP Determination Department of Communications and Guidance Engineering National Taiwan Ocean University, Keelung, TAIWAN Phone: +886-96654, FAX: +886--463349
More informationSecrets of Matrix Factorization: Supplementary Document
Secrets of Matrix Factorization: Supplementary Document Je Hyeong Hong University of Cambridge jhh37@cam.ac.uk Andrew Fitzgibbon Microsoft, Cambridge, UK awf@microsoft.com Abstract This document consists
More informationSum-of-exponentials modeling via Hankel low-rank approximation with palindromic kernel structure
Sum-of-exponentials modeling via Hankel low-rank approximation with palindromic kernel structure Ivan Markovsky and Dieter Toon Verbeke Vrije Universiteit Brussel Pleinlaan 2, 1050 Brussels, Belgium Email:
More informationLow-Rank Approximation
Ivan Markovsky Low-Rank Approximation Algorithms, Implementation, Applications September 2, 2014 Springer Preface Mathematical models are obtained from first principles (natural laws, interconnection,
More informationFiltering via Rank-Reduced Hankel Matrix CEE 690, ME 555 System Identification Fall, 2013 c H.P. Gavin, September 24, 2013
Filtering via Rank-Reduced Hankel Matrix CEE 690, ME 555 System Identification Fall, 2013 c H.P. Gavin, September 24, 2013 This method goes by the terms Structured Total Least Squares and Singular Spectrum
More informationDimension Reduction. David M. Blei. April 23, 2012
Dimension Reduction David M. Blei April 23, 2012 1 Basic idea Goal: Compute a reduced representation of data from p -dimensional to q-dimensional, where q < p. x 1,...,x p z 1,...,z q (1) We want to do
More informationA Log-Frequency Approach to the Identification of the Wiener-Hammerstein Model
A Log-Frequency Approach to the Identification of the Wiener-Hammerstein Model The MIT Faculty has made this article openly available Please share how this access benefits you Your story matters Citation
More informationSignal Identification Using a Least L 1 Norm Algorithm
Optimization and Engineering, 1, 51 65, 2000 c 2000 Kluwer Academic Publishers. Manufactured in The Netherlands. Signal Identification Using a Least L 1 Norm Algorithm J. BEN ROSEN Department of Computer
More informationEL 625 Lecture 10. Pole Placement and Observer Design. ẋ = Ax (1)
EL 625 Lecture 0 EL 625 Lecture 0 Pole Placement and Observer Design Pole Placement Consider the system ẋ Ax () The solution to this system is x(t) e At x(0) (2) If the eigenvalues of A all lie in the
More informationI. D. Landau, A. Karimi: A Course on Adaptive Control Adaptive Control. Part 9: Adaptive Control with Multiple Models and Switching
I. D. Landau, A. Karimi: A Course on Adaptive Control - 5 1 Adaptive Control Part 9: Adaptive Control with Multiple Models and Switching I. D. Landau, A. Karimi: A Course on Adaptive Control - 5 2 Outline
More information6.241 Dynamic Systems and Control
6.241 Dynamic Systems and Control Lecture 24: H2 Synthesis Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology May 4, 2011 E. Frazzoli (MIT) Lecture 24: H 2 Synthesis May
More informationIntroduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p.
Preface p. xiii Acknowledgment p. xix Introduction p. 1 Fundamental Problems p. 2 Core of Fundamental Theory and General Mathematical Ideas p. 3 Classical Statistical Decision p. 4 Bayes Decision p. 5
More informationSparsity Models. Tong Zhang. Rutgers University. T. Zhang (Rutgers) Sparsity Models 1 / 28
Sparsity Models Tong Zhang Rutgers University T. Zhang (Rutgers) Sparsity Models 1 / 28 Topics Standard sparse regression model algorithms: convex relaxation and greedy algorithm sparse recovery analysis:
More information1-Bit Matrix Completion
1-Bit Matrix Completion Mark A. Davenport School of Electrical and Computer Engineering Georgia Institute of Technology Yaniv Plan Mary Wootters Ewout van den Berg Matrix Completion d When is it possible
More informationIntroduction to Machine Learning
Introduction to Machine Learning Brown University CSCI 1950-F, Spring 2012 Prof. Erik Sudderth Lecture 25: Markov Chain Monte Carlo (MCMC) Course Review and Advanced Topics Many figures courtesy Kevin
More informationChapter 5. Standard LTI Feedback Optimization Setup. 5.1 The Canonical Setup
Chapter 5 Standard LTI Feedback Optimization Setup Efficient LTI feedback optimization algorithms comprise a major component of modern feedback design approach: application problems involving complex models
More informationNOISE ROBUST RELATIVE TRANSFER FUNCTION ESTIMATION. M. Schwab, P. Noll, and T. Sikora. Technical University Berlin, Germany Communication System Group
NOISE ROBUST RELATIVE TRANSFER FUNCTION ESTIMATION M. Schwab, P. Noll, and T. Sikora Technical University Berlin, Germany Communication System Group Einsteinufer 17, 1557 Berlin (Germany) {schwab noll
More informationStatistics 910, #15 1. Kalman Filter
Statistics 910, #15 1 Overview 1. Summary of Kalman filter 2. Derivations 3. ARMA likelihoods 4. Recursions for the variance Kalman Filter Summary of Kalman filter Simplifications To make the derivations
More informationSGN Advanced Signal Processing Project bonus: Sparse model estimation
SGN 21006 Advanced Signal Processing Project bonus: Sparse model estimation Ioan Tabus Department of Signal Processing Tampere University of Technology Finland 1 / 12 Sparse models Initial problem: solve
More informationMitigating the Effect of Noise in Iterative Projection Phase Retrieval
Mitigating the Effect of Noise in Iterative Projection Phase Retrieval & David Hyland Imaging and Applied Optics: Optics and Photonics Congress Introduction Filtering Results Conclusion Outline Introduction
More informationTime Series Prediction by Kalman Smoother with Cross-Validated Noise Density
Time Series Prediction by Kalman Smoother with Cross-Validated Noise Density Simo Särkkä E-mail: simo.sarkka@hut.fi Aki Vehtari E-mail: aki.vehtari@hut.fi Jouko Lampinen E-mail: jouko.lampinen@hut.fi Abstract
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationData-Enabled Predictive Control : In the Shallows of the DeePC
Data-Enabled Predictive Control : In the Shallows of the DeePC Florian Do rfler Automatic Control Laboratory, ETH Zu rich Acknowledgements Linbin Huang Jeremy Coulson Brain-storming: P. Mohajerin Esfahani,
More informationEstimation, Detection, and Identification CMU 18752
Estimation, Detection, and Identification CMU 18752 Graduate Course on the CMU/Portugal ECE PhD Program Spring 2008/2009 Instructor: Prof. Paulo Jorge Oliveira pjcro @ isr.ist.utl.pt Phone: +351 21 8418053
More informationSYSTEMTEORI - KALMAN FILTER VS LQ CONTROL
SYSTEMTEORI - KALMAN FILTER VS LQ CONTROL 1. Optimal regulator with noisy measurement Consider the following system: ẋ = Ax + Bu + w, x(0) = x 0 where w(t) is white noise with Ew(t) = 0, and x 0 is a stochastic
More informationIntroduction to Optimal Control Theory and Hamilton-Jacobi equations. Seung Yeal Ha Department of Mathematical Sciences Seoul National University
Introduction to Optimal Control Theory and Hamilton-Jacobi equations Seung Yeal Ha Department of Mathematical Sciences Seoul National University 1 A priori message from SYHA The main purpose of these series
More informationObservability and state estimation
EE263 Autumn 2015 S Boyd and S Lall Observability and state estimation state estimation discrete-time observability observability controllability duality observers for noiseless case continuous-time observability
More informationECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 7: Matrix completion Yuejie Chi The Ohio State University Page 1 Reference Guaranteed Minimum-Rank Solutions of Linear
More informationy k = ( ) x k + v k. w q wk i 0 0 wk
Four telling examples of Kalman Filters Example : Signal plus noise Measurement of a bandpass signal, center frequency.2 rad/sec buried in highpass noise. Dig out the quadrature part of the signal while
More informationEfficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter
arxiv:physics/0511236v2 [physics.data-an] 29 Dec 2006 Efficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter Brian R. Hunt Institute for Physical Science and Technology
More informationRealization and identification of autonomous linear periodically time-varying systems
Realization and identification of autonomous linear periodically time-varying systems Ivan Markovsky, Jan Goos, Konstantin Usevich, and Rik Pintelon Department ELEC, Vrije Universiteit Brussel, Pleinlaan
More informationSignal theory: Part 1
Signal theory: Part 1 Ivan Markovsky\ Vrije Universiteit Brussel Contents 1 Introduction 2 2 The behavioral approach 3 2.1 Definition of a system....................................... 3 2.2 Dynamical
More informationLecture 19 Observability and state estimation
EE263 Autumn 2007-08 Stephen Boyd Lecture 19 Observability and state estimation state estimation discrete-time observability observability controllability duality observers for noiseless case continuous-time
More informationImage restoration. An example in astronomy
Image restoration Convex approaches: penalties and constraints An example in astronomy Jean-François Giovannelli Groupe Signal Image Laboratoire de l Intégration du Matériau au Système Univ. Bordeaux CNRS
More informationRobust Subspace System Identification via Weighted Nuclear Norm Optimization
Robust Subspace System Identification via Weighted Nuclear Norm Optimization Dorsa Sadigh Henrik Ohlsson, S. Shankar Sastry Sanjit A. Seshia University of California, Berkeley, Berkeley, CA 9470 USA. {dsadigh,ohlsson,sastry,sseshia}@eecs.berkeley.edu
More informationHow data assimilation helps to illuminate complex biology
How data assimilation helps to illuminate complex biology The dynamic elastic-net Maik Kschischo Department of Mathematics and Technology University of Applied Sciences Koblenz RheinAhrCampus Remagen Joseph-Rovan-Allee
More informationThe Essentials of Linear State-Space Systems
:or-' The Essentials of Linear State-Space Systems J. Dwight Aplevich GIFT OF THE ASIA FOUNDATION NOT FOR RE-SALE John Wiley & Sons, Inc New York Chichester Weinheim OAI HOC OUOC GIA HA N^l TRUNGTAMTHANCTINTHUVIIN
More informationA Constraint Generation Approach to Learning Stable Linear Dynamical Systems
A Constraint Generation Approach to Learning Stable Linear Dynamical Systems Sajid M. Siddiqi Byron Boots Geoffrey J. Gordon Carnegie Mellon University NIPS 2007 poster W22 steam Application: Dynamic Textures
More informationParameter Estimation in a Moving Horizon Perspective
Parameter Estimation in a Moving Horizon Perspective State and Parameter Estimation in Dynamical Systems Reglerteknik, ISY, Linköpings Universitet State and Parameter Estimation in Dynamical Systems OUTLINE
More information10. Rank-nullity Definition Let A M m,n (F ). The row space of A is the span of the rows. The column space of A is the span of the columns.
10. Rank-nullity Definition 10.1. Let A M m,n (F ). The row space of A is the span of the rows. The column space of A is the span of the columns. The nullity ν(a) of A is the dimension of the kernel. The
More informationSparse Regularization via Convex Analysis
Sparse Regularization via Convex Analysis Ivan Selesnick Electrical and Computer Engineering Tandon School of Engineering New York University Brooklyn, New York, USA 29 / 66 Convex or non-convex: Which
More informationDesign of Norm-Optimal Iterative Learning Controllers: The Effect of an Iteration-Domain Kalman Filter for Disturbance Estimation
Design of Norm-Optimal Iterative Learning Controllers: The Effect of an Iteration-Domain Kalman Filter for Disturbance Estimation Nicolas Degen, Autonomous System Lab, ETH Zürich Angela P. Schoellig, University
More information1-Bit Matrix Completion
1-Bit Matrix Completion Mark A. Davenport School of Electrical and Computer Engineering Georgia Institute of Technology Yaniv Plan Mary Wootters Ewout van den Berg Matrix Completion d When is it possible
More informationFurther Results on Model Structure Validation for Closed Loop System Identification
Advances in Wireless Communications and etworks 7; 3(5: 57-66 http://www.sciencepublishinggroup.com/j/awcn doi:.648/j.awcn.735. Further esults on Model Structure Validation for Closed Loop System Identification
More informationAccelerated Block-Coordinate Relaxation for Regularized Optimization
Accelerated Block-Coordinate Relaxation for Regularized Optimization Stephen J. Wright Computer Sciences University of Wisconsin, Madison October 09, 2012 Problem descriptions Consider where f is smooth
More informationThree Generalizations of Compressed Sensing
Thomas Blumensath School of Mathematics The University of Southampton June, 2010 home prev next page Compressed Sensing and beyond y = Φx + e x R N or x C N x K is K-sparse and x x K 2 is small y R M or
More informationSystem Identification by Nuclear Norm Minimization
Dept. of Information Engineering University of Pisa (Italy) System Identification by Nuclear Norm Minimization eng. Sergio Grammatico grammatico.sergio@gmail.com Class of Identification of Uncertain Systems
More informationStochastic gradient descent on Riemannian manifolds
Stochastic gradient descent on Riemannian manifolds Silvère Bonnabel 1 Centre de Robotique - Mathématiques et systèmes Mines ParisTech SMILE Seminar Mines ParisTech Novembre 14th, 2013 1 silvere.bonnabel@mines-paristech
More informationREGULARIZED STRUCTURED LOW-RANK APPROXIMATION WITH APPLICATIONS
REGULARIZED STRUCTURED LOW-RANK APPROXIMATION WITH APPLICATIONS MARIYA ISHTEVA, KONSTANTIN USEVICH, AND IVAN MARKOVSKY Abstract. We consider the problem of approximating an affinely structured matrix,
More informationRecovery of Low-Rank Plus Compressed Sparse Matrices with Application to Unveiling Traffic Anomalies
July 12, 212 Recovery of Low-Rank Plus Compressed Sparse Matrices with Application to Unveiling Traffic Anomalies Morteza Mardani Dept. of ECE, University of Minnesota, Minneapolis, MN 55455 Acknowledgments:
More informationStructured matrix factorizations. Example: Eigenfaces
Structured matrix factorizations Example: Eigenfaces An extremely large variety of interesting and important problems in machine learning can be formulated as: Given a matrix, find a matrix and a matrix
More informationStatistical Geometry Processing Winter Semester 2011/2012
Statistical Geometry Processing Winter Semester 2011/2012 Linear Algebra, Function Spaces & Inverse Problems Vector and Function Spaces 3 Vectors vectors are arrows in space classically: 2 or 3 dim. Euclidian
More informationIEOR 265 Lecture 3 Sparse Linear Regression
IOR 65 Lecture 3 Sparse Linear Regression 1 M Bound Recall from last lecture that the reason we are interested in complexity measures of sets is because of the following result, which is known as the M
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationChap. 3. Controlled Systems, Controllability
Chap. 3. Controlled Systems, Controllability 1. Controllability of Linear Systems 1.1. Kalman s Criterion Consider the linear system ẋ = Ax + Bu where x R n : state vector and u R m : input vector. A :
More informationDeep Linear Networks with Arbitrary Loss: All Local Minima Are Global
homas Laurent * 1 James H. von Brecht * 2 Abstract We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima
More informationStructured Matrices and Solving Multivariate Polynomial Equations
Structured Matrices and Solving Multivariate Polynomial Equations Philippe Dreesen Kim Batselier Bart De Moor KU Leuven ESAT/SCD, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium. Structured Matrix Days,
More informationBlind Deconvolution Using Convex Programming. Jiaming Cao
Blind Deconvolution Using Convex Programming Jiaming Cao Problem Statement The basic problem Consider that the received signal is the circular convolution of two vectors w and x, both of length L. How
More informationOn-off Control: Audio Applications
On-off Control: Audio Applications Graham C. Goodwin Day 4: Lecture 3 16th September 2004 International Summer School Grenoble, France 1 Background In this lecture we address the issue of control when
More informationSparse Approximation and Variable Selection
Sparse Approximation and Variable Selection Lorenzo Rosasco 9.520 Class 07 February 26, 2007 About this class Goal To introduce the problem of variable selection, discuss its connection to sparse approximation
More information4. DATA ASSIMILATION FUNDAMENTALS
4. DATA ASSIMILATION FUNDAMENTALS... [the atmosphere] "is a chaotic system in which errors introduced into the system can grow with time... As a consequence, data assimilation is a struggle between chaotic
More informationNoisy Word Recognition Using Denoising and Moment Matrix Discriminants
Noisy Word Recognition Using Denoising and Moment Matrix Discriminants Mila Nikolova Département TSI ENST, rue Barrault, 753 Paris Cedex 13, France, nikolova@tsi.enst.fr Alfred Hero Dept. of EECS, Univ.
More informationCamera Calibration The purpose of camera calibration is to determine the intrinsic camera parameters (c 0,r 0 ), f, s x, s y, skew parameter (s =
Camera Calibration The purpose of camera calibration is to determine the intrinsic camera parameters (c 0,r 0 ), f, s x, s y, skew parameter (s = cotα), and the lens distortion (radial distortion coefficient
More informationMissing dependent variables in panel data models
Missing dependent variables in panel data models Jason Abrevaya Abstract This paper considers estimation of a fixed-effects model in which the dependent variable may be missing. For cross-sectional units
More informationOWL to the rescue of LASSO
OWL to the rescue of LASSO IISc IBM day 2018 Joint Work R. Sankaran and Francis Bach AISTATS 17 Chiranjib Bhattacharyya Professor, Department of Computer Science and Automation Indian Institute of Science,
More informationStochastic Realization of Binary Exchangeable Processes
Stochastic Realization of Binary Exchangeable Processes Lorenzo Finesso and Cecilia Prosdocimi Abstract A discrete time stochastic process is called exchangeable if its n-dimensional distributions are,
More informationDerivation of the Kalman Filter
Derivation of the Kalman Filter Kai Borre Danish GPS Center, Denmark Block Matrix Identities The key formulas give the inverse of a 2 by 2 block matrix, assuming T is invertible: T U 1 L M. (1) V W N P
More informationThe 1d Kalman Filter. 1 Understanding the forward model. Richard Turner
The d Kalman Filter Richard Turner This is a Jekyll and Hyde of a document and should really be split up. We start with Jekyll which contains a very short derivation for the d Kalman filter, the purpose
More informationTemporal-Difference Q-learning in Active Fault Diagnosis
Temporal-Difference Q-learning in Active Fault Diagnosis Jan Škach 1 Ivo Punčochář 1 Frank L. Lewis 2 1 Identification and Decision Making Research Group (IDM) European Centre of Excellence - NTIS University
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More informationSchur parametrizations and balanced realizations of real discrete-time stable all-pass systems.
1 Schur parametrizations and balanced realizations of real discrete-time stable all-pass systems Martine Olivi Jean-Paul Marmorat Bernard Hanzon Ralf LM Peeters Abstract We investigate the parametrization
More informationGI07/COMPM012: Mathematical Programming and Research Methods (Part 2) 2. Least Squares and Principal Components Analysis. Massimiliano Pontil
GI07/COMPM012: Mathematical Programming and Research Methods (Part 2) 2. Least Squares and Principal Components Analysis Massimiliano Pontil 1 Today s plan SVD and principal component analysis (PCA) Connection
More informationKrylov subspace iterative methods for nonsymmetric discrete ill-posed problems in image restoration
Krylov subspace iterative methods for nonsymmetric discrete ill-posed problems in image restoration D. Calvetti a, B. Lewis b and L. Reichel c a Department of Mathematics, Case Western Reserve University,
More informationStructured State Space Realizations for SLS Distributed Controllers
Structured State Space Realizations for SLS Distributed Controllers James Anderson and Nikolai Matni Abstract In recent work the system level synthesis (SLS) paradigm has been shown to provide a truly
More informationCSE446: non-parametric methods Spring 2017
CSE446: non-parametric methods Spring 2017 Ali Farhadi Slides adapted from Carlos Guestrin and Luke Zettlemoyer Linear Regression: What can go wrong? What do we do if the bias is too strong? Might want
More informationSimplex Algorithm Using Canonical Tableaus
41 Simplex Algorithm Using Canonical Tableaus Consider LP in standard form: Min z = cx + α subject to Ax = b where A m n has rank m and α is a constant In tableau form we record it as below Original Tableau
More informationRank reduction of parameterized time-dependent PDEs
Rank reduction of parameterized time-dependent PDEs A. Spantini 1, L. Mathelin 2, Y. Marzouk 1 1 AeroAstro Dpt., MIT, USA 2 LIMSI-CNRS, France UNCECOMP 2015 (MIT & LIMSI-CNRS) Rank reduction of parameterized
More informationLecture 4 Noisy Channel Coding
Lecture 4 Noisy Channel Coding I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw October 9, 2015 1 / 56 I-Hsiang Wang IT Lecture 4 The Channel Coding Problem
More information