Parameter Selection Techniques and Surrogate Models


 Homer Hamilton
 10 months ago
 Views:
Transcription
1 Parameter Selection Techniques and Surrogate Models Model Reduction: Will discuss two forms Parameter space reduction Surrogate models to reduce model complexity Input Representation Local Sensitivity Analysis Global Sensitivity Analysis Parameter Selection Model Discrepancy Surrogate Models Sparse Grids Model Calibration Stochastic Spectral Methods Uncertainty Propagation Sparse Grids
2 Parameter Selection Techniques and Surrogate Models Parameter Space Reduction: SIR Model ds = N  S  kis, S() =S dt di dt = kis  (r + )I, I() =I dr = ri  R, R() =R dt Susceptible Infectious Recovered Parameters: Response: : Infection coefficient k: Interaction coefficient r: Recovery rate : Birth/death rate y = Z 5 R(t, q)dt Note: Parameters q =[, k, r, ] not uniquely determined by data
3 Parameter Selection Techniques and Surrogate Models First Issue: Parameters often not identifiable in the sense that they are not uniquely determined by the data. Example: Spring model m d 2 z dt 2 Problem: z() =z, + c dz dt + kz = f cos(! F t) dz dt () =z 1 Parameters q =[m, c, k, f ] and q = 1, c m, k m, f m yield same displacements
4 Parameter Selection Techniques and Surrogate Models First Issue: Parameters often not identifiable in the sense that they are not uniquely determined by the data. Example: Spring model m d 2 z dt 2 Problem: z() =z, Solution: Reformulate problem as d 2 z dt 2 + C dz dt + Kz = F cos(! F t) z() =z, + c dz dt + kz = f cos(! F t) dz dt () =z 1 Parameters q =[m, c, k, f ] and q = 1, c m, k m, f m yield same displacements dz dt () =z 1 where C = c m, K = k m and F = f m Techniques for General Models: Linear algebra analysis; e.g., SVD or QR algorithms Identifiability analysis: M. Eisenberg Sensitivity analysis: M. Hyman Active Subspaces: Present discussion Surrogates: Present and later talks
5 Parameter Selection Techniques and Surrogate Models Second Issue: Coupled compartment can have 1 s of nonidentifiable parameters Example: Metabolism Techniques for General Models: Identifiability and sensitivity analysis: M. Eisenberg, M. Hyman Active Subspaces: Present discussion
6 Example: Varies most in [.7,.3] direction No variation in orthogonal direction Active Subspaces Note: Functions may vary significantly in only a few directions Active directions may be linear combination of inputs y = exp(.7q 1 +.3q 2 ) Strategy: Linearly parameterized problems: Employ SVD or QR decomposition. Nonlinear problems: Construct approximate gradient matrix and employ SVD or QR. q2 q 1
7 Parameter Space Reduction Techniques: Linear Problems Second Issue: Models depends on very large number of parameters e.g., millions but only a few are significant. Linear Algebra Techniques: Linearly parameterized problems y = Aq, q 2 R p,y2 R m Singular Value Decomposition (SVD): Rank Revealing QR Decomposition: Problem: Neither is directly applicable when m or p are very large; e.g., millions. Solution: Random range finding algorithms.
8 Random Range Finding Algorithms: Linear Problems Algorithm: Halko, Martinsson and Tropp, SIAM Review, 211 Example: y i = px q k sin(2 kt i ),i=1,,m k=1
9 Random Range Finding Algorithms: Linear Problems Example: m = 11, p = 1: Analytic value for rank is 49 Column Entries of A A(:,1) A(:,11) A(:,21) A(:,91) Time (s) Column Entries of A Aliasing A(:,1) A(:,51) Time (s).8 1 Singular Values Singular Values Deterministic Algorithm Random Algorithm Absolute Difference in Singular Values Absolute Difference Between Singular Values Example: m = 11, p = 1,,: Random algorithm still viable
10 Random Range Finding Algorithms: Linear Problems Example: m = 11, p = 1 Column Entries of A A(:,1) A(:,11) A(:,21) A(:,91) Time (s) Column Entries of A A(:,1) A(:,51) Time (s).8 1 Aliasing Later: Reproduce these results using the code alias.m. You can experiment with the size of the random sample. See how far you can increase the dimension. Singular Values Deterministic Algorithm Random Algorithm Singular Values Absolute Difference in Singular Values Absolute Difference Between Singular Values
11 Example: Varies most in [.7,.3] direction No variation in orthogonal direction Active Subspaces Note: Functions may vary significantly in only a few directions Active directions may be linear combination of inputs y = exp(.7q 1 +.3q 2 ) Strategy: Linearly parameterized problems: Employ SVD or QR decomposition. Nonlinear problems: Construct approximate gradient matrix and employ SVD or QR. q2 q 1
12 GradientBased Active Subspace Construction Active Subspace: See [Constantine, SIAM 215]. Consider f = f (q), q 2 Q R p and r q f (q) = Construct outer product C @q p Z (r q f )(r q f ) T dq T (q): Distribution of input parameters q Partition eigenvalues: C = W W T = apple 1 Rotated Coordinates: 2, W =[W 1 W 2 ] y = W T 1 q 2 Rn and z = W T 2 q 2 Rpn Active Variables Active Subspace: Range of eigenvectors in W 1
13 GradientBased Active Subspace Construction Active Subspace: Construction based on random sampling 1. Draw M independent samples {q j } from 2. Evaluate r q f j = r q f (q j ) 3. Approximate outer product C e C = 1 M MX (r q f j )(r q f j ) T j=1 Note: e C = GG T where G = 1 p M [r q f 1,..., r q f M ] 4. Take SVD of G = W p V T Monte Carlo Quadrature: More on Wednesday Active subspace of dimension n is first n columns of W Research: Develop algorithms for codes that do not have derivative capabilities Note: Finitedifference approximations tempting but not very effective Strategy: Algorithms based on linearization and sampling in active directions.
14 GradientBased Active Subspace Construction Example: y = exp(.7q 1 +.3q 2 ) Active Subspace: For gradient matrix G, form SVD G = U V T Eigenvalue spectrum indicates 1D active subspace with basis U(:, 1) = [.91,.39] g(y) Response Surface Testing Points y Exercise: Construct a gradient matrix G and evaluate it at random points in the interval [ 1, 1] [ 1, 1] Now take the SVD to determine a basis for the active subspace. How might you construct a response surface g(y) on the 1D subspace y?
15 Later Talk: C. Webster Steps in Uncertainty Quantification
16 Surrogate Models: Motivation SinkoStreifer Model Notation and Assumptions: u (t, x ): Number of fish of size x at time t µ: Death rate Growth rate of samesized individuals is same and denoted by g (t, x ) dx = g (t, x ) dt Flux Balance: u (t, x ) is a density and rate is (t, x ) = g (t, x )u (t, @ ) + = µu + = Model: ) + Z g (t, x )u (t, x ) u (, x ) = x =x (x ) = x1 x k (t, )u (t, )d
17 Example: Consider the model Surrogate = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 u(t, x, y, z; q)dxdydzdt t 1 x, y, z Notes: Requires approximation of PDE in 3D What would be a simple surrogate?
18 Recall: Consider = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 Surrogate Models u(t, x, y, z; q)dxdydzdt t Surrogate: Quadratic 1 x, y, z y s (q) =(q.25) Question: How do you construct a polynomial surrogate? Regression Interpolation
19 Recall: Consider = f(q) with the response y(q) = Boundary Conditions Initial Conditions Z 1 Z 1 Z 1 Z 1 Surrogate Models u(t, x, y, z; q)dxdydzdt t Surrogate: Quadratic 1 x, y, z y s (q) =(q.25) Question: How do you construct a polynomial surrogate? Interpolation Regression Statistical Model: y s (q): Emulator for y(q) y m = y s (q m )+" m,m=1,...,m M=7 k=2
20 Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k  2 log[ (y q)] Bayesian Information Criterion (AIC) BIC = k log(m)  2 log[ (y q)] Likelihood: 1 2 (y q) = Maximize (2 2 ) M/2 ess q/2 MX SS q = [y m  y s (q m )] 2 Minimize m=1 Later Talks: M. Eisenberg, B. Fitzpatrick, R. Smith M=7 k=2
21 Surrogate Models Question: How do we keep from fitting noise? Akaike Information Criterion (AIC) AIC = 2k  2 log[ (y q)] Bayesian Information Criterion (AIC) Example: Exercise: BIC = k log(m)  2 log[ (y q)] Likelihood: 1 2 (y q) = (2 2 ) M/2 ess q/2 MX SS q = [y m  y s (q m )] 2 m=1 y = exp(.7q 1 +.3q 2 ) Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate? Maximize Minimize q2 q 1 M=7 k=2
22 Example: m = 11, p = 1 y i = Code: Computer Tutorials px q k sin(2 kt i ),i=1,,m k=1 Reproduce these results using the code alias.m. You can experiment with the size of the random sample and tol. See how far you can increase the dimension. Column Entries of A A(:,1) A(:,51) Time (s).8 1 Example: y = exp(.7q 1 +.3q 2 ) Construct a polynomial surrogate using the code response_surface.m. What order seems appropriate. Construct a gradient matrix G and evaluate it at random points in the interval [ 1, 1] [ 1, 1] q2 Take SVD to determine basis for the active subspace. q 1
Spectral Representation of Random Processes
Spectral Representation of Random Processes Example: Represent u(t,x,q) by! u K (t, x, Q) = u k (t, x) k(q) where k(q) are orthogonal polynomials. Single Random Variable:! Let k (Q) be orthogonal with
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2
MA 575 Linear Models: Cedric E. Ginestet, Boston University Regularization: Ridge Regression and Lasso Week 14, Lecture 2 1 Ridge Regression Ridge regression and the Lasso are two forms of regularized
More informationDSGA 1002 Lecture notes 10 November 23, Linear models
DSGA 2 Lecture notes November 23, 2 Linear functions Linear models A linear model encodes the assumption that two quantities are linearly related. Mathematically, this is characterized using linear functions.
More informationBackground Mathematics (2/2) 1. David Barber
Background Mathematics (2/2) 1 David Barber University College London Modified by Samson Cheung (sccheung@ieee.org) 1 These slides accompany the book Bayesian Reasoning and Machine Learning. The book and
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationStochastic Collocation Methods for Polynomial Chaos: Analysis and Applications
Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Dongbin Xiu Department of Mathematics, Purdue University Support: AFOSR FA95581353 (Computational Math) SF CAREER DMS64535
More informationQuadratic forms. Here. Thus symmetric matrices are diagonalizable, and the diagonalization can be performed by means of an orthogonal matrix.
Quadratic forms 1. Symmetric matrices An n n matrix (a ij ) n ij=1 with entries on R is called symmetric if A T, that is, if a ij = a ji for all 1 i, j n. We denote by S n (R) the set of all n n symmetric
More informationStat 451 Lecture Notes Numerical Integration
Stat 451 Lecture Notes 03 12 Numerical Integration Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapter 5 in Givens & Hoeting, and Chapters 4 & 18 of Lange 2 Updated: February 11, 2016 1 / 29
More informationTENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY
TENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY Eugene Tyrtyshnikov Institute of Numerical Mathematics Russian Academy of Sciences (joint work with Ivan Oseledets) WHAT ARE TENSORS? Tensors
More informationrandomized block krylov methods for stronger and faster approximate svd
randomized block krylov methods for stronger and faster approximate svd Cameron Musco and Christopher Musco December 2, 25 Massachusetts Institute of Technology, EECS singular value decomposition n d left
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationLINEAR ALGEBRA (PMTH213) Tutorial Questions
Tutorial Questions The tutorial exercises range from revision and routine practice, through lling in details in the notes, to applications of the theory. While the tutorial problems are not compulsory,
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationarxiv: v2 [math.na] 5 Dec 2013
ACTIVE SUBSPACE METHODS IN THEORY AND PRACTICE: APPLICATIONS TO KRIGING SURFACES PAUL G. CONSTANTINE, ERIC DOW, AND QIQI WANG arxiv:1304.2070v2 [math.na 5 Dec 2013 Abstract. Many multivariate functions
More informationI. Multiple Choice Questions (Answer any eight)
Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam  Course Instructor : Prashanth L.A. Date : Sep24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY
More informationLinear Models 1. Isfahan University of Technology Fall Semester, 2014
Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and
More informationA Tour of the Lanczos Algorithm and its Convergence Guarantees through the Decades
A Tour of the Lanczos Algorithm and its Convergence Guarantees through the Decades Qiaochu Yuan Department of Mathematics UC Berkeley Joint work with Prof. Ming Gu, Bo Li April 17, 2018 Qiaochu Yuan Tour
More information3. Identify and find the general solution of each of the following first order differential equations.
Final Exam MATH 33, Sample Questions. Fall 6. y = Cx 3 3 is the general solution of a differential equation. Find the equation. Answer: y = 3y + 9 xy. y = C x + C is the general solution of a differential
More informationDSGA 1002 Lecture notes 12 Fall Linear regression
DSGA Lecture notes 1 Fall 16 1 Linear models Linear regression In statistics, regression consists of learning a function relating a certain quantity of interest y, the response or dependent variable,
More informationProposition 42. Let M be an m n matrix. Then (32) N (M M)=N (M) (33) R(MM )=R(M)
RODICA D. COSTIN. Singular Value Decomposition.1. Rectangular matrices. For rectangular matrices M the notions of eigenvalue/vector cannot be defined. However, the products MM and/or M M (which are square,
More informationPreliminary Examination in Numerical Analysis
Department of Applied Mathematics Preliminary Examination in Numerical Analysis August 7, 06, 0 am pm. Submit solutions to four (and no more) of the following six problems. Show all your work, and justify
More information1. Select the unique answer (choice) for each problem. Write only the answer.
MATH 5 Practice Problem Set Spring 7. Select the unique answer (choice) for each problem. Write only the answer. () Determine all the values of a for which the system has infinitely many solutions: x +
More informationTheorems. Least squares regression
Theorems In this assignment we are trying to classify AML and ALL samples by use of penalized logistic regression. Before we indulge on the adventure of classification we should first explain the most
More informationPositive Definite Matrix
1/29 ChiaPing Chen Professor Department of Computer Science and Engineering National Sun Yatsen University Linear Algebra Positive Definite, Negative Definite, Indefinite 2/29 Pure Quadratic Function
More informationLecture 4 and 5 Controllability and Observability: Kalman decompositions
1 Lecture 4 and 5 Controllability and Observability: Kalman decompositions Spring 2013  EE 194, Advanced Control (Prof. Khan) January 30 (Wed.) and Feb. 04 (Mon.), 2013 I. OBSERVABILITY OF DT LTI SYSTEMS
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationLinear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions
Linear Algebra, part 2 Eigenvalues, eigenvectors and least squares solutions AnnaKarin Tornberg Mathematical Models, Analysis and Simulation Fall semester, 2013 Main problem of linear algebra 2: Given
More informationCS168: The Modern Algorithmic Toolbox Lecture #8: How PCA Works
CS68: The Modern Algorithmic Toolbox Lecture #8: How PCA Works Tim Roughgarden & Gregory Valiant April 20, 206 Introduction Last lecture introduced the idea of principal components analysis (PCA). The
More informationCalculating determinants for larger matrices
Day 26 Calculating determinants for larger matrices We now proceed to define det A for n n matrices A As before, we are looking for a function of A that satisfies the product formula det(ab) = det A det
More informationFinal Exam, Machine Learning, Spring 2009
Name: Andrew ID: Final Exam, 10701 Machine Learning, Spring 2009  The exam is openbook, opennotes, no electronics other than calculators.  The maximum possible score on this exam is 100. You have 3
More information10725/36725: Convex Optimization Prerequisite Topics
10725/36725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the
More informationMath 224, Fall 2007 Exam 3 Thursday, December 6, 2007
Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007 You have 1 hour and 20 minutes. No notes, books, or other references. You are permitted to use Maple during this exam, but you must start with a blank
More information(a)
Chapter 8 Subspace Methods 8. Introduction Principal Component Analysis (PCA) is applied to the analysis of time series data. In this context we discuss measures of complexity and subspace methods for
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationNumerical Mathematics
Alfio Quarteroni Riccardo Sacco Fausto Saleri Numerical Mathematics Second Edition With 135 Figures and 45 Tables 421 Springer Contents Part I Getting Started 1 Foundations of Matrix Analysis 3 1.1 Vector
More informationCollaborative Filtering: A Machine Learning Perspective
Collaborative Filtering: A Machine Learning Perspective Chapter 6: Dimensionality Reduction Benjamin Marlin Presenter: Chaitanya Desai Collaborative Filtering: A Machine Learning Perspective p.1/18 Topics
More informationMatrix Algebra, part 2
Matrix Algebra, part 2 MingChing Luoh 2005.9.12 1 / 38 Diagonalization and Spectral Decomposition of a Matrix Optimization 2 / 38 Diagonalization and Spectral Decomposition of a Matrix Also called Eigenvalues
More informationFINITEDIMENSIONAL LINEAR ALGEBRA
DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITEDIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More information18.06 Problem Set 8  Solutions Due Wednesday, 14 November 2007 at 4 pm in
806 Problem Set 8  Solutions Due Wednesday, 4 November 2007 at 4 pm in 206 08 03 Problem : 205+5+5+5 Consider the matrix A 02 07 a Check that A is a positive Markov matrix, and find its steady state
More informationAn Empirical Chaos Expansion Method for Uncertainty Quantification
An Empirical Chaos Expansion Method for Uncertainty Quantification Melvin Leok and Gautam Wilkins Abstract. Uncertainty quantification seeks to provide a quantitative means to understand complex systems
More informationModeling Data with Linear Combinations of Basis Functions. Read Chapter 3 in the text by Bishop
Modeling Data with Linear Combinations of Basis Functions Read Chapter 3 in the text by Bishop A Type of Supervised Learning Problem We want to model data (x 1, t 1 ),..., (x N, t N ), where x i is a vector
More informationCS 231A Section 1: Linear Algebra & Probability Review
CS 231A Section 1: Linear Algebra & Probability Review 1 Topics Support Vector Machines Boosting ViolaJones face detector Linear Algebra Review Notation Operations & Properties Matrix Calculus Probability
More informationCS 231A Section 1: Linear Algebra & Probability Review. Kevin Tang
CS 231A Section 1: Linear Algebra & Probability Review Kevin Tang Kevin Tang Section 11 9/30/2011 Topics Support Vector Machines Boosting Viola Jones face detector Linear Algebra Review Notation Operations
More informationHigh Performance Nonlinear Solvers
What is a nonlinear system? High Performance Nonlinear Solvers Michael McCourt Division Argonne National Laboratory IIT Meshfree Seminar September 19, 2011 Every nonlinear system of equations can be described
More informationRandomized algorithms for the approximation of matrices
Randomized algorithms for the approximation of matrices Luis Rademacher The Ohio State University Computer Science and Engineering (joint work with Amit Deshpande, Santosh Vempala, Grant Wang) Two topics
More informationDifferential equations
Differential equations Math 7 Spring Practice problems for April Exam Problem Use the method of elimination to find the xcomponent of the general solution of x y = 6x 9x + y = x 6y 9y Soln: The system
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Linear Classifiers. Blaine Nelson, Tobias Scheffer
Universität Potsdam Institut für Informatik Lehrstuhl Linear Classifiers Blaine Nelson, Tobias Scheffer Contents Classification Problem Bayesian Classifier Decision Linear Classifiers, MAP Models Logistic
More informationLeast Squares Optimization
Least Squares Optimization The following is a brief review of least squares optimization and constrained optimization techniques. I assume the reader is familiar with basic linear algebra, including the
More informationA q x k+q + A q 1 x k+q A 0 x k = 0 (1.1) where k = 0, 1, 2,..., N q, or equivalently. A(σ)x k = 0, k = 0, 1, 2,..., N q (1.
A SPECTRAL CHARACTERIZATION OF THE BEHAVIOR OF DISCRETE TIME ARREPRESENTATIONS OVER A FINITE TIME INTERVAL E.N.Antoniou, A.I.G.Vardulakis, N.P.Karampetakis Aristotle University of Thessaloniki Faculty
More informationFundamentals of Data Assimila1on
014 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO July 1416, 014 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More informationNumerical Methods I Singular Value Decomposition
Numerical Methods I Singular Value Decomposition Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATHGA 2011.003 / CSCIGA 2945.003, Fall 2014 October 9th, 2014 A. Donev (Courant Institute)
More informationSPRING 2006 PRELIMINARY EXAMINATION SOLUTIONS
SPRING 006 PRELIMINARY EXAMINATION SOLUTIONS 1A. Let G be the subgroup of the free abelian group Z 4 consisting of all integer vectors (x, y, z, w) such that x + 3y + 5z + 7w = 0. (a) Determine a linearly
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Classical Iterations We have a problem, We assume that the matrix comes from a discretization of a PDE. The best and most popular model problem is, The matrix will be as large
More informationLinear Subspace Models
Linear Subspace Models Goal: Explore linear models of a data set. Motivation: A central question in vision concerns how we represent a collection of data vectors. The data vectors may be rasterized images,
More informationTHE GRADIENT PROJECTION ALGORITHM FOR ORTHOGONAL ROTATION. 2 The gradient projection algorithm
THE GRADIENT PROJECTION ALGORITHM FOR ORTHOGONAL ROTATION 1 The problem Let M be the manifold of all k by m columnwise orthonormal matrices and let f be a function defined on arbitrary k by m matrices.
More information* Tuesday 17 January :3016:30 (2 hours) Recored on ESSE3 General introduction to the course.
Name of the course Statistical methods and data analysis Audience The course is intended for students of the first or second year of the Graduate School in Materials Engineering. The aim of the course
More informationLECTURE 16: PCA AND SVD
Instructor: Sael Lee CS549 Computational Biology LECTURE 16: PCA AND SVD Resource: PCA Slide by Iyad Batal Chapter 12 of PRML Shlens, J. (2003). A tutorial on principal component analysis. CONTENT Principal
More information1 Linearity and Linear Systems
Mathematical Tools for Neuroscience (NEU 34) Princeton University, Spring 26 Jonathan Pillow Lecture 78 notes: Linear systems & SVD Linearity and Linear Systems Linear system is a kind of mapping f( x)
More informationB553 Lecture 5: Matrix Algebra Review
B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations
More informationContents as of 12/8/2017. Preface. 1. Overview...1
Contents as of 12/8/2017 Preface 1. Overview...1 1.1 Introduction...1 1.2 Finite element data...1 1.3 Matrix notation...3 1.4 Matrix partitions...8 1.5 Special finite element matrix notations...9 1.6 Finite
More informationLinearization of Differential Equation Models
Linearization of Differential Equation Models 1 Motivation We cannot solve most nonlinear models, so we often instead try to get an overall feel for the way the model behaves: we sometimes talk about looking
More informationFullState Feedback Design for a MultiInput System
FullState Feedback Design for a MultiInput System A. Introduction The openloop system is described by the following state space model. x(t) = Ax(t)+Bu(t), y(t) =Cx(t)+Du(t) () 4 8.5 A =, B =.5.5, C
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationPreface. Figures Figures appearing in the text were prepared using MATLAB R. For product information, please contact:
Linear algebra forms the basis for much of modern mathematics theoretical, applied, and computational. The purpose of this book is to provide a broad and solid foundation for the study of advanced mathematics.
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More informationSTOCHASTIC SAMPLING METHODS
STOCHASTIC SAMPLING METHODS APPROXIMATING QUANTITIES OF INTEREST USING SAMPLING METHODS Recall that quantities of interest often require the evaluation of stochastic integrals of functions of the solutions
More informationStructure in Data. A major objective in data analysis is to identify interesting features or structure in the data.
Structure in Data A major objective in data analysis is to identify interesting features or structure in the data. The graphical methods are very useful in discovering structure. There are basically two
More informationThe Jordan Canonical Form
The Jordan Canonical Form The Jordan canonical form describes the structure of an arbitrary linear transformation on a finitedimensional vector space over an algebraically closed field. Here we develop
More informationOverview. Bayesian assimilation of experimental data into simulation (for Goland wing flutter) Why not uncertainty quantification?
Delft University of Technology Overview Bayesian assimilation of experimental data into simulation (for Goland wing flutter), Simao Marques 1. Why not uncertainty quantification? 2. Why uncertainty quantification?
More informationThe antitriangular factorisation of saddle point matrices
The antitriangular factorisation of saddle point matrices J. Pestana and A. J. Wathen August 29, 2013 Abstract Mastronardi and Van Dooren [this journal, 34 (2013) pp. 173 196] recently introduced the block
More informationLinear Models. DSGA 1013 / MATHGA 2824 Optimizationbased Data Analysis.
Linear Models DSGA 1013 / MATHGA 2824 Optimizationbased Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos FernandezGranda Linear regression Leastsquares estimation
More informationDerivativeFree TrustRegion methods
DerivativeFree TrustRegion methods MTH6418 S. Le Digabel, École Polytechnique de Montréal Fall 2015 (v4) MTH6418: DFTR 1/32 Plan Quadratic models Model Quality DerivativeFree TrustRegion Framework
More informationLinear Regression (continued)
Linear Regression (continued) Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 6, 2017 1 / 39 Outline 1 Administration 2 Review of last lecture 3 Linear regression
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationSubset Selection. Ilse Ipsen. North Carolina State University, USA
Subset Selection Ilse Ipsen North Carolina State University, USA Subset Selection Given: real or complex matrix A integer k Determine permutation matrix P so that AP = ( A 1 }{{} k A 2 ) Important columns
More informationFundamentals of Data Assimila1on
2015 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO August 1114, 2015 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University
More informationLeast Squares Optimization
Least Squares Optimization The following is a brief review of least squares optimization and constrained optimization techniques. Broadly, these techniques can be used in data analysis and visualization
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationOn the Nature of Random System Matrices in Structural Dynamics
On the Nature of Random System Matrices in Structural Dynamics S. ADHIKARI AND R. S. LANGLEY Cambridge University Engineering Department Cambridge, U.K. Nature of Random System Matrices p.1/20 Outline
More informationCOMP 558 lecture 18 Nov. 15, 2010
Least squares We have seen several least squares problems thus far, and we will see more in the upcoming lectures. For this reason it is good to have a more general picture of these problems and how to
More informationCME 345: MODEL REDUCTION
CME 345: MODEL REDUCTION Proper Orthogonal Decomposition (POD) Charbel Farhat & David Amsallem Stanford University cfarhat@stanford.edu 1 / 43 Outline 1 Timecontinuous Formulation 2 Method of Snapshots
More informationMA2501 Numerical Methods Spring 2015
Norwegian University of Science and Technology Department of Mathematics MA5 Numerical Methods Spring 5 Solutions to exercise set 9 Find approximate values of the following integrals using the adaptive
More informationIntroduction to Differential Equations
Introduction to Differential Equations J. M. Veal, Ph. D. version 13.08.30 Contents 1 Introduction to Differential Equations 2 1.1 Definitions and Terminology.................... 2 1.2 InitialValue Problems.......................
More informationProbabilistic Latent Semantic Analysis
Probabilistic Latent Semantic Analysis Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongamro, Namgu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationGraphs, Vectors, and Matrices Daniel A. Spielman Yale University. AMS Josiah Willard Gibbs Lecture January 6, 2016
Graphs, Vectors, and Matrices Daniel A. Spielman Yale University AMS Josiah Willard Gibbs Lecture January 6, 2016 From Applied to Pure Mathematics Algebraic and Spectral Graph Theory Sparsification: approximating
More informationData Mining Techniques
Data Mining Techniques CS 6220  Section 3  Fall 2016 Lecture 12 JanWillem van de Meent (credit: Yijun Zhao, Percy Liang) DIMENSIONALITY REDUCTION Borrowing from: Percy Liang (Stanford) Linear Dimensionality
More informationSubspace Identification
Chapter 10 Subspace Identification Given observations of m 1 input signals, and p 1 signals resulting from those when fed into a dynamical system under study, can we estimate the internal dynamics regulating
More informationLinear Algebra Final Exam Review
Linear Algebra Final Exam Review. Let A be invertible. Show that, if v, v, v 3 are linearly independent vectors, so are Av, Av, Av 3. NOTE: It should be clear from your answer that you know the definition.
More informationLecture 9: SVD, Low Rank Approximation
CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 9: SVD, Low Rank Approimation Lecturer: Shayan Oveis Gharan April 25th Scribe: Koosha Khalvati Disclaimer: hese notes have not been subjected
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University October 17, 005 Lecture 3 3 he Singular Value Decomposition
More informationNumerical Methods I: Eigenvalues and eigenvectors
1/25 Numerical Methods I: Eigenvalues and eigenvectors Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu November 2, 2017 Overview 2/25 Conditioning Eigenvalues and eigenvectors How hard are they
More informationComputational methods for mixed models
Computational methods for mixed models Douglas Bates Department of Statistics University of Wisconsin Madison March 27, 2018 Abstract The lme4 package provides R functions to fit and analyze several different
More informationELEC633: Graphical Models
ELEC633: Graphical Models Tahira isa Saleem Scribe from 7 October 2008 References: Casella and George Exploring the Gibbs sampler (1992) Chib and Greenberg Understanding the MetropolisHastings algorithm
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationS. Freitag, B. T. Cao, J. Ninić & G. Meschke
SFB 837 Interaction Modeling Mechanized Tunneling S Freitag, B T Cao, J Ninić & G Meschke Institute for Structural Mechanics Ruhr University Bochum 1 Content 1 Motivation 2 Processoriented FE model for
More information1. Find the solution of the following uncontrolled linear system. 2 α 1 1
Appendix B Revision Problems 1. Find the solution of the following uncontrolled linear system 0 1 1 ẋ = x, x(0) =. 2 3 1 Class test, August 1998 2. Given the linear system described by 2 α 1 1 ẋ = x +
More informationThe Kalman Filter ImPr Talk
The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman
More information