# The line, the circle, and the ray. R + x r. Science is linear, is nt? But behaviors take place in nonlinear spaces. The line The circle The ray

Save this PDF as:

Size: px
Start display at page:

Download "The line, the circle, and the ray. R + x r. Science is linear, is nt? But behaviors take place in nonlinear spaces. The line The circle The ray"

## Transcription

1 Science is linear, is nt The line, the circle, and the ray Nonlinear spaces with efficient linearizations R. Sepulchre -- University of Cambridge Francqui Chair UCL, 05 Page rank algorithm Consensus algorithms Power method all share the same linear iteration + = A But behaviors take place in nonlinear spaces The line The circle The ray The power algorithm is an iteration on the projective space (orthogonality constraint, i.e. the circle) Perron-Frobenius theorem is a fied point in the projective space (positivity constraints, i.e. the ray) R S R + r

2 Part I: Homogeneity is essential to nonlinear behaviors The line : linear spaces Homogeneity is the net best thing to linearity It is necessary to account for the nonlinear nature of data y + y R Linear combinations: the basis of calculus It is sufficient to make local analysis efficient R, R n, R n n, C, C n, Sym(n), Skew(n),... The circle : phase and rotation spaces The ray : intensity spaces e i Embedding: Projection: S! C :! e i C! S : z! arg(z) Embedding: R +! R : r! log r S, S n, SO(n), SU(n),... R +, + (n), GL + (n),... phase, rotation, attitude, orthogonal matrices, unitary matrices,... Representation: linear spaces + orthogonality constraints radius, intensity, concentration, probability, density, Representation: linear spaces + positivity constraints

3 Polar coordinates r z A fundamental result of linear matri analysis A = QR Linear transformations (matrices) have intensities and orientation A = U V T z = re i C Any linear transformation can be decomposed as two rotations and one diagonal scaling. Linear objects (vectors) have intensities and orientation Nonlinear data Eamples of nonlinear measurements Most sensors have a preference for phase or intensity. For good reasons. concentration signals Our ears favor amplitude. Our eyes favor phase. Our nose favors concentrations. phase & intensity signals intensity signals

4 Behaviors (Willems) Linear behaviors (a mature theory) (V, B) V T! W vector space The Universum : space in which we observe/measure/collect the data The Behavior : mathematical law that govern the data B linear subspace E.g. Dynamical systems: V signal space, i.e. T! W :(t, )! w(t, ) B local law in (t, ) : F (w, ẇ,..., w (m) )=0 Linearization = local (and efficient) calculus Linearized behaviors Filtering Interpolating Optimizing (least squares, grad) Averaging,... (lecture 4) Linearization principle (Newton): linear behaviors capture local behaviors near a nominal solution w B w w + w w B(w ) linearized behavior around w From Taylor s epansion: DF(w ) w =0 Note: this requires W to be a linear space...

5 Linearized phase behavior Local analysis of nonlinear behaviors Embed the space W in a linear space and make the embedding part of the behavior W = S, B : F ( ) =0 B(w ) B(w ) B(w 3) is equivalent to z W = C, B : F ( ) =0 z = e i B Patching linearized behaviors: intractable Note: conceptually, the embedding trick works for arbitrary differentiable manifolds. What is special about the phase constraint Invariance and nonlinear data We like to think of local laws over to be invariant with respect to data phases intensities rotation scaling Homogeneity: linearization should look the same everywhere... Moon phase measured in Tokyo or Paris T measured in C or F...

6 Behaviors in invariant spaces LT(S)I behaviors are maimally invariant Laws independent from the locality of our data are invariant to specific transformations of the space The law is the same everywhere and everytime W = R W = S W = R + Invariance to translation of data Invariance to rotation of data Invariance to scaling of data shift-invariant in T translation invariant in W Note: those are the eact assumptions under which behavioral theory is mature and efficient. The key property The geometry of scaling and rotating Phase and rotation spaces are homogeneous spaces. In those spaces, linearization (and hence local calculus) can be made the same everywhere. The line, the circle, and the ray share a common mathematical structure. + z e i.e iz e.e z Transitive group action. Reaching any point from any point.

7 Lie groups Matri Lie groups From Wikipedia: three major themes in 9th century mathematics were combined by Lie in creating his new theory: the idea of symmetry, as eemplified by Galois through the algebraic notion of a group; geometric theory and the eplicit solutions of differential equations of mechanics, worked out by Poisson and Jacobi; and the new understanding of geometry that emerged in the works of Plücker, Möbius, Grassmann and others, and culminated in Riemann's revolutionary vision of the subject. R n n SO(n) matri translation matri rotation Pioneers in engineering: e.g. A.S. Willsky, Dynamical Systems Defined on Groups: Structural Properties and Estimation, Ph.D. Thesis, MIT Dept. of Aeronautics and Astronautics, May 973. Thesis Advisors: R. W. Brockett, Wallace E. Vander Velde. GL(n) matri scaling Homogeneity is essential to nonlinear modeling The local description of the law can be made independent from the locality of the data window only if W is homogeneous. Homogeneity is key to tractability: local coordinates can be made the same everywhere... Part II: Eamples of behaviors on homogeneous spaces A homogeneous space is a space with a transitive group action by a Lie group. (The sphere is not a Lie group but it is a homogenous space)

8 Homogeneous spaces A homogeneous space M is a space with a transitive group action by a Lie group G. Notation: M G/H H is the stabilizer: Two (important) eamples S + (n) Symmetric positive definite matrices (Behaviors in homogeneous spaces; positivity constraints) H = {g G g.e = e} Gr(p, n) Sphere: S O(3)/O() Behaviors on the space of positive definite matrices the set of p-dimensional subspaces of Rn (Homogeneous behaviors in linear spaces; orthogonality and rank constraints) Diffusion Tensor Imaging voel data = local measure of diffusion of water molecules Phase Intensity Homogeneous data processing (Filtering, interpolating, registering,...)

9 Quadratic forms on linear data Zero-mean gaussian distributions are characterized by covariance matrices A linear transformation of the data points results in the group action GL(n) X = E( T )! A X! AXA T S + (n) The group acts transitively on by congruence. S + (n) GL(n)/O(n) Other eamples of quadratic forms: kernels, distance matrices,... Engineering impact of affine-invariant geometry of the cone Statistical engineering S. T. Smith, Covariance, subspace, and intrinsic Cramer-Rao bounds, IEEE Trans. Signal Process., 53 (005), pp Conve optimization Yu. E. Nesterov and M. J. Todd, On the Riemannian geometry defined for selfconcordant barriers and interior point methods, Found. Comput. Math., (00), pp DTI imaging X. Pennec, P. Fillard, and N. Ayache, A Riemannian framework for tensor computing, International Journal of Computer Vision, 66 (006), pp Behaviors on quadratic forms: gaussian processes, kernel optimization,... Different ways to make a space homogeneous Diffusion Tensor Imaging : filtering and interpolating smarties X = AA T = U U T = ep(z) Affine-invariant geometry (intrinsic) X GL(n)/O(n) Group embedding (etrinsic) (U, ) O(n) + linear embedding (etrinsic) Z Sym(n) Issues: computation, singularities, invariance properties (PhD Anne Collard, 03: anisotropy preserving midpoints)

10 Matri completion: a popular benchmark Big data behaviors ~ 07 known ratings (0.0% - 0.%) A recurrent theme: Scarcity of data points in huge dimensional spaces make behaviors ill-posed. Remedy: rank and orthogonality constraints ~ 05 items ~ 06 users Matri completion with a low-rank prior Statistics with scarce data Make the search space dimension consistent with the number of data points spots are gene epression levels each row is an eperiment (~0) each column is a gene (~04) DNA. mrna Protein

11 Statistics with a low-rank /sparsity prior The Grassmann manifold Gr(p, n) the set of p-dimensional subspaces of Rn A subspace is determined by the first p columns of an orthogonal matri Gr(p, n) O(n)/stab e epression of a component for all eperiments test correlation with clinical data gene signature of a component test overlap with pathways, regulatory modules Rank constraints M (p, m n) Transitive group action : A key homogeneous space of behavioral theory Miing rank and positivity constraints Space of m by n matrices of rank p (A, B)! AXB T M (p, m n) Gl(n) Gl(m)/stab ep The set of positive semidefinite matrices of size n and rank p S+ (p, n) = {X Rn n X = X T 0} p Transitive group action: A! AXAT S+ (p, n) GL(n)/stabe 44

12 The line, the circle, and the ray Nonlinear spaces with efficient linearizations Nonlinear data is a source of nonlinear behaviors. Phase and intensities spaces are homogeneous. Rank, orthogonality, and positivity constraints are homogeneous. Behaviors on homogeneous spaces can be made independent from the locality of data. Calculus on homogeneous spaces can be made invariant (lecture 3). Even if they are not invariant, behaviors on homogeneous spaces might have invariant properties (see lecture 6). Behaviors defined on non homogeneous spaces are ill-posed Behaviors with invariant properties are tractable

### Lyapunov functions on nonlinear spaces S 1 R R + V =(1 cos ) Constructing Lyapunov functions: a personal journey

Constructing Lyapunov functions: a personal journey Lyapunov functions on nonlinear spaces R. Sepulchre -- University of Liege, Belgium Reykjavik - July 2013 Lyap functions in linear spaces (1994-1997)

### 1 Kernel methods & optimization

Machine Learning Class Notes 9-26-13 Prof. David Sontag 1 Kernel methods & optimization One eample of a kernel that is frequently used in practice and which allows for highly non-linear discriminant functions

### II. DIFFERENTIABLE MANIFOLDS. Washington Mio CENTER FOR APPLIED VISION AND IMAGING SCIENCES

II. DIFFERENTIABLE MANIFOLDS Washington Mio Anuj Srivastava and Xiuwen Liu (Illustrations by D. Badlyans) CENTER FOR APPLIED VISION AND IMAGING SCIENCES Florida State University WHY MANIFOLDS? Non-linearity

### The nonsmooth Newton method on Riemannian manifolds

The nonsmooth Newton method on Riemannian manifolds C. Lageman, U. Helmke, J.H. Manton 1 Introduction Solving nonlinear equations in Euclidean space is a frequently occurring problem in optimization and

### Ranking from Crowdsourced Pairwise Comparisons via Matrix Manifold Optimization

Ranking from Crowdsourced Pairwise Comparisons via Matrix Manifold Optimization Jialin Dong ShanghaiTech University 1 Outline Introduction FourVignettes: System Model and Problem Formulation Problem Analysis

### CS168: The Modern Algorithmic Toolbox Lecture #8: How PCA Works

CS68: The Modern Algorithmic Toolbox Lecture #8: How PCA Works Tim Roughgarden & Gregory Valiant April 20, 206 Introduction Last lecture introduced the idea of principal components analysis (PCA). The

### Numerical Integration (Quadrature) Another application for our interpolation tools!

Numerical Integration (Quadrature) Another application for our interpolation tools! Integration: Area under a curve Curve = data or function Integrating data Finite number of data points spacing specified

### 33A Linear Algebra and Applications: Practice Final Exam - Solutions

33A Linear Algebra and Applications: Practice Final Eam - Solutions Question Consider a plane V in R 3 with a basis given by v = and v =. Suppose, y are both in V. (a) [3 points] If [ ] B =, find. (b)

### Generalized Shifted Inverse Iterations on Grassmann Manifolds 1

Proceedings of the Sixteenth International Symposium on Mathematical Networks and Systems (MTNS 2004), Leuven, Belgium Generalized Shifted Inverse Iterations on Grassmann Manifolds 1 J. Jordan α, P.-A.

### ABSTRACT ALGEBRA WITH APPLICATIONS

ABSTRACT ALGEBRA WITH APPLICATIONS IN TWO VOLUMES VOLUME I VECTOR SPACES AND GROUPS KARLHEINZ SPINDLER Darmstadt, Germany Marcel Dekker, Inc. New York Basel Hong Kong Contents f Volume I Preface v VECTOR

### Manifolds, Lie Groups, Lie Algebras, with Applications. Kurt W.A.J.H.Y. Reillag (alias Jean Gallier) CIS610, Spring 2005

Manifolds, Lie Groups, Lie Algebras, with Applications Kurt W.A.J.H.Y. Reillag (alias Jean Gallier) CIS610, Spring 2005 1 Motivations and Goals 1. Motivations Observation: Often, the set of all objects

### Visual SLAM Tutorial: Bundle Adjustment

Visual SLAM Tutorial: Bundle Adjustment Frank Dellaert June 27, 2014 1 Minimizing Re-projection Error in Two Views In a two-view setting, we are interested in finding the most likely camera poses T1 w

### Numerical Methods I Singular Value Decomposition

Numerical Methods I Singular Value Decomposition Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 9th, 2014 A. Donev (Courant Institute)

### Lecture 4.2 Finite Difference Approximation

Lecture 4. Finite Difference Approimation 1 Discretization As stated in Lecture 1.0, there are three steps in numerically solving the differential equations. They are: 1. Discretization of the domain by

### Lecture 13 The Fundamental Forms of a Surface

Lecture 13 The Fundamental Forms of a Surface In the following we denote by F : O R 3 a parametric surface in R 3, F(u, v) = (x(u, v), y(u, v), z(u, v)). We denote partial derivatives with respect to the

### Computational Methods CMSC/AMSC/MAPL 460. EigenValue decomposition Singular Value Decomposition. Ramani Duraiswami, Dept. of Computer Science

Computational Methods CMSC/AMSC/MAPL 460 EigenValue decomposition Singular Value Decomposition Ramani Duraiswami, Dept. of Computer Science Hermitian Matrices A square matrix for which A = A H is said

### Clifford Algebras and Spin Groups

Clifford Algebras and Spin Groups Math G4344, Spring 2012 We ll now turn from the general theory to examine a specific class class of groups: the orthogonal groups. Recall that O(n, R) is the group of

### Spectral Algorithms I. Slides based on Spectral Mesh Processing Siggraph 2010 course

Spectral Algorithms I Slides based on Spectral Mesh Processing Siggraph 2010 course Why Spectral? A different way to look at functions on a domain Why Spectral? Better representations lead to simpler solutions

### Discriminative Direction for Kernel Classifiers

Discriminative Direction for Kernel Classifiers Polina Golland Artificial Intelligence Lab Massachusetts Institute of Technology Cambridge, MA 02139 polina@ai.mit.edu Abstract In many scientific and engineering

### Adjoint Orbits, Principal Components, and Neural Nets

Adjoint Orbits, Principal Components, and Neural Nets Some facts about Lie groups and examples Examples of adjoint orbits and a distance measure Descent equations on adjoint orbits Properties of the double

### Exact and Approximate Numbers:

Eact and Approimate Numbers: The numbers that arise in technical applications are better described as eact numbers because there is not the sort of uncertainty in their values that was described above.

### Courses: Mathematics (MATH)College: Natural Sciences & Mathematics. Any TCCN equivalents are indicated in square brackets [ ].

Courses: Mathematics (MATH)College: Natural Sciences & Mathematics Any TCCN equivalents are indicated in square brackets [ ]. MATH 1300: Fundamentals of Mathematics Cr. 3. (3-0). A survey of precollege

### An introduction to arithmetic groups. Lizhen Ji CMS, Zhejiang University Hangzhou , China & Dept of Math, Univ of Michigan Ann Arbor, MI 48109

An introduction to arithmetic groups Lizhen Ji CMS, Zhejiang University Hangzhou 310027, China & Dept of Math, Univ of Michigan Ann Arbor, MI 48109 June 27, 2006 Plan. 1. Examples of arithmetic groups

### Review of Linear Algebra

Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources

### α x x 0 α x x f(x) α x x α x ( 1) f(x) x f(x) x f(x) α x = α x x 2

Quadratic speedup for unstructured search - Grover s Al- CS 94- gorithm /8/07 Spring 007 Lecture 11 01 Unstructured Search Here s the problem: You are given an efficient boolean function f : {1,,} {0,1},

### Lecture 7: Weak Duality

EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly

### THE EULER CHARACTERISTIC OF A LIE GROUP

THE EULER CHARACTERISTIC OF A LIE GROUP JAY TAYLOR 1 Examples of Lie Groups The following is adapted from [2] We begin with the basic definition and some core examples Definition A Lie group is a smooth

### Chapter 3 Numerical Methods

Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2

### Welcome to Copenhagen!

Welcome to Copenhagen! Schedule: Monday Tuesday Wednesday Thursday Friday 8 Registration and welcome 9 Crash course on Crash course on Introduction to Differential and Differential and Information Geometry

### The convex algebraic geometry of rank minimization

The convex algebraic geometry of rank minimization Pablo A. Parrilo Laboratory for Information and Decision Systems Massachusetts Institute of Technology International Symposium on Mathematical Programming

### FINITE-DIMENSIONAL LINEAR ALGEBRA

DISCRETE MATHEMATICS AND ITS APPLICATIONS Series Editor KENNETH H ROSEN FINITE-DIMENSIONAL LINEAR ALGEBRA Mark S Gockenbach Michigan Technological University Houghton, USA CRC Press Taylor & Francis Croup

### Linear Algebra Methods for Data Mining

Linear Algebra Methods for Data Mining Saara Hyvönen, Saara.Hyvonen@cs.helsinki.fi Spring 2007 Linear Discriminant Analysis Linear Algebra Methods for Data Mining, Spring 2007, University of Helsinki Principal

### 2. Preliminaries. x 2 + y 2 + z 2 = a 2 ( 1 )

x 2 + y 2 + z 2 = a 2 ( 1 ) V. Kumar 2. Preliminaries 2.1 Homogeneous coordinates When writing algebraic equations for such geometric objects as planes and circles, we are used to writing equations that

### 8 Eigenvectors and the Anisotropic Multivariate Gaussian Distribution

Eigenvectors and the Anisotropic Multivariate Gaussian Distribution Eigenvectors and the Anisotropic Multivariate Gaussian Distribution EIGENVECTORS [I don t know if you were properly taught about eigenvectors

### Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

### Linear Algebra for Machine Learning. Sargur N. Srihari

Linear Algebra for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Overview Linear Algebra is based on continuous math rather than discrete math Computer scientists have little experience with it

### 2.29 Numerical Fluid Mechanics Spring 2015 Lecture 4

2.29 Spring 2015 Lecture 4 Review Lecture 3 Truncation Errors, Taylor Series and Error Analysis Taylor series: 2 3 n n i1 i i i i i n f( ) f( ) f '( ) f ''( ) f '''( )... f ( ) R 2! 3! n! n1 ( n1) Rn f

### Digital Image Processing

Digital Image Processing 2D SYSTEMS & PRELIMINARIES Hamid R. Rabiee Fall 2015 Outline 2 Two Dimensional Fourier & Z-transform Toeplitz & Circulant Matrices Orthogonal & Unitary Matrices Block Matrices

### Topics in Representation Theory: Cultural Background

Topics in Representation Theory: Cultural Background This semester we will be covering various topics in representation theory, see the separate syllabus for a detailed list of topics, including some that

### arxiv: v1 [math.dg] 2 Mar 2015

Geometric Properties of Conformal Transformations on R p,q Matvei Libine and Surya Raghavendran February 13, 2018 arxiv:1503.00520v1 [math.dg] 2 Mar 2015 Abstract We show that conformal transformations

### 18.303: Introduction to Green s functions and operator inverses

8.33: Introduction to Green s functions and operator inverses S. G. Johnson October 9, 2 Abstract In analogy with the inverse A of a matri A, we try to construct an analogous inverse Â of differential

### 3.3.1 Linear functions yet again and dot product In 2D, a homogenous linear scalar function takes the general form:

3.3 Gradient Vector and Jacobian Matri 3 3.3 Gradient Vector and Jacobian Matri Overview: Differentiable functions have a local linear approimation. Near a given point, local changes are determined by

### Parallel Singular Value Decomposition. Jiaxing Tan

Parallel Singular Value Decomposition Jiaxing Tan Outline What is SVD? How to calculate SVD? How to parallelize SVD? Future Work What is SVD? Matrix Decomposition Eigen Decomposition A (non-zero) vector

### Weyl Group Representations and Unitarity of Spherical Representations.

Weyl Group Representations and Unitarity of Spherical Representations. Alessandra Pantano, University of California, Irvine Windsor, October 23, 2008 β ν 1 = ν 2 S α S β ν S β ν S α ν S α S β S α S β ν

### The Symmetric Space for SL n (R)

The Symmetric Space for SL n (R) Rich Schwartz November 27, 2013 The purpose of these notes is to discuss the symmetric space X on which SL n (R) acts. Here, as usual, SL n (R) denotes the group of n n

### PARTIAL DIFFERENTIAL EQUATIONS

MATHEMATICAL METHODS PARTIAL DIFFERENTIAL EQUATIONS I YEAR B.Tech By Mr. Y. Prabhaker Reddy Asst. Professor of Mathematics Guru Nanak Engineering College Ibrahimpatnam, Hyderabad. SYLLABUS OF MATHEMATICAL

### Tangent spaces, normals and extrema

Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent

### RIEMANNIAN METRIC AND GEOMETRIC MEAN FOR POSITIVE SEMIDEFINITE MATRICES OF FIXED RANK

SIAM J. MATRIX ANAL. APPL. Vol. 0, No. 0, pp. 000 000 c 2009 Society for Industrial and Applied Mathematics RIEMANNIAN METRIC AND GEOMETRIC MEAN FOR POSITIVE SEMIDEFINITE MATRICES OF FIXED RANK SILVÈRE

### MATHEMATICAL METHODS INTERPOLATION

MATHEMATICAL METHODS INTERPOLATION I YEAR BTech By Mr Y Prabhaker Reddy Asst Professor of Mathematics Guru Nanak Engineering College Ibrahimpatnam, Hyderabad SYLLABUS OF MATHEMATICAL METHODS (as per JNTU

### 1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)

Notes for 2017-01-30 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty

### Robust Fisher Discriminant Analysis

Robust Fisher Discriminant Analysis Seung-Jean Kim Alessandro Magnani Stephen P. Boyd Information Systems Laboratory Electrical Engineering Department, Stanford University Stanford, CA 94305-9510 sjkim@stanford.edu

### 5. Nonholonomic constraint Mechanics of Manipulation

5. Nonholonomic constraint Mechanics of Manipulation Matt Mason matt.mason@cs.cmu.edu http://www.cs.cmu.edu/~mason Carnegie Mellon Lecture 5. Mechanics of Manipulation p.1 Lecture 5. Nonholonomic constraint.

### R. Courant and D. Hilbert METHODS OF MATHEMATICAL PHYSICS Volume II Partial Differential Equations by R. Courant

R. Courant and D. Hilbert METHODS OF MATHEMATICAL PHYSICS Volume II Partial Differential Equations by R. Courant CONTENTS I. Introductory Remarks S1. General Information about the Variety of Solutions.

### Affine iterations on nonnegative vectors

Affine iterations on nonnegative vectors V. Blondel L. Ninove P. Van Dooren CESAME Université catholique de Louvain Av. G. Lemaître 4 B-348 Louvain-la-Neuve Belgium Introduction In this paper we consider

### MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS

T H I R D E D I T I O N MULTIVARIABLE CALCULUS, LINEAR ALGEBRA, AND DIFFERENTIAL EQUATIONS STANLEY I. GROSSMAN University of Montana and University College London SAUNDERS COLLEGE PUBLISHING HARCOURT BRACE

### Or, more succinctly, lim

Lecture 7. Functions and Stuff PCMI Summer 2015 Undergraduate Lectures on Flag Varieties Lecture 7. Functions and differentiable and analytic manifolds. The implicit function theorem, cotangent spaces

### Linear and affine transformations

Linear and affine transformations Linear Algebra Review Matrices Transformations Affine transformations in Euclidean space 1 The linear transformation given b a matri Let A be an mn matri. The function

### Riemannian geometry of positive definite matrices: Matrix means and quantum Fisher information

Riemannian geometry of positive definite matrices: Matrix means and quantum Fisher information Dénes Petz Alfréd Rényi Institute of Mathematics Hungarian Academy of Sciences POB 127, H-1364 Budapest, Hungary

### 2 Linear Classifiers and Perceptrons

Linear lassifiers and Perceptrons 7 Linear lassifiers and Perceptrons LASSIFIERS You are given sample of n observations, each with d features. Some observations belong to class ; some do not. Eample: Observations

### Camera calibration. Outline. Pinhole camera. Camera projection models. Nonlinear least square methods A camera calibration tool

Outline Camera calibration Camera projection models Camera calibration i Nonlinear least square methods A camera calibration tool Applications Digital Visual Effects Yung-Yu Chuang with slides b Richard

### Group theoretical methods in Machine Learning

Group theoretical methods in Machine Learning Risi Kondor Columbia University Tutorial at ICML 2007 Tiger, tiger, burning bright In the forests of the night, What immortal hand or eye Dare frame thy fearful

### Solve Wave Equation from Scratch [2013 HSSP]

1 Solve Wave Equation from Scratch [2013 HSSP] Yuqi Zhu MIT Department of Physics, 77 Massachusetts Ave., Cambridge, MA 02139 (Dated: August 18, 2013) I. COURSE INFO Topics Date 07/07 Comple number, Cauchy-Riemann

CS249: ADVANCED DATA MINING Clustering Evaluation and Practical Issues Instructor: Yizhou Sun yzsun@cs.ucla.edu May 2, 2017 Announcements Homework 2 due later today Due May 3 rd (11:59pm) Course project

### Lines and points. Lines and points

omogeneous coordinates in the plane Homogeneous coordinates in the plane A line in the plane a + by + c is represented as (a, b, c). A line is a subset of points in the plane. All vectors (ka, kb, kc)

### Topological K-theory

Topological K-theory Robert Hines December 15, 2016 The idea of topological K-theory is that spaces can be distinguished by the vector bundles they support. Below we present the basic ideas and definitions

### Linear Subspace Models

Linear Subspace Models Goal: Explore linear models of a data set. Motivation: A central question in vision concerns how we represent a collection of data vectors. The data vectors may be rasterized images,

### Singular Value Decomposition

Chapter 6 Singular Value Decomposition In Chapter 5, we derived a number of algorithms for computing the eigenvalues and eigenvectors of matrices A R n n. Having developed this machinery, we complete our

### Image Registration Lecture 2: Vectors and Matrices

Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this

### Fast ADMM for Sum of Squares Programs Using Partial Orthogonality

Fast ADMM for Sum of Squares Programs Using Partial Orthogonality Antonis Papachristodoulou Department of Engineering Science University of Oxford www.eng.ox.ac.uk/control/sysos antonis@eng.ox.ac.uk with

### FALL 2011, LECTURE 1 (9/8/11) This is subject to revision. Current version: Thu, Sep 8, 1:00 PM

18700 FALL 2011, LECTURE 1 (9/8/11) TRAVIS SCHEDLER This is subject to revision Current version: Thu, Sep 8, 1:00 PM Abstract In this lecture, I will broadly sketch what linear algebra is and why we should

### Stratification of 3 3 Matrices

Stratification of 3 3 Matrices Meesue Yoo & Clay Shonkwiler March 2, 2006 1 Warmup with 2 2 Matrices { Best matrices of rank 2} = O(2) S 3 ( 2) { Best matrices of rank 1} S 3 (1) 1.1 Viewing O(2) S 3 (

### 3D Computer Vision - WT 2004

3D Computer Vision - WT 2004 Singular Value Decomposition Darko Zikic CAMP - Chair for Computer Aided Medical Procedures November 4, 2004 1 2 3 4 5 Properties For any given matrix A R m n there exists

### Algebra I Fall 2007

MIT OpenCourseWare http://ocw.mit.edu 18.701 Algebra I Fall 007 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 18.701 007 Geometry of the Special Unitary

### Exercise Sheet 1. 1 Probability revision 1: Student-t as an infinite mixture of Gaussians

Exercise Sheet 1 1 Probability revision 1: Student-t as an infinite mixture of Gaussians Show that an infinite mixture of Gaussian distributions, with Gamma distributions as mixing weights in the following

### LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization

### Multi-Frame Factorization Techniques

Multi-Frame Factorization Techniques Suppose { x j,n } J,N j=1,n=1 is a set of corresponding image coordinates, where the index n = 1,...,N refers to the n th scene point and j = 1,..., J refers to the

### Computation of the Quadrifocal Tensor

Computation of the Quadrifocal Tensor Richard I. Hartley G.E. Corporate Research and Development Research Circle, Niskayuna, NY 2309, USA Abstract. This paper gives a practical and accurate algorithm for

### Lecture 5: September Time Complexity Analysis of Local Alignment

CSCI1810: Computational Molecular Biology Fall 2017 Lecture 5: September 21 Lecturer: Sorin Istrail Scribe: Cyrus Cousins Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes

### Sparse Levenberg-Marquardt algorithm.

Sparse Levenberg-Marquardt algorithm. R. I. Hartley and A. Zisserman: Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2004. Appendix 6 was used in part. The Levenberg-Marquardt

### Least Squares with Examples in Signal Processing 1. 2 Overdetermined equations. 1 Notation. The sum of squares of x is denoted by x 2 2, i.e.

Least Squares with Eamples in Signal Processing Ivan Selesnick March 7, 3 NYU-Poly These notes address (approimate) solutions to linear equations by least squares We deal with the easy case wherein the

Extended Thesis Abstracts The Finite Radon Transform Andrea Zentz Andrea Zentz graduated Magna Cum Laude from Ball State University in May 2008 with a major in mathematics education. She currently teaches

### ENTANGLED STATES ARISING FROM INDECOMPOSABLE POSITIVE LINEAR MAPS. 1. Introduction

Trends in Mathematics Information Center for Mathematical Sciences Volume 6, Number 2, December, 2003, Pages 83 91 ENTANGLED STATES ARISING FROM INDECOMPOSABLE POSITIVE LINEAR MAPS SEUNG-HYEOK KYE Abstract.

### A q x k+q + A q 1 x k+q A 0 x k = 0 (1.1) where k = 0, 1, 2,..., N q, or equivalently. A(σ)x k = 0, k = 0, 1, 2,..., N q (1.

A SPECTRAL CHARACTERIZATION OF THE BEHAVIOR OF DISCRETE TIME AR-REPRESENTATIONS OVER A FINITE TIME INTERVAL E.N.Antoniou, A.I.G.Vardulakis, N.P.Karampetakis Aristotle University of Thessaloniki Faculty

### Positive and Monotone Systems

History of Control, 2012, Lund May 29, 2012 Outline. 1 Positive Systems Definition Fathers Occurrence Example Publications. 2 Monotone Systems Definition Early days Publications Positive System A continuous

### THE STABLE EMBEDDING PROBLEM

THE STABLE EMBEDDING PROBLEM R. Zavala Yoé C. Praagman H.L. Trentelman Department of Econometrics, University of Groningen, P.O. Box 800, 9700 AV Groningen, The Netherlands Research Institute for Mathematics

### Small sample size in high dimensional space - minimum distance based classification.

Small sample size in high dimensional space - minimum distance based classification. Ewa Skubalska-Rafaj lowicz Institute of Computer Engineering, Automatics and Robotics, Department of Electronics, Wroc

### 6.241 Dynamic Systems and Control

6.41 Dynamic Systems and Control Lecture 5: H Synthesis Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology May 11, 011 E. Frazzoli (MIT) Lecture 5: H Synthesis May 11, 011

### STEM-Prep Pathway SLOs

STEM-Prep Pathway SLOs Background: The STEM-Prep subgroup of the MMPT adopts a variation of the student learning outcomes for STEM from the courses Reasoning with Functions I and Reasoning with Functions

### Window-based Tensor Analysis on High-dimensional and Multi-aspect Streams

Window-based Tensor Analysis on High-dimensional and Multi-aspect Streams Jimeng Sun Spiros Papadimitriou Philip S. Yu Carnegie Mellon University Pittsburgh, PA, USA IBM T.J. Watson Research Center Hawthorne,

### Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision Prasanna Sahoo Department of Mathematics University of Louisville 1 Basic Information Instructor: Professor Ron Sahoo Office: NS 218 Tel: (502) 852-2731 Fax: (502)

### Lecture 7 Spectral methods

CSE 291: Unsupervised learning Spring 2008 Lecture 7 Spectral methods 7.1 Linear algebra review 7.1.1 Eigenvalues and eigenvectors Definition 1. A d d matrix M has eigenvalue λ if there is a d-dimensional

### Proposition 42. Let M be an m n matrix. Then (32) N (M M)=N (M) (33) R(MM )=R(M)

RODICA D. COSTIN. Singular Value Decomposition.1. Rectangular matrices. For rectangular matrices M the notions of eigenvalue/vector cannot be defined. However, the products MM and/or M M (which are square,

### Corner detection: the basic idea

Corner detection: the basic idea At a corner, shifting a window in any direction should give a large change in intensity flat region: no change in all directions edge : no change along the edge direction

### Using the TI-92+ : examples

Using the TI-9+ : eamples Michel Beaudin École de technologie supérieure 1100, rue Notre-Dame Ouest Montréal (Québec) Canada, H3C 1K3 mbeaudin@seg.etsmtl.ca 1- Introduction We incorporated the use of the

### NORCO COLLEGE SLO to PLO MATRIX

SLO to PLO MATRI CERTIFICATE/PROGRAM: Math ADT COURSE: MAT-1A Calculus I Calculate the limit of a function. SLO 2 Determine the continuity of a function. Find the derivatives of algebraic and transcendental

### MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

### Fractional Spectral and Spectral Element Methods

Fractional Calculus, Probability and Non-local Operators: Applications and Recent Developments Nov. 6th - 8th 2013, BCAM, Bilbao, Spain Fractional Spectral and Spectral Element Methods (Based on PhD thesis

### T -equivariant tensor rank varieties and their K-theory classes

T -equivariant tensor rank varieties and their K-theory classes 2014 July 18 Advisor: Professor Anders Buch, Department of Mathematics, Rutgers University Overview 1 Equivariant K-theory Overview 2 Determinantal