CS 523: Computer Graphics, Spring Shape Modeling. PCA Applications + SVD. Andrew Nealen, Rutgers, /15/2011 1
|
|
- James Cook
- 5 years ago
- Views:
Transcription
1 CS 523: Computer Graphcs, Sprng 20 Shape Modelng PCA Applcatons + SVD Andrew Nealen, utgers, 20 2/5/20
2 emnder: PCA Fnd prncpal components of data ponts Orthogonal drectons that are domnant n the data (have varance etrema) y y Scatter matr S X X v λ λ2 v S v v 2 v 2 d λ d v d Andrew Nealen, utgers, 20 2/5/20 2
3 More applcatons of PCA Morphable models of faces Data base of face scans: 3D geometry + teture (photo) 0,000 ponts n each scan, y, z,, G, B 6 numbers for each pont hus, each scan s a 0,000*6 60,000-dmensonal vector See: V. Blanzand. Vetter, A MorphableModel for the Synthess of 3D Faces, SIGGAPH 99 Andrew Nealen, utgers, 20 2/5/20 3
4 More applcatons of PCA Morphable models of faces How to fnd nterestng aes s ths dmensonal space? aes that measures age, gender, etc here s hope: the faces are lkely to be governed by a small set of parameters (much less than 60,000 ) age as gender as FaceGen demo Andrew Nealen, utgers, 20 2/5/20 4
5 Sngular Value Decomposton Andrew Nealen, utgers, 20 2/5/20 5
6 Geometrc analyss of lnear We want to know what a lnear transformaton Adoes transformatons Need some smple and comprehensble representaton of the matr A Let s look what Adoes to some vectors Snce A(αv) αa(v), t s enough to look at vectors vof unt length A Andrew Nealen, utgers, 20 2/5/20 6
7 Geometrc analyss of lnear transformatons A lnear (non-sngular) transform Aalways takes hyper-spheres to hyper-ellpses. A A Andrew Nealen, utgers, 20 2/5/20 7
8 Geometrc analyss of lnear transformatons hus, one good way to understand what A does s to fnd whch vectors are mapped to the man aes of the ellpsod A A Andrew Nealen, utgers, 20 2/5/20 8
9 Geometrc analyss of lnear transformatons If As symmetrc: A V D V, Vorthogonal he egenvectors of Aare the aes of the ellpse A Andrew Nealen, utgers, 20 2/5/20 9
10 Symmetrc matr: egendecomposton In ths case As just a scalng matr. he egendecompostonof Atells us whch orthogonal aes t scales, and by how much A λ 2 λ A λ 2 [ v v Kv ] [ v v Kv ] 2 n 2 Andrew Nealen, utgers, 20 2/5/20 λ O Av λv λn n 0
11 General lnear transformatons: Sngular Value Decomposton In general Awll also contan rotatons, not just scales A σ 2 σ A U Σ V A σ σ 2 [ u u Ku ] [ v v Kv ] O σ n 2 n 2 n Andrew Nealen, utgers, 20 2/5/20
12 General lnear transformatons: Sngular Value Decomposton A σ 2 σ A V U Σ A orthonormal orthonormal [ v v Kv ] [ u u Ku ] 2 n 2 n σ σ 2 O σ n Av σ u, σ 0 Andrew Nealen, utgers, 20 2/5/20 2
13 Some hstory SVD was dscovered by the followng people: E. Beltram ( ) M. Jordan ( ) J. Sylvester (84 897) E. Schmdt ( ) H. Weyl ( ) Andrew Nealen, utgers, 20 2/5/20 3
14 SVD SVD ests for any matr Formal defnton: For square matrces A n n, there est orthogonal matrces U, V n n and a dagonal matr Σ, such that all the dagonal values σ of Σare non-negatve and A U Σ V A U Σ V Andrew Nealen, utgers, 20 2/5/20 4
15 SVD he dagonal values of Σare called the sngular values.it s accustomed to sort them: σ σ 2 σ n he columns of U(u,, u n ) are called the left sngular vectors. hey are the aes of the ellpsod. he columns of V(v,, v n ) are called the rght sngular vectors. hey are the premages of the aes of the ellpsod. A U Σ V A U Σ V Andrew Nealen, utgers, 20 2/5/20 5
16 educed SVD For rectangular matrces, we have two forms of SVD. he reduced SVD looks lke ths: he columns of Uare orthonormal Cheaper form for computaton and storage A U Σ V Andrew Nealen, utgers, 20 2/5/20 6
17 Full SVD We can complete Uto a full orthogonal matr and pad Σby zeros accordngly A U Σ V Andrew Nealen, utgers, 20 2/5/20 7
18 here are stable numercal algorthms to SVD Applcatons compute SVD (albet not cheap). Once you have t, you have many thngs: Matr nverse can solve square lnear systems Numercal rank of a matr Can solve lnear least-squares systems PCA Many more Andrew Nealen, utgers, 20 2/5/20 8
19 Matr nverse and solvng lnear systems Matr nverse So, to solve Andrew Nealen, utgers, 20 2/5/20 9 ( ) ( ) U V U V V U A V U A n Σ Σ Σ σ σ O b b U V A Σ
20 Matr rank he rank of As the number of non-zero sngular values n σ σ2 m σ n A U Σ V Andrew Nealen, utgers, 20 2/5/20 20
21 Numercal rank If there are very small sngular values, then A s close to beng sngular. We can set a threshold t, so that numerc_rank(a) #{σ σ > t} Usng SVD s a numercally stable way! he determnant s not a good way to check sngularty Andrew Nealen, utgers, 20 2/5/20 2
22 PCA Construct the matr X of the centered data ponts he prncpal aes are egenvectors of S XX Andrew Nealen, utgers, 20 2/5/20 22 p n p p L 2 X U U XX S λ d λ O
23 PCA We can compute the prncpal components by SVD of X: X UΣV XX UΣV (UΣV ) UΣV VΣU UΣ 2 U hus, the left sngular vectors of Xare the prncpal components! We sort them by the sze of the sngular values of X. Andrew Nealen, utgers, 20 2/5/20 23
24 Least-squares rotaton wth SVD Andrew Nealen, utgers, 20 2/5/20 24
25 Shape matchng We have two objects n correspondence Want to fnd the rgd transformaton that algns them Andrew Nealen, utgers, 20 2/5/20 25
26 Shape matchng When the objects are algned, the lengths of the connectng lnes are small Andrew Nealen, utgers, 20 2/5/20 26
27 Optmal local rotaton We wll use ths for mesh deformaton v v j2 v v j v j v j 2 Andrew Nealen, utgers, 20 2/5/20 27
28 Shape matchng formalzaton Algn two pont sets P p, K, p and Q q, K, q. { } { } n Fnd a translaton vector tand rotaton matr so that n n 2 ( p + t) s mnmzed q Andrew Nealen, utgers, 20 2/5/20 28
29 Shape matchng soluton Solve translaton and rotaton separately If (, t)s the optmal transformaton, then the pont sets {p + t}and {q }have the same centers of mass Andrew Nealen, utgers, 20 2/5/20 29 n n p p n n q q ( ) p q t t p t p t p q n n n n
30 Fndng the rotaton o fnd the optmal, we brng the centrods of both pont sets to the orgn We want to fnd that mnmzes Andrew Nealen, utgers, 20 2/5/20 30 q q y p p n 2 y
31 ( ) ( ) ( ) + n n n 2 y y y y y y y Fndng the rotaton Andrew Nealen, utgers, 20 2/5/20 hese terms do not depend on, so we can gnore them n the mnmzaton 3 I
32 Fndng the rotaton Andrew Nealen, utgers, 20 2/5/20 ths s a scalar 32 ( ) ( ) ( ) + n n n argma ma mn y y y y y y y y
33 Fndng the rotaton n y tr ( Y X) tr n ( A) A y y 2 M 2 L n y y 2 M 2 L n y n y n Y X Andrew Nealen, utgers, 20 2/5/20 33
34 Fndng the rotaton n y tr ( Y X) tr n ( A) A y y 2 M 2 L n y y 2 2 O y n y n n Andrew Nealen, utgers, 20 2/5/20 34
35 Fndng the rotaton Fnd that mamzes tr ( ) ( Y X tr XY ) (because tr(ab) tr(ba)) Let s do SVD on S XY tr S XY UΣV ( ) ( ) ( ( XY tr UΣV trσ V U ) orthogonal matr Andrew Nealen, utgers, 20 2/5/20 35
36 Fndng the rotaton We want to mamze tr ( ( Σ V U ) orthogonal matr all entres σ σ 2 σ 3 m M L m 22 L M m 33 tr ( ( Σ V U ) m 3 3 σ σ Andrew Nealen, utgers, 20 2/5/20 36
37 tr Fndng the rotaton ( ( Σ V U ) m Our best shot s m,.e. to make V U I V U 3 U VU 3 σ σ V I Andrew Nealen, utgers, 20 2/5/20 37
38 Summary of rgd algnment ranslate the nput ponts to the centrods Compute the covarance matr Compute the SVD of S p p y q he optmal orthogonal s S S XY Andrew Nealen, utgers, 20 2/5/20 38 UΣV VU n y q
39 Sgn correcton It s possble that det(vu ) : sometmes reflecton s the best orthogonal transform Andrew Nealen, utgers, 20 2/5/20 39
40 Sgn correcton It s possble that det(vu ) : sometmes reflecton s the best orthogonal transform Andrew Nealen, utgers, 20 2/5/20 40
41 Sgn correcton It s possble that det(vu ) : sometmes reflecton s the best orthogonal transform o restrct ourselves to rotatons only: take the last column of V(correspondng to the smallest sngular value) and nvert ts sgn. Why? See the PDF Andrew Nealen, utgers, 20 2/5/20 4
42 Complety Numercal SVD s an epensve operaton O(mn(mn 2,nm 2 )) We always need to pay attenton to the dmensons of the matr we re applyng SVD to. Andrew Nealen, utgers, 20 2/5/20 42
43 SVD for anmaton compresson Chcken anmaton See: epresentng Anmatons by Prncpal Components, M. Alea and W. Muller, Eurographcs 2000 Compresson of Soft-Body Anmaton Sequences, Z. Karn and C. Gotsman, Computers&Graphcs 28(): 25-34, 2004 Key Pont Subspace Acceleraton and Soft Cachng, M. Meyer and J. Anderson, SIGGAPH 2007 Andrew Nealen, utgers, 20 2/5/20 43
44 3D anmatons Each frame s a 3D model (mesh) Andrew Nealen, utgers, 20 2/5/20 44
45 3D anmatons Connectvty s usually constant (at least on large segments of the anmaton) he geometry changes n each frame vast amount of data! 3 seconds, 3000 vertces/frame, 26 MB Andrew Nealen, utgers, 20 2/5/20 45
46 46 Anmaton compresson by dmensonalty reducton he geometry of each frame s a vector n 3N space (N #vertces) N N N y y z z M M M 3N #f 2/5/20 Andrew Nealen, utgers, 20
47 47 Anmaton compresson by dmensonalty reducton Fnd a few vectors of 3N that wll best represent our frame vectors! N N N y y z z M M M 2 f σ σ σ O 2 f σ σ σ O V U 3N f Σ f f V f f 2 f σ σ σ K 2/5/20 Andrew Nealen, utgers, 20
48 48 Anmaton compresson by dmensonalty reducton he frst prncpal components are the mportant ones N N N y y z z M M M u u 2 u 3 2 f σ σ σ O V σ σ σ 2/5/20 Andrew Nealen, utgers, 20
49 Anmaton compresson by dmensonalty reducton Appromate each frame by lnear combnaton of the frst prncpal components he more components we use, the better the appromaton Usually, the number of components needed s much smaller than f. Andrew Nealen, utgers, 20 M N y M yn z M z N α u + α 2 u 2 + α 3 u 3 2/5/20 49
50 Anmaton compresson by dmensonalty reducton Compressed representaton: he chosen prncpal component vectors u Coeffcents α for each frame Anmaton wth only 2 prncpal components Anmaton wth 20 out of 400 prncpal components Andrew Nealen, utgers, 20 2/5/20 50
51 Egenfaces Same prncpal components analyss can be appled to mages Andrew Nealen, utgers, 20 2/5/20 5
52 Egenfaces Each mage s a vector n Want to fnd the prncpal aes vectors that best represent the nput database of mages Andrew Nealen, utgers, 20 2/5/20 52
53 econstructon wth a few vectors epresent each mage by the frst few (n) prncpal components (,, K, ) v αu + αu + Kα u α α α 2 2 n n 2 n Andrew Nealen, utgers, 20 2/5/20 53
54 Face recognton Gven a new mage of a face, w epresent wusng the frst n PCA vectors: (,, K, ) w αu + αu + Kα u α α α 2 2 n n 2 n Now fnd an mage n the database whose representaton n the PCA bass s the closest: w w (,, K, ) w α α2 α n w, w s the largest he angle between wand w s the smallest w w Andrew Nealen, utgers, 20 2/5/20 54
55 Non-lnear dmensonalty reducton More sophstcated methods can dscover non-lnear structures n the face datasets Isomap, Scence, Dec Andrew Nealen, utgers, 20 2/5/20 55
Singular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationρ some λ THE INVERSE POWER METHOD (or INVERSE ITERATION) , for , or (more usually) to
THE INVERSE POWER METHOD (or INVERSE ITERATION) -- applcaton of the Power method to A some fxed constant ρ (whch s called a shft), x λ ρ If the egenpars of A are { ( λ, x ) } ( ), or (more usually) to,
More informationStructure from Motion. Forsyth&Ponce: Chap. 12 and 13 Szeliski: Chap. 7
Structure from Moton Forsyth&once: Chap. 2 and 3 Szelsk: Chap. 7 Introducton to Structure from Moton Forsyth&once: Chap. 2 Szelsk: Chap. 7 Structure from Moton Intro he Reconstructon roblem p 3?? p p 2
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationCS4495/6495 Introduction to Computer Vision. 3C-L3 Calibrating cameras
CS4495/6495 Introducton to Computer Vson 3C-L3 Calbratng cameras Fnally (last tme): Camera parameters Projecton equaton the cumulatve effect of all parameters: M (3x4) f s x ' 1 0 0 0 c R 0 I T 3 3 3 x1
More informationEfficient, General Point Cloud Registration with Kernel Feature Maps
Effcent, General Pont Cloud Regstraton wth Kernel Feature Maps Hanchen Xong, Sandor Szedmak, Justus Pater Insttute of Computer Scence Unversty of Innsbruck 30 May 2013 Hanchen Xong (Un.Innsbruck) 3D Regstraton
More informationError Bars in both X and Y
Error Bars n both X and Y Wrong ways to ft a lne : 1. y(x) a x +b (σ x 0). x(y) c y + d (σ y 0) 3. splt dfference between 1 and. Example: Prmordal He abundance: Extrapolate ft lne to [ O / H ] 0. [ He
More information3D Geometry for Computer Graphics. Lesson 2: PCA & SVD
3D Geometry for Computer Graphcs Lesso 2: PCA & SVD Last week - egedecomposto We wat to lear how the matrx A works: A 2 Last week - egedecomposto If we look at arbtrary vectors, t does t tell us much.
More informationStatistical pattern recognition
Statstcal pattern recognton Bayes theorem Problem: decdng f a patent has a partcular condton based on a partcular test However, the test s mperfect Someone wth the condton may go undetected (false negatve
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More information1 Derivation of Point-to-Plane Minimization
1 Dervaton of Pont-to-Plane Mnmzaton Consder the Chen-Medon (pont-to-plane) framework for ICP. Assume we have a collecton of ponts (p, q ) wth normals n. We want to determne the optmal rotaton and translaton
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More informationWhich Separator? Spring 1
Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal
More informationFace Recognition CS 663
Face Recognton CS 663 Importance of face recognton The most common way for humans to recognze each other Study of the process of face recognton has applcatons n () securty/survellance/authentcaton, ()
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationMEM 255 Introduction to Control Systems Review: Basics of Linear Algebra
MEM 255 Introducton to Control Systems Revew: Bascs of Lnear Algebra Harry G. Kwatny Department of Mechancal Engneerng & Mechancs Drexel Unversty Outlne Vectors Matrces MATLAB Advanced Topcs Vectors A
More informationQuantum Mechanics I - Session 4
Quantum Mechancs I - Sesson 4 Aprl 3, 05 Contents Operators Change of Bass 4 3 Egenvectors and Egenvalues 5 3. Denton....................................... 5 3. Rotaton n D....................................
More information332600_08_1.qxp 4/17/08 11:29 AM Page 481
336_8_.qxp 4/7/8 :9 AM Page 48 8 Complex Vector Spaces 8. Complex Numbers 8. Conjugates and Dvson of Complex Numbers 8.3 Polar Form and DeMovre s Theorem 8.4 Complex Vector Spaces and Inner Products 8.5
More informationMatrix Approximation via Sampling, Subspace Embedding. 1 Solving Linear Systems Using SVD
Matrx Approxmaton va Samplng, Subspace Embeddng Lecturer: Anup Rao Scrbe: Rashth Sharma, Peng Zhang 0/01/016 1 Solvng Lnear Systems Usng SVD Two applcatons of SVD have been covered so far. Today we loo
More informationFormulas for the Determinant
page 224 224 CHAPTER 3 Determnants e t te t e 2t 38 A = e t 2te t e 2t e t te t 2e 2t 39 If 123 A = 345, 456 compute the matrx product A adj(a) What can you conclude about det(a)? For Problems 40 43, use
More informationCSci 6974 and ECSE 6966 Math. Tech. for Vision, Graphics and Robotics Lecture 21, April 17, 2006 Estimating A Plane Homography
CSc 6974 and ECSE 6966 Math. Tech. for Vson, Graphcs and Robotcs Lecture 21, Aprl 17, 2006 Estmatng A Plane Homography Overvew We contnue wth a dscusson of the major ssues, usng estmaton of plane projectve
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationLinear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space.
Lnear, affne, and convex sets and hulls In the sequel, unless otherwse specfed, X wll denote a real vector space. Lnes and segments. Gven two ponts x, y X, we defne xy = {x + t(y x) : t R} = {(1 t)x +
More informationChat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980
MT07: Multvarate Statstcal Methods Mke Tso: emal mke.tso@manchester.ac.uk Webpage for notes: http://www.maths.manchester.ac.uk/~mkt/new_teachng.htm. Introducton to multvarate data. Books Chat eld, C. and
More informationReport on Image warping
Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.
More informationHongyi Miao, College of Science, Nanjing Forestry University, Nanjing ,China. (Received 20 June 2013, accepted 11 March 2014) I)ϕ (k)
ISSN 1749-3889 (prnt), 1749-3897 (onlne) Internatonal Journal of Nonlnear Scence Vol.17(2014) No.2,pp.188-192 Modfed Block Jacob-Davdson Method for Solvng Large Sparse Egenproblems Hongy Mao, College of
More informationUnit 5: Quadratic Equations & Functions
Date Perod Unt 5: Quadratc Equatons & Functons DAY TOPIC 1 Modelng Data wth Quadratc Functons Factorng Quadratc Epressons 3 Solvng Quadratc Equatons 4 Comple Numbers Smplfcaton, Addton/Subtracton & Multplcaton
More informationPattern Classification
Pattern Classfcaton All materals n these sldes ere taken from Pattern Classfcaton (nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wley & Sons, 000 th the permsson of the authors and the publsher
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationAdaptive Manifold Learning
Adaptve Manfold Learnng Jng Wang, Zhenyue Zhang Department of Mathematcs Zhejang Unversty, Yuquan Campus, Hangzhou, 327, P. R. Chna wroarng@sohu.com zyzhang@zju.edu.cn Hongyuan Zha Department of Computer
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More informationp 1 c 2 + p 2 c 2 + p 3 c p m c 2
Where to put a faclty? Gven locatons p 1,..., p m n R n of m houses, want to choose a locaton c n R n for the fre staton. Want c to be as close as possble to all the house. We know how to measure dstance
More informationLecture 4: Constant Time SVD Approximation
Spectral Algorthms and Representatons eb. 17, Mar. 3 and 8, 005 Lecture 4: Constant Tme SVD Approxmaton Lecturer: Santosh Vempala Scrbe: Jangzhuo Chen Ths topc conssts of three lectures 0/17, 03/03, 03/08),
More informationLecture 3. Ax x i a i. i i
18.409 The Behavor of Algorthms n Practce 2/14/2 Lecturer: Dan Spelman Lecture 3 Scrbe: Arvnd Sankar 1 Largest sngular value In order to bound the condton number, we need an upper bound on the largest
More informationCSE 252C: Computer Vision III
CSE 252C: Computer Vson III Lecturer: Serge Belonge Scrbe: Catherne Wah LECTURE 15 Kernel Machnes 15.1. Kernels We wll study two methods based on a specal knd of functon k(x, y) called a kernel: Kernel
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus
More informationAn Algorithm to Solve the Inverse Kinematics Problem of a Robotic Manipulator Based on Rotation Vectors
An Algorthm to Solve the Inverse Knematcs Problem of a Robotc Manpulator Based on Rotaton Vectors Mohamad Z. Al-az*, Mazn Z. Othman**, and Baker B. Al-Bahr* *AL-Nahran Unversty, Computer Eng. Dep., Baghdad,
More informationNorms, Condition Numbers, Eigenvalues and Eigenvectors
Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b
More informationHowever, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values
Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly
More informationCME 302: NUMERICAL LINEAR ALGEBRA FALL 2005/06 LECTURE 13
CME 30: NUMERICAL LINEAR ALGEBRA FALL 005/06 LECTURE 13 GENE H GOLUB 1 Iteratve Methods Very large problems (naturally sparse, from applcatons): teratve methods Structured matrces (even sometmes dense,
More informationMinimizing Algebraic Error in Geometric Estimation Problems
Mnmzng Algebrac n Geometrc Estmaton Problems Rchard I. Hartley G.E. Corporate Research and Development PO Box 8, Schenectady, NY 39 Emal : hartley@crd.ge.com Abstract Ths paper gves a wdely applcable technque
More informationProgramming Project 1: Molecular Geometry and Rotational Constants
Programmng Project 1: Molecular Geometry and Rotatonal Constants Center for Computatonal Chemstry Unversty of Georga Athens, Georga 30602 Summer 2012 1 Introducton Ths programmng project s desgned to provde
More informationFisher Linear Discriminant Analysis
Fsher Lnear Dscrmnant Analyss Max Wellng Department of Computer Scence Unversty of Toronto 10 Kng s College Road Toronto, M5S 3G5 Canada wellng@cs.toronto.edu Abstract Ths s a note to explan Fsher lnear
More informationLecture 6/7 (February 10/12, 2014) DIRAC EQUATION. The non-relativistic Schrödinger equation was obtained by noting that the Hamiltonian 2
P470 Lecture 6/7 (February 10/1, 014) DIRAC EQUATION The non-relatvstc Schrödnger equaton was obtaned by notng that the Hamltonan H = P (1) m can be transformed nto an operator form wth the substtutons
More informationSo far: simple (planar) geometries
Physcs 06 ecture 5 Torque and Angular Momentum as Vectors SJ 7thEd.: Chap. to 3 Rotatonal quanttes as vectors Cross product Torque epressed as a vector Angular momentum defned Angular momentum as a vector
More informationThe Prncpal Component Transform The Prncpal Component Transform s also called Karhunen-Loeve Transform (KLT, Hotellng Transform, oregenvector Transfor
Prncpal Component Transform Multvarate Random Sgnals A real tme sgnal x(t can be consdered as a random process and ts samples x m (m =0; ;N, 1 a random vector: The mean vector of X s X =[x0; ;x N,1] T
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and
More informationCS 3710: Visual Recognition Classification and Detection. Adriana Kovashka Department of Computer Science January 13, 2015
CS 3710: Vsual Recognton Classfcaton and Detecton Adrana Kovashka Department of Computer Scence January 13, 2015 Plan for Today Vsual recognton bascs part 2: Classfcaton and detecton Adrana s research
More informationNumber of cases Number of factors Number of covariates Number of levels of factor i. Value of the dependent variable for case k
ANOVA Model and Matrx Computatons Notaton The followng notaton s used throughout ths chapter unless otherwse stated: N F CN Y Z j w W Number of cases Number of factors Number of covarates Number of levels
More informationLecture 10: Dimensionality reduction
Lecture : Dmensonalt reducton g The curse of dmensonalt g Feature etracton s. feature selecton g Prncpal Components Analss g Lnear Dscrmnant Analss Intellgent Sensor Sstems Rcardo Guterrez-Osuna Wrght
More informationCHALMERS, GÖTEBORGS UNIVERSITET. SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS. COURSE CODES: FFR 135, FIM 720 GU, PhD
CHALMERS, GÖTEBORGS UNIVERSITET SOLUTIONS to RE-EXAM for ARTIFICIAL NEURAL NETWORKS COURSE CODES: FFR 35, FIM 72 GU, PhD Tme: Place: Teachers: Allowed materal: Not allowed: January 2, 28, at 8 3 2 3 SB
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More informationModule 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur
Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More informationA Novel Biometric Feature Extraction Algorithm using Two Dimensional Fisherface in 2DPCA subspace for Face Recognition
A Novel ometrc Feature Extracton Algorthm usng wo Dmensonal Fsherface n 2DPA subspace for Face Recognton R. M. MUELO, W.L. WOO, and S.S. DLAY School of Electrcal, Electronc and omputer Engneerng Unversty
More informationTensor Subspace Analysis
Tensor Subspace Analyss Xaofe He 1 Deng Ca Partha Nyog 1 1 Department of Computer Scence, Unversty of Chcago {xaofe, nyog}@cs.uchcago.edu Department of Computer Scence, Unversty of Illnos at Urbana-Champagn
More informationQuadratic speedup for unstructured search - Grover s Al-
Quadratc speedup for unstructured search - Grover s Al- CS 94- gorthm /8/07 Sprng 007 Lecture 11 001 Unstructured Search Here s the problem: You are gven a boolean functon f : {1,,} {0,1}, and are promsed
More informationSome Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)
Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998
More informationA New Refinement of Jacobi Method for Solution of Linear System Equations AX=b
Int J Contemp Math Scences, Vol 3, 28, no 17, 819-827 A New Refnement of Jacob Method for Soluton of Lnear System Equatons AX=b F Naem Dafchah Department of Mathematcs, Faculty of Scences Unversty of Gulan,
More informationHomework Notes Week 7
Homework Notes Week 7 Math 4 Sprng 4 #4 (a Complete the proof n example 5 that s an nner product (the Frobenus nner product on M n n (F In the example propertes (a and (d have already been verfed so we
More informationactuary )uj, problem L It X Projection approximation Orthogonal subspace of !- ( iii. ( run projection of Pcxzll rim orthogonal ( Nuu, y H, ye W )
dmensonal ( Note thot rm t w ; n ( ( run u! ( t E 21 k f } actuary uj n ( Nuu h rate h Hutt O Snce when Truth Klment we also have wht Lu te It k} Orthogonal Projecton let W be an un subspace a Eucldean
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationReview: Fit a line to N data points
Revew: Ft a lne to data ponts Correlated parameters: L y = a x + b Orthogonal parameters: J y = a (x ˆ x + b For ntercept b, set a=0 and fnd b by optmal average: ˆ b = y, Var[ b ˆ ] = For slope a, set
More informationGeometric Registration for Deformable Shapes. 2.1 ICP + Tangent Space optimization for Rigid Motions
Geometrc Regstraton for Deformable Shapes 2.1 ICP + Tangent Space optmzaton for Rgd Motons Regstraton Problem Gven Two pont cloud data sets P (model) and Q (data) sampled from surfaces Φ P and Φ Q respectvely.
More informationBody Models I-2. Gerard Pons-Moll and Bernt Schiele Max Planck Institute for Informatics
Body Models I-2 Gerard Pons-Moll and Bernt Schele Max Planck Insttute for Informatcs December 18, 2017 What s mssng Gven correspondences, we can fnd the optmal rgd algnment wth Procrustes. PROBLEMS: How
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationAffine transformations and convexity
Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/
More informationVECTORS AND MATRICES:
VECTORS AND MATRICES: Matrx: a rectangular array of elements Dmenson: rxc means r rows by b columns Example: A = [a ], =1,2,3; j=1,2 j In general: A = [a ], =1, r; j=1,...c j Multvarate observatons=vector:
More informationBezier curves. Michael S. Floater. August 25, These notes provide an introduction to Bezier curves. i=0
Bezer curves Mchael S. Floater August 25, 211 These notes provde an ntroducton to Bezer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of the
More informationVector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.
Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm
More informationProtein Structure Comparison
Proten Structure Comparson Proten Structure Representaton CPK: hard sphere model Ball-and-stck Cartoon Degrees of Freedom n Protens Bond length Dhedral angle 3 4 Bond angle + Proten Structure: Varables
More informationMathematical Preparations
1 Introducton Mathematcal Preparatons The theory of relatvty was developed to explan experments whch studed the propagaton of electromagnetc radaton n movng coordnate systems. Wthn expermental error the
More informationLECTURE :FACTOR ANALYSIS
LCUR :FACOR ANALYSIS Rta Osadchy Based on Lecture Notes by A. Ng Motvaton Dstrbuton coes fro MoG Have suffcent aount of data: >>n denson Use M to ft Mture of Gaussans nu. of tranng ponts If
More informationPoint cloud to point cloud rigid transformations. Minimizing Rigid Registration Errors
Pont cloud to pont cloud rgd transformatons Russell Taylor 600.445 1 600.445 Fall 000-015 Mnmzng Rgd Regstraton Errors Typcally, gven a set of ponts {a } n one coordnate system and another set of ponts
More information7. Products and matrix elements
7. Products and matrx elements 1 7. Products and matrx elements Based on the propertes of group representatons, a number of useful results can be derved. Consder a vector space V wth an nner product ψ
More informationMachine Learning for Signal Processing
Machne Learnng for Sgnal Processng Lecture 6: Optmzaton Data-drven Representatons Abelno Jmenez Generalzes to nequalty constrants Optmzaton problem wth constrants mn s. t. g ( x) Lagrange multplers l ³
More informationBézier curves. Michael S. Floater. September 10, These notes provide an introduction to Bézier curves. i=0
Bézer curves Mchael S. Floater September 1, 215 These notes provde an ntroducton to Bézer curves. 1 Bernsten polynomals Recall that a real polynomal of a real varable x R, wth degree n, s a functon of
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationNon-linear Canonical Correlation Analysis Using a RBF Network
ESANN' proceedngs - European Smposum on Artfcal Neural Networks Bruges (Belgum), 4-6 Aprl, d-sde publ., ISBN -97--, pp. 57-5 Non-lnear Canoncal Correlaton Analss Usng a RBF Network Sukhbnder Kumar, Elane
More informationFall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede
Fall 0 Analyss of Expermental easurements B. Esensten/rev. S. Errede We now reformulate the lnear Least Squares ethod n more general terms, sutable for (eventually extendng to the non-lnear case, and also
More informationProblem Do any of the following determine homomorphisms from GL n (C) to GL n (C)?
Homework 8 solutons. Problem 16.1. Whch of the followng defne homomomorphsms from C\{0} to C\{0}? Answer. a) f 1 : z z Yes, f 1 s a homomorphsm. We have that z s the complex conjugate of z. If z 1,z 2
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationSimilarities, Distances and Manifold Learning
Smlartes, Dstances and Manfold Learnng Prof. Rchard C. Wlson Dept. of Computer Scence Unversty of York Part I: Eucldean Space Poston, Smlarty and Dstance Manfold Learnng n Eucldean space Some famous technques
More informationCOMPLEX NUMBERS AND QUADRATIC EQUATIONS
COMPLEX NUMBERS AND QUADRATIC EQUATIONS INTRODUCTION We know that x 0 for all x R e the square of a real number (whether postve, negatve or ero) s non-negatve Hence the equatons x, x, x + 7 0 etc are not
More informationQuantum Mechanics for Scientists and Engineers. David Miller
Quantum Mechancs for Scentsts and Engneers Davd Mller Types of lnear operators Types of lnear operators Blnear expanson of operators Blnear expanson of lnear operators We know that we can expand functons
More information5 The Rational Canonical Form
5 The Ratonal Canoncal Form Here p s a monc rreducble factor of the mnmum polynomal m T and s not necessarly of degree one Let F p denote the feld constructed earler n the course, consstng of all matrces
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationParameter estimation class 5
Parameter estmaton class 5 Multple Ve Geometr Comp 9-89 Marc Pollefes Content Background: Projectve geometr (D, 3D), Parameter estmaton, Algortm evaluaton. Sngle Ve: Camera model, Calbraton, Sngle Ve Geometr.
More informationExplicit constructions of all separable two-qubits density matrices and related problems for three-qubits systems
Explct constructons of all separable two-qubts densty matrces and related problems for three-qubts systems Y. en-ryeh and. Mann Physcs Department, Technon-Israel Insttute of Technology, Hafa 2000, Israel
More informationMATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS
MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples
More informationTracking with Kalman Filter
Trackng wth Kalman Flter Scott T. Acton Vrgna Image and Vdeo Analyss (VIVA), Charles L. Brown Department of Electrcal and Computer Engneerng Department of Bomedcal Engneerng Unversty of Vrgna, Charlottesvlle,
More informationWeek 11: Chapter 11. The Vector Product. The Vector Product Defined. The Vector Product and Torque. More About the Vector Product
The Vector Product Week 11: Chapter 11 Angular Momentum There are nstances where the product of two vectors s another vector Earler we saw where the product of two vectors was a scalar Ths was called the
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationFour Bar Linkages in Two Dimensions. A link has fixed length and is joined to other links and also possibly to a fixed point.
Four bar lnkages 1 Four Bar Lnkages n Two Dmensons lnk has fed length and s oned to other lnks and also possbly to a fed pont. The relatve velocty of end B wth regard to s gven by V B = ω r y v B B = +y
More informationMachine Learning for Signal Processing Linear Gaussian Models
Machne Learnng for Sgnal rocessng Lnear Gaussan Models lass 2. 2 Nov 203 Instructor: Bhsha Raj 2 Nov 203 755/8797 HW3 s up. Admnstrva rojects please send us an update 2 Nov 203 755/8797 2 Recap: MA stmators
More information