CS 523: Computer Graphcs, Sprng 20 Shape Modelng PCA Applcatons + SVD Andrew Nealen, utgers, 20 2/5/20
emnder: PCA Fnd prncpal components of data ponts Orthogonal drectons that are domnant n the data (have varance etrema) y y Scatter matr S X X v λ λ2 v S v v 2 v 2 d λ d v d Andrew Nealen, utgers, 20 2/5/20 2
More applcatons of PCA Morphable models of faces Data base of face scans: 3D geometry + teture (photo) 0,000 ponts n each scan, y, z,, G, B 6 numbers for each pont hus, each scan s a 0,000*6 60,000-dmensonal vector See: V. Blanzand. Vetter, A MorphableModel for the Synthess of 3D Faces, SIGGAPH 99 Andrew Nealen, utgers, 20 2/5/20 3
More applcatons of PCA Morphable models of faces How to fnd nterestng aes s ths 60000-dmensonal space? aes that measures age, gender, etc here s hope: the faces are lkely to be governed by a small set of parameters (much less than 60,000 ) age as gender as FaceGen demo Andrew Nealen, utgers, 20 2/5/20 4
Sngular Value Decomposton Andrew Nealen, utgers, 20 2/5/20 5
Geometrc analyss of lnear We want to know what a lnear transformaton Adoes transformatons Need some smple and comprehensble representaton of the matr A Let s look what Adoes to some vectors Snce A(αv) αa(v), t s enough to look at vectors vof unt length A Andrew Nealen, utgers, 20 2/5/20 6
Geometrc analyss of lnear transformatons A lnear (non-sngular) transform Aalways takes hyper-spheres to hyper-ellpses. A A Andrew Nealen, utgers, 20 2/5/20 7
Geometrc analyss of lnear transformatons hus, one good way to understand what A does s to fnd whch vectors are mapped to the man aes of the ellpsod A A Andrew Nealen, utgers, 20 2/5/20 8
Geometrc analyss of lnear transformatons If As symmetrc: A V D V, Vorthogonal he egenvectors of Aare the aes of the ellpse A Andrew Nealen, utgers, 20 2/5/20 9
Symmetrc matr: egendecomposton In ths case As just a scalng matr. he egendecompostonof Atells us whch orthogonal aes t scales, and by how much A λ 2 λ A λ 2 [ v v Kv ] [ v v Kv ] 2 n 2 Andrew Nealen, utgers, 20 2/5/20 λ O Av λv λn n 0
General lnear transformatons: Sngular Value Decomposton In general Awll also contan rotatons, not just scales A σ 2 σ A U Σ V A σ σ 2 [ u u Ku ] [ v v Kv ] O σ n 2 n 2 n Andrew Nealen, utgers, 20 2/5/20
General lnear transformatons: Sngular Value Decomposton A σ 2 σ A V U Σ A orthonormal orthonormal [ v v Kv ] [ u u Ku ] 2 n 2 n σ σ 2 O σ n Av σ u, σ 0 Andrew Nealen, utgers, 20 2/5/20 2
Some hstory SVD was dscovered by the followng people: E. Beltram (835 900) M. Jordan (838 922) J. Sylvester (84 897) E. Schmdt (876-959) H. Weyl (885-955) Andrew Nealen, utgers, 20 2/5/20 3
SVD SVD ests for any matr Formal defnton: For square matrces A n n, there est orthogonal matrces U, V n n and a dagonal matr Σ, such that all the dagonal values σ of Σare non-negatve and A U Σ V A U Σ V Andrew Nealen, utgers, 20 2/5/20 4
SVD he dagonal values of Σare called the sngular values.it s accustomed to sort them: σ σ 2 σ n he columns of U(u,, u n ) are called the left sngular vectors. hey are the aes of the ellpsod. he columns of V(v,, v n ) are called the rght sngular vectors. hey are the premages of the aes of the ellpsod. A U Σ V A U Σ V Andrew Nealen, utgers, 20 2/5/20 5
educed SVD For rectangular matrces, we have two forms of SVD. he reduced SVD looks lke ths: he columns of Uare orthonormal Cheaper form for computaton and storage A U Σ V Andrew Nealen, utgers, 20 2/5/20 6
Full SVD We can complete Uto a full orthogonal matr and pad Σby zeros accordngly A U Σ V Andrew Nealen, utgers, 20 2/5/20 7
here are stable numercal algorthms to SVD Applcatons compute SVD (albet not cheap). Once you have t, you have many thngs: Matr nverse can solve square lnear systems Numercal rank of a matr Can solve lnear least-squares systems PCA Many more Andrew Nealen, utgers, 20 2/5/20 8
Matr nverse and solvng lnear systems Matr nverse So, to solve Andrew Nealen, utgers, 20 2/5/20 9 ( ) ( ) U V U V V U A V U A n Σ Σ Σ σ σ O b b U V A Σ
Matr rank he rank of As the number of non-zero sngular values n σ σ2 m σ n A U Σ V Andrew Nealen, utgers, 20 2/5/20 20
Numercal rank If there are very small sngular values, then A s close to beng sngular. We can set a threshold t, so that numerc_rank(a) #{σ σ > t} Usng SVD s a numercally stable way! he determnant s not a good way to check sngularty Andrew Nealen, utgers, 20 2/5/20 2
PCA Construct the matr X of the centered data ponts he prncpal aes are egenvectors of S XX Andrew Nealen, utgers, 20 2/5/20 22 p n p p L 2 X U U XX S λ d λ O
PCA We can compute the prncpal components by SVD of X: X UΣV XX UΣV (UΣV ) UΣV VΣU UΣ 2 U hus, the left sngular vectors of Xare the prncpal components! We sort them by the sze of the sngular values of X. Andrew Nealen, utgers, 20 2/5/20 23
Least-squares rotaton wth SVD Andrew Nealen, utgers, 20 2/5/20 24
Shape matchng We have two objects n correspondence Want to fnd the rgd transformaton that algns them Andrew Nealen, utgers, 20 2/5/20 25
Shape matchng When the objects are algned, the lengths of the connectng lnes are small Andrew Nealen, utgers, 20 2/5/20 26
Optmal local rotaton We wll use ths for mesh deformaton v v j2 v v j v j v j 2 Andrew Nealen, utgers, 20 2/5/20 27
Shape matchng formalzaton Algn two pont sets P p, K, p and Q q, K, q. { } { } n Fnd a translaton vector tand rotaton matr so that n n 2 ( p + t) s mnmzed q Andrew Nealen, utgers, 20 2/5/20 28
Shape matchng soluton Solve translaton and rotaton separately If (, t)s the optmal transformaton, then the pont sets {p + t}and {q }have the same centers of mass Andrew Nealen, utgers, 20 2/5/20 29 n n p p n n q q ( ) p q t t p t p t p q + + + n n n n
Fndng the rotaton o fnd the optmal, we brng the centrods of both pont sets to the orgn We want to fnd that mnmzes Andrew Nealen, utgers, 20 2/5/20 30 q q y p p n 2 y
( ) ( ) ( ) + n n n 2 y y y y y y y Fndng the rotaton Andrew Nealen, utgers, 20 2/5/20 hese terms do not depend on, so we can gnore them n the mnmzaton 3 I
Fndng the rotaton Andrew Nealen, utgers, 20 2/5/20 ths s a scalar 32 ( ) ( ) ( ) + n n n argma ma mn y y y y y y y y
Fndng the rotaton n y tr ( Y X) tr n ( A) A y y 2 M 2 L n y y 2 M 2 L n y n y n Y X Andrew Nealen, utgers, 20 2/5/20 33
Fndng the rotaton n y tr ( Y X) tr n ( A) A y y 2 M 2 L n y y 2 2 O y n y n n Andrew Nealen, utgers, 20 2/5/20 34
Fndng the rotaton Fnd that mamzes tr ( ) ( Y X tr XY ) (because tr(ab) tr(ba)) Let s do SVD on S XY tr S XY UΣV ( ) ( ) ( ( XY tr UΣV trσ V U ) orthogonal matr Andrew Nealen, utgers, 20 2/5/20 35
Fndng the rotaton We want to mamze tr ( ( Σ V U ) orthogonal matr all entres σ σ 2 σ 3 m M L m 22 L M m 33 tr ( ( Σ V U ) m 3 3 σ σ Andrew Nealen, utgers, 20 2/5/20 36
tr Fndng the rotaton ( ( Σ V U ) m Our best shot s m,.e. to make V U I V U 3 U VU 3 σ σ V I Andrew Nealen, utgers, 20 2/5/20 37
Summary of rgd algnment ranslate the nput ponts to the centrods Compute the covarance matr Compute the SVD of S p p y q he optmal orthogonal s S S XY Andrew Nealen, utgers, 20 2/5/20 38 UΣV VU n y q
Sgn correcton It s possble that det(vu ) : sometmes reflecton s the best orthogonal transform Andrew Nealen, utgers, 20 2/5/20 39
Sgn correcton It s possble that det(vu ) : sometmes reflecton s the best orthogonal transform Andrew Nealen, utgers, 20 2/5/20 40
Sgn correcton It s possble that det(vu ) : sometmes reflecton s the best orthogonal transform o restrct ourselves to rotatons only: take the last column of V(correspondng to the smallest sngular value) and nvert ts sgn. Why? See the PDF Andrew Nealen, utgers, 20 2/5/20 4
Complety Numercal SVD s an epensve operaton O(mn(mn 2,nm 2 )) We always need to pay attenton to the dmensons of the matr we re applyng SVD to. Andrew Nealen, utgers, 20 2/5/20 42
SVD for anmaton compresson Chcken anmaton See: epresentng Anmatons by Prncpal Components, M. Alea and W. Muller, Eurographcs 2000 Compresson of Soft-Body Anmaton Sequences, Z. Karn and C. Gotsman, Computers&Graphcs 28(): 25-34, 2004 Key Pont Subspace Acceleraton and Soft Cachng, M. Meyer and J. Anderson, SIGGAPH 2007 Andrew Nealen, utgers, 20 2/5/20 43
3D anmatons Each frame s a 3D model (mesh) Andrew Nealen, utgers, 20 2/5/20 44
3D anmatons Connectvty s usually constant (at least on large segments of the anmaton) he geometry changes n each frame vast amount of data! 3 seconds, 3000 vertces/frame, 26 MB Andrew Nealen, utgers, 20 2/5/20 45
46 Anmaton compresson by dmensonalty reducton he geometry of each frame s a vector n 3N space (N #vertces) N N N y y z z M M M 3N #f 2/5/20 Andrew Nealen, utgers, 20
47 Anmaton compresson by dmensonalty reducton Fnd a few vectors of 3N that wll best represent our frame vectors! N N N y y z z M M M 2 f σ σ σ O 2 f σ σ σ O V U 3N f Σ f f V f f 2 f σ σ σ K 2/5/20 Andrew Nealen, utgers, 20
48 Anmaton compresson by dmensonalty reducton he frst prncpal components are the mportant ones N N N y y z z M M M u u 2 u 3 2 f σ σ σ O V 2 3 0 0 σ σ σ 2/5/20 Andrew Nealen, utgers, 20
Anmaton compresson by dmensonalty reducton Appromate each frame by lnear combnaton of the frst prncpal components he more components we use, the better the appromaton Usually, the number of components needed s much smaller than f. Andrew Nealen, utgers, 20 M N y M yn z M z N α u + α 2 u 2 + α 3 u 3 2/5/20 49
Anmaton compresson by dmensonalty reducton Compressed representaton: he chosen prncpal component vectors u Coeffcents α for each frame Anmaton wth only 2 prncpal components Anmaton wth 20 out of 400 prncpal components Andrew Nealen, utgers, 20 2/5/20 50
Egenfaces Same prncpal components analyss can be appled to mages Andrew Nealen, utgers, 20 2/5/20 5
Egenfaces Each mage s a vector n 250 300 Want to fnd the prncpal aes vectors that best represent the nput database of mages Andrew Nealen, utgers, 20 2/5/20 52
econstructon wth a few vectors epresent each mage by the frst few (n) prncpal components (,, K, ) v αu + αu + Kα u α α α 2 2 n n 2 n Andrew Nealen, utgers, 20 2/5/20 53
Face recognton Gven a new mage of a face, w 250 300 epresent wusng the frst n PCA vectors: (,, K, ) w αu + αu + Kα u α α α 2 2 n n 2 n Now fnd an mage n the database whose representaton n the PCA bass s the closest: w w (,, K, ) w α α2 α n w, w s the largest he angle between wand w s the smallest w w Andrew Nealen, utgers, 20 2/5/20 54
Non-lnear dmensonalty reducton More sophstcated methods can dscover non-lnear structures n the face datasets Isomap, Scence, Dec. 2000 Andrew Nealen, utgers, 20 2/5/20 55