3D Geometry for Computer Graphics. Lesson 2: PCA & SVD
|
|
- Marlene Randall
- 6 years ago
- Views:
Transcription
1 3D Geometry for Computer Graphcs Lesso 2: PCA & SVD
2 Last week - egedecomposto We wat to lear how the matrx A works: A 2
3 Last week - egedecomposto If we look at arbtrary vectors, t does t tell us much. A 3
4 Spectra ad dagoalzato If A s symmetrc, the egevectors are orthogoal (ad there s always a egebass). A A = UΛU λ1 λ2 Λ= Au = λ u λ 4
5 Why SVD Dagoalzable matrx s essetally a scalg. Most matrces are ot dagoalzable they do other thgs alog wth scalg (such as rotato) So, to uderstad how geeral matrces behave, oly egevalues are ot eough SVD tells us how geeral lear trasformatos behave, ad other thgs 5
6 he pla for today Frst we ll see some applcatos of PCA Prcpal Compoet Aalyss that uses spectral decomposto. he look at the theory. SVD Basc tuto Formal defto Applcatos 6
7 PCA the geeral dea PCA fds a orthogoal bass that best represets gve data set. y y x x he sum of dstaces 2 from the x axs s mmzed. 7
8 PCA the geeral dea PCA fds a orthogoal bass that best represets gve data set. z 3D pot set stadard bass x y PCA fds a best approxmatg plae (aga, terms of Σdstaces 2 ) 8
9 Applcato: fdg tght boudg box A axs-alged boudg box: agrees wth the axes y maxy mx maxx x my 9
10 Applcato: fdg tght boudg box Oreted boudg box: we fd better axes! y x 10
11 Applcato: fdg tght boudg box hs s ot the optmal boudg box z x y 11
12 Applcato: fdg tght boudg box Oreted boudg box: we fd better axes! 12
13 Usage of boudg boxes (boudg volumes) Serve as very smple approxmato of the object Fast collso detecto, vsblty queres Wheever we eed to kow the dmesos (sze) of the object he models cosst of thousads of polygos o quckly test that they do t tersect, the boudg boxes are tested Sometmes a herarchy of BB s s used he tghter the BB the less false alarms we have Sample 13
14 Scaed meshes 14
15 Pot clouds Scaers gve us raw pot cloud data How to compute ormals to shade the surface? ormal 15
16 Pot clouds Local PCA, take the thrd vector 16
17 Notatos Deote our data pots by x 1, x 2,, x R d x 1 x 2 x x1 x2 x x,,, 1 = x = = 2 x d d d x 1 x 2 x 17
18 he org of the ew axes he org s zero-order approxmato of our data set (a pot) It wll be the ceter of mass: m It ca be show that: = 1 = 1 x m = argm x -x x =
19 Scatter matrx Deote y = x m, = 1, 2,, S = YY where Y s d matrx wth y k as colums (k = 1, 2,, ) S d y1 y2 y y1 y1 y d y1 y2 y y2 y2 y2 = d d d 1 2 d y1 y2 y y y y Y Y 19
20 Varace of projected pots I a way, S measures varace (= scatteress) of the data dfferet drectos. Let s look at a le L through the ceter of mass m, ad project our pots x oto t. he varace of the projected pots x s: 1 2 var( L) = x m = 1 L L L L Orgal set Small varace Large varace 20
21 Varace of projected pots Gve a drecto v, v = 1, the projecto of x oto L = m + vt s: x m = < v, x m> / v = < v, y > = v y x L m v x 21
22 Varace of projected pots So, x-m vy v Y = 1 = 1 var( L) = = ( ) = = = Y v = < Y v, Y v>= v YY v= v Sv= < Sv, v> y y1 y2 y d y 1 2 d y1 y2 y 2 ( ) ( v v v ) ( v v v ) vy = = = vy = 1 = 1 d d d d y y1 y2 y 22
23 Drectos of maxmal varace So, we have: var(l) = <Sv, v> heorem: Let f : {v R d v = 1} R, f (v) = <Sv, v> (ad S s a symmetrc matrx). he, the extrema of f are attaed at the egevectors of S. So, egevectors of S are drectos of maxmal/mmal varace! 23
24 Summary so far We take the cetered data vectors y 1, y 2,, y R d Costruct the scatter matrx S = YY S measures the varace of the data pots Egevectors of S are drectos of maxmal varace. 24
25 Scatter matrx - egedecomposto S s symmetrc S has egedecomposto: S = VΛV λ 1λ2 v 1 S = v 1 v 2 v d v 2 λ d v d he egevectors form orthogoal bass 25
26 Prcpal compoets Egevectors that correspod to bg egevalues are the drectos whch the data has strog compoets (= large varace). If the egevalues are more or less the same there s o preferable drecto. Note: the egevalues are always o-egatve. 26
27 Prcpal compoets here s o preferable drecto S looks lke ths: V λ V λ Ay vector s a egevector here s a clear preferable drecto S looks lke ths: λ V V μ μ s close to zero, much smaller tha λ. 27
28 How to use what we got For fdg oreted boudg box we smply compute the boudg box wth respect to the axes defed by the egevectors. he org s at the mea pot m. v 3 v 1 v 2 28
29 For approxmato y v 2 v 1 x y y x x hs le segmet approxmates the orgal data set he projected data set approxmates the orgal data set 29
30 For approxmato I geeral dmeso d, the egevalues are sorted descedg order: λ 1 λ 2 λ d he egevectors are sorted accordgly. o get a approxmato of dmeso d < d, we take the d frst egevectors ad look at the subspace they spa (d = 1 s a le, d = 2 s a plae ) 30
31 For approxmato o get a approxmatg set, we project the orgal data pots oto the chose subspace: x = m + α 1 v 1 + α 2 v α d v d + +α d v d Projecto: x = m + α 1 v 1 + α 2 v α d v d +0 v d v d 31
32 SVD
33 Geometrc aalyss of lear trasformatos We wat to kow what a lear trasformato A does Need some smple ad comprehedble represetato of the matrx of A. Let s look what A does to some vectors Sce A(αv) = αa(v), t s eough to look at vectors v of ut legth A 33
34 he geometry of lear trasformatos A lear (o-sgular) trasform A always takes hyper-spheres to hyper-ellpses. A A 34
35 he geometry of lear trasformatos hus, oe good way to uderstad what A does s to fd whch vectors are mapped to the ma axes of the ellpsod. A A 35
36 Geometrc aalyss of lear trasformatos If we are lucky: A = V Λ V, V orthogoal (true f A s symmetrc) he egevectors of A are the axes of the ellpse A 36
37 37 Symmetrc matrx: ege decomposto I ths case A s just a scalg matrx. he ege decomposto of A tells us whch orthogoal axes t scales, ad by how much: [ ] [ ] A λ λ λ = v v v v v v A 1 1 λ 2 λ 1 A v v λ =
38 Geeral lear trasformatos: SVD I geeral A wll also cota rotatos, ot just scales: 1 1 A σ 2 σ 1 A = U V A = σ 1 σ 2 [ u u u ] [ v v v ] σ
39 Geeral lear trasformatos: SVD 1 1 A σ 2 σ 1 AV = U A orthoormal [ v v v ] = [ u u u ] orthoormal σ 1 σ 2 σ A v = σ u, σ 0 39
40 SVD more formally SVD exsts for ay matrx Formal defto: For square matrces A R, there exst orthogoal matrces U, V R ad a dagoal matrx Σ, such that all the dagoal values σ of Σ are o-egatve ad A = UΣV = A U Σ V 40
41 SVD more formally he dagoal values of Σ (σ 1,, σ ) are called the sgular values. It s accustomed to sort them: σ 1 σ 2 σ he colums of U (u 1,, u ) are called the left sgular vectors. hey are the axes of the ellpsod. he colums of V (v 1,, v ) are called the rght sgular vectors. hey are the premages of the axes of the ellpsod. A = UΣV = A U Σ V 41
42 Reduced SVD For rectagular matrces, we have two forms of SVD. he reduced SVD looks lke ths: he colums of U are orthoormal Cheaper form for computato ad storage = A U Σ V 42
43 Full SVD We ca complete U to a full orthogoal matrx ad pad Σ by zeros accordgly = A U Σ V 43
44 Some hstory SVD was dscovered by the followg people: E. Beltram ( ) M. Jorda ( ) J. Sylvester ( ) E. Schmdt ( ) H. Weyl ( ) 44
45 SVD s the workg horse of lear algebra here are umercal algorthms to compute SVD. Oce you have t, you have may thgs: Matrx verse ca solve square lear systems Numercal rak of a matrx Ca solve least-squares systems PCA May more 45
46 Matrx verse ad solvg lear systems Matrx verse: So, to solve Ax= x A = U V ( ) ( ) A = U V = V U = = V 1 σ 1 = V U 1 σ b U 1 b 46
47 Matrx rak he rak of A s the umber of o-zero sgular values σ 1σ2 m = σ A U Σ V 47
48 Numercal rak If there are very small sgular values, the A s close to beg sgular. We ca set a threshold t, so that umerc_rak(a) = #{σ σ > t} If rak(a) < the A s sgular. It maps the etre space R oto some subspace, lke a plae (so A s some sort of projecto). 48
49 Back to PCA We wated to fd prcpal compoets y y x x 49
50 PCA Move the ceter of mass to the org p = p m y y x x 50
51 PCA Costructed the matrx X of the data pots. X p p p = 1 2 he prcpal axes are egevectors of S = XX λ 1 XX = S = U U λ d 51
52 PCA We ca compute the prcpal compoets by SVD of X: X = UΣV XX = UΣV ( UΣ V ) = = Σ Σ = Σ 2 U V V U U U hus, the left sgular vectors of X are the prcpal compoets! We sort them by the sze of the sgular values of X. 52
53 Shape matchg We have two objects correspodece Wat to fd the rgd trasformato that algs them 53
54 Shape matchg Whe the objects are alged, the legths of the coectg les are small. 54
55 Shape matchg formalzato Alg two pot sets P = p,, p ad Q= q,, q. { } { } 1 1 Fd a traslato vector t ad rotato matrx R so that: p = 1 2 ( Rq + t) s mmzed 55
56 Shape matchg soluto urs out we ca solve the traslato ad rotato separately. heorem: f (R, t) s the optmal trasformato, the the pots {p } ad {Rq + t} have the same ceters of mass. p 1 p = 1 = q 1 q = 1 = p 1 = + = 1 ( Rq t ) 1 p = R q + = R + t q t = 1 t = p Rq 56
57 Fdg the rotato R o fd the optmal R, we brg the cetrods of both pot sets to the org: p = p p q = q q We wat to fd R that mmzes p Rq =
58 Fdg the rotato R 2 ( ) ( ) = 1 = 1 = 1 p Rq = p Rq p Rq = ( p ) p p Rq q R p q R Rq = + I hese terms do ot deped o R, so we ca gore them the mmzato 58
59 Fdg the rotato R ( ) ( p ) Rq q R p = p Rq + q R p m max. = 1 = 1 R = R = R ( ) q p q p p q ( ) ( p ) Rq p Rq max 2 = max = 1 = 1 ths s a scalar 59
60 Fdg the rotato R p q qp qp = 1 = 1 = 1 ( R ) = race R = race R H = = 1 qp = = 3 3 race( A) = = 1 A 60
61 Fdg the rotato R So, we wat to fd R that maxmzes race( RH ) heorem: f M s symmetrc postve defte (all egevalues of M are postve) ad B s ay orthogoal matrx the race M race BM ( ) ( ) So, let s fd R so that RH s symmetrc postve defte. he we kow for sure that race(rh) s maxmal. 61
62 Fdg the rotato R hs s easy! Compute SVD of H: Defe R: Check RH: H = UΣV R = VU ( )( ) RH = VU UΣ V = VΣV hs s a symmetrc matrx, Its egevalues are σ 0 So RH s postve! 62
63 Summary of rgd algmet: raslate the put pots to the cetrods: Compute the covarace matrx Compute the SVD of H: he optmal rotato s he traslato vector s p = p p = q q H = qp = 1 H = UΣV R = VU t = p Rq q 63
64 Complexty Numercal SVD s a expesve operato We always eed to pay atteto to the dmesos of the matrx we re applyg SVD to. 64
65 Small somewhat related example 65
66 he Ed
Chapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More informationSingular Value Decomposition. Linear Algebra (3) Singular Value Decomposition. SVD and Eigenvectors. Solving LEs with SVD
Sgular Value Decomosto Lear Algera (3) m Cootes Ay m x matrx wth m ca e decomosed as follows Dagoal matrx A UWV m x x Orthogoal colums U U I w1 0 0 w W M M 0 0 x Orthoormal (Pure rotato) VV V V L 0 L 0
More informationQR Factorization and Singular Value Decomposition COS 323
QR Factorzato ad Sgular Value Decomposto COS 33 Why Yet Aother Method? How do we solve least-squares wthout currg codto-squarg effect of ormal equatos (A T A A T b) whe A s sgular, fat, or otherwse poorly-specfed?
More information4 Inner Product Spaces
11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More information1 Convergence of the Arnoldi method for eigenvalue problems
Lecture otes umercal lear algebra Arold method covergece Covergece of the Arold method for egevalue problems Recall that, uless t breaks dow, k steps of the Arold method geerates a orthogoal bass of a
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationHomework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015
Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts
More informationAnnouncements. Recognition II. Computer Vision I. Example: Face Detection. Evaluating a binary classifier
Aoucemets Recogto II H3 exteded to toght H4 to be aouced today. Due Frday 2/8. Note wll take a whle to ru some thgs. Fal Exam: hursday 2/4 at 7pm-0pm CSE252A Lecture 7 Example: Face Detecto Evaluatg a
More informationA scalar t is an eigenvalue of A if and only if t satisfies the characteristic equation of A: det (A ti) =0
Chapter 5 a glace: Let e a lear operator whose stadard matrx s wth sze x. he, a ozero vector x s sad to e a egevector of ad f there exsts a scalar sch that (x) x x. he scalar s called a egevale of (or
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationGeometric Suffix Tree: A New Index Structure for Protein 3-D Structures
CPM 2006 Geometrc Suffx ree: A New Idex Structure for Prote 3-D Structures etsuo Shbuya Huma Geome Ceter, Isttute of Medcal Scece, Uversty of okyo oday's alk Backgrouds Prote structures Suffx rees Geometrc
More informationPrincipal Components. Analysis. Basic Intuition. A Method of Self Organized Learning
Prcpal Compoets Aalss A Method of Self Orgazed Learg Prcpal Compoets Aalss Stadard techque for data reducto statstcal patter matchg ad sgal processg Usupervsed learg: lear from examples wthout a teacher
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationIntroduction to Matrices and Matrix Approach to Simple Linear Regression
Itroducto to Matrces ad Matrx Approach to Smple Lear Regresso Matrces Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people,
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationIntegral Equation Methods. Jacob White. Thanks to Deepak Ramaswamy, Michal Rewienski, Xin Wang and Karen Veroy
Itroducto to Smulato - Lecture 22 Itegral Equato ethods Jacob Whte Thaks to Deepak Ramaswamy, chal Rewesk, X Wag ad Kare Veroy Outle Itegral Equato ethods Exteror versus teror problems Start wth usg pot
More informationPROJECTION PROBLEM FOR REGULAR POLYGONS
Joural of Mathematcal Sceces: Advaces ad Applcatos Volume, Number, 008, Pages 95-50 PROJECTION PROBLEM FOR REGULAR POLYGONS College of Scece Bejg Forestry Uversty Bejg 0008 P. R. Cha e-mal: sl@bjfu.edu.c
More informationComputational Geometry
Problem efto omputatoal eometry hapter 6 Pot Locato Preprocess a plaar map S. ve a query pot p, report the face of S cotag p. oal: O()-sze data structure that eables O(log ) query tme. pplcato: Whch state
More informationIII-16 G. Brief Review of Grand Orthogonality Theorem and impact on Representations (Γ i ) l i = h n = number of irreducible representations.
III- G. Bref evew of Grad Orthogoalty Theorem ad mpact o epresetatos ( ) GOT: h [ () m ] [ () m ] δδ δmm ll GOT puts great restrcto o form of rreducble represetato also o umber: l h umber of rreducble
More informationGG313 GEOLOGICAL DATA ANALYSIS
GG33 GEOLOGICAL DATA ANALYSIS 3 GG33 GEOLOGICAL DATA ANALYSIS LECTURE NOTES PAUL WESSEL SECTION 3 LINEAR (MATRIX ALGEBRA OVERVIEW OF MATRIX ALGEBRA (or All you ever wated to kow about Lear Algebra but
More informationApplication of Legendre Bernstein basis transformations to degree elevation and degree reduction
Computer Aded Geometrc Desg 9 79 78 www.elsever.com/locate/cagd Applcato of Legedre Berste bass trasformatos to degree elevato ad degree reducto Byug-Gook Lee a Yubeom Park b Jaechl Yoo c a Dvso of Iteret
More informationLattices. Mathematical background
Lattces Mathematcal backgroud Lattces : -dmesoal Eucldea space. That s, { T x } x x = (,, ) :,. T T If x= ( x,, x), y = ( y,, y), the xy, = xy (er product of xad y) x = /2 xx, (Eucldea legth or orm of
More informationDiscrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b
CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More informationMean is only appropriate for interval or ratio scales, not ordinal or nominal.
Mea Same as ordary average Sum all the data values ad dvde by the sample sze. x = ( x + x +... + x Usg summato otato, we wrte ths as x = x = x = = ) x Mea s oly approprate for terval or rato scales, ot
More informationInvestigating Cellular Automata
Researcher: Taylor Dupuy Advsor: Aaro Wootto Semester: Fall 4 Ivestgatg Cellular Automata A Overvew of Cellular Automata: Cellular Automata are smple computer programs that geerate rows of black ad whte
More information1 Mixed Quantum State. 2 Density Matrix. CS Density Matrices, von Neumann Entropy 3/7/07 Spring 2007 Lecture 13. ψ = α x x. ρ = p i ψ i ψ i.
CS 94- Desty Matrces, vo Neuma Etropy 3/7/07 Sprg 007 Lecture 3 I ths lecture, we wll dscuss the bascs of quatum formato theory I partcular, we wll dscuss mxed quatum states, desty matrces, vo Neuma etropy
More informationDerivation of 3-Point Block Method Formula for Solving First Order Stiff Ordinary Differential Equations
Dervato of -Pot Block Method Formula for Solvg Frst Order Stff Ordary Dfferetal Equatos Kharul Hamd Kharul Auar, Kharl Iskadar Othma, Zara Bb Ibrahm Abstract Dervato of pot block method formula wth costat
More informationOn the construction of symmetric nonnegative matrix with prescribed Ritz values
Joural of Lear ad Topologcal Algebra Vol. 3, No., 14, 61-66 O the costructo of symmetrc oegatve matrx wth prescrbed Rtz values A. M. Nazar a, E. Afshar b a Departmet of Mathematcs, Arak Uversty, P.O. Box
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationLinear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
More informationPTAS for Bin-Packing
CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More information( q Modal Analysis. Eigenvectors = Mode Shapes? Eigenproblem (cont) = x x 2 u 2. u 1. x 1 (4.55) vector and M and K are matrices.
4.3 - Modal Aalyss Physcal coordates are ot always the easest to work Egevectors provde a coveet trasformato to modal coordates Modal coordates are lear combato of physcal coordates Say we have physcal
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More informationNumerical Analysis Formulae Booklet
Numercal Aalyss Formulae Booklet. Iteratve Scemes for Systems of Lear Algebrac Equatos:.... Taylor Seres... 3. Fte Dfferece Approxmatos... 3 4. Egevalues ad Egevectors of Matrces.... 3 5. Vector ad Matrx
More informationCentroids Method of Composite Areas
Cetrods Method of Composte reas small boy swallowed some cos ad was take to a hosptal. Whe hs gradmother telephoed to ask how he was a urse sad 'No chage yet'. Cetrods Prevously, we developed a geeral
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More informationMOLECULAR VIBRATIONS
MOLECULAR VIBRATIONS Here we wsh to vestgate molecular vbratos ad draw a smlarty betwee the theory of molecular vbratos ad Hückel theory. 1. Smple Harmoc Oscllator Recall that the eergy of a oe-dmesoal
More informationLecture Notes 2. The ability to manipulate matrices is critical in economics.
Lecture Notes. Revew of Matrces he ablt to mapulate matrces s crtcal ecoomcs.. Matr a rectagular arra of umbers, parameters, or varables placed rows ad colums. Matrces are assocated wth lear equatos. lemets
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationCS 523: Computer Graphics, Spring Shape Modeling. PCA Applications + SVD. Andrew Nealen, Rutgers, /15/2011 1
CS 523: Computer Graphcs, Sprng 20 Shape Modelng PCA Applcatons + SVD Andrew Nealen, utgers, 20 2/5/20 emnder: PCA Fnd prncpal components of data ponts Orthogonal drectons that are domnant n the data (have
More informationC-1: Aerodynamics of Airfoils 1 C-2: Aerodynamics of Airfoils 2 C-3: Panel Methods C-4: Thin Airfoil Theory
ROAD MAP... AE301 Aerodyamcs I UNIT C: 2-D Arfols C-1: Aerodyamcs of Arfols 1 C-2: Aerodyamcs of Arfols 2 C-3: Pael Methods C-4: Th Arfol Theory AE301 Aerodyamcs I Ut C-3: Lst of Subects Problem Solutos?
More informationA conic cutting surface method for linear-quadraticsemidefinite
A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008 Outle: Secod-order coe: defto
More informationTHE COMPLETE ENUMERATION OF FINITE GROUPS OF THE FORM R 2 i ={R i R j ) k -i=i
ENUMERATON OF FNTE GROUPS OF THE FORM R ( 2 = (RfR^'u =1. 21 THE COMPLETE ENUMERATON OF FNTE GROUPS OF THE FORM R 2 ={R R j ) k -= H. S. M. COXETER*. ths paper, we vestgate the abstract group defed by
More information( ) ( ) ( ( )) ( ) ( ) ( ) ( ) ( ) = ( ) ( ) + ( ) ( ) = ( ( )) ( ) + ( ( )) ( ) Review. Second Derivatives for f : y R. Let A be an m n matrix.
Revew + v, + y = v, + v, + y, + y, Cato! v, + y, + v, + y geeral Let A be a atr Let f,g : Ω R ( ) ( ) R y R Ω R h( ) f ( ) g ( ) ( ) ( ) ( ( )) ( ) dh = f dg + g df A, y y A Ay = = r= c= =, : Ω R he Proof
More informationDr. Shalabh. Indian Institute of Technology Kanpur
Aalyss of Varace ad Desg of Expermets-I MODULE -I LECTURE - SOME RESULTS ON LINEAR ALGEBRA, MATRIX THEORY AND DISTRIBUTIONS Dr. Shalabh Departmet t of Mathematcs t ad Statstcs t t Ida Isttute of Techology
More informationMEASURES OF DISPERSION
MEASURES OF DISPERSION Measure of Cetral Tedecy: Measures of Cetral Tedecy ad Dsperso ) Mathematcal Average: a) Arthmetc mea (A.M.) b) Geometrc mea (G.M.) c) Harmoc mea (H.M.) ) Averages of Posto: a) Meda
More information13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations
Lecture 7 3. Parametrc ad No-Parametrc Ucertates, Radal Bass Fuctos ad Neural Network Approxmatos he parameter estmato algorthms descrbed prevous sectos were based o the assumpto that the system ucertates
More informationCentroids & Moments of Inertia of Beam Sections
RCH 614 Note Set 8 S017ab Cetrods & Momets of erta of Beam Sectos Notato: b C d d d Fz h c Jo L O Q Q = ame for area = ame for a (base) wdth = desgato for chael secto = ame for cetrod = calculus smbol
More information8.1 Hashing Algorithms
CS787: Advaced Algorthms Scrbe: Mayak Maheshwar, Chrs Hrchs Lecturer: Shuch Chawla Topc: Hashg ad NP-Completeess Date: September 21 2007 Prevously we looked at applcatos of radomzed algorthms, ad bega
More informationOut of sample extensions of PCA, kernel PCA, and MDS
Out of sample extesos of PCA, erel PCA, ad MDS Math 85 Project, Fall 05 Wlso A. Florero-Salas Da L ABLE OF CONENS. Itroducto.... Prcpal Compoet Aalyss (PCA).... he out of sample exteso of PCA... 3.3 he
More informationMatrix Algebra Tutorial With Examples in Matlab
Matr Algebra Tutoral Wth Eamples Matlab by Klaus Moelter Departmet of Agrcultural ad Appled Ecoomcs Vrga Tech emal: moelter@vt.edu web: http://faculty.ageco.vt.edu/moelter/ Specfcally desged as a / day
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationSolving Constrained Flow-Shop Scheduling. Problems with Three Machines
It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632
More informationA tighter lower bound on the circuit size of the hardest Boolean functions
Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the
More informationDimensionality Reduction
Dmesoalty Reducto Sav Kumar, Google Research, NY EECS-6898, Columba Uversty - Fall, 010 Sav Kumar 11/16/010 EECS6898 Large Scale Mache Learg 1 Curse of Dmesoalty May learg techques scale poorly wth data
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationOn Singular Value Decomposition of Rectangular Matrices
-Najah Natoal Uversty Faculty of Graduate Studes O Sgular Value Decomposto of Rectagular Matrces By Shree Najeh Issa Odeh Supervsor Dr"Mohammad Othma" Omra Ths thess s submtted Partal Fulfllmet of the
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More information( ) 2 2. Multi-Layer Refraction Problem Rafael Espericueta, Bakersfield College, November, 2006
Mult-Layer Refracto Problem Rafael Espercueta, Bakersfeld College, November, 006 Lght travels at dfferet speeds through dfferet meda, but refracts at layer boudares order to traverse the least-tme path.
More information1 Lyapunov Stability Theory
Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may
More informationMotion Estimation Based on Unit Quaternion Decomposition of the Rotation Matrix
Moto Estmato Based o Ut Qatero Decomposto of the Rotato Matrx Hag Y Ya Baozog (Isttte of Iformato Scece orther Jaotog Uversty Bejg 00044 PR Cha Abstract Based o the t qatero decomposto of rotato matrx
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationTaylor s Series and Interpolation. Interpolation & Curve-fitting. CIS Interpolation. Basic Scenario. Taylor Series interpolates at a specific
CIS 54 - Iterpolato Roger Crawfs Basc Scearo We are able to prod some fucto, but do ot kow what t really s. Ths gves us a lst of data pots: [x,f ] f(x) f f + x x + August 2, 25 OSU/CIS 54 3 Taylor s Seres
More informationAssignment 7/MATH 247/Winter, 2010 Due: Friday, March 19. Powers of a square matrix
Assgmet 7/MATH 47/Wter, 00 Due: Frday, March 9 Powers o a square matrx Gve a square matrx A, ts powers A or large, or eve arbtrary, teger expoets ca be calculated by dagoalzg A -- that s possble (!) Namely,
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationNewton s Power Flow algorithm
Power Egeerg - Egll Beedt Hresso ewto s Power Flow algorthm Power Egeerg - Egll Beedt Hresso The ewto s Method of Power Flow 2 Calculatos. For the referece bus #, we set : V = p.u. ad δ = 0 For all other
More informationSome Different Perspectives on Linear Least Squares
Soe Dfferet Perspectves o Lear Least Squares A stadard proble statstcs s to easure a respose or depedet varable, y, at fed values of oe or ore depedet varables. Soetes there ests a deterstc odel y f (,,
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationAn Introduction to Relevant Graph Theory and Matrix Theory
Chapter 0 A Itroducto to Relevat Graph heory ad Matrx heory hs boo s cocered wth the calculato of the umber of spag trees of a multgraph usg algebrac ad aalytc techques. We also clude several results o
More informationLebesgue Measure of Generalized Cantor Set
Aals of Pure ad Appled Mathematcs Vol., No.,, -8 ISSN: -8X P), -888ole) Publshed o 8 May www.researchmathsc.org Aals of Lebesgue Measure of Geeralzed ator Set Md. Jahurul Islam ad Md. Shahdul Islam Departmet
More informationTransforms that are commonly used are separable
Trasforms s Trasforms that are commoly used are separable Eamples: Two-dmesoal DFT DCT DST adamard We ca the use -D trasforms computg the D separable trasforms: Take -D trasform of the rows > rows ( )
More informationn -dimensional vectors follow naturally from the one
B. Vectors ad sets B. Vectors Ecoomsts study ecoomc pheomea by buldg hghly stylzed models. Uderstadg ad makg use of almost all such models requres a hgh comfort level wth some key mathematcal sklls. I
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationParameter, Statistic and Random Samples
Parameter, Statstc ad Radom Samples A parameter s a umber that descrbes the populato. It s a fxed umber, but practce we do ot kow ts value. A statstc s a fucto of the sample data,.e., t s a quatty whose
More informationMaps on Triangular Matrix Algebras
Maps o ragular Matrx lgebras HMED RMZI SOUROUR Departmet of Mathematcs ad Statstcs Uversty of Vctora Vctora, BC V8W 3P4 CND sourour@mathuvcca bstract We surveys results about somorphsms, Jorda somorphsms,
More informationANALYSIS ON THE NATURE OF THE BASIC EQUATIONS IN SYNERGETIC INTER-REPRESENTATION NETWORK
Far East Joural of Appled Mathematcs Volume, Number, 2008, Pages Ths paper s avalable ole at http://www.pphm.com 2008 Pushpa Publshg House ANALYSIS ON THE NATURE OF THE ASI EQUATIONS IN SYNERGETI INTER-REPRESENTATION
More information1 Onto functions and bijections Applications to Counting
1 Oto fuctos ad bectos Applcatos to Coutg Now we move o to a ew topc. Defto 1.1 (Surecto. A fucto f : A B s sad to be surectve or oto f for each b B there s some a A so that f(a B. What are examples of
More informationModel Fitting, RANSAC. Jana Kosecka
Model Fttg, RANSAC Jaa Kosecka Fttg: Issues Prevous strateges Le detecto Hough trasform Smple parametrc model, two parameters m, b m + b Votg strateg Hard to geeralze to hgher dmesos a o + a + a 2 2 +
More informationUNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS
Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method
More informationα1 α2 Simplex and Rectangle Elements Multi-index Notation of polynomials of degree Definition: The set P k will be the set of all functions:
Smplex ad Rectagle Elemets Mult-dex Notato = (,..., ), o-egatve tegers = = β = ( β,..., β ) the + β = ( + β,..., + β ) + x = x x x x = x x β β + D = D = D D x x x β β Defto: The set P of polyomals of degree
More informationSummary of the lecture in Biostatistics
Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the
More informationAN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET
AN UPPER BOUND FOR THE PERMANENT VERSUS DETERMINANT PROBLEM BRUNO GRENET Abstract. The Permaet versus Determat problem s the followg: Gve a matrx X of determates over a feld of characterstc dfferet from
More information2.28 The Wall Street Journal is probably referring to the average number of cubes used per glass measured for some population that they have chosen.
.5 x 54.5 a. x 7. 786 7 b. The raked observatos are: 7.4, 7.5, 7.7, 7.8, 7.9, 8.0, 8.. Sce the sample sze 7 s odd, the meda s the (+)/ 4 th raked observato, or meda 7.8 c. The cosumer would more lkely
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationDecomposition of Hadamard Matrices
Chapter 7 Decomposto of Hadamard Matrces We hae see Chapter that Hadamard s orgal costructo of Hadamard matrces states that the Kroecer product of Hadamard matrces of orders m ad s a Hadamard matrx of
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationLecture 07: Poles and Zeros
Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto
More informationPinaki Mitra Dept. of CSE IIT Guwahati
Pak Mtra Dept. of CSE IIT Guwahat Hero s Problem HIGHWAY FACILITY LOCATION Faclty Hgh Way Farm A Farm B Illustrato of the Proof of Hero s Theorem p q s r r l d(p,r) + d(q,r) = d(p,q) p d(p,r ) + d(q,r
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationENGI 3423 Simple Linear Regression Page 12-01
ENGI 343 mple Lear Regresso Page - mple Lear Regresso ometmes a expermet s set up where the expermeter has cotrol over the values of oe or more varables X ad measures the resultg values of aother varable
More informationSolution of General Dual Fuzzy Linear Systems. Using ABS Algorithm
Appled Mathematcal Sceces, Vol 6, 0, o 4, 63-7 Soluto of Geeral Dual Fuzzy Lear Systems Usg ABS Algorthm M A Farborz Aragh * ad M M ossezadeh Departmet of Mathematcs, Islamc Azad Uversty Cetral ehra Brach,
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationMA/CSSE 473 Day 27. Dynamic programming
MA/CSSE 473 Day 7 Dyamc Programmg Bomal Coeffcets Warshall's algorthm (Optmal BSTs) Studet questos? Dyamc programmg Used for problems wth recursve solutos ad overlappg subproblems Typcally, we save (memoze)
More information