CS434/54: Pttern Recognition Prof. Olg Veksler Lecture
Outline Review of Liner Algebr vectors nd mtrices products nd norms vector spces nd liner trnsformtions eigenvlues nd eigenvectors Introduction to Mtlb
Wh Liner Algebr? For ech dt point, we will represent set of fetures s feture vector [length, weight, color, ] Collected dt will be represented s collection of (feture) vectors [l, w, c, ] [l, w, c, ] [l, w, c, ] 3 3 3 Liner models re simple nd computtionll fesible
Vectors X [, ] z n-dimensionl row vector [ ] rnspose of row vector is column vector Vector product (or inner or dot product), X -z + + + n n n i k i i n
More on Vectors Euclidin norm or length i i n If we s is normlized or unit length Angle θ between vectors nd cos θ cos θ 0 0 orthogonl to cos θ > 0, hus inner product cptures direction reltionship between nd cosθ < 0
More on Vectors Vectors nd re orthonorml if the re orthogonl nd Euclidin distnce between vectors nd ( ) i i i n -
Liner Dependence nd Independence Vectors,,, n re linerl dependent if there eist constnts α, α,, α n s.t.. α + α + + α n n 0 0. t lest one α i Vectors,,, n re linerl independent if α + α + + α 0 α α 0 n n n
Vector Spces nd Bsis he set of ll n-dimensionl vectors is clled vector spce V { } A set of vectors u, u,, u n re clled bsis for vector spce if n v in V cn be written s v α u + α u + + αnu n u, u,, u n re independent implies the form bsis, nd vice vers u, u,, u n. u i i. u i u j i give n orthonorml bsis if j
Mtrices n b m mtri A nd its m b n trnspose A A n n m m nm A m m n n nm
Mtri Product AB n n 3 n3 d nd b b b b b3 b b d b m m 3m dm c ij i b j c ij i, b C is row i of A is column j of B j # of columns of A # of rows of B even if defined, in generl AB BA
Mtrices Rnk of mtri is the number of linerl independent rows (or equivlentl columns) A squre mtri is non-singulr if its rnk equl to the number of rows. If its rnk is less thn number of rows it is singulr. 0 0 Identit mtri I 0 0 0 0 0 0 AIIAA Mtri A is smmetric if AA 9 5 7 4 8 9 4 3 6 5 8 6 4
Mtrices Mtri A is positive definite if A A 0 i, j i, j i j > Mtri A is positive semi-definite if A A 0 i, j i, j i j rce of squre mtri A is sum on the elements on the digonl tr n [ A] i ii
Mtrices Inverse of squre mtri A is mtri A - s.t. AA I - If A is singulr or not squre, inverse does not eist. Pseudo-inverse A is defined whenever A A is not singulr (it is squre) A (A A ) A - - AA (A A) AAI
Mtrices Determinnt of n b n mtri A is det n ( ) ( ) k+ i A ( ) ik det Aik A ik k Where obtined from A b removing the ith row nd kth column Absolute vlue of determinnt gives the volume of prllelepiped spnned b the mtri rows { β + β +... + β n } n β i [ 0,] i
Liner rnsformtions A liner trnsformtion from vector spce V to vector spce U is mpping which cn be represented b mtri M: u Mv If U nd V hve the sme dimension, M is squre mtri V U v M Mv In pttern recognition, often U hs smller dimensionlit thn V, i.e. trnsformtion M is used to reduce the number of fetures. M v u
Eigenvectors nd Eigenvlues Given n b n mtri A, nd nonzero vector. Suppose there is λ which stisfies A λ is clled n eigenvector of A λ is clled n eigenvlue of A Note: A0λ0 for n λ, not interesting Liner trnsformtion A mps n eigenvector v in simple w. Mgnitude chnges b λ, direction If λ > 0 If λ < 0 Av Av v v
Eigenvectors nd Eigenvlues If A is rel nd smmetric, then ll eigenvlues re rel (not comple) If A is non singulr, ll eigenvlues re non zero If A is positive definite, ll eigenvlues re positive
MALAB
Strting mtlb term -fn X4 mtlb Bsic Nvigtion quit more help generl Sclrs, vribles, bsic rithmetic Cler + - * / ^ help rith Reltionl opertors,&,,~,or help relop Lists, vectors, mtrices A[ 3;4 5] A Mtri nd vector opertions find(a>3), colon opertor * / ^.*./.^ ee(n),norm(a),det(a),eig(a) m,min,std help mtfun Elementr functions help elfun Dt tpes double Chr Progrmming in Mtlb.m files scripts function squre() help lng Flow control if i else end, if else if end for i:0.5: end while i end Return help lng Grphics help grphics help grph3d File I/O lod,sve fopen, fclose, fprintf, fscnf