CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 2

Transcription

1 CS434/54: Pttern Recognition Prof. Olg Veksler Lecture

2 Outline Review of Liner Algebr vectors nd mtrices products nd norms vector spces nd liner trnsformtions eigenvlues nd eigenvectors Introduction to Mtlb

3 Wh Liner Algebr? For ech dt point, we will represent set of fetures s feture vector [length, weight, color, ] Collected dt will be represented s collection of (feture) vectors [l, w, c, ] [l, w, c, ] [l, w, c, ] Liner models re simple nd computtionll fesible

4 Vectors X [, ] z n-dimensionl row vector [ ] rnspose of row vector is column vector Vector product (or inner or dot product), X -z n n n i k i i n

5 More on Vectors Euclidin norm or length i i n If we s is normlized or unit length Angle θ between vectors nd cos θ cos θ 0 0 orthogonl to cos θ > 0, hus inner product cptures direction reltionship between nd cosθ < 0

6 More on Vectors Vectors nd re orthonorml if the re orthogonl nd Euclidin distnce between vectors nd ( ) i i i n -

7 Liner Dependence nd Independence Vectors,,, n re linerl dependent if there eist constnts α, α,, α n s.t.. α + α + + α n n 0 0. t lest one α i Vectors,,, n re linerl independent if α + α + + α 0 α α 0 n n n

8 Vector Spces nd Bsis he set of ll n-dimensionl vectors is clled vector spce V { } A set of vectors u, u,, u n re clled bsis for vector spce if n v in V cn be written s v α u + α u + + αnu n u, u,, u n re independent implies the form bsis, nd vice vers u, u,, u n. u i i. u i u j i give n orthonorml bsis if j

9 Mtrices n b m mtri A nd its m b n trnspose A A n n m m nm A m m n n nm

10 Mtri Product AB n n 3 n3 d nd b b b b b3 b b d b m m 3m dm c ij i b j c ij i, b C is row i of A is column j of B j # of columns of A # of rows of B even if defined, in generl AB BA

11 Mtrices Rnk of mtri is the number of linerl independent rows (or equivlentl columns) A squre mtri is non-singulr if its rnk equl to the number of rows. If its rnk is less thn number of rows it is singulr. 0 0 Identit mtri I AIIAA Mtri A is smmetric if AA

12 Mtrices Mtri A is positive definite if A A 0 i, j i, j i j > Mtri A is positive semi-definite if A A 0 i, j i, j i j rce of squre mtri A is sum on the elements on the digonl tr n [ A] i ii

13 Mtrices Inverse of squre mtri A is mtri A - s.t. AA I - If A is singulr or not squre, inverse does not eist. Pseudo-inverse A is defined whenever A A is not singulr (it is squre) A (A A ) A - - AA (A A) AAI

14 Mtrices Determinnt of n b n mtri A is det n ( ) ( ) k+ i A ( ) ik det Aik A ik k Where obtined from A b removing the ith row nd kth column Absolute vlue of determinnt gives the volume of prllelepiped spnned b the mtri rows { β + β β n } n β i [ 0,] i

15 Liner rnsformtions A liner trnsformtion from vector spce V to vector spce U is mpping which cn be represented b mtri M: u Mv If U nd V hve the sme dimension, M is squre mtri V U v M Mv In pttern recognition, often U hs smller dimensionlit thn V, i.e. trnsformtion M is used to reduce the number of fetures. M v u

16 Eigenvectors nd Eigenvlues Given n b n mtri A, nd nonzero vector. Suppose there is λ which stisfies A λ is clled n eigenvector of A λ is clled n eigenvlue of A Note: A0λ0 for n λ, not interesting Liner trnsformtion A mps n eigenvector v in simple w. Mgnitude chnges b λ, direction If λ > 0 If λ < 0 Av Av v v

17 Eigenvectors nd Eigenvlues If A is rel nd smmetric, then ll eigenvlues re rel (not comple) If A is non singulr, ll eigenvlues re non zero If A is positive definite, ll eigenvlues re positive

18 MALAB

19 Strting mtlb term -fn X4 mtlb Bsic Nvigtion quit more help generl Sclrs, vribles, bsic rithmetic Cler + - * / ^ help rith Reltionl opertors,&,,~,or help relop Lists, vectors, mtrices A[ 3;4 5] A Mtri nd vector opertions find(a>3), colon opertor * / ^.*./.^ ee(n),norm(a),det(a),eig(a) m,min,std help mtfun Elementr functions help elfun Dt tpes double Chr Progrmming in Mtlb.m files scripts function squre() help lng Flow control if i else end, if else if end for i:0.5: end while i end Return help lng Grphics help grphics help grph3d File I/O lod,sve fopen, fclose, fprintf, fscnf