The Eigenfaces method
Plan of the lecture Principal Components Analysis main idea Feature extraction by PCA face recognition Eigenfaces training feature extraction
Literature M.A.Turk, A.P.Pentland Face Recognition Using Eigenfaces
Phases of recognition Detection Normalisation Feature vectors comparison Feature extraction
Example...
Example...
Example...
PCA main issues Dimensionality input space data Coordinates system suited to a set Dimensionality reduction reduction error Principal Components Analysis (PCA) statistical method
PCA main issues Input data set vectors / points input space Space orthonormal basis each vector from the input space can be expressed as a linear combination of basis vectors basis vector defines a dimension
PCA main issues Dimensions are sorted each basis vector assigned with a weight Main directions of variance in the input set each direction generates one dimension Dimension importance proportional to variance
Input face space Normalised image described by pixel values Image as a point in a space e.g. 64x75 pixels 4800 dimensions Information excess too many dimensions Elimination of redundant information relevant information must remain feature extraction
PCA for feature extraction Set of normalised images input vectors (a.k.a. training set) PCA: finds new orthogonal basis number of dimensions can be reduced creates face space Feature extraction by PCA find a corresponding point in a new space works for any face image (not only from training set) Eigenfaces method PCA for face images
The Eigenfaces method Training 1. create covariance matrix for the training set 2. calculate eigenvalues and eigenvectors eigenvectors are the orthonormal basis and define the face space eigenvalues are associated with eigenvectors eigenvalues proportional to variance 3. select eigenvectors Feature extraction face image mapped to the new space (coordinates must be found) any face image may be processed
C Eigenfaces: training Input vectors: u,...,uu 1 M u N-dimensional vector M number of vectors in the set Average vector: μ Covariance matrix: = = 1 M u M i= 1 1 M ( u μ)( u μ) Τ or i i M C = 1 i= 1 i M A A T
Training example Training set (M=4, N=3): [1, 0, 2] [0, 3, 1] [4, 1, 2] [3, 0, -1] Average vector, covariance matrix
Eigenfaces: training Characteristic equation: eigenvalues Eigenvectors (v) det( C λ I ) = (λ) one for each eigenvalue C v Jacobi method (numerical method) = λ OpenCV - cveigenvv function v 0
Training optimisation Problem: large size of covariance matrix (NxN), e.g. 4800x4800 T Trick: A A T A A A T A v = λ v v' = λ ' v ' ( A v') = λ' ( A v') λ = λ' v = A Av desired eigenvectors v'
Eigenfaces: training Eigenvectors properties dimensionality equal to input vectors orthonormal (length = 1) sorted by corresponding eigenvalues may be scaled to pixel value range Eigenfaces eigenvectors transformed to images example dimensionality reduction New space, less dimensions
Eigenfaces: training C 00... C 0n......... C n0... C nn Normalised images Covariance matrix Eigenfaces
Eigenfaces: feature extraction Feature extraction ti input: set of eigenvectors and eigenvalues (delivered by training) normalised image Projection: x'= ψ T x ti ith i t ψ -matrix with eigenvectors x normalised image after average face subtraction x transformed vector
Feature extraction - example 2-dimensional space: eigenvectors: 2 2 2 2 [ ; ] [ ; ] 2 2 2 2 Vectors projection: [3; 1], [-2; -2], [10, 9] Dimensionality i reduction
Eigenfaces: feature extraction ψ matrix can be cut to reduce dimensions ψ ψ ψ Feature vector element is a scalar product: wi = v T x i w = ψ' T x Feature vector cut projected vector x
Eigenfaces: feature extraction K1 K2... Scalar products between normalised image and eigenvectors K3... Feature vector
Back projection: face image Feature vector face description information reduction Back projection: face image recovered from feature vector reduced information are lost Projection error: depends on similarity to the training set 2D example face images
Back projection: detection Back projection of images: face -> slightly modified face image flower -> image similar to a face Back projection error is higher for non-face images Can be used as a verifier threshold of accepted projection error
Summary Eigenfaces a basic face recognition method many derived methods Training and feature extraction Holistic approach High speed Average / low effectiveness may be improved
Thank you for your attention!