Recognition Using Class Specific Linear Projection Magali Segal Stolrasky Nadav Ben Jakov April, 2015
Articles Eigenfaces vs. Fisherfaces Recognition Using Class Specific Linear Projection, Peter N. Belhumeur, Joao P. Hespanha, David J. Kriegman, 1996 Eigenfaces for Recognition, Turk and Pentland, 1991
Intro Definition Face recognition usages Eigenfaces method Fisherfaces method Experimental Results
Face recognition - definition?
Face recognition what is it good for? Using our face as our password When searching for criminals Adapting advertisements personally Family album organization (Picasa) Demo
Recognition steps
Eigenfaces
Eigenfaces?
Eigenfaces -Dataset x 11 x 21 x 31 x 41 = x 12 x 1n = x 22 x 2n = x 32 x 3n = x 42 x 4n x 51 x 61 x 71 x 81 = x 52 x 5n = x 62 x 6n = x 72 x 7n = x 82 x 8n 9
Eigenfaces - PCA We need to align the data to the center 10
Eigenfaces - PCA We need to align the data to the center 11
Eigenfaces - PCA First, we calculate the mean face from the dataset, then we align each image from the dataset, by subtracting the mean image μ = 1 N N i=1 x i Φ j = x j μ 12
Eigenfaces - PCA Then, using PCA, we can find orthonormal vectors that best account the distribution of face images within the entire space w k MAX 1 N N i=1 w k T Φ i 2,k=1..m 13
Eigenfaces - PCA 14
Eigenfaces - PCA N N C = 1 N i=1 Φ i Φ i T = WW T W = n Note that C is n x n, so finding its eigenvectors is REALLY ineffective We need a better approach 15
Eigenfaces - PCA Assuming that N << n We will only need N 1 vectors to represent the dataset 16
Eigenfaces - PCA Using the opposite multiplication W T Wv i = μ i v i CWv i = WW T Wv i = μ i Wv i We can see that Wv i are the eigenvectors of the Covariance matrix Where A T A is an N x N matrix, which is much more manageable 17
Eigenfaces - PCA So the new calculation of the eigenvectors is as follows u l = M i=1 v li Φ i, l = 1.. M (M<N) 18
Eigenfaces - checkpoint = α 1 +α 2 +α 3 +α 4 19
Eigenfaces - Usages Given an image, we want to output whether it s a face (detection), and who is he/she (recognition) = α 1 +α 2 +α 3 +α 4? 20
Eigenfaces - Usages 21
Eigenfaces - Usages 22
Eigenfaces - Usages 23
Eigenfaces - Usages 24
Eigenfaces - Usages 25
Eigenfaces - Usages 26
Eigenfaces - Usages 27
Eigenfaces - Usages Given a large image, we want to locate all of the faces it contains Assumption we already have a central face image of the same size (multiple sizes will be required otherwise) We also assume the faces within the given image experience the same condition as the images in the dataset 28
Eigenfaces - Usages 29
Eigenfaces - Usages Body detection using spatiotemporal filters, and thresholding it over the pictures-array, assuming o Static background o Straight body and head o Smooth movement Head detection using face detection and recognition on the upper part 30
Eigenfaces - Usages 31
Eigenfaces - Usages We can store all of the unknown faces, and then cluster them into new face classes, and update our dataset and data structure 32
Eigenfaces - limitations Background is being learned in the process. Optional workaround apply radial Gaussian mask, centered on the face affected by illumination Optional workaround remove the 3 highest eigenfaces from the set 33
Eigenfaces - limitations Highly affected by size changes Optional workaround keep the data in multiple sizes Affected by head orientation Optional workaround keep the dataset faces in several notations 34
Fisherfaces
Fisherfaces (FLD) FLD maximizes the between-class scatter FLD minimizes the within-class scatter Class A Class B μ 1 μ μ 2 36
Fisherfaces
FLD vs PCA
Fisherfaces Assumptions o N is the number of images in the database o c is the number of persons in the database
Fisherfaces the database x 11 x 21 x 31 x 41 = x 12 x 1n = x 22 x 2n = x 32 x 3n = x 42 x 4n x 51 x 61 x 71 x 81 = x 52 x 5n = x 62 x 6n = x 72 x 7n = x 82 x 8n 40
Fisherfaces - the algorithm Computing the average of all faces μ = 1 N x 11 x 12 x 1n + + + x 21 x 22 x 2n +.. + +.. + +.. + x 81 x 82 x 8n, where N = 8 41
Fisherfaces - the algorithm Computing the average face of each person μ 1 = 1 2 x 11 x 12 x 1n + + x 21 x 22 + x 2n μ 2 = 1 2 x 31 x 32 x 3n + + x 41 x 42 + x 4n μ 3 = 1 2 x 51 x 52 x 5n + + x 61 x 62 + x 6n μ 4 = 1 2 x 71 x 72 x 7n + + x 81 x 82 + x 8n 42
Fisherfaces, the algorithm Subtracting average of each person from the training faces x 11 μ 1 x 21 μ 1 x 31 μ 2 x 41 μ 2 x 1 = x 12 μ 1 x 2 = x 22 μ 1 x 3 = x 32 μ 2 x 4 = x 42 μ 2 x 1n μ 1 x 2n μ 1 x 3n μ 2 x 4n μ 2 x 51 μ 3 x 61 μ 3 x 71 μ 4 x 81 μ 4 x 5 = x 52 μ 3 x 6 = x 62 μ 3 x 7 = x 72 μ 4 x 8 = x 82 μ 4 x 5n μ 3 x 6n μ 3 x 7n μ 4 x 8n μ 4
Illustration x2 S W S 1 S 2 Within class scatter S 1 S 2 S B x1 Between class scatter
Fisherfaces - the algorithm Scatter matrices S 1 = (x 1 x 1 T +x 2 x 2 T ), S 2 = (x 3 x 3 T +x 4 x 4 T ) S 3 = (x 5 x 5 T +x 6 x 6 T ), S 4 = (x 7 x 7 T +x 8 x 8 T ) Within-class scatter matrix S w = S 1 + S 2 + S 3 + S 4
Fisherfaces - the algorithm The between-class scatter matrix S B = 2( μ 1 - μ ) ( μ 1 - μ ) T + 2( μ 2 - μ ) ( μ 2 - μ ) T + 2( μ 3 - μ ) ( μ 3 - μ ) T + 2( μ 4 - μ ) ( μ 4 - μ ) T We are seeking W opt = argmax w W T S B W W T S w W = w 1, w 2,, w m
Fisherfaces - the algorithm If S w is nonsingular The columns of w w 1, w 2,, w m are the eigenvectors of S w 1 S B Compute S w 1 Multiply the matrices S w 1 S B Compute the eigenvectors w 1, w 2,, w m
Fisherfaces, the algorithm If S w is singular Apply PCA first to reduce dimensionality of faces W pca = argmax w W T S T W ; S T = N k=1 (x k μ)(x k μ) T Apply FLD on the output W fld = argmax w W T W T pca S B W pca w W T W T pca S w W pca w W opt = W fld W pca
Recognition with Fisherfaces o Project to FLD subspace - W opt multiplication Class A o Classify by nearest neighbor Class B
Correlation Method description o Normalize learning images to have zero mean and unit variance o Assign to the image the label with the closest point in the learning set. o The normalization allows the independency of light source intensity. Limitations o On light variations images, the points will not be tightly clustered o Expensive calculation time o Large storage
Linear subspaces o Method description o For each face, use 3 or more images taken under different lighting directions to construct a 3-D basis for the linear subspace. o Recognition compute the distance of a new image to each linear subspace and choose the face corresponding to the shortest distance. o If there is no noise or self-shadowing there will be no errors when classifying under any lighting conditions. o Limitations o Does not work well with varying facial expressions o Expensive calculation time o Large storage
Experimental Results
Variation in light experiment On each image in this DB a subject held his\her head steady while being illuminated by a dominant light source. 5 subsets consisting of images of 5 people with varying light were used for this experiment. Recognition was performed using nearest neighbor classifier.
Variation in light experiment 30 Images (15 ) 105 images (75 ) 45 images (30 ) 65 images (45 ) 85 images (60 )
Variation in light experiment extrapolation experiment Each method was trained on samples from subset 1 and tested using samples from subsets 1,2,3
Variation in light experiment Interpolation experiment Each method was trained on subsets 1 & 5 and then tested on subsets 2,3,4.
Variation in Facial Expression, Eyewear and lighting experiment The Database contains 160 frontal face images covering 16 individuals taken under 10 different conditions A normal image under ambient lighting, one with or without glasses, 3 images taken with different point light sources, and 5 different facial expressions Recognition was performed using nearest neighbor classifier.
Variation in Facial Expression, Eyewear and lighting experiment
Experiments Conclusions All methods perform well if presented with an image in the test set which is similar to an image in the training set. Removing the initial three principal components improves the performance of the eigenface method in the presence of lighting variation. The Fisherface method is the best in handling variation in lighting and expressions The linear subspace method suffers when confronted with variation in facial expression, but it is a close second after the fisherface method.
Thank You!