Recognition Using Class Specific Linear Projection. Magali Segal Stolrasky Nadav Ben Jakov April, 2015

Transcription

1 Recognition Using Class Specific Linear Projection Magali Segal Stolrasky Nadav Ben Jakov April, 2015

2 Articles Eigenfaces vs. Fisherfaces Recognition Using Class Specific Linear Projection, Peter N. Belhumeur, Joao P. Hespanha, David J. Kriegman, 1996 Eigenfaces for Recognition, Turk and Pentland, 1991

3 Intro Definition Face recognition usages Eigenfaces method Fisherfaces method Experimental Results

4 Face recognition - definition?

5 Face recognition what is it good for? Using our face as our password When searching for criminals Adapting advertisements personally Family album organization (Picasa) Demo

6 Recognition steps

7 Eigenfaces

8 Eigenfaces?

9 Eigenfaces -Dataset x 11 x 21 x 31 x 41 = x 12 x 1n = x 22 x 2n = x 32 x 3n = x 42 x 4n x 51 x 61 x 71 x 81 = x 52 x 5n = x 62 x 6n = x 72 x 7n = x 82 x 8n 9

10 Eigenfaces - PCA We need to align the data to the center 10

11 Eigenfaces - PCA We need to align the data to the center 11

12 Eigenfaces - PCA First, we calculate the mean face from the dataset, then we align each image from the dataset, by subtracting the mean image μ = 1 N N i=1 x i Φ j = x j μ 12

13 Eigenfaces - PCA Then, using PCA, we can find orthonormal vectors that best account the distribution of face images within the entire space w k MAX 1 N N i=1 w k T Φ i 2,k=1..m 13

14 Eigenfaces - PCA 14

15 Eigenfaces - PCA N N C = 1 N i=1 Φ i Φ i T = WW T W = n Note that C is n x n, so finding its eigenvectors is REALLY ineffective We need a better approach 15

16 Eigenfaces - PCA Assuming that N << n We will only need N 1 vectors to represent the dataset 16

17 Eigenfaces - PCA Using the opposite multiplication W T Wv i = μ i v i CWv i = WW T Wv i = μ i Wv i We can see that Wv i are the eigenvectors of the Covariance matrix Where A T A is an N x N matrix, which is much more manageable 17

18 Eigenfaces - PCA So the new calculation of the eigenvectors is as follows u l = M i=1 v li Φ i, l = 1.. M (M<N) 18

19 Eigenfaces - checkpoint = α 1 +α 2 +α 3 +α 4 19

20 Eigenfaces - Usages Given an image, we want to output whether it s a face (detection), and who is he/she (recognition) = α 1 +α 2 +α 3 +α 4? 20

21 Eigenfaces - Usages 21

22 Eigenfaces - Usages 22

23 Eigenfaces - Usages 23

24 Eigenfaces - Usages 24

25 Eigenfaces - Usages 25

26 Eigenfaces - Usages 26

27 Eigenfaces - Usages 27

28 Eigenfaces - Usages Given a large image, we want to locate all of the faces it contains Assumption we already have a central face image of the same size (multiple sizes will be required otherwise) We also assume the faces within the given image experience the same condition as the images in the dataset 28

29 Eigenfaces - Usages 29

30 Eigenfaces - Usages Body detection using spatiotemporal filters, and thresholding it over the pictures-array, assuming o Static background o Straight body and head o Smooth movement Head detection using face detection and recognition on the upper part 30

31 Eigenfaces - Usages 31

32 Eigenfaces - Usages We can store all of the unknown faces, and then cluster them into new face classes, and update our dataset and data structure 32

33 Eigenfaces - limitations Background is being learned in the process. Optional workaround apply radial Gaussian mask, centered on the face affected by illumination Optional workaround remove the 3 highest eigenfaces from the set 33

34 Eigenfaces - limitations Highly affected by size changes Optional workaround keep the data in multiple sizes Affected by head orientation Optional workaround keep the dataset faces in several notations 34

35 Fisherfaces

36 Fisherfaces (FLD) FLD maximizes the between-class scatter FLD minimizes the within-class scatter Class A Class B μ 1 μ μ 2 36

37 Fisherfaces

38 FLD vs PCA

39 Fisherfaces Assumptions o N is the number of images in the database o c is the number of persons in the database

40 Fisherfaces the database x 11 x 21 x 31 x 41 = x 12 x 1n = x 22 x 2n = x 32 x 3n = x 42 x 4n x 51 x 61 x 71 x 81 = x 52 x 5n = x 62 x 6n = x 72 x 7n = x 82 x 8n 40

41 Fisherfaces - the algorithm Computing the average of all faces μ = 1 N x 11 x 12 x 1n x 21 x 22 x 2n x 81 x 82 x 8n, where N = 8 41

42 Fisherfaces - the algorithm Computing the average face of each person μ 1 = 1 2 x 11 x 12 x 1n + + x 21 x 22 + x 2n μ 2 = 1 2 x 31 x 32 x 3n + + x 41 x 42 + x 4n μ 3 = 1 2 x 51 x 52 x 5n + + x 61 x 62 + x 6n μ 4 = 1 2 x 71 x 72 x 7n + + x 81 x 82 + x 8n 42

43 Fisherfaces, the algorithm Subtracting average of each person from the training faces x 11 μ 1 x 21 μ 1 x 31 μ 2 x 41 μ 2 x 1 = x 12 μ 1 x 2 = x 22 μ 1 x 3 = x 32 μ 2 x 4 = x 42 μ 2 x 1n μ 1 x 2n μ 1 x 3n μ 2 x 4n μ 2 x 51 μ 3 x 61 μ 3 x 71 μ 4 x 81 μ 4 x 5 = x 52 μ 3 x 6 = x 62 μ 3 x 7 = x 72 μ 4 x 8 = x 82 μ 4 x 5n μ 3 x 6n μ 3 x 7n μ 4 x 8n μ 4

44 Illustration x2 S W S 1 S 2 Within class scatter S 1 S 2 S B x1 Between class scatter

45 Fisherfaces - the algorithm Scatter matrices S 1 = (x 1 x 1 T +x 2 x 2 T ), S 2 = (x 3 x 3 T +x 4 x 4 T ) S 3 = (x 5 x 5 T +x 6 x 6 T ), S 4 = (x 7 x 7 T +x 8 x 8 T ) Within-class scatter matrix S w = S 1 + S 2 + S 3 + S 4

46 Fisherfaces - the algorithm The between-class scatter matrix S B = 2( μ 1 - μ ) ( μ 1 - μ ) T + 2( μ 2 - μ ) ( μ 2 - μ ) T + 2( μ 3 - μ ) ( μ 3 - μ ) T + 2( μ 4 - μ ) ( μ 4 - μ ) T We are seeking W opt = argmax w W T S B W W T S w W = w 1, w 2,, w m

47 Fisherfaces - the algorithm If S w is nonsingular The columns of w w 1, w 2,, w m are the eigenvectors of S w 1 S B Compute S w 1 Multiply the matrices S w 1 S B Compute the eigenvectors w 1, w 2,, w m

48 Fisherfaces, the algorithm If S w is singular Apply PCA first to reduce dimensionality of faces W pca = argmax w W T S T W ; S T = N k=1 (x k μ)(x k μ) T Apply FLD on the output W fld = argmax w W T W T pca S B W pca w W T W T pca S w W pca w W opt = W fld W pca

49 Recognition with Fisherfaces o Project to FLD subspace - W opt multiplication Class A o Classify by nearest neighbor Class B

50 Correlation Method description o Normalize learning images to have zero mean and unit variance o Assign to the image the label with the closest point in the learning set. o The normalization allows the independency of light source intensity. Limitations o On light variations images, the points will not be tightly clustered o Expensive calculation time o Large storage

51 Linear subspaces o Method description o For each face, use 3 or more images taken under different lighting directions to construct a 3-D basis for the linear subspace. o Recognition compute the distance of a new image to each linear subspace and choose the face corresponding to the shortest distance. o If there is no noise or self-shadowing there will be no errors when classifying under any lighting conditions. o Limitations o Does not work well with varying facial expressions o Expensive calculation time o Large storage

52 Experimental Results

53 Variation in light experiment On each image in this DB a subject held his\her head steady while being illuminated by a dominant light source. 5 subsets consisting of images of 5 people with varying light were used for this experiment. Recognition was performed using nearest neighbor classifier.

54 Variation in light experiment 30 Images (15 ) 105 images (75 ) 45 images (30 ) 65 images (45 ) 85 images (60 )

55 Variation in light experiment extrapolation experiment Each method was trained on samples from subset 1 and tested using samples from subsets 1,2,3

56 Variation in light experiment Interpolation experiment Each method was trained on subsets 1 & 5 and then tested on subsets 2,3,4.

57 Variation in Facial Expression, Eyewear and lighting experiment The Database contains 160 frontal face images covering 16 individuals taken under 10 different conditions A normal image under ambient lighting, one with or without glasses, 3 images taken with different point light sources, and 5 different facial expressions Recognition was performed using nearest neighbor classifier.

58 Variation in Facial Expression, Eyewear and lighting experiment

59 Experiments Conclusions All methods perform well if presented with an image in the test set which is similar to an image in the training set. Removing the initial three principal components improves the performance of the eigenface method in the presence of lighting variation. The Fisherface method is the best in handling variation in lighting and expressions The linear subspace method suffers when confronted with variation in facial expression, but it is a close second after the fisherface method.

60 Thank You!