The Mathematics of Facial Recognition

Transcription

1 William Dean Gowin Graduate Student Appalachian State University July 26, 2007

2 Outline

3 EigenFaces Deconstruct a known face into an N-dimensional facespace where N is the number of faces in our data set. Deconstruct new faces and check how close they are in the facespace to the known face.

4 Objectives Explain Principle Component Analysis (PCA), and Singular Value Decomposition (SVD) Test any visual differences between PCA and SVD and then check the computational efficiency

5 Outline

6 PCA Utilized in: genomic studies biochemistry neurology biometrics The goal of PCA is to find the most important basis to represent a large data set, which filters out redundant or extraneous information.

7 Reduces dimensionality of the data set and filters out the redundant data. Calculate the eigenvalues and eigenvectors of the covariance matrix AA T. The k largest eigenvalues and their subsequent eigenvectors represent the most important patterns or characteristics in the data. That is you can reproduce the original faces reasonably well with only these k eigenvectors.

8 A simple Example of PCA

9 Subtract means, calculate eigenvalues and eigenvectors of the covariance matrix.

10

11 x y x restore y restore

12

13

14 Let A be an n 2 m matrix, where m is the number of images in the data set, and each image is: n n, and n m. AA T has n 2 eigenvalues (λ i ), and eigenvectors (u i ), but n 2 is much to large to compute and therefore intractable. A T A has m eigenvalues and eigenvectors (v i ), and are the largest m eigenvalues of AA T. The eigenvectors of AA T are related to A T A by: u i = Av i. References: M. Turk and A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neuroscience, Vol. 3, no. 1, pp , 1991

15 Outline

16 SVD Every m n matrix can be factored into a singular value decomposition (SVD) of the form: A = UΣV T (1) where the columns of U and V are the left and right (respectively) singular vectors of A and they are both orthogonal matrices: U T = U 1, V T = V 1, and Σ = diag(σ 1, σ 2,..., σ n ) where the σ i s are the singular values of A which are non-negative. References: Carl Meyer, Matrix Analysis and Applied Linear Algebra, Siam, Philadelphia, Jonathon Shlens, A Tutorial on Principal Component Analysis, shlens/notes.html

17 To see how the singular values relate to the eigenvalues, the SVD of A can be rewritten as: AV = UΣ and A T U = V Σ, which leads to: Av i = σ i u i A T Av i = A T σ i u i = σ i A T u i = σ 2 i v i A T Av i = λ i v i where v i represents the i th column vector of V, u i represents the i th column vector of U and λ i is the i th eigenvalue of A T A. The above proof demonstrates the relationship that the squared singular values of A are the eigenvalues of A T A and the column vectors of V are the eigenvectors of A T A and the column vectors of U are the corresponding eigenvectors of AA T.

18 Outline

19 Figure: Faces in the data set.

20 Figure: PCA facebasis.

21 Figure: SVD facebasis.

22 Figure: Recreating with top 5 PCA facebasis. Figure: Recreating with top 5 SVD facebasis.

23 No visual difference between PCA and SVD. On average the time to compute was twice as fast with PCA compared to SVD. Using only 5 eigenfaces produced similar results to using 20. Was able to identify most faces even ones that were quite different than the original.

24 Figure: Top 5 Closest faces in facespace using PCA. Figure: Top 5 Closest faces in facespace using SVD.

25 Figure: Reconstructing a face that is smiling and has eyes closed. Figure: Recreating with glasses, smiling and further away.

26 Figure: Recreating a face that is smiling. Figure: Recreating a face that is smiling.

27 Figure: Recreating a face that is turned and smiling. Figure: Recreating a face that is smiling.

28 Figure: Recreating face with only 5 eigenfaces. Figure: Recreating face with only 5 eigenfaces.

29

30

31 Outline

32 Improvements A larger data set Cropped closer to the face. Subtract off the background Use Eigenfaces first to check and then use face proportions (eye width...), instead of human input or threshold

33 Difficulties Aligning the faces (trying to get the camera centered & perpendicular to the face) can be quite difficult. Other factors: distance away from camera, head tilt, lighting, image resolution,... Cropping - not including hair, ears, neck. Working in RGB space is more difficult and computational expensive.