Image Analysis & Retrieval Lec 15 - Graph Embedding & Laplacianface

Transcription

1 CS/EE 5590 / ENG 401 Special Topics, Spring 2018 Image Analysis & Retrieval Lec 15 - Graph Embedding & Laplacianface Zhu Li Dept of CSEE, UMKC Office Hour: Tue/Thr 2:30-4pm@FH560E, Contact: lizhu@umkc.edu, Ph: x Created using WPS office and EqualX latex equation editor p.1

2 Outline q Recap: Eigenface Fisherface HW-3: Fisherface and Laplacian face q Graph Embedding Laplacianface Graph Fourier Transform qsummary p.2

3 Subspace Learning for Face Recognition q Project face images to a subspace with basis A Matlab: x=faces*a(:,1:kd) eigf 2 eigf 3 eigf 1 = 10.9* + 0.4* + 4.7* p.3

4 PCA & Fisher s Linear Discriminant Between-class scatter c S B = å c i ( mi - m)( mi - m) i= 1 Within-class scatter T c 1 c 2 m 2 S W = Where c å å i= 1 x Îc k ( x i k - m )( m - m ) c is the number of classes i m i is the mean of class c i c i is number of samples of c i.. k i T m 1 m p.4

5 Eigen vs Fisher Projection PCA c 1 c 2 Fisher PCA (Eigenfaces) W PCA = arg maxw W Maximizes projected total scatter Fisher s Linear Discriminant W fld W = arg max W W W Maximizes ratio of projected between-class to projected within-class scatter, solved by the generalized Eigen problem: T T T S S S T B W W W p.5

6 q mylda.m LDA implementation % compute class mean mx = mean(x); ids = unique(y); m = length(ids); Sb = zeros(kd, kd); for k=1:m indx = find(y==ids(k)); nk(k) = length(indx); % class mean m_cx(k,:) = mean(x(indx, :)); % between class scatter Sb = Sb + nk(k)*(m_cx(k,:) - mx)'*(m_cx(k,:) - mx); end % compute intra-class scatter Sw = zeros(kd, kd); for k=1:m indx = find(y==ids(k)); nk(k) = length(indx); % remove mean xk = x(indx, :) - repmat(m_cx(k,:), [nk(k), 1]); % adding up Sw = Sw + (xk'*xk); end p.6

7 LDA Implementation q Find projection by generalized Eigen problem solution % generalized eigen problem [A, v]=eigs(sb, Sw); if dbg figure(31); subplot(2,2,1); imagesc(sb); colormap('gray'); title('s_b'); subplot(2,2,2); imagesc(sw); colormap('gray'); title('s_w'); z = x*a; dist = pdist2(z, z); subplot(2,2,3); imagesc(dist); title('dist(j,k)'); subplot(2,2,4); stem(diag(v), '.'); grid on; hold on; title('eig v'); end p.7

8 Fisherface Basis q It is interesting to compare Fisherface with Eigenface basis Eigenface Fisherface p.8

9 Fisherface Performance q Fisher vs Eigenface performance: (fisherface.m) 1200 face images, 144 subjects Eigen kd=32 ROC 144 subjects p.9

10 q Recap: Eigenface Fisherface Outline q Graph Embedding Locality Preserving Projection Laplacian Face Graph Fourier Transform qsummary p.10

11 Locality Preserving Projection q Recall the dimension reduction formulations: find w, s.t y=wx: PCA: LDA: p.11

12 LPP Formulation Affinity q To preserve local affinity relationship Affinity map Selection of heat kernel size and threshold are important Hint: affinity matrix should be sparse affinity histogram p.12

13 LPP Formulation Affinity Supervised q How to utilize the label info? Heat map mapping is good for intra-class affinity modelling, but how about intra-class affinity? One direct solution is to set affinity to zero for intra class pairs % LPP - compute affinity f_dist1 = pdist2(x1, x1); % heat kernel size mdist = mean(f_dist1(:)); h = -log(0.15)/mdist; S1 = exp(-h*f_dist1); id_dist = pdist2(ids, ids); subplot(2,2,3); imagesc(id_dist); title('label distance'); S2=S1; S2(find(id_dist~=0)) = 0; subplot(2,2,4); imagesc(s1); colormap('gray'); title('affinitysupervised'); p.13

14 LPP- Affinity Preserving Projection q Find a projection that best preserves the affinity matrix p.14

15 LPP Formulation ql = D-S: nxn matrix, called graph Laplacian qnormalizing factor: nxn D Diagonal matrix, entry D ii = sum of col/row affinity The larger the value, the more important data point is q Constraint on D: p.15

16 q Now the formulation is, Generalized Eigen Problem q Lagranian q by KKT (Karush-Khun-Tucker) Condition, it is solved by a generalized Eigen problem X He, S Yan, Y Hu, P Niyogi, HJ Zhang, Face Recognition Using Laplacianface, IEEE Trans PAMI, vol. 27 (3), , p.16

17 q laplacianface.m Matlab Implementation %LPP n_face = 1200; n_subj = length(unique(ids(1:n_face))); % eigenface kd=32; x1 = faces(1:n_face,:)*a1(:,1:kd); ids=ids(1:n_face); % LPP - compute affinity f_dist1 = pdist2(x1, x1); % heat kernel size mdist = mean(f_dist1(:)); h = -log(0.15)/mdist; S1 = exp(-h*f_dist1); figure(32); subplot(2,2,1); imagesc(f_dist1); colormap('gray'); title('d(x_i, d_j)'); subplot(2,2,2); imagesc(s1); colormap('gray'); title('affinity'); %subplot(2,2,3); grid on; hold on; [h_aff, v_aff]=hist(s(:), 40); plot(v_aff, h_aff, '.-'); % utilize supervised info id_dist = pdist2(ids, ids); subplot(2,2,3); imagesc(id_dist); title('label distance'); S2=S1; S2(find(id_dist~=0)) = 0; subplot(2,2,4); imagesc(s1); colormap('gray'); title('affinity-supervised'); % laplacian face lpp_opt.pcaratio = 1; [A2, eigv2]=lpp(s2, lpp_opt, x1); p.17

18 Laplacian Face q Now, we can model face as a LPP projection: Eigenface Laplacian Face p.18

19 q 1200 faces, 144 subjects Laplacian vs Eigenface p.19

20 LPP and PCA q Graph Embedding is an unifying theory on dimension reduction PCA becomes special case of LPP, if we do not enforce local affinity p.20

21 qhow about LDA? Recall within class scatter: LPP and LDA This is i-th class Data covariance L i has diagonal entry of 1/n i, Equal affinity among data points p.21

22 LPP and LDA q Now consider the between class scatter C is the data covariance, regardless of label L is graph Laplacian computed from the affinity rule that, p.22

23 LDA as a special case of LPP q The same generalized Eigen problem p.23

24 Graph Embedding Interpretation of PCA/LDA/LPP q Affinity graph S, determines the embedding subspace W, via qpca and LDA are special cases of Graph Embedding PCA: LDA LPP p.24

25 Applications: facial expression embedding q Facial expressions embedded in a 2-d space via LPP frown happy sad neutral p.25

26 Application: Compression of SIFT q Compression of SIFT, preserve matching relationship, rather than reconstruction: p.26

27 Homework-3: Subspace Methods q Objective: Understand the graph embedding connections among popular subspace methods like PCA, LDA and LPP Practical experiences with serious size data set q Data Set: 7ww 417 subjects, 6650 image face data set, pre-processed to 20x20 pel images, intensity normalized to [0, 1] Add your own face images, 10~15, frontal q Tasks: Compute Eigenface, Fisherface and Laplacianface models ROC plot on verification performance map for retrieval/identification performance p.27

28 HW-3 test run q Laplacian face is powerful. J p.28

29 Graph Fourier Transform q David I. Shuman, Sunil K. Narang, Pascal Frossard, Antonio Ortega, Pierre Vandergheynst: The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains. IEEE Signal Process. Mag. 30(3): (2013) p.29

30 q non-uniformly sampled Signal on Graph p.30

31 Graph Fourier Transform q GFT is different from Laplacian Embedding: p.31

32 GFT Example qgraph Laplacian p.32

33 Normalized Graph Laplacian qnormalize by edge pair degree p.33

34 q Analogous to FT Graph Frourier Transform p.34

35 Graph Spectrum p.35

36 q Quadratic form on L: Graph Signal Smoothness p.36

37 Summary q Graph Laplacian Embedding is an unifying theory for feature space dimension reduction PCA is a special case of graph embedding o Fully connected affinity map, equal importance LDA is a special case of graph embedding o Fully connected intra class o Zero affinity inter class LPP: preserves pair wise affinity. GFT: eigen vectors of graph Laplacian, has Fourier Transform like characteristics. q Many applications in Face recognition Pose estimation Facial expression modeling Compression of Graph signals. p.37