Image Analysis & Retrieval Lec 13 - Feature Dimension Reduction

Transcription

1 CS/EE 5590 / ENG 401 Special Topics, Spring 2018 Image Analysis & Retrieval Lec 13 - Feature Dimension Reduction Zhu Li Dept of CSEE, UMKC Office Hour: Tue/Thr 2:30-4pm@FH560E, Contact: lizhu@umkc.edu, Ph: x Created using WPS office and EqualX latex equation editor Z. Li, Image Analysis & Retrv, Spring 2018 p.1

2 q Recap: Part I q Linear Algebra Refresher q SVD Outline q Principal Component Analysis (PCA) q Laplacian Eigen Map (LEM) Z. Li, Image Analysis & Retrv, Spring 2018 p.2

3 Handcrafted Feature Pipeline q An image retrieval pipeline (hand crafted features) Image Formation Feature Computing Feature Aggregation Classification Homography, Color space Color histogram Filtering, Edge Detection HoG, Harris Detector, LoG Scale space, SIFT Bow VLAD Fisher Vector Supervector Knowledge /Data Base TPR, FPR, Precision, Recall, map knn, Bayesian SVM, Kernel Machine Z. Li, Image Analysis & Retrv, Spring 2018 p.3

4 Vector and Matrix Notations q Vector qmatrix Z. Li, Image Analysis & Retrv, Spring 2018 p.4

5 Vector Products q Inner Product qouter Product Z. Li, Image Analysis & Retrv, Spring 2018 p.5

6 q y=ax Matrix-Vector Product So, y is a linear combination of basis {a k } with weights from x Z. Li, Image Analysis & Retrv, Spring 2018 p.6

7 q C=AB Matrix Product A: nxp B: pxm A: nxm = q Associative: ABC = (AB)C = A(BC) qdistributive: A(B+C) = AB + AC Z. Li, Image Analysis & Retrv, Spring 2018 p.7

8 q Vector outer product: Outer Product/Kron qexample Z. Li, Image Analysis & Retrv, Spring 2018 p.8

9 Matrix Transpose q Transpose Z. Li, Image Analysis & Retrv, Spring 2018 p.9

10 Matrix Trace and Determinant q Trace:Tr(A): only for nxn square matrix q Determinant: Det(A): The size of volumes spanned by A, All possible linear combinations of a 1 and a 2 Det(A) = 2-9 = 7; Z. Li, Image Analysis & Retrv, Spring 2018 p.10

11 Eigen Values and Eigen Vectors q Definition: for nxn matrix A: q In Matlab: [P, V]=eig(A); Z. Li, Image Analysis & Retrv, Spring 2018 p.11

12 Eigen Vectors of Symmetric Matrix Z. Li, Image Analysis & Retrv, Spring 2018 p.12

13 SVD q for non square matrix: A mxn: Z. Li, Image Analysis & Retrv, Spring 2018 p.13

14 SVD as Signal Expansion A (mxn) = U (mxm) S (mxn) V (nxn) The 1 st order SVD approx. of A is: Z. Li, Image Analysis & Retrv, Spring 2018 p.14

15 q Very easy SVD approximation of an image function [x]=svd_approx(x0, k) dbg=0; if dbg x0= fix(100*randn(4,6)); k=2; end [u, s, v]=svd(x0); [m, n]=size(s); x = zeros(m, n); sgm = diag(s); for j=1:k x = x + sgm(j)*u(:,j)*v(:,j)'; end Z. Li, Image Analysis & Retrv, Spring 2018 p.15

16 SVD for Separable Filtering q Take LoG filter for eg. h=fspecial('log', 11, 2.0); [u,s,v]=svd(h); h1=s(1,1)*u(:,1)*v(:,1)'; h 1 is 1-SVD approx of LoG Many implications for Deep networks acceleration! Z. Li, Image Analysis & Retrv, Spring 2018 p.16

17 Norm qvector Norm: Length of the vector Euclidean Norm (L2 Norm): norm(x, 2) L p norm: qmatrix Norm: Forbenius Norm Z. Li, Image Analysis & Retrv, Spring 2018 p.17

18 Quadratic Form q Quadratic form f(x)=x T Ax in R: qpositive Definite (PD): For non-zero x, x T Ax > 0 qpositive Semi-Definite (PSD): For non-zero x, x T Ax >= 0 q Indefinite: Exists x 1, x 2 non zero, but x 1 TAx 1 >0, while x 2 TAx 2 < 0; Z. Li, Image Analysis & Retrv, Spring 2018 p.18

19 Matrix Calculus q Gradient of f(a): q Matrix Gradient Properties Z. Li, Image Analysis & Retrv, Spring 2018 p.19

20 Hessian of f(x) Z. Li, Image Analysis & Retrv, Spring 2018 p.20

21 q Recap: About HW-2 Quiz-1 q Linear Algebra Refresher q SVD Outline q Principal Component Analysis (PCA) Z. Li, Image Analysis & Retrv, Spring 2018 p.21

22 PCA -Dimension Reduction in Retrieval q A typical image retrieval pipeline R d -> R p Image Formation Feature Computing Feature Aggregation Classification e.g, dense SIFT: x 128 e.g, Fisher Vector: k=64, d=128 Knowledge /Data Base Z. Li, Image Analysis & Retrv, Spring 2018 p.22

23 Principal Component Analysis q The formulation: for data points {x 1, x 2,, } in R n, find a lower dimensional representation in R m, via a projection W,: mxn, s.t., the energy of the data is preserved Z. Li, Image Analysis & Retrv, Spring 2018 p.23

24 PCA solution q Take the Lagrangian of the problem q Take the derivative w.r.t. to w, and KKT condition gives us, This is an Eigen problem, finding projection s.t. it is just a scaling along the scatter matrix eigen vectors. Z. Li, Image Analysis & Retrv, Spring 2018 p.24

25 PCA how to compute q PCA via SVD on the Covariance matrix S: covariance, nxn Z. Li, Image Analysis & Retrv, Spring 2018 p.25

26 2d Data Z. Li, Image Analysis & Retrv, Spring 2018 p.26

27 Principal Components qgives best axis to project qminimum RMS error qprincipal vectors are orthogonal st principal vector nd principal vector Z. Li, Image Analysis & Retrv, Spring 2018 p.27

28 PCA on HoGs q Matlab Implementation of PCA: [A, s, eig_values]=princomp(hogs); HoG basis function Z. Li, Image Analysis & Retrv, Spring 2018 p.28

29 PCA Application in Aggregation q SIFT aggregation Usually a PCA is done on SIFT features, to reduce the dimension from 128 to say 24, 32. Then a GMM is trained in R 32 space, for FV encoding q Homework-2 Aggregation Fisher Vector Aggregation of SIFT load../../dataset/cdvs_sift_aggregation_te st_data.mat; [n_sift, kd_sift]=size(gd_sift_cdvs); offs = randperm(n_sift); offs = offs(1:200*2^10); % PCA [A1, s1, lat1]=princomp(double(gd_sift_cdvs(off s,:))); figure(41); hold on; grid on; stem(lat1, '.'); title('sift pca eigen values'); Z. Li, Image Analysis & Retrv, Spring 2018 p.29

30 qeigen values SIFT PCA That is why we have kd=[24, 32 48] for SIFT GMM in FV aggregation Z. Li, Image Analysis & Retrv, Spring 2018 p.30

31 SIFT PCA Basis Functions q Capturing max variation directions Z. Li, Image Analysis & Retrv, Spring 2018 p.31

32 Visualizing SIFT in lower dimensional space q Project SIFTs from 2 images to 2D space Z. Li, Image Analysis & Retrv, Spring 2018 p.32

33 q Recap: Part I and Quiz-1 q Linear Algebra Refresher q SVD Outline q Principal Component Analysis (PCA) q Laplacian Eigen Map (LEM) Z. Li, Image Analysis & Retrv, Spring 2018 p.33

34 Laplacian Eigen Map q Directly compute an embedding {y k } from input {x k } without the explicit projection model A, s.t. Y=AX q Objective function where the nxn affinity matrix W reflects the data points relationship in the original space X. M. Belkin and P. Niyogi. Laplacian Eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems, volume 14, pages , Cambridge, MA, USA, The MIT Press Z. Li, Image Analysis & Retrv, Spring 2018 p.34

35 qgraph Laplacian: L= D - W Graph Laplacian Z. Li, Image Analysis & Retrv, Spring 2018 p.35

36 q Minimizing the following Laplacian Eigenmap Is equivalent to Where D is the degree matrix (diagonal) with Z. Li, Image Analysis & Retrv, Spring 2018 p.36

37 Laplacian Eigen Map Solution q Numerically, solve the eigen problem: where the first d smallest eigen values corresponding eigenvectors, will form a d-dimensional feature of {y k } Z. Li, Image Analysis & Retrv, Spring 2018 p.37

38 Locally Linear Embedding (LLE) q History: S.T. Roweis and L.K. Saul. Nonlinear dimensionality reduction by Locally Linear Embedding.Science, 290(5500): , 2000 Procedure 1. Identify the neighbors of each data point 2. Compute weights that best linearly reconstruct the point from its neighbors 3. Find the low-dimensional embedding vector {y i } which is best reconstructed by the weights determined in Step 2 In matrix form: Z. Li, Image Analysis & Retrv, Spring

39 LLE Solution qsimilar to Laplacian Eigenmap, the first d eigenvectors provides the best local embedding. q Many successful applications in embedding 2d embedding (pose) with Laplacian Eigenmap 2d embedding with LLE Z. Li, Image Analysis & Retrv, Spring 2018 p.39

40 Summary q SVD and PCA SVD non-square matrix decomposition, left transform and right transform, with scaling in between SVD as an image decomposition, linear combination of outer-product basis PCA eigen values indicate amount of info/energy in each dimension, PCA basis are eigen vectors to the covariance matrix LEM direct data embedding without explicit projection Many applications Z. Li, Image Analysis & Retrv, Spring 2018 p.40