Eigenimages. Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 1
|
|
- Ethan Kevin Johnson
- 6 years ago
- Views:
Transcription
1 Eigenimages " Unitary transforms" Karhunen-Loève transform" Eigenimages for recognition" Sirovich and Kirby method" Example: eigenfaces" Eigenfaces vs. Fisherfaces" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 1
2 Unitary transforms Sort samples f [x,y] of an MxN image (or a rectangular portion of the image) into column vector of length MN" Compute transform coefficients" " " " " where A is a matrix of size MNxMN The transform A is unitary, iff" c = Af A 1 = A *T A H Hermitian conjugate" If A is real-valued, i.e., A=A*, transform is orthonormal " Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 2
3 Energy conservation with unitary transforms c = A f For any unitary transform we obtain" c 2 = c H c = f H A H A f = f 2 Interpretation: every unitary transform is simply a rotation of the coordinate system (and, possibly, sign flips)" Vector length ( energy ) is conserved." Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 3
4 Energy distribution for unitary transforms Energy is conserved, but, in general, unevenly distributed among coefficients." Autocorrelation matrix" R cc = E c c H = E A f f H A H = AR ff AH Mean squared values ( average energies ) of the coefficients c i live on the diagonal of R cc " 2 E c i = R cc i,i = AR ff A H i,i Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 4
5 Eigenmatrix of the autocorrelation matrix Definition: eigenmatrix Φ of autocorrelation matrix R ff " l Φ is unitary" l The columns of Φ form a set of eigenvectors of R ff, i.e., " " " R ff Φ = ΦΛ Λ is a diagonal matrix of eigenvalues λ i " Λ = λ 0 0 l R ff is symmetric nonnegative definite, hence" λ i 0 for all i l R ff is normal matrix, i.e., R H H ff R ff = R ff R ff, hence unitary eigenmatrix exists ( spectral theorem )" λ 1 0 λ MN 1 Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 5
6 Karhunen-Loève transform Unitary transform with matrix" A = Φ H "where the columns of Φ are ordered according to decreasing eigenvalues." Transform coefficients are pairwise uncorrelated" Energy concentration property: " R cc = AR ff A H = Φ H R ff Φ = Φ H ΦΛ = Λ l Mean squared approximation error by choosing only first J coefficients is minimized. " l No other unitary transform packs as much energy into the first J coefficients, for any J" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 6
7 Illustration of energy concentration Strongly correlated samples, equal energies" f 2 f 1 A = cosθ sinθ sinθ cosθ c 2 c 1 After KLT: uncorrelated samples, most of the energy in first coefficient" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 7
8 Basis images and eigenimages For any unitary transform, the inverse transform" f = A H c "can be interpreted in terms of the superposition of basis images (columns of A H ) of size MN." If the transform is a KL transform, the basis images (aka eigenvectors of the autocorrelation matrix R ff ) are called eigenimages. " If energy concentration works well, only a limited number of eigenimages is needed to approximate a set of images with small error. These eigenimages form an optimal linear subspace of dimensionality J." Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 8
9 Eigenimages for recognition To recognize complex patterns (e.g., faces), large portions of an image (say of size MN) might have to be considered" High dimensionality of image space means high computational burden for many recognition techniques" Example: nearest-neigbor search requires pairwise comparison with every image in a database" Transform can reduce dimensionality from MN to J by representing the c = Wf image by J coefficients " Idea: tailor a KLT to the specific set of images of the recognition task to preserve the salient features" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 9
10 Eigenimages for recognition (cont.) [Ruiz-del-Solar and Navarrete, 2005]! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 10
11 Computing eigenimages from a training set How to measure MNxMN covariance matrix?" l Use training set Γ 1, Γ 2,, Γ L (each column vector represents one image)" l Let be the mean image of all samples" l " Problem 1: Training set size should be L >> MN" "" If L < MN, covariance matrix R is rank-deficient Problem 2: Finding eigenvectors of an MNxMN matrix." " µ Define training set matrix S = ( Γ 1 µ, Γ 2 µ, Γ 3 µ,, Γ L µ ), L and calculate R = Γ l µ Γ l µ l=1 ( )( ) H = SS H Can we find a small set of the most important eigenimages from a small training set L << MN? " Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 11
12 Sirovich and Kirby method " Instead of eigenvectors of SS H, consider the eigenvectors of S H S, i.e., Premultiply both sides by S S H S v i = λ i vi SS H S v i = λ i S v i By inspection, we find that S v i are eigenvectors of SS H For L << MN this gives rise to great computational savings, by " l Computing the LxL matrix S H S " l Computing L eigenvectors of S H S " l Computing eigenimages corresponding to the L 0 L largest eigenvalues according as" " S v i v i L. Sirovich and M. Kirby, "Low-dimensional procedure for the characterization of human faces," " Journal of the Optical Society of America A, 4(3), pp , 1987." Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 12
13 Example: eigenfaces The first 8 eigenfaces obtained from a training set of 180 male and 180 female training images (without mean removal)" Can be used to generate faces by adjusting 8 coefficients." A. Diaco, J. DiCarlo and J. Santos, EE368 class project, Spring 2000." Can be used for face recognition by nearest neighbor search in 8-d face space. " Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 13
14 Gender recognition using eigenfaces " Recognition accuracy using 8 eigenimages" Female face samples" A. Diaco, J. DiCarlo and J. Santos, EE368 class project, Spring 2000." Male face samples" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 14
15 Fisher linear discriminant analysis " Eigenimage method maximizes scatter within the linear subspace over the entire image set regardless of classification task" W opt = argmax W ( det( WRW H )) Fisher linear discriminant analysis (1936): maximize between-class scatter, while minimizing within-class scatter W opt = argmax W ( ) ( ) det WR B W H det WR W W H R B = Samples in class i R W = c c i=1 i=1 Γ l N ( i µ ) i µ i Class(i) µ ( ) H µ i µ Mean in class i ( Γ ) l Γ l µ i ( ) H Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 15
16 Fisher linear discriminant analysis (cont.) " Solution: Generalized eigenvectors w i K largest eigenvalues { λ i i = 1,2,..., K}, i.e. R B w i corresponding to the = λ i R W w i, i = 1,2,..., K Problem: within-class scatter matrix R w at most of rank L-c, hence usually singular." Apply KLT first to reduce dimension of feature space to L-c (or less), proceed with Fisher LDA in low-dimensional space Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 16
17 Eigenimages vs. Fisherimages 2d example:" Samples for 2 classes are projected onto 1d subspace using the" KLT (aka PCA) or" Fisher LDA (FLD)." PCA preserves maximum energy, but the 2 classes are no longer distinguishable. FLD separates the classes by choosing a better 1d subspace." " +" +" +" +" +" +" +" +" +" +" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 17 [Belhumeur, Hespanha, Kriegman, 1997]!
18 Fisherimages and varying iillumination Differences due to varying illumination can be much larger than differences between faces!" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 18 [Belhumeur, Hespanha, Kriegman, 1997]!
19 Fisherimages and varying iillumination All images of same Lambertian surface with different illumination (without shadows) lie in a 3d linear subspace" Single point source at infinity" surface normal n l light source direction f ( x, y) = a( x, y) l T n( x, y) Surface albedo" Light source intensity" Superposition of arbitrary number of point sources at infinity still in same 3d linear subspace, due to linear superposition of each contribution to image " Fisherimages can eliminate within-class scatter" ( ) L Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 19
20 Face recognition with Eigenfaces and Fisherfaces FERET data base," 254 classes," 3 images per class" [Belhumeur, Hespanha, Kriegman, 1997]! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 20
21 Fisherface trained to recognize glasses [Belhumeur, Hespanha, Kriegman, 1997]! Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 21
22 Fisherface trained to recognize gender Female face samples" Mean image" Female mean" Male mean" µ µ 1 µ 2 Male face samples" Fisherface" Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 22
23 Gender recognition using fisherface EER = 6.5%! Male Female 0.1 Probability Projection Score Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 23
24 Gender recognition using 2 nd eigenface EER = 24%! Male Female 0.1 Probability Projection Score Digital Image Processing: Bernd Girod, 2013 Stanford University -- Eigenimages 24
Example: Face Detection
Announcements HW1 returned New attendance policy Face Recognition: Dimensionality Reduction On time: 1 point Five minutes or more late: 0.5 points Absent: 0 points Biometrics CSE 190 Lecture 14 CSE190,
More informationRecognition Using Class Specific Linear Projection. Magali Segal Stolrasky Nadav Ben Jakov April, 2015
Recognition Using Class Specific Linear Projection Magali Segal Stolrasky Nadav Ben Jakov April, 2015 Articles Eigenfaces vs. Fisherfaces Recognition Using Class Specific Linear Projection, Peter N. Belhumeur,
More informationImage Analysis & Retrieval. Lec 14. Eigenface and Fisherface
Image Analysis & Retrieval Lec 14 Eigenface and Fisherface Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph: x 2346. http://l.web.umkc.edu/lizhu Z. Li, Image Analysis & Retrv, Spring
More informationLecture: Face Recognition
Lecture: Face Recognition Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 12-1 What we will learn today Introduction to face recognition The Eigenfaces Algorithm Linear
More informationFace Recognition. Face Recognition. Subspace-Based Face Recognition Algorithms. Application of Face Recognition
ace Recognition Identify person based on the appearance of face CSED441:Introduction to Computer Vision (2017) Lecture10: Subspace Methods and ace Recognition Bohyung Han CSE, POSTECH bhhan@postech.ac.kr
More informationImage Analysis & Retrieval Lec 14 - Eigenface & Fisherface
CS/EE 5590 / ENG 401 Special Topics, Spring 2018 Image Analysis & Retrieval Lec 14 - Eigenface & Fisherface Zhu Li Dept of CSEE, UMKC http://l.web.umkc.edu/lizhu Office Hour: Tue/Thr 2:30-4pm@FH560E, Contact:
More informationDimensionality Reduction Using PCA/LDA. Hongyu Li School of Software Engineering TongJi University Fall, 2014
Dimensionality Reduction Using PCA/LDA Hongyu Li School of Software Engineering TongJi University Fall, 2014 Dimensionality Reduction One approach to deal with high dimensional data is by reducing their
More informationA Unified Bayesian Framework for Face Recognition
Appears in the IEEE Signal Processing Society International Conference on Image Processing, ICIP, October 4-7,, Chicago, Illinois, USA A Unified Bayesian Framework for Face Recognition Chengjun Liu and
More informationPCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
PCA & ICA CE-717: Machine Learning Sharif University of Technology Spring 2015 Soleymani Dimensionality Reduction: Feature Selection vs. Feature Extraction Feature selection Select a subset of a given
More informationSubspace Methods for Visual Learning and Recognition
This is a shortened version of the tutorial given at the ECCV 2002, Copenhagen, and ICPR 2002, Quebec City. Copyright 2002 by Aleš Leonardis, University of Ljubljana, and Horst Bischof, Graz University
More informationLecture 17: Face Recogni2on
Lecture 17: Face Recogni2on Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei-Fei Li Stanford Vision Lab Lecture 17-1! What we will learn today Introduc2on to face recogni2on Principal Component Analysis
More informationEnhanced Fisher Linear Discriminant Models for Face Recognition
Appears in the 14th International Conference on Pattern Recognition, ICPR 98, Queensland, Australia, August 17-2, 1998 Enhanced isher Linear Discriminant Models for ace Recognition Chengjun Liu and Harry
More informationLecture 17: Face Recogni2on
Lecture 17: Face Recogni2on Dr. Juan Carlos Niebles Stanford AI Lab Professor Fei-Fei Li Stanford Vision Lab Lecture 17-1! What we will learn today Introduc2on to face recogni2on Principal Component Analysis
More informationComparative Assessment of Independent Component. Component Analysis (ICA) for Face Recognition.
Appears in the Second International Conference on Audio- and Video-based Biometric Person Authentication, AVBPA 99, ashington D. C. USA, March 22-2, 1999. Comparative Assessment of Independent Component
More informationVectors To begin, let us describe an element of the state space as a point with numerical coordinates, that is x 1. x 2. x =
Linear Algebra Review Vectors To begin, let us describe an element of the state space as a point with numerical coordinates, that is x 1 x x = 2. x n Vectors of up to three dimensions are easy to diagram.
More informationPrincipal Component Analysis -- PCA (also called Karhunen-Loeve transformation)
Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) PCA transforms the original input space into a lower dimensional space, by constructing dimensions that are linear combinations
More informationCourse 495: Advanced Statistical Machine Learning/Pattern Recognition
Course 495: Advanced Statistical Machine Learning/Pattern Recognition Deterministic Component Analysis Goal (Lecture): To present standard and modern Component Analysis (CA) techniques such as Principal
More informationWhen Fisher meets Fukunaga-Koontz: A New Look at Linear Discriminants
When Fisher meets Fukunaga-Koontz: A New Look at Linear Discriminants Sheng Zhang erence Sim School of Computing, National University of Singapore 3 Science Drive 2, Singapore 7543 {zhangshe, tsim}@comp.nus.edu.sg
More informationReconnaissance d objetsd et vision artificielle
Reconnaissance d objetsd et vision artificielle http://www.di.ens.fr/willow/teaching/recvis09 Lecture 6 Face recognition Face detection Neural nets Attention! Troisième exercice de programmation du le
More informationFace Recognition. Lecture-14
Face Recognition Lecture-14 Face Recognition imple Approach Recognize faces (mug shots) using gray levels (appearance). Each image is mapped to a long vector of gray levels. everal views of each person
More informationLecture 24: Principal Component Analysis. Aykut Erdem May 2016 Hacettepe University
Lecture 4: Principal Component Analysis Aykut Erdem May 016 Hacettepe University This week Motivation PCA algorithms Applications PCA shortcomings Autoencoders Kernel PCA PCA Applications Data Visualization
More information1 Singular Value Decomposition and Principal Component
Singular Value Decomposition and Principal Component Analysis In these lectures we discuss the SVD and the PCA, two of the most widely used tools in machine learning. Principal Component Analysis (PCA)
More informationThe Singular Value Decomposition
The Singular Value Decomposition Philippe B. Laval KSU Fall 2015 Philippe B. Laval (KSU) SVD Fall 2015 1 / 13 Review of Key Concepts We review some key definitions and results about matrices that will
More informationW vs. QCD Jet Tagging at the Large Hadron Collider
W vs. QCD Jet Tagging at the Large Hadron Collider Bryan Anenberg: anenberg@stanford.edu; CS229 December 13, 2013 Problem Statement High energy collisions of protons at the Large Hadron Collider (LHC)
More informationFace Recognition. Lecture-14
Face Recognition Lecture-14 Face Recognition imple Approach Recognize faces mug shots) using gray levels appearance). Each image is mapped to a long vector of gray levels. everal views of each person are
More informationPattern Recognition 2
Pattern Recognition 2 KNN,, Dr. Terence Sim School of Computing National University of Singapore Outline 1 2 3 4 5 Outline 1 2 3 4 5 The Bayes Classifier is theoretically optimum. That is, prob. of error
More informationLinear & Non-Linear Discriminant Analysis! Hugh R. Wilson
Linear & Non-Linear Discriminant Analysis! Hugh R. Wilson PCA Review! Supervised learning! Fisher linear discriminant analysis! Nonlinear discriminant analysis! Research example! Multiple Classes! Unsupervised
More informationStatistical Pattern Recognition
Statistical Pattern Recognition Feature Extraction Hamid R. Rabiee Jafar Muhammadi, Alireza Ghasemi, Payam Siyari Spring 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Agenda Dimensionality Reduction
More informationModel-based Characterization of Mammographic Masses
Model-based Characterization of Mammographic Masses Sven-René von der Heidt 1, Matthias Elter 2, Thomas Wittenberg 2, Dietrich Paulus 1 1 Institut für Computervisualistik, Universität Koblenz-Landau 2
More informationFocus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations.
Previously Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations y = Ax Or A simply represents data Notion of eigenvectors,
More informationLecture 13 Visual recognition
Lecture 13 Visual recognition Announcements Silvio Savarese Lecture 13-20-Feb-14 Lecture 13 Visual recognition Object classification bag of words models Discriminative methods Generative methods Object
More informationKeywords Eigenface, face recognition, kernel principal component analysis, machine learning. II. LITERATURE REVIEW & OVERVIEW OF PROPOSED METHODOLOGY
Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Eigenface and
More informationFace Recognition and Biometric Systems
The Eigenfaces method Plan of the lecture Principal Components Analysis main idea Feature extraction by PCA face recognition Eigenfaces training feature extraction Literature M.A.Turk, A.P.Pentland Face
More informationAdvanced Introduction to Machine Learning CMU-10715
Advanced Introduction to Machine Learning CMU-10715 Principal Component Analysis Barnabás Póczos Contents Motivation PCA algorithms Applications Some of these slides are taken from Karl Booksh Research
More informationPrincipal Component Analysis
B: Chapter 1 HTF: Chapter 1.5 Principal Component Analysis Barnabás Póczos University of Alberta Nov, 009 Contents Motivation PCA algorithms Applications Face recognition Facial expression recognition
More informationPrincipal Component Analysis and Linear Discriminant Analysis
Principal Component Analysis and Linear Discriminant Analysis Ying Wu Electrical Engineering and Computer Science Northwestern University Evanston, IL 60208 http://www.eecs.northwestern.edu/~yingwu 1/29
More informationEigenfaces and Fisherfaces
Eigenfaces and Fisherfaces Dimension Reduction and Component Analysis Jason Corso University of Michigan EECS 598 Fall 2014 Foundations of Computer Vision JJ Corso (University of Michigan) Eigenfaces and
More informationPrincipal Component Analysis (PCA)
Principal Component Analysis (PCA) Additional reading can be found from non-assessed exercises (week 8) in this course unit teaching page. Textbooks: Sect. 6.3 in [1] and Ch. 12 in [2] Outline Introduction
More informationDecember 20, MAA704, Multivariate analysis. Christopher Engström. Multivariate. analysis. Principal component analysis
.. December 20, 2013 Todays lecture. (PCA) (PLS-R) (LDA) . (PCA) is a method often used to reduce the dimension of a large dataset to one of a more manageble size. The new dataset can then be used to make
More informationCOS 429: COMPUTER VISON Face Recognition
COS 429: COMPUTER VISON Face Recognition Intro to recognition PCA and Eigenfaces LDA and Fisherfaces Face detection: Viola & Jones (Optional) generic object models for faces: the Constellation Model Reading:
More informationEigenimaging for Facial Recognition
Eigenimaging for Facial Recognition Aaron Kosmatin, Clayton Broman December 2, 21 Abstract The interest of this paper is Principal Component Analysis, specifically its area of application to facial recognition
More informationECE 661: Homework 10 Fall 2014
ECE 661: Homework 10 Fall 2014 This homework consists of the following two parts: (1) Face recognition with PCA and LDA for dimensionality reduction and the nearest-neighborhood rule for classification;
More informationRegularized Discriminant Analysis and Reduced-Rank LDA
Regularized Discriminant Analysis and Reduced-Rank LDA Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Regularized Discriminant Analysis A compromise between LDA and
More informationRobot Image Credit: Viktoriya Sukhanova 123RF.com. Dimensionality Reduction
Robot Image Credit: Viktoriya Sukhanova 13RF.com Dimensionality Reduction Feature Selection vs. Dimensionality Reduction Feature Selection (last time) Select a subset of features. When classifying novel
More informationAn Efficient Pseudoinverse Linear Discriminant Analysis method for Face Recognition
An Efficient Pseudoinverse Linear Discriminant Analysis method for Face Recognition Jun Liu, Songcan Chen, Daoqiang Zhang, and Xiaoyang Tan Department of Computer Science & Engineering, Nanjing University
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationStructure in Data. A major objective in data analysis is to identify interesting features or structure in the data.
Structure in Data A major objective in data analysis is to identify interesting features or structure in the data. The graphical methods are very useful in discovering structure. There are basically two
More informationThe Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)
Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will
More informationMachine Learning. Data visualization and dimensionality reduction. Eric Xing. Lecture 7, August 13, Eric Xing Eric CMU,
Eric Xing Eric Xing @ CMU, 2006-2010 1 Machine Learning Data visualization and dimensionality reduction Eric Xing Lecture 7, August 13, 2010 Eric Xing Eric Xing @ CMU, 2006-2010 2 Text document retrieval/labelling
More informationMain matrix factorizations
Main matrix factorizations A P L U P permutation matrix, L lower triangular, U upper triangular Key use: Solve square linear system Ax b. A Q R Q unitary, R upper triangular Key use: Solve square or overdetrmined
More informationDimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
More informationFace recognition Computer Vision Spring 2018, Lecture 21
Face recognition http://www.cs.cmu.edu/~16385/ 16-385 Computer Vision Spring 2018, Lecture 21 Course announcements Homework 6 has been posted and is due on April 27 th. - Any questions about the homework?
More informationDiscriminant Uncorrelated Neighborhood Preserving Projections
Journal of Information & Computational Science 8: 14 (2011) 3019 3026 Available at http://www.joics.com Discriminant Uncorrelated Neighborhood Preserving Projections Guoqiang WANG a,, Weijuan ZHANG a,
More informationCOMP 408/508. Computer Vision Fall 2017 PCA for Recognition
COMP 408/508 Computer Vision Fall 07 PCA or Recognition Recall: Color Gradient by PCA v λ ( G G, ) x x x R R v, v : eigenvectors o D D with v ^v (, ) x x λ, λ : eigenvalues o D D with λ >λ v λ ( B B, )
More informationSymmetric Two Dimensional Linear Discriminant Analysis (2DLDA)
Symmetric Two Dimensional inear Discriminant Analysis (2DDA) Dijun uo, Chris Ding, Heng Huang University of Texas at Arlington 701 S. Nedderman Drive Arlington, TX 76013 dijun.luo@gmail.com, {chqding,
More informationCSC 411 Lecture 12: Principal Component Analysis
CSC 411 Lecture 12: Principal Component Analysis Roger Grosse, Amir-massoud Farahmand, and Juan Carrasquilla University of Toronto UofT CSC 411: 12-PCA 1 / 23 Overview Today we ll cover the first unsupervised
More informationDimension Reduction and Component Analysis Lecture 4
Dimension Reduction and Component Analysis Lecture 4 Jason Corso SUNY at Buffalo 28 January 2009 J. Corso (SUNY at Buffalo) Lecture 4 28 January 2009 1 / 103 Plan In lecture 3, we learned about estimating
More informationMultiple Similarities Based Kernel Subspace Learning for Image Classification
Multiple Similarities Based Kernel Subspace Learning for Image Classification Wang Yan, Qingshan Liu, Hanqing Lu, and Songde Ma National Laboratory of Pattern Recognition, Institute of Automation, Chinese
More information«Random Vectors» Lecture #2: Introduction Andreas Polydoros
«Random Vectors» Lecture #2: Introduction Andreas Polydoros Introduction Contents: Definitions: Correlation and Covariance matrix Linear transformations: Spectral shaping and factorization he whitening
More informationMeasures. Wendy S. Yambor, Bruce A. Draper, and J. Ross Beveridge 1. Colorado State University.
Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures Wendy S. Yambor, Bruce A. Draper, and J. Ross Beveridge 1 Computer Science Department Colorado State University
More informationFace Recognition in Subspaces
MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com Face Recognition in Subspaces Gregory Shakhnarovich Baback Moghaddam TR2004-041 May 2004 Abstract Images of faces, represented as high-dimensional
More informationPrincipal Component Analysis (PCA)
Principal Component Analysis (PCA) Salvador Dalí, Galatea of the Spheres CSC411/2515: Machine Learning and Data Mining, Winter 2018 Michael Guerzhoy and Lisa Zhang Some slides from Derek Hoiem and Alysha
More informationMachine Learning. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395
Machine Learning Dimensionality reduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1395 1 / 47 Table of contents 1 Introduction
More informationEvolutionary Pursuit and Its Application to Face Recognition
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 50-52, 2000. Evolutionary Pursuit and Its Application to Face Recognition Chengjun Liu, Member, IEEE, and Harry Wechsler, Fellow,
More informationEigenfaces. Face Recognition Using Principal Components Analysis
Eigenfaces Face Recognition Using Principal Components Analysis M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive Neuroscience, 3(1), pp. 71-86, 1991. Slides : George Bebis, UNR
More informationModeling Classes of Shapes Suppose you have a class of shapes with a range of variations: System 2 Overview
4 4 4 6 4 4 4 6 4 4 4 6 4 4 4 6 4 4 4 6 4 4 4 6 4 4 4 6 4 4 4 6 Modeling Classes of Shapes Suppose you have a class of shapes with a range of variations: System processes System Overview Previous Systems:
More informationLinear Algebra using Dirac Notation: Pt. 2
Linear Algebra using Dirac Notation: Pt. 2 PHYS 476Q - Southern Illinois University February 6, 2018 PHYS 476Q - Southern Illinois University Linear Algebra using Dirac Notation: Pt. 2 February 6, 2018
More informationLecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed in the
More informationCS4495/6495 Introduction to Computer Vision. 8B-L2 Principle Component Analysis (and its use in Computer Vision)
CS4495/6495 Introduction to Computer Vision 8B-L2 Principle Component Analysis (and its use in Computer Vision) Wavelength 2 Wavelength 2 Principal Components Principal components are all about the directions
More informationDimension Reduction and Component Analysis
Dimension Reduction and Component Analysis Jason Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Dimension Reduction and Component Analysis 1 / 103 Plan We learned about estimating parametric models and
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationFace Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi
Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi Overview Introduction Linear Methods for Dimensionality Reduction Nonlinear Methods and Manifold
More informationLinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Principal Component Analysis 3 Factor Analysis
More informationTable of Contents. Multivariate methods. Introduction II. Introduction I
Table of Contents Introduction Antti Penttilä Department of Physics University of Helsinki Exactum summer school, 04 Construction of multinormal distribution Test of multinormality with 3 Interpretation
More informationRandom Sampling LDA for Face Recognition
Random Sampling LDA for Face Recognition Xiaogang Wang and Xiaoou ang Department of Information Engineering he Chinese University of Hong Kong {xgwang1, xtang}@ie.cuhk.edu.hk Abstract Linear Discriminant
More informationLEC 2: Principal Component Analysis (PCA) A First Dimensionality Reduction Approach
LEC 2: Principal Component Analysis (PCA) A First Dimensionality Reduction Approach Dr. Guangliang Chen February 9, 2016 Outline Introduction Review of linear algebra Matrix SVD PCA Motivation The digits
More informationFisher s Linear Discriminant Analysis
Fisher s Linear Discriminant Analysis Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationPCA and LDA. Man-Wai MAK
PCA and LDA Man-Wai MAK Dept. of Electronic and Information Engineering, The Hong Kong Polytechnic University enmwmak@polyu.edu.hk http://www.eie.polyu.edu.hk/ mwmak References: S.J.D. Prince,Computer
More informationIEEE. Proof. LINEAR DISCRIMINANT analysis (LDA) [1], [16] has
TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART B: CYBERNETICS, VOL 35, NO 6, DECEMBER 2005 1 GA-Fisher: A New LDA-Based Face Recognition Algorithm With Selection of Principal Components Wei-Shi Zheng,
More informationProblem Session #5. EE368/CS232 Digital Image Processing
Problem Session #5 EE368/CS232 Digital Image Processing 1. Solving a Jigsaw Puzzle Please download the image hw5_puzzle_pieces.jpg from the handouts webpage, which shows the pieces of a jigsaw puzzle.
More informationCITS 4402 Computer Vision
CITS 4402 Computer Vision A/Prof Ajmal Mian Adj/A/Prof Mehdi Ravanbakhsh Lecture 06 Object Recognition Objectives To understand the concept of image based object recognition To learn how to match images
More informationLecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 11-1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed
More informationKarhunen-Loève Transform KLT. JanKees van der Poel D.Sc. Student, Mechanical Engineering
Karhunen-Loève Transform KLT JanKees van der Poel D.Sc. Student, Mechanical Engineering Karhunen-Loève Transform Has many names cited in literature: Karhunen-Loève Transform (KLT); Karhunen-Loève Decomposition
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More informationIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 31, NO. 5, MAY ASYMMETRIC PRINCIPAL COMPONENT ANALYSIS
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 31, NO. 5, MAY 2009 931 Short Papers Asymmetric Principal Component and Discriminant Analyses for Pattern Classification Xudong Jiang,
More informationLecture 16: Small Sample Size Problems (Covariance Estimation) Many thanks to Carlos Thomaz who authored the original version of these slides
Lecture 16: Small Sample Size Problems (Covariance Estimation) Many thanks to Carlos Thomaz who authored the original version of these slides Intelligent Data Analysis and Probabilistic Inference Lecture
More informationSystem 1 (last lecture) : limited to rigidly structured shapes. System 2 : recognition of a class of varying shapes. Need to:
System 2 : Modelling & Recognising Modelling and Recognising Classes of Classes of Shapes Shape : PDM & PCA All the same shape? System 1 (last lecture) : limited to rigidly structured shapes System 2 :
More informationFace Recognition. Lauren Barker
Face Recognition Lauren Barker 24th April 2011 Abstract This report presents an exploration into the various techniques involved in attempting to solve the problem of face recognition. Focus is paid to
More informationMultilinear Analysis of Image Ensembles: TensorFaces
Multilinear Analysis of Image Ensembles: TensorFaces M Alex O Vasilescu and Demetri Terzopoulos Courant Institute, New York University, USA Department of Computer Science, University of Toronto, Canada
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More informationChapter 2 Face Recognition in Subspaces
Chapter 2 Face Recognition in Subspaces Gregory Shakhnarovich and Baback Moghaddam 2.1 Introduction Images of faces, represented as high-dimensional pixel arrays, often belong to a manifold of intrinsically
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationEECS490: Digital Image Processing. Lecture #26
Lecture #26 Moments; invariant moments Eigenvector, principal component analysis Boundary coding Image primitives Image representation: trees, graphs Object recognition and classes Minimum distance classifiers
More informationIV. Matrix Approximation using Least-Squares
IV. Matrix Approximation using Least-Squares The SVD and Matrix Approximation We begin with the following fundamental question. Let A be an M N matrix with rank R. What is the closest matrix to A that
More informationIntroduction to Machine Learning
10-701 Introduction to Machine Learning PCA Slides based on 18-661 Fall 2018 PCA Raw data can be Complex, High-dimensional To understand a phenomenon we measure various related quantities If we knew what
More informationA Tutorial on Data Reduction. Principal Component Analysis Theoretical Discussion. By Shireen Elhabian and Aly Farag
A Tutorial on Data Reduction Principal Component Analysis Theoretical Discussion By Shireen Elhabian and Aly Farag University of Louisville, CVIP Lab November 2008 PCA PCA is A backbone of modern data
More informationMethods for sparse analysis of high-dimensional data, II
Methods for sparse analysis of high-dimensional data, II Rachel Ward May 26, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 55 High dimensional
More information1 Principal Components Analysis
Lecture 3 and 4 Sept. 18 and Sept.20-2006 Data Visualization STAT 442 / 890, CM 462 Lecture: Ali Ghodsi 1 Principal Components Analysis Principal components analysis (PCA) is a very popular technique for
More informationNumerical Methods I Singular Value Decomposition
Numerical Methods I Singular Value Decomposition Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 9th, 2014 A. Donev (Courant Institute)
More informationMachine Learning 11. week
Machine Learning 11. week Feature Extraction-Selection Dimension reduction PCA LDA 1 Feature Extraction Any problem can be solved by machine learning methods in case of that the system must be appropriately
More information