Kronecker Decomposition for Image Classification
|
|
- Blaise Black
- 5 years ago
- Views:
Transcription
1 university of innsbruck institute of computer science intelligent and interactive systems Kronecker Decomposition for Image Classification Sabrina Fontanella 1,2, Antonio Rodríguez-Sánchez 1, Justus Piater 1, and Sandor Szedmak 3 1 University of Innsbruck 2 University of Salerno 3 Aalto University Évora, September 2016
2 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 1/41
3 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 2/41
4 Image classification I I I Images are classified according to their visual content Applicability: 1. Recognition of specific objects 2. Indoor/outdoor recognition 3. Analysis of medical images Antonio Rodrı guez-sa nchez (CLEF 2016) 3/41
5 Image classification II Example of classification algorithm, Bag of Words: 1. Features extraction, stored into feature vectors 2. Approximation of the distribution of the features by an histogram 3. Apply a classification algorithm (Support Vector Machine, Neural Network, Markov Random Field, etc) Antonio Rodríguez-Sánchez (CLEF 2016) 4/41
6 Relations between objects are of interest Is it possible to recognize relationships between the objects appearing in a scene? This is of interest, since this relationship can provide knowledge necessary to identify and classify the image E.g. A car is quite likely to be in an image where there is also buildings and people. E.g. A zebra is quite likely to be outdoors, surrounded by Savanna plants or animals. Antonio Rodríguez-Sánchez (CLEF 2016) 5/41
7 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 6/41
8 Decomposing the environment Structured decomposition of the environment Learning structured output is a popular stream of machine learning By decomposing the matrix that represent the image, the structure behind the scene could be captured Let us consider 2D image decomposition Points close to each other within continuous 2D blocks can strongly relate to each other Antonio Rodríguez-Sánchez (CLEF 2016) 7/41
9 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 8/41
10 Tensor decomposition A tensor is a multidimensional or N-way array an N- way or Nth-order tensor is an element of the tensor product of N vector spaces Tensor decomposition can be considered as a higher- order generalization of the matrix singular value decomposition (SVD) and principal component analysis (PCA) The tensor decomposition for a same image is not unique Given an RGB image of size (256,256,3), it is possible to perform the following decompositions: (16,16,3),(16,16,1) tensor + matrix (2 components) (8,8,3), (8,8,1), (4,4,1) tensor + 2 matrices (3 components) Antonio Rodríguez-Sánchez (CLEF 2016) 9/41
11 Tensor decomposition Concerning computer vision, the tensor decomposition could be used to represent: Color images, where three matrices express the RGB images and we can use a tensor of order three (for example (1024,1024,3)). Video stream of color images where the dimensions are R, G, B and the time. Antonio Rodríguez-Sánchez (CLEF 2016) 10/41
12 The Kronecker product Given two matrices A R m A n A and B R m B n B, the Kronecker product X can be expressed as: A 1,1 B A 1,2 B A 1,nA B A 2,1 B A 2,2 B A 2,nA B X = A B A ma,1b A ma,2b A ma,n A B with m X = m A m B, n X = n A n B If X is given (the image), how can we compute A and B (its components)? B can be considered as a 2D filter of the image represented by the matrix X components)? Antonio Rodríguez-Sánchez (CLEF 2016) 11/41
13 The Kronecker decomposition and SVD The Kronecker decomposition can be carried out by Singular Value Decomposition(SVD) Given an arbitrary matrix X with size m n the SVD is given by X = USV T where U R mxm is an orthogonal matrix of left singular vectors, where UU T = I m, V R nxn, is an orthogonal matrix of right singular vectors, where VV T = I n, S R mxn, is a diagonal matrix containing the singular values with nonnegative components in its diagonal Antonio Rodríguez-Sánchez (CLEF 2016) 12/41
14 Note The algorithm solving the SVD does not depend on the order of the elements of the matrix Thus, any permutation of the indexes, reordering, of the columns and (or) rows preserves the same solution We can then work on a reordered representation of the matrix X Antonio Rodríguez-Sánchez (CLEF 2016) 13/41
15 Algorithm for solving Kronecker decomposition 1. Reorder the matrix 2. Compute SVD decomposition 3. Compute the approximation of X 4. Invert the reordering Antonio Rodríguez-Sánchez (CLEF 2016) 14/41
16 Nearest Kronecker Product (NKP) Given a matrix X R mxn, the NKP problem involves minimizing: φ(a, B) = X A B F F is the Frobenius norm This problem can be solved using SVD, working on a reordered representation of X Antonio Rodríguez-Sánchez (CLEF 2016) 15/41
17 Step 1: Reorder matrix X 1 X x 11 x 12 x 13 x 14 x 15 x 16 x 21 x 22 x 23 x 24 x 25 x 26 x 31 x 32 x 33 x 34 x 35 x 36 x 41 x 42 x 43 x 44 x 45 x 46 x 51 x 52 x 53 x 54 x 55 x 56 x 61 x 62 x 63 x 64 x 65 x 66 = = A B a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 [ b11 b 12 b 21 b 22 ], can be reordered into = X = Ã B x 11 x 13 x 15 x 31 x 33 x 35 x 51 x 53 x 55 x 12 x 14 x 16 x 32 x 34 x 36 x 52 x 54 x 56 x 21 x 23 x 25 x 41 x 43 x 45 x 61 x 63 x 65 x 22 x 24 x 26 x 42 x 44 x 46 x 62 x 64 x 66 b 11 b 12 b 21 b 22 [ a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 ], 1 C.F.V. Loan. The ubiquitous Kronecker product. Journal of Computational and Applied Mathematics, 123:85-100, Antonio Rodríguez-Sánchez (CLEF 2016) 16/41
18 Approximation of X and reordering X vec(a) vec(b) F Vec() is a vectorization operator which stacks columns of a matrix on top of each other Problem of finding the nearest rank-1 matrix to X Well known solutions using SVD Antonio Rodríguez-Sánchez (CLEF 2016) 17/41
19 Step 2: Compute SVD decomposition X vec(a) vec(b) F Let X = USV T the decomposition of X The best à and B are defined as: à = σ 1 U(:, 1) and B = σ 1 V (:; 1) where σ 1 is the largest singular value and U and V are the corresponding singular vectors Antonio Rodríguez-Sánchez (CLEF 2016) 18/41
20 Steps 3 and 4: Approximation and reordering Once we have à and B is possible to compute the approximation of X Since at beginning we have changed the order of values into matrix, invert the reordering is necessary for obtain the original A and B Antonio Rodríguez-Sánchez (CLEF 2016) 19/41
21 Components and factorization The number of components and factorization influence the level of details Given, for example, a gray image of size (1024,1024): If it has many details, is better chose many components with small factorization: Example: (4,4)(4,4)(4,4)(4,4)(4,4) If is less detailed, less component with high factorization: Example: (32,32)(32,32) Antonio Rodríguez-Sánchez (CLEF 2016) 20/41
22 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 21/41
23 Compression I The tensor decomposition can provide a very high level of images compression It takes consideration only the largest singular values (Eckart-Young theorem) The level of compression is given by the total number of: elements in image matrix elements of components in the decomposition Antonio Rodríguez-Sánchez (CLEF 2016) 22/41
24 Compression II Let nsv number of singular values taken in consideration nf number of factors for component v value of factors n c the number of components used Then the total number of elements of components is given by: nsv n c v n f For simplify the notation we assume that the all factors are equal for every component Decomposition with different factors can be taken in consideration For example (32,28)(16,8)(2,4) Antonio Rodríguez-Sánchez (CLEF 2016) 23/41
25 Compression III: Example Given an image of size (1024,1024). It can be compressed with components (32,32)(32,32) and with 10 singular values by: = 51.2 (4,4),(4,4),(4,4),(4,4),(4,4) and with 10 singular values by: = Antonio Rodríguez-Sánchez (CLEF 2016) 24/41
26 Compression IV: Example Compression ratio: 202 Compression ratio: 99 Figure: Example of compression on toys room image. Antonio Rodríguez-Sánchez (CLEF 2016) 25/41
27 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 26/41
28 Interpretation of image components I X = A B B can be interpreted like an image filter It finds the boundary of the critical regions where most of the structural information concentrates This represents a big advantage: In general, in image filtering processes, a predetermined filter is used The Kronecker decomposition automatically tries to predict the optimal filters Antonio Rodríguez-Sánchez (CLEF 2016) 27/41
29 Interpretation of image components II Highest components (A) Lowest components (B) Figure: Toys room picture and its components. The Highest component and the Lowest component correspond to the matrices A1,... and B1,... respectively. Antonio Rodríguez-Sánchez (CLEF 2016) 28/41
30 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 29/41
31 Learning Sample set of pairs of output and input objects {(y i, x i ) : y i Y, x i X, i = 1,..., m} Define two functions, φ and ψ, that map the input and output objects respectively into linear vector spaces feature space in case of the input label space in case of the output φ : X H φ and ψ : Y H ψ Antonio Rodríguez-Sánchez (CLEF 2016) 30/41
32 Objective Find a linear function acting on the feature space f (φ(x)) = Wφ(x) + b that produces a prediction of every input object in the label space The output corresponding to X is: y = ψ 1 (f (φ(x))) Antonio Rodríguez-Sánchez (CLEF 2016) 31/41
33 MMR (Maximum Margin Regression) vs SVM (Support Vector Machine) MMR is a framework for multilabel classification Is based on Support Vector Machine (SVM) Key idea: reinterpretation of the normal vector w SVM w is the normal vector of the separating hyperplane. y i { 1, +1} binary outputs. The labels are equal to the binary objects. Extended View W is a linear operator projecting the feature space into the label space y i Y arbitrary outputs ψ(y i ) H ψ are the labels, the embedded outputs in a linear vector space Antonio Rodríguez-Sánchez (CLEF 2016) 32/41
34 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 33/41
35 ImageCLEF dataset Task: multi-label classification Figure: The hierarchy of classes in ImageCLEF multi-label challenge. Antonio Rodríguez-Sánchez (CLEF 2016) 34/41
36 Results on ImageCLEF F1 score F1 score Degree of polynomial (a) Standard deviation (b) Figure: Results for six filter sizes: 4, 8, 12, 20, 18 and 32 using 3 components, training with two different kernel: a) polynomial b) Gaussian. The parameter varied in F1 measure are degree of polynomial from 1 to 10 for polynomial kernel and values of standard deviation of Gaussian for Gaussian kernel. Antonio Rodríguez-Sánchez (CLEF 2016) 35/41
37 Outline Image classification The problem Decomposing the environment The tensor decomposition What is it Compression Interpretation of the image components Learning approach Maximum Margin Regression Experimental evaluation ImageCLEF 2015 Experimental evaluation Pascal and Flickr Antonio Rodríguez-Sánchez (CLEF 2016) 36/41
38 Pascal and Flickr: Features to compare to Feature Dimension Source Descriptor Hsv 4096 color HSV Lab 4096 color LAB Rgb 4096 color RGB HsvV3H color HSV LabV3H color LAB RgbV3H color RGB DenseHue 100 texture hue HarrisHue 100 texture Hue DenseHueV3H1 300 texture hue HarrisHueV3H1 300 texture Hue DenseSift 1000 texture sift HarrisSift 1000 texture sift DenseSiftV3H texture sift HarrisSiftV3H texture sift Figure: Comparing tensor decomposition with other features 1 on Pascal07 dataset with Gaussian and Polynomial kernel. The decomposition chosen is 3 components with factorization (22,22). 1 Matthieu Guillaumin, Thomas Mensink, Jakob Verbeek, and Cordelia Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation, Antonio Rodríguez-Sánchez (CLEF 2016) 37/41
39 Results on Pascal07 dataset Gaussian kernel Feature P(%) R(%) F1(%) TD HarrisSiftV3H HarrisSift DenseSiftV3H DenseSift LabV3H DenseHueV3H HarrisHueV3H RgbV3H HsvV3H Hsv Lab Rgb HarrisHue DenseHue Polynomial kernel Feature P(%) R(%) F1(%) TD HarrisSiftV3H HarrisSift DenseSiftV3H DenseSift HsvV3H RgbV3H LabV3H HarrisHueV3H DenseHueV3H Hsv HarrisHue Rgb Lab DenseHue Figure: Comparing tensor decomposition with other features 1 on Pascal07 dataset with Gaussian and Polynomial kernel. The decomposition chosen is 3 components with factorization (22,22). 1 Matthieu Guillaumin, Thomas Mensink, Jakob Verbeek, and Cordelia Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation, Antonio Rodríguez-Sánchez (CLEF 2016) 38/41
40 Results on Flickr dataset Gaussian kernel Feature P(%) R(%) F1(%) TD HarrisSiftV3H DenseSift HarrisSift DenseSiftV3H LabV3H HarrisHueV3H DenseHueV3H HsvV3H HarrisHue RgbV3H Lab DenseHue Rgb Hsv Polynomial kernel Feature P(%) R(%) F1(%) TD HarrisSiftV3H DenseSiftV3H HarrisSift DenseSift LabV3H HsvV3H HarrisHueV3H DenseHueV3H RgbV3H Lab DenseHue HarrisHue Hsv Rgb Figure: Comparing tensor decomposition with other features 1 on Flickr dataset with Gaussian and Polynomial kernel. The decomposition chosen is 3 components with factorization (22,22). 1 Matthieu Guillaumin, Thomas Mensink, Jakob Verbeek, and Cordelia Schmid. Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation, Antonio Rodríguez-Sánchez (CLEF 2016) 39/41
41 Conclusions We have presented a method for feature extraction based on decomposition of environment Pro: 1. Compression 2. Automatic prediction of the best filters to use for extracting features Cons: 1. Different decompositions can strong influence the final result 2. Lack of a mechanism for automatically choose the best parameters Antonio Rodríguez-Sánchez (CLEF 2016) 40/41
Joint SVM for Accurate and Fast Image Tagging
his copy is for personal use, the final version will be publisehd in the proceedings of the 22th European Symposium on Artificial Neural Networks, Computation Intelligence and Machine Learning (ESANN 2014)
More informationCS 231A Section 1: Linear Algebra & Probability Review
CS 231A Section 1: Linear Algebra & Probability Review 1 Topics Support Vector Machines Boosting Viola-Jones face detector Linear Algebra Review Notation Operations & Properties Matrix Calculus Probability
More informationCS 231A Section 1: Linear Algebra & Probability Review. Kevin Tang
CS 231A Section 1: Linear Algebra & Probability Review Kevin Tang Kevin Tang Section 1-1 9/30/2011 Topics Support Vector Machines Boosting Viola Jones face detector Linear Algebra Review Notation Operations
More informationFantope Regularization in Metric Learning
Fantope Regularization in Metric Learning CVPR 2014 Marc T. Law (LIP6, UPMC), Nicolas Thome (LIP6 - UPMC Sorbonne Universités), Matthieu Cord (LIP6 - UPMC Sorbonne Universités), Paris, France Introduction
More informationCITS 4402 Computer Vision
CITS 4402 Computer Vision A/Prof Ajmal Mian Adj/A/Prof Mehdi Ravanbakhsh Lecture 06 Object Recognition Objectives To understand the concept of image based object recognition To learn how to match images
More informationGlobal Scene Representations. Tilke Judd
Global Scene Representations Tilke Judd Papers Oliva and Torralba [2001] Fei Fei and Perona [2005] Labzebnik, Schmid and Ponce [2006] Commonalities Goal: Recognize natural scene categories Extract features
More informationScalable, Accurate Image Annotation with Joint SVMs and Output Kernels
Scalable, Accurate Image Annotation with Joint SVMs and Output Kernels Hanchen Xiong 1, Sandor Szedmak, Justus Piater Institute of Computer Science, University of Innsbruck, Technikerstr.21a, A-6020 Innsbruck,
More informationFace Recognition. Face Recognition. Subspace-Based Face Recognition Algorithms. Application of Face Recognition
ace Recognition Identify person based on the appearance of face CSED441:Introduction to Computer Vision (2017) Lecture10: Subspace Methods and ace Recognition Bohyung Han CSE, POSTECH bhhan@postech.ac.kr
More informationMachine Learning. B. Unsupervised Learning B.2 Dimensionality Reduction. Lars Schmidt-Thieme, Nicolas Schilling
Machine Learning B. Unsupervised Learning B.2 Dimensionality Reduction Lars Schmidt-Thieme, Nicolas Schilling Information Systems and Machine Learning Lab (ISMLL) Institute for Computer Science University
More informationLecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed in the
More informationMachine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function.
Bayesian learning: Machine learning comes from Bayesian decision theory in statistics. There we want to minimize the expected value of the loss function. Let y be the true label and y be the predicted
More informationMachine Learning Basics
Security and Fairness of Deep Learning Machine Learning Basics Anupam Datta CMU Spring 2019 Image Classification Image Classification Image classification pipeline Input: A training set of N images, each
More informationLecture: Face Recognition and Feature Reduction
Lecture: Face Recognition and Feature Reduction Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 11-1 Recap - Curse of dimensionality Assume 5000 points uniformly distributed
More informationCS4495/6495 Introduction to Computer Vision. 8C-L3 Support Vector Machines
CS4495/6495 Introduction to Computer Vision 8C-L3 Support Vector Machines Discriminative classifiers Discriminative classifiers find a division (surface) in feature space that separates the classes Several
More informationFisher Vector image representation
Fisher Vector image representation Machine Learning and Category Representation 2014-2015 Jakob Verbeek, January 9, 2015 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.14.15 A brief recap on kernel
More informationStatistical Machine Learning
Statistical Machine Learning Christoph Lampert Spring Semester 2015/2016 // Lecture 12 1 / 36 Unsupervised Learning Dimensionality Reduction 2 / 36 Dimensionality Reduction Given: data X = {x 1,..., x
More informationIntroduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin
1 Introduction to Machine Learning PCA and Spectral Clustering Introduction to Machine Learning, 2013-14 Slides: Eran Halperin Singular Value Decomposition (SVD) The singular value decomposition (SVD)
More informationInformation Retrieval
Introduction to Information CS276: Information and Web Search Christopher Manning and Pandu Nayak Lecture 13: Latent Semantic Indexing Ch. 18 Today s topic Latent Semantic Indexing Term-document matrices
More informationLecture 7: Kernels for Classification and Regression
Lecture 7: Kernels for Classification and Regression CS 194-10, Fall 2011 Laurent El Ghaoui EECS Department UC Berkeley September 15, 2011 Outline Outline A linear regression problem Linear auto-regressive
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationFace detection and recognition. Detection Recognition Sally
Face detection and recognition Detection Recognition Sally Face detection & recognition Viola & Jones detector Available in open CV Face recognition Eigenfaces for face recognition Metric learning identification
More informationLecture 13 Visual recognition
Lecture 13 Visual recognition Announcements Silvio Savarese Lecture 13-20-Feb-14 Lecture 13 Visual recognition Object classification bag of words models Discriminative methods Generative methods Object
More informationTUTORIAL PART 1 Unsupervised Learning
TUTORIAL PART 1 Unsupervised Learning Marc'Aurelio Ranzato Department of Computer Science Univ. of Toronto ranzato@cs.toronto.edu Co-organizers: Honglak Lee, Yoshua Bengio, Geoff Hinton, Yann LeCun, Andrew
More informationLecture 24: Principal Component Analysis. Aykut Erdem May 2016 Hacettepe University
Lecture 4: Principal Component Analysis Aykut Erdem May 016 Hacettepe University This week Motivation PCA algorithms Applications PCA shortcomings Autoencoders Kernel PCA PCA Applications Data Visualization
More informationDiscriminative Direction for Kernel Classifiers
Discriminative Direction for Kernel Classifiers Polina Golland Artificial Intelligence Lab Massachusetts Institute of Technology Cambridge, MA 02139 polina@ai.mit.edu Abstract In many scientific and engineering
More informationIntroduction to Machine Learning. Introduction to ML - TAU 2016/7 1
Introduction to Machine Learning Introduction to ML - TAU 2016/7 1 Course Administration Lecturers: Amir Globerson (gamir@post.tau.ac.il) Yishay Mansour (Mansour@tau.ac.il) Teaching Assistance: Regev Schweiger
More informationImage Analysis & Retrieval. Lec 14. Eigenface and Fisherface
Image Analysis & Retrieval Lec 14 Eigenface and Fisherface Zhu Li Dept of CSEE, UMKC Office: FH560E, Email: lizhu@umkc.edu, Ph: x 2346. http://l.web.umkc.edu/lizhu Z. Li, Image Analysis & Retrv, Spring
More informationOBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES
OBJECT DETECTION AND RECOGNITION IN DIGITAL IMAGES THEORY AND PRACTICE Bogustaw Cyganek AGH University of Science and Technology, Poland WILEY A John Wiley &. Sons, Ltd., Publication Contents Preface Acknowledgements
More informationMachine Learning. Principal Components Analysis. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Machine Learning CSE6740/CS7641/ISYE6740, Fall 2012 Principal Components Analysis Le Song Lecture 22, Nov 13, 2012 Based on slides from Eric Xing, CMU Reading: Chap 12.1, CB book 1 2 Factor or Component
More informationc 4, < y 2, 1 0, otherwise,
Fundamentals of Big Data Analytics Univ.-Prof. Dr. rer. nat. Rudolf Mathar Problem. Probability theory: The outcome of an experiment is described by three events A, B and C. The probabilities Pr(A) =,
More informationChemometrics: Classification of spectra
Chemometrics: Classification of spectra Vladimir Bochko Jarmo Alander University of Vaasa November 1, 2010 Vladimir Bochko Chemometrics: Classification 1/36 Contents Terminology Introduction Big picture
More informationUNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013 Exam policy: This exam allows two one-page, two-sided cheat sheets; No other materials. Time: 2 hours. Be sure to write your name and
More informationMachine learning for pervasive systems Classification in high-dimensional spaces
Machine learning for pervasive systems Classification in high-dimensional spaces Department of Communications and Networking Aalto University, School of Electrical Engineering stephan.sigg@aalto.fi Version
More informationMultiple Similarities Based Kernel Subspace Learning for Image Classification
Multiple Similarities Based Kernel Subspace Learning for Image Classification Wang Yan, Qingshan Liu, Hanqing Lu, and Songde Ma National Laboratory of Pattern Recognition, Institute of Automation, Chinese
More informationLearning to Rank and Quadratic Assignment
Learning to Rank and Quadratic Assignment Thomas Mensink TVPA - XRCE & LEAR - INRIA Grenoble, France Jakob Verbeek LEAR Team INRIA Rhône-Alpes Grenoble, France Abstract Tiberio Caetano Machine Learning
More informationAdvanced Introduction to Machine Learning CMU-10715
Advanced Introduction to Machine Learning CMU-10715 Principal Component Analysis Barnabás Póczos Contents Motivation PCA algorithms Applications Some of these slides are taken from Karl Booksh Research
More informationIntroduction to Support Vector Machines
Introduction to Support Vector Machines Hsuan-Tien Lin Learning Systems Group, California Institute of Technology Talk in NTU EE/CS Speech Lab, November 16, 2005 H.-T. Lin (Learning Systems Group) Introduction
More informationPAC-learning, VC Dimension and Margin-based Bounds
More details: General: http://www.learning-with-kernels.org/ Example of more complex bounds: http://www.research.ibm.com/people/t/tzhang/papers/jmlr02_cover.ps.gz PAC-learning, VC Dimension and Margin-based
More informationLinear Algebra for Machine Learning. Sargur N. Srihari
Linear Algebra for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Overview Linear Algebra is based on continuous math rather than discrete math Computer scientists have little experience with it
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf-Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2018/19 Part 6: Some Other Stuff PD Dr.
More informationPrincipal Component Analysis
B: Chapter 1 HTF: Chapter 1.5 Principal Component Analysis Barnabás Póczos University of Alberta Nov, 009 Contents Motivation PCA algorithms Applications Face recognition Facial expression recognition
More informationProblems. Looks for literal term matches. Problems:
Problems Looks for literal term matches erms in queries (esp short ones) don t always capture user s information need well Problems: Synonymy: other words with the same meaning Car and automobile 电脑 vs.
More informationDiscriminative Learning and Big Data
AIMS-CDT Michaelmas 2016 Discriminative Learning and Big Data Lecture 2: Other loss functions and ANN Andrew Zisserman Visual Geometry Group University of Oxford http://www.robots.ox.ac.uk/~vgg Lecture
More information14 Singular Value Decomposition
14 Singular Value Decomposition For any high-dimensional data analysis, one s first thought should often be: can I use an SVD? The singular value decomposition is an invaluable analysis tool for dealing
More informationMAT 343 Laboratory 6 The SVD decomposition and Image Compression
MA 4 Laboratory 6 he SVD decomposition and Image Compression In this laboratory session we will learn how to Find the SVD decomposition of a matrix using MALAB Use the SVD to perform Image Compression
More informationStyle-aware Mid-level Representation for Discovering Visual Connections in Space and Time
Style-aware Mid-level Representation for Discovering Visual Connections in Space and Time Experiment presentation for CS3710:Visual Recognition Presenter: Zitao Liu University of Pittsburgh ztliu@cs.pitt.edu
More informationFocus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations.
Previously Focus was on solving matrix inversion problems Now we look at other properties of matrices Useful when A represents a transformations y = Ax Or A simply represents data Notion of eigenvectors,
More informationDimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
More information15 Singular Value Decomposition
15 Singular Value Decomposition For any high-dimensional data analysis, one s first thought should often be: can I use an SVD? The singular value decomposition is an invaluable analysis tool for dealing
More informationDimensionality Reduction:
Dimensionality Reduction: From Data Representation to General Framework Dong XU School of Computer Engineering Nanyang Technological University, Singapore What is Dimensionality Reduction? PCA LDA Examples:
More informationThe Singular Value Decomposition (SVD) and Principal Component Analysis (PCA)
Chapter 5 The Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) 5.1 Basics of SVD 5.1.1 Review of Key Concepts We review some key definitions and results about matrices that will
More information26 : Spectral GMs. Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G.
10-708: Probabilistic Graphical Models, Spring 2015 26 : Spectral GMs Lecturer: Eric P. Xing Scribes: Guillermo A Cidre, Abelino Jimenez G. 1 Introduction A common task in machine learning is to work with
More informationImage Analysis & Retrieval Lec 14 - Eigenface & Fisherface
CS/EE 5590 / ENG 401 Special Topics, Spring 2018 Image Analysis & Retrieval Lec 14 - Eigenface & Fisherface Zhu Li Dept of CSEE, UMKC http://l.web.umkc.edu/lizhu Office Hour: Tue/Thr 2:30-4pm@FH560E, Contact:
More informationCPSC 340: Machine Learning and Data Mining. More PCA Fall 2017
CPSC 340: Machine Learning and Data Mining More PCA Fall 2017 Admin Assignment 4: Due Friday of next week. No class Monday due to holiday. There will be tutorials next week on MAP/PCA (except Monday).
More informationLecture Notes 2: Matrices
Optimization-based data analysis Fall 2017 Lecture Notes 2: Matrices Matrices are rectangular arrays of numbers, which are extremely useful for data analysis. They can be interpreted as vectors in a vector
More informationCOMP 551 Applied Machine Learning Lecture 13: Dimension reduction and feature selection
COMP 551 Applied Machine Learning Lecture 13: Dimension reduction and feature selection Instructor: Herke van Hoof (herke.vanhoof@cs.mcgill.ca) Based on slides by:, Jackie Chi Kit Cheung Class web page:
More informationLeast Squares Optimization
Least Squares Optimization The following is a brief review of least squares optimization and constrained optimization techniques. Broadly, these techniques can be used in data analysis and visualization
More informationAn Introduction to Machine Learning
An Introduction to Machine Learning L6: Structured Estimation Alexander J. Smola Statistical Machine Learning Program Canberra, ACT 0200 Australia Alex.Smola@nicta.com.au Tata Institute, Pune, January
More informationLinear Algebra Background
CS76A Text Retrieval and Mining Lecture 5 Recap: Clustering Hierarchical clustering Agglomerative clustering techniques Evaluation Term vs. document space clustering Multi-lingual docs Feature selection
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationSingular Value Decomposition. 1 Singular Value Decomposition and the Four Fundamental Subspaces
Singular Value Decomposition This handout is a review of some basic concepts in linear algebra For a detailed introduction, consult a linear algebra text Linear lgebra and its pplications by Gilbert Strang
More informationTensor Methods for Feature Learning
Tensor Methods for Feature Learning Anima Anandkumar U.C. Irvine Feature Learning For Efficient Classification Find good transformations of input for improved classification Figures used attributed to
More informationSupport Vector Machine. Industrial AI Lab.
Support Vector Machine Industrial AI Lab. Classification (Linear) Autonomously figure out which category (or class) an unknown item should be categorized into Number of categories / classes Binary: 2 different
More informationNearest Neighbors Methods for Support Vector Machines
Nearest Neighbors Methods for Support Vector Machines A. J. Quiroz, Dpto. de Matemáticas. Universidad de Los Andes joint work with María González-Lima, Universidad Simón Boĺıvar and Sergio A. Camelo, Universidad
More informationFrom Binary to Multiclass Classification. CS 6961: Structured Prediction Spring 2018
From Binary to Multiclass Classification CS 6961: Structured Prediction Spring 2018 1 So far: Binary Classification We have seen linear models Learning algorithms Perceptron SVM Logistic Regression Prediction
More informationSingular Value Decomposition and Digital Image Compression
Singular Value Decomposition and Digital Image Compression Chris Bingham December 1, 016 Page 1 of Abstract The purpose of this document is to be a very basic introduction to the singular value decomposition
More informationLarge Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Linear Algebra U Kang Seoul National University U Kang 1 In This Lecture Overview of linear algebra (but, not a comprehensive survey) Focused on the subset
More informationMathematical Formulation of Our Example
Mathematical Formulation of Our Example We define two binary random variables: open and, where is light on or light off. Our question is: What is? Computer Vision 1 Combining Evidence Suppose our robot
More informationExample: Face Detection
Announcements HW1 returned New attendance policy Face Recognition: Dimensionality Reduction On time: 1 point Five minutes or more late: 0.5 points Absent: 0 points Biometrics CSE 190 Lecture 14 CSE190,
More informationKernel Methods and Support Vector Machines
Kernel Methods and Support Vector Machines Oliver Schulte - CMPT 726 Bishop PRML Ch. 6 Support Vector Machines Defining Characteristics Like logistic regression, good for continuous input features, discrete
More informationActive Appearances. Statistical Appearance Models
Active Appearances The material following is based on T.F. Cootes, G.J. Edwards, and C.J. Taylor, Active Appearance Models, Proc. Fifth European Conf. Computer Vision, H. Burkhardt and B. Neumann, eds.,
More informationSingular Value Decompsition
Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost
More informationCS 559: Machine Learning Fundamentals and Applications 5 th Set of Notes
CS 559: Machine Learning Fundamentals and Applications 5 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 25 Project:
More informationSalt Dome Detection and Tracking Using Texture Analysis and Tensor-based Subspace Learning
Salt Dome Detection and Tracking Using Texture Analysis and Tensor-based Subspace Learning Zhen Wang*, Dr. Tamir Hegazy*, Dr. Zhiling Long, and Prof. Ghassan AlRegib 02/18/2015 1 /42 Outline Introduction
More informationBare minimum on matrix algebra. Psychology 588: Covariance structure and factor models
Bare minimum on matrix algebra Psychology 588: Covariance structure and factor models Matrix multiplication 2 Consider three notations for linear combinations y11 y1 m x11 x 1p b11 b 1m y y x x b b n1
More informationImage Analysis. PCA and Eigenfaces
Image Analysis PCA and Eigenfaces Christophoros Nikou cnikou@cs.uoi.gr Images taken from: D. Forsyth and J. Ponce. Computer Vision: A Modern Approach, Prentice Hall, 2003. Computer Vision course by Svetlana
More informationSingular Value Decomposition: Compression of Color Images
1/26 Singular Value Decomposition: Compression of Color Images Bethany Adams and Nina Magnoni Introduction The SVD has very useful applications. It can be used in least squares approximations, search engines,
More informationThe Singular-Value Decomposition
Mathematical Tools for Data Science Spring 2019 1 Motivation The Singular-Value Decomposition The singular-value decomposition (SVD) is a fundamental tool in linear algebra. In this section, we introduce
More informationUncorrelated Multilinear Principal Component Analysis through Successive Variance Maximization
Uncorrelated Multilinear Principal Component Analysis through Successive Variance Maximization Haiping Lu 1 K. N. Plataniotis 1 A. N. Venetsanopoulos 1,2 1 Department of Electrical & Computer Engineering,
More informationWhat is Image Deblurring?
What is Image Deblurring? When we use a camera, we want the recorded image to be a faithful representation of the scene that we see but every image is more or less blurry, depending on the circumstances.
More informationDiscriminative Models
No.5 Discriminative Models Hui Jiang Department of Electrical Engineering and Computer Science Lassonde School of Engineering York University, Toronto, Canada Outline Generative vs. Discriminative models
More informationNeural networks and optimization
Neural networks and optimization Nicolas Le Roux INRIA 8 Nov 2011 Nicolas Le Roux (INRIA) Neural networks and optimization 8 Nov 2011 1 / 80 1 Introduction 2 Linear classifier 3 Convolutional neural networks
More informationCS 3710: Visual Recognition Describing Images with Features. Adriana Kovashka Department of Computer Science January 8, 2015
CS 3710: Visual Recognition Describing Images with Features Adriana Kovashka Department of Computer Science January 8, 2015 Plan for Today Presentation assignments + schedule changes Image filtering Feature
More informationClustering with k-means and Gaussian mixture distributions
Clustering with k-means and Gaussian mixture distributions Machine Learning and Category Representation 2012-2013 Jakob Verbeek, ovember 23, 2012 Course website: http://lear.inrialpes.fr/~verbeek/mlcr.12.13
More informationLecture 8. Principal Component Analysis. Luigi Freda. ALCOR Lab DIAG University of Rome La Sapienza. December 13, 2016
Lecture 8 Principal Component Analysis Luigi Freda ALCOR Lab DIAG University of Rome La Sapienza December 13, 2016 Luigi Freda ( La Sapienza University) Lecture 8 December 13, 2016 1 / 31 Outline 1 Eigen
More informationMatrix decompositions
Matrix decompositions Zdeněk Dvořák May 19, 2015 Lemma 1 (Schur decomposition). If A is a symmetric real matrix, then there exists an orthogonal matrix Q and a diagonal matrix D such that A = QDQ T. The
More informationSystem 1 (last lecture) : limited to rigidly structured shapes. System 2 : recognition of a class of varying shapes. Need to:
System 2 : Modelling & Recognising Modelling and Recognising Classes of Classes of Shapes Shape : PDM & PCA All the same shape? System 1 (last lecture) : limited to rigidly structured shapes System 2 :
More informationSupport Vector Machines (SVM) in bioinformatics. Day 1: Introduction to SVM
1 Support Vector Machines (SVM) in bioinformatics Day 1: Introduction to SVM Jean-Philippe Vert Bioinformatics Center, Kyoto University, Japan Jean-Philippe.Vert@mines.org Human Genome Center, University
More informationBasic Calculus Review
Basic Calculus Review Lorenzo Rosasco ISML Mod. 2 - Machine Learning Vector Spaces Functionals and Operators (Matrices) Vector Space A vector space is a set V with binary operations +: V V V and : R V
More informationClassification and Support Vector Machine
Classification and Support Vector Machine Yiyong Feng and Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) ELEC 5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline
More informationLinear Subspace Models
Linear Subspace Models Goal: Explore linear models of a data set. Motivation: A central question in vision concerns how we represent a collection of data vectors. The data vectors may be rasterized images,
More informationLecture: Face Recognition
Lecture: Face Recognition Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab Lecture 12-1 What we will learn today Introduction to face recognition The Eigenfaces Algorithm Linear
More informationDimensionality Reduction and Principle Components Analysis
Dimensionality Reduction and Principle Components Analysis 1 Outline What is dimensionality reduction? Principle Components Analysis (PCA) Example (Bishop, ch 12) PCA vs linear regression PCA as a mixture
More informationNotes on Latent Semantic Analysis
Notes on Latent Semantic Analysis Costas Boulis 1 Introduction One of the most fundamental problems of information retrieval (IR) is to find all documents (and nothing but those) that are semantically
More informationSparse Kernel Machines - SVM
Sparse Kernel Machines - SVM Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Support
More informationStatistical Pattern Recognition
Statistical Pattern Recognition Feature Extraction Hamid R. Rabiee Jafar Muhammadi, Alireza Ghasemi, Payam Siyari Spring 2014 http://ce.sharif.edu/courses/92-93/2/ce725-2/ Agenda Dimensionality Reduction
More informationMachine Learning - MT & 14. PCA and MDS
Machine Learning - MT 2016 13 & 14. PCA and MDS Varun Kanade University of Oxford November 21 & 23, 2016 Announcements Sheet 4 due this Friday by noon Practical 3 this week (continue next week if necessary)
More informationDimensionality Reduction
Dimensionality Reduction Le Song Machine Learning I CSE 674, Fall 23 Unsupervised learning Learning from raw (unlabeled, unannotated, etc) data, as opposed to supervised data where a classification of
More informationIV. Matrix Approximation using Least-Squares
IV. Matrix Approximation using Least-Squares The SVD and Matrix Approximation We begin with the following fundamental question. Let A be an M N matrix with rank R. What is the closest matrix to A that
More informationPrincipal Component Analysis (PCA)
Principal Component Analysis (PCA) Additional reading can be found from non-assessed exercises (week 8) in this course unit teaching page. Textbooks: Sect. 6.3 in [1] and Ch. 12 in [2] Outline Introduction
More information