Dimension Reduction and Classification Using PCA and Factor. Overview
|
|
- Lionel Ryan
- 6 years ago
- Views:
Transcription
1 Dimension Reduction and Classification Using PCA and - A Short Overview Laboratory for Interdisciplinary Statistical Analysis Department of Statistics Virginia Tech March 2, 2009
2 Outline - Disussions Difference between and
3 The Problem What to do when you have too many predictors in a model? For example you have expression level data for 1000 genes! Or you have customer attributes in hundreds and you are interested in making a predictive model based on customer attributes! Or you have second by second stock market data over a trading day for stocks! Or in survey data where multiple questions might capture the same kind of information (highly correlated)
4 The Cars Outline A researcher wants to build a model to find out which variables are most significant in predicting the demand for cars but believes that a lot of variables have high correlation and the study can be effectively done on a small number of variables without losing much information.
5 The Problem Outline Given a data set with N observations like X = (x 1,..., x p ) for a very large p. Figure: Data with 11 possible predictors
6 The Problem Outline How do we reduce the number of columns in X but still not throw away too much information?
7 The Problem Outline JMP Analyze Multivariate Methods Principal Components Mutlivariate(Tab) Scatterplot Matrix
8 The Problem Notice the highly correlated variables! We will attempt to explain most of the variability in the data, but use a small number of principal components (parsimony) if it is possible.
9 The Geometric Interpretation We intend to come up with rotations and projections in p dimensions that captures most of the variability. Figure: Plot of in three dimensions
10 The Geometric Interpretation - Eigens We can write the principal components as: Y 1 = a 1 X... Y p = a p X such that the Y s are uncorrelated and the variances for each Y is as large as possible. We find out the eigenvalues λ of the data matrix and rank them in terms of their size. The a s are obtained from the corresponding eigenvectors and the eigenvalues correspond to corresponding variances. Since Total population Variance = λ λ p Variance explained by the k th principal component = λ k λ 1 + +λ p
11 The Geometric Interpretation - Eigens Summary Principal components are determined by our predictors There is a principal component for every eigenvalue The value of the eigenvalue gives a measure of much variation the corresponding principal component explains
12 The Geometric Interpretation - Eigens Summary By choosing the first few principal components (and hence eigenvalues) we might be able to explain a lot of the variation among the predictors (not all!) Hence we throw away some information but hopefully not much
13 The Cars We have data about 387 cars with the following variables Suggested Retail Price Invoice price Engine Size (liters) Number of Cylinders (=-1 if rotary engine) Horsepower City Miles Per Gallon Highway Miles Per Gallon Weight (Pounds) Wheel Base (inches) Length (inches) Width (inches)
14 The Cars Again A researcher wants to build a model to find out which variables are most significant in predicting the demand for cars but believes that a lot of variables have high correlation and the study can be effectively done on a small number of variables without losing much information. But how to choose a fewer number of predictors? Analysis!
15 The Cars Outline Use JMP Analyze Multivariate Methods Principal Components
16 The Cars Outline Let us first look at the correlations between the variables. Figure: Correlations
17 The Cars Outline What about the principal components? Can we interpret them? Figure:
18 The Cars Outline How many principal components do we need? How much of the variation is explained?
19 Key Points Principal components are functions of the predictors The first few principal components can give us almost all the information in terms of the variability in the data
20 - Discussion To reduce the number of predictors As a first step for a predictive model where we would like to remove correlated variables General dimension reduction - expecting a low dimensional structure where higher dimensions are basically noise
21 - Disussions Difference between and The Problem Sometimes inherent structure of the data motivates the researcher to group the data based on some unseen underlying factors. This inherent structure can be identified through the correlation matrix of X.
22 - Disussions Difference between and The Subject Scores Problem Consider examination scores in 6 subjects for 220 male students. The 6 subjects are Latin, English, History, Arithmetic, Algebra and Geometry. Consider the correlation matrix for the scores
23 - Disussions Difference between and The Problem The researcher believes that the subject scores will be correlated amongst themselves in groups. A possible hypothesis might be that there are probably two underlying factors for the students scores - a factor that captures the liberal arts scores and another that captures the science scores. But how to verify such a hypothesis?!
24 - Disussions Difference between and Factor Loadings For our problem the researcher thinks that there are two underlying factors. The underlying factors correspond to two different loadings on the 6 subjects. Latin = L 11 F 1 + L 12 F 2 + ɛ 1 English = L 21 F 1 + L 22 F 2 + ɛ 2... Geometry = L 61 F 1 + L 62 F 2 + ɛ 6 The loadings L s will hopefully help us interpret the factors.
25 - Disussions Difference between and The Approach Data has underlying factors researcher determines number of factors factor loadings to be obtained through the covariance matrix researcher interprets factors based on loadings
26 - Disussions Difference between and Factor Loadings for the Subject Scores Variable F 1 F 2 Communalities Latin English History Arithmetic Algebra Geometry The factor loadings do not give us any immediately identifiable groups or factor interpretation. Or DOES it? Communalities give a measure of how much of the variance of the variable is explained by the factor structure.
27 - Disussions Difference between and Factor Loadings Plot Figure: Plot of factor loadings with two factors for the scores example
28 - Disussions Difference between and The Factor Rotation The factors are not immediately identifiable What do we do now? Factor structure in terms of variance explained remains unchanged if we rotate the factors Lets rotate and see if the factor loadings become interpretable
29 - Disussions Difference between and Rotated Factor Loadings for the Subject Scores Variable F 1 F 2 Communalities Latin English History Arithmetic Algebra Geometry Rotation makes the two factors immediately identifiable
30 - Disussions Difference between and Rotated Factor Loadings Plot Figure: Plot of factor loadings with two factors for the scores example
31 - Disussions Difference between and Approach - Summary Decide on number of factors Obtain factor loadings for the variables Interpret factors If interpretation not obvious rotate factors and check loadings again
32 - Disussions Difference between and Psychometrics, Psychology, human factors - identify factors that explain a variety of results on different tests Marketing - Identify the salient attributes consumers use to evaluate products in this category. Physical sciences, geochemistry, ecology, and hydrochemistry
33 - Disussions Difference between and Differences Principal components capture most of the variability in data by using fewer dimensions that where the data exists Hence the principal components lie in the same space as data Factor analysis conceptually tries to search for underlying but unobserved factors that define the correlation in the data Hence factors lie in a different space than the data
34 Richard Johnson, Dean Wishern - Applied Multivariate Statistical Analysis, 5e
Machine Learning. Principal Components Analysis. Le Song. CSE6740/CS7641/ISYE6740, Fall 2012
Machine Learning CSE6740/CS7641/ISYE6740, Fall 2012 Principal Components Analysis Le Song Lecture 22, Nov 13, 2012 Based on slides from Eric Xing, CMU Reading: Chap 12.1, CB book 1 2 Factor or Component
More informationDimensionality Reduction Techniques (DRT)
Dimensionality Reduction Techniques (DRT) Introduction: Sometimes we have lot of variables in the data for analysis which create multidimensional matrix. To simplify calculation and to get appropriate,
More informationChapter 4: Factor Analysis
Chapter 4: Factor Analysis In many studies, we may not be able to measure directly the variables of interest. We can merely collect data on other variables which may be related to the variables of interest.
More informationTAMS39 Lecture 10 Principal Component Analysis Factor Analysis
TAMS39 Lecture 10 Principal Component Analysis Factor Analysis Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content - Lecture Principal component analysis
More informationPRINCIPAL COMPONENTS ANALYSIS
121 CHAPTER 11 PRINCIPAL COMPONENTS ANALYSIS We now have the tools necessary to discuss one of the most important concepts in mathematical statistics: Principal Components Analysis (PCA). PCA involves
More informationPrincipal Component Analysis
Principal Component Analysis Yingyu Liang yliang@cs.wisc.edu Computer Sciences Department University of Wisconsin, Madison [based on slides from Nina Balcan] slide 1 Goals for the lecture you should understand
More informationPrincipal Components Analysis. Sargur Srihari University at Buffalo
Principal Components Analysis Sargur Srihari University at Buffalo 1 Topics Projection Pursuit Methods Principal Components Examples of using PCA Graphical use of PCA Multidimensional Scaling Srihari 2
More informationMultivariate Data Analysis a survey of data reduction and data association techniques: Principal Components Analysis
Multivariate Data Analysis a survey of data reduction and data association techniques: Principal Components Analysis For example Data reduction approaches Cluster analysis Principal components analysis
More informationPrincipal Component Analysis (PCA) Principal Component Analysis (PCA)
Recall: Eigenvectors of the Covariance Matrix Covariance matrices are symmetric. Eigenvectors are orthogonal Eigenvectors are ordered by the magnitude of eigenvalues: λ 1 λ 2 λ p {v 1, v 2,..., v n } Recall:
More informationUCLA STAT 233 Statistical Methods in Biomedical Imaging
UCLA STAT 233 Statistical Methods in Biomedical Imaging Instructor: Ivo Dinov, Asst. Prof. In Statistics and Neurology University of California, Los Angeles, Spring 2004 http://www.stat.ucla.edu/~dinov/
More informationMultidimensional scaling (MDS)
Multidimensional scaling (MDS) Just like SOM and principal curves or surfaces, MDS aims to map data points in R p to a lower-dimensional coordinate system. However, MSD approaches the problem somewhat
More informationApplied Multivariate Analysis
Department of Mathematics and Statistics, University of Vaasa, Finland Spring 2017 Dimension reduction Exploratory (EFA) Background While the motivation in PCA is to replace the original (correlated) variables
More informationVector Space Models. wine_spectral.r
Vector Space Models 137 wine_spectral.r Latent Semantic Analysis Problem with words Even a small vocabulary as in wine example is challenging LSA Reduce number of columns of DTM by principal components
More information12.2 Dimensionality Reduction
510 Chapter 12 of this dimensionality problem, regularization techniques such as SVD are almost always needed to perform the covariance matrix inversion. Because it appears to be a fundamental property
More information2/26/2017. This is similar to canonical correlation in some ways. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2
PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 What is factor analysis? What are factors? Representing factors Graphs and equations Extracting factors Methods and criteria Interpreting
More informationPrincipal Component Analysis -- PCA (also called Karhunen-Loeve transformation)
Principal Component Analysis -- PCA (also called Karhunen-Loeve transformation) PCA transforms the original input space into a lower dimensional space, by constructing dimensions that are linear combinations
More informationMultivariate Fundamentals: Rotation. Exploratory Factor Analysis
Multivariate Fundamentals: Rotation Exploratory Factor Analysis PCA Analysis A Review Precipitation Temperature Ecosystems PCA Analysis with Spatial Data Proportion of variance explained Comp.1 + Comp.2
More informationNotes on Latent Semantic Analysis
Notes on Latent Semantic Analysis Costas Boulis 1 Introduction One of the most fundamental problems of information retrieval (IR) is to find all documents (and nothing but those) that are semantically
More informationPrincipal Components Analysis (PCA)
Principal Components Analysis (PCA) Principal Components Analysis (PCA) a technique for finding patterns in data of high dimension Outline:. Eigenvectors and eigenvalues. PCA: a) Getting the data b) Centering
More information1 A factor can be considered to be an underlying latent variable: (a) on which people differ. (b) that is explained by unknown variables
1 A factor can be considered to be an underlying latent variable: (a) on which people differ (b) that is explained by unknown variables (c) that cannot be defined (d) that is influenced by observed variables
More informationLinear Algebra in a Nutshell: PCA. can be seen as a dimensionality reduction technique Baroni & Evert. Baroni & Evert
What is? a Nutshell: a Nutshell: can be seen as a dimensionality technique to find the inherent underlying dimensions of a Co matrix a Nutshell Dimensions & Marco Baroni & Stefan Evert Co matrix exploits
More informationAP Statistics Unit 2 (Chapters 7-10) Warm-Ups: Part 1
AP Statistics Unit 2 (Chapters 7-10) Warm-Ups: Part 1 2. A researcher is interested in determining if one could predict the score on a statistics exam from the amount of time spent studying for the exam.
More informationIntermediate Social Statistics
Intermediate Social Statistics Lecture 5. Factor Analysis Tom A.B. Snijders University of Oxford January, 2008 c Tom A.B. Snijders (University of Oxford) Intermediate Social Statistics January, 2008 1
More informationQuantitative Understanding in Biology Principal Components Analysis
Quantitative Understanding in Biology Principal Components Analysis Introduction Throughout this course we have seen examples of complex mathematical phenomena being represented as linear combinations
More informationStructure in Data. A major objective in data analysis is to identify interesting features or structure in the data.
Structure in Data A major objective in data analysis is to identify interesting features or structure in the data. The graphical methods are very useful in discovering structure. There are basically two
More informationThird Grade Report Card Rubric 1 Exceeding 2 Meeting 3 Developing 4 Area of Concern
Concepts Assessed by Unit and Trimester Units 5, 6, 7, 8 Units 5, 6, 7 Units 5, 6, 7, 8 1 Exceeding 2 Meeting 3 Developing 4 Area of Concern Student exceeds expectations of this unit Student is meeting
More informationLECTURE 4 PRINCIPAL COMPONENTS ANALYSIS / EXPLORATORY FACTOR ANALYSIS
LECTURE 4 PRINCIPAL COMPONENTS ANALYSIS / EXPLORATORY FACTOR ANALYSIS NOTES FROM PRE- LECTURE RECORDING ON PCA PCA and EFA have similar goals. They are substantially different in important ways. The goal
More informationThe response variable depends on the explanatory variable.
A response variable measures an outcome of study. > dependent variables An explanatory variable attempts to explain the observed outcomes. > independent variables The response variable depends on the explanatory
More informationChapter 3: Examining Relationships
Chapter 3 Review Chapter 3: Examining Relationships 1. A study is conducted to determine if one can predict the yield of a crop based on the amount of yearly rainfall. The response variable in this study
More information********************************************************************************************************
QUESTION # 1 1. Let the random variable X represent the number of telephone lines in use by the technical support center of a software manufacturer at noon each day. The probability distribution of X is
More informationDATA MINING LECTURE 8. Dimensionality Reduction PCA -- SVD
DATA MINING LECTURE 8 Dimensionality Reduction PCA -- SVD The curse of dimensionality Real data usually have thousands, or millions of dimensions E.g., web documents, where the dimensionality is the vocabulary
More informationFeature Transformation
Página 1 de 31 On this page Introduction to Nonnegative Matrix Factorization Principal Component Analysis (PCA) Quality of Life in U.S. Cities Factor Analysis Introduction to Feature transformation is
More information[ z = 1.48 ; accept H 0 ]
CH 13 TESTING OF HYPOTHESIS EXAMPLES Example 13.1 Indicate the type of errors committed in the following cases: (i) H 0 : µ = 500; H 1 : µ 500. H 0 is rejected while H 0 is true (ii) H 0 : µ = 500; H 1
More informationPRINCIPAL COMPONENT ANALYSIS
PRINCIPAL COMPONENT ANALYSIS 1 INTRODUCTION One of the main problems inherent in statistics with more than two variables is the issue of visualising or interpreting data. Fortunately, quite often the problem
More informationMultivariate Statistics
Multivariate Statistics Chapter 4: Factor analysis Pedro Galeano Departamento de Estadística Universidad Carlos III de Madrid pedro.galeano@uc3m.es Course 2017/2018 Master in Mathematical Engineering Pedro
More informationData Mining. Dimensionality reduction. Hamid Beigy. Sharif University of Technology. Fall 1395
Data Mining Dimensionality reduction Hamid Beigy Sharif University of Technology Fall 1395 Hamid Beigy (Sharif University of Technology) Data Mining Fall 1395 1 / 42 Outline 1 Introduction 2 Feature selection
More informationIntroduction to Machine Learning
10-701 Introduction to Machine Learning PCA Slides based on 18-661 Fall 2018 PCA Raw data can be Complex, High-dimensional To understand a phenomenon we measure various related quantities If we knew what
More informationKey Algebraic Results in Linear Regression
Key Algebraic Results in Linear Regression James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) 1 / 30 Key Algebraic Results in
More informationPCA, Kernel PCA, ICA
PCA, Kernel PCA, ICA Learning Representations. Dimensionality Reduction. Maria-Florina Balcan 04/08/2015 Big & High-Dimensional Data High-Dimensions = Lot of Features Document classification Features per
More informationUnconstrained Ordination
Unconstrained Ordination Sites Species A Species B Species C Species D Species E 1 0 (1) 5 (1) 1 (1) 10 (4) 10 (4) 2 2 (3) 8 (3) 4 (3) 12 (6) 20 (6) 3 8 (6) 20 (6) 10 (6) 1 (2) 3 (2) 4 4 (5) 11 (5) 8 (5)
More informationDecember 20, MAA704, Multivariate analysis. Christopher Engström. Multivariate. analysis. Principal component analysis
.. December 20, 2013 Todays lecture. (PCA) (PLS-R) (LDA) . (PCA) is a method often used to reduce the dimension of a large dataset to one of a more manageble size. The new dataset can then be used to make
More informationVAR Model. (k-variate) VAR(p) model (in the Reduced Form): Y t-2. Y t-1 = A + B 1. Y t + B 2. Y t-p. + ε t. + + B p. where:
VAR Model (k-variate VAR(p model (in the Reduced Form: where: Y t = A + B 1 Y t-1 + B 2 Y t-2 + + B p Y t-p + ε t Y t = (y 1t, y 2t,, y kt : a (k x 1 vector of time series variables A: a (k x 1 vector
More informationMethods for sparse analysis of high-dimensional data, II
Methods for sparse analysis of high-dimensional data, II Rachel Ward May 23, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 47 High dimensional
More informationVocabulary: Samples and Populations
Vocabulary: Samples and Populations Concept Different types of data Categorical data results when the question asked in a survey or sample can be answered with a nonnumerical answer. For example if we
More informationEigenvalues, Eigenvectors, and an Intro to PCA
Eigenvalues, Eigenvectors, and an Intro to PCA Eigenvalues, Eigenvectors, and an Intro to PCA Changing Basis We ve talked so far about re-writing our data using a new set of variables, or a new basis.
More informationEigenvalues, Eigenvectors, and an Intro to PCA
Eigenvalues, Eigenvectors, and an Intro to PCA Eigenvalues, Eigenvectors, and an Intro to PCA Changing Basis We ve talked so far about re-writing our data using a new set of variables, or a new basis.
More informationPROCESS MONITORING OF THREE TANK SYSTEM. Outline Introduction Automation system PCA method Process monitoring with T 2 and Q statistics Conclusions
PROCESS MONITORING OF THREE TANK SYSTEM Outline Introduction Automation system PCA method Process monitoring with T 2 and Q statistics Conclusions Introduction Monitoring system for the level and temperature
More informationPrincipal Component Analysis
Principal Component Analysis Anders Øland David Christiansen 1 Introduction Principal Component Analysis, or PCA, is a commonly used multi-purpose technique in data analysis. It can be used for feature
More informationMS-E2112 Multivariate Statistical Analysis (5cr) Lecture 8: Canonical Correlation Analysis
MS-E2112 Multivariate Statistical (5cr) Lecture 8: Contents Canonical correlation analysis involves partition of variables into two vectors x and y. The aim is to find linear combinations α T x and β
More informationLesson 3 - Linear Functions
Lesson 3 - Linear Functions Introduction As an overview for the course, in Lesson's 1 and 2 we discussed the importance of functions to represent relationships and the associated notation of these functions
More informationNew York State Testing Program Grade 8 Common Core Mathematics Test. Released Questions. June 2017
New York State Testing Program Grade 8 Common Core Mathematics Test Released Questions June 2017 New York State administered the Mathematics Common Core Tests in May 2017 and is now making approximately
More informationMultiple Regression. Midterm results: AVG = 26.5 (88%) A = 27+ B = C =
Economics 130 Lecture 6 Midterm Review Next Steps for the Class Multiple Regression Review & Issues Model Specification Issues Launching the Projects!!!!! Midterm results: AVG = 26.5 (88%) A = 27+ B =
More informationDeriving Principal Component Analysis (PCA)
-0 Mathematical Foundations for Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Deriving Principal Component Analysis (PCA) Matt Gormley Lecture 11 Oct.
More informationDimensionality Reduction: PCA. Nicholas Ruozzi University of Texas at Dallas
Dimensionality Reduction: PCA Nicholas Ruozzi University of Texas at Dallas Eigenvalues λ is an eigenvalue of a matrix A R n n if the linear system Ax = λx has at least one non-zero solution If Ax = λx
More informationPrincipal component analysis (PCA) for clustering gene expression data
Principal component analysis (PCA) for clustering gene expression data Ka Yee Yeung Walter L. Ruzzo Bioinformatics, v17 #9 (2001) pp 763-774 1 Outline of talk Background and motivation Design of our empirical
More informationMachine Learning 2nd Edition
INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010
More informationDIMENSION REDUCTION AND CLUSTER ANALYSIS
DIMENSION REDUCTION AND CLUSTER ANALYSIS EECS 833, 6 March 2006 Geoff Bohling Assistant Scientist Kansas Geological Survey geoff@kgs.ku.edu 864-2093 Overheads and resources available at http://people.ku.edu/~gbohling/eecs833
More informationPrincipal Component Analysis (PCA) Theory, Practice, and Examples
Principal Component Analysis (PCA) Theory, Practice, and Examples Data Reduction summarization of data with many (p) variables by a smaller set of (k) derived (synthetic, composite) variables. p k n A
More informationLearning From Data: Modelling as an Optimisation Problem
Learning From Data: Modelling as an Optimisation Problem Iman Shames April 2017 1 / 31 You should be able to... Identify and formulate a regression problem; Appreciate the utility of regularisation; Identify
More informationDimension Reduction Techniques. Presented by Jie (Jerry) Yu
Dimension Reduction Techniques Presented by Jie (Jerry) Yu Outline Problem Modeling Review of PCA and MDS Isomap Local Linear Embedding (LLE) Charting Background Advances in data collection and storage
More informationNew York State Testing Program Grade 8 Common Core Mathematics Test Released Questions June 2017
New York State Testing Program Grade 8 Common Core Mathematics Test Released Questions June 2017 New York State administered the Mathematics Common Core Tests in May 2017 and is now making approximately
More informationIntroduction to Linear Regression
Introduction to Linear Regression James H. Steiger Department of Psychology and Human Development Vanderbilt University James H. Steiger (Vanderbilt University) Introduction to Linear Regression 1 / 46
More informationCS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
More informationComputer exercise 3: PCA, CCA and factors. Principal component analysis. Eigenvalues and eigenvectors
UPPSALA UNIVERSITY Department of Mathematics Måns Thulin Multivariate Methods Spring 2011 thulin@math.uu.se Computer exercise 3: PCA, CCA and factors In this computer exercise the following topics are
More informationAnnouncements (repeat) Principal Components Analysis
4/7/7 Announcements repeat Principal Components Analysis CS 5 Lecture #9 April 4 th, 7 PA4 is due Monday, April 7 th Test # will be Wednesday, April 9 th Test #3 is Monday, May 8 th at 8AM Just hour long
More informationPrincipal component analysis
Principal component analysis Motivation i for PCA came from major-axis regression. Strong assumption: single homogeneous sample. Free of assumptions when used for exploration. Classical tests of significance
More information1 Principal Components Analysis
Lecture 3 and 4 Sept. 18 and Sept.20-2006 Data Visualization STAT 442 / 890, CM 462 Lecture: Ali Ghodsi 1 Principal Components Analysis Principal components analysis (PCA) is a very popular technique for
More informationExploratory Factor Analysis and Principal Component Analysis
Exploratory Factor Analysis and Principal Component Analysis Today s Topics: What are EFA and PCA for? Planning a factor analytic study Analysis steps: Extraction methods How many factors Rotation and
More informationPrincipal Components Theory Notes
Principal Components Theory Notes Charles J. Geyer August 29, 2007 1 Introduction These are class notes for Stat 5601 (nonparametrics) taught at the University of Minnesota, Spring 2006. This not a theory
More informationPrincipal component analysis, PCA
CHEM-E3205 Bioprocess Optimization and Simulation Principal component analysis, PCA Tero Eerikäinen Room D416d tero.eerikainen@aalto.fi Data Process or system measurements New information from the gathered
More informationEigenfaces. Face Recognition Using Principal Components Analysis
Eigenfaces Face Recognition Using Principal Components Analysis M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive Neuroscience, 3(1), pp. 71-86, 1991. Slides : George Bebis, UNR
More informationRegression Models REVISED TEACHING SUGGESTIONS ALTERNATIVE EXAMPLES
M04_REND6289_10_IM_C04.QXD 5/7/08 2:49 PM Page 46 4 C H A P T E R Regression Models TEACHING SUGGESTIONS Teaching Suggestion 4.1: Which Is the Independent Variable? We find that students are often confused
More informationPollution Sources Detection via Principal Component Analysis and Rotation
Pollution Sources Detection via Principal Component Analysis and Rotation Vanessa Kuentz 1 in collaboration with : Marie Chavent 1 Hervé Guégan 2 Brigitte Patouille 1 Jérôme Saracco 1,3 1 IMB, Université
More informationWhat is Principal Component Analysis?
What is Principal Component Analysis? Principal component analysis (PCA) Reduce the dimensionality of a data set by finding a new set of variables, smaller than the original set of variables Retains most
More informationPrincipal Component Analysis-I Geog 210C Introduction to Spatial Data Analysis. Chris Funk. Lecture 17
Principal Component Analysis-I Geog 210C Introduction to Spatial Data Analysis Chris Funk Lecture 17 Outline Filters and Rotations Generating co-varying random fields Translating co-varying fields into
More informationCollege Algebra. Word Problems
College Algebra Word Problems Example 2 (Section P6) The table shows the numbers N (in millions) of subscribers to a cellular telecommunication service in the United States from 2001 through 2010, where
More information1 Correlation and Inference from Regression
1 Correlation and Inference from Regression Reading: Kennedy (1998) A Guide to Econometrics, Chapters 4 and 6 Maddala, G.S. (1992) Introduction to Econometrics p. 170-177 Moore and McCabe, chapter 12 is
More informationPrincipal components
Principal components Principal components is a general analysis technique that has some application within regression, but has a much wider use as well. Technical Stuff We have yet to define the term covariance,
More informationPractice A. Name Date. Evaluate the expression for the given value of the variable. Match the equation with its solution. Solve the equation.
mm mm OFF ON ZERO Name Date Practice A For use with pages 390 395 Evaluate the expression for the given value of the variable.. 2 x 2 3 4; 25 2. 3 2x 8 2 0; 3 3. 24 x 2 9 ; 26 Match the equation with its
More informationTable of Contents. Multivariate methods. Introduction II. Introduction I
Table of Contents Introduction Antti Penttilä Department of Physics University of Helsinki Exactum summer school, 04 Construction of multinormal distribution Test of multinormality with 3 Interpretation
More informationQuantitative Understanding in Biology Short Course Session 9 Principal Components Analysis
Quantitative Understanding in Biology Short Course Session 9 Principal Components Analysis Jinhyun Ju Jason Banfelder Luce Skrabanek June 21st, 218 1 Preface For the last session in this course, we ll
More informationHW Unit 7: Connections (Graphs, Equations and Inequalities)
Math Fundamentals for Statistics I (Math 5) HW Unit 7: Connections (Graphs, Equations and Inequalities) By Scott Fallstrom and Brent Pickett The How and Whys Guys This work is licensed under a Creative
More informationBasics of Multivariate Modelling and Data Analysis
Basics of Multivariate Modelling and Data Analysis Kurt-Erik Häggblom 2. Overview of multivariate techniques 2.1 Different approaches to multivariate data analysis 2.2 Classification of multivariate techniques
More informationAP Statistics - Chapter 2A Extra Practice
AP Statistics - Chapter 2A Extra Practice 1. A study is conducted to determine if one can predict the yield of a crop based on the amount of yearly rainfall. The response variable in this study is A) yield
More informationExploratory Factor Analysis and Principal Component Analysis
Exploratory Factor Analysis and Principal Component Analysis Today s Topics: What are EFA and PCA for? Planning a factor analytic study Analysis steps: Extraction methods How many factors Rotation and
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Introduction Edps/Psych/Stat/ 584 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c Board of Trustees,
More informationLecture 02 Linear Algebra Basics
Introduction to Computational Data Analysis CX4240, 2019 Spring Lecture 02 Linear Algebra Basics Chao Zhang College of Computing Georgia Tech These slides are based on slides from Le Song and Andres Mendez-Vazquez.
More information3.1. The probabilistic view of the principal component analysis.
301 Chapter 3 Principal Components and Statistical Factor Models This chapter of introduces the principal component analysis (PCA), briefly reviews statistical factor models PCA is among the most popular
More informationPractice Questions for Exam 1
Practice Questions for Exam 1 1. A used car lot evaluates their cars on a number of features as they arrive in the lot in order to determine their worth. Among the features looked at are miles per gallon
More informationPREPARING FOR THE CLAST MATHEMATICS Ignacio Bello
PREPARING FOR THE CLAST MATHEMATICS Ignacio Bello CLAST T.V. PRACTICE TEST Page. Find: + 8. Find: ( ) 9 4. Find: (6 )( ) (6 x 4)( x ) ( 6 + 6 + 6 + 6)( + + + + ) 0 (6 x ) (6 x 6 x 6 x 6)( x x x x ) 4.
More information-Principal components analysis is by far the oldest multivariate technique, dating back to the early 1900's; ecologists have used PCA since the
1 2 3 -Principal components analysis is by far the oldest multivariate technique, dating back to the early 1900's; ecologists have used PCA since the 1950's. -PCA is based on covariance or correlation
More informationRegression: Ordinary Least Squares
Regression: Ordinary Least Squares Mark Hendricks Autumn 2017 FINM Intro: Regression Outline Regression OLS Mathematics Linear Projection Hendricks, Autumn 2017 FINM Intro: Regression: Lecture 2/32 Regression
More informationPrinciple Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA
Principle Components Analysis (PCA) Relationship Between a Linear Combination of Variables and Axes Rotation for PCA Principle Components Analysis: Uses one group of variables (we will call this X) In
More informationApplied Regression Modeling: A Business Approach Chapter 3: Multiple Linear Regression Sections
Applied Regression Modeling: A Business Approach Chapter 3: Multiple Linear Regression Sections 3.1 3.3.2 by Iain Pardoe 3.1 Probability model for (X 1, X 2,...) and Y 2 Multiple linear regression................................................
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 4 for Applied Multivariate Analysis Outline 1 Eigen values and eigen vectors Characteristic equation Some properties of eigendecompositions
More informationStatistics: A review. Why statistics?
Statistics: A review Why statistics? What statistical concepts should we know? Why statistics? To summarize, to explore, to look for relations, to predict What kinds of data exist? Nominal, Ordinal, Interval
More informationPrincipal Component Analysis
I.T. Jolliffe Principal Component Analysis Second Edition With 28 Illustrations Springer Contents Preface to the Second Edition Preface to the First Edition Acknowledgments List of Figures List of Tables
More informationSem. 1 Review Ch. 1-3
AP Stats Sem. 1 Review Ch. 1-3 Name 1. You measure the age, marital status and earned income of an SRS of 1463 women. The number and type of variables you have measured is a. 1463; all quantitative. b.
More informationMultilevel Analysis, with Extensions
May 26, 2010 We start by reviewing the research on multilevel analysis that has been done in psychometrics and educational statistics, roughly since 1985. The canonical reference (at least I hope so) is
More information2011 Pearson Education, Inc
Statistics for Business and Economics Chapter 7 Inferences Based on Two Samples: Confidence Intervals & Tests of Hypotheses Content 1. Identifying the Target Parameter 2. Comparing Two Population Means:
More information