Contingency Table Analysis via Matrix Factorization
|
|
|
- Austen Fields
- 5 years ago
- Views:
Transcription
1 Contingency Table Analysis via Matrix Factorization Kumer Pial Das 1, Jay Powell 2, Myron Katzoff 3, S. Stanley Young 4 1 Department of Mathematics,Lamar University, TX 2 Better Schooling Systems, Pittsburgh, PA 3 Center for Disease Control-Retired 4 National Institute of Statistical Sciences May 10, 2013 SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 1
2 Outline Introduction 1 Introduction SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 2
3 One of the goals of the OCER working group "Singular value decompositions can be used to simultaneously cluster the rows (patients) and columns (variables) of a two-way table. New research indicates that non-negative matrix factorization, NMF, has distinct advantages in the identification of subgroups with distinct etiologies, Lee and Seung (1999). Scores computed via NMF might be better clustering variables for Local Control analysis." SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 3
4 Introduction Contingency tables of numeric data are often analyzed using dimension reduction methods like the singular value decomposition (SVD), and principal component analysis (PCA). This analysis produces score and loading matrices representing the rows and the columns of the original table and these matrices may be used for both prediction purposes and to gain structural understanding of the data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 4
5 Introduction Contingency tables of numeric data are often analyzed using dimension reduction methods like the singular value decomposition (SVD), and principal component analysis (PCA). This analysis produces score and loading matrices representing the rows and the columns of the original table and these matrices may be used for both prediction purposes and to gain structural understanding of the data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 4
6 Extracting factors from a contingency table IJ Good (1969) proposed using SVD to extract factors from a contingency table. We think Non-negative Matrix Factorization (NMF) will likely produce better results and interpretation. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 5
7 Extracting factors from a contingency table IJ Good (1969) proposed using SVD to extract factors from a contingency table. We think Non-negative Matrix Factorization (NMF) will likely produce better results and interpretation. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 5
8 Non-negative Matrix Factorization In some tables, the data entries are necessarily non-negative, and so the matrix factors meant to represent them should arguably also contain only non-negative elements. We describe here the use of NMF, an algorithm based on decomposition by parts that can reduce the dimension of data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 6
9 Non-negative Matrix Factorization In some tables, the data entries are necessarily non-negative, and so the matrix factors meant to represent them should arguably also contain only non-negative elements. We describe here the use of NMF, an algorithm based on decomposition by parts that can reduce the dimension of data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 6
10 Non-negative Matrix Factorization Let A be a n p non-negative matrix, and k > 0 an integer. NMF consists in finding an approximation, A WH where W, H are n k and k p non-negative matrices, respectively. Columns of W are the underlying basis vectors. Columns of H give the weights associated with each basis vector. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 7
11 Non-negative Matrix Factorization Let A be a n p non-negative matrix, and k > 0 an integer. NMF consists in finding an approximation, A WH where W, H are n k and k p non-negative matrices, respectively. Columns of W are the underlying basis vectors. Columns of H give the weights associated with each basis vector. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 7
12 Non-negative Matrix Factorization Let A be a n p non-negative matrix, and k > 0 an integer. NMF consists in finding an approximation, A WH where W, H are n k and k p non-negative matrices, respectively. Columns of W are the underlying basis vectors. Columns of H give the weights associated with each basis vector. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 7
13 In practice, the factorization rank k is often chosen such that k << min(n, p). In general, k can be bounded as (n + p)k < np. The objective behind this choice is to summarize and split the information contained in A into k factors: the columns of W. Depending on the application field, these factors are given different names:basis images, metagenes, source signals. We ll be using the term source signals in this presentation. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 8
14 In practice, the factorization rank k is often chosen such that k << min(n, p). In general, k can be bounded as (n + p)k < np. The objective behind this choice is to summarize and split the information contained in A into k factors: the columns of W. Depending on the application field, these factors are given different names:basis images, metagenes, source signals. We ll be using the term source signals in this presentation. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 8
15 Objective Introduction We study the use of PCA, SVD, and NMF to reduce the dimensionality of count data presented in a contingency table. Our primary goal is to remove noise and uncertainty. In theory, NMF can also be used for better interpretation of factoring matrices. As the rank k increases the method uncovers substructures, whose robustness can be evaluated by a cophenetic correlation coefficient. These substructures may also give evidence of nesting subtypes. Thus, NMF can reveal hierarchical structure when it exists but does not force such structure on the data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 9
16 Objective Introduction We study the use of PCA, SVD, and NMF to reduce the dimensionality of count data presented in a contingency table. Our primary goal is to remove noise and uncertainty. In theory, NMF can also be used for better interpretation of factoring matrices. As the rank k increases the method uncovers substructures, whose robustness can be evaluated by a cophenetic correlation coefficient. These substructures may also give evidence of nesting subtypes. Thus, NMF can reveal hierarchical structure when it exists but does not force such structure on the data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 9
17 Cophenetic Correlation Coefficient The cophenetic correlation coefficient is based on the consensus matrix (i.e. the average of connectivity matrices) and was proposed by Brunet et al. (2004) to measure the stability of the clusters obtained from NMF. It is defined as the Pearson correlation between the samples distances induced by the consensus matrix, seen as a similarity matrix and their cophenetic distances from a hierarchical clustering based on these very distances, by default an average linkage is used. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 10
18 Cophenetic Correlation Coefficient The cophenetic correlation coefficient is based on the consensus matrix (i.e. the average of connectivity matrices) and was proposed by Brunet et al. (2004) to measure the stability of the clusters obtained from NMF. It is defined as the Pearson correlation between the samples distances induced by the consensus matrix, seen as a similarity matrix and their cophenetic distances from a hierarchical clustering based on these very distances, by default an average linkage is used. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 10
19 We may consider a data set consisting of the expression levels of M genes in N samples. For gene expression studies, M is typically in the thousands, and N is typically <100 The data can be represented by an expression matrix A of size M N. One possible goal could be to find a small number of metagenes, each defined as a positive linear combination of the M genes. We can then approximate the gene expression pattern of samples as positive linear combinations of these metagenes. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 11
20 We may consider a data set consisting of the expression levels of M genes in N samples. For gene expression studies, M is typically in the thousands, and N is typically <100 The data can be represented by an expression matrix A of size M N. One possible goal could be to find a small number of metagenes, each defined as a positive linear combination of the M genes. We can then approximate the gene expression pattern of samples as positive linear combinations of these metagenes. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 11
21 We may consider a data set consisting of the expression levels of M genes in N samples. For gene expression studies, M is typically in the thousands, and N is typically <100 The data can be represented by an expression matrix A of size M N. One possible goal could be to find a small number of metagenes, each defined as a positive linear combination of the M genes. We can then approximate the gene expression pattern of samples as positive linear combinations of these metagenes. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 11
22 National Institute on Aging (NIA) and NIH are conducting research by collecting data from 29 NIA funded Alzheimer s Disease Centers. There are approximately 25,000 subjects. The data set includes more than 700 variables, representing demographics, behavioral status, cognitive testing, medical history, family history, clinical impressions, and diagnoses. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 12
23 National Institute on Aging (NIA) and NIH are conducting research by collecting data from 29 NIA funded Alzheimer s Disease Centers. There are approximately 25,000 subjects. The data set includes more than 700 variables, representing demographics, behavioral status, cognitive testing, medical history, family history, clinical impressions, and diagnoses. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 12
24 National Institute on Aging (NIA) and NIH are conducting research by collecting data from 29 NIA funded Alzheimer s Disease Centers. There are approximately 25,000 subjects. The data set includes more than 700 variables, representing demographics, behavioral status, cognitive testing, medical history, family history, clinical impressions, and diagnoses. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 12
25 2013 Geoffrey Beene Global NeuroDiscovery Challenge The Foundation for the National Institute of Health (FNIH), in association with the Geoffrey Beene Foundation Alzheimer s initiative, is conducting a study to identify male/female differences in the early cognitive decline that precedes Alzheimer s disease. The 2013 Geoffrey Beene Global NeuroDiscovery Challenge calls for original and innovative hypotheses on gender differences in pathology and neuro degenerative decline leading to Alzheimer s disease. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 13
26 2013 Geoffrey Beene Global NeuroDiscovery Challenge The Foundation for the National Institute of Health (FNIH), in association with the Geoffrey Beene Foundation Alzheimer s initiative, is conducting a study to identify male/female differences in the early cognitive decline that precedes Alzheimer s disease. The 2013 Geoffrey Beene Global NeuroDiscovery Challenge calls for original and innovative hypotheses on gender differences in pathology and neuro degenerative decline leading to Alzheimer s disease. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 13
27 2013 Geoffrey Beene Global NeuroDiscovery Challenge The winning submission(s) will receive $100,000 in prize awards. For more information you may visit SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 14
28 2013 Geoffrey Beene Global NeuroDiscovery Challenge The winning submission(s) will receive $100,000 in prize awards. For more information you may visit SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 14
29 Description of the data set J. Powell and his colleagues designed various education studies. Most of these studies originated with an observation made in the early 1960 s wherein some "wrong" answers given to multiple tests appeared to reflect systematic selection by test subjects. One of these studies has used a multiple choice test (known as "the Proverbs Test") contains 40 items, each with 4 alternatives. The test was given twice in each year. We have added another alternative for missing data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 15
30 Description of the data set J. Powell and his colleagues designed various education studies. Most of these studies originated with an observation made in the early 1960 s wherein some "wrong" answers given to multiple tests appeared to reflect systematic selection by test subjects. One of these studies has used a multiple choice test (known as "the Proverbs Test") contains 40 items, each with 4 alternatives. The test was given twice in each year. We have added another alternative for missing data. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 15
31 Netherlandish Proverbs by P. Bruegel the Elder, 1559 SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 16
32 Q32: Don t cast pearls before swine 1 Put your efforts where they are appreciated 2 Don t give pearl to fools 3 Don t be wasteful 4 Don t always put yourself before everybody SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 17
33 Age Group SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 18
34 All 40 questions The Windsor education data is represented by an expresssion matrix A of size whose rows contain 2,293 students and coulmn contains 160 variables (40 questions with four alternatives). Matrix W has size 2293 k, with each of the k columns defining a source signal. Matrix H has size k 160, with each of the 160 columns representing a source signal expression pattern of the corresponding sample. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 19
35 All 40 questions The Windsor education data is represented by an expresssion matrix A of size whose rows contain 2,293 students and coulmn contains 160 variables (40 questions with four alternatives). Matrix W has size 2293 k, with each of the k columns defining a source signal. Matrix H has size k 160, with each of the 160 columns representing a source signal expression pattern of the corresponding sample. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 19
36 All 40 questions The Windsor education data is represented by an expresssion matrix A of size whose rows contain 2,293 students and coulmn contains 160 variables (40 questions with four alternatives). Matrix W has size 2293 k, with each of the k columns defining a source signal. Matrix H has size k 160, with each of the 160 columns representing a source signal expression pattern of the corresponding sample. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 19
37 Estimating the Factorization Rank A critical parameter in NMF is the factorization rank k. It defines the number of source signals used to approximate the target matrix. Given a NMF method and the trgaet matrix, a common approach of deciding on k is to try different values, compute some quality measure of the results, and choose the best value according to this quality criteria. We use both R and JMP to estimate the value of k. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 20
38 Estimating the Factorization Rank A critical parameter in NMF is the factorization rank k. It defines the number of source signals used to approximate the target matrix. Given a NMF method and the trgaet matrix, a common approach of deciding on k is to try different values, compute some quality measure of the results, and choose the best value according to this quality criteria. We use both R and JMP to estimate the value of k. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 20
39 Estimating the Factorization Rank A critical parameter in NMF is the factorization rank k. It defines the number of source signals used to approximate the target matrix. Given a NMF method and the trgaet matrix, a common approach of deciding on k is to try different values, compute some quality measure of the results, and choose the best value according to this quality criteria. We use both R and JMP to estimate the value of k. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 20
40 Interpreting the consensus matrix The consensus matrices are computed at k = 2 to 10 for the Windsor education data. Samples are hierarchically clustered by using distances derived from consensus clustering matrix entries, colored from 0(deep blue, samples are never in the same cluster) to 1 (dark red, samples are always in the same cluster). SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 21
41 Interpreting the consensus matrix The consensus matrices are computed at k = 2 to 10 for the Windsor education data. Samples are hierarchically clustered by using distances derived from consensus clustering matrix entries, colored from 0(deep blue, samples are never in the same cluster) to 1 (dark red, samples are always in the same cluster). SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 21
42 Consensus Map of all questions for rank 2 SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 22
43 Consensus Map of all questions for rank 3 SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 23
44 Consensus Map of all questions for rank 4 SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 24
45 Consensus Map of all questions for rank 2-10 SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 25
46 Cophenetic Correlation Coefficeint Although a visual inspection is important, it s also important to have quantitative measure of stability for each value of k. One measure proposed by (Brunet, 2004) is cophenetic coefficient, which indicates the dispersion of the consensus matrix, defined as the average connectivity matrix over many clustering runs. We observe how cophenetic coefficient changes as k increases. We select values of k where the magnitude of cophenetic coefficient begins to fall. (Hutchins et al. 2008) suggested to choose the first value where the RSS curve presents an inflection point. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 26
47 Cophenetic Correlation Coefficeint Although a visual inspection is important, it s also important to have quantitative measure of stability for each value of k. One measure proposed by (Brunet, 2004) is cophenetic coefficient, which indicates the dispersion of the consensus matrix, defined as the average connectivity matrix over many clustering runs. We observe how cophenetic coefficient changes as k increases. We select values of k where the magnitude of cophenetic coefficient begins to fall. (Hutchins et al. 2008) suggested to choose the first value where the RSS curve presents an inflection point. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 26
48 Cophenetic Correlation Coefficeint Although a visual inspection is important, it s also important to have quantitative measure of stability for each value of k. One measure proposed by (Brunet, 2004) is cophenetic coefficient, which indicates the dispersion of the consensus matrix, defined as the average connectivity matrix over many clustering runs. We observe how cophenetic coefficient changes as k increases. We select values of k where the magnitude of cophenetic coefficient begins to fall. (Hutchins et al. 2008) suggested to choose the first value where the RSS curve presents an inflection point. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 26
49 Estimating k Introduction SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 27
50 Estimating k Introduction SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 28
51 NMF is looking at all answer columns without paying attention to the natural grouping of 4 answers for each question. NMF does not "know" the correct answer. NMF is "unsupervised". NMF is looking at how students of different ages answer proverb questions. It is looking at the shape of the age response curve. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 29
52 NMF is looking at all answer columns without paying attention to the natural grouping of 4 answers for each question. NMF does not "know" the correct answer. NMF is "unsupervised". NMF is looking at how students of different ages answer proverb questions. It is looking at the shape of the age response curve. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 29
53 χ 2 partitioning We group this data by the usual χ 2 partitioning approach. But we have found that the problem of dimension reduction remains. We believe this is where NMF should be useful. Ideally, the factors that distinguish the members of the ultimate groups are the same but why must that be the case? Some factors may be active in one group that are not actors in another; but what are they? We think NMF answers that question. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 30
54 χ 2 partitioning We group this data by the usual χ 2 partitioning approach. But we have found that the problem of dimension reduction remains. We believe this is where NMF should be useful. Ideally, the factors that distinguish the members of the ultimate groups are the same but why must that be the case? Some factors may be active in one group that are not actors in another; but what are they? We think NMF answers that question. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 30
55 χ 2 partitioning We group this data by the usual χ 2 partitioning approach. But we have found that the problem of dimension reduction remains. We believe this is where NMF should be useful. Ideally, the factors that distinguish the members of the ultimate groups are the same but why must that be the case? Some factors may be active in one group that are not actors in another; but what are they? We think NMF answers that question. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 30
56 What s after PCA/SVD/NMF? A possible way forward is to use NMF to get response variables for recursive partitioning or local control approach. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 31
57 What s after PCA/SVD/NMF? A possible way forward is to use NMF to get response variables for recursive partitioning or local control approach. SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 31
58 References Introduction Brunet, J.P., Tamayo, P., Gould, T.R., and Mesirov, J.P.(2004) Metagenes and Molecular Pattern Discovery using Matrix Factorization, Proceedings of the National Academy of Sciences. 101, Good, I.J.(1969) Some Applications of the Singular Decomposition of a Matrix, Technometrics. 11(4), Lee, D.D., and Seung, H.S.(1999) Algorithms for Non-negative Matrix Factorization, Nature. 401, SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 32
59 Thank You SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 33
60 Thank You Questions SAMSI Transitional Workshop Contingency Table Analysis via Matrix Factorization 33
COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017
COMS 4721: Machine Learning for Data Science Lecture 18, 4/4/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University TOPIC MODELING MODELS FOR TEXT DATA
Machine Learning (BSMC-GA 4439) Wenke Liu
Machine Learning (BSMC-GA 4439) Wenke Liu 02-01-2018 Biomedical data are usually high-dimensional Number of samples (n) is relatively small whereas number of features (p) can be large Sometimes p>>n Problems
EUSIPCO
EUSIPCO 2013 1569741067 CLUSERING BY NON-NEGAIVE MARIX FACORIZAION WIH INDEPENDEN PRINCIPAL COMPONEN INIIALIZAION Liyun Gong 1, Asoke K. Nandi 2,3 1 Department of Electrical Engineering and Electronics,
Dimension Reduction and Iterative Consensus Clustering
Dimension Reduction and Iterative Consensus Clustering Southeastern Clustering and Ranking Workshop August 24, 2009 Dimension Reduction and Iterative 1 Document Clustering Geometry of the SVD Centered
Introduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin
1 Introduction to Machine Learning PCA and Spectral Clustering Introduction to Machine Learning, 2013-14 Slides: Eran Halperin Singular Value Decomposition (SVD) The singular value decomposition (SVD)
Consensus NMF: A unified approach to two-sided testing of micro array data
Consensus NMF: A unified approach to two-sided testing of micro array data Paul Fogel, Consultant, Paris, France S. Stanley Young, National Institute of Statistical Sciences JMP Users Group October 11-14
Non-Negative Factorization for Clustering of Microarray Data
INT J COMPUT COMMUN, ISSN 1841-9836 9(1):16-23, February, 2014. Non-Negative Factorization for Clustering of Microarray Data L. Morgos Lucian Morgos Dept. of Electronics and Telecommunications Faculty
AN ENHANCED INITIALIZATION METHOD FOR NON-NEGATIVE MATRIX FACTORIZATION. Liyun Gong 1, Asoke K. Nandi 2,3 L69 3BX, UK; 3PH, UK;
213 IEEE INERNAIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEP. 22 25, 213, SOUHAMPON, UK AN ENHANCED INIIALIZAION MEHOD FOR NON-NEGAIVE MARIX FACORIZAION Liyun Gong 1, Asoke K. Nandi 2,3
Statistics 202: Data Mining. c Jonathan Taylor. Week 2 Based in part on slides from textbook, slides of Susan Holmes. October 3, / 1
Week 2 Based in part on slides from textbook, slides of Susan Holmes October 3, 2012 1 / 1 Part I Other datatypes, preprocessing 2 / 1 Other datatypes Document data You might start with a collection of
Part I. Other datatypes, preprocessing. Other datatypes. Other datatypes. Week 2 Based in part on slides from textbook, slides of Susan Holmes
Week 2 Based in part on slides from textbook, slides of Susan Holmes Part I Other datatypes, preprocessing October 3, 2012 1 / 1 2 / 1 Other datatypes Other datatypes Document data You might start with
CS570 Introduction to Data Mining
CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong Data Exploration and Data Preprocessing Data and Attributes Data exploration Data pre-processing Data cleaning
Clustering. Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein. Some slides adapted from Jacques van Helden
Clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein Some slides adapted from Jacques van Helden Gene expression profiling A quick review Which molecular processes/functions
Dimension reduction, PCA & eigenanalysis Based in part on slides from textbook, slides of Susan Holmes. October 3, Statistics 202: Data Mining
Dimension reduction, PCA & eigenanalysis Based in part on slides from textbook, slides of Susan Holmes October 3, 2012 1 / 1 Combinations of features Given a data matrix X n p with p fairly large, it can
Lecture 13. Principal Component Analysis. Brett Bernstein. April 25, CDS at NYU. Brett Bernstein (CDS at NYU) Lecture 13 April 25, / 26
Principal Component Analysis Brett Bernstein CDS at NYU April 25, 2017 Brett Bernstein (CDS at NYU) Lecture 13 April 25, 2017 1 / 26 Initial Question Intro Question Question Let S R n n be symmetric. 1
Grouping of correlated feature vectors using treelets
Grouping of correlated feature vectors using treelets Jing Xiang Department of Machine Learning Carnegie Mellon University Pittsburgh, PA 15213 jingx@cs.cmu.edu Abstract In many applications, features
Data Mining and Matrices
Data Mining and Matrices 05 Semi-Discrete Decomposition Rainer Gemulla, Pauli Miettinen May 16, 2013 Outline 1 Hunting the Bump 2 Semi-Discrete Decomposition 3 The Algorithm 4 Applications SDD alone SVD
Recommender Systems. Dipanjan Das Language Technologies Institute Carnegie Mellon University. 20 November, 2007
Recommender Systems Dipanjan Das Language Technologies Institute Carnegie Mellon University 20 November, 2007 Today s Outline What are Recommender Systems? Two approaches Content Based Methods Collaborative
Non-Negative Matrix Factorization
Chapter 3 Non-Negative Matrix Factorization Part : Introduction & computation Motivating NMF Skillicorn chapter 8; Berry et al. (27) DMM, summer 27 2 Reminder A T U Σ V T T Σ, V U 2 Σ 2,2 V 2.8.6.6.3.6.5.3.6.3.6.4.3.6.4.3.3.4.5.3.5.8.3.8.3.3.5
Nonnegative Matrix Factorization
Nonnegative Matrix Factorization Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
Matrix Factorization & Latent Semantic Analysis Review. Yize Li, Lanbo Zhang
Matrix Factorization & Latent Semantic Analysis Review Yize Li, Lanbo Zhang Overview SVD in Latent Semantic Indexing Non-negative Matrix Factorization Probabilistic Latent Semantic Indexing Vector Space
Clustering. Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein. Some slides adapted from Jacques van Helden
Clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein Some slides adapted from Jacques van Helden Gene expression profiling A quick review Which molecular processes/functions
A Hierarchical Perspective on Lee-Carter Models
A Hierarchical Perspective on Lee-Carter Models Paul Eilers Leiden University Medical Centre L-C Workshop, Edinburgh 24 The vantage point Previous presentation: Iain Currie led you upward From Glen Gumbel
Ordinal Variables in 2 way Tables
Ordinal Variables in 2 way Tables Edps/Psych/Soc 589 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2018 C.J. Anderson (Illinois) Ordinal Variables
GAUSS-SEIDEL BASED NON-NEGATIVE MATRIX FACTORIZATION FOR GENE EXPRESSION CLUSTERING
GAUSS-SEIDEL BASED NON-NEGAIVE MARIX FACORIZAION FOR GENE EXPRESSION CLUSERING Qing Liao, Naiyang Guan, Qian Zhang Dept. of Computer Science & echnology, he Hong Kong University of Science and echnology
Freeman (2005) - Graphic Techniques for Exploring Social Network Data
Freeman (2005) - Graphic Techniques for Exploring Social Network Data The analysis of social network data has two main goals: 1. Identify cohesive groups 2. Identify social positions Moreno (1932) was
Statistical Inference: Estimation and Confidence Intervals Hypothesis Testing
Statistical Inference: Estimation and Confidence Intervals Hypothesis Testing 1 In most statistics problems, we assume that the data have been generated from some unknown probability distribution. We desire
Deep Learning Basics Lecture 7: Factor Analysis. Princeton University COS 495 Instructor: Yingyu Liang
Deep Learning Basics Lecture 7: Factor Analysis Princeton University COS 495 Instructor: Yingyu Liang Supervised v.s. Unsupervised Math formulation for supervised learning Given training data x i, y i
Dimensionality Reduction for Exponential Family Data
Dimensionality Reduction for Exponential Family Data Yoonkyung Lee* Department of Statistics The Ohio State University *joint work with Andrew Landgraf July 2-6, 2018 Computational Strategies for Large-Scale
Expression Data Exploration: Association, Patterns, Factors & Regression Modelling
Expression Data Exploration: Association, Patterns, Factors & Regression Modelling Exploring gene expression data Scale factors, median chip correlation on gene subsets for crude data quality investigation
ORIE 4741 Final Exam
ORIE 4741 Final Exam December 15, 2016 Rules for the exam. Write your name and NetID at the top of the exam. The exam is 2.5 hours long. Every multiple choice or true false question is worth 1 point. Every
Similarity and Dissimilarity
1//015 Similarity and Dissimilarity COMP 465 Data Mining Similarity of Data Data Preprocessing Slides Adapted From : Jiawei Han, Micheline Kamber & Jian Pei Data Mining: Concepts and Techniques, 3 rd ed.
Section 29: What s an Inverse?
Section 29: What s an Inverse? Our investigations in the last section showed that all of the matrix operations had an identity element. The identity element for addition is, for obvious reasons, called
What is a Decision Problem?
Alberto Colorni 1 Alexis Tsoukiàs 2 1 INDACO, Politecnico di Milano, alberto.colorni@polimi.it 2 LAMSADE - CNRS, Université Paris-Dauphine tsoukias@lamsade.dauphine.fr Napoli, 28/04/2013 Outline 1 2 3
Testing for group differences in brain functional connectivity
Testing for group differences in brain functional connectivity Junghi Kim, Wei Pan, for ADNI Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN 55455 Banff Feb
ECE 501b Homework #6 Due: 11/26
ECE 51b Homework #6 Due: 11/26 1. Principal Component Analysis: In this assignment, you will explore PCA as a technique for discerning whether low-dimensional structure exists in a set of data and for
Chapter 2 Class Notes
Chapter 2 Class Notes Probability can be thought of in many ways, for example as a relative frequency of a long series of trials (e.g. flips of a coin or die) Another approach is to let an expert (such
Factor Analysis (FA) Non-negative Matrix Factorization (NMF) CSE Artificial Intelligence Grad Project Dr. Debasis Mitra
Factor Analysis (FA) Non-negative Matrix Factorization (NMF) CSE 5290 - Artificial Intelligence Grad Project Dr. Debasis Mitra Group 6 Taher Patanwala Zubin Kadva Factor Analysis (FA) 1. Introduction Factor
Research Methods in Environmental Science
Research Methods in Environmental Science Module 5: Research Methods in the Physical Sciences Research Methods in the Physical Sciences Obviously, the exact techniques and methods you will use will vary
Goodness of Fit Tests
Goodness of Fit Tests Marc H. Mehlman marcmehlman@yahoo.com University of New Haven (University of New Haven) Goodness of Fit Tests 1 / 38 Table of Contents 1 Goodness of Fit Chi Squared Test 2 Tests of
Latent Semantic Analysis. Hongning Wang
Latent Semantic Analysis Hongning Wang CS@UVa VS model in practice Document and query are represented by term vectors Terms are not necessarily orthogonal to each other Synonymy: car v.s. automobile Polysemy:
4 Bias-Variance for Ridge Regression (24 points)
Implement Ridge Regression with λ = 0.00001. Plot the Squared Euclidean test error for the following values of k (the dimensions you reduce to): k = {0, 50, 100, 150, 200, 250, 300, 350, 400, 450, 500,
MODULE 9 NORMAL DISTRIBUTION
MODULE 9 NORMAL DISTRIBUTION Contents 9.1 Characteristics of a Normal Distribution........................... 62 9.2 Simple Areas Under the Curve................................. 63 9.3 Forward Calculations......................................
Data Mining 4. Cluster Analysis
Data Mining 4. Cluster Analysis 4.2 Spring 2010 Instructor: Dr. Masoud Yaghini Outline Data Structures Interval-Valued (Numeric) Variables Binary Variables Categorical Variables Ordinal Variables Variables
Singular Value Decompsition
Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost
a Short Introduction
Collaborative Filtering in Recommender Systems: a Short Introduction Norm Matloff Dept. of Computer Science University of California, Davis matloff@cs.ucdavis.edu December 3, 2016 Abstract There is a strong
Principal Component Analysis, A Powerful Scoring Technique
Principal Component Analysis, A Powerful Scoring Technique George C. J. Fernandez, University of Nevada - Reno, Reno NV 89557 ABSTRACT Data mining is a collection of analytical techniques to uncover new
Structure in Data. A major objective in data analysis is to identify interesting features or structure in the data.
Structure in Data A major objective in data analysis is to identify interesting features or structure in the data. The graphical methods are very useful in discovering structure. There are basically two
Basics of Multivariate Modelling and Data Analysis
Basics of Multivariate Modelling and Data Analysis Kurt-Erik Häggblom 6. Principal component analysis (PCA) 6.1 Overview 6.2 Essentials of PCA 6.3 Numerical calculation of PCs 6.4 Effects of data preprocessing
Machine Learning (Spring 2012) Principal Component Analysis
1-71 Machine Learning (Spring 1) Principal Component Analysis Yang Xu This note is partly based on Chapter 1.1 in Chris Bishop s book on PRML and the lecture slides on PCA written by Carlos Guestrin in
Binary Principal Component Analysis in the Netflix Collaborative Filtering Task
Binary Principal Component Analysis in the Netflix Collaborative Filtering Task László Kozma, Alexander Ilin, Tapani Raiko first.last@tkk.fi Helsinki University of Technology Adaptive Informatics Research
Clustering Lecture 1: Basics. Jing Gao SUNY Buffalo
Clustering Lecture 1: Basics Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics Clustering
MLCC 2015 Dimensionality Reduction and PCA
MLCC 2015 Dimensionality Reduction and PCA Lorenzo Rosasco UNIGE-MIT-IIT June 25, 2015 Outline PCA & Reconstruction PCA and Maximum Variance PCA and Associated Eigenproblem Beyond the First Principal Component
Practical Statistics for the Analytical Scientist Table of Contents
Practical Statistics for the Analytical Scientist Table of Contents Chapter 1 Introduction - Choosing the Correct Statistics 1.1 Introduction 1.2 Choosing the Right Statistical Procedures 1.2.1 Planning
Lab 2 Worksheet. Problems. Problem 1: Geometry and Linear Equations
Lab 2 Worksheet Problems Problem : Geometry and Linear Equations Linear algebra is, first and foremost, the study of systems of linear equations. You are going to encounter linear systems frequently in
L 2,1 Norm and its Applications
L 2, Norm and its Applications Yale Chang Introduction According to the structure of the constraints, the sparsity can be obtained from three types of regularizers for different purposes.. Flat Sparsity.
Designing Information Devices and Systems II Fall 2018 Elad Alon and Miki Lustig Homework 9
EECS 16B Designing Information Devices and Systems II Fall 18 Elad Alon and Miki Lustig Homework 9 This homework is due Wednesday, October 31, 18, at 11:59pm. Self grades are due Monday, November 5, 18,
Notes on Latent Semantic Analysis
Notes on Latent Semantic Analysis Costas Boulis 1 Introduction One of the most fundamental problems of information retrieval (IR) is to find all documents (and nothing but those) that are semantically
Clustering. Genome 373 Genomic Informatics Elhanan Borenstein. Some slides adapted from Jacques van Helden
Clustering Genome 373 Genomic Informatics Elhanan Borenstein Some slides adapted from Jacques van Helden The clustering problem The goal of gene clustering process is to partition the genes into distinct
Advanced Statistical Methods: Beyond Linear Regression
Advanced Statistical Methods: Beyond Linear Regression John R. Stevens Utah State University Notes 3. Statistical Methods II Mathematics Educators Worshop 28 March 2009 1 http://www.stat.usu.edu/~jrstevens/pcmi
Image Registration Lecture 2: Vectors and Matrices
Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this
Embeddings Learned By Matrix Factorization
Embeddings Learned By Matrix Factorization Benjamin Roth; Folien von Hinrich Schütze Center for Information and Language Processing, LMU Munich Overview WordSpace limitations LinAlgebra review Input matrix
Lecture 5: ANOVA and Correlation
Lecture 5: ANOVA and Correlation Ani Manichaikul amanicha@jhsph.edu 23 April 2007 1 / 62 Comparing Multiple Groups Continous data: comparing means Analysis of variance Binary data: comparing proportions
Structural Learning and Integrative Decomposition of Multi-View Data
Structural Learning and Integrative Decomposition of Multi-View Data, Department of Statistics, Texas A&M University JSM 2018, Vancouver, Canada July 31st, 2018 Dr. Gen Li, Columbia University, Mailman
2.3. Clustering or vector quantization 57
Multivariate Statistics non-negative matrix factorisation and sparse dictionary learning The PCA decomposition is by construction optimal solution to argmin A R n q,h R q p X AH 2 2 under constraint :
Dimension Reduction and Classification Using PCA and Factor. Overview
Dimension Reduction and Classification Using PCA and - A Short Overview Laboratory for Interdisciplinary Statistical Analysis Department of Statistics Virginia Tech http://www.stat.vt.edu/consult/ March
Clustering. Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein. Some slides adapted from Jacques van Helden
Clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein Some slides adapted from Jacques van Helden Small vs. large parsimony A quick review Fitch s algorithm:
Data Mining Techniques
Data Mining Techniques CS 6220 - Section 3 - Fall 2016 Lecture 12 Jan-Willem van de Meent (credit: Yijun Zhao, Percy Liang) DIMENSIONALITY REDUCTION Borrowing from: Percy Liang (Stanford) Linear Dimensionality
COMP 551 Applied Machine Learning Lecture 13: Dimension reduction and feature selection
COMP 551 Applied Machine Learning Lecture 13: Dimension reduction and feature selection Instructor: Herke van Hoof (herke.vanhoof@cs.mcgill.ca) Based on slides by:, Jackie Chi Kit Cheung Class web page:
Multilevel Analysis, with Extensions
May 26, 2010 We start by reviewing the research on multilevel analysis that has been done in psychometrics and educational statistics, roughly since 1985. The canonical reference (at least I hope so) is
Quick Introduction to Nonnegative Matrix Factorization
Quick Introduction to Nonnegative Matrix Factorization Norm Matloff University of California at Davis 1 The Goal Given an u v matrix A with nonnegative elements, we wish to find nonnegative, rank-k matrices
Collaborative Filtering. Radek Pelánek
Collaborative Filtering Radek Pelánek 2017 Notes on Lecture the most technical lecture of the course includes some scary looking math, but typically with intuitive interpretation use of standard machine
Introduction How it works Theory behind Compressed Sensing. Compressed Sensing. Huichao Xue. CS3750 Fall 2011
Compressed Sensing Huichao Xue CS3750 Fall 2011 Table of Contents Introduction From News Reports Abstract Definition How it works A review of L 1 norm The Algorithm Backgrounds for underdetermined linear
Functional SVD for Big Data
Functional SVD for Big Data Pan Chao April 23, 2014 Pan Chao Functional SVD for Big Data April 23, 2014 1 / 24 Outline 1 One-Way Functional SVD a) Interpretation b) Robustness c) CV/GCV 2 Two-Way Problem
A Modular NMF Matching Algorithm for Radiation Spectra
A Modular NMF Matching Algorithm for Radiation Spectra Melissa L. Koudelka Sensor Exploitation Applications Sandia National Laboratories mlkoude@sandia.gov Daniel J. Dorsey Systems Technologies Sandia
Sample Size and Power Considerations for Longitudinal Studies
Sample Size and Power Considerations for Longitudinal Studies Outline Quantities required to determine the sample size in longitudinal studies Review of type I error, type II error, and power For continuous
7 Principal Component Analysis
7 Principal Component Analysis This topic will build a series of techniques to deal with high-dimensional data. Unlike regression problems, our goal is not to predict a value (the y-coordinate), it is
Using Matrix Decompositions in Formal Concept Analysis
Using Matrix Decompositions in Formal Concept Analysis Vaclav Snasel 1, Petr Gajdos 1, Hussam M. Dahwa Abdulla 1, Martin Polovincak 1 1 Dept. of Computer Science, Faculty of Electrical Engineering and
Robust Asymmetric Nonnegative Matrix Factorization
Robust Asymmetric Nonnegative Matrix Factorization Hyenkyun Woo Korea Institue for Advanced Study Math @UCLA Feb. 11. 2015 Joint work with Haesun Park Georgia Institute of Technology, H. Woo (KIAS) RANMF
A primer on matrices
A primer on matrices Stephen Boyd August 4, 2007 These notes describe the notation of matrices, the mechanics of matrix manipulation, and how to use matrices to formulate and solve sets of simultaneous
Chapter 4: Factor Analysis
Chapter 4: Factor Analysis In many studies, we may not be able to measure directly the variables of interest. We can merely collect data on other variables which may be related to the variables of interest.
Matrix Decomposition in Privacy-Preserving Data Mining JUN ZHANG DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF KENTUCKY
Matrix Decomposition in Privacy-Preserving Data Mining JUN ZHANG DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY OF KENTUCKY OUTLINE Why We Need Matrix Decomposition SVD (Singular Value Decomposition) NMF (Nonnegative
Multiple Correspondence Analysis
Multiple Correspondence Analysis 18 Up to now we have analysed the association between two categorical variables or between two sets of categorical variables where the row variables are different from
Lectures of STA 231: Biostatistics
Lectures of STA 231: Biostatistics Second Semester Academic Year 2016/2017 Text Book Biostatistics: Basic Concepts and Methodology for the Health Sciences (10 th Edition, 2014) By Wayne W. Daniel Prepared
SVD, discrepancy, and regular structure of contingency tables
SVD, discrepancy, and regular structure of contingency tables Marianna Bolla Institute of Mathematics Technical University of Budapest (Research supported by the TÁMOP-4.2.2.B-10/1-2010-0009 project) marib@math.bme.hu
Chapter 7. Scatterplots, Association, and Correlation. Copyright 2010 Pearson Education, Inc.
Chapter 7 Scatterplots, Association, and Correlation Copyright 2010 Pearson Education, Inc. Looking at Scatterplots Scatterplots may be the most common and most effective display for data. In a scatterplot,
14 Singular Value Decomposition
14 Singular Value Decomposition For any high-dimensional data analysis, one s first thought should often be: can I use an SVD? The singular value decomposition is an invaluable analysis tool for dealing
Robustness of Principal Components
PCA for Clustering An objective of principal components analysis is to identify linear combinations of the original variables that are useful in accounting for the variation in those original variables.
How to analyze multiple distance matrices
DISTATIS How to analyze multiple distance matrices Hervé Abdi & Dominique Valentin Overview. Origin and goal of the method DISTATIS is a generalization of classical multidimensional scaling (MDS see the
Quantitative Understanding in Biology Principal Components Analysis
Quantitative Understanding in Biology Principal Components Analysis Introduction Throughout this course we have seen examples of complex mathematical phenomena being represented as linear combinations
Problem Set #2 Due: 1:00pm on Monday, Oct 15th
Chris Piech PSet #2 CS109 October 5, 2018 Problem Set #2 Due: 1:00pm on Monday, Oct 15th For each problem, briefly explain/justify how you obtained your answer in order to obtain full credit. Your explanations
Chapter 26: Comparing Counts (Chi Square)
Chapter 6: Comparing Counts (Chi Square) We ve seen that you can turn a qualitative variable into a quantitative one (by counting the number of successes and failures), but that s a compromise it forces
Collection of Biostatistics Research Archive
Collection of Biostatistics Research Archive COBRA Preprint Series Year 2016 Paper 115 hpcnmf: A high-performance toolbox for non-negative matrix factorization Karthik Devarajan Guoli Wang Fox Chase Cancer
Parallel Singular Value Decomposition. Jiaxing Tan
Parallel Singular Value Decomposition Jiaxing Tan Outline What is SVD? How to calculate SVD? How to parallelize SVD? Future Work What is SVD? Matrix Decomposition Eigen Decomposition A (non-zero) vector
Linear Models 1. Isfahan University of Technology Fall Semester, 2014
Linear Models 1 Isfahan University of Technology Fall Semester, 2014 References: [1] G. A. F., Seber and A. J. Lee (2003). Linear Regression Analysis (2nd ed.). Hoboken, NJ: Wiley. [2] A. C. Rencher and
Distances and similarities Based in part on slides from textbook, slides of Susan Holmes. October 3, Statistics 202: Data Mining
Distances and similarities Based in part on slides from textbook, slides of Susan Holmes October 3, 2012 1 / 1 Similarities Start with X which we assume is centered and standardized. The PCA loadings were
These Choice Boards cover the fourteen units for eighth grade Common Core math.
A Note to the Teacher: Thank you so much for purchasing this Choice Board BUNDLE from The Math Station on Teachers Pay Teachers. I hope you like it! I started using choice boards last year and LOVE them!
Unsupervised machine learning
Chapter 9 Unsupervised machine learning Unsupervised machine learning (a.k.a. cluster analysis) is a set of methods to assign objects into clusters under a predefined distance measure when class labels
Data Mining Techniques
Data Mining Techniques CS 622 - Section 2 - Spring 27 Pre-final Review Jan-Willem van de Meent Feedback Feedback https://goo.gl/er7eo8 (also posted on Piazza) Also, please fill out your TRACE evaluations!
Factor Analysis (10/2/13)
STA561: Probabilistic machine learning Factor Analysis (10/2/13) Lecturer: Barbara Engelhardt Scribes: Li Zhu, Fan Li, Ni Guan Factor Analysis Factor analysis is related to the mixture models we have studied.
Hypothesis Testing: Chi-Square Test 1
Hypothesis Testing: Chi-Square Test 1 November 9, 2017 1 HMS, 2017, v1.0 Chapter References Diez: Chapter 6.3 Navidi, Chapter 6.10 Chapter References 2 Chi-square Distributions Let X 1, X 2,... X n be