Knowledge Discovery with Iterative Denoising
|
|
|
- Beatrix Williamson
- 5 years ago
- Views:
Transcription
1 Knowledge Discovery with Iterative Denoising Assistant Professor Department of Statistics and Operations Research Virginia Commonwealth University Associate Staff Scientist Human Language Technology Center for Excellence (HLTCOE) Johns Hopkins University
2 What is Data? Object xi has q measurements: x i = x i1, x i2,..., x iq T ℝ q * All n objects in the dataset can be expressed as an nxq data matrix * known as vector-space model Examples: 1.) Text Mining: n documents, q weights or scores for particular words or phrases 2.) Image Analysis: n images, q pixel color or intensity values 3.) Computer Network Traffic: n application or protocol flows, q network traffic counts or scores 4.) DNA Expression Microarrays: n genes (nucleotide sequences), q cell samples 2
3 Growth of (Complex) Data Amount of Data is Increasing * Figure: annotated collection of all available DNA sequences. * PUB MED: abstracts for 12 million research papers on life sciences topics. 40,000 new biomedical abstracts are added every month. * Similar growth of data in other fields: computer network traffic, text documents, images. Complexity of Data is Increasing * n observations, q features/measurements/dimensions * Data that we need to analyze is high-dimensional: q large * In complex data, often q >> n 3
4 Concerns with High-Dimensional Data High-dimensionality causes analysis algorithm performance problems * scaling * curse of dimensionality * if q >> n, overfitting issues Dimensionality Reduction! * Most feature selection is done univariately, but complex data is multivariate - a variable that is useless by itself can provide a significant performance improvement when taken with others Common procedure is to decouple feature selection from classification algorithm * but concern is better classification performance, not necessarily what features are important * difficult to do with high-dimensional data * especially for new datasets, there is some concern about the loss of useful information when throwing away features 4
5 Many (Statistical) Approaches Class-conditional densities: * known - Bayes Decision Theory * unknown - supervised - unsupervised - parametric and nonparametric 5
6 Example 1: Find SPAM 6
7 Example 1: Find SPAM Supervised learning 7
8 Example 2: Find EVIL-DOERS Enron Figure: The New York Times Carey Priebe and Youngser Park, Johns Hopkins University 8
9 Example 2: Find EVIL-DOERS Enron Figure: The New York Times Carey Priebe and Youngser Park, Johns Hopkins University Unsupervised learning 9
10 Implementation Issues * large databases * distance * high dimensionality * overfitting * missing/noisy data * local structure * user involvement 10
11 Desired Classification Methodology Attributes: 1. Unsupervised 2. Nonparametric 3. Scalable (large n) 4. Curse-aware (high-dimensional + sparse) 5. Intuitive visualization interface 6. Framework flexible 7. Useful for data analyst 8. Data domain agnostic 9. Distinguish global similarity from local dissimilarity 11
12 The Iterative Denoising Methodology In a nutshell: Process a set of high-dimensional data; perform a local structurepreserving projection into a low-dimensional space; provide a visualization and interaction interface; partition and iteratively denoise. Motivated by: Priebe, Marchette, Healy, 2004, Integrated Sensing and Processing Decision Trees, IEEE PAMI. Priebe, et al., 2004, Iterative Denoising for Cross-Corpus Discovery, COMPSTAT. 12
13 cd Denoise The Concept: An Iterative Denoising Tree fe label 13
14 Iterative Denoising Framework 14
15 Denoising Detail A proximity metric A low dim space Clusters Nonlinear version uses Laplacian Eigenmaps 15
16 Laplacian Eigenmaps Nonlinear dimensionality reduction technique that distorts geometry in such a way that enhances some types of clustering L = D A is large, sparse L symmetric, positive semi-definite 0 < λ1 λ2... λd Corresponding d eigenvectors --> Fiedler Space Eigenvectors corresponding to two smallest non-zero eigenvalues --> visualization 16
17 Great, but does it work? 17
18 A Text Document Example: a Science News Corpus n = 1047 documents q = words (ngrams) 18
19 Science News corpus: a clustering hierarchy 1. Anthropology 2. Astronomy 3. Behavioral Sciences 4. Earth Sciences 5. Life Sciences 6. Math & CS 7. Medicine 8. Physics 19
20 Science News corpus: a clustering hierarchy 1. Anthropology 2. Astronomy 3. Behavioral Sciences 4. Earth Sciences 5. Life Sciences 6. Math & CS 7. Medicine 8. Physics 20
21 Science News corpus, Fiedler Space projection Anthropology: yellow Astronomy: black Behavioral Sciences: magenta Earth Sciences: lightgray Life Sciences: orange Math & CS: red Medicine: green Physics: blue 21
22 Science News corpus, Fiedler Space projection Anthropology: yellow Astronomy: black Behavioral Sciences: magenta Life Sciences: orange Earth Sciences: lightgray Math & CS: red Medicine: green Physics: blue 22
23 Science News corpus, Fiedler Space projection Anthropology: yellow Astronomy: black Behavioral Sciences: magenta Earth Sciences: lightgray Life Sciences: orange Math & CS: red Medicine: green Physics: blue 23
24 Science News corpus, Fiedler Space projection Anthropology: yellow Astronomy: black Behavioral Sciences: magenta Earth Sciences: lightgray Life Sciences: orange Math & CS: red Medicine: green Physics: blue 24
25 Interesting Grouping 1. Math enthusiast wins Science Talent Search 2. Message in DNA tops Science Talent Search 3. Chinks in Digital Armor: exploiting faults to break smart-card cryptosystems 4. Motor City hosts top science fair winners 5. Science Talent Search winners shine bright 6. Neutrinos to buckyballs: 10 talents tower 7. Logic in the Blocks: simple puzzles can give computers an unexpected workout 8. How to trick other people's computers into solving your math problems Anthropology: yellow Astronomy: black Behavioral Sciences: magenta Earth Sciences: lightgray Life Sciences: orange Math & CS: red Medicine: green Physics: blue 25
26 Stopping Criteria Variety of methods can be used for stopping the tree build: * max tree height * min observations/node * (supervised): min node purity * (supervised): entropy or divergence Here, we used min observations/node = 10d/k 26
27 Semi-supervised Classification Performance 1.) Embed using all observations (labeled + unlabeled) 2.) Label Iterative Denoising Tree leaves using only labeled observations 3.) Calculate performance based on unlabeled observations, 10-fold cross validation See: M.W. Trosset, C.E. Priebe, Y. Park, and M.I. Miller, "Semisupervised Learning from Dissimilarity Data," Computational Statistics and Data Analysis, accepted for publication, February,
28 4-Class Science News: Iterative Denoising Astronomy: black Math & CS: red Medicine: green Physics: blue n = 579 documents, q = unique monograms 28
29 4-Class Science News Astronomy: black Math & CS: red Medicine: green Physics: blue 29
30 20-Newsgroups * Collection of newsgroup documents partitioned into 20 different newsgroups. --> * As a simple test, we chose three roughly disparate groups, and two groups on the same topic but with differing viewpoints. 1.) rec.autos 2.) talk.politics.misc 3.) comp.graphics 4.) soc.religion.christian 5.) alt.athiesm n = 2722 documents, with unique monograms * NB: we did not do typical text pre-processing tasks such as stop-word removal, removal of infrequent words, etc. 30
31 20-Newsgroups: no natural class separation Image from FINE: Fisher Information Non-parametric Embedding by Kevin M. Carter, Raviv Raich, William G. Finn, Alfred O. Hero 31
32 20-Newsgroups: Iterative Denoising K_knn = 20, m = 2, dim = 9: error = , sd = green black red gray blue rec.autos talk.politics.misc soc.religion.christian comp.graphics alt.athiesm 32
33 20-Newsgroups green black red gray blue rec.autos talk.politics.misc soc.religion.christian comp.graphics alt.athiesm 33
34 20-Newsgroups green black red gray blue rec.autos talk.politics.misc soc.religion.christian comp.graphics alt.athiesm 34
35 20-Newsgroups green black red gray blue rec.autos talk.politics.misc soc.religion.christian comp.graphics alt.athiesm 35
36 20-Newsgroups green black red gray blue rec.autos talk.politics.misc soc.religion.christian comp.graphics alt.athiesm 36
37 ...It follows from this that God must give everyone the same revelation of truth, and thus anyone who comes to a different conclusion is intentionally choosing the wrong path......we will grant that a God exists, and uses revelation to communicate with humans... Global Similarity green black red gray blue rec.autos talk.politics.misc soc.religion.christian comp.graphics alt.athiesm 37
38 20-Newsgroups...What I see as arrogance and the problem I have with it is not a sense of personal certainty, but a lack of respect for others who come to differing conclusions......now, based on this, can this person be blamed for concluding, absent a personal revelation of their own, that there is almost certainly nothing to this 'revelation' thing?... Local Dissimilarity 38
39 Summary 1. Unsupervised 2. Nonparametric Iterative Denoising Methodology Feature selection inherent in methodology Scalable and tailored for high-dimensionality Ability to visualize relationships in complex data 3. Scalable (large n) 4. Curse-aware (high-dimensional + sparse) 5. Intuitive visualization interface 6. Framework flexible 7. Useful for data analyst 8. Data domain agnostic 9. Distinguish global similarity from local dissimilarity Enhances global similarities and local dissimilarities Good performance on text datasets Also good performance on intrusion detection dataset Preliminary testing on backscatter, biological data, images 39
40 Also: Iterative Denoising,, Michael Trosset, David Marchette, Carey Priebe. Computational Statistics, Accepted for Publication, September Collaborators: * Dr. Carey Priebe, Department of Applied Mathematics and Statistics, Johns Hopkins University * Dr. David Marchette, Senior Scientist, Naval Surface Warfare Center * Dr. Michael Trosset, Department of Statistics, Indiana University * Dr. Jeff Solka, Principal Scientist, Naval Surface Warfare Center * Dr. Youngser Park, Senior Research Analyst, Johns Hopkins University *
41 We are drowning in information and starving for knowledge. -- Rutherford D. Roger 41
Nonlinear Dimensionality Reduction. Jose A. Costa
Nonlinear Dimensionality Reduction Jose A. Costa Mathematics of Information Seminar, Dec. Motivation Many useful of signals such as: Image databases; Gene expression microarrays; Internet traffic time
Graph Theoretic Latent Class Discovery
Graph Theoretic Latent Class Discovery Jeff Solka jsolka@gmu.edu NSWCDD/GMU GMU BINF Colloquium 2/24/04 p.1/28 Agenda What is latent class discovery? What are some approaches to the latent class discovery
Unsupervised Anomaly Detection for High Dimensional Data
Unsupervised Anomaly Detection for High Dimensional Data Department of Mathematics, Rowan University. July 19th, 2013 International Workshop in Sequential Methodologies (IWSM-2013) Outline of Talk Motivation
CPSC 340: Machine Learning and Data Mining. More PCA Fall 2016
CPSC 340: Machine Learning and Data Mining More PCA Fall 2016 A2/Midterm: Admin Grades/solutions posted. Midterms can be viewed during office hours. Assignment 4: Due Monday. Extra office hours: Thursdays
CS 6375 Machine Learning
CS 6375 Machine Learning Nicholas Ruozzi University of Texas at Dallas Slides adapted from David Sontag and Vibhav Gogate Course Info. Instructor: Nicholas Ruozzi Office: ECSS 3.409 Office hours: Tues.
Graph Metrics and Dimension Reduction
Graph Metrics and Dimension Reduction Minh Tang 1 Michael Trosset 2 1 Applied Mathematics and Statistics The Johns Hopkins University 2 Department of Statistics Indiana University, Bloomington November
Investigating the structure of high dimensional pattern recognition problems
Investigating the structure of high dimensional pattern recognition problems Carey E. Priebe Department of Mathematical Sciences Whiting School of Engineering Johns Hopkins University altimore,
Iterative Laplacian Score for Feature Selection
Iterative Laplacian Score for Feature Selection Linling Zhu, Linsong Miao, and Daoqiang Zhang College of Computer Science and echnology, Nanjing University of Aeronautics and Astronautics, Nanjing 2006,
ECE 521. Lecture 11 (not on midterm material) 13 February K-means clustering, Dimensionality reduction
ECE 521 Lecture 11 (not on midterm material) 13 February 2017 K-means clustering, Dimensionality reduction With thanks to Ruslan Salakhutdinov for an earlier version of the slides Overview K-means clustering
Dimension reduction methods: Algorithms and Applications Yousef Saad Department of Computer Science and Engineering University of Minnesota
Dimension reduction methods: Algorithms and Applications Yousef Saad Department of Computer Science and Engineering University of Minnesota Université du Littoral- Calais July 11, 16 First..... to the
Data Mining Techniques
Data Mining Techniques CS 622 - Section 2 - Spring 27 Pre-final Review Jan-Willem van de Meent Feedback Feedback https://goo.gl/er7eo8 (also posted on Piazza) Also, please fill out your TRACE evaluations!
Lecture 7: DecisionTrees
Lecture 7: DecisionTrees What are decision trees? Brief interlude on information theory Decision tree construction Overfitting avoidance Regression trees COMP-652, Lecture 7 - September 28, 2009 1 Recall:
Clustering using Mixture Models
Clustering using Mixture Models The full posterior of the Gaussian Mixture Model is p(x, Z, µ,, ) =p(x Z, µ, )p(z )p( )p(µ, ) data likelihood (Gaussian) correspondence prob. (Multinomial) mixture prior
CS570 Introduction to Data Mining
CS570 Introduction to Data Mining Department of Mathematics and Computer Science Li Xiong Data Exploration and Data Preprocessing Data and Attributes Data exploration Data pre-processing Data cleaning
From statistics to data science. BAE 815 (Fall 2017) Dr. Zifei Liu
From statistics to data science BAE 815 (Fall 2017) Dr. Zifei Liu Zifeiliu@ksu.edu Why? How? What? How much? How many? Individual facts (quantities, characters, or symbols) The Data-Information-Knowledge-Wisdom
Text Mining. Dr. Yanjun Li. Associate Professor. Department of Computer and Information Sciences Fordham University
Text Mining Dr. Yanjun Li Associate Professor Department of Computer and Information Sciences Fordham University Outline Introduction: Data Mining Part One: Text Mining Part Two: Preprocessing Text Data
What is semi-supervised learning?
What is semi-supervised learning? In many practical learning domains, there is a large supply of unlabeled data but limited labeled data, which can be expensive to generate text processing, video-indexing,
Department of Computer Science and Engineering
Linear algebra methods for data mining with applications to materials Yousef Saad Department of Computer Science and Engineering University of Minnesota ICSC 2012, Hong Kong, Jan 4-7, 2012 HAPPY BIRTHDAY
Part-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287
Part-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287 Review: Neural Networks One-layer multi-layer perceptron architecture, NN MLP1 (x) = g(xw 1 + b 1 )W 2 + b 2 xw + b; perceptron x is the
PCA & ICA. CE-717: Machine Learning Sharif University of Technology Spring Soleymani
PCA & ICA CE-717: Machine Learning Sharif University of Technology Spring 2015 Soleymani Dimensionality Reduction: Feature Selection vs. Feature Extraction Feature selection Select a subset of a given
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Data & Data Preprocessing & Classification (Basic Concepts) Huan Sun, CSE@The Ohio State University Slides adapted from UIUC CS412, Fall 2017, by Prof. Jiawei Han Chapter
Support Vector Machine & Its Applications
Support Vector Machine & Its Applications A portion (1/3) of the slides are taken from Prof. Andrew Moore s SVM tutorial at http://www.cs.cmu.edu/~awm/tutorials Mingyue Tan The University of British Columbia
Correlation Preserving Unsupervised Discretization. Outline
Correlation Preserving Unsupervised Discretization Jee Vang Outline Paper References What is discretization? Motivation Principal Component Analysis (PCA) Association Mining Correlation Preserving Discretization
The Out-of-Sample Problem for Classical Multidimensional Scaling
The Out-of-Sample Problem for Classical Multidimensional Scaling Michael W. Trosset a and Carey E. Priebe b December 21, 2006; revised February 10, 2008 Technical Report 06-04 Department of Statistics
Informal Definition: Telling things apart
9. Decision Trees Informal Definition: Telling things apart 2 Nominal data No numeric feature vector Just a list or properties: Banana: longish, yellow Apple: round, medium sized, different colors like
Modern Information Retrieval
Modern Information Retrieval Chapter 8 Text Classification Introduction A Characterization of Text Classification Unsupervised Algorithms Supervised Algorithms Feature Selection or Dimensionality Reduction
Graph Inference with Imperfect Edge Classifiers
Graph Inference with Imperfect Edge Classifiers Michael W. Trosset Department of Statistics Indiana University Joint work with David Brinda (Yale University) and Shantanu Jain (Indiana University), supported
Computer Vision Group Prof. Daniel Cremers. 14. Clustering
Group Prof. Daniel Cremers 14. Clustering Motivation Supervised learning is good for interaction with humans, but labels from a supervisor are hard to obtain Clustering is unsupervised learning, i.e. it
CS249: ADVANCED DATA MINING
CS249: ADVANCED DATA MINING Clustering Evaluation and Practical Issues Instructor: Yizhou Sun yzsun@cs.ucla.edu May 2, 2017 Announcements Homework 2 due later today Due May 3 rd (11:59pm) Course project
UVA CS 4501: Machine Learning
UVA CS 4501: Machine Learning Lecture 21: Decision Tree / Random Forest / Ensemble Dr. Yanjun Qi University of Virginia Department of Computer Science Where are we? è Five major sections of this course
Classification Semi-supervised learning based on network. Speakers: Hanwen Wang, Xinxin Huang, and Zeyu Li CS Winter
Classification Semi-supervised learning based on network Speakers: Hanwen Wang, Xinxin Huang, and Zeyu Li CS 249-2 2017 Winter Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions Xiaojin
Gaussian Models
Gaussian Models ddebarr@uw.edu 2016-04-28 Agenda Introduction Gaussian Discriminant Analysis Inference Linear Gaussian Systems The Wishart Distribution Inferring Parameters Introduction Gaussian Density
Computational Biology Course Descriptions 12-14
Computational Biology Course Descriptions 12-14 Course Number and Title INTRODUCTORY COURSES BIO 311C: Introductory Biology I BIO 311D: Introductory Biology II BIO 325: Genetics CH 301: Principles of Chemistry
Ruslan Salakhutdinov Joint work with Geoff Hinton. University of Toronto, Machine Learning Group
NON-LINEAR DIMENSIONALITY REDUCTION USING NEURAL NETORKS Ruslan Salakhutdinov Joint work with Geoff Hinton University of Toronto, Machine Learning Group Overview Document Retrieval Present layer-by-layer
Dimensionality Reduction
Dimensionality Reduction Le Song Machine Learning I CSE 674, Fall 23 Unsupervised learning Learning from raw (unlabeled, unannotated, etc) data, as opposed to supervised data where a classification of
Topic Models and Applications to Short Documents
Topic Models and Applications to Short Documents Dieu-Thu Le Email: dieuthu.le@unitn.it Trento University April 6, 2011 1 / 43 Outline Introduction Latent Dirichlet Allocation Gibbs Sampling Short Text
Unsupervised Image Segmentation Using Comparative Reasoning and Random Walks
Unsupervised Image Segmentation Using Comparative Reasoning and Random Walks Anuva Kulkarni Carnegie Mellon University Filipe Condessa Carnegie Mellon, IST-University of Lisbon Jelena Kovacevic Carnegie
Self Organizing Maps. We are drowning in information and starving for knowledge. A New Approach for Integrated Analysis of Geological Data.
Radiometrics Processed Landsat TM Self Organizing Maps A New Approach for Integrated Analysis of Geological Data. We are drowning in information and starving for knowledge. Rutherford D. Roger Stephen.Fraser@csiro.au
Be able to define the following terms and answer basic questions about them:
CS440/ECE448 Section Q Fall 2017 Final Review Be able to define the following terms and answer basic questions about them: Probability o Random variables, axioms of probability o Joint, marginal, conditional
Statistical Machine Learning
Statistical Machine Learning Christoph Lampert Spring Semester 2015/2016 // Lecture 12 1 / 36 Unsupervised Learning Dimensionality Reduction 2 / 36 Dimensionality Reduction Given: data X = {x 1,..., x
Statistical Learning. Dong Liu. Dept. EEIS, USTC
Statistical Learning Dong Liu Dept. EEIS, USTC Chapter 6. Unsupervised and Semi-Supervised Learning 1. Unsupervised learning 2. k-means 3. Gaussian mixture model 4. Other approaches to clustering 5. Principle
Models, Data, Learning Problems
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Models, Data, Learning Problems Tobias Scheffer Overview Types of learning problems: Supervised Learning (Classification, Regression,
Machine Learning for Software Engineering
Machine Learning for Software Engineering Dimensionality Reduction Prof. Dr.-Ing. Norbert Siegmund Intelligent Software Systems 1 2 Exam Info Scheduled for Tuesday 25 th of July 11-13h (same time as the
Machine Learning 3. week
Machine Learning 3. week Entropy Decision Trees ID3 C4.5 Classification and Regression Trees (CART) 1 What is Decision Tree As a short description, decision tree is a data classification procedure which
Shared Segmentation of Natural Scenes. Dependent Pitman-Yor Processes
Shared Segmentation of Natural Scenes using Dependent Pitman-Yor Processes Erik Sudderth & Michael Jordan University of California, Berkeley Parsing Visual Scenes sky skyscraper sky dome buildings trees
Nonlinear Methods. Data often lies on or near a nonlinear low-dimensional curve aka manifold.
Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27 Laplacian Eigenmaps Linear methods Lower-dimensional linear projection that preserves distances between all
COMP 551 Applied Machine Learning Lecture 5: Generative models for linear classification
COMP 55 Applied Machine Learning Lecture 5: Generative models for linear classification Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp55 Unless otherwise noted, all material
DIMENSION REDUCTION AND CLUSTER ANALYSIS
DIMENSION REDUCTION AND CLUSTER ANALYSIS EECS 833, 6 March 2006 Geoff Bohling Assistant Scientist Kansas Geological Survey geoff@kgs.ku.edu 864-2093 Overheads and resources available at http://people.ku.edu/~gbohling/eecs833
ECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Classification Clustering Frequent Pattern Mining Matrix Data Decision
Unsupervised Learning: K- Means & PCA
Unsupervised Learning: K- Means & PCA Unsupervised Learning Supervised learning used labeled data pairs (x, y) to learn a func>on f : X Y But, what if we don t have labels? No labels = unsupervised learning
SPECTRAL CLUSTERING AND KERNEL PRINCIPAL COMPONENT ANALYSIS ARE PURSUING GOOD PROJECTIONS
SPECTRAL CLUSTERING AND KERNEL PRINCIPAL COMPONENT ANALYSIS ARE PURSUING GOOD PROJECTIONS VIKAS CHANDRAKANT RAYKAR DECEMBER 5, 24 Abstract. We interpret spectral clustering algorithms in the light of unsupervised
CPSC 340: Machine Learning and Data Mining. Linear Least Squares Fall 2016
CPSC 340: Machine Learning and Data Mining Linear Least Squares Fall 2016 Assignment 2 is due Friday: Admin You should already be started! 1 late day to hand it in on Wednesday, 2 for Friday, 3 for next
Locally-biased analytics
Locally-biased analytics You have BIG data and want to analyze a small part of it: Solution 1: Cut out small part and use traditional methods Challenge: cutting out may be difficult a priori Solution 2:
Principal Component Analysis, A Powerful Scoring Technique
Principal Component Analysis, A Powerful Scoring Technique George C. J. Fernandez, University of Nevada - Reno, Reno NV 89557 ABSTRACT Data mining is a collection of analytical techniques to uncover new
Introduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin
1 Introduction to Machine Learning PCA and Spectral Clustering Introduction to Machine Learning, 2013-14 Slides: Eran Halperin Singular Value Decomposition (SVD) The singular value decomposition (SVD)
Data Mining Techniques
Data Mining Techniques CS 6220 - Section 3 - Fall 2016 Lecture 12 Jan-Willem van de Meent (credit: Yijun Zhao, Percy Liang) DIMENSIONALITY REDUCTION Borrowing from: Percy Liang (Stanford) Linear Dimensionality
Mid-year Report Linear and Non-linear Dimentionality. Reduction. applied to gene expression data of cancer tissue samples
Mid-year Report Linear and Non-linear Dimentionality applied to gene expression data of cancer tissue samples Franck Olivier Ndjakou Njeunje Applied Mathematics, Statistics, and Scientific Computation
Online Social Networks and Media. Link Analysis and Web Search
Online Social Networks and Media Link Analysis and Web Search How to Organize the Web First try: Human curated Web directories Yahoo, DMOZ, LookSmart How to organize the web Second try: Web Search Information
Feature selection and extraction Spectral domain quality estimation Alternatives
Feature selection and extraction Error estimation Maa-57.3210 Data Classification and Modelling in Remote Sensing Markus Törmä markus.torma@tkk.fi Measurements Preprocessing: Remove random and systematic
Machine Learning 2nd Edition
INTRODUCTION TO Lecture Slides for Machine Learning 2nd Edition ETHEM ALPAYDIN, modified by Leonardo Bobadilla and some parts from http://www.cs.tau.ac.il/~apartzin/machinelearning/ The MIT Press, 2010
Clustering Lecture 1: Basics. Jing Gao SUNY Buffalo
Clustering Lecture 1: Basics Jing Gao SUNY Buffalo 1 Outline Basics Motivation, definition, evaluation Methods Partitional Hierarchical Density-based Mixture model Spectral methods Advanced topics Clustering
Fast Hierarchical Clustering from the Baire Distance
Fast Hierarchical Clustering from the Baire Distance Pedro Contreras 1 and Fionn Murtagh 1,2 1 Department of Computer Science. Royal Holloway, University of London. 57 Egham Hill. Egham TW20 OEX, England.
Linear Classification and SVM. Dr. Xin Zhang
Linear Classification and SVM Dr. Xin Zhang Email: eexinzhang@scut.edu.cn What is linear classification? Classification is intrinsically non-linear It puts non-identical things in the same class, so a
PATTERN CLASSIFICATION
PATTERN CLASSIFICATION Second Edition Richard O. Duda Peter E. Hart David G. Stork A Wiley-lnterscience Publication JOHN WILEY & SONS, INC. New York Chichester Weinheim Brisbane Singapore Toronto CONTENTS
A Least Squares Formulation for Canonical Correlation Analysis
A Least Squares Formulation for Canonical Correlation Analysis Liang Sun, Shuiwang Ji, and Jieping Ye Department of Computer Science and Engineering Arizona State University Motivation Canonical Correlation
CS145: INTRODUCTION TO DATA MINING
CS145: INTRODUCTION TO DATA MINING 4: Vector Data: Decision Tree Instructor: Yizhou Sun yzsun@cs.ucla.edu October 10, 2017 Methods to Learn Vector Data Set Data Sequence Data Text Data Classification Clustering
Issues and Techniques in Pattern Classification
Issues and Techniques in Pattern Classification Carlotta Domeniconi www.ise.gmu.edu/~carlotta Machine Learning Given a collection of data, a machine learner eplains the underlying process that generated
Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi
Face Recognition Using Laplacianfaces He et al. (IEEE Trans PAMI, 2005) presented by Hassan A. Kingravi Overview Introduction Linear Methods for Dimensionality Reduction Nonlinear Methods and Manifold
Mining Newsgroups Using Networks Arising From Social Behavior by Rakesh Agrawal et al. Presented by Will Lee
Mining Newsgroups Using Networks Arising From Social Behavior by Rakesh Agrawal et al. Presented by Will Lee wwlee1@uiuc.edu September 28, 2004 Motivation IR on newsgroups is challenging due to lack of
Predicting New Search-Query Cluster Volume
Predicting New Search-Query Cluster Volume Jacob Sisk, Cory Barr December 14, 2007 1 Problem Statement Search engines allow people to find information important to them, and search engine companies derive
CS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu November 16, 2015 Methods to Learn Classification Clustering Frequent Pattern Mining Matrix Data Decision
Entropic Affinities: Properties and Efficient Numerical Computation
Entropic Affinities: Properties and Efficient Numerical Computation Max Vladymyrov and Miguel Carreira-Perpiñán Electrical Engineering and Computer Science University of California, Merced http://eecs.ucmerced.edu
Advanced Machine Learning & Perception
Advanced Machine Learning & Perception Instructor: Tony Jebara Topic 1 Introduction, researchy course, latest papers Going beyond simple machine learning Perception, strange spaces, images, time, behavior
On the conditionality principle in pattern recognition
On the conditionality principle in pattern recognition Carey E. Priebe cep@jhu.edu Johns Hopkins University Department of Applied Mathematics & Statistics Department of Computer Science Center for Imaging
A short introduction to supervised learning, with applications to cancer pathway analysis Dr. Christina Leslie
A short introduction to supervised learning, with applications to cancer pathway analysis Dr. Christina Leslie Computational Biology Program Memorial Sloan-Kettering Cancer Center http://cbio.mskcc.org/leslielab
P leiades: Subspace Clustering and Evaluation
P leiades: Subspace Clustering and Evaluation Ira Assent, Emmanuel Müller, Ralph Krieger, Timm Jansen, and Thomas Seidl Data management and exploration group, RWTH Aachen University, Germany {assent,mueller,krieger,jansen,seidl}@cs.rwth-aachen.de
Spectral Techniques for Clustering
Nicola Rebagliati 1/54 Spectral Techniques for Clustering Nicola Rebagliati 29 April, 2010 Nicola Rebagliati 2/54 Thesis Outline 1 2 Data Representation for Clustering Setting Data Representation and Methods
Principles of Pattern Recognition. C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata
Principles of Pattern Recognition C. A. Murthy Machine Intelligence Unit Indian Statistical Institute Kolkata e-mail: murthy@isical.ac.in Pattern Recognition Measurement Space > Feature Space >Decision
Data Preprocessing. Cluster Similarity
1 Cluster Similarity Similarity is most often measured with the help of a distance function. The smaller the distance, the more similar the data objects (points). A function d: M M R is a distance on M
Data Exploration vis Local Two-Sample Testing
Data Exploration vis Local Two-Sample Testing 0 20 40 60 80 100 40 20 0 20 40 Freeman, Kim, and Lee (2017) Astrostatistics at Carnegie Mellon CMU Astrostatistics Network Graph 2017 (not including collaborations
Chap 1. Overview of Statistical Learning (HTF, , 2.9) Yongdai Kim Seoul National University
Chap 1. Overview of Statistical Learning (HTF, 2.1-2.6, 2.9) Yongdai Kim Seoul National University 0. Learning vs Statistical learning Learning procedure Construct a claim by observing data or using logics
Anomaly (outlier) detection. Huiping Cao, Anomaly 1
Anomaly (outlier) detection Huiping Cao, Anomaly 1 Outline General concepts What are outliers Types of outliers Causes of anomalies Challenges of outlier detection Outlier detection approaches Huiping
STA 414/2104: Machine Learning
STA 414/2104: Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistics! rsalakhu@cs.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 8 Continuous Latent Variable
Non-parametric Methods
Non-parametric Methods Machine Learning Alireza Ghane Non-Parametric Methods Alireza Ghane / Torsten Möller 1 Outline Machine Learning: What, Why, and How? Curve Fitting: (e.g.) Regression and Model Selection
Overview of Statistical Tools. Statistical Inference. Bayesian Framework. Modeling. Very simple case. Things are usually more complicated
Fall 3 Computer Vision Overview of Statistical Tools Statistical Inference Haibin Ling Observation inference Decision Prior knowledge http://www.dabi.temple.edu/~hbling/teaching/3f_5543/index.html Bayesian
Learning from Labeled and Unlabeled Data: Semi-supervised Learning and Ranking p. 1/31
Learning from Labeled and Unlabeled Data: Semi-supervised Learning and Ranking Dengyong Zhou zhou@tuebingen.mpg.de Dept. Schölkopf, Max Planck Institute for Biological Cybernetics, Germany Learning from
Multiscale Wavelets on Trees, Graphs and High Dimensional Data
Multiscale Wavelets on Trees, Graphs and High Dimensional Data ICML 2010, Haifa Matan Gavish (Weizmann/Stanford) Boaz Nadler (Weizmann) Ronald Coifman (Yale) Boaz Nadler Ronald Coifman Motto... the relationships
Comparing Dissimilarity Representations
Comparing Dissimilarity Representations Adam Cardinal-Stakenas Department of Applied Mathematics & Statistics Johns Hopkins University Interface, Durham, NC 1 / 18 Comparing Dissimilarity Representations
CS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2010 Lecture 24: Perceptrons and More! 4/22/2010 Pieter Abbeel UC Berkeley Slides adapted from Dan Klein Announcements W7 due tonight [this is your last written for
Decision T ree Tree Algorithm Week 4 1
Decision Tree Algorithm Week 4 1 Team Homework Assignment #5 Read pp. 105 117 of the text book. Do Examples 3.1, 3.2, 3.3 and Exercise 3.4 (a). Prepare for the results of the homework assignment. Due date
HYPERGRAPH BASED SEMI-SUPERVISED LEARNING ALGORITHMS APPLIED TO SPEECH RECOGNITION PROBLEM: A NOVEL APPROACH
HYPERGRAPH BASED SEMI-SUPERVISED LEARNING ALGORITHMS APPLIED TO SPEECH RECOGNITION PROBLEM: A NOVEL APPROACH Hoang Trang 1, Tran Hoang Loc 1 1 Ho Chi Minh City University of Technology-VNU HCM, Ho Chi
Neural Networks. Single-layer neural network. CSE 446: Machine Learning Emily Fox University of Washington March 10, /9/17
3/9/7 Neural Networks Emily Fox University of Washington March 0, 207 Slides adapted from Ali Farhadi (via Carlos Guestrin and Luke Zettlemoyer) Single-layer neural network 3/9/7 Perceptron as a neural
Data Mining. Preamble: Control Application. Industrial Researcher s Approach. Practitioner s Approach. Example. Example. Goal: Maintain T ~Td
Data Mining Andrew Kusiak 2139 Seamans Center Iowa City, Iowa 52242-1527 Preamble: Control Application Goal: Maintain T ~Td Tel: 319-335 5934 Fax: 319-335 5669 andrew-kusiak@uiowa.edu http://www.icaen.uiowa.edu/~ankusiak
On Improving the k-means Algorithm to Classify Unclassified Patterns
On Improving the k-means Algorithm to Classify Unclassified Patterns Mohamed M. Rizk 1, Safar Mohamed Safar Alghamdi 2 1 Mathematics & Statistics Department, Faculty of Science, Taif University, Taif,
Machine Learning. CUNY Graduate Center, Spring Lectures 11-12: Unsupervised Learning 1. Professor Liang Huang.
Machine Learning CUNY Graduate Center, Spring 2013 Lectures 11-12: Unsupervised Learning 1 (Clustering: k-means, EM, mixture models) Professor Liang Huang huang@cs.qc.cuny.edu http://acl.cs.qc.edu/~lhuang/teaching/machine-learning
Semi-Supervised Learning in Gigantic Image Collections. Rob Fergus (New York University) Yair Weiss (Hebrew University) Antonio Torralba (MIT)
Semi-Supervised Learning in Gigantic Image Collections Rob Fergus (New York University) Yair Weiss (Hebrew University) Antonio Torralba (MIT) Gigantic Image Collections What does the world look like? High
Machine Learning - MT Clustering
Machine Learning - MT 2016 15. Clustering Varun Kanade University of Oxford November 28, 2016 Announcements No new practical this week All practicals must be signed off in sessions this week Firm Deadline:
Semi-Supervised Learning
Semi-Supervised Learning getting more for less in natural language processing and beyond Xiaojin (Jerry) Zhu School of Computer Science Carnegie Mellon University 1 Semi-supervised Learning many human
Classification: The rest of the story
U NIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN CS598 Machine Learning for Signal Processing Classification: The rest of the story 3 October 2017 Today s lecture Important things we haven t covered yet Fisher