RaRE: Social Rank Regulated Large-scale Network Embedding
|
|
- Suzanna Mosley
- 5 years ago
- Views:
Transcription
1 RaRE: Social Rank Regulated Large-scale Network Embedding Authors: Yupeng Gu 1, Yizhou Sun 1, Yanen Li 2, Yang Yang 3 04/26/2018 The Web Conference, University of California, Los Angeles 2 Snapchat Inc. 3 Zhejiang University, China
2 Outline Background Representation learning on networks Traditional models Motivation Our Approach: rank regulated network embedding model How does social rank play a role in link generation? How to model them jointly with dependency? Experiments Does it help improve the quality of traditional embeddings? Can we make better predictions using both embeddings?
3 Outline Background Representation learning on networks Traditional models Motivation Our Approach: rank regulated network embedding model How does social rank play a role in link generation? How to model them jointly with dependency? Experiments Does it help improve the quality of traditional embeddings? Can we make better predictions using both embeddings?
4 Background - Representation Learning on Networks Informative features are crucial in all kinds of machine learning tasks. Features from node attribute Features from network structure Rely on the link information only Generalize well to other domains Less labor-intensive and feature engineering
5 Background - Traditional Representation Models One-hot encoding Too sparse and inefficient. Dimension reduction methods on adjacency matrix, e.g. PCA, multidimension scaling (MDS) Require expensive matrix computations; do not scale well. Block models Limited representation power; scalability. Matrix factorization-based approaches Only model the observed interactions.
6 Outline Background Representation learning on networks Traditional models Motivation Our Approach: rank regulated network embedding model How does social rank play a role in link generation? How to model them jointly with dependency? Experiments Does it help improve the quality of traditional embeddings? Can we make better predictions using both embeddings?
7 Motivation How is the latent representation learned from the network? Latent space model [McPherson 2001] Homophily assumption: connected nodes tend to have more similar representations than random pairs of nodes. Word2vec model [Mikolov 2013], DeepWalk [Perozzi 2014] Two nodes (words) are likely to have similar roles, and thus have similar representations, if they share similar neighbors (context). Later generalized to various models with similar assumptions and architectures (e.g. LINE [Tang 2015], GraRep [Cao 2015], Node2vec [Grover & Leskovec 2016], etc.)
8 Motivation Image courtesy of [Perozzi 2014]
9 Motivation Is this assumption accurate? Case 1. Whom to cite in research paper?
10 Motivation Is this assumption accurate? Case 1. Whom to cite in research paper? Links happen from nodes to Similar nodes Popular nodes
11 Motivation Is this assumption accurate? Case 1. Whom to cite in research paper? Links happen from nodes to Similar nodes Popular nodes ( social rank ) Case 2. Whom to follow on Twitter?
12 Outline Background Representation learning on networks Traditional models Motivation Our Approach: rank regulated network embedding model How does social rank play a role in link generation? How to model them jointly with dependency? Experiments Does it help improve the quality of traditional embeddings? Can we make better predictions using both embeddings?
13 Problem Formulation An information network can be represented as! = ($, &), where, $ = ( ) )*+ is the set of nodes, and & $ $ is the set of edges. Our goal is to infer both: Latent proximity-based embedding vectors / 0 1 $ 3 4. Latent social rank $ 3 6 (smaller value means higher rank, similar to ordinal numbers).
14 Our Approach Base Probabilistic Model Advanced Model Final Model
15 Our Approach - Base Model We aim to explain the existence of a link! "# by the interaction of two nodes proximity-based embeddings $ ", $ # and social ranks & ", & #. The random variable! "# is assumed to be the outcome of a Bernoulli distribution '!&((* "# ), with parameter * "# = *(! "# = 1) =.(& ", & #, $ ", $ # ) where. is a probabilistic function to be defined later.
16 Our Approach Base Probabilistic Model Intuition and findings in related work Advanced Model Final Model
17 Our Approach In addition, we have the following two observations: The probability depends on whether two people share similar opinions or latent social characteristics or not, and the similarity can be measured by the Euclidean distance between two embeddings (! ",! $ ). The probability can also be simplified to the difference of two actors ranks (% ", % $ ) [Ball & Newman, 2013]. Therefore, the previous equation can be rewritten as & "$ = ((% ", % $,! ",! $ ) = ((+%, +,) where +% = % " % $ and +, =! "! $ /
18 Our Approach Base Probabilistic Model Intuition and findings in related work Advanced Model Guess? Final Model
19 Our Approach Base Probabilistic Model Intuition and findings in related work Advanced Model Prior & Posterior distribution Derivation (Bayes theorem) Final Model
20 Our Approach - Posterior Distribution I In order to define!, we think about what are the characteristics of "# and "$ under different circumstances. Why the link is present (% &' = 1): Because the latter is famous * "# "$, % &' = 1 =, - h "$, (Gaussian) We set h "$ = based on several properties. Because they share similar ideas * "$ % &' = 1,(0, ) (truncated Gaussian)
21 Our Approach - Posterior Distribution II When the link is absent (! "# = 0): Since majority of the links are absent, we treat as background and do not put strong assumption on the underlying parameters & '( '),! "# = 0 = + 0,, -. (Gaussian) & ')! "# = 0 +(0,,.. ) (truncated Gaussian) Note that the distribution of ') should be much flatter when the link is absent, thus,.. >, 3..
22 Our Approach Base Probabilistic Model Intuition and findings in related work Advanced Model Prior & Posterior distribution Derivation (Bayes theorem) Final Model
23 Our Approach - Final Model Bayes Theorem:!(# %) = (() *)((*) (()) In our scenario: +,- + /,- = 1 12, , /,- = /,- = 1 +(/,- = 1) = , /,- = /,- = 1 + /,- = , /,- = /,- = 0 +(/,- = 0) = 89:;<91 =,- where 89:;<91 > = 1/(1 + ) and =,- = log E 12 14, /,- = 1 E 14 /,- = 1 E F GH IJ E 12 14, /,- = 0 E 14 /,- = 0. E(F GH IK) Simplifying =,- yields +,- = 89:;<91(L M 12 h 14 L P 14 Q + L K )
24 Our Approach - Final Model Bayes Theorem:!(# %) = (() *)((*) (()) In our scenario: +,- + /,- = 1 12, , /,- = /,- = 1 +(/,- = 1) = , /,- = /,- = 1 + /,- = , /,- = /,- = 0 +(/,- = 0) = 89:;<91 =,- where 89:;<91 > = 1/(1 + ) and =,- = log E 12 14, /,- = 1 E 14 /,- = 1 E F GH IJ E 12 14, /,- = 0 E 14 /,- = 0. E(F GH IK) Simplifying =,- yields +,- = 89:;<91(L M 12 h 14 L P 14 Q + L K )
25 Our Approach - Final Model Bayes Theorem:!(# %) = (() *)((*) (()) In our scenario: +,- + /,- = 1 12, , /,- = /,- = 1 +(/,- = 1) = , /,- = /,- = 1 + /,- = , /,- = /,- = 0 +(/,- = 0) = 89:;<91 =,- where 89:;<91 > = 1/(1 + ) and =,- = log E 12 14, /,- = 1 E 14 /,- = 1 E F GH IJ E 12 14, /,- = 0 E 14 /,- = 0. E(F GH IK) Simplifying =,- yields +,- = 89:;<91(L M 12 h 14 L P 14 Q + L K ) Importance of rank factor Importance of interest factor Sparsity
26 Implementation Misc. Maximum a posteriori (MAP) estimation: arg max log ()*+,-.)- = arg max log 0.1,0.h))3 + arg max log (-.)- Optimized w.r.t. model parameters using stochastic gradient ascent. Negative sampling tricks. Hyper-parameters: only need to care about 5 6 and 5 7. Their range can be easily limited to a reasonable range using intuitions. Scalable: able to handle networks of millions of nodes within minutes on a single machine.
27 Outline Background Representation learning on networks Traditional models Motivation Our Approach: rank regulated network embedding model How does social rank play a role in link generation? How to model them jointly with dependency? Experiments Does it help improve the quality of traditional embeddings? Can we make better predictions using both embeddings?
28 Experiments Baselines: Matrix factorization techniques for recommender systems (MF) Graph Factorization (GF) Large-scale information network embedding (LINE) Node2vec Rank-regulated network embedding (RaRE)
29 Experiments - Tasks Node level: multi-class / multi-label classification Edge level: link prediction Case study and visualization
30 Experiments - Tasks Node level: multi-class / multi-label classification Jaccard Index Hamming Loss F1 score Edge level: link prediction Area under the ROC curve (AUC) Case study and visualization
31
32 Experiments - Tasks Node level: multi-class / multi-label classification Jaccard Index Hamming Loss F1 score Edge level: link prediction Area under the ROC curve (AUC) Case study and visualization
33
34 Experiments - Tasks Node level: multi-class / multi-label classification Jaccard Index Hamming Loss F1 score Edge level: link prediction Area under the ROC curve (AUC) Case study and visualization
35 Visualization of CS Venues
36 Conclusion A brand new network embedding approach considering both proximity-based factor and social rank factor. Outstanding performance on various prediction tasks. Provide a solid reasoning about the link generation, transparent white box model and high explanability. Scalable to several real-world large-scale networks. Training can be done within minutes for networks of millions of nodes/edges, on a single machine.
37 Reference [McPherson 2001] M. McPherson, L. Smith-Lovin, and J. M. Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, pages , [Ball & Newman 2013] B. Ball and M. E. Newman, Friendship networks and social status. Network Science, 1(01):16 30, 2013 [Mikolov 2013] Mikolov, Tomas, et al. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems [Perozzi 2014] B. Perozzi, R. Al-Rfou, and S. Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages ACM, 2014 [Tang 2015] J. Tang, M. Qu, M. Wang, M. Zhang, J. Yan, and Q. Mei. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web, pages ACM, 2015 [Cao 2015] S. Cao, W. Lu, and Q. Xu. Grarep: Learning graph representations with global structural information. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pages ACM, 2015 [Grover & Leskovec 2016] A. Grover and J. Leskovec. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages ACM, 2016
38 Questions? Contact: Yupeng Gu
Community Preserving Network Embedding
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) Community Preserving Network Embedding Xiao Wang, 1 Peng Cui, 1 Jing Wang, 2 Jian Pei, 3 Wenwu Zhu, 1 Shiqiang Yang
More informationGlobal and Local Feature Learning for Ego-Network Analysis
Global and Local Feature Learning for Ego-Network Analysis Fatemeh Salehi Rizi Michael Granitzer and Konstantin Ziegler TIR Workshop 29 August 2017 (TIR Workshop) University of Passau 29 August 2017 1
More informationword2vec Parameter Learning Explained
word2vec Parameter Learning Explained Xin Rong ronxin@umich.edu Abstract The word2vec model and application by Mikolov et al. have attracted a great amount of attention in recent two years. The vector
More informationLink Prediction. Eman Badr Mohammed Saquib Akmal Khan
Link Prediction Eman Badr Mohammed Saquib Akmal Khan 11-06-2013 Link Prediction Which pair of nodes should be connected? Applications Facebook friend suggestion Recommendation systems Monitoring and controlling
More informationL 2,1 Norm and its Applications
L 2, Norm and its Applications Yale Chang Introduction According to the structure of the constraints, the sparsity can be obtained from three types of regularizers for different purposes.. Flat Sparsity.
More informationJure Leskovec Joint work with Jaewon Yang, Julian McAuley
Jure Leskovec (@jure) Joint work with Jaewon Yang, Julian McAuley Given a network, find communities! Sets of nodes with common function, role or property 2 3 Q: How and why do communities form? A: Strength
More informationData Mining & Machine Learning
Data Mining & Machine Learning CS57300 Purdue University April 10, 2018 1 Predicting Sequences 2 But first, a detour to Noise Contrastive Estimation 3 } Machine learning methods are much better at classifying
More informationA Shrinkage Approach for Modeling Non-Stationary Relational Autocorrelation
A Shrinkage Approach for Modeling Non-Stationary Relational Autocorrelation Pelin Angin Department of Computer Science Purdue University pangin@cs.purdue.edu Jennifer Neville Department of Computer Science
More informationarxiv: v2 [cs.cl] 1 Jan 2019
Variational Self-attention Model for Sentence Representation arxiv:1812.11559v2 [cs.cl] 1 Jan 2019 Qiang Zhang 1, Shangsong Liang 2, Emine Yilmaz 1 1 University College London, London, United Kingdom 2
More informationNaïve Bayes classification
Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss
More informationOverlapping Community Detection at Scale: A Nonnegative Matrix Factorization Approach
Overlapping Community Detection at Scale: A Nonnegative Matrix Factorization Approach Author: Jaewon Yang, Jure Leskovec 1 1 Venue: WSDM 2013 Presenter: Yupeng Gu 1 Stanford University 1 Background Community
More informationMIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October,
MIDTERM: CS 6375 INSTRUCTOR: VIBHAV GOGATE October, 23 2013 The exam is closed book. You are allowed a one-page cheat sheet. Answer the questions in the spaces provided on the question sheets. If you run
More informationSupporting Statistical Hypothesis Testing Over Graphs
Supporting Statistical Hypothesis Testing Over Graphs Jennifer Neville Departments of Computer Science and Statistics Purdue University (joint work with Tina Eliassi-Rad, Brian Gallagher, Sergey Kirshner,
More informationCS230: Lecture 8 Word2Vec applications + Recurrent Neural Networks with Attention
CS23: Lecture 8 Word2Vec applications + Recurrent Neural Networks with Attention Today s outline We will learn how to: I. Word Vector Representation i. Training - Generalize results with word vectors -
More informationProbabilistic Machine Learning. Industrial AI Lab.
Probabilistic Machine Learning Industrial AI Lab. Probabilistic Linear Regression Outline Probabilistic Classification Probabilistic Clustering Probabilistic Dimension Reduction 2 Probabilistic Linear
More informationApplying Latent Dirichlet Allocation to Group Discovery in Large Graphs
Lawrence Livermore National Laboratory Applying Latent Dirichlet Allocation to Group Discovery in Large Graphs Keith Henderson and Tina Eliassi-Rad keith@llnl.gov and eliassi@llnl.gov This work was performed
More informationNaïve Bayes classification. p ij 11/15/16. Probability theory. Probability theory. Probability theory. X P (X = x i )=1 i. Marginal Probability
Probability theory Naïve Bayes classification Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. s: A person s height, the outcome of a coin toss Distinguish
More informationThe Forward-Backward Embedding of Directed Graphs
The Forward-Backward Embedding of Directed Graphs Anonymous authors Paper under double-blind review Abstract We introduce a novel embedding of directed graphs derived from the singular value decomposition
More informationECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Oct 18, 2016 Outline One versus all/one versus one Ranking loss for multiclass/multilabel classification Scaling to millions of labels Multiclass
More informationClassification. Classification is similar to regression in that the goal is to use covariates to predict on outcome.
Classification Classification is similar to regression in that the goal is to use covariates to predict on outcome. We still have a vector of covariates X. However, the response is binary (or a few classes),
More informationECS289: Scalable Machine Learning
ECS289: Scalable Machine Learning Cho-Jui Hsieh UC Davis Oct 27, 2015 Outline One versus all/one versus one Ranking loss for multiclass/multilabel classification Scaling to millions of labels Multiclass
More informationPrediction of Citations for Academic Papers
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationDeep Learning for NLP Part 2
Deep Learning for NLP Part 2 CS224N Christopher Manning (Many slides borrowed from ACL 2012/NAACL 2013 Tutorials by me, Richard Socher and Yoshua Bengio) 2 Part 1.3: The Basics Word Representations The
More informationECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
More informationOverlapping Communities
Overlapping Communities Davide Mottin HassoPlattner Institute Graph Mining course Winter Semester 2017 Acknowledgements Most of this lecture is taken from: http://web.stanford.edu/class/cs224w/slides GRAPH
More informationSemantics with Dense Vectors. Reference: D. Jurafsky and J. Martin, Speech and Language Processing
Semantics with Dense Vectors Reference: D. Jurafsky and J. Martin, Speech and Language Processing 1 Semantics with Dense Vectors We saw how to represent a word as a sparse vector with dimensions corresponding
More informationIntroduction to Probabilistic Machine Learning
Introduction to Probabilistic Machine Learning Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course 1) Nov 03, 2015 Piyush Rai (IIT Kanpur) Introduction to Probabilistic Machine Learning 1 Machine Learning
More informationCPSC 340: Machine Learning and Data Mining. More PCA Fall 2017
CPSC 340: Machine Learning and Data Mining More PCA Fall 2017 Admin Assignment 4: Due Friday of next week. No class Monday due to holiday. There will be tutorials next week on MAP/PCA (except Monday).
More informationFrom perceptrons to word embeddings. Simon Šuster University of Groningen
From perceptrons to word embeddings Simon Šuster University of Groningen Outline A basic computational unit Weighting some input to produce an output: classification Perceptron Classify tweets Written
More informationVariational Autoencoder
Variational Autoencoder Göker Erdo gan August 8, 2017 The variational autoencoder (VA) [1] is a nonlinear latent variable model with an efficient gradient-based training procedure based on variational
More informationCSC321 Lecture 18: Learning Probabilistic Models
CSC321 Lecture 18: Learning Probabilistic Models Roger Grosse Roger Grosse CSC321 Lecture 18: Learning Probabilistic Models 1 / 25 Overview So far in this course: mainly supervised learning Language modeling
More informationAlgorithm-Independent Learning Issues
Algorithm-Independent Learning Issues Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2007 c 2007, Selim Aksoy Introduction We have seen many learning
More informationContent-based Recommendation
Content-based Recommendation Suthee Chaidaroon June 13, 2016 Contents 1 Introduction 1 1.1 Matrix Factorization......................... 2 2 slda 2 2.1 Model................................. 3 3 flda 3
More informationSparse vectors recap. ANLP Lecture 22 Lexical Semantics with Dense Vectors. Before density, another approach to normalisation.
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Previous lectures: Sparse vectors recap How to represent
More informationANLP Lecture 22 Lexical Semantics with Dense Vectors
ANLP Lecture 22 Lexical Semantics with Dense Vectors Henry S. Thompson Based on slides by Jurafsky & Martin, some via Dorota Glowacka 5 November 2018 Henry S. Thompson ANLP Lecture 22 5 November 2018 Previous
More informationIterative Laplacian Score for Feature Selection
Iterative Laplacian Score for Feature Selection Linling Zhu, Linsong Miao, and Daoqiang Zhang College of Computer Science and echnology, Nanjing University of Aeronautics and Astronautics, Nanjing 2006,
More informationPart-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287
Part-of-Speech Tagging + Neural Networks 3: Word Embeddings CS 287 Review: Neural Networks One-layer multi-layer perceptron architecture, NN MLP1 (x) = g(xw 1 + b 1 )W 2 + b 2 xw + b; perceptron x is the
More informationhsnim: Hyper Scalable Network Inference Machine for Scale-Free Protein-Protein Interaction Networks Inference
CS 229 Project Report (TR# MSB2010) Submitted 12/10/2010 hsnim: Hyper Scalable Network Inference Machine for Scale-Free Protein-Protein Interaction Networks Inference Muhammad Shoaib Sehgal Computer Science
More informationDistance Metric Learning in Data Mining (Part II) Fei Wang and Jimeng Sun IBM TJ Watson Research Center
Distance Metric Learning in Data Mining (Part II) Fei Wang and Jimeng Sun IBM TJ Watson Research Center 1 Outline Part I - Applications Motivation and Introduction Patient similarity application Part II
More informationLink Prediction with Personalized Social Influence
Link Prediction with Personalized Social Influence Zepeng Huo 1, Xiao Huang 1, Xia Hu 1,2 1 Department of Computer Science and Engineering, Texas A&M University 2 Center for Remote Health Technologies
More informationECE521 Lecture7. Logistic Regression
ECE521 Lecture7 Logistic Regression Outline Review of decision theory Logistic regression A single neuron Multi-class classification 2 Outline Decision theory is conceptually easy and computationally hard
More informationCollaborative topic models: motivations cont
Collaborative topic models: motivations cont Two topics: machine learning social network analysis Two people: " boy Two articles: article A! girl article B Preferences: The boy likes A and B --- no problem.
More informationDynamic Probabilistic Models for Latent Feature Propagation in Social Networks
Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks Creighton Heaukulani and Zoubin Ghahramani University of Cambridge TU Denmark, June 2013 1 A Network Dynamic network data
More informationMODULE -4 BAYEIAN LEARNING
MODULE -4 BAYEIAN LEARNING CONTENT Introduction Bayes theorem Bayes theorem and concept learning Maximum likelihood and Least Squared Error Hypothesis Maximum likelihood Hypotheses for predicting probabilities
More informationNetwork Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec Jiezhong Qiu Tsinghua University February 21, 2018 Joint work with Yuxiao Dong (MSR), Hao Ma (MSR), Jian Li (IIIS,
More informationarxiv: v3 [cs.cl] 30 Jan 2016
word2vec Parameter Learning Explained Xin Rong ronxin@umich.edu arxiv:1411.2738v3 [cs.cl] 30 Jan 2016 Abstract The word2vec model and application by Mikolov et al. have attracted a great amount of attention
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Mining Graph/Network Data Instructor: Yizhou Sun yzsun@ccs.neu.edu March 16, 2016 Methods to Learn Classification Clustering Frequent Pattern Mining Matrix Data Decision
More informationFeature Engineering, Model Evaluations
Feature Engineering, Model Evaluations Giri Iyengar Cornell University gi43@cornell.edu Feb 5, 2018 Giri Iyengar (Cornell Tech) Feature Engineering Feb 5, 2018 1 / 35 Overview 1 ETL 2 Feature Engineering
More informationMeasuring Segregation in Social Networks
Measuring Segregation in Social Networks Micha l Bojanowski Rense Corten ICS/Sociology, Utrecht University July 2, 2010 Sunbelt XXX, Riva del Garda Outline 1 Introduction Homophily and segregation 2 Problem
More informationCPSC 340: Machine Learning and Data Mining. MLE and MAP Fall 2017
CPSC 340: Machine Learning and Data Mining MLE and MAP Fall 2017 Assignment 3: Admin 1 late day to hand in tonight, 2 late days for Wednesday. Assignment 4: Due Friday of next week. Last Time: Multi-Class
More informationLearning to translate with neural networks. Michael Auli
Learning to translate with neural networks Michael Auli 1 Neural networks for text processing Similar words near each other France Spain dog cat Neural networks for text processing Similar words near each
More information6.047 / Computational Biology: Genomes, Networks, Evolution Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.047 / 6.878 Computational Biology: Genomes, etworks, Evolution Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationThe exam is closed book, closed notes except your one-page cheat sheet.
CS 189 Fall 2015 Introduction to Machine Learning Final Please do not turn over the page before you are instructed to do so. You have 2 hours and 50 minutes. Please write your initials on the top-right
More informationECE521 Lectures 9 Fully Connected Neural Networks
ECE521 Lectures 9 Fully Connected Neural Networks Outline Multi-class classification Learning multi-layer neural networks 2 Measuring distance in probability space We learnt that the squared L2 distance
More informationA graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
More informationCompressed Sensing and Neural Networks
and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications
More informationProbabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
More informationDeep Poisson Factorization Machines: a factor analysis model for mapping behaviors in journalist ecosystem
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
More informationSigned Network Embedding in Social Media
Signed Network Embedding in Social Media Suhang Wang Jiliang Tang Charu Aggarwal Yi Chang Huan Liu Abstract Network embedding is to learn lowdimensional vector representations for nodes of a given social
More informationIntroduction: MLE, MAP, Bayesian reasoning (28/8/13)
STA561: Probabilistic machine learning Introduction: MLE, MAP, Bayesian reasoning (28/8/13) Lecturer: Barbara Engelhardt Scribes: K. Ulrich, J. Subramanian, N. Raval, J. O Hollaren 1 Classifiers In this
More informationReview: Probabilistic Matrix Factorization. Probabilistic Matrix Factorization (PMF)
Case Study 4: Collaborative Filtering Review: Probabilistic Matrix Factorization Machine Learning for Big Data CSE547/STAT548, University of Washington Emily Fox February 2 th, 214 Emily Fox 214 1 Probabilistic
More informationCS-E3210 Machine Learning: Basic Principles
CS-E3210 Machine Learning: Basic Principles Lecture 4: Regression II slides by Markus Heinonen Department of Computer Science Aalto University, School of Science Autumn (Period I) 2017 1 / 61 Today s introduction
More informationLogistic Regression: Online, Lazy, Kernelized, Sequential, etc.
Logistic Regression: Online, Lazy, Kernelized, Sequential, etc. Harsha Veeramachaneni Thomson Reuter Research and Development April 1, 2010 Harsha Veeramachaneni (TR R&D) Logistic Regression April 1, 2010
More informationDiscrete Latent Variable Models
Discrete Latent Variable Models Stefano Ermon, Aditya Grover Stanford University Lecture 14 Stefano Ermon, Aditya Grover (AI Lab) Deep Generative Models Lecture 14 1 / 29 Summary Major themes in the course
More informationGenerative Models for Sentences
Generative Models for Sentences Amjad Almahairi PhD student August 16 th 2014 Outline 1. Motivation Language modelling Full Sentence Embeddings 2. Approach Bayesian Networks Variational Autoencoders (VAE)
More informationBeyond the Point Cloud: From Transductive to Semi-Supervised Learning
Beyond the Point Cloud: From Transductive to Semi-Supervised Learning Vikas Sindhwani, Partha Niyogi, Mikhail Belkin Andrew B. Goldberg goldberg@cs.wisc.edu Department of Computer Sciences University of
More informationNonparametric Bayesian Matrix Factorization for Assortative Networks
Nonparametric Bayesian Matrix Factorization for Assortative Networks Mingyuan Zhou IROM Department, McCombs School of Business Department of Statistics and Data Sciences The University of Texas at Austin
More informationAn overview of word2vec
An overview of word2vec Benjamin Wilson Berlin ML Meetup, July 8 2014 Benjamin Wilson word2vec Berlin ML Meetup 1 / 25 Outline 1 Introduction 2 Background & Significance 3 Architecture 4 CBOW word representations
More informationDimensionality Reduction
Lecture 5 1 Outline 1. Overview a) What is? b) Why? 2. Principal Component Analysis (PCA) a) Objectives b) Explaining variability c) SVD 3. Related approaches a) ICA b) Autoencoders 2 Example 1: Sportsball
More informationCollaborative Topic Modeling for Recommending Scientific Articles
Collaborative Topic Modeling for Recommending Scientific Articles Chong Wang and David M. Blei Best student paper award at KDD 2011 Computer Science Department, Princeton University Presented by Tian Cao
More informationExploiting Local and Global Social Context for Recommendation
Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence Exploiting Local and Global Social Context for Recommendation Jiliang Tang, Xia Hu, Huiji Gao, Huan Liu Computer
More informationMachine Learning! in just a few minutes. Jan Peters Gerhard Neumann
Machine Learning! in just a few minutes Jan Peters Gerhard Neumann 1 Purpose of this Lecture Foundations of machine learning tools for robotics We focus on regression methods and general principles Often
More informationLecture 10: Logistic Regression
BIOINF 585: Machine Learning for Systems Biology & Clinical Informatics Lecture 10: Logistic Regression Jie Wang Department of Computational Medicine & Bioinformatics University of Michigan 1 Outline An
More informationRecommendation Systems
Recommendation Systems Popularity Recommendation Systems Predicting user responses to options Offering news articles based on users interests Offering suggestions on what the user might like to buy/consume
More informationCollaborative Filtering. Radek Pelánek
Collaborative Filtering Radek Pelánek 2017 Notes on Lecture the most technical lecture of the course includes some scary looking math, but typically with intuitive interpretation use of standard machine
More informationApproximate Inference Part 1 of 2
Approximate Inference Part 1 of 2 Tom Minka Microsoft Research, Cambridge, UK Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ Bayesian paradigm Consistent use of probability theory
More informationAlgorithmisches Lernen/Machine Learning
Algorithmisches Lernen/Machine Learning Part 1: Stefan Wermter Introduction Connectionist Learning (e.g. Neural Networks) Decision-Trees, Genetic Algorithms Part 2: Norman Hendrich Support-Vector Machines
More informationApproximate Inference Part 1 of 2
Approximate Inference Part 1 of 2 Tom Minka Microsoft Research, Cambridge, UK Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ 1 Bayesian paradigm Consistent use of probability theory
More informationTopic Modeling Using Latent Dirichlet Allocation (LDA)
Topic Modeling Using Latent Dirichlet Allocation (LDA) Porter Jenkins and Mimi Brinberg Penn State University prj3@psu.edu mjb6504@psu.edu October 23, 2017 Porter Jenkins and Mimi Brinberg (PSU) LDA October
More informationUnsupervised Learning
CS 3750 Advanced Machine Learning hkc6@pitt.edu Unsupervised Learning Data: Just data, no labels Goal: Learn some underlying hidden structure of the data P(, ) P( ) Principle Component Analysis (Dimensionality
More informationBayesian Paragraph Vectors
Bayesian Paragraph Vectors Geng Ji 1, Robert Bamler 2, Erik B. Sudderth 1, and Stephan Mandt 2 1 Department of Computer Science, UC Irvine, {gji1, sudderth}@uci.edu 2 Disney Research, firstname.lastname@disneyresearch.com
More informationIntroduction to Machine Learning. PCA and Spectral Clustering. Introduction to Machine Learning, Slides: Eran Halperin
1 Introduction to Machine Learning PCA and Spectral Clustering Introduction to Machine Learning, 2013-14 Slides: Eran Halperin Singular Value Decomposition (SVD) The singular value decomposition (SVD)
More informationWeb Structure Mining Nodes, Links and Influence
Web Structure Mining Nodes, Links and Influence 1 Outline 1. Importance of nodes 1. Centrality 2. Prestige 3. Page Rank 4. Hubs and Authority 5. Metrics comparison 2. Link analysis 3. Influence model 1.
More informationRepresentation Learning in Continuous Entity-Set Associations
Representation Learning in Continuous Entity-Set Associations ABSTRACT Srijan Kumar srijan@cs.stanford.edu Modeling a continuous sequence of associations between entities and sets of entities is crucial
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 218 Outlines Overview Introduction Linear Algebra Probability Linear Regression 1
More informationMining Triadic Closure Patterns in Social Networks
Mining Triadic Closure Patterns in Social Networks Hong Huang, University of Goettingen Jie Tang, Tsinghua University Sen Wu, Stanford University Lu Liu, Northwestern University Xiaoming Fu, University
More informationDeep Learning for NLP
Deep Learning for NLP CS224N Christopher Manning (Many slides borrowed from ACL 2012/NAACL 2013 Tutorials by me, Richard Socher and Yoshua Bengio) Machine Learning and NLP NER WordNet Usually machine learning
More informationVariational Autoencoders
Variational Autoencoders Recap: Story so far A classification MLP actually comprises two components A feature extraction network that converts the inputs into linearly separable features Or nearly linearly
More informationLearning a Degree-Augmented Distance Metric from a Network. Bert Huang, U of Maryland Blake Shaw, Foursquare Tony Jebara, Columbia U
Learning a Degree-Augmented Distance Metric from a Network Bert Huang, U of Maryland Blake Shaw, Foursquare Tony Jebara, Columbia U Beyond Mahalanobis: Supervised Large-Scale Learning of Similarity NIPS
More information11/3/15. Deep Learning for NLP. Deep Learning and its Architectures. What is Deep Learning? Advantages of Deep Learning (Part 1)
11/3/15 Machine Learning and NLP Deep Learning for NLP Usually machine learning works well because of human-designed representations and input features CS224N WordNet SRL Parser Machine learning becomes
More informationDescriptive Data Summarization
Descriptive Data Summarization Descriptive data summarization gives the general characteristics of the data and identify the presence of noise or outliers, which is useful for successful data cleaning
More informationOrthogonal Nonnegative Matrix Factorization: Multiplicative Updates on Stiefel Manifolds
Orthogonal Nonnegative Matrix Factorization: Multiplicative Updates on Stiefel Manifolds Jiho Yoo and Seungjin Choi Department of Computer Science Pohang University of Science and Technology San 31 Hyoja-dong,
More informationCSC 576: Variants of Sparse Learning
CSC 576: Variants of Sparse Learning Ji Liu Department of Computer Science, University of Rochester October 27, 205 Introduction Our previous note basically suggests using l norm to enforce sparsity in
More informationNeutron inverse kinetics via Gaussian Processes
Neutron inverse kinetics via Gaussian Processes P. Picca Politecnico di Torino, Torino, Italy R. Furfaro University of Arizona, Tucson, Arizona Outline Introduction Review of inverse kinetics techniques
More informationDeep Convolutional Neural Networks for Pairwise Causality
Deep Convolutional Neural Networks for Pairwise Causality Karamjit Singh, Garima Gupta, Lovekesh Vig, Gautam Shroff, and Puneet Agarwal TCS Research, Delhi Tata Consultancy Services Ltd. {karamjit.singh,
More informationNotes on Machine Learning for and
Notes on Machine Learning for 16.410 and 16.413 (Notes adapted from Tom Mitchell and Andrew Moore.) Choosing Hypotheses Generally want the most probable hypothesis given the training data Maximum a posteriori
More informationTensor Methods for Feature Learning
Tensor Methods for Feature Learning Anima Anandkumar U.C. Irvine Feature Learning For Efficient Classification Find good transformations of input for improved classification Figures used attributed to
More informationFast Nonnegative Matrix Factorization with Rank-one ADMM
Fast Nonnegative Matrix Factorization with Rank-one Dongjin Song, David A. Meyer, Martin Renqiang Min, Department of ECE, UCSD, La Jolla, CA, 9093-0409 dosong@ucsd.edu Department of Mathematics, UCSD,
More informationData Mining & Machine Learning
Data Mining & Machine Learning CS57300 Purdue University March 1, 2018 1 Recap of Last Class (Model Search) Forward and Backward passes 2 Feedforward Neural Networks Neural Networks: Architectures 2-layer
More information