o A set of very exciting (to me, may be others) questions at the interface of all of the above and more
|
|
- Sharon Atkinson
- 6 years ago
- Views:
Transcription
1 Devavrat Shah Laboratory for Information and Decision Systems Department of EECS Massachusetts Institute of Technology (list of relevant references in the last set of slides)
2 o Ideally o Graphical models o Belief propagation o Connections to Probability, Statistics, EE, CS, o In reality o A set of very exciting (to me, may be others) questions at the interface of all of the above and more o Seemingly unrelated to graphical model o However, provide fertile ground to understand everything about graphical models (algorithms, analysis)
3 o Recommendations o What movie to watch o Which restaurant to eat o o Precisely, o Suggest what you may like o Given what others have liked o By finding others like you and what they had liked
4 o Ranking o Players and/or Teams o Based on outcome of games o Papers at a competitive conference o Using reviews o Graduate admissions o From feedback of professors o Precisely, o Global ranking of objects from partial preferences
5 o Partial preferences are revealed in different forms o Sports: Win and Loss o Social: Starred rating o Conferences: Scores o All can be viewed as pair- wise comparisons o IND beats AUS: IND AUS o Clio ***** vs No 9 Park ****: Clio No 9 Park o Ranking Paper 9/10 vs Other Paper 5/10: Ranking Other
6 o Revealed preferences lead to o Bag of pair- wise comparisons o Question of interest o Recommendations o Suggest what you may like given what others have liked o Ranking o Global ranking of objects given outcome of games/ Ø This requires understanding (computing) choice model o What people like/dislike from pair- wise comparisons
7 o Rational view: Axiom of revealed preferences [Samuelson 37] o There is one ordering over all objects consistent across population o Unlikely (lack of transitivity in people s preferences) o Meaningful view discrete choice model o Distribution over orderings of objects o consistent with population s revealed preferences Data Choice Model Decision B B A B C C B C A A C A A B B C C A B C A
8 o Object tracking (cf. Huang, Guestrin, Guibas 08) o Noisy observations of locations o Feasible to maintain partial information only o Q=[Q ij ] first- order information 1 2 Q 11 = P(1è P1) Q 13 Q 12 P1 P2 3 P3 Objects Locations
9 o Object tracking o Noisy observations of locations o Feasible to maintain partial information only o Q=[Q ij ] first- order information Data Choice Model Decision 1 2 Q 11 Q 12 Q 13 P1 P2 1 2 P1 P2 1 2 P1 P2 3 P3 3 P3 3 P3
10 o Recommendation o Ranking o Object tracking o Policy making o Business operations (assortment optimization) o Display advertising o Polling, o Canonical question o Decision using choice model learnt from partial preference data
11 1 2 Q 11 Q 12 Q 13 P1 P2 3 P3
12 Q 11 1 Q 12 P1 Q 13 2 P2 3 P3 o Q. Given weighted bipartite graph G=(V, E, Q) o Find matching of objects/positions o That is `most likely
13 1 P1 2 P2 3 P3 o Answer: maximum weight matching o Weight of a matching equals o summation of Q- entries of edges participating in the matching
14 A B C C B C A A
15 # times 1 defeats 2 1 A 12 A o Q1. Given weighted comparison graph G=(V, E, A) o Find ranking of/scores associated with objects o Q2. When possible (e.g. Conference/Crowd- Sourcing), choose G so as to o Minimize the number of comparisons required to find ranking/scores
16 1 A 12 A o Random walk on comparison graph G=(V,E,A) o d = max (undirected) vertex degree of G o For each edge (i,j): o P ij = (A ji +1)/(A ij +A ij +2) x 1/d o For each node i: o P ii = 1- Σ j i P ij o Let G be connected o Let s be the unique stationary distribution of RW P o Ranking: o Use s as scores of objects s T = s T P 5 4 3
17 1 A 12 A o Random walk on comparison graph G=(V,E,A) o d = max (undirected) vertex degree of G o For each edge (i,j): o P ij = (A ji +1)/(A ij +A ij +2) x 1/d o For each node i: o P ii = 1- Σ j i P ij o Ranking: use s as scores of objects, where o s be the unique stationary distribution of RW P o Choice of graph G s T = s T P o Subject to constraints, choose G so that o Spectral gap of natural RW on G is maximized o SDP [Boyd, Diaconis, Xiao 04] 5 4 3
18 1 2 Q 11 Q 12 Q 13 P1 P2 A B C B C A 3 P3 C A o Maximum Weight Matching o How to compute it? o Belief propagation o Why does it make sense? o Max- likelihood estimation w.r.t. exponential family o Rank centrality o How to compute it? o Power- iteration o Why does it make sense? o Mode for Bradley- Terry- Luce (or MNL) model
19 1 2 Q 11 Q 12 Q 13 P1 P2 3 P3
20 (all of below explained using class- board) o Computation o Belief propagation o Algorithm o Why it works o Model o Maximum entropy (max- ent) consistent distribution o Maximum Likelihood in exponential family o Maximum weight matching o First- order approximation of mode of this distribution o Exact computation of max- ent o Via dual gradient o Belief propagation/mcmc at rescue
21 A B C C B C A A
22 o Choice model (distribution over permutations) [Bradley- Terry- Luce (BTL) or MNL (cf. McFadden) Model] o Each object i has an associated weight w i 0 o When objects i and j are compared o P(i j) = w i /(w i + w j ) o Sampling model o Edges E of graph G are selected o For each (i,j) ε E, sample k pair- wise comparisons
23 o Error(s) = 1 w ( (w i -w j ) 2 I {(s(i)-s(j))(w i -w j )<0 ij }) 1/2 o G: Erdos- Renyi graph with edge prob. d/n 0.1 Ratio Matrix L1 ranking Rank Centrality ML estimate 0.1 Ratio Matrix L1 ranking Rank Centrality ML estimate k d/n
24 o Theorem 1. o Let R= (max ij w i /w j ). o Let G be Erdos- Renyi graph. o Under Rank centrality, with d = Ω(log n) s-w w C R 5 log n kd o That is, sufficient to have O(R 5 n log n) samples o Optimal dependence on n for ER graph o Dependence on R?
25 o Theorem 1. o Let R= (max ij w i /w j ). o Let G be Erdos- Renyi graph. o Under Rank centrality, with d = Ω(log n) s-w w C R 5 log n kd o Information theoretic lower- bound: for any algorithm s-w w C' 1 kd
26 o Theorem 2. o Let R= (max ij w i /w j ). o Let G be any connected graph: o L = D - 1 E be natural RW transition matrix o Δ = 1- λ max (L) o κ = d max /d min o Under Rank centrality, with kd = Ω(log n) s-w w C Δ κ R5 log n kd o That is, number of samples required O(R 5 κ 2 n log n x Δ - 2 ) o Graph structure plays role through it s Laplacian
27 o Theorem 2. o Under Rank centrality, with kd = Ω(log n) s-w w C Δ κ R5 log n kd o That is, number of samples required O(R 5 κ 2 n log n x Δ - 2 ) o Choice of graph G o Subject to constraints, choose G so that o Spectral gap Δ is maximized o SDP [Boyd, Diaconis, Xiao 04]
28
29
30 o Bound on o Use of comparison theorem [Diaconis- Saloff Coste 94]++ o Bound on o Use of (modified) concentration of measure inequality for matrices o Finally, use this to further bound Error(s)
31 Washington Post: Allourideas
32 o Ground truth: algorithm s result with complete data o Error: average position discrepancy L1 ranking Rank Centrality
33 o Input: complete preference (not comparisons) o Axiomatic impossibility [Arrow 51] o Some algorithms o Kemeny optimal: minimize disagreements o Extended Condorcet Criteria o NP- hard, 2- approx algorithm [Dwork et al 01] A A 12 3 o Borda count: average position is score o Simple o Useful axiomatic properties [Young 74]
34 o Algorithms with partial data o Let pair- wise data available for all pairs o Kemeny distance depends on pair- wise marginal only argmin σ A ij Ι(σ (i) < σ (j)) o Data is consistent with a distribution on permutations o For example, obtained as the max- ent approximation o Kemeny optimal of this distribution is the same as above o NP- hard o 2- approx for this distribution acts as 2- approx for the above [Ammar- Shah 11 12]
35 o Borda count o Average position o But, comparison do not have position information o Given pair- wise marginal p ij for all i j o For any distribution consistent with pair- wise marginal o Borda count is given as c(i) p j ij [Ammar- Shah 11 12]
36 o Finding winner and BTL choice model [Adler, Gemmell, Harchol- Balter, Karp, Kenyon 87] o O(log n) iteration adaptive algorithm, O(n) total comparisons o Noisy sorting and Mallow s model [Braverman, Mossel 09] o O(n log n) samples in total (complete ordering) o Average position (Borda count) algorithm++ o Polynomial(n) time algorithm
37 o Choice model o A powerful model to tackle a range of questions o Many are in it s infancy (e.g. recommendations) o Challenge being computation + statistics o Excellent play ground to resolve challenges of graphical models o Two examples in this tutorial o Object tracking o Learning from first- order marginals o Ranking o Using pair- wise comparisons
38 o Open direction: o Learning graphical models efficiently o Computationally and statistically o A concrete question: o Given pair- wise comparisons data o When can we learn the choice model efficiently? o For example, if exact pair- wise comparison marginals available o Then, can learn `sparse choice model up to o(log n) sparsity [Farias + Jagabathula + S 09 12] o But what about noisy setting? o Or, max- ent learning?
39 o Part I: o A. Ammar, D. Shah, ``Efficient rank aggregation from partial data, Proceedings of ACM Sigmetrics o M. Bayati, D. Shah, M. Sharma, ``Max- product for maximum weight matching: convergence, correctness and LP duality, IEEE Transactions on Information Theory, o S. Jagabathula, D. Shah, ``Inferring ranking using constrained sensing, IEEE Transactions on Information Theory, o S. Agrawal, Z. Wang, Y. Ye, ``Parimutuel betting on permutations, Internet and Network Economics, o J. Huang, C. Guestrin, L. Guibas, ``Fourier theoretic probabilistic inference over permutations, Journal of Machine Learning Research, Color coding covered in tutorial sparse choice model that is v. relevant others/ closely related/definitely worth a read
40 o Part II: o S. Negahban, S. Oh, D. Shah, ``Iterative ranking using pair- wise comparisons, Proceedings of NIPS o V. Farias, S. Jagabathula, D. Shah, ``Data driven approach to modeling choice, Proceedings of NIPS, Also Management Science, 2012 (and Arxiv). o V. Farias, S. Jagabathula, D. Shah, ``Sparse choice model, available on Arxiv, o A. Ammar, D. Shah, ``Compare, don t score, Proceedings of Allerton, Color coding covered in tutorial sparse choice model that is v. relevant others/ closely related/definitely worth a read
41 o At large: o H. Varian, ``Revealed preferences, Samuelsonian economics and the twenty- first century, o D. McFadden, ``Disaggregate Behavioral Travel Demand s RUM Side, A 30- year Retrospective, available online o P. Diaconis, ``Group representation in probability and statistics, Lecture notes- monograph series, o M. Wainwright, M. Jordan, ``Graphical models, exponential families, and variational inference, Foundations and Trends in Machine Learning, Color coding covered in tutorial sparse choice model that is v. relevant others/ closely related/definitely worth a read
Ranking from Pairwise Comparisons
Ranking from Pairwise Comparisons Sewoong Oh Department of ISE University of Illinois at Urbana-Champaign Joint work with Sahand Negahban(MIT) and Devavrat Shah(MIT) / 9 Rank Aggregation Data Algorithm
More informationSpectral Method and Regularized MLE Are Both Optimal for Top-K Ranking
Spectral Method and Regularized MLE Are Both Optimal for Top-K Ranking Yuxin Chen Electrical Engineering, Princeton University Joint work with Jianqing Fan, Cong Ma and Kaizheng Wang Ranking A fundamental
More informationMinimax-optimal Inference from Partial Rankings
Minimax-optimal Inference from Partial Rankings Bruce Hajek UIUC b-hajek@illinois.edu Sewoong Oh UIUC swoh@illinois.edu Jiaming Xu UIUC jxu8@illinois.edu Abstract This paper studies the problem of rank
More informationRecoverabilty Conditions for Rankings Under Partial Information
Recoverabilty Conditions for Rankings Under Partial Information Srikanth Jagabathula Devavrat Shah Abstract We consider the problem of exact recovery of a function, defined on the space of permutations
More informationELE 538B: Mathematics of High-Dimensional Data. Spectral methods. Yuxin Chen Princeton University, Fall 2018
ELE 538B: Mathematics of High-Dimensional Data Spectral methods Yuxin Chen Princeton University, Fall 2018 Outline A motivating application: graph clustering Distance and angles between two subspaces Eigen-space
More informationInferring Rankings Using Constrained Sensing Srikanth Jagabathula and Devavrat Shah
7288 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 11, NOVEMBER 2011 Inferring Rankings Using Constrained Sensing Srikanth Jagabathula Devavrat Shah Abstract We consider the problem of recovering
More informationSYNC-RANK: ROBUST RANKING, CONSTRAINED RANKING AND RANK AGGREGATION VIA EIGENVECTOR AND SDP SYNCHRONIZATION
SYNC-RANK: ROBUST RANKING, CONSTRAINED RANKING AND RANK AGGREGATION VIA EIGENVECTOR AND SDP SYNCHRONIZATION MIHAI CUCURINGU Abstract. We consider the classic problem of establishing a statistical ranking
More informationRanking: Compare, don t Score
Forty-Ninth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 28-30, 2011 Ranking: Compare, don t Score Ammar Ammar, Devavrat Shah Laboratory for Information and Decision Systems
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More informationRecent Advances in Ranking: Adversarial Respondents and Lower Bounds on the Bayes Risk
Recent Advances in Ranking: Adversarial Respondents and Lower Bounds on the Bayes Risk Vincent Y. F. Tan Department of Electrical and Computer Engineering, Department of Mathematics, National University
More informationLecture 14: Random Walks, Local Graph Clustering, Linear Programming
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 14: Random Walks, Local Graph Clustering, Linear Programming Lecturer: Shayan Oveis Gharan 3/01/17 Scribe: Laura Vonessen Disclaimer: These
More informationThe non-backtracking operator
The non-backtracking operator Florent Krzakala LPS, Ecole Normale Supérieure in collaboration with Paris: L. Zdeborova, A. Saade Rome: A. Decelle Würzburg: J. Reichardt Santa Fe: C. Moore, P. Zhang Berkeley:
More informationSocial Choice and Networks
Social Choice and Networks Elchanan Mossel UC Berkeley All rights reserved Logistics 1 Different numbers for the course: Compsci 294 Section 063 Econ 207A Math C223A Stat 206A Room: Cory 241 Time TuTh
More informationLIMITATION OF LEARNING RANKINGS FROM PARTIAL INFORMATION. By Srikanth Jagabathula Devavrat Shah
00 AIM Workshop on Ranking LIMITATION OF LEARNING RANKINGS FROM PARTIAL INFORMATION By Srikanth Jagabathula Devavrat Shah Interest is in recovering distribution over the space of permutations over n elements
More information3.1 Arrow s Theorem. We study now the general case in which the society has to choose among a number of alternatives
3.- Social Choice and Welfare Economics 3.1 Arrow s Theorem We study now the general case in which the society has to choose among a number of alternatives Let R denote the set of all preference relations
More informationCombinatorial Hodge Theory and a Geometric Approach to Ranking
Combinatorial Hodge Theory and a Geometric Approach to Ranking Yuan Yao 2008 SIAM Annual Meeting San Diego, July 7, 2008 with Lek-Heng Lim et al. Outline 1 Ranking on networks (graphs) Netflix example
More informationOverlapping Communities
Overlapping Communities Davide Mottin HassoPlattner Institute Graph Mining course Winter Semester 2017 Acknowledgements Most of this lecture is taken from: http://web.stanford.edu/class/cs224w/slides GRAPH
More informationApproximating a single component of the solution to a linear system
Approximating a single component of the solution to a linear system Christina E. Lee, Asuman Ozdaglar, Devavrat Shah celee@mit.edu asuman@mit.edu devavrat@mit.edu MIT LIDS 1 How do I compare to my competitors?
More information15-850: Advanced Algorithms CMU, Fall 2018 HW #4 (out October 17, 2018) Due: October 28, 2018
15-850: Advanced Algorithms CMU, Fall 2018 HW #4 (out October 17, 2018) Due: October 28, 2018 Usual rules. :) Exercises 1. Lots of Flows. Suppose you wanted to find an approximate solution to the following
More informationLecture: Aggregation of General Biased Signals
Social Networks and Social Choice Lecture Date: September 7, 2010 Lecture: Aggregation of General Biased Signals Lecturer: Elchanan Mossel Scribe: Miklos Racz So far we have talked about Condorcet s Jury
More information14 : Theory of Variational Inference: Inner and Outer Approximation
10-708: Probabilistic Graphical Models 10-708, Spring 2014 14 : Theory of Variational Inference: Inner and Outer Approximation Lecturer: Eric P. Xing Scribes: Yu-Hsin Kuo, Amos Ng 1 Introduction Last lecture
More informationLecture 9: PGM Learning
13 Oct 2014 Intro. to Stats. Machine Learning COMP SCI 4401/7401 Table of Contents I Learning parameters in MRFs 1 Learning parameters in MRFs Inference and Learning Given parameters (of potentials) and
More informationLecture 13: Spectral Graph Theory
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 13: Spectral Graph Theory Lecturer: Shayan Oveis Gharan 11/14/18 Disclaimer: These notes have not been subjected to the usual scrutiny reserved
More informationECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning
ECE 6504: Advanced Topics in Machine Learning Probabilistic Graphical Models and Large-Scale Learning Topics Summary of Class Advanced Topics Dhruv Batra Virginia Tech HW1 Grades Mean: 28.5/38 ~= 74.9%
More informationGraph Helmholtzian and Rank Learning
Graph Helmholtzian and Rank Learning Lek-Heng Lim NIPS Workshop on Algebraic Methods in Machine Learning December 2, 2008 (Joint work with Xiaoye Jiang, Yuan Yao, and Yinyu Ye) L.-H. Lim (NIPS 2008) Graph
More informationCO759: Algorithmic Game Theory Spring 2015
CO759: Algorithmic Game Theory Spring 2015 Instructor: Chaitanya Swamy Assignment 1 Due: By Jun 25, 2015 You may use anything proved in class directly. I will maintain a FAQ about the assignment on the
More informationMachine Learning for Data Science (CS4786) Lecture 24
Machine Learning for Data Science (CS4786) Lecture 24 Graphical Models: Approximate Inference Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ BELIEF PROPAGATION OR MESSAGE PASSING Each
More information1 Primals and Duals: Zero Sum Games
CS 124 Section #11 Zero Sum Games; NP Completeness 4/15/17 1 Primals and Duals: Zero Sum Games We can represent various situations of conflict in life in terms of matrix games. For example, the game shown
More informationSimple, Robust and Optimal Ranking from Pairwise Comparisons
Journal of Machine Learning Research 8 (208) -38 Submitted 4/6; Revised /7; Published 4/8 Simple, Robust and Optimal Ranking from Pairwise Comparisons Nihar B. Shah Machine Learning Department and Computer
More informationA physical model for efficient rankings in networks
A physical model for efficient rankings in networks Daniel Larremore Assistant Professor Dept. of Computer Science & BioFrontiers Institute March 5, 2018 CompleNet danlarremore.com @danlarremore The idea
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationUndirected Graphical Models
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Properties Properties 3 Generative vs. Conditional
More informationMarkov Chains and Spectral Clustering
Markov Chains and Spectral Clustering Ning Liu 1,2 and William J. Stewart 1,3 1 Department of Computer Science North Carolina State University, Raleigh, NC 27695-8206, USA. 2 nliu@ncsu.edu, 3 billy@ncsu.edu
More informationSpring 2016 Network Science. Solution of Quiz I
Spring 2016 Network Science Department of Electrical and Computer Engineering National Chiao Tung University Solution of Quiz I Problem Points Your Score 1 5 2 20 25 20 Total 100 Read all of the following
More informationRestricted Boltzmann Machines for Collaborative Filtering
Restricted Boltzmann Machines for Collaborative Filtering Authors: Ruslan Salakhutdinov Andriy Mnih Geoffrey Hinton Benjamin Schwehn Presentation by: Ioan Stanculescu 1 Overview The Netflix prize problem
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationApproximating the Partition Function by Deleting and then Correcting for Model Edges (Extended Abstract)
Approximating the Partition Function by Deleting and then Correcting for Model Edges (Extended Abstract) Arthur Choi and Adnan Darwiche Computer Science Department University of California, Los Angeles
More informationStatistical ranking problems and Hodge decompositions of graphs and skew-symmetric matrices
Statistical ranking problems and Hodge decompositions of graphs and skew-symmetric matrices Lek-Heng Lim and Yuan Yao 2008 Workshop on Algorithms for Modern Massive Data Sets June 28, 2008 (Contains joint
More informationToday: Linear Programming (con t.)
Today: Linear Programming (con t.) COSC 581, Algorithms April 10, 2014 Many of these slides are adapted from several online sources Reading Assignments Today s class: Chapter 29.4 Reading assignment for
More informationCS242: Probabilistic Graphical Models Lecture 4A: MAP Estimation & Graph Structure Learning
CS242: Probabilistic Graphical Models Lecture 4A: MAP Estimation & Graph Structure Learning Professor Erik Sudderth Brown University Computer Science October 4, 2016 Some figures and materials courtesy
More informationSF2972 Game Theory Exam with Solutions March 15, 2013
SF2972 Game Theory Exam with s March 5, 203 Part A Classical Game Theory Jörgen Weibull and Mark Voorneveld. (a) What are N, S and u in the definition of a finite normal-form (or, equivalently, strategic-form)
More informationEECS 495: Combinatorial Optimization Lecture Manolis, Nima Mechanism Design with Rounding
EECS 495: Combinatorial Optimization Lecture Manolis, Nima Mechanism Design with Rounding Motivation Make a social choice that (approximately) maximizes the social welfare subject to the economic constraints
More informationMulticlass Classification-1
CS 446 Machine Learning Fall 2016 Oct 27, 2016 Multiclass Classification Professor: Dan Roth Scribe: C. Cheng Overview Binary to multiclass Multiclass SVM Constraint classification 1 Introduction Multiclass
More information- Well-characterized problems, min-max relations, approximate certificates. - LP problems in the standard form, primal and dual linear programs
LP-Duality ( Approximation Algorithms by V. Vazirani, Chapter 12) - Well-characterized problems, min-max relations, approximate certificates - LP problems in the standard form, primal and dual linear programs
More informationSocial Choice and Social Networks. Aggregation of General Biased Signals (DRAFT)
Social Choice and Social Networks Aggregation of General Biased Signals (DRAFT) All rights reserved Elchanan Mossel UC Berkeley 8 Sep 2010 Extending Condorect s Jury Theorem We want to consider extensions
More information14 : Theory of Variational Inference: Inner and Outer Approximation
10-708: Probabilistic Graphical Models 10-708, Spring 2017 14 : Theory of Variational Inference: Inner and Outer Approximation Lecturer: Eric P. Xing Scribes: Maria Ryskina, Yen-Chia Hsu 1 Introduction
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationLower bounds on the size of semidefinite relaxations. David Steurer Cornell
Lower bounds on the size of semidefinite relaxations David Steurer Cornell James R. Lee Washington Prasad Raghavendra Berkeley Institute for Advanced Study, November 2015 overview of results unconditional
More informationApproximate Inference in Practice Microsoft s Xbox TrueSkill TM
1 Approximate Inference in Practice Microsoft s Xbox TrueSkill TM Daniel Hernández-Lobato Universidad Autónoma de Madrid 2014 2 Outline 1 Introduction 2 The Probabilistic Model 3 Approximate Inference
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 5: Multi-class and preference learning Juho Rousu 11. October, 2017 Juho Rousu 11. October, 2017 1 / 37 Agenda from now on: This week s theme: going
More informationGraph Sparsifiers: A Survey
Graph Sparsifiers: A Survey Nick Harvey UBC Based on work by: Batson, Benczur, de Carli Silva, Fung, Hariharan, Harvey, Karger, Panigrahi, Sato, Spielman, Srivastava and Teng Approximating Dense Objects
More informationRecap Social Choice Fun Game Voting Paradoxes Properties. Social Choice. Lecture 11. Social Choice Lecture 11, Slide 1
Social Choice Lecture 11 Social Choice Lecture 11, Slide 1 Lecture Overview 1 Recap 2 Social Choice 3 Fun Game 4 Voting Paradoxes 5 Properties Social Choice Lecture 11, Slide 2 Formal Definition Definition
More informationProbabilistic and Bayesian Machine Learning
Probabilistic and Bayesian Machine Learning Day 4: Expectation and Belief Propagation Yee Whye Teh ywteh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit University College London http://www.gatsby.ucl.ac.uk/
More informationOutline. Spring It Introduction Representation. Markov Random Field. Conclusion. Conditional Independence Inference: Variable elimination
Probabilistic Graphical Models COMP 790-90 Seminar Spring 2011 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Outline It Introduction ti Representation Bayesian network Conditional Independence Inference:
More informationProject Discussions: SNL/ADMM, MDP/Randomization, Quadratic Regularization, and Online Linear Programming
Project Discussions: SNL/ADMM, MDP/Randomization, Quadratic Regularization, and Online Linear Programming Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305,
More informationNetwork Theory with Applications to Economics and Finance
Network Theory with Applications to Economics and Finance Instructor: Michael D. König University of Zurich, Department of Economics, Schönberggasse 1, CH - 8001 Zurich, email: michael.koenig@econ.uzh.ch.
More informationCOMS 4721: Machine Learning for Data Science Lecture 20, 4/11/2017
COMS 4721: Machine Learning for Data Science Lecture 20, 4/11/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University SEQUENTIAL DATA So far, when thinking
More informationMarkov Networks. l Like Bayes Nets. l Graph model that describes joint probability distribution using tables (AKA potentials)
Markov Networks l Like Bayes Nets l Graph model that describes joint probability distribution using tables (AKA potentials) l Nodes are random variables l Labels are outcomes over the variables Markov
More informationBreaking the 1/ n Barrier: Faster Rates for Permutation-based Models in Polynomial Time
Proceedings of Machine Learning Research vol 75:1 6, 2018 31st Annual Conference on Learning Theory Breaking the 1/ n Barrier: Faster Rates for Permutation-based Models in Polynomial Time Cheng Mao Massachusetts
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Chapter 8&9: Classification: Part 3 Instructor: Yizhou Sun yzsun@ccs.neu.edu March 12, 2013 Midterm Report Grade Distribution 90-100 10 80-89 16 70-79 8 60-69 4
More informationInference as Optimization
Inference as Optimization Sargur Srihari srihari@cedar.buffalo.edu 1 Topics in Inference as Optimization Overview Exact Inference revisited The Energy Functional Optimizing the Energy Functional 2 Exact
More informationMatrix completion: Fundamental limits and efficient algorithms. Sewoong Oh Stanford University
Matrix completion: Fundamental limits and efficient algorithms Sewoong Oh Stanford University 1 / 35 Low-rank matrix completion Low-rank Data Matrix Sparse Sampled Matrix Complete the matrix from small
More informationDeep Reinforcement Learning. STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 19, 2017
Deep Reinforcement Learning STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 19, 2017 Outline Introduction to Reinforcement Learning AlphaGo (Deep RL for Computer Go)
More informationJunction Tree, BP and Variational Methods
Junction Tree, BP and Variational Methods Adrian Weller MLSALT4 Lecture Feb 21, 2018 With thanks to David Sontag (MIT) and Tony Jebara (Columbia) for use of many slides and illustrations For more information,
More informationPolyhedral Approaches to Online Bipartite Matching
Polyhedral Approaches to Online Bipartite Matching Alejandro Toriello joint with Alfredo Torrico, Shabbir Ahmed Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Industrial
More informationLinear Programming. Jie Wang. University of Massachusetts Lowell Department of Computer Science. J. Wang (UMass Lowell) Linear Programming 1 / 47
Linear Programming Jie Wang University of Massachusetts Lowell Department of Computer Science J. Wang (UMass Lowell) Linear Programming 1 / 47 Linear function: f (x 1, x 2,..., x n ) = n a i x i, i=1 where
More informationAlgorithm Design and Analysis
Algorithm Design and Analysis LECTURE 26 Computational Intractability Polynomial Time Reductions Sofya Raskhodnikova S. Raskhodnikova; based on slides by A. Smith and K. Wayne L26.1 What algorithms are
More informationData Mining Recitation Notes Week 3
Data Mining Recitation Notes Week 3 Jack Rae January 28, 2013 1 Information Retrieval Given a set of documents, pull the (k) most similar document(s) to a given query. 1.1 Setup Say we have D documents
More informationLecture 6: Graphical Models: Learning
Lecture 6: Graphical Models: Learning 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering, University of Cambridge February 3rd, 2010 Ghahramani & Rasmussen (CUED)
More informationHigh-dimensional graphical model selection: Practical and information-theoretic limits
1 High-dimensional graphical model selection: Practical and information-theoretic limits Martin Wainwright Departments of Statistics, and EECS UC Berkeley, California, USA Based on joint work with: John
More informationCS 188: Artificial Intelligence Fall 2011
CS 188: Artificial Intelligence Fall 2011 Lecture 20: HMMs / Speech / ML 11/8/2011 Dan Klein UC Berkeley Today HMMs Demo bonanza! Most likely explanation queries Speech recognition A massive HMM! Details
More informationCSC 412 (Lecture 4): Undirected Graphical Models
CSC 412 (Lecture 4): Undirected Graphical Models Raquel Urtasun University of Toronto Feb 2, 2016 R Urtasun (UofT) CSC 412 Feb 2, 2016 1 / 37 Today Undirected Graphical Models: Semantics of the graph:
More informationSubmodular Functions Properties Algorithms Machine Learning
Submodular Functions Properties Algorithms Machine Learning Rémi Gilleron Inria Lille - Nord Europe & LIFL & Univ Lille Jan. 12 revised Aug. 14 Rémi Gilleron (Mostrare) Submodular Functions Jan. 12 revised
More informationBlind Identification of Invertible Graph Filters with Multiple Sparse Inputs 1
Blind Identification of Invertible Graph Filters with Multiple Sparse Inputs Chang Ye Dept. of ECE and Goergen Institute for Data Science University of Rochester cye7@ur.rochester.edu http://www.ece.rochester.edu/~cye7/
More informationApproximate Nash Equilibria with Near Optimal Social Welfare
Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 015) Approximate Nash Equilibria with Near Optimal Social Welfare Artur Czumaj, Michail Fasoulakis, Marcin
More information13 : Variational Inference: Loopy Belief Propagation
10-708: Probabilistic Graphical Models 10-708, Spring 2014 13 : Variational Inference: Loopy Belief Propagation Lecturer: Eric P. Xing Scribes: Rajarshi Das, Zhengzhong Liu, Dishan Gupta 1 Introduction
More informationMarkov Networks. l Like Bayes Nets. l Graphical model that describes joint probability distribution using tables (AKA potentials)
Markov Networks l Like Bayes Nets l Graphical model that describes joint probability distribution using tables (AKA potentials) l Nodes are random variables l Labels are outcomes over the variables Markov
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationMatrix estimation by Universal Singular Value Thresholding
Matrix estimation by Universal Singular Value Thresholding Courant Institute, NYU Let us begin with an example: Suppose that we have an undirected random graph G on n vertices. Model: There is a real symmetric
More informationMarkov Chains, Random Walks on Graphs, and the Laplacian
Markov Chains, Random Walks on Graphs, and the Laplacian CMPSCI 791BB: Advanced ML Sridhar Mahadevan Random Walks! There is significant interest in the problem of random walks! Markov chain analysis! Computer
More informationfsat We next show that if sat P, then fsat has a polynomial-time algorithm. c 2010 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 425
fsat fsat is this function problem: Let φ(x 1, x 2,..., x n ) be a boolean expression. If φ is satisfiable, then return a satisfying truth assignment. Otherwise, return no. We next show that if sat P,
More informationProbabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Variational Inference II: Mean Field Method and Variational Principle Junming Yin Lecture 15, March 7, 2012 X 1 X 1 X 1 X 1 X 2 X 3 X 2 X 2 X 3
More informationFast Linear Iterations for Distributed Averaging 1
Fast Linear Iterations for Distributed Averaging 1 Lin Xiao Stephen Boyd Information Systems Laboratory, Stanford University Stanford, CA 943-91 lxiao@stanford.edu, boyd@stanford.edu Abstract We consider
More informationCS6220: DATA MINING TECHNIQUES
CS6220: DATA MINING TECHNIQUES Matrix Data: Classification: Part 2 Instructor: Yizhou Sun yzsun@ccs.neu.edu September 21, 2014 Methods to Learn Matrix Data Set Data Sequence Data Time Series Graph & Network
More informationUnsupervised Image Segmentation Using Comparative Reasoning and Random Walks
Unsupervised Image Segmentation Using Comparative Reasoning and Random Walks Anuva Kulkarni Carnegie Mellon University Filipe Condessa Carnegie Mellon, IST-University of Lisbon Jelena Kovacevic Carnegie
More informationCS264: Beyond Worst-Case Analysis Lecture #18: Smoothed Complexity and Pseudopolynomial-Time Algorithms
CS264: Beyond Worst-Case Analysis Lecture #18: Smoothed Complexity and Pseudopolynomial-Time Algorithms Tim Roughgarden March 9, 2017 1 Preamble Our first lecture on smoothed analysis sought a better theoretical
More informationarxiv: v1 [cs.lg] 30 Dec 2015
Simple, Robust and Optimal Ranking from Pairwise Comparisons Nihar B. Shah nihar@eecs.berkeley.edu Martin J. Wainwright, wainwrig@berkeley.edu arxiv:5.08949v [cs.lg] 30 Dec 05 Department of EECS, and Department
More informationDoes Better Inference mean Better Learning?
Does Better Inference mean Better Learning? Andrew E. Gelfand, Rina Dechter & Alexander Ihler Department of Computer Science University of California, Irvine {agelfand,dechter,ihler}@ics.uci.edu Abstract
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec, Stanford University http://cs224w.stanford.edu 10/24/2012 Jure Leskovec, Stanford CS224W: Social and Information Network Analysis, http://cs224w.stanford.edu
More informationHigh-dimensional graphical model selection: Practical and information-theoretic limits
1 High-dimensional graphical model selection: Practical and information-theoretic limits Martin Wainwright Departments of Statistics, and EECS UC Berkeley, California, USA Based on joint work with: John
More informationA Graphical Transformation for Belief Propagation: Maximum Weight Matchings and Odd-Sized Cycles
A Graphical Transformation for Belief Propagation: Maximum Weight Matchings and Odd-Sized Cycles Jinwoo Shin Department of Electrical Engineering Korea Advanced Institute of Science and Technology Daejeon,
More informationOverview. 1 Introduction. 2 Preliminary Background. 3 Unique Game. 4 Unique Games Conjecture. 5 Inapproximability Results. 6 Unique Game Algorithms
Overview 1 Introduction 2 Preliminary Background 3 Unique Game 4 Unique Games Conjecture 5 Inapproximability Results 6 Unique Game Algorithms 7 Conclusion Antonios Angelakis (NTUA) Theory of Computation
More informationMachine Learning Summer School
Machine Learning Summer School Lecture 3: Learning parameters and structure Zoubin Ghahramani zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ Department of Engineering University of Cambridge,
More informationInference in Graphical Models Variable Elimination and Message Passing Algorithm
Inference in Graphical Models Variable Elimination and Message Passing lgorithm Le Song Machine Learning II: dvanced Topics SE 8803ML, Spring 2012 onditional Independence ssumptions Local Markov ssumption
More informationThanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides
Thanks to Jure Leskovec, Stanford and Panayiotis Tsaparas, Univ. of Ioannina for slides Web Search: How to Organize the Web? Ranking Nodes on Graphs Hubs and Authorities PageRank How to Solve PageRank
More informationDiscrepancy Theory in Approximation Algorithms
Discrepancy Theory in Approximation Algorithms Rajat Sen, Soumya Basu May 8, 2015 1 Introduction In this report we would like to motivate the use of discrepancy theory in algorithms. Discrepancy theory
More informationLearning Binary Classifiers for Multi-Class Problem
Research Memorandum No. 1010 September 28, 2006 Learning Binary Classifiers for Multi-Class Problem Shiro Ikeda The Institute of Statistical Mathematics 4-6-7 Minami-Azabu, Minato-ku, Tokyo, 106-8569,
More informationCS264: Beyond Worst-Case Analysis Lecture #15: Smoothed Complexity and Pseudopolynomial-Time Algorithms
CS264: Beyond Worst-Case Analysis Lecture #15: Smoothed Complexity and Pseudopolynomial-Time Algorithms Tim Roughgarden November 5, 2014 1 Preamble Previous lectures on smoothed analysis sought a better
More informationLecture 10. Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis
CS-621 Theory Gems October 18, 2012 Lecture 10 Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis 1 Introduction In this lecture, we will see how one can use random walks to
More informationMath 201 Statistics for Business & Economics. Definition of Statistics. Two Processes that define Statistics. Dr. C. L. Ebert
Math 201 Statistics for Business & Economics Dr. C. L. Ebert Chapter 1 Introduction Definition of Statistics Statistics - the study of the collection, organization, presentation, and characterization of
More information