Belief Propagation for Traffic forecasting
|
|
|
- Madlyn Davidson
- 5 years ago
- Views:
Transcription
1 Belief Propagation for Traffic forecasting Cyril Furtlehner (INRIA Saclay - Tao team) context : Travesti project http ://travesti.gforge.inria.fr/) Anne Auger (INRIA Saclay) Dimo Brockhoff (INRIA Lille) Yufei Han (CAOR Armines and INRIA Rocquencourt) Jean-Marc Lasgouttes (INRIA Rocquencourt) Fabrice Marchal (formerly at CNRS LET) Victorin Martin (INRIA) Fabien Moutarde (CAOR Armines) Maxim Samsonov (formerly at INRIA Saclay) C. Furtlehner IWI INRIA-PARIS June
2 Problem at hand Goal : reconstruct and predict road traffic on secondary network Existing solutions based on data coming from static sensors (magnetic loops) on main roads. Too expensive to be installed in all streets. Solution : more and more vehicles are equipped with GPS and able to exchanging data (through cellular phone connections) : Floating Car Data (FCD). = we want to build an inference schema adapted to these FCD C. Furtlehner IWI INRIA-PARIS June
3 Project guidelines (Traffic Volume Estimation by Spatio-Temporal Inference) (http ://travesti.gforge.inria.fr/) Goal 1 : is to find a macroscopic description of a large scale traffic network machine learning approach, clustering & dimensional reduction At microscopic level we consider travel time along each segment (FCD observations) Hypothesis : large scale behaviour is dominated by various dynamical congestion patterns. Multi-modal joint measure of travel time mixing macroscopic & microscopic variables Goal 2 : Setting of a MRF to encode dependencies between degrees of freedom. (dynamics at the macroscopic level) Real time inference : search for a model compatible with the belief propagation algorithm Related project : FUI Pumas (http ://pumas.inria.fr/), concrete implementation in the Rouen agglomeration (1000 probe vehicles) C. Furtlehner IWI INRIA-PARIS June
4 An Ising model for traffic? Is it possible to encode traffic data on the basis of a binary latent state s i,t { 1, 1} (Ising) corresponding to congested/non-congested. Input data x i,t R : static sensor : density and speed. floating car data : speed and travel times. 2 questions : Assuming an historical dataset of {ˆx i,t } : which probabilistic model P ({x i,t Ω})? Given actual observations {x i,t, (i, t) Ω } : how to infer {x i,t, (i, t) Ω\Ω }? (b) (a) C. Furtlehner IWI INRIA-PARIS June
5 Ising based proposed solution x j,t x i,t i : spatial segment label t : discretized time label x R s { 1 1} s j,t s i,t Single variable Statistical model Λ(x) = P (s = 1 x) Pairwise interaction Statistical model ˆp 11 = P (s = 1, s = 1) Probabilistic pairwise MRF model 4 components of the model : single variable statistical model translating real-valued observations into binary latent states pairwise statistical model of the dependency between latent states MRF model to encode the network of dependencies Belief propagation algorithm to decode a partially observed network {x i,t } C. Furtlehner IWI INRIA-PARIS June
6 Building and testing the model Input : a learning set {x k i, i {1... N}, k = 1... M}. Step 1 : define a mapping Λ(x) = P (s = 1 x) for each link. This fixes the set of ˆp i (s i ) Step 2 : build (EM) set of pairwise marginals ˆp ij (s i, s j ) based on the model hypothesis : P (x i, x j, s i, s j ) = ˆp ij (s i, s j )P (x i s i )P (x j s j ). Step 3 : from the set {ˆp i, ˆp ij } find a MRF inverse Ising model Experimental Test : from a test set {y k i, i = 1... N, k = 1... Q}, for each k, reveal one by one the variables at random and infer the other hidden one with BP. plot the prediction error as a function ρ of the fraction of observed variables. C. Furtlehner IWI INRIA-PARIS June
7 Inverse Ising Problem Inverse Problem : from observations to an underlying Markov Random field. Given a set of M joint observations {ŝ k i, i {1... N}, k = 1... M}, look for a model P Ising (s) = 1 Z(h, J) exp( i h i s i + i,j J ij s i s j ) Find the set h, J of parameters s.t. the Log Likelihood is maximal L(h, J) = D KL ( ˆP P Ising ) = log(z(h, J)) i h i Ê(s i ) ij J ij Ê(s i s j ). L(h, J) is concave and optimal when the moment matching constraints are satisfied : log Z h i [h, J] = Ê(s i) log Z J ij [h, J] = Ê(s is j ). Problem : Z(h, J) is difficult to evaluate in general (exponential cost). C. Furtlehner IWI INRIA-PARIS June
8 Inverse Ising Problem Generic question encountered in biology, image processing... Statistical physics based methods : Boltzmann machines (Hinton, Sejnowski 83) Linear response theory, Plefka expansion (Welling Teh 03) Susceptibility-Propagation (Mézard-Mora 07) Advanced linear response theory (Yasuda,Tanaka 09) Adaptative cluster expansion for neural coupling determination (Cocco et. al 09) Sparse optimization methods in Machine learning : L 1 optimization with belief propagation (Lee,Ganapathi,Koller 2006) L 1 regularization for graph selection (Wainwright,Ravikumar,Lafferty 2008) L 1 penalized pseudo-likelihood (Höfling,Tibshirani 2009) C. Furtlehner IWI INRIA-PARIS June
9 Mean-Field methods Input are the magnetization ˆm i = Ê(s i) and susceptibilities ˆχ ij = Cov(s i, s j ). Small interaction expansion : Naive Mean Field : J MF ij = Thouless-Anderson-Palmer (TAP, 77) J T AP ij ˆχ ij (1 ˆm 2 i )(1 ˆm2 j ) [ ˆχ 1 ] ij. J T AP 2[ ˆχ 1 ] ij ij = 1 +, 1 8 ˆm i ˆm j [ ˆχ 1 ] ij Tree-like interactions graph : Bethe approximation P Bethe (s) = (i,j) T ˆp ij (s i, s j ) ˆp i (s i )ˆp j (s j ) i ˆp i (s i ). with ˆp ij (s i, s j ) = 1 4 ( 1 + ˆmi s i + ˆm j s j + ( ˆχ ij + ˆm i ˆm j )s i s j ) C. Furtlehner IWI INRIA-PARIS June
10 Mean-Field methods Direct identification leads to J B1 ij J B1 ij = 1 ( 4 log ˆpij (1, 1) ˆp ij ( 1, 1 ). ˆp ij ( 1, 1) ˆp ij (1, 1) Remark : Z Bethe (h, J) explicit function of ˆp i and ˆp ij. Inverse susceptibility (Nguyen, Berg 2012) [ 1 [ ˆχ 1 di ] ij = + 1 m 2 ] k ˆχ ij 1 m 2 i (1 m 2 k i i )(1 m2 k ) δ ij ˆχ2 ik (1 m 2 i )(1 m2 j ) ˆχ2 ij δ j i. leads to J B2 ij J B2 (equivalent to susceptibility propagation) ij = 1 ( 2 atanh 2[ ˆχ 1 ] ij 1 + 4(1 ˆm 2 i )(1 ˆm2 j )[ ˆχ 1 ] 2 ij 2 ˆm i ˆm j [ ˆχ 1 ] ij ). C. Furtlehner IWI INRIA-PARIS June
11 Two-parameters Bethe Model Additional constraint : compatibility with belief propagation In practice we consider 2 calibration parameters : a global rescaling factor α of the interactions ( inverse temperature ) : J ij αj ij for all (i, j) a mean connectivity K of the graph various pruning procedures Bethe model 1 P α Bethe (s) = ( ˆpij (s i, s j ) ) α ˆp i (s i ). ˆp i (s i )ˆp j (s j ) (i,j) G Various pruning procedure : e.g. Max spanning tree + thresholding on other links. i C. Furtlehner IWI INRIA-PARIS June
12 BP fixed points corresponds to congestion patterns Real-valued synthetic multimodal data : combining the approximate MRF with the mapping Λ. L_1 tt Error N=200 C=5 a=0.1 b=0.4 dd=0.1 BP (alpha = 0.08 K=90) median tt prediction mean tt prediction KNN predictor ,2 0,4 0,6 0,8 1 ρ C. Furtlehner IWI INRIA-PARIS June
13 Simulation data : Sioux Falls Network Data generated by the METROPOLIS traffic simulator written by Fabrice Marchal (formerly at LET, CNRS-Lyon) and André de Palma (ENS Cachan) C. Furtlehner IWI INRIA-PARIS June
14 Experiments I Variable are revealed one by one in a random order. L_1 TT error Median TT predictor Mean TT predictor BP1 alpha = K=28 BP2 alpha=0.098 K=8 Mean Field alpha=0.044 K=20 Hopfield alpha=0.042 K=18 KNN Sioux Falls N= ,2 0,4 0,6 0,8 1 ρ (ρ = fraction of revealed variables) C. Furtlehner IWI INRIA-PARIS June
15 Variable are revealed one by one in a random order. Experiments II 150 IAURIF N= L_1 Error 50 BP tt predictor ENCOD 3 BP tt predictor ENCOD 7 BP tt predictor ENCOD 9 Median tt predictor Mean tt predictor K-NN tt predictor 0 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 ρ (ρ = fraction of revealed variables) C. Furtlehner IWI INRIA-PARIS June
16 Traffic index for real data 0,01 For a given travel time tt the index is defined by : x i = f(tt) def = P (tt i < tt), using a weighted cumulative based on the segmentation Probability Density 0,001 0,0001 1e-05 1e-06 1e Travel Time 1 0,8 travel time distribution Traffic index 0,6 0,4 0, Travel time (s) Various index maps C. Furtlehner IWI INRIA-PARIS June
17 Highway data RMSE on unrevealed variables segments, t = minutes, strong correlations Extraction of a spanning 180 tree with maximal weight 160 the weight is the mutual 140 information 3 temporal layers 120 Decimation % of revealed variables BP-prediction t-mean t0-prediction mean C. Furtlehner IWI INRIA-PARIS June
18 Conclusion Ising model can be used as a building block Macroscopic variables have to be incorporated in the model Real data needed to check various hypothesis C. Furtlehner IWI INRIA-PARIS June
An Ising Model for Road Traffic Inference
An Ising Model for Road Traffic Inference Cyril Furtlehner To cite this version: Cyril Furtlehner. An Ising Model for Road Traffic Inference. Xavier Leoncini and Marc Leonetti. From Hamiltonian Chaos to
Latent binary MRF for online reconstruction of large scale systems
Latent binary MRF for online reconstruction of large scale systems Victorin Martin, Jean-Marc Lasgouttes, Cyril Furtlehner To cite this version: Victorin Martin, Jean-Marc Lasgouttes, Cyril Furtlehner.
Bayesian Learning in Undirected Graphical Models
Bayesian Learning in Undirected Graphical Models Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK http://www.gatsby.ucl.ac.uk/ Work with: Iain Murray and Hyun-Chul
A Belief Propagation Approach to Traffic Prediction using Probe Vehicles
A Belief Propagation Approach to Traffic Prediction using Probe Vehicles Cyril Furtlehner Jean-Marc Lasgouttes Arnaud de La Fortelle Abstract This paper deals with real-time prediction of traffic conditions
Lecture 16 Deep Neural Generative Models
Lecture 16 Deep Neural Generative Models CMSC 35246: Deep Learning Shubhendu Trivedi & Risi Kondor University of Chicago May 22, 2017 Approach so far: We have considered simple models and then constructed
Bayesian Machine Learning - Lecture 7
Bayesian Machine Learning - Lecture 7 Guido Sanguinetti Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh gsanguin@inf.ed.ac.uk March 4, 2015 Today s lecture 1
Inference in kinetic Ising models: mean field and Bayes estimators
Inference in kinetic Ising models: mean field and Bayes estimators Ludovica Bachschmid-Romano, Manfred Opper Artificial Intelligence group, Computer Science, TU Berlin, Germany October 23, 2015 L. Bachschmid-Romano,
Causal Modeling with Generative Neural Networks
Causal Modeling with Generative Neural Networks Michele Sebag TAO, CNRS INRIA LRI Université Paris-Sud Joint work: D. Kalainathan, O. Goudet, I. Guyon, M. Hajaiej, A. Decelle, C. Furtlehner https://arxiv.org/abs/1709.05321
A brief introduction to the inverse Ising problem and some algorithms to solve it
A brief introduction to the inverse Ising problem and some algorithms to solve it Federico Ricci-Tersenghi Physics Department Sapienza University, Roma original results in collaboration with Jack Raymond
Bayesian Learning in Undirected Graphical Models
Bayesian Learning in Undirected Graphical Models Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London, UK http://www.gatsby.ucl.ac.uk/ and Center for Automated Learning and
Introduction to Graphical Models. Srikumar Ramalingam School of Computing University of Utah
Introduction to Graphical Models Srikumar Ramalingam School of Computing University of Utah Reference Christopher M. Bishop, Pattern Recognition and Machine Learning, Jonathan S. Yedidia, William T. Freeman,
Probabilistic and Bayesian Machine Learning
Probabilistic and Bayesian Machine Learning Day 4: Expectation and Belief Propagation Yee Whye Teh ywteh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit University College London http://www.gatsby.ucl.ac.uk/
Path and travel time inference from GPS probe vehicle data
Path and travel time inference from GPS probe vehicle data Timothy Hunter Department of Electrical Engineering and Computer Science University of California, Berkeley tjhunter@eecs.berkeley.edu Pieter
Lecture 15. Probabilistic Models on Graph
Lecture 15. Probabilistic Models on Graph Prof. Alan Yuille Spring 2014 1 Introduction We discuss how to define probabilistic models that use richly structured probability distributions and describe how
Introduction to Graphical Models. Srikumar Ramalingam School of Computing University of Utah
Introduction to Graphical Models Srikumar Ramalingam School of Computing University of Utah Reference Christopher M. Bishop, Pattern Recognition and Machine Learning, Jonathan S. Yedidia, William T. Freeman,
13 : Variational Inference: Loopy Belief Propagation and Mean Field
10-708: Probabilistic Graphical Models 10-708, Spring 2012 13 : Variational Inference: Loopy Belief Propagation and Mean Field Lecturer: Eric P. Xing Scribes: Peter Schulam and William Wang 1 Introduction
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2014
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2014 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
VISUAL EXPLORATION OF SPATIAL-TEMPORAL TRAFFIC CONGESTION PATTERNS USING FLOATING CAR DATA. Candra Kartika 2015
VISUAL EXPLORATION OF SPATIAL-TEMPORAL TRAFFIC CONGESTION PATTERNS USING FLOATING CAR DATA Candra Kartika 2015 OVERVIEW Motivation Background and State of The Art Test data Visualization methods Result
Information, Physics, and Computation
Information, Physics, and Computation Marc Mezard Laboratoire de Physique Thdorique et Moales Statistiques, CNRS, and Universit y Paris Sud Andrea Montanari Department of Electrical Engineering and Department
Junction Tree, BP and Variational Methods
Junction Tree, BP and Variational Methods Adrian Weller MLSALT4 Lecture Feb 21, 2018 With thanks to David Sontag (MIT) and Tony Jebara (Columbia) for use of many slides and illustrations For more information,
Approximate Inference Part 1 of 2
Approximate Inference Part 1 of 2 Tom Minka Microsoft Research, Cambridge, UK Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ Bayesian paradigm Consistent use of probability theory
Rapid Introduction to Machine Learning/ Deep Learning
Rapid Introduction to Machine Learning/ Deep Learning Hyeong In Choi Seoul National University 1/24 Lecture 5b Markov random field (MRF) November 13, 2015 2/24 Table of contents 1 1. Objectives of Lecture
Approximate Inference Part 1 of 2
Approximate Inference Part 1 of 2 Tom Minka Microsoft Research, Cambridge, UK Machine Learning Summer School 2009 http://mlg.eng.cam.ac.uk/mlss09/ 1 Bayesian paradigm Consistent use of probability theory
The Origin of Deep Learning. Lili Mou Jan, 2015
The Origin of Deep Learning Lili Mou Jan, 2015 Acknowledgment Most of the materials come from G. E. Hinton s online course. Outline Introduction Preliminary Boltzmann Machines and RBMs Deep Belief Nets
Introduction to graphical models: Lecture III
Introduction to graphical models: Lecture III Martin Wainwright UC Berkeley Departments of Statistics, and EECS Martin Wainwright (UC Berkeley) Some introductory lectures January 2013 1 / 25 Introduction
ECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4
ECE52 Tutorial Topic Review ECE52 Winter 206 Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides ECE52 Tutorial ECE52 Winter 206 Credits to Alireza / 4 Outline K-means, PCA 2 Bayesian
Undirected Graphical Models: Markov Random Fields
Undirected Graphical Models: Markov Random Fields 40-956 Advanced Topics in AI: Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2015 Markov Random Field Structure: undirected
13: Variational inference II
10-708: Probabilistic Graphical Models, Spring 2015 13: Variational inference II Lecturer: Eric P. Xing Scribes: Ronghuo Zheng, Zhiting Hu, Yuntian Deng 1 Introduction We started to talk about variational
UNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS
UNDERSTANDING BELIEF PROPOGATION AND ITS GENERALIZATIONS JONATHAN YEDIDIA, WILLIAM FREEMAN, YAIR WEISS 2001 MERL TECH REPORT Kristin Branson and Ian Fasel June 11, 2003 1. Inference Inference problems
Some recent results on the inverse Ising problem. Federico Ricci-Tersenghi Physics Department Sapienza University, Roma
Some recent results on the inverse Ising problem Federico Ricci-Tersenghi Physics Department Sapienza University, Roma Outline of the talk Bethe approx. for inverse Ising problem Comparison among several
Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks
Dynamic Probabilistic Models for Latent Feature Propagation in Social Networks Creighton Heaukulani and Zoubin Ghahramani University of Cambridge TU Denmark, June 2013 1 A Network Dynamic network data
Reconstruction for Models on Random Graphs
1 Stanford University, 2 ENS Paris October 21, 2007 Outline 1 The Reconstruction Problem 2 Related work and applications 3 Results The Reconstruction Problem: A Story Alice and Bob Alice, Bob and G root
Probabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Variational Inference II: Mean Field Method and Variational Principle Junming Yin Lecture 15, March 7, 2012 X 1 X 1 X 1 X 1 X 2 X 3 X 2 X 2 X 3
Chapter 17: Undirected Graphical Models
Chapter 17: Undirected Graphical Models The Elements of Statistical Learning Biaobin Jiang Department of Biological Sciences Purdue University bjiang@purdue.edu October 30, 2014 Biaobin Jiang (Purdue)
Undirected graphical models
Undirected graphical models Semantics of probabilistic models over undirected graphs Parameters of undirected models Example applications COMP-652 and ECSE-608, February 16, 2017 1 Undirected graphical
Spatial Bayesian Nonparametrics for Natural Image Segmentation
Spatial Bayesian Nonparametrics for Natural Image Segmentation Erik Sudderth Brown University Joint work with Michael Jordan University of California Soumya Ghosh Brown University Parsing Visual Scenes
6.867 Machine learning, lecture 23 (Jaakkola)
Lecture topics: Markov Random Fields Probabilistic inference Markov Random Fields We will briefly go over undirected graphical models or Markov Random Fields (MRFs) as they will be needed in the context
Probabilistic Graphical Models
Probabilistic Graphical Models Lecture 12 Dynamical Models CS/CNS/EE 155 Andreas Krause Homework 3 out tonight Start early!! Announcements Project milestones due today Please email to TAs 2 Parameter learning
Max$Sum(( Exact(Inference(
Probabilis7c( Graphical( Models( Inference( MAP( Max$Sum(( Exact(Inference( Product Summation a 1 b 1 8 a 1 b 2 1 a 2 b 1 0.5 a 2 b 2 2 a 1 b 1 3 a 1 b 2 0 a 2 b 1-1 a 2 b 2 1 Max-Sum Elimination in Chains
Chapter 11. Stochastic Methods Rooted in Statistical Mechanics
Chapter 11. Stochastic Methods Rooted in Statistical Mechanics Neural Networks and Learning Machines (Haykin) Lecture Notes on Self-learning Neural Algorithms Byoung-Tak Zhang School of Computer Science
Linear Response for Approximate Inference
Linear Response for Approximate Inference Max Welling Department of Computer Science University of Toronto Toronto M5S 3G4 Canada welling@cs.utoronto.ca Yee Whye Teh Computer Science Division University
Knowledge Extraction from DBNs for Images
Knowledge Extraction from DBNs for Images Son N. Tran and Artur d Avila Garcez Department of Computer Science City University London Contents 1 Introduction 2 Knowledge Extraction from DBNs 3 Experimental
Bias-Variance Trade-Off in Hierarchical Probabilistic Models Using Higher-Order Feature Interactions
- Trade-Off in Hierarchical Probabilistic Models Using Higher-Order Feature Interactions Simon Luo The University of Sydney Data61, CSIRO simon.luo@data61.csiro.au Mahito Sugiyama National Institute of
UNSUPERVISED LEARNING
UNSUPERVISED LEARNING Topics Layer-wise (unsupervised) pre-training Restricted Boltzmann Machines Auto-encoders LAYER-WISE (UNSUPERVISED) PRE-TRAINING Breakthrough in 2006 Layer-wise (unsupervised) pre-training
Inferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines. Eric W. Tramel. itwist 2016 Aalborg, DK 24 August 2016
Inferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines Eric W. Tramel itwist 2016 Aalborg, DK 24 August 2016 Andre MANOEL, Francesco CALTAGIRONE, Marylou GABRIE, Florent
14 : Theory of Variational Inference: Inner and Outer Approximation
10-708: Probabilistic Graphical Models 10-708, Spring 2017 14 : Theory of Variational Inference: Inner and Outer Approximation Lecturer: Eric P. Xing Scribes: Maria Ryskina, Yen-Chia Hsu 1 Introduction
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
An Introduction to Bayesian Machine Learning
1 An Introduction to Bayesian Machine Learning José Miguel Hernández-Lobato Department of Engineering, Cambridge University April 8, 2013 2 What is Machine Learning? The design of computational systems
Probabilistic Graphical Models Lecture Notes Fall 2009
Probabilistic Graphical Models Lecture Notes Fall 2009 October 28, 2009 Byoung-Tak Zhang School of omputer Science and Engineering & ognitive Science, Brain Science, and Bioinformatics Seoul National University
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
Learning in Markov Random Fields An Empirical Study
Learning in Markov Random Fields An Empirical Study Sridevi Parise, Max Welling University of California, Irvine sparise,welling@ics.uci.edu Abstract Learning the parameters of an undirected graphical
CSC 412 (Lecture 4): Undirected Graphical Models
CSC 412 (Lecture 4): Undirected Graphical Models Raquel Urtasun University of Toronto Feb 2, 2016 R Urtasun (UofT) CSC 412 Feb 2, 2016 1 / 37 Today Undirected Graphical Models: Semantics of the graph:
Approximating the Partition Function by Deleting and then Correcting for Model Edges (Extended Abstract)
Approximating the Partition Function by Deleting and then Correcting for Model Edges (Extended Abstract) Arthur Choi and Adnan Darwiche Computer Science Department University of California, Los Angeles
Loopy Belief Propagation for Bipartite Maximum Weight b-matching
Loopy Belief Propagation for Bipartite Maximum Weight b-matching Bert Huang and Tony Jebara Computer Science Department Columbia University New York, NY 10027 Outline 1. Bipartite Weighted b-matching 2.
Random Field Models for Applications in Computer Vision
Random Field Models for Applications in Computer Vision Nazre Batool Post-doctorate Fellow, Team AYIN, INRIA Sophia Antipolis Outline Graphical Models Generative vs. Discriminative Classifiers Markov Random
11. Learning graphical models
Learning graphical models 11-1 11. Learning graphical models Maximum likelihood Parameter learning Structural learning Learning partially observed graphical models Learning graphical models 11-2 statistical
Variational Inference (11/04/13)
STA561: Probabilistic machine learning Variational Inference (11/04/13) Lecturer: Barbara Engelhardt Scribes: Matt Dickenson, Alireza Samany, Tracy Schifeling 1 Introduction In this lecture we will further
COMP90051 Statistical Machine Learning
COMP90051 Statistical Machine Learning Semester 2, 2017 Lecturer: Trevor Cohn 24. Hidden Markov Models & message passing Looking back Representation of joint distributions Conditional/marginal independence
Improving forecasting under missing data on sparse spatial networks
Improving forecasting under missing data on sparse spatial networks J. Haworth 1, T. Cheng 1, E. J. Manley 1 1 SpaceTimeLab, Department of Civil, Environmental and Geomatic Engineering, University College
3 : Representation of Undirected GM
10-708: Probabilistic Graphical Models 10-708, Spring 2016 3 : Representation of Undirected GM Lecturer: Eric P. Xing Scribes: Longqi Cai, Man-Chia Chang 1 MRF vs BN There are two types of graphical models:
2 : Directed GMs: Bayesian Networks
10-708: Probabilistic Graphical Models 10-708, Spring 2017 2 : Directed GMs: Bayesian Networks Lecturer: Eric P. Xing Scribes: Jayanth Koushik, Hiroaki Hayashi, Christian Perez Topic: Directed GMs 1 Types
A Deep Interpretation of Classifier Chains
A Deep Interpretation of Classifier Chains Jesse Read and Jaakko Holmén http://users.ics.aalto.fi/{jesse,jhollmen}/ Aalto University School of Science, Department of Information and Computer Science and
Independent Component Analysis and Unsupervised Learning. Jen-Tzung Chien
Independent Component Analysis and Unsupervised Learning Jen-Tzung Chien TABLE OF CONTENTS 1. Independent Component Analysis 2. Case Study I: Speech Recognition Independent voices Nonparametric likelihood
Spatio-Temporal Analytics of Network Data
Spatio-Temporal Analytics of Network Data Tao Cheng, James Haworth SpaceTimeLab Team University College London http://www.ucl.ac.uk/spacetimelab GIS Ostrava 2014 - Geoinformatics for Intelligent Transportation
Does Better Inference mean Better Learning?
Does Better Inference mean Better Learning? Andrew E. Gelfand, Rina Dechter & Alexander Ihler Department of Computer Science University of California, Irvine {agelfand,dechter,ihler}@ics.uci.edu Abstract
Deep Learning Basics Lecture 8: Autoencoder & DBM. Princeton University COS 495 Instructor: Yingyu Liang
Deep Learning Basics Lecture 8: Autoencoder & DBM Princeton University COS 495 Instructor: Yingyu Liang Autoencoder Autoencoder Neural networks trained to attempt to copy its input to its output Contain
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, Spis treści
Deep learning / Ian Goodfellow, Yoshua Bengio and Aaron Courville. - Cambridge, MA ; London, 2017 Spis treści Website Acknowledgments Notation xiii xv xix 1 Introduction 1 1.1 Who Should Read This Book?
Active and Semi-supervised Kernel Classification
Active and Semi-supervised Kernel Classification Zoubin Ghahramani Gatsby Computational Neuroscience Unit University College London Work done in collaboration with Xiaojin Zhu (CMU), John Lafferty (CMU),
Probabilistic Graphical Models
Probabilistic Graphical Models Brown University CSCI 295-P, Spring 213 Prof. Erik Sudderth Lecture 11: Inference & Learning Overview, Gaussian Graphical Models Some figures courtesy Michael Jordan s draft
LARGE-SCALE TRAFFIC STATE ESTIMATION
Hans van Lint, Yufei Yuan & Friso Scholten A localized deterministic Ensemble Kalman Filter LARGE-SCALE TRAFFIC STATE ESTIMATION CONTENTS Intro: need for large-scale traffic state estimation Some Kalman
ABSTRACT INTRODUCTION
ABSTRACT Presented in this paper is an approach to fault diagnosis based on a unifying review of linear Gaussian models. The unifying review draws together different algorithms such as PCA, factor analysis,
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
Probabilistic Graphical Models
School of Computer Science Probabilistic Graphical Models Variational Inference IV: Variational Principle II Junming Yin Lecture 17, March 21, 2012 X 1 X 1 X 1 X 1 X 2 X 3 X 2 X 2 X 3 X 3 Reading: X 4
Dynamic Approaches: The Hidden Markov Model
Dynamic Approaches: The Hidden Markov Model Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Machine Learning: Neural Networks and Advanced Models (AA2) Inference as Message
Deep unsupervised learning
Deep unsupervised learning Advanced data-mining Yongdai Kim Department of Statistics, Seoul National University, South Korea Unsupervised learning In machine learning, there are 3 kinds of learning paradigm.
Lecture 18 Generalized Belief Propagation and Free Energy Approximations
Lecture 18, Generalized Belief Propagation and Free Energy Approximations 1 Lecture 18 Generalized Belief Propagation and Free Energy Approximations In this lecture we talked about graphical models and
Clique trees & Belief Propagation. Siamak Ravanbakhsh Winter 2018
Graphical Models Clique trees & Belief Propagation Siamak Ravanbakhsh Winter 2018 Learning objectives message passing on clique trees its relation to variable elimination two different forms of belief
Learning in Bayesian Networks
Learning in Bayesian Networks Florian Markowetz Max-Planck-Institute for Molecular Genetics Computational Molecular Biology Berlin Berlin: 20.06.2002 1 Overview 1. Bayesian Networks Stochastic Networks
Graphical Models for Collaborative Filtering
Graphical Models for Collaborative Filtering Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Sequence modeling HMM, Kalman Filter, etc.: Similarity: the same graphical model topology,
Clustering. CSL465/603 - Fall 2016 Narayanan C Krishnan
Clustering CSL465/603 - Fall 2016 Narayanan C Krishnan ckn@iitrpr.ac.in Supervised vs Unsupervised Learning Supervised learning Given x ", y " "%& ', learn a function f: X Y Categorical output classification
Improving the travel time prediction by using the real-time floating car data
Improving the travel time prediction by using the real-time floating car data Krzysztof Dembczyński Przemys law Gawe l Andrzej Jaszkiewicz Wojciech Kot lowski Adam Szarecki Institute of Computing Science,
Statistical Approaches to Learning and Discovery
Statistical Approaches to Learning and Discovery Graphical Models Zoubin Ghahramani & Teddy Seidenfeld zoubin@cs.cmu.edu & teddy@stat.cmu.edu CALD / CS / Statistics / Philosophy Carnegie Mellon University
Graphical Models and Kernel Methods
Graphical Models and Kernel Methods Jerry Zhu Department of Computer Sciences University of Wisconsin Madison, USA MLSS June 17, 2014 1 / 123 Outline Graphical Models Probabilistic Inference Directed vs.
CSCI-567: Machine Learning (Spring 2019)
CSCI-567: Machine Learning (Spring 2019) Prof. Victor Adamchik U of Southern California Mar. 19, 2019 March 19, 2019 1 / 43 Administration March 19, 2019 2 / 43 Administration TA3 is due this week March
Uncertainty quantification and visualization for functional random variables
Uncertainty quantification and visualization for functional random variables MascotNum Workshop 2014 S. Nanty 1,3 C. Helbert 2 A. Marrel 1 N. Pérot 1 C. Prieur 3 1 CEA, DEN/DER/SESI/LSMR, F-13108, Saint-Paul-lez-Durance,
CSCE 478/878 Lecture 6: Bayesian Learning
Bayesian Methods Not all hypotheses are created equal (even if they are all consistent with the training data) Outline CSCE 478/878 Lecture 6: Bayesian Learning Stephen D. Scott (Adapted from Tom Mitchell
p L yi z n m x N n xi
y i z n x n N x i Overview Directed and undirected graphs Conditional independence Exact inference Latent variables and EM Variational inference Books statistical perspective Graphical Models, S. Lauritzen
High-dimensional graphical model selection: Practical and information-theoretic limits
1 High-dimensional graphical model selection: Practical and information-theoretic limits Martin Wainwright Departments of Statistics, and EECS UC Berkeley, California, USA Based on joint work with: John
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013
UNIVERSITY of PENNSYLVANIA CIS 520: Machine Learning Final, Fall 2013 Exam policy: This exam allows two one-page, two-sided cheat sheets; No other materials. Time: 2 hours. Be sure to write your name and
Restricted Boltzmann Machines for Collaborative Filtering
Restricted Boltzmann Machines for Collaborative Filtering Authors: Ruslan Salakhutdinov Andriy Mnih Geoffrey Hinton Benjamin Schwehn Presentation by: Ioan Stanculescu 1 Overview The Netflix prize problem
Learning Deep Architectures for AI. Part II - Vijay Chakilam
Learning Deep Architectures for AI - Yoshua Bengio Part II - Vijay Chakilam Limitations of Perceptron x1 W, b 0,1 1,1 y x2 weight plane output =1 output =0 There is no value for W and b such that the model
CPSC 540: Machine Learning
CPSC 540: Machine Learning Mark Schmidt University of British Columbia Winter 2018 Last Time: Learning and Inference in DAGs We discussed learning in DAG models, log p(x W ) = n d log p(x i j x i pa(j),
Algorithmisches Lernen/Machine Learning
Algorithmisches Lernen/Machine Learning Part 1: Stefan Wermter Introduction Connectionist Learning (e.g. Neural Networks) Decision-Trees, Genetic Algorithms Part 2: Norman Hendrich Support-Vector Machines
Directed and Undirected Graphical Models
Directed and Undirected Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Machine Learning: Neural Networks and Advanced Models (AA2) Last Lecture Refresher Lecture Plan Directed
5. Sum-product algorithm
Sum-product algorithm 5-1 5. Sum-product algorithm Elimination algorithm Sum-product algorithm on a line Sum-product algorithm on a tree Sum-product algorithm 5-2 Inference tasks on graphical models consider
Modeling Freeway Traffic with Coupled HMMs
Modeling Freeway Traffic with Coupled HMMs Jaimyoung Kwon Department of Statistics University of California Berkeley, CA 947 kwon@stat.berkeley.edu Kevin Murphy Department of Computer Science University
7.1 Basis for Boltzmann machine. 7. Boltzmann machines
7. Boltzmann machines this section we will become acquainted with classical Boltzmann machines which can be seen obsolete being rarely applied in neurocomputing. It is interesting, after all, because is
High-dimensional graphical model selection: Practical and information-theoretic limits
1 High-dimensional graphical model selection: Practical and information-theoretic limits Martin Wainwright Departments of Statistics, and EECS UC Berkeley, California, USA Based on joint work with: John
Independent Component Analysis and Unsupervised Learning
Independent Component Analysis and Unsupervised Learning Jen-Tzung Chien National Cheng Kung University TABLE OF CONTENTS 1. Independent Component Analysis 2. Case Study I: Speech Recognition Independent