University of Washington Department of Electrical Engineering EE512 Spring, 2006 Graphical Models
|
|
|
- Nathaniel Shepherd
- 5 years ago
- Views:
Transcription
1 University of Washington Department of Electrical Engineering EE512 Spring, 2006 Graphical Models Jeff A. Bilmes Lecture 1 Slides March 28 th, 2006 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 1
2 Outline of Today s Lecture Class overview What are graphical models Semantics of Bayesian networks Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 2
3 Books and Sources for Today Jordan: Chapters 1 and 2 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 3
4 Class Road Map L1: Tues, 3/28: Overview, GMs, Intro BNs. L2: Thur, 3/30 L3: Tues, 4/4 L4: Thur, 4/6 L5: Tue, 4/11 L6: Thur, 4/13 L7: Tues, 4/18 L8: Thur, 4/20 L9: Tue, 4/25 L10: Thur, 4/27 L11: Tues, 5/2 L12: Thur, 5/4 L13: Tues, 5/9 L14: Thur, 5/11 L15: Tue, 5/16 L16: Thur, 5/18 L17: Tues, 5/23 L18: Thur, 5/25 L19: Tue, 5/30 L20: Thur, 6/1: final presentations Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 4
5 Announcements READING: Chapter 1,2 in Jordan s book (pick up book from basement of communications copy center). List handout: name, department, and List handout: regular makeup slot, and discussion section Syllabus Course web page: Goal: powerpoint slides this quarter, they will be on the web page after lecture (but not before as you are getting them hot off the press). Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 5
6 Graphical Models A graphical model is a visual, abstract, and mathematically formal description of properties of families of probability distributions (densities, mass functions) There are many different types of Graphical model, ex: Bayesian Networks Factor Graph Markov Random Fields Chain Graph Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 6
7 GMs cover many well-known methods Graphical Models Other Semantics Causal Models Chain Graphs Factor Graphs Dependency Networks DGMs AR UGMs FST HMM ZMs DBNs Factorial HMM/Mixed Memory Markov Models BMMs Bayesian Networks Mixture Models Kalman Decision Trees Segment Models Simple Models PCA LDA MRFs Gibbs/Boltzman Distributions Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 7
8 Graphical Models Provide: Structure Algorithms Language Approximations Data-Bases Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 8
9 Graphical Models Provide GMs give us: I. Structure: A method to explore the structure of natural phenomena (causal vs. correlated relations, properties of natural signals and scenes) II. Algorithms: A set of algorithms that provide efficient probabilistic inference and statistical decision making III. Language: A mathematically formal, abstract, visual language with which to efficiently discuss families of probabilistic models and their properties. Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 9
10 GMs Provide GMs give us (cont): IV. Approximation: Methods to explore systems of approximation and their implications. E.g., what are the consequences of a (perhaps known to be) wrong assumption? V. Data-base: Provide a probabilistic data-base and corresponding search algorithms for making queries about properties in such model families. Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 10
11 GMs There are many different types of GM. Each GM has its semantics A GM (under the current semantics) is really a set of constraints. The GM represents all probability distributions that obey these constraints, including those that obey additional constraints (but not including those that obey fewer constraints). Most often, the constraints are some form of factorization property, e.g., f() factorizes (is composed of a product of factors of subsets of arguments). Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 11
12 Types of Queries Several types of queries we may be interested in: Compute: p(one subset of vars) Compute: p(one subset of vars another subset of vars) Find the N most probable configurations of one subset of variables given assignments of values to some other sets Q: Is one subset independent of another subset? Q: Is one subset independent of another given a third? How efficiently can we do this? Can this question be answered? What if it is too costly, can we approximate, and if so, how well? These are questions we will answer this term. GMs are like a probabilistic data-base (or data structure), a system that can be queried to provide answers to these sorts of questions. Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 12
13 Example Typical goal of pattern recognition: training (say, EM or gradient descent), need query of form: In this form, we need to compute p(o,h) efficiently. Bayes decision rule, need to find best class for a given unknown pattern: but this is yet another query on a probability distribution. We can train, and perform Bayes decision theory quickly if we can compute with probabilities quickly. Graphical models provide a way to reason about, and understand when this is possible, and if not, how to reasonably approximate. Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 13
14 Some Notation Random variables ψ ψ, ψ ψ, ψ, ψ (scalar or vector) Distributions: Subsets: Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 14
15 Main types of Graphical Models Markov Random Fields a form of undirected graphical model relatively simple to understand their semantics also, log-linear models, Gibbs distributions, Boltzman distributions, many exponential models, conditional random fields (CRFs), etc. Bayesian networks a form of directed graphical model originally developed to represent a form of causality, but not ideal for that (they still represent factorization) Semantics more interesting (but trickier) than MRFs Factor Graphs pure, the assembly language models for factorization properties came out of coding theory community (LDPC, Turbo codes) Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 15
16 Chain graphs: Main types of Graphical Models Hybrid between Bayesian networks and MRFs A set of clusters of undirected nodes connected as directed links Not as widely used, but very powerful. Ancestral graphs we probably won t cover these. Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 16
17 Bayesian Network Examples Mixture models C Markov Chains Q 1 Q 2 Q 3 Q 4 p( qt q1 : t 1) = p( qt qt 1) X px ( ) = cpxi i ( ) i Q 1 Q 2 Q 3 Q 4 p( qt q1 : t 1) = p( qt qt 1, qt 2) Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 17
18 GMs: PCA and Factor Analysis Y μ = AX X Y + Q R X N(0, Q) N( μ, R) PCA: Q = Λ, R 0, A = ortho Y = ΓX + μ FA: Q = I, R = diagonal Y = Γ X + u+ μ Other generalizations possible E.g., Q = gen. diagonal, or capture using general A since T Y N( μ, AQA + R) Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 18
19 Independent Component Analysis I 1 I 2 I I 1 2 X 1 X 2 X 3 X 4 The data X 1:4 is explained by the two (marginally) independent causes. Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 19
20 Linear Discriminant Analysis fi ( X) p( i) PC ( = i X) = f ( X) p( k) f ( X) = N( μ, Σ) j k Class conditional data has diff. mean but common covariance matrix. Fisher s formulation: project onto space spanned by the means. j k μ C X Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 20
21 Extensions to LDA HDA/QDA MDA HMDA C C C μ M M X μ μ Heteroschedastic Discriminant Analysis/Quadratic Discriminant Analysis Mixture Discriminant Analsysis X X Both Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 21
22 Generalized Decision Trees I D 1 D 2 Generalized Probabilistic Decision Trees Hierarchical Mixtures of Experts (Jordan) D 3 O Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 22
23 Example: Printer Troubleshooting Dechter Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 23
24 Classifier Combination Mixture of Experts (Sum rule) Naive Bayes (prod. rule) Other Combination Schemes X X 1 X 2... X N C S C 1 C 2 C C X 1 X 2 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 24
25 Discriminative and Generative Models Generative Model C Discriminative Model C X X Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 25
26 HMMs/Kalman Filter HMM Q 1 Q 2 Q 3 Q 4 X 1 X 2 X 3 X 4 Autoregressive HMM Q 1 Q 2 Q 3 Q 4 X 1 X 2 X 3 X 4 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 26
27 Switching Kalman Filter Q 1 Q 2 Q 3 Q 4 S 1 S 2 S 3 S 4 X 1 X 2 X 3 X 4 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 27
28 Factorial HMM Q 1 Q 2 Q 3 Q 4 Q 1 Q 2 Q 3 Q 4 X 1 X 2 X 3 X 4 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 28
29 Standard Language Modeling Example: standard 4-gram Pw ( h) = Pw ( w, w, w ) t t t t 1 t 2 t 3 W t 4 W t 3 W t 2 Wt 1 W t Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 29
30 Interpolated Uni-,Bi-,Tri-Grams Pw ( h) = P( α = 1) Pw ( ) + P( α = 2) Pw ( w ) t t t t t t t 1 + P( α = 3) p( w w, w ) t t t 1 t 2 Nothing gets zero probability Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 30
31 Conditional mixture tri-grams Pw ( h) = P( α = 1 w, w ) Pw ( ) t t t t 1 t 2 t + P( α = 2 w, w ) P( w w ) t t 1 t 2 t t 1 + P( α = 3 w, w ) p( w w, w ) t t 1 t 2 t t 1 t 2 Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 31
32 Bayesian Networks and so on We need to be more formal about what BNs mean. In the rest of this, and in the next, lecture, we start with basic semantics of BNs, move on to undirected models, and then come back to BNs again to clean up Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 32
33 Bayesian Networks Has nothing to do with Bayesian statistical models (there are Bayesian and non-bayesian Bayesian networks). valid: invalid: Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 33
34 BN: Alarm Network Example Compact representation: factors of probabilities of children given parents (J. Pearl, 1988) Sub-family specification: Directed acyclic graph (DAG) Nodes - random variables Edges - direct influence Radio Earthquake Burglary Alarm E e e e e B b b b b P(A E,B) Together (graph and inst.): Defines a unique distribution in a factored form PBEACR (,,,, ) = PBPEPABEPR ( ) ( ) (, ) ( EPC ) ( A) Instantiation: Set of conditional probability distributions Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 34 Call
35 Bayesian Networks Reminder: chain rule of probability. For any order σ Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 35
36 Bayesian Networks Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 36
37 Conditional Independence Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 37
38 Conditional Independence & BNs GMs can help answer CI queries: X A B Z X Z Y??? C E D No!! Y Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 38
39 Bayesian Networks & CI Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 39
40 Bayesian Networks & CI Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 40
41 Bayesian Networks & CI Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 41
42 Bayesian Networks & CI Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 42
43 Pictorial d-separation: blocked/unblocked paths Blocked Paths Unblocked Paths Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 43
44 Three Canonical Cases Three 3-node examples of BNs and their conditional independence statements. V 1 V 1 V 2 V 3 V 2 V 3 V V V V V 2 3 V 2 V 3 V V V V V 2 3 V 1 V V 2 3 V V V Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 44
45 Case 1 Markov Chain Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 45
46 Case 2 Still a Markov Chain Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 46
47 Case 3 NOT a Markov Chain Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 47
48 Examples of the three cases SUVs Greenhouse Gasses Global Warming Lung Cancer Smoking Bad Breath Genetics Cancer Smoking Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 48
49 Bayes Ball simple algorithm, ball bouncing along path in graph, to tell if path is blocked or not (more details in text). Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 49
50 What are implied conditional independences? Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 50
51 Two Views of a Family Lec 1: March 28th, 2006 EE512 - Graphical Models - J. Bilmes Page 51
Bayesian Networks Inference with Probabilistic Graphical Models
4190.408 2016-Spring Bayesian Networks Inference with Probabilistic Graphical Models Byoung-Tak Zhang intelligence Lab Seoul National University 4190.408 Artificial (2016-Spring) 1 Machine Learning? Learning
CS 2750: Machine Learning. Bayesian Networks. Prof. Adriana Kovashka University of Pittsburgh March 14, 2016
CS 2750: Machine Learning Bayesian Networks Prof. Adriana Kovashka University of Pittsburgh March 14, 2016 Plan for today and next week Today and next time: Bayesian networks (Bishop Sec. 8.1) Conditional
COS402- Artificial Intelligence Fall Lecture 10: Bayesian Networks & Exact Inference
COS402- Artificial Intelligence Fall 2015 Lecture 10: Bayesian Networks & Exact Inference Outline Logical inference and probabilistic inference Independence and conditional independence Bayes Nets Semantics
Directed Graphical Models
CS 2750: Machine Learning Directed Graphical Models Prof. Adriana Kovashka University of Pittsburgh March 28, 2017 Graphical Models If no assumption of independence is made, must estimate an exponential
Lecture 17: May 29, 2002
EE596 Pat. Recog. II: Introduction to Graphical Models University of Washington Spring 2000 Dept. of Electrical Engineering Lecture 17: May 29, 2002 Lecturer: Jeff ilmes Scribe: Kurt Partridge, Salvador
Bayesian Networks BY: MOHAMAD ALSABBAGH
Bayesian Networks BY: MOHAMAD ALSABBAGH Outlines Introduction Bayes Rule Bayesian Networks (BN) Representation Size of a Bayesian Network Inference via BN BN Learning Dynamic BN Introduction Conditional
Based on slides by Richard Zemel
CSC 412/2506 Winter 2018 Probabilistic Learning and Reasoning Lecture 3: Directed Graphical Models and Latent Variables Based on slides by Richard Zemel Learning outcomes What aspects of a model can we
EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS
EE562 ARTIFICIAL INTELLIGENCE FOR ENGINEERS Lecture 16, 6/1/2005 University of Washington, Department of Electrical Engineering Spring 2005 Instructor: Professor Jeff A. Bilmes Uncertainty & Bayesian Networks
3 : Representation of Undirected GM
10-708: Probabilistic Graphical Models 10-708, Spring 2016 3 : Representation of Undirected GM Lecturer: Eric P. Xing Scribes: Longqi Cai, Man-Chia Chang 1 MRF vs BN There are two types of graphical models:
Intelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 18 Oct, 21, 2015 Slide Sources Raymond J. Mooney University of Texas at Austin D. Koller, Stanford CS - Probabilistic Graphical Models CPSC
Conditional Independence and Factorization
Conditional Independence and Factorization Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
Graphical Models and Kernel Methods
Graphical Models and Kernel Methods Jerry Zhu Department of Computer Sciences University of Wisconsin Madison, USA MLSS June 17, 2014 1 / 123 Outline Graphical Models Probabilistic Inference Directed vs.
Lecture 5: Bayesian Network
Lecture 5: Bayesian Network Topics of this lecture What is a Bayesian network? A simple example Formal definition of BN A slightly difficult example Learning of BN An example of learning Important topics
EE512 Graphical Models Fall 2009
EE512 Graphical Models Fall 2009 Prof. Jeff Bilmes University of Washington, Seattle Department of Electrical Engineering Fall Quarter, 2009 http://ssli.ee.washington.edu/~bilmes/ee512fa09 Lecture 3 -
A Brief Introduction to Graphical Models. Presenter: Yijuan Lu November 12,2004
A Brief Introduction to Graphical Models Presenter: Yijuan Lu November 12,2004 References Introduction to Graphical Models, Kevin Murphy, Technical Report, May 2001 Learning in Graphical Models, Michael
CS 188: Artificial Intelligence Spring Announcements
CS 188: Artificial Intelligence Spring 2011 Lecture 16: Bayes Nets IV Inference 3/28/2011 Pieter Abbeel UC Berkeley Many slides over this course adapted from Dan Klein, Stuart Russell, Andrew Moore Announcements
Machine Learning Lecture 14
Many slides adapted from B. Schiele, S. Roth, Z. Gharahmani Machine Learning Lecture 14 Undirected Graphical Models & Inference 23.06.2015 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de/ leibe@vision.rwth-aachen.de
Directed Graphical Models or Bayesian Networks
Directed Graphical Models or Bayesian Networks Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Bayesian Networks One of the most exciting recent advancements in statistical AI Compact
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
p L yi z n m x N n xi
y i z n x n N x i Overview Directed and undirected graphs Conditional independence Exact inference Latent variables and EM Variational inference Books statistical perspective Graphical Models, S. Lauritzen
6.867 Machine learning, lecture 23 (Jaakkola)
Lecture topics: Markov Random Fields Probabilistic inference Markov Random Fields We will briefly go over undirected graphical models or Markov Random Fields (MRFs) as they will be needed in the context
CS Lecture 4. Markov Random Fields
CS 6347 Lecture 4 Markov Random Fields Recap Announcements First homework is available on elearning Reminder: Office hours Tuesday from 10am-11am Last Time Bayesian networks Today Markov random fields
Intelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 19 Oct, 24, 2016 Slide Sources Raymond J. Mooney University of Texas at Austin D. Koller, Stanford CS - Probabilistic Graphical Models D. Page,
Sum-Product Networks. STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 17, 2017
Sum-Product Networks STAT946 Deep Learning Guest Lecture by Pascal Poupart University of Waterloo October 17, 2017 Introduction Outline What is a Sum-Product Network? Inference Applications In more depth
2 : Directed GMs: Bayesian Networks
10-708: Probabilistic Graphical Models 10-708, Spring 2017 2 : Directed GMs: Bayesian Networks Lecturer: Eric P. Xing Scribes: Jayanth Koushik, Hiroaki Hayashi, Christian Perez Topic: Directed GMs 1 Types
Undirected Graphical Models
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Introduction 2 Properties Properties 3 Generative vs. Conditional
CS 484 Data Mining. Classification 7. Some slides are from Professor Padhraic Smyth at UC Irvine
CS 484 Data Mining Classification 7 Some slides are from Professor Padhraic Smyth at UC Irvine Bayesian Belief networks Conditional independence assumption of Naïve Bayes classifier is too strong. Allows
Chris Bishop s PRML Ch. 8: Graphical Models
Chris Bishop s PRML Ch. 8: Graphical Models January 24, 2008 Introduction Visualize the structure of a probabilistic model Design and motivate new models Insights into the model s properties, in particular
Graphical Models for Automatic Speech Recognition
Graphical Models for Automatic Speech Recognition Advanced Signal Processing SE 2, SS05 Stefan Petrik Signal Processing and Speech Communication Laboratory Graz University of Technology GMs for Automatic
Intelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 11 Oct, 3, 2016 CPSC 422, Lecture 11 Slide 1 422 big picture: Where are we? Query Planning Deterministic Logics First Order Logics Ontologies
Conditional Random Field
Introduction Linear-Chain General Specific Implementations Conclusions Corso di Elaborazione del Linguaggio Naturale Pisa, May, 2011 Introduction Linear-Chain General Specific Implementations Conclusions
Intelligent Systems: Reasoning and Recognition. Reasoning with Bayesian Networks
Intelligent Systems: Reasoning and Recognition James L. Crowley ENSIMAG 2 / MoSIG M1 Second Semester 2016/2017 Lesson 13 24 march 2017 Reasoning with Bayesian Networks Naïve Bayesian Systems...2 Example
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
Introduction to Artificial Intelligence. Unit # 11
Introduction to Artificial Intelligence Unit # 11 1 Course Outline Overview of Artificial Intelligence State Space Representation Search Techniques Machine Learning Logic Probabilistic Reasoning/Bayesian
Machine learning: lecture 20. Tommi S. Jaakkola MIT CSAIL
Machine learning: lecture 20 ommi. Jaakkola MI CAI tommi@csail.mit.edu opics Representation and graphical models examples Bayesian networks examples, specification graphs and independence associated distribution
Intelligent Systems:
Intelligent Systems: Undirected Graphical models (Factor Graphs) (2 lectures) Carsten Rother 15/01/2015 Intelligent Systems: Probabilistic Inference in DGM and UGM Roadmap for next two lectures Definition
CS 188: Artificial Intelligence Fall 2009
CS 188: Artificial Intelligence Fall 2009 Lecture 14: Bayes Nets 10/13/2009 Dan Klein UC Berkeley Announcements Assignments P3 due yesterday W2 due Thursday W1 returned in front (after lecture) Midterm
Probabilistic Graphical Models (I)
Probabilistic Graphical Models (I) Hongxin Zhang zhx@cad.zju.edu.cn State Key Lab of CAD&CG, ZJU 2015-03-31 Probabilistic Graphical Models Modeling many real-world problems => a large number of random
Introduction to Graphical Models
Introduction to Graphical Models The 15 th Winter School of Statistical Physics POSCO International Center & POSTECH, Pohang 2018. 1. 9 (Tue.) Yung-Kyun Noh GENERALIZATION FOR PREDICTION 2 Probabilistic
Probabilistic Reasoning. (Mostly using Bayesian Networks)
Probabilistic Reasoning (Mostly using Bayesian Networks) Introduction: Why probabilistic reasoning? The world is not deterministic. (Usually because information is limited.) Ways of coping with uncertainty
CSC 412 (Lecture 4): Undirected Graphical Models
CSC 412 (Lecture 4): Undirected Graphical Models Raquel Urtasun University of Toronto Feb 2, 2016 R Urtasun (UofT) CSC 412 Feb 2, 2016 1 / 37 Today Undirected Graphical Models: Semantics of the graph:
What you need to know about Kalman Filters
Readings: K&F: 4.5, 12.2, 12.3, 12.4, 18.1, 18.2, 18.3, 18.4 Switching Kalman Filter Dynamic Bayesian Networks Graphical Models 10708 Carlos Guestrin Carnegie Mellon University November 27 th, 2006 1 What
Intelligent Systems (AI-2)
Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 19 Oct, 23, 2015 Slide Sources Raymond J. Mooney University of Texas at Austin D. Koller, Stanford CS - Probabilistic Graphical Models D. Page,
Outline. CSE 573: Artificial Intelligence Autumn Bayes Nets: Big Picture. Bayes Net Semantics. Hidden Markov Models. Example Bayes Net: Car
CSE 573: Artificial Intelligence Autumn 2012 Bayesian Networks Dan Weld Many slides adapted from Dan Klein, Stuart Russell, Andrew Moore & Luke Zettlemoyer Outline Probabilistic models (and inference)
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2014
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2014 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
Machine learning: lecture 20. Tommi S. Jaakkola MIT CSAIL
Machine learning: lecture 20 ommi. Jaakkola MI AI tommi@csail.mit.edu Bayesian networks examples, specification graphs and independence associated distribution Outline ommi Jaakkola, MI AI 2 Bayesian networks
Extensions of Bayesian Networks. Outline. Bayesian Network. Reasoning under Uncertainty. Features of Bayesian Networks.
Extensions of Bayesian Networks Outline Ethan Howe, James Lenfestey, Tom Temple Intro to Dynamic Bayesian Nets (Tom Exact inference in DBNs with demo (Ethan Approximate inference and learning (Tom Probabilistic
Sum-Product Networks: A New Deep Architecture
Sum-Product Networks: A New Deep Architecture Pedro Domingos Dept. Computer Science & Eng. University of Washington Joint work with Hoifung Poon 1 Graphical Models: Challenges Bayesian Network Markov Network
Probabilistic Models
Bayes Nets 1 Probabilistic Models Models describe how (a portion of) the world works Models are always simplifications May not account for every variable May not account for all interactions between variables
Review: Bayesian learning and inference
Review: Bayesian learning and inference Suppose the agent has to make decisions about the value of an unobserved query variable X based on the values of an observed evidence variable E Inference problem:
Computational Genomics
Computational Genomics http://www.cs.cmu.edu/~02710 Introduction to probability, statistics and algorithms (brief) intro to probability Basic notations Random variable - referring to an element / event
Preliminaries Bayesian Networks Graphoid Axioms d-separation Wrap-up. Bayesian Networks. Brandon Malone
Preliminaries Graphoid Axioms d-separation Wrap-up Much of this material is adapted from Chapter 4 of Darwiche s book January 23, 2014 Preliminaries Graphoid Axioms d-separation Wrap-up 1 Preliminaries
Probabilistic Graphical Models: MRFs and CRFs. CSE628: Natural Language Processing Guest Lecturer: Veselin Stoyanov
Probabilistic Graphical Models: MRFs and CRFs CSE628: Natural Language Processing Guest Lecturer: Veselin Stoyanov Why PGMs? PGMs can model joint probabilities of many events. many techniques commonly
Outline of Today s Lecture
University of Washington Department of Electrical Engineering Computer Speech Processing EE516 Winter 2005 Jeff A. Bilmes Lecture 12 Slides Feb 23 rd, 2005 Outline of Today s
Bayesian Networks. Vibhav Gogate The University of Texas at Dallas
Bayesian Networks Vibhav Gogate The University of Texas at Dallas Intro to AI (CS 4365) Many slides over the course adapted from either Dan Klein, Luke Zettlemoyer, Stuart Russell or Andrew Moore 1 Outline
Pattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
Dynamic models 1 Kalman filters, linearization,
Koller & Friedman: Chapter 16 Jordan: Chapters 13, 15 Uri Lerner s Thesis: Chapters 3,9 Dynamic models 1 Kalman filters, linearization, Switching KFs, Assumed density filters Probabilistic Graphical Models
Announcements. CS 188: Artificial Intelligence Fall Causality? Example: Traffic. Topology Limits Distributions. Example: Reverse Traffic
CS 188: Artificial Intelligence Fall 2008 Lecture 16: Bayes Nets III 10/23/2008 Announcements Midterms graded, up on glookup, back Tuesday W4 also graded, back in sections / box Past homeworks in return
Template-Based Representations. Sargur Srihari
Template-Based Representations Sargur srihari@cedar.buffalo.edu 1 Topics Variable-based vs Template-based Temporal Models Basic Assumptions Dynamic Bayesian Networks Hidden Markov Models Linear Dynamical
Bayes Networks. CS540 Bryan R Gibson University of Wisconsin-Madison. Slides adapted from those used by Prof. Jerry Zhu, CS540-1
Bayes Networks CS540 Bryan R Gibson University of Wisconsin-Madison Slides adapted from those used by Prof. Jerry Zhu, CS540-1 1 / 59 Outline Joint Probability: great for inference, terrible to obtain
Implementing Machine Reasoning using Bayesian Network in Big Data Analytics
Implementing Machine Reasoning using Bayesian Network in Big Data Analytics Steve Cheng, Ph.D. Guest Speaker for EECS 6893 Big Data Analytics Columbia University October 26, 2017 Outline Introduction Probability
Probabilistic Graphical Models
Probabilistic Graphical Models David Sontag New York University Lecture 4, February 16, 2012 David Sontag (NYU) Graphical Models Lecture 4, February 16, 2012 1 / 27 Undirected graphical models Reminder
Bayesian Networks (Part II)
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Bayesian Networks (Part II) Graphical Model Readings: Murphy 10 10.2.1 Bishop 8.1,
Graphical models and causality: Directed acyclic graphs (DAGs) and conditional (in)dependence
Graphical models and causality: Directed acyclic graphs (DAGs) and conditional (in)dependence General overview Introduction Directed acyclic graphs (DAGs) and conditional independence DAGs and causal effects
Bayesian Networks. instructor: Matteo Pozzi. x 1. x 2. x 3 x 4. x 5. x 6. x 7. x 8. x 9. Lec : Urban Systems Modeling
12735: Urban Systems Modeling Lec. 09 Bayesian Networks instructor: Matteo Pozzi x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 1 outline example of applications how to shape a problem as a BN complexity of the inference
CPSC 540: Machine Learning
CPSC 540: Machine Learning Undirected Graphical Models Mark Schmidt University of British Columbia Winter 2016 Admin Assignment 3: 2 late days to hand it in today, Thursday is final day. Assignment 4:
CS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Bayes Nets: Independence Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials available at http://ai.berkeley.edu.]
Bayesian Networks. Vibhav Gogate The University of Texas at Dallas
Bayesian Networks Vibhav Gogate The University of Texas at Dallas Intro to AI (CS 6364) Many slides over the course adapted from either Dan Klein, Luke Zettlemoyer, Stuart Russell or Andrew Moore 1 Outline
Recall from last time. Lecture 3: Conditional independence and graph structure. Example: A Bayesian (belief) network.
ecall from last time Lecture 3: onditional independence and graph structure onditional independencies implied by a belief network Independence maps (I-maps) Factorization theorem The Bayes ball algorithm
STA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 11 Project
ECE521 Tutorial 11. Topic Review. ECE521 Winter Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides. ECE521 Tutorial 11 / 4
ECE52 Tutorial Topic Review ECE52 Winter 206 Credits to Alireza Makhzani, Alex Schwing, Rich Zemel and TAs for slides ECE52 Tutorial ECE52 Winter 206 Credits to Alireza / 4 Outline K-means, PCA 2 Bayesian
CS 5522: Artificial Intelligence II
CS 5522: Artificial Intelligence II Bayes Nets Instructor: Alan Ritter Ohio State University [These slides were adapted from CS188 Intro to AI at UC Berkeley. All materials available at http://ai.berkeley.edu.]
CS839: Probabilistic Graphical Models. Lecture 2: Directed Graphical Models. Theo Rekatsinas
CS839: Probabilistic Graphical Models Lecture 2: Directed Graphical Models Theo Rekatsinas 1 Questions Questions? Waiting list Questions on other logistics 2 Section 1 1. Intro to Bayes Nets 3 Section
Announcements. CS 188: Artificial Intelligence Spring Probability recap. Outline. Bayes Nets: Big Picture. Graphical Model Notation
CS 188: Artificial Intelligence Spring 2010 Lecture 15: Bayes Nets II Independence 3/9/2010 Pieter Abbeel UC Berkeley Many slides over the course adapted from Dan Klein, Stuart Russell, Andrew Moore Current
Graphical Models - Part I
Graphical Models - Part I Oliver Schulte - CMPT 726 Bishop PRML Ch. 8, some slides from Russell and Norvig AIMA2e Outline Probabilistic Models Bayesian Networks Markov Random Fields Inference Outline Probabilistic
Bayesian Networks Introduction to Machine Learning. Matt Gormley Lecture 24 April 9, 2018
10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Bayesian Networks Matt Gormley Lecture 24 April 9, 2018 1 Homework 7: HMMs Reminders
Lecture 6: Graphical Models
Lecture 6: Graphical Models Kai-Wei Chang CS @ Uniersity of Virginia kw@kwchang.net Some slides are adapted from Viek Skirmar s course on Structured Prediction 1 So far We discussed sequence labeling tasks:
Bayes Nets: Independence
Bayes Nets: Independence [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Bayes Nets A Bayes
CS 343: Artificial Intelligence
CS 343: Artificial Intelligence Bayes Nets Prof. Scott Niekum The University of Texas at Austin [These slides based on those of Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188
Introduction to Artificial Intelligence (AI)
Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 9 Oct, 11, 2011 Slide credit Approx. Inference : S. Thrun, P, Norvig, D. Klein CPSC 502, Lecture 9 Slide 1 Today Oct 11 Bayesian
Rapid Introduction to Machine Learning/ Deep Learning
Rapid Introduction to Machine Learning/ Deep Learning Hyeong In Choi Seoul National University 1/32 Lecture 5a Bayesian network April 14, 2016 2/32 Table of contents 1 1. Objectives of Lecture 5a 2 2.Bayesian
Probabilistic Models. Models describe how (a portion of) the world works
Probabilistic Models Models describe how (a portion of) the world works Models are always simplifications May not account for every variable May not account for all interactions between variables All models
A graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
Directed and Undirected Graphical Models
Directed and Undirected Graphical Models Adrian Weller MLSALT4 Lecture Feb 26, 2016 With thanks to David Sontag (NYU) and Tony Jebara (Columbia) for use of many slides and illustrations For more information,
CS 188: Artificial Intelligence Fall 2008
CS 188: Artificial Intelligence Fall 2008 Lecture 14: Bayes Nets 10/14/2008 Dan Klein UC Berkeley 1 1 Announcements Midterm 10/21! One page note sheet Review sessions Friday and Sunday (similar) OHs on
Recall from last time: Conditional probabilities. Lecture 2: Belief (Bayesian) networks. Bayes ball. Example (continued) Example: Inference problem
Recall from last time: Conditional probabilities Our probabilistic models will compute and manipulate conditional probabilities. Given two random variables X, Y, we denote by Lecture 2: Belief (Bayesian)
2 : Directed GMs: Bayesian Networks
10-708: Probabilistic Graphical Models, Spring 2015 2 : Directed GMs: Bayesian Networks Lecturer: Eric P. Xing Scribes: Yi Cheng, Cong Lu 1 Notation Here the notations used in this course are defined:
Introduction to Bayesian Learning
Course Information Introduction Introduction to Bayesian Learning Davide Bacciu Dipartimento di Informatica Università di Pisa bacciu@di.unipi.it Apprendimento Automatico: Fondamenti - A.A. 2016/2017 Outline
Tópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863
Tópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863 Daniel, Edmundo, Rosa Terceiro trimestre de 2012 UFRJ - COPPE Programa de Engenharia de Sistemas e Computação Bayesian Networks
Probability. CS 3793/5233 Artificial Intelligence Probability 1
CS 3793/5233 Artificial Intelligence 1 Motivation Motivation Random Variables Semantics Dice Example Joint Dist. Ex. Axioms Agents don t have complete knowledge about the world. Agents need to make decisions
COMS 4771 Probabilistic Reasoning via Graphical Models. Nakul Verma
COMS 4771 Probabilistic Reasoning via Graphical Models Nakul Verma Last time Dimensionality Reduction Linear vs non-linear Dimensionality Reduction Principal Component Analysis (PCA) Non-linear methods
Bayesian belief networks. Inference.
Lecture 13 Bayesian belief networks. Inference. Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Midterm exam Monday, March 17, 2003 In class Closed book Material covered by Wednesday, March 12 Last
Bayesian Networks. Semantics of Bayes Nets. Example (Binary valued Variables) CSC384: Intro to Artificial Intelligence Reasoning under Uncertainty-III
CSC384: Intro to Artificial Intelligence Reasoning under Uncertainty-III Bayesian Networks Announcements: Drop deadline is this Sunday Nov 5 th. All lecture notes needed for T3 posted (L13,,L17). T3 sample
Random Field Models for Applications in Computer Vision
Random Field Models for Applications in Computer Vision Nazre Batool Post-doctorate Fellow, Team AYIN, INRIA Sophia Antipolis Outline Graphical Models Generative vs. Discriminative Classifiers Markov Random
Bayesian Networks Representation
Bayesian Networks Representation Machine Learning 10701/15781 Carlos Guestrin Carnegie Mellon University March 19 th, 2007 Handwriting recognition Character recognition, e.g., kernel SVMs a c z rr r r
Outline. Spring It Introduction Representation. Markov Random Field. Conclusion. Conditional Independence Inference: Variable elimination
Probabilistic Graphical Models COMP 790-90 Seminar Spring 2011 The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Outline It Introduction ti Representation Bayesian network Conditional Independence Inference:
Bayesian Networks. Motivation
Bayesian Networks Computer Sciences 760 Spring 2014 http://pages.cs.wisc.edu/~dpage/cs760/ Motivation Assume we have five Boolean variables,,,, The joint probability is,,,, How many state configurations
CSE 473: Artificial Intelligence Autumn 2011
CSE 473: Artificial Intelligence Autumn 2011 Bayesian Networks Luke Zettlemoyer Many slides over the course adapted from either Dan Klein, Stuart Russell or Andrew Moore 1 Outline Probabilistic models
Lecture 4 October 18th
Directed and undirected graphical models Fall 2017 Lecture 4 October 18th Lecturer: Guillaume Obozinski Scribe: In this lecture, we will assume that all random variables are discrete, to keep notations
15-780: Graduate Artificial Intelligence. Bayesian networks: Construction and inference
15-780: Graduate Artificial Intelligence ayesian networks: Construction and inference ayesian networks: Notations ayesian networks are directed acyclic graphs. Conditional probability tables (CPTs) P(Lo)