Chapter 03: Bayesian Networks

Transcription

1 LEARNING AND INFERENCE IN GRAPHICAL MODELS Chapter 03: Bayesian Networks Dr. Martin Lauer University of Freiburg Machine Learning Lab Karlsruhe Institute of Technology Institute of Measurement and Control Systems Learning and Inference in Graphical Models. Chapter 03 p. 1/15

2 References for this chapter Christopher M. Bishop, Pattern Recognition and Machine Learning, ch. 8, Springer, 2006 Stuart Russell and Peter Norvig, Artificial Intelligenece: A Modern Approach, ch. 14, Prentice Hall, 2003 Learning and Inference in Graphical Models. Chapter 03 p. 2/15

3 Motivation Bayesian inference very useful for non-trivial problems: modeling with several random variables which depend in a complex way on each other understanding the inference process becomes hard graphical representation of dependencies in a graph structure Learning and Inference in Graphical Models. Chapter 03 p. 3/15

4 Bayesian networks a Bayesian network is a directed, weakly connected, acyclic graph each node represents a random variable open circles indicate non-observed random variables filled circles indicate observed random variables dots indicate given constants A B C links (arrows) indicate an explicitly modeled stochastic dependence in the form of a conditional density D Bayesian networks are also called belief networks. Learning and Inference in Graphical Models. Chapter 03 p. 4/15

5 Bayesian networks Joint probability in a Bayesian network over all nodes Definition: the joint probability of a Bayesian network with nodesa 1,...,A ν is given by A p(a 1,...,A ν ) = ν p(a j Pred(A j )) B C j=1 wherepred(a) denotes the set of all nodesb for which a connectionb A exists in the network. D Example: p(a,b,c,d) = p(a) p(b A) p(c) p(d B,C) Learning and Inference in Graphical Models. Chapter 03 p. 5/15

6 Bayesian networks Examples: tossing a coin 5 times (α 1,α 2 ) (q 1,q 2 ) X 1 X 2 X 3 X 4 X 5 p(q 1,q 2,X 1,...,X 5 ) = p(q 1,q 2 α 1,α 2 ) p(x 1 q 1,q 2 ) p(x 2 q 1,q 2 ) p(x 3 q 1,q 2 ) p(x 4 q 1,q 2 ) p(x 5 q 1,q 2 ) Remark: constants do not contribute to the left side of conditionals Learning and Inference in Graphical Models. Chapter 03 p. 6/15

7 Bayesian networks Copying nodes for independently identically distributed random variables is annoying plates (α 1,α 2 ) (q 1,q 2 ) X i 5 Learning and Inference in Graphical Models. Chapter 03 p. 7/15

8 Independence in Bayesian networks Example: consider the network on the right how can we simplify p(e A,B,C,D,F,G,H)? node E is independent on which of the other nodes? blackboard A B C D p(e A,B,C,D,F,G,H) = p(e B,C,F,G) E F E is stochastically independent of A, D, and H if we know B,C,F, and G G H Learning and Inference in Graphical Models. Chapter 03 p. 8/15

9 Independence in Bayesian networks More general: p(x rest) = p(x Pred(X) Succ(X) Cop(X)) with Cop(X) = ( Y Succ(X) Pred(Y)) \{X} the coparents of X Proof: let s definen = Succ(X) Pred(X) Cop(X) andr = {all nodes}\(n {X}) then, by definition of a Bayesian network, the joint probability can be split as p(x,n,r)=p(x Pred(X)) Y Succ(X) p(y X,Pred(Y)\{X}) } {{ } =:V(X,N) =V(X,N) C(N,R)... }{{} =:C(N,R) Learning and Inference in Graphical Models. Chapter 03 p. 9/15

10 Independence in Bayesian networks Then, p(x =x N = n,r = r) = = p(x = x,n = n,r = r) p(n = n,r = r) p(x = x,n = n,r = r) p(x = x,n = n,r = r)dx = V(x,n) C(n,r) V(x,n) C(n,r)dx = V(x,n) C(n,r) V(x,n)dx C(n,r) = V(x,n) V(x,n)dx V(x,n) C(n,r )dr = = V(x,n)dx C(n,r )dr V(x,n) C(n,r )dr V(x,n) C(n,r )dx dr = p(x = x,n = n,r = r )dr p(x = x,n = n,r = r )dx dr = p(x = x,n = n) p(n = n) = p(x = x N = n) Learning and Inference in Graphical Models. Chapter 03 p. 10/15

11 Independence in Bayesian networks The set Pred(X) Succ(X) Cop(X) is called the Markov blanket ofx Why do the coparents belong to the Markov blanket? X Example: the exam problem the lecturer cannot access the mental mood. Hence, he will consider p(comprehension exam) the student knows his mental state. He want s the lecturer to consider p(comprehension exam, mood) level of comprehension mental mood result of exam Learning and Inference in Graphical Models. Chapter 03 p. 11/15

12 Examples Develop Bayesian networks for the following tasks on the blackboard. Discuss the structure of the network, the type of network nodes, and conditional distribution that can be assigned to each node. level of maturity fruit type classification of euro coins depending on diameter, thickness, and weight color size shape extend the exam problem making multiple measurements with a biased measurement tool/calibrating a biased measurement tool contour linear regression reconsider the fruit type problem from chapter 1 image Learning and Inference in Graphical Models. Chapter 03 p. 12/15

13 Gibbs distribution A Gibbs distribution is often useful for things that can be described hardly with standard distributions for which a function E can be determined which models whether an outcome is likely (E(x) small) or unlikely (E(x) large) IfE(x) is a real-valued function andβ > 0 and e βe(x) dx <, then p(x) = 1 Z(β) e βe(x) is a Gibbs distribution (or Boltzmann distribution). The normalization constant Z(β) = e βe(x) dx is called the partition function. Learning and Inference in Graphical Models. Chapter 03 p. 13/15

14 Gibbs distribution Examples: modeling the distribution of images of apples function E measures the similarity of template and image pixelwise, e.g. E(image) = u v g(u,v) t(u,v) β can be used to model how much deviation from the template is tolerated β large: only images contribute to the distribution which are very similar to the template β close to0: images which are dissimilar also have a probability significantly larger than0 template of an applet(u,v) image of an appleg(u,v) Learning and Inference in Graphical Models. Chapter 03 p. 14/15

15 Summary structure of Bayesian networks joint probability independence in Bayesian networks examples Gibbs distribution follows in subsequent chapters: various methods for stochastic inference in Bayesian networks hidden Markov models as special form of Bayesian networks Learning and Inference in Graphical Models. Chapter 03 p. 15/15