Graphical Models Chris Bishop

Transcription

1 Graphical Models Chris Bishop Microsoft Research Cambridge Machine Learning Summer School 2013, Tübingen

2 Chapter 8: Graphical Models (PDF download)

3

4

5 1. Introduction

6 Traditional machine learning K-means clustering logistic regression random forest neural networks support vector machines Markov random field RVM Gaussian mixture Kalman filter HMM principal components deep networks kernel PCA ICA linear regression Radial basis functions Gaussian process factor analysis Boltzmann machines decision trees

7

8

9

10 infra-red emitter infra-red camera

11

12 (w 1, w 2, w N )

13 Fast depth image features Depth comparisons: f(x i ; Δ) = d(x i ) d(x i ) where x i = x i + Δ/d(x i ) i Δ i i Δ Δ Δ i i i i i input depth image

14

15 2. Model-based Machine Learning

16 Model-based machine learning Goal: A single development framework which supports the creation of a wide range of bespoke models Traditional: how do I map my problem into standard tools? Model-based: what is the model that represents my problem?

17 Potential benefits of MBML Models optimised for each new application Transparent functionality Models expressed as compact code Community of model builders Segregate model from training/inference code Newcomers learn one modelling environment Does the right thing automatically

18 Intelligent software Goal: software that can adapt, learn, and reason Player skill Movie preferences Words Game result Ratings Can be described by a model Ink

19 Intelligent software Goal: software that can adapt, learn, and reason Player skill Movie preferences Words Game result Ratings Reasoning backwards Ink

20 3. Uncertainty

21 Handling uncertainty We are uncertain about a player s skill Each result provides relevant information But we are never completely certain How can we compute with uncertainty in a principled way?

22 Uncertainty everywhere Which movie should the user watch next? Which word did the user write? What did the user say? Which web page is the user trying to find? Which link will user click on? What kind of product does the user wish to buy? Which gesture is the user making? Many others

23 Probability Limit of infinite number of trials Quantification of uncertainty 60% 40%

24 Movie Recommender Demo Matchbox

25 Xbox Live Recommendation Over 50M users Serves more than 100M requests per day Spans verticals: games, TV programmes, movies

26 4. Probabilities

27 A murder mystery A fiendish murder has been committed Whodunit? There are two suspects: the Butler the Cook There are three possible murder weapons: a butcher s Knife a Pistol a fireplace Poker

28 Prior distribution Butler has served family well for many years Cook hired recently, rumours of dodgy history P(Culprit = Butler) = 20% P(Culprit = Cook) = 80% Probabilities add to 100% P(Culprit) This is called a factor graph (we ll see why later) Culprit = {Butler, Cook}

29 Conditional distribution Butler is ex-army, keeps a gun in a locked drawer Cook has access to lots of knives Butler is older and getting frail P(Weapon Culprit)

30 Factor graph P(Culprit) Prior distribution Culprit = {Butler, Cook} Conditional distribution P(Weapon Culprit) Weapon = {Pistol, Knife, Poker}

31 Joint distribution What is the probability that the Cook committed the murder using the Pistol? P(Culprit = Cook) = 80% P(Weapon = Pistol Culprit = Cook) = 5% P(Weapon = Pistol, Culprit = Cook) = 80% x 5% = 4% Likewise for the other five combinations of Culprit and Weapon

32 Joint distribution = 100% P(Weapon, Culprit) = P(Weapon Culprit) P(Culprit) P x, y = P y x P(x) Product rule

33 Factor graphs P(Culprit) Culprit = {Butler, Cook} P(Weapon Culprit) Generative model Weapon = {Pistol, Knife, Poker} P(Weapon, Culprit) = P(Weapon Culprit) P(Culprit)

34 Generative viewpoint Murderer Cook Butler Cook Cook Cook Butler Cook Cook Butler Cook Weapon Knife Knife Pistol Poker Knife Pistol Poker Knife Pistol Knife

35 Marginal distribution of Culprit P x = y P x, y Sum rule

36 Marginal distribution of Weapon P x = y P x, y Sum rule

37 Posterior distribution We discover a Pistol at the scene of the crime = 20% = 80% This looks bad for the Butler!

38 Generative viewpoint Murderer Cook Butler Cook Cook Cook Butler Cook Cook Butler Cook Weapon Knife Knife Pistol Poker Knife Pistol Poker Knife Pistol Knife

39 Reasoning backwards Culprit Weapon

40 Bayes theorem posterior P x, y = P y x P x = P x y P(y) likelihood P x y P(y) P y x = P(x) prior Prior belief before making a particular obs. Posterior belief after making the obs. Posterior is the prior for the next observation Intrinsically incremental

41 Two views of probability Frequency: limit of infinite number of trials Bayesian: quantification of uncertainty

42 The Rules of Probability Sum rule P x = P x, y Product rule y P x, y = P y x P(x) Bayes theorem Denominator P y x = P x = y P x y P(y) P(x) P x y)p(y

43

44 5. Directed Graphs

45 Probabilistic Graphical Models Combine probability theory with graphs new insights into existing models framework for designing new models Graph-based algorithms for calculation and computation (c.f. Feynman diagrams in physics) efficient software implementation

46 Three types of graphical model Directed graphs useful for designing models Undirected graphs good for some domains, e.g. computer vision Factor graphs useful for inference and learning

47 Decomposition Consider an arbitrary joint distribution By successive application of the product rule:

48 Directed Graphs Arrows may indicate causal relationships

49 Special cases PCA, ICA, factor analysis, linear regression, logistic regression, mixture models Kalman filters, hidden Markov models

50 We re hiring! Interns, Postdocs, Researchers, Developers

51 6. Conditional Independence

52 Conditional Independence

53 Conditional Independence: Example 1

54 Conditional Independence: Example 1

55 Conditional Independence: Example 2

56 Conditional Independence: Example 2

57 Conditional Independence: Example 3

58 Conditional Independence: Example 3

59 D-separation

60 Two coins p(h) = 1/2 p(h) = 1/2 Coin 1 {H,T} Coin 2 {H,T} Both heads {true, false}

61 What is the probability of two heads? p(h) = 1/2 p(h) = 1/2 Coin 1 {H,T} Coin 2 {H,T} Generative model Coin 1 T T H H Coin 2 T H T H Both heads false false false true Both heads {true, false} p(true) = 1/4

62 Reasoning backwards p(h) = 1/3 p(h) = 1/3 Coin 1 {H,T} Coin 2 {H,T} false Inference Both heads {true, false} Coin 1 T T H H Coin 2 T H T H Both heads false false false true

63 Reasoning backwards p(h) = 1/2 Coin 1 {H,T} tails Coin 2 {H,T} false Both heads {true, false} Coin 1 T T H H Coin 2 T H T H Both heads false false false true

64 Reasoning backwards p(h) = 0 Coin 1 {H,T} heads Coin 2 {H,T} false Both heads {true, false} Coin 1 T T H H Coin 2 T H T H Both heads false false false true Explaining away

65 7. Undirected Graphs

66 Undirected Graphs Markov random fields

67 Factorization Clique Maximal Clique M K-state variables K M terms in Z

68 Illustration: Image De-Noising

69

70

71 Directed versus Undirected

72 8. Factor Graphs

73 Factorization Directed graphs: Undirected graphs: Both have the form of products of factors:

74 Factor Graphs

75 From Directed Graph to Factor Graph

76 9. Inference

77 Efficient inference

78 The Sum-Product Algorithm u f 1 (u, w) w f 2 (w, x) x f 3 (x, y) f 4 (x, z) y z

79

80

81

82 The Sum-Product Algorithm Three update equations Message schedule from root to leaves and back One message in each direction on each link

83 What if the graph is not a tree? Condition on variables to break loops cut-set conditioning (exact) Transform graph into tree of composite nodes junction tree algorithm (exact) Approximate: keep iterating the messages: loopy belief propagation (approximate)

84 What if the messages are intractable? True distribution Monte Carlo Variational Message Passing Expectation propagation

85 Learning is just inference!

86 10. Example: Kalman filter

87 Hand location Noisy position sensor

88 Finding the true location x

89 The Gaussian distribution standard deviation σ variance σ 2 precision λ = 1/σ 2 mean μ

90 The multi-dimensional Gaussian

91 Learning the mean p(μ) μ p(x μ) x

92 Learning the mean μ x

93 Plates μ x N

94 μ

95 Hand tracking Noisy position sensor and moving hand

96 μ x t = 1 t = 2 t = 3 t = 4 The Kalman filter (The hidden Markov model)

97

98

99

100 What about the noise level? p(λ) λ p(μ) μ p(x μ, λ) x

101 The gamma distribution An example of a conjugate prior

102 Predictions p(λ) λ p(μ) μ p(x μ, λ) N x

103 current Big data? voltage

104

105 11. Case Study: TrueSkill

106 TrueSkill TM Ralf Herbrich, Tom Minka, and Thore Graepel (NIPS, 2007)

107 Elo International standard for chess grading A single rating for each player Limitations: not applicable to more than two players not applicable to team games

108 Stages of MBML 1. Build a model: joint probability distribution of all of the relevant variables (e.g. as a graph) 2. Incorporate the observed data 3. Compute the distributions over the desired variables: inference Iterate 2 and 3 in real-time applications Extend model as required

109 Gaussian Gaussian s 1 s 2 Gaussian Gaussian 1 2 I( 1 > 2 ) y 12

110 s 1 s y 12

111 s 1 s y 12

112 s 1 s y 12

113 Expectation propagation (EP)

114 Level Convergence char (TrueSkill ) SQLWildman (TrueSkill ) 5 char (Elo) SQLWildman (Elo) Number of Games

115 13. Probabilistic Programming

116 CSOFT A representation language for probabilistic models. Takes C# and adds support for: random variables constraints on variables inference Can be embedded in ordinary C# to allow integration of deterministic + stochastic code

117 Random variables Normal variables have a fixed single value int length=6 Random variables have a probability distribution int length = random(uniform(0,10))

118 Constraints Constraints on random variables constrain(visible==true) constrain(length==4) constrain(length>0) constrain(i==j)

119 Inference Compute posterior distribution int i = random(uniform(1,10)); bool b = (i*i>50); Dist bdist = infer(b); //Bernoulli(0.3)

120 Random variables Probabilistic program double x = random(gaussian(0,1)); Graphical model Gaussian(0,1) x

121 Bayesian networks Probabilistic program double x = random(gaussian(0,1)); double y = random(gamma(1,1)); double z = random(gaussian(x,y)); Graphical model Gaussian(0,1) Gamma(1,1)) x y Gaussian z

122 Loops plates Probabilistic program double x = random(gaussian(0,1)); double y = random(gamma(1,1)); for(int i=0;i<10;i++) { double z = random(gaussian(x,y)); } Graphical model Gaussian(0,1) Gamma(1,1)) x y Gaussian z i=0..9

123 If statement gates Probabilistic program bool b = random(bernoulli(0.5)); double x; if (b) { x = random(gaussian(0,1)); } else { x = random(gaussian(10,1)); } Graphical model Bernoulli(0.5) b T Gaussian(0,1) F Gaussian(10,1) x Gates (Minka and Winn, NIPS 2008)

124 Other language features Probabilistic program Functions/recursion Indexing Jagged arrays Mutation: x=x+1 Objects... Graphical model No common equivalent

125 Sampling interpretation Imagine running program many times, where random(dist) draws a random number from dist constrain(b) stops the run if b is not true infer(x) accumulates the value of x into memory

126 John Winn, Tom Minka, John Guiver, et al.

127 How Infer.NET works Probabilistic program (model) Observed values Inference Engine Infer.NET compiler C# C# compiler Algorithm Algorithm execution Probability distributions

128 Standard models supported Mixture models Factor analysis / PCA / ICA Logistic regression Discrete Bayesian networks Hidden Markov models Ranking models Kalman filters Hierarchical models

129 s 1 s y 12

130 // model variables Variable<double> skill1, skill2; Variable<double> performance1, performance2; Gaussian skillposterior1, skillposterior2; // model skill1 = Variable.GaussianFromMeanAndPrecision(0, 1); skill2 = Variable.GaussianFromMeanAndPrecision(0, 1); performance1 = Variable.GaussianFromMeanAndPrecision(skill1, beta); performance2 = Variable.GaussianFromMeanAndPrecision(skill2, beta); Variable.ConstrainPositive(performance1 performance2); // infer new posterior skills InferenceEngine engine = new InferenceEngine(); skillposterior1 = engine.infer<gaussian>(skill1); skillposterior2 = engine.infer<gaussian>(skill2);

131 Extension to Multiple players s 1 s 2 s y 12 y 23

132 // model variables Variable<double> skill1, skill2, skill3; Variable<double> performance1, performance2, performance3; Gaussian skillposterior1,skillposterior2, skillposterior3; // model skill1 = Variable.GaussianFromMeanAndPrecision(0, 1); skill2 = Variable.GaussianFromMeanAndPrecision(0, 1); skill3 = Variable.GaussianFromMeanAndPrecision(0, 1); performance1 = Variable.GaussianFromMeanAndPrecision(skill1, beta); performance2 = Variable.GaussianFromMeanAndPrecision(skill2, beta); performance3 = Variable.GaussianFromMeanAndPrecision(skill3, beta); Variable.ConstrainPositive(performance1 - performance2); Variable.ConstrainPositive(performance2 - performance3); // infer new posterior skills InferenceEngine engine = new InferenceEngine(); skillposterior1 = engine.infer<gaussian>(skill1); skillposterior2 = engine.infer<gaussian>(skill2); skillposterior3 = engine.infer<gaussian>(skill3);

133 Extension to Teams s 1 s 2 s 3 s y 12

134 // model variables Variable<double> skill1, skill2, skill3, skill4; Variable<double> performance1, performance2, performance3, performance4; Gaussian skillposterior1,skillposterior2, skillposterior3, skillposterior4; // model skill1 = Variable.GaussianFromMeanAndPrecision(0, 1); skill2 = Variable.GaussianFromMeanAndPrecision(0, 1); skill3 = Variable.GaussianFromMeanAndPrecision(0, 1); skill4 = Variable.GaussianFromMeanAndPrecision(0, 1); performance1 = Variable.GaussianFromMeanAndPrecision(skill1 + skill2, beta); performance2 = Variable.GaussianFromMeanAndPrecision(skill3 + skill4, beta); Variable.ConstrainPositive(performance1 - performance2); // infer new posterior skills InferenceEngine engine = new InferenceEngine(); skillposterior1 = engine.infer<gaussian>(skill1); skillposterior2 = engine.infer<gaussian>(skill2); skillposterior3 = engine.infer<gaussian>(skill3); skillposterior4 = engine.infer<gaussian>(skill4);

135 TrueSkill TM through time s 1 s 2 ~ ~ s 1 s ~ ~ 1 2 y 12 ~ y 12

136 // model variables Variable<double> skill1, skill2; Variable<double> performance1, performance2; Gaussian skillposterior1, skillposterior2; // model skill1 = Variable.GaussianFromMeanAndPrecision(oldskill1, alpha); skill2 = Variable.GaussianFromMeanAndPrecision(oldskill2, alpha); performance1 = Variable.GaussianFromMeanAndPrecision(skill1, beta); performance2 = Variable.GaussianFromMeanAndPrecision(skill2,beta); Variable.ConstrainPositive(performance1 - performance2); // infer new posterior skills InferenceEngine engine = new InferenceEngine(); skillposterior1 = engine.infer<gaussian>(skill1); skillposterior2 = engine.infer<gaussian>(skill2);

137 Skill estimate ChessBase Analysis: M game outcomes 20 million variables (200,000 players in each year of lifetime + latent variables) 40 million factors 3000 Garry Kasparov Robert James Fischer Anatoly Karpov 2400 Paul Morphy Mikhail Botvinnik 2200 Whilhelm Steinitz Boris V Spassky 2000 Emanuel Lasker Adolf Anderssen Jose Raul Capablanca Year

138

139 Any questions?

140