Tópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863

Transcription

1 Tópicos Especiais em Modelagem e Análise - Aprendizado por Máquina CPS863 Daniel, Edmundo, Rosa Terceiro trimestre de 2012 UFRJ - COPPE Programa de Engenharia de Sistemas e Computação

2 Motivação Grandes quantidades de dados disponíveis, levam a diversas questões: Como prever preços de ações a partir do histórico? Dada uma lista de filmes que um usuário gosta, que outros filmes ele gostaria? Como identificar aspectos da saúde de um paciente que são indicativos de uma certa doença? Que documentos de um dado conjunto são relevantes para uma dada busca (Google)?

3 Joint Distributions Let (X 1, X 2, X 3, X 4 ) denote a vector of RV and (x 1, x 2, x 3, x 4 ) a particular realization of these RV Relevant questions: (1) What is the probability of a particular realization? For example, P(X1 =x1, X2 = x2, X3 = x3, X4 = x4) (2) What are the marginal probability distributions? For example, P(X1, X2)

4 Relevant questions: Joint Distributions (3) What are some of the conditional distributions that can be computed? For example, P(X1, X2 X3). (4) What are expected values of functions of random variables? For example, E[ f(x1, X2)] (5) Questions related to independence of the random variables. For example, is X1, X2 conditionally independent of X3?

5 Complexity Issues It is important to keep in mind the huge storage and retrieval issues involved in the representation of the joint. Suppose we have n binary valued random variables, then a naive representation of the joint would translate into storing 2 n elements.

6 Graphical Models Graphical models provide mechanisms for encoding such high-dimensional distributions over a set of RV structuring them compactly The nodes correspond to the RV and the edges correspond to the direct probabilistic interactions between the variables Examples: Markov Chains, HMM, Bayesian Networks

7 Graphical Models

8 Graphical Models Let {X 1,X 2,,X n } be a set of random variables. Questions about them: What are the conditional probabilities? What are the independencies? GM are an economic representation of joint distributions taking advantage of the local relation among the random variables.

9 Directed Graphical Models (DGM) A directed GM is a Directed Acyclic Graph (DAG) G(V, E), where V, the set of vertices, represents the random variables E, the edges, represents the direct dependence among the variables ( parent of relationships) X 1 parent X 2 child

10 Directed Graphical Models (DGM) Descendants: set of nodes that can be reached on a direct path from the node Ancestors: set of nodes from which the node can be reached on a direct path We also define: Π i = set of parents of X i X 1 X 2 X 3 X 4 X 5

11 DAG example Π 6 = {X 2, X 3 } p(x 1:6 ) = p(x 1 )p(x 2 x 1 )p(x 3 x 1 )p(x 4 x 2 )p(x 5 x 3 ) p(x 6 x 2, x 5 )

12 DAG example Only 4 entries If X 1:6 are binary RV the complete table of the joint distribution has 2 6 entries The number of entries of the table using the GM representation is : 6 i=1 2 π i +1 Exponential growth in number of nodes Exponential growth in number of parents

13 Bayesian Networks Definition A Bayesian network B is a Directed Acyclic Graph It represents a joint probability distribution over a set of random variables

14 Bayesian Networks Definition The network is defined by a pair B=(G,Θ), where G is the DAG whose nodes X 1,X 2,,X n represent the direct dependences between these variables If the variable represented by a node is observed, the node is an evidence node, otherwise it is a hidden node

15 Bayesian Networks Definition The graph G encodes independence assumptions, by which each variable X i is independent of its nondescendants given its parents in G. Θ is the set of parameters of the network. This set contains the parameters: θ xi /π i =P(x i /π i ) (for each realization x i of X i )

16 Joint Distribution Chain rule of probability: n p(x 1:n )= p(x i / x 1:i 1 ) i=1 p(x 1:5 )= p(x 1 ) p(x 2 / x 1 ) p(x 3 / x 1, x 2 ) p(x 4 / x 3, x 2, x 1 ) p(x 5 / x 4, x 3, x 2, x 1 ) Independence assumptioms: n p(x )= 1 :n i=1 p(x i / x πi ) Π i is the set of indices of the parents of i

17 Bayesian Networks Definition Θ is the set of parameters of the network. This set contains the parameters: θ xi /π i =P(x i /π i ) (for each realization x i of X i ) B defines a unique joint probability distribution over V: n P(X 1, X 2,..., X n )= i=1 n P(X i /π i )= i=1 θ X i /π i

18 Conditional Independence Independence: X A X B p(x A, x B )= p(x A ) p(x B ) Conditional Independence: X A X B / X C p(x A, x B / x C )= p(x A / x C ) p(x B / x C ) p(x A / x B, x C )= p(x A / x C )

19 Bayesian Networks Example

20 Bayesian Networks Example Paremeters reduced from to 10! P(C, S,W,B, A)=P(C)P(S /C)P(W / S,C) P(B/W, S, C) P( A /B,W, S,C) P(C, S,W, B, A)=P(C)P(S)P(W /C)P(B/S,C)P( A / B)

21 Inference via BN BN is used to compute marginal distributions of one or more query nodes P( X / E=e)= P(X, e) P(e) = w x P( X, e, w) P(x, e) E is the set of observed variables (the evidence nodes) X is the set of unobserved variables whose values we are interested in estimating (query nodes) W are the random variables that are neither query nor evidence nodes

22 BN Inference Example P(C=T, A=T ) P(C=T / A=T )= P( A=T ) P(C=T, A=T )= S,W, B [T, F ] What is the probability of uncomfortable chair given the observation that the person suffers from backache? P(C=T) P(S)P(W /C=T )P(B /S, C=T) P(A=T /B) The number of terms in the sum will grow exponentially P( A=T )= S,W, B,C [T, F ] with the number of hidden nodes P(C) P(S)P(W /C) P(B/ S,C)P( A=T /B)

23 BN Inference Exact inference is an NP-hard problem Some algorithms to restricted classes networks (ex: message passing) Approximate inference methods (ex: Markov Chain Monte Carlo)

24 BN Learning BN is unknown and we want to learn it from the data Given training data and prior information (e.g., expert knowledge, casual relationships), estimate the graph topology (network structure) and the parameters of the JPD in the BN.