Modeling and Inference with Relational Dynamic Bayesian Networks Cristina Manfredotti cristina.manfredotti@gmail.com
The importance of the context Cristina Manfredotti 2
Complex activity recognition Cristina Manfredotti 3
Complex activity recognition Cristina Manfredotti 4
Desiderata Need to model of dynamic contexts and to maintain beliefs over particular relations between objects In order to: 1) Improve tracking with better predictions 2) Identify activities based on observations and prior knowledge Cristina Manfredotti 5
Relational Domain Relational Domain: set of object-types with relations and/or predicates between them Car A Id color ( position(t ( velocity(t ( direction(t ( DecreasingVelocity(t ( SameDirection(t ( distance(t ( Before(t Car B Id color ( position(t ( velocity(t ( direction(t ( DecreasingVelocity(t ( SameDirection(t ( distance(t ( Before(t Cristina Manfredotti 6
Relational State The State of a Relational Domain is the set of instantiations of the objects in the Domain, their attributes and their relations Relational State = State of instantiations (s i ) + State of relations (s r ) Cristina Manfredotti 7
A parenthesis
Bayesian Networks: Encode the joint probability distribution of a set of variables, as a Direct Acyclic Graph A Direct Acyclic Graph which nodes are conditionally independent of its non-descendent given its parents C T T F F B T F T F A P(B A P(C) T ).90 0.01 P(D C,B ).95.94.29.01 D P(E D ) T.90 F.05 C B D E F.05 D P(F D) T.70 F.01 Cristina Manfredotti 9 F A P(A) 0.01
The Alarm example: Cristina Manfredotti 10
Each node is a variable: BNs: a drawback Two different nodes If we would have 4 neighbors? We have to construct a graph with 2 more nodes. Cristina Manfredotti 11
A large BN Thanks to Mark Chavira Cristina Manfredotti 12
The Alarm Relational Domain (1) Object-types neighbor alarm burglar neighbor s attributes: his capacity of hearing, his attention,... alarm s attributes: its volume, its sensibility,... Relational Domain contains a set of objects with relations and/or predicates between them e.g.: Relation e.g.: Predicate tocall (the honer of the house) tohear (the alarm) toring Cristina Manfredotti 13
Alarm RBN: Earthquacke Neigh.DegOfDef Alarm.Volume Neigh.NoiseAround NeighborCalls Cristina Manfredotti 14
Closing the parenthesis... Syntax RBN: a set of nodes, one per variable a directed, acyclic graph a conditional distribution for each node given its parents Cristina Manfredotti 15
Closing the parenthesis... Syntax RBN: a set of nodes, one per predicate/relation/attribute a directed, acyclic graph a conditional distribution for each node given its parents, this distribution must take into account the actual complexity of the nodes! Cristina Manfredotti 16
Dynamics The State of a Relational Domain is the set of instantiations of the objects in the Domain, their attributes an their relations State evolves with time We extend a RBN to a RDBN as we extend a BN to a DBN. Intra-slice distribution: sensor model Inter-slice distribution: transition model Cristina Manfredotti 17
Inference Under Markov assumption Bayesian Filter algorithm: Belief: bel(s t ) = p(s t z 1:t ) = kp(z t s t ) p(s t s t-1 )bel(s t-1 )ds t-1 Relations in the State result in correlating the State of different instantiations between them Cristina Manfredotti 18
Measurement model (1 st assumpt.) part of the state relative to relations, s r, not directly observable p(z t s t ) = p(z t s i t) observation z t independent by the relations between objects. This measurement model only depends on the part of the state of instances. Cristina Manfredotti 19
Transition Model (2 nd assumpt.) p(s t s t-1 )=p(s i t,s r t s i t-1, s r t-1): S i t-1 S i t S r t-1 S r t Cristina Manfredotti 20
Transition Model (2 nd assumpt.) p(s t s t-1 )=p(s i t,s r t s i t-1, s r t-1): S i t-1 S i t S r t-1 S r t Cristina Manfredotti 21
Conditional Probability Distribution CPD relation t (x,y): relation t-1 (x,y) T F(pos t (x), pos t (y)) F CPD pos t (x): G(pos t (x), pos t (y)) F(pos t-1 (x), pos t (y)) relation t-1 (x,y) G(pos t-1 (x)) FOPT: a Probabilistic Tree whose nodes are FOL formulas T y.pos_{t-1} Right StraightLeft x.pos_t Right 0.5 0.15 0.3 Straight 0.35 0.7 0.2 Left 0.15 0.15 0.5 F Cristina Manfredotti 22
Particle Filtering* (general case) Fix the number of particles: M 1. Particle generation x ( m) k ~ p( xk xk 1) Sense the measure at time k: z k 2a. Weight computation 2b. Weight normalization 3. Resampling w = p( z *( m) ( m) k k k w ( m) k = M w m= 1 *( m) k w *( m) k x ) * It is a technique that implements a recursive Bayesian Filter through a Monte Carlo simulation. The key idea is to represent the posterior pdf as a set of samples (particles) paired with weights and to filter the mesurament based on these weights.. Cristina Manfredotti 23
A trick p(s t s t-1 ) = p(s i t,s r t s i t-1, s r t-1) S i t-1 S i t S r t-1 S r t Cristina Manfredotti 24
Relational Relational Transition Inference Model p(s i t,s r t s i t-1,s r t-1) = p(s i t s i t-1,s r t-1) p(s r t s i t-1,s r t-1, s i t) But s r t independent by s i t-1 given s r t-1 and s i t p(s i t,s r t s i t-1,s r t-1) = p(s i t s i t-1,s r t-1) p(s r t s r t-1, s i t) p(z t s i t,s r t) = p(z t s i t) bel(s t ) = p(s t z 1:t ) = p(s i t,s r t z 1:t ) bel(s t )=kp(z t s i t,s r t) p(s i t,s r t s i t-1,s r t-1)bel(s t-1 )ds t-1 Cristina Manfredotti 25
Relational Particle Filter (1) Cristina Manfredotti 26
Relational Particle Filter (2) X i t,(m) X r t,(m) X i t,(m) p(xi t,(m) s i t-1,s r t-1) X i t,(m) X r t,(m) p(xr t,(m) s i t-1 = x r t,(m),s r t-1) Cristina Manfredotti 27
Relational Particle Filter (3) X i t,(m) Weight ( ) p(zt xi t) X r t,(m) The weighting step is done according to the instantiation part of each particle only, the relational part follows. The consistency of the probability function ensures the convergence of the algorithm. Cristina Manfredotti 28
To conclude... Modeling Relations as dynamic context: To improve multi target tracking To recognize complex activities Inference in Dynamic Relational Domain In theory complex BUT Simplified by smart decomposition of the transition model non-relational sensor model... Cristina Manfredotti 29
Challenges Cristina Manfredotti 30