Methods that transform the original network into a tighter and tighter representations

Similar documents
Consistency algorithms. Chapter 3

Consistency algorithms

Reasoning with Deterministic and Probabilistic graphical models Class 2: Inference in Constraint Networks Rina Dechter

Min-Max Message Passing and Local Consistency in Constraint Networks

Automata and Formal Languages - CM0081 Finite Automata and Regular Expressions

Graphical Model Inference with Perfect Graphs

13 : Variational Inference: Loopy Belief Propagation

Announcements. Solution to Assignment 3 is posted Assignment 4 is available. c D. Poole and A. Mackworth 2017 CPSC 322 Lecture 8 1 / 25

Probabilistic Graphical Models

Artificial Intelligence

Figure 1: Bayesian network for problem 1. P (A = t) = 0.3 (1) P (C = t) = 0.6 (2) Table 1: CPTs for problem 1. C P (D = t) f 0.9 t 0.

Search and Lookahead. Bernhard Nebel, Julien Hué, and Stefan Wölfl. June 4/6, 2012

6.867 Machine learning, lecture 23 (Jaakkola)

Exam 1. Problem 1: True or false

FACTOR GRAPH OF NON-COMMUTATIVE RING

8. INTRACTABILITY I. Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley. Last updated on 2/6/18 2:16 AM

arxiv:math/ v1 [math.lo] 5 Mar 2007

On shredders and vertex connectivity augmentation

Multi-Agent Systems. Bernhard Nebel, Felix Lindner, and Thorsten Engesser. Summer Term Albert-Ludwigs-Universität Freiburg

Clique trees & Belief Propagation. Siamak Ravanbakhsh Winter 2018

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.

Introduction to Arti Intelligence

SOLVING FINITE-DOMAIN LINEAR CONSTRAINTS IN PRESENCE OF THE ALLDIFFERENT

Classical Complexity and Fixed-Parameter Tractability of Simultaneous Consecutive Ones Submatrix & Editing Problems

COMPSCI 276 Fall 2007

Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science Algorithms For Inference Fall 2014

Basic graph theory 18.S995 - L26.

Solutions to Exercises

Directed Graphical Models or Bayesian Networks

Algorithms and Data Structures 2016 Week 5 solutions (Tues 9th - Fri 12th February)

Lifted Inference: Exact Search Based Algorithms

Generalized Micro-Structures for Non-Binary CSP

Qualifying Exam Logic August 2005

Multi-join Query Evaluation on Big Data Lecture 2

Energy minimization via graph-cuts

Variational algorithms for marginal MAP

Recitation 9: Loopy BP

Discrete (and Continuous) Optimization Solutions of Exercises 2 WI4 131

Inference and Representation

Inference in Graphical Models Variable Elimination and Message Passing Algorithm

Approximate Message Passing

CHAPTER 3 LOW DENSITY PARITY CHECK CODES

Physical Mapping. Restriction Mapping. Lecture 12. A physical map of a DNA tells the location of precisely defined sequences along the molecule.

Loopy Belief Propagation for Bipartite Maximum Weight b-matching

CS264: Beyond Worst-Case Analysis Lecture #11: LP Decoding

Topics in Algorithms. 1 Generation of Basic Combinatorial Objects. Exercises. 1.1 Generation of Subsets. 1. Consider the sum:

COMPSCI 650 Applied Information Theory Apr 5, Lecture 18. Instructor: Arya Mazumdar Scribe: Hamed Zamani, Hadi Zolfaghari, Fatemeh Rezaei

13 : Variational Inference: Loopy Belief Propagation and Mean Field

CS 347 Parallel and Distributed Data Processing

Fast Sorting and Selection. A Lower Bound for Worst Case

Fundamental Algorithms 11

14 : Theory of Variational Inference: Inner and Outer Approximation

Computational Complexity of Inference

Symmetry Breaking Ordering Constraints

Lecture 11 October 11, Information Dissemination through Social Networks

Maxime GUILLAUD. Huawei Technologies Mathematical and Algorithmic Sciences Laboratory, Paris

Multi-coloring and Mycielski s construction

Definable Graphs and Dominating Reals

Knowledge representation DATA INFORMATION KNOWLEDGE WISDOM. Figure Relation ship between data, information knowledge and wisdom.

Petri nets. s 1 s 2. s 3 s 4. directed arcs.

NP and Computational Intractability

Computational complexity theory

Machine Learning 4771

The core of solving constraint problems using Constraint Programming (CP), with emphasis on:

Probabilistic Graphical Models Homework 2: Due February 24, 2014 at 4 pm

14 : Theory of Variational Inference: Inner and Outer Approximation

Lecture 19 : Reed-Muller, Concatenation Codes & Decoding problem

Variational Inference (11/04/13)

Graph Theorizing Peg Solitaire. D. Paul Hoilman East Tennessee State University

Distributed Optimization. Song Chong EE, KAIST

12 : Variational Inference I

Fuzzy and Rough Sets Part I

arxiv: v3 [math.ds] 21 Jan 2018

Cuts. Cuts. Consistent cuts and consistent global states. Global states and cuts. A cut C is a subset of the global history of H

Convergence Analysis of the Variance in. Gaussian Belief Propagation

Math 5490 Network Flows

Lecture 4 : Introduction to Low-density Parity-check Codes

Global constraints and decompositions

UC Berkeley Department of Electrical Engineering and Computer Science Department of Statistics. EECS 281A / STAT 241A Statistical Learning Theory

Locally triangular graphs and normal quotients of n-cubes

cs/ee/ids 143 Communication Networks

Hardness of Embedding Metric Spaces of Equal Size

Distributed Optimization over Random Networks

PAC Learning. prof. dr Arno Siebes. Algorithmic Data Analysis Group Department of Information and Computing Sciences Universiteit Utrecht

Minimum cost transportation problem

1.1 P, NP, and NP-complete

Lecture Notes on Ordered Proofs as Concurrent Programs

arxiv: v1 [cs.dc] 4 Oct 2018

Submodular Functions, Optimization, and Applications to Machine Learning

d 2 -coloring of a Graph

The Complexity of a Reliable Distributed System

Intelligent Systems:

CS 6820 Fall 2014 Lectures, October 3-20, 2014

MATH 61-02: PRACTICE PROBLEMS FOR FINAL EXAM

CS Lecture 4. Markov Random Fields

Computational complexity theory

Generalized Loopy 2U: A New Algorithm for Approximate Inference in Credal Networks

ACO Comprehensive Exam March 20 and 21, Computability, Complexity and Algorithms

Transcription:

hapter

pproximation of inference: rc, path and i-consistecy Methods that transform the original network into a tighter and tighter representations all 00 IS 75 - onstraint Networks

rc-consistency X, Y, Z, T X < Y Y = Z T < Z X T X,, Y,,,,,, T < = < Z all 00 IS 75 - onstraint Networks

rc-consistency X, Y, Z, T X < Y Y = Z T < Z X T X T < Y < Z = all 00 IS 75 - onstraint Networks 4

all 00 IS 75 - onstraint Networks 5

R RX = {,,}, RY = {,,}, constriant reduces domain of X to R = {,}. R X X < Y all 00 IS 75 - onstraint Networks 6

i i π i ( Rij j ) all 00 IS 75 - onstraint Networks 7

!" all 00 IS 75 - onstraint Networks 8

# omplexity (Mackworth and reuder, 986): e = number of arcs, n variables,k values O ( enk (ek^, each loop, nk number of loops), best-case = ek, rc-consistency is: Ω( ek ) ) all 00 IS 75 - onstraint Networks 9

omplexity: O( ek ) est case O(ek), since each arc may be processed in O(k) all 00 IS 75 - onstraint Networks 0

$%&"" ' (%( &&" all 00 IS 75 - onstraint Networks

) omplexity: O( ek ) (ounter is the number of supports to ai in xi from xj. S_(xi,ai) is the set of pairs that (xi,ai) supports) all 00 IS 75 - onstraint Networks

* && Implement - distributedly. Node x_j sends the message to node x_i i h j i i π π i ( Rij j i ( Rij j ) ) Node x_i updates its domain: i π ( R ) i i ij j = i i h j i Messages can be sent asynchronously or scheduled in a topological order all 00 IS 75 - onstraint Networks

+, Example: a triangle graph-coloring with values. Is it arc-consistent? Is it consistent? It is not path, or -consistent. all 00 IS 75 - onstraint Networks 4

- all 00 IS 75 - onstraint Networks 5

- all 00 IS 75 - onstraint Networks 6

R ij omplexity: O(k^) est-case: O(t) Worst-case O(tk) R ij π ij ( Rik k Rkj ) all 00 IS 75 - onstraint Networks 7

-# omplexity: O( n 5 k 5 O(n^) triplets, each take O(k^) steps O(n^ k^) Max number of loops: O(n^ k^). ) all 00 IS 75 - onstraint Networks 8

-. omplexity: Optimal P-4: O( n k 5 O( n ) k ) (each pair deleted may add: n- triplets, number of pairs: O(n^ k^) size of Q is O(n^ k^), processing is O(k^)) all 00 IS 75 - onstraint Networks 9

$%&"& P- requires processings of each arc while P- may not an we do path-consistency distributedly? all 00 IS 75 - onstraint Networks 0

+ all 00 IS 75 - onstraint Networks

/"" ' "" all 00 IS 75 - onstraint Networks

omplexity: for binary constraints i or arbitrary constraints: O((k) ) O( k i ) all 00 IS 75 - onstraint Networks

)!%&" all 00 IS 75 - onstraint Networks 4

+ all 00 IS 75 - onstraint Networks 5

0"( x x π ( R }) x S S {x omplexity: O(t k), t bounds number of tuples. Relational arc-consistency: R π }( R S { x} S { x S x ) all 00 IS 75 - onstraint Networks 6

$%&""( {x+y+z <= 5, z >= } x<=, y<= Example of relational arc-consistency { G, G}, all 00 IS 75 - onstraint Networks 7

Global constraints: e.g., all-different constraints Special semantic constraints that appears often in practice and a specialized constraint propagation. Used in constraint programming. ounds-consistency: pruning the boundaries of domains o exercise 6 all 00 IS 75 - onstraint Networks 8

$%&""" = {,4,5,6} = {,4} = {,,4,5} = {,,4} E = {,4} lldiff (,,,,E} rc-consistency does nothing pply G to sol(,,,,e)? = {6}, = {}. lg: bipartite matching kn^.5 (Lopez-Ortiz, et. l, IJI-0 pp 45 ( fast and simple algorithm for bounds consistency of alldifferent constraint) all 00 IS 75 - onstraint Networks 9

0"" lldifferent Sum constraint Global cardinality constraint (a value can be assigned a bounded number of times) The cummulative constraint (related to scheduling tasks) all 00 IS 75 - onstraint Networks 0

all 00 IS 75 - onstraint Networks

"" all 00 IS 75 - onstraint Networks

"&& ( V ~) and () is arc-consistent relative to but not vice-versa rc-consistency achieved by resolution: res(( V ~),) = Given also ( V ), path-consistency means: res(( V ~),( V )) = ( V ) What can generalized arc-consistency do to cnfs? Relational arc-consistency rule = unit-resolution all 00 IS 75 - onstraint Networks

"&& If lex goes, then ecky goes: If hris goes, then lex goes: Query: Is it possible that hris goes to the party but ecky does not? Is propositional theory ϕ = {,,, } (or, ) (or, ) satisfiable? all 00 IS 75 - onstraint Networks 4

&&" && all 00 IS 75 - onstraint Networks 5

" "" "( Think about the following: G-i apply -i to the dual problem when singleton variables are explicit: the bi-partite representation. What is the complexity? Relational arc-consistency: imitate unit propagation. pply - on the dual problem where each subset of a scope is presented. Is unit propagation equivalent to -4? all 00 IS 75 - onstraint Networks 6

x [,0], x + arc by y = 0 adding y [5,5], consistency x + y x [,5], = 0, y y [5,9] 5 z [ 0,0], y + z path consistency x z 7 obtained by adding, x + y = 0, y z all 00 IS 75 - onstraint Networks 7

"" all 00 IS 75 - onstraint Networks 8

&" '& ) all 00 IS 75 - onstraint Networks 9

* && Implement - distributedly. i i π i ( Rij j ) Node x_j sends the message to node x_i h j i π i ( Rij j ) Node x_i updates its domain: i i h j i Generalized arc-consistency can be implemented distributedly: sending messages between constraints over the dual graph: R π }( R S { x} S { x S x ) all 00 IS 75 - onstraint Networks 40

all 00 IS 75 - onstraint Networks 4 G R R R 4 R R 5 R 6 *"" %&" G The message that R sends to R is R updates its relation and domains and sends messages to neighbors

* R- can be applied to the dual problem of any constraint network. G b) onstraint network all 00 IS 75 - onstraint Networks 4

all 00 IS 75 - onstraint Networks 4 G R R R 4 R R 5 R 6 G 4 5 6 *"4&

all 00 IS 75 - onstraint Networks 44 G +# G 4 5 6 h h h h h 4 h 5 h 4 h 5 4 h 4 h 6 4 h 6 h 5 6 h 4 5 h 5 h 5 h 6 R R R 4 R R 5 R 6

all 00 IS 75 - onstraint Networks 45 G R R R 4 R R 5 R 6 G 4 5 6 +#

all 00 IS 75 - onstraint Networks 46 G G 4 5 6 h h h h h 4 h 5 h 4 h 5 4 h 4 h 6 4 h 6 h 5 6 h 4 5 h 5 h 5 h 6 R R R 4 R R 5 R 6 +.

+. R R R R 4 4 5 6 G R 5 R 6 G all 00 IS 75 - onstraint Networks 47

h 5 + h 4 h R R h h h 4 R h h 5 4 h 6 4 h 4 h R 4 4 5 6 G R 5 5 h 5 h 5 h 6 R 6 6 h 4 6 h 5 G all 00 IS 75 - onstraint Networks 48

+ R R R R 4 4 5 6 G R 5 R 6 G all 00 IS 75 - onstraint Networks 49

h 5 +) h 4 h R R h h h 4 R h h 5 4 h 6 4 h 4 h R 4 4 5 6 G R 5 5 h 5 h 5 h 6 R 6 G 6 h 4 6 h 5 all 00 IS 75 - onstraint Networks 50

+) R R R R 4 4 5 6 G R 5 R 6 G all 00 IS 75 - onstraint Networks 5

+5 R h h h 4 h 5 h 4 h R R h h 5 4 h 6 4 h 4 h R 4 4 5 6 G R 5 5 h 5 h 5 h 6 R 6 G 6 h 4 6 h 5 all 00 IS 75 - onstraint Networks 5

+5 R R R R 4 4 5 6 G R 5 R 6 G all 00 IS 75 - onstraint Networks 5