Tractable Lineages on Treelike Instances: Limits and Extensions

Transcription

1 Tractable Lineages on Treelike Instances: Limits and Extensions Antoine Amarilli 1, Pierre Bourhis 2, Pierre enellart 1,3 June 29th, Télécom ParisTech 2 CNR CRItAL 3 National University of ingapore 1/20

2 Tuple independent databases (TID) Probabilistic databases: model uncertainty about data implest model: tuple independent databases (TID) A relational database I A probability valuation π mapping each fact of I to [0, 1] emantics of a TID (I, π): a probability distribution on I I: Each fact F I is either present or absent with probability π(f) Assume independence across facts 2/20

3 Example: TID a a 1 b v.5 b w.2 3/20

4 Example: TID a a 1 b v.5 b w.2 This TID (I, π) represents the following probability distribution: 3/20

5 Example: TID a a 1 b v.5 b w.2 This TID (I, π) represents the following probability distribution:.5.2 a b b a v w 3/20

6 Example: TID a a 1 b v.5 b w.2 This TID (I, π) represents the following probability distribution: (1.2) a a a a b v b v b w 3/20

7 Example: TID a a 1 b v.5 b w.2 This TID (I, π) represents the following probability distribution: (1.2) (1.5).2 a a a a a a b v b v b w b w 3/20

8 Example: TID a a 1 b v.5 b w.2 This TID (I, π) represents the following probability distribution: (1.2) (1.5).2 (1.5) (1.2) a a a a a a a a b v b v b w b w 3/20

9 Probabilistic query evaluation (PQE) Let us fix: Relational signature σ Class I of relational instances on σ (e.g., acyclic, treelike) Class Q of Boolean queries (e.g., CQs, acyclic CQs) 4/20

10 Probabilistic query evaluation (PQE) Let us fix: Relational signature σ Class I of relational instances on σ (e.g., acyclic, treelike) Class Q of Boolean queries (e.g., CQs, acyclic CQs) Probabilistic query evaluation (PQE) problem for Q and I: Fix a query q Q Read an instance I I and a probability valuation π Compute the probability that (I, π) satisfies q Data complexity: measured as a function of (I, π) 4/20

11 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. 5/20

12 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) 5/20

13 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 5/20

14 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 5/20

15 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 5/20

16 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 5/20

17 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 5/20

18 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: 5/20

19 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here 5/20

20 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here 5/20

21 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here Probability: 5/20

22 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here Probability:.4 5/20

23 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here Probability:.4 (1 5/20

24 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here Probability:.4 (1 (1.5.3) 5/20

25 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here Probability:.4 (1 (1.5.3) (1.2.7)) 5/20

26 Example: PQE ignature σ, class Q of conjunctive queries, class I of all instances. q : x y R(x) (x, y) T(y) R a 1 b.4 c.6 a a 1 b v.5 b w.2 T v.3 w.7 b 1 The query is true iff R(b) is here and one of: (b, v) and T(v) are here (b, w) and T(w) are here Probability:.4 (1 (1.5.3) (1.2.7)) = /20

27 Complexity of probabilistic query evaluation (PQE) Question: what is the complexity of PQE depending on the class Q of queries and class I of instances? 6/20

28 Complexity of probabilistic query evaluation (PQE) Question: what is the complexity of PQE depending on the class Q of queries and class I of instances? Existing dichotomy result on Q [Dalvi and uciu, 2012] Q are unions of CQs, I is all instances There is a class Q of safe queries 6/20

29 Complexity of probabilistic query evaluation (PQE) Question: what is the complexity of PQE depending on the class Q of queries and class I of instances? Existing dichotomy result on Q [Dalvi and uciu, 2012] Q are unions of CQs, I is all instances There is a class Q of safe queries PQE is PTIME for any q on all instances 6/20

30 Complexity of probabilistic query evaluation (PQE) Question: what is the complexity of PQE depending on the class Q of queries and class I of instances? Existing dichotomy result on Q [Dalvi and uciu, 2012] Q are unions of CQs, I is all instances There is a class Q of safe queries PQE is PTIME for any q on all instances PQE is #P-hard for any q Q\ on all instances 6/20

31 Complexity of probabilistic query evaluation (PQE) Question: what is the complexity of PQE depending on the class Q of queries and class I of instances? Existing dichotomy result on Q [Dalvi and uciu, 2012] Q are unions of CQs, I is all instances There is a class Q of safe queries PQE is PTIME for any q on all instances PQE is #P-hard for any q Q\ on all instances q : x y R(x) (x, y) T(y) is unsafe! 6/20

32 Instance-based dichotomy We show a dichotomy result on I instead: Q are Boolean monadic second-order queries (includes all UCQs) 7/20

33 Instance-based dichotomy We show a dichotomy result on I instead: Q are Boolean monadic second-order queries (includes all UCQs) PQE is in linear time for Q on any bounded-treewidth class I up to arithmetic costs [ame authors, ICALP 15] 7/20

34 Instance-based dichotomy We show a dichotomy result on I instead: Q are Boolean monadic second-order queries (includes all UCQs) PQE is in linear time for Q on any bounded-treewidth class I up to arithmetic costs [ame authors, ICALP 15] This talk: PQE is intractable for Q on any unbounded-treewidth class I (under some assumptions) 7/20

35 Instance-based dichotomy We show a dichotomy result on I instead: Q are Boolean monadic second-order queries (includes all UCQs) PQE is in linear time for Q on any bounded-treewidth class I up to arithmetic costs [ame authors, ICALP 15] This talk: PQE is intractable for Q on any unbounded-treewidth class I (under some assumptions) Treewidth measures how much data is close to a tree Trees have treewidth 1 Cycles have treewidth 2 k-cliques and (k 1)-grids have treewidth k 1 7/20

36 Main result: hardness of PQE Theorem For any arity-2 signature σ, there is a first-order query q such that for any constructible unbounded-treewidth class I, the PQE problem for Q = {q} and I is #P-hard under RP reductions Arity-2: Relations have arity 2 (and one has arity 2), i.e., graphs Unbounded-treewidth: for all k N, there is I k I of treewidth k Constructible: given k, we can compute such an instance I k in PTIME #P-hard under RP reductions: reduce in PTIME with high probability from the problem of counting accepting paths of a PTIME machine 8/20

37 Table of contents Introduction Proof sketch Extensions Conclusion 9/20

38 Idea: Topological minors G is a topological minor of H if: 10/20

39 Idea: Topological minors G is a topological minor of H if: G H 10/20

40 Idea: Topological minors G is a topological minor of H if: 1 3 G 2 Embedding: maps edges to vertex-disjoint paths maps vertices to vertices H /20

41 Topological minor extraction results Theorem [Robertson and eymour, 1986] For any planar graph G of degree 3, for any graph H of sufficiently high treewidth, G is a topological minor of H. 11/20

42 Topological minor extraction results Theorem [Robertson and eymour, 1986] For any planar graph G of degree 3, for any graph H of sufficiently high treewidth, G is a topological minor of H. More recently: Theorem [Chekuri and Chuzhoy, 2014] There is a certain constant c N such that for any planar graph G of degree 3 and graph H of treewidth G c, we can embed G as a topological minor of H in PTIME with high proba 11/20

43 Proof sketch Choose a problem from which to reduce: Must be #P-hard on planar degree-3 graphs Must be encodable to an FO query q We use the problem of counting graph matchings 12/20

44 Proof sketch Choose a problem from which to reduce: Must be #P-hard on planar degree-3 graphs Must be encodable to an FO query q We use the problem of counting graph matchings Given an input graph G, compute k = G c 12/20

45 Proof sketch Choose a problem from which to reduce: Must be #P-hard on planar degree-3 graphs Must be encodable to an FO query q We use the problem of counting graph matchings Given an input graph G, compute k = G c Compute in PTIME an instance I k of I of treewidth k 12/20

46 Proof sketch Choose a problem from which to reduce: Must be #P-hard on planar degree-3 graphs Must be encodable to an FO query q We use the problem of counting graph matchings Given an input graph G, compute k = G c Compute in PTIME an instance I k of I of treewidth k Compute in randomized PTIME an embedding of G in I k 12/20

47 Proof sketch Choose a problem from which to reduce: Must be #P-hard on planar degree-3 graphs Must be encodable to an FO query q We use the problem of counting graph matchings Given an input graph G, compute k = G c Compute in PTIME an instance I k of I of treewidth k Compute in randomized PTIME an embedding of G in I k Construct a probability valuation π of I k such that: Unneccessary edges of I k are removed PQE for q gives the answer to the hard problem Easy for MO but trickier for FO 12/20

48 Table of contents Introduction Proof sketch Extensions Conclusion 13/20

49 Non-probabilistic evaluation Apply [Chekuri and Chuzhoy, 2014] to method of [Ganian et al., 2014] for MO non-probabilistic query evaluation (QE): Theorem For any arity-2 signature σ and level Σ P i of the polynomial hierarchy, there is a MO query q i such that, for any constructible, subinstance-closed, unbounded-tw class I, the QE problem for Q = {q i } and I is Σ P i -hard under RP reductions 14/20

50 Non-probabilistic evaluation Apply [Chekuri and Chuzhoy, 2014] to method of [Ganian et al., 2014] for MO non-probabilistic query evaluation (QE): Theorem For any arity-2 signature σ and level Σ P i of the polynomial hierarchy, there is a MO query q i such that, for any constructible, subinstance-closed, unbounded-tw class I, the QE problem for Q = {q i } and I is Σ P i -hard under RP reductions Variant: we also show a result like in [Ganian et al., 2014], with: weaker constructibility requirement on I: I is densely unbounded poly-logarithmically stronger complexity hypothesis: PH does not admit 2 o(n) -sized circuits 14/20

51 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds 15/20

52 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds q : x y R(x) (x, y) T(y) R a r 1 b r 2 c r 3 a a s 1 b v s 2 b w s 3 T v t 1 w t 2 b t 3 15/20

53 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds q : x y R(x) (x, y) T(y) r 2? 1... R a r 1 a a s 1 T v t 1 0 b r 2 b v s 2 w t 2 c r 3 b w s 3 b t 3 0

54 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds q : x y R(x) (x, y) T(y) R T a r 1 a a s 1 v t 1 b r 2 b v s 2 w t 2 0 r 2? 1 s 2? c r 3 b w s 3 b t 3 0

55 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds q : x y R(x) (x, y) T(y) R T a r 1 a a s 1 v t 1 b r 2 b v s 2 w t 2 c r 3 b w s 3 b t 3 0 r 2? 1 s 2? t 1? 1 0 1

56 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds q : x y R(x) (x, y) T(y) R T a r 1 a a s 1 v t 1 b r 2 b v s 2 w t 2 c r 3 b w s 3 b t r 2? 1 s 2? 0 0 s 3? 1 t 1?

57 Lower bound on OBDD representations OBDD for a query q on instance I: ordered decision diagram on the facts of I to decide whether q holds q : x y R(x) (x, y) T(y) R T a r 1 a a s 1 v t 1 b r 2 b v s 2 w t 2 c r 3 b w s 3 b t r 2? 1 s 2? 1 0 t 1? 0 s 3? t 2? /20

58 Lower bound on OBDD representations (result) If we find an OBDD of q on I in PTIME, then PQE of q on I also is how inexistence of poly-size OBDDs (rather than PQE hardness) 16/20

59 Lower bound on OBDD representations (result) If we find an OBDD of q on I in PTIME, then PQE of q on I also is how inexistence of poly-size OBDDs (rather than PQE hardness) Theorem For any arity-2 signature σ, there is a UCQ with inequalities q s.t. for any I of treewidth densely unbounded poly-logarithmically, q has no OBDDs of polynomial size on instances of I 16/20

60 Lower bound on OBDD representations (result) If we find an OBDD of q on I in PTIME, then PQE of q on I also is how inexistence of poly-size OBDDs (rather than PQE hardness) Theorem For any arity-2 signature σ, there is a UCQ with inequalities q s.t. for any I of treewidth densely unbounded poly-logarithmically, q has no OBDDs of polynomial size on instances of I Meta-dichotomy on the connected UCQ for which this holds 16/20

61 Also in paper Hardness results for MO match counting: Count how many subsets X are such that I satisfies q(x) Connect the tractability result of MO on treelike TID to the study of tractable lineages: d-dnnfs, OBDDs, formulae, etc. Connect the same result to tractability of safe queries: Inversion-free queries: subclass of safe queries, tractable OBDDs We can always rewrite their input instances to treelike instances in a lineage-preserving way (hence, probability-preserving) 17/20

62 Table of contents Introduction Proof sketch Extensions Conclusion 18/20

63 ummary of our dichotomy Upper. PQE for MO on treelike instances has linear data complexity up to arithmetic costs 19/20

64 ummary of our dichotomy Upper. PQE for MO on treelike instances has linear data complexity up to arithmetic costs Lower. PQE for FO on any constructible, arity-2, unbounded-tw instance family is #P-hard under RP reductions 19/20

65 ummary of our dichotomy Upper. PQE for MO on treelike instances has linear data complexity up to arithmetic costs Lower. PQE for FO on any constructible, arity-2, unbounded-tw instance family is #P-hard under RP reductions Bounded treewidth is the right notion for tractability of PQE 19/20

66 Future work Can we show #P-hardness under usual P reductions? Depends on [Chekuri and Chuzhoy, 2014] 20/20

67 Future work Can we show #P-hardness under usual P reductions? Depends on [Chekuri and Chuzhoy, 2014] Can we extend the result to arbitrary arity signatures? 20/20

68 Future work Can we show #P-hardness under usual P reductions? Depends on [Chekuri and Chuzhoy, 2014] Can we extend the result to arbitrary arity signatures? Can we extend the result to weaker query languages like UCQ? Conjecture PQE is hard on any constructible unbounded-tw family for: q : (E(x, y) E(y, x)) (E(y, z) E(z, y)) x z This query is alerady hard in terms of OBDDs 20/20

69 Future work Can we show #P-hardness under usual P reductions? Depends on [Chekuri and Chuzhoy, 2014] Can we extend the result to arbitrary arity signatures? Can we extend the result to weaker query languages like UCQ? Conjecture PQE is hard on any constructible unbounded-tw family for: q : (E(x, y) E(y, x)) (E(y, z) E(z, y)) x z This query is alerady hard in terms of OBDDs Thanks for your attention! 20/20

70 References I Amarilli, A., Bourhis, P., and enellart, P. (2015). Provenance circuits for trees and treelike instances. In Proc. ICALP, LNC, pages Chaudhuri,. and Vardi, M. Y. (1992). On the equivalence of recursive and nonrecursive Datalog programs. In POD. Chekuri, C. and Chuzhoy, J. (2014). Polynomial bounds for the grid-minor theorem. In TOC.

71 References II Dalvi, N. and uciu, D. (2012). The dichotomy of probabilistic inference for unions of conjunctive queries. JACM, 59(6):30. Ganian, R., Hliněnỳ, P., Langer, A., Obdržálek, J., Rossmanith, P., and ikdar,. (2014). Lower bounds on the complexity of MO1 model-checking. JC, 1(80). Robertson, N. and eymour, P. D. (1986). Graph minors. V. Excluding a planar graph. J. Comb. Theory, er. B, 41(1).

72 Encoding treelike instances [Chaudhuri and Vardi, 1992] Instance: N a b c d e b c d e f a b c e

73 Encoding treelike instances [Chaudhuri and Vardi, 1992] Instance: Gaifman graph: N a f a b b c c d b e d e e f c d a b c e

74 Encoding treelike instances [Chaudhuri and Vardi, 1992] Instance: Gaifman graph: Tree decomp.: N a f a b c a b c d e b c d e f b c e d c d e b c e e f a b c e

75 Encoding treelike instances [Chaudhuri and Vardi, 1992] Instance: Gaifman graph: Tree decomp.: Tree encoding: N a f a b c N(a 1, a 2 ) a b c d e b c d e f b c e d c d e b c e e f N(a 2, a 3 ) (a 1, a 3 ) (a 2, a 4 ) N(a 3, a 1 ) N(a 4, a 1 ) a b c e N(a 1, a 4 )

76 Defining the query G I k In the embedding, edges of G can become long paths in I k q must answer the hard problem on G despite subdivisions Easy in MO but tricky in FO!

77 Defining the query G I k In the embedding, edges of G can become long paths in I k q must answer the hard problem on G despite subdivisions Easy in MO but tricky in FO! Our q restricts to a subset of the worlds of known weight and gives the right answer up to renormalization