APPROXIMATION COMPLEXITY OF MAP INFERENCE IN SUM-PRODUCT NETWORKS

Transcription

1 APPROXIMATION COMPLEXITY OF MAP INFERENCE IN SUM-PRODUCT NETWORKS Diarmaid Conaty Queen s University Belfast, UK Denis D. Mauá Universidade de São Paulo, Brazil Cassio P. de Campos Queen s University Belfast, UK UAI 27

2 SUM-PRODUCT NETWORKS Efficient representation of the arithmetic expression describing probability function SUM-PRODUCT NETWORK I Rooted DAG with internal nodes + and, and univariate distributions on leaves I I Product nodes have disjoint scope Sum nodes have identical scope I Represents rich mixture probability distribution (with exponentially many components) I Allows efficient marginal inference; usually learned from data 2 / 7

3 SUM-PRODUCT NETWORK X X S(X, )=.2P (X )P 2 ( )+.5P (X )P 2 ( )+.3P (X )P 2 ( ) 3 / 7

4 MAXIMUM A POSTERIORI INFERENCE MAP Given a sum-product net S specified with rational weights and an assignment e, find x such that S(x )=max x e S(x) 4 / 7

5 COMPLEXITY OF MAXIMUM A POSTERIORI INFERENCE Previously known: I Peharz (25) proved NP-hardness from MAXSAT I Peharz et al. (26) adapted NP-hardness of MAP in Naive-Bayes BN (de Campos, 2) I Is tractable when supports of children of sum node are disjoint (Peharz et al. 26) I In practice: Max-Product Algorithm (Poon & Domingos 2) 5 / 7

6 MAX-PRODUCT ALGORITHM.2.5 max X X / 7

7 MAX-PRODUCT ALGORITHM.2.84 max X X / 7

8 MAX-PRODUCT ALGORITHM.2.24 max X X / 7

9 MAX-PRODUCT ALGORITHM.2.26 max X X / 7

10 MAX-PRODUCT ALGORITHM.2.5 max X X maxprod(s) =(, ) S(X =, = ) =.3 6 / 7

11 MAX-PRODUCT ALGORITHM.2.5 max X X Linear time 6 / 7

12 COMPLEXITY OF APPROXIMATE MAP This work: I New proof of NP-hardness I Establish approximation complexity of MAP I Worst-case bounds for Max-Product quality I Improved approximate algorithm: ArgMax-Product I Empirical results comparing both algorithms 7 / 7

13 THEOREM MAP in sum-product networks is NP-complete even if there is no evidence, and the underlying graph is a tree of height 2 Proof: Reduction from maximum independent set: S /6 /3 + /6 /3 4 3 S S 2 S 3 S 4 2 X X 3 X 4 X X 3 X 4 X X 3 X 4 X X 3 X 4 /2 /2 8 / 7

14 COROLLARY Unless P = NP, there is no (m - ) " -approximation algorithm for MAP in sum-product networks for any 6 "<, where m is the number of internal nodes of the networks, even if there is no evidence and the underlying graph is a tree of height 2 9 / 7

15 COROLLARY Unless P = NP, there is no (m - ) " -approximation algorithm for MAP in sum-product networks for any 6 "<, where m is the number of internal nodes of the networks, even if there is no evidence and the underlying graph is a tree of height 2 THEOREM Max-Product returns a (m - )-approximation for sum-product networks whose underlying graph has height at most 2, where m is the number of internal nodes. 9 / 7

16 THEOREM Unless P=NP, there is no 2 s" -approximation algorithm for MAP in sum-product-networks for any 6 "<, where s is the size of the input, even if there is no evidence and the underlying graph is a tree of height 3 Proof: Reduction from SAT using a polynomial number of independent copies of SAT-solving network. E.g. ( X X 3 ) ^ ( X _ X 3 _ X 4 ) /2 /2 /2 /2 /2 /2 /2 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 + X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 X X2 X3 X4 /2 /2 /2 /2 /2 /2 /2 / 7

17 THEOREM Let S + denote the sum nodes in sum-product network S, and d i be the number of children of sum node S i 2 S +. Then Max-Product finds an ( Q S i 2S + d i)-approximation COROLLARY Max-Product returns a 2 " s -approximation for some <"<, where s is the size of the network / 7

18 SO FAR... I Max-Product is optimal (in the worst-case) I Can we find an algorithm with same worst-case performance but better average-case performance? 2 / 7

19 SO FAR... I Max-Product is optimal (in the worst-case) I Can we find an algorithm with same worst-case performance but better average-case performance? I Yes, if we admit quadratic runtime Argmax-Product Algorithm: Single upward pass propagating (partial) configurations and selecting best configuration at sum nodes 2 / 7

20 ARGMAX-PRODUCT ALGORITHM x =.48 x 2 =.72 x 3 = X X amap(s) = arg max x2{x,...,x t } tx w j S j (x) j= 3 / 7

21 ARGMAX-PRODUCT ALGORITHM x =.48 x 2 =.72 x 3 = X X amap(s) = arg max x2{x,...,x t } tx w j S j (x) j= 3 / 7

22 ARGMAX-PRODUCT ALGORITHM x =.48 x 2 =.72 x 3 = X X amap(s) = arg max x2{x,...,x t } tx w j S j (x) j= 3 / 7

23 ARGMAX-PRODUCT ALGORITHM x =.48 x 2 =.72 x 3 = X X amap(s) = arg max x2{x,...,x t } tx w j S j (x) j= 3 / 7

24 ARGMAX-PRODUCT ALGORITHM x 3 = x =.48 x 2 =.72 x 3 = X X amap(s) = arg max x2{x,...,x t } tx w j S j (x) j= 3 / 7

25 THEOREM I Argmax-Product always finds a configuration that is at least as good as the configuration found by MaxProduct: S(amap(S, e)) > S(maxprod(S, e)) I There is a network such that Argmax-Product returns an exponentially better configuration than Max-Product: S(amap(S, e)) > 2 m S(maxprod(S, e)), where m is the number of sum nodes in S 4 / 7

26 EXPERIMENTS: ARGMAX-PRODUCT VS. MAX-PRODUCT Vertices % Edges Nodes Ratio StDev Table: SPNs encoding randomly generated instances of the maximum independent set problem 5 / 7

27 EXPERIMENTS: ARGMAX-PRODUCT VS. MAX-PRODUCT Dataset No. of No. of No Evidence 5% Evidence Height variables samples Ratio Ratio StDev audiology breast-cancer car cylinder-bands flags ionosphere nursery primary-tumor sonar vowel Table: SPNs learned using LearnSPN from UCI datasets 6 / 7

28 CONCLUSION HEIGHT LOWER BOUND UPPER BOUND 2 (m - ) " m - > 3 2 s" 2 s I MAP is hard even to approximate in SPNs I Max-Product performance is optimal in worst-case I Simple modification to algorithm can significantly improve quality (with a decrease in runtime performance) I Goal: quality of Argmax-Product with runtime of Max-Product 7 / 7