arxiv: v1 [cs.ai] 28 Jun 2016

Transcription

1 On the Semantic Relationship beteen Probabilistic Soft Logic and Markov Logic Joohyung Lee and Yi Wang School of Computing, Informatics, and Decision Systems Engineering Arizona State University Tempe, AZ, 85287, USA arxiv: v1 [cs.ai] 28 Jun 2016 Abstract Markov Logic Netorks MLN and Probabilistic Soft Logic PSL are idely applied formalisms in Statistical Relational Learning, an emerging area in Artificial Intelligence that is concerned ith combining logical and statistical AI. Despite their resemblance, the relationship has not been formally stated. In this paper, e describe the precise semantic relationship beteen them from a logical perspective. This is facilitated by first extending fuzzy logic to allo eights, hich can be also vieed as a generalization of PSL, and then relate that generalization to MLN. We observe that the relationship beteen PSL and MLN is analogous to the knon relationship beteen fuzzy logic and Boolean logic, and furthermore the eight scheme of PSL is essentially a generalization of the eight scheme of MLN for the many-valued setting. 1 Introduction Statistical relational learning SRL is an emerging area in Artificial Intelligence that is concerned ith combining logical and statistical AI. Markov Logic Netorks MLN Richardson and Domingos 2006 and Probabilistic Soft Logic PSL Kimmig et al. 2012; Bach et al are ell-knon formalisms in statistical relational learning, and have been successfully applied to a ide range of AI applications, such as natural language processing, entity resolution, collective classification, and social netork modeling. Both of them combine logic and probabilistic graphical model in a single representation, here each formula is associated ith a eight, and the probability distribution over possible orlds is derived from the eights of the formulas that are satisfied by the possible orlds. Hoever, despite their resemblance to each other, the precise relationship beteen their semantics is not obvious. PSL is based on fuzzy interpretations that range over reals in[0,1], and in this sense is more general than MLN. On the other hand, its syntax is restricted to formulas in clausal form, unlike MLN that allos any complex formulas. It is also not obvious ho their models eights are related to each other due to the different ays that the eights are associated ith models. Originating from the machine learning research, these formalisms 1 In Working Notes of the 6th International Workshop on Statistical Relational AI StarAI 2016 are equipped ith several efficient inference and learning algorithms, and some paper compares the suitability of one formalism over the other by experiments on specific applications Beltagy, Erk, and Mooney On the other hand, the precise relationship beteen the to formalisms has not been formally stated. In this paper, e present a precise semantic relationship beteen them. We observe that the relationship is analogous to the ell-knon relationship beteen fuzzy logic and classical logic. Moreover, despite the different ays that eights of models are defined in each formalism, it turns out that they are essentially of the same kind. Toards this end, e introduce a eighted fuzzy logic as a proper generalization of PSL, hich is also interesting on its on as an extension of the standard fuzzy logic to incorporate eighted models. The eighted fuzzy logic uses the same eight scheme as PSL, but associates eights to arbitrary fuzzy formulas. This intermediate formalism facilitates the comparison beteen PSL and MLN. We observe that the same analogy beteen fuzzy logic and Boolean logic carries over to beteen PSL and MLN. Analogous to that fuzzy logic agrees ith Boolean logic on crisp interpretations, PSL and MLN agree on crisp interpretations, here their eights are proportional to each other. Hoever, their maximum a posteriori MAP estimates do not necessarily coincide due to the differences beteen many-valued vs. Boolean models. The paper is organized as follos. We first revie each of MLN, fuzzy propositional logic, and PSL. Then e define a eighted fuzzy logic as a generalization of PSL. Using this e study the semantic relationship beteen PSL and MLN. Preliminaries Although both PSL and MLN allo atoms to contain variables, those variables are understood in terms of grounding over finite domains here a universally quantified sentence is turned into multiple conjunctions and an existentially quantified sentence is turned into multiple disjunctions, essentially resulting in propositional theories. For example, the ground atoms of the first-order signature σ = {p,a,b}, herepis a unary predicate constant anda,bare object constants, can be identified ith the propositional atoms of the propositional signature{pa, pb}. Thus for simplicity but ithout losing generality, e assume that the programs are

2 propositional. 2 Revie: Markov Logic Netorks The folloing is a revie of Markov Logic from Richardson and Domingos A Markov Logic Netork MLN L of a propositional signature σ is a finite set of pairs : F, here F is a propositional formula ofσ and is a real number. For any MLN L of signature σ, e define L I to be the set of eighted formulas : F in L such that I = F. The unnormalized eight of an interpretation I under L is defined as W L I = exp, :F L I and the normalized eight a.k.a. probability of I under L is defined as P L I = W L I J PW W LJ, here PW Possible Worlds is the set of all interpretations ofσ. The basic idea of Markov Logic is to allo formulas to be soft constrained, here a model does not have to satisfy all formulas, but is associated ith the eight that is obtained from the satisfied formulas. An interpretation that does not satisfy certain formulas receives an indirect penalty because such formulas do not contribute to the eight of that interpretation. Revie: Fuzzy Propositional Formula The folloing is a revie of fuzzy propositional formulas from Hajek A fuzzy propositional signature σ is a set of symbols called fuzzy atoms. In addition, e assume the presence of a setconj of fuzzy conjunction symbols, a set DISJ of fuzzy disjunction symbols, a set NEG of fuzzy negation symbols, and a set IMPL of fuzzy implication symbols. A fuzzy propositional formula of σ is defined recursively as follos. every fuzzy atom p σ is a fuzzy formula; every numeric constant c, here c is a real number in [0,1], is a fuzzy formula; iff is a fuzzy formula, then F is a fuzzy formula, here NEG; if F and G are fuzzy formulas, then F G, F G, and F G are fuzzy formulas, here CONJ, DISJ, and IMPL. The models of a fuzzy formula are defined as follos. The fuzzy truth values are the real numbers in the range[0,1]. A fuzzy interpretation I of σ is a mapping from σ into[0, 1]. 2 Inference and learning algorithms in these languages indeed utilize the relational structure, but in terms of defining the semantics, the assumption simplifies the presentation ithout the need to refer to fuzzy predicate logic. Symbol Name Definition l Lukasieicz t-norm l x,y = maxx+y 1,0 l Lukasieicz t-conorm l x,y = minx + y,1 m Gödel t-norm mx,y = minx,y m Gödel t-conorm mx,y = maxx,y p product t-norm px,y = x y p product t-conorm px,y = x + y x y s standard negator sx = 1 x 1 if x y r R-implicator induced by m rx,y = y otherise s S-implicator induced by m sx,y = max1 x,y l Implicator induced by l l x,y = min1 x + y,1 Figure 1: Some t-norms, t-conorms, negator, and implicators The fuzzy operators are functions mapping one or a pair of truth values into a truth value. Among the operators, denotes a function from [0,1] into [0,1];,, and denote functions from[0,1] [0,1] into[0,1]. The actual mapping performed by each operator can be defined in many different ays, but all of them satisfy the properties that they are generalizations of the corresponding Boolean connectives. Figure 1 lists some examples of fuzzy operators. The truth value of a fuzzy propositional formula F underi, denotedυ I F, is defined recursively as follos: for any atomp σ, υ I p = Ip; for any numeric constantc, υ I c = c; υ I F = υ I F; υ I F G = υ I F,υ I G {,, }. For simplicity, e identify the symbols for the fuzzy operators ith the truth value functions represented by them. Definition 1 We say that a fuzzy interpretation I satisfies a fuzzy formula F if υ I F = 1, and denote it by I = F. We call suchi a fuzzy model off. We say that a fuzzy interpretation I is Boolean if Ip is either 0 or 1 for each fuzzy atom p. Clearly, e may identify a Boolean fuzzy interpretation I ith the classical propositional interpretation by identifying 1 ith TRUE and 0 ith FALSE. Any fuzzy propositional formula hose numeric constants are restricted to0and1can be identified ith a classical propositional formula. For such a formula F, due to the fact that fuzzy operators are generalizations of their Boolean counterparts, it is clear that Boolean fuzzy models of F are precisely the Boolean models of F hen F is vieed as a classical propositional formula. Revie: Probabilistic Soft Logic The folloing is a revie of PSL from Kimmig et al. 2012, but is stated using the terminology from fuzzy logic. A PSL program Π is a set of eighted formulas : R ˆk here is a nonnegative real number, R is a fuzzy propositional formula of the form 3 a l b 1 l... l b n 1 3 We understand G F as an alternative notation for F G.

3 heren 0, each of a,b 1,...,b n is a fuzzy atom possibly preceded by the standard negator, and k {1,2}. 4 For each rule R of the form 1, the distance to satisfaction under interpretation I is defined as d R I = max{0, υ I b 1 l l b n υ I a}. 2 Given an interpretation I of Π, the unnormalized density function over I under Π is defined as ˆf Π I = exp d R I, k :Rˆk Π and the probability density function over I under Π is defined as f Π I = ˆf Π I, Z Π herez Π is the normalization factor ˆf Π I. I The probability density function f Π I is defined similar to the eight W L I in MLN. Different from MLN here the eight of an interpretation comes from the sum over the eights of all formulas that are satisfied thus the penalty is implicit, in PSL, the probability density function of an interpretation is obtained from the sum over the penalty i.e., the eight times the distance to satisfaction from each formula, here the penalty is 0 hen the formula is satisfied, and becomes bigger as the formula gets unsatisfied more i.e., the fuzzy truth value of the body gets bigger than the fuzzy truth value of the head. When the formula is most unsatisfied i.e., the body evaluates to 1 and the head evaluates to0, the penalty is, the maximum. A novel idea here is that each formula contributes to the penalty to a certain graded truth degree including 0. Along ith the restriction imposed on the syntax of fuzzy formulas using the rule form 1, MAP inference in PSL can be reduced to a convex optimization problem in continuous space, thereby enabling efficient computation. Weighted Fuzzy Logic as a Generalization of PSL Weighted Fuzzy Logic Here e define a eighted fuzzy logic as a generalization of PSL. The idea is simple. We take the standard fuzzy logic and extend it by applying the log-linear eight scheme of PSL. A eighted propositional fuzzy logic theory Π is a set of eighted formulas : F ˆk, here is a real number, F is a fuzzy propositional formula, and 4 PSL also allos linear equality and inequality constraints, hich is outside logical theories, and e omit here for simplicity. InterpretationI that violates any of them gets f ΠI = 0. k {1,2}. The unnormalized density function of a fuzzy interpretationi underπis defined as ˆf Π I = exp 1 υ I F, k :F ˆk Π and the probability density function of I under Π is defined as f Π I = ˆf Π I, Z Π herez Π is the normalization factor ˆf Π I. I Notice that 1 υ I F represents the distance to satisfaction in the general case. It is 0 hen I satisfies F, and becomes bigger as υ I F gets farther from 1. This notion of distance to satisfaction for an arbitrary formula is also used in Probabilistic Similarity Logic Bröcheler, Mihalkova, and Getoor 2010, and indeed, the eighted fuzzy logic is very similar to Probabilistic Similarity Logic. Both of them employ arbitrary fuzzy operators, not restricted to the Lukasieicz fuzzy operators. Hoever, the languages are not the same. In Probabilistic Similarity Logic, atomic sentences are of the form called similarity statements,a = s B, heresis some similarity measure, and A, B are entities or sets that can even be represented in an object-oriented syntax. On the other hand, atomic sentences of the eighted logic is a fuzzy atom, same as in PSL. As e sho belo it is easy to vie the eighted fuzzy logic as a generalization of PSL, and it serves as a convenient intermediate language to relate PSL and MLN. 5 Relation to PSL The folloing lemma tells us ho the notions of distance to satisfaction in PSL and in the eighted fuzzy logic are related. Lemma 1 For any rule R of the form 1 and any interpretation I, d R I = 1 υ I R. Proof. 1 υ IR = max{0, 1 υ IR} = max{0, 1 υ Ia l b 1 l... l b n} = max{0, 1 min{1 υ Ib 1 l l b n+υ Ia, 1}} = max{0, υ Ib 1 l l b n υ Ia} = d RI. In Lemma 1, it is essential that rules 1 use Lukasieicz fuzzy operators. The lemma does not hold ith an arbitrary selection of fuzzy operators as the folloing example indicates. 5 Although PSL and Probabilistic Similarity Logic seem to be closely related, the formal relationship beteen them has not been discussed in the literature to the best of our knoledge.

4 Example 1 Consider Gödel t-norm m and its residual implicator r. Let R be q r p and I an interpretation {p,0.6,q,0.4}. d R I is 0.2, hile 1 υ I p r q is1 0.4 = 0.6. It follos from Lemma 1 that PSL can be easily vieed as a special case of the eighted fuzzy logic. Theorem 1 Given any PSL programπand any fuzzy interpretation I, the definition of f Π I hen Π is vieed as the eighted fuzzy logic coincides ith the definition of f Π I henπis vieed as a PSL program. Proof. Immediate from Lemma 1. Due to this theorem, e ill call the eighted fuzzy logic also as generalized PSL GPSL. Vieing PSL as a special case of the eighted fuzzy logic allos us to apply the mathematical results knon from fuzzy logic to the context of PSL. Here is one example, hich tells us that the different versions of PSL defined in Kimmig et al and Bach et al are equivalent despite the different syntax adopted in each of them. To be precise, PSL in Bach et al is defined for clausal form only, such as 3 belo, hile in Kimmig et al it is defined for rule form 1 only. When L is either an atom A or s A, by øl e denote a literal complementary to L, i.e., øl = s A if L is A, and øl = A ifl = s A. The folloing equivalences are knon from fuzzy logic. Lemma 2 For any formulas F and G, and any literals L i 1 i n, a F l G is equivalent to s F l G. b s L 1 l l L n is equivalent toøl 1 l l øl n. The folloing lemma tells us that the clausal form using Lukasieicz t-conorm can be ritten in many different forms. Lemma 3 For any literalsl i 1 i n, is equivalent to L 1 l l L m l L m+1 l l L n 3 øl 1 l l øl m l L m+1 l l L n here n m 0. Proof. By Lemma 2 a, formula 3 is equivalent to s L 1 l l L m l L m+1 l l L n and by Lemma 2 b, the latter is equivalent to øl 1 l l øl m l L m+1 l l L n. It follos from Lemma 1 that the probability density of an interpretation does not change hen the formula is replaced ith another equivalent formula. This tells us that PSL rules of the form 1 can be reritten as any other equivalent formulas. For instance, PSL rule : a l b l c ˆ1 4 can be equivalently reritten as any of the folloing ones. : s b l s a l c ˆ1, : s c l s a l b ˆ1, : a l s b l c ˆ1, : a l s c l b ˆ1, : s b l s c l s a ˆ1, : a l s b l s c ˆ1, : 0 l s a l b l c ˆ1. 5 As noted above, the syntax of PSL in Bach et al is clausal form only, such as the second to the last formula in 5, hile the syntax of PSL in Kimmig et al is rule form such as 4. The result above tells us that the definitions of PSL defined in Kimmig et al and Bach et al are equivalent despite the different syntax adopted there. On the other hand, similar reriting using other t-norms and their derived operators may not necessarily yield an equivalent formula because not every selection of fuzzy operators satisfy Lemma 2 even if they are generalizations of the corresponding Boolean connectives. Example 2 Consider again Gödel t-norm m and its residual implicator r. The negation m induced from m x = x r 0 is { 1 if x = 0 m x = 0 if x > 0. For the interpretation I = {p, 0.4,q, 0.5}, e have υ I m p m q = 0 m 0.5 = 0.5, butυ I p r q = 1. In other ords, m p m q is not equivalent top r q. In the literature on PSL Kimmig et al. 2012; Bach et al. 2015, the selection of Lukasieicz t-norm is motivated by the computational efficiency gained by reducing MAP inferences to convex optimization problems. This section presents yet another justification of Lukasieicz t-norm in PSL from the logical perspective. GPSL : MLN = Fuzzy Logic : Boolean Logic Like fuzzy logic is a many-valued extension of Boolean logic, e may vie GPSL as a many-valued extension of MLN. For any classical propositional formula F, let F fuzzy be the fuzzy formula obtained fromf by replacing ith0, ith1, ith any fuzzy negation symbol, ith any fuzzy conjunction symbol, ith any fuzzy disjunction symbol, and ith any fuzzy implication symbol. For any GPSL program Π, by TW Π total eight e denote exp. :F ˆk Π For any MLN L, let Π L be the GPSL program obtained from L by replacing each eighted formula : F in L ith : F fuzzyˆk, here k is either 1 or 2. The folloing theorem tells us that, for any Boolean interpretation I, its eight under MLN L is proportional to the unnormalized probability density under the GPSL programπ L.

5 Theorem 2 For any MLN L and any Boolean interpretationi, W L I = TW ΠL ˆf ΠL I. Proof. W L I = exp = exp = exp,f L I,F L,F L = exp exp,f L,F L\L I,F L\L I exp 1+ 0.,F L\L I,F L I Note that hen I is Boolean, 1 υ I F = 1 if I = F, and1 υ I F = 0 if I = F. So 6 is equal to TW ΠL exp = TW ΠL ˆf ΠL I. :F fuzzyˆk Π L 1 υ I F fuzzy k This theorem tells us that the problem of computing the eight of an interpretation in MLN can be reduced to computing the probability density of an interpretation in GPSL. By Theorem 1, since PSL is a special case of GPSL, the folloing relation beteen PSL and MLN follos easily. Corollary 1 For any PSL programπand any fuzzy Boolean interpretation I, let L be the MLN obtained from Π by replacing each fuzzy operator ith its Boolean counterpart. We have ˆf Π I = W LI TW Π. Example 3 Let Π be the folloing PSL program 1 : p l q ˆ1 2 : q l p ˆ1 and let L be the corresponding MLN as described in Corollary 1. The folloing table shos, for each Boolean interpretation I, its eight according to the MLN semantics W L I is TW Π, hich is e 3, multiplied by its unnormalized probability density function ˆf Π I according to the PSL semantics. We identify a Boolean interpretation ith the set of atoms that are true in it. Interpretation I W L I ˆfΠ I e 3 e 0 {p} e 1 e 2 {q} e 2 e 1 {p,q} e 3 e 0 6 Hoever, MAP states in MLN and PSL can be different because most probable interpretations in PSL may be non- Boolean. Example 4 Consider the PSL program: 1 : p l s p ˆ1 1 : s p ˆ1 and the corresponding MLN: 1 : p s p 1 : s p The most probable Boolean interpretations for MLN 8 are I 1 = and I 2 = {p}, each ith eight e 1. Their unnormalized probability density for PSL program 7 is e 1. Hoever, they are not the most probable interpretations according to the PSL semantics: I 3 = {p,0.5} has the largest unnormalized probability densitye 0.5. The difference can be closed by adding to the eighted propositional fuzzy logic theory a crispifying rule for each atom. For any atom p σ, let CRSPp be the formula defined as CRSPp = p l p l p. It is easy to check thatυ I p l p l p = 1 iffυ I p is either 0 or 1. Note that although this formula uses Lukasieicz operators, it is not expressible in PSL because l occurs in the body of the rule. For any MLN L of signature σ, let Π L be the GPSL program obtained from L by replacing each eighted formula : F in L ith : F fuzzyˆk here k could be either 1 or 2. LetCR be the GPSL program { α : CRSPpˆ1 p σ }. The folloing theorem tells us that the most probable interpretations of MLN L coincides ith the most probable interpretations of GPSL programπ L CR. Theorem 3 For any MLNL, henα, argmax I W L I = argmax J ˆf ΠL CRJ here I ranges over all Boolean interpretations and J ranges over all fuzzy interpretations. Proof. We first sho that, hen α, for any Boolean interpretation I and any non-boolean interpretation J, e have ˆf ΠL CRJ < ˆf ΠL CRI, hich implies that no non- Boolean interpretation can be the most probable interpretations. First, for any non-boolean interpretationj, let W ΠL J = 1 υ J F k and Then W CR J = :Fˆk Π L :F ˆ1 CR α 1 υ J F. ˆf ΠL CRJ = expw ΠL J+W CR J expw CR J. 7 8

6 Since J is not Boolean, there is at least one eighted formulaα : p l p l p ˆ1 CR that is not satisfied by J, so that ˆf ΠL CRJ exp α 1 υ J p l p l p here1 υ J p l p l p > 0. Notice that lim ˆf ΠL CRJ α lim exp α 1 υ J p l p l p = 0. α On the other hand, for any Boolean interpretationi, ˆf ΠL CRI = expw ΠL I+W CR I = expw ΠL I. SinceexpW ΠL I does not containα, e have lim ˆf ΠL CRI > 0. α Thus e have ˆf ΠL CRJ < ˆf ΠL CRI henα. It follos that any fuzzy interpretation K that satisfies argmax J P ΠL CRJ = K must be Boolean. By Theorem 2, for any Boolean interpretationi, e have Since ˆf ΠL CRI = exp σ α TW ΠL CR exp σ α TW ΠL CR W L I. is constant for all interpretations, ˆf ΠL CRI W L I. It follos that argmax I W L I = argmax J ˆf ΠL CRJ. Example 5 Consider the GPSL program: 1 : p l s p ˆ1 1 : s p ˆ1 α : p l p l p ˆ1. When α the most probable fuzzy interpretations are Boolean, and they are the same as the most probable interpretations for the MLN 8. It is knon that the MAP problem in PSL can be solved in polynomial time Bröcheler, Mihalkova, and Getoor 2010, hile the same problem in MLN is #P-hard. The reduction from MLN to GPSL in Theorem 3 tells us that the MAP problem in GPSL is #P-hard as ell. This implies that GPSL is strictly more expressive than PSL even hen e restrict attention to Lukasieicz operators. Related to Theorem 3, relation beteen discrete and soft MAP states as also studied in Bach, Huang, and Getoor 2015; Bach et al. 2015, but from a different, computational perspective. There, inference on discrete MAP states is vieed as an instance of MAX SAT problems, and then approximated by relaxation to linear programming ith rounding guarantee of solutions. The result indirectly tells us ho MAP states in PSL are related to MAP states in MLN, but this is different from Theorem 3, hich completely closes the semantic gap beteen them via crispifying rules. Conclusion In this note, e studied the to ell-knon formalisms in statistical relational learning from a logical perspective. Vieing PSL in terms of the eighted fuzzy logic gives us some useful insights knon from fuzzy logic. Besides the reducibility to convex optimization problems, the restriction to the Lukasieicz fuzzy operators in clausal form allos intuitive equivalent transformations resembling those from Boolean logic. On the other hand, it prohibits us from using some other intuitive fuzzy operators. In our previous ork Lee and Wang 2014; Lee and Wang 2016a e used fuzzy anser set programs to describe temporal projection in dynamic domains, here e had to use Gödel t-norm as ell as Lukasieicz t-norm. 6 There, Gödel t-norm is necessary in expressing the commonsense la of inertia. For example, Trusta,b,t m s s Trusta,b,t+1 r Trusta,b,t+1, 9 expresses that the degree that a trustsbat time t+1 is equal to the degree at time t if it can be assumed ithout contradicting any of the facts that can be derived. 7 The fuzzy conjunction used here needs to satisfy that x,y is equal to eitherxory otherise the trust degree at next time step ould change for no reason. Obviously Lukasieicz t-norm does not satisfy the requirement: l x,y < x hen y < 1. In other ords, if e replace m ith l, the trust degree at next time drops for no reason, hich is unintuitive. The restriction to Lukasieicz t-norm in PSL accounts for the difficulty in directly applying PSL to temporal reasoning problems like the above example. Indeed, most ork on PSL has been limited to static domains. Since computing marginal probabilities in MLN can be reduced to computing marginal probabilities in GPSL as indicated by Theorem 2, computing marginal probabilities in GPSL is at least #P-hard. Hoever, a sampling method could be used for such an inference. A naive sampling method is outlined belo: suppose e are approximating the probability that the truth value of formula F falls into l,u for some0 l u 1 denoted aspl F u. 1. Generate N interpretations at random; 2. For each of the N interpretations, compute its probability density; 3. ApproximatePl F u by X N, here X is the number of interpretations I that satisfies l υ I F u among the N interpretations. It can be shon that X N is the estimation of Pl F u that maximizes the likelihood of then samples. The ay that MLN extends propositional logic is similar to the ay that PSL extends a restricted version of fuzzy propositional logic. GPSL is simply taking the fuzzy propositional logic in full generality and applying the log-linear eight scheme. In our recent ork Lee and Wang 2016b, 6 The main example as ho the trust degree beteen people changes over time. 7 We refer the reader to Lee and Wang 2016a for the precise semantics of this language.

7 e adopted the similar eight scheme to anser set programs in order to overcome the deterministic nature of the stable model semantics providing ays to resolve inconsistencies in anser set programs, to rank stable models, to associate probability to stable models, and to apply statistical inference to computing eighted stable models. Perhaps this indicates the universality of the log-linear eight scheme first adopted in MLN, hich provides a uniform method to turn the crisp logic be it fuzzy logic, propositional logic, or anser set programs soft. Acknoledgements We are grateful to Michael Bartholome and the anonymous referees for their useful comments. This ork as partially supported by the National Science Foundation under Grants IIS , IIS , and a gift funding from Robert Bosch LLC. References [Bach et al. 2015] Bach, S. H.; Broecheler, M.; Huang, B.; and Getoor, L Hinge-loss markov random fields and probabilistic soft logic. arxiv: [cs.lg]. [Bach, Huang, and Getoor 2015] Bach, S. H.; Huang, B.; and Getoor, L Unifying local consistency and MAX SAT relaxations for scalable inference ith rounding guarantees. In Artificial Intelligence and Statistics AISTATS. [Beltagy, Erk, and Mooney 2014] Beltagy, I.; Erk, K.; and Mooney, R Probabilistic soft logic for semantic textual similarity. Proceedings of Association for Computational Linguistics ACL-14. [Bröcheler, Mihalkova, and Getoor 2010] Bröcheler, M.; Mihalkova, L.; and Getoor, L Probabilistic similarity logic. In UAI 2010, Proceedings of the Tenty-Sixth Conference on Uncertainty in Artificial Intelligence, [Hajek 1998] Hajek, P Mathematics of Fuzzy Logic. Kluer. [Kimmig et al. 2012] Kimmig, A.; Bach, S.; Broecheler, M.; Huang, B.; and Getoor, L A short introduction to probabilistic soft logic. In Proceedings of the NIPS Workshop on Probabilistic Programming: Foundations and Applications, 1 4. [Lee and Wang 2014] Lee, J., and Wang, Y Stable models of fuzzy propositional formulas. In Proceedings of European Conference on Logics in Artificial Intelligence JELIA, [Lee and Wang 2016a] Lee, J., and Wang, Y. 2016a. Fuzzy propositional formulas under the stable model semantics. the Special Issue on Logics for Reasoning about Preferences, Uncertainty and Vagueness of the IfCoLog Journal of Logics and their Applications. To appear. [Lee and Wang 2016b] Lee, J., and Wang, Y. 2016b. Weighted rules under the stable model semantics. In Proceedings of International Conference on Principles of Knoledge Representation and Reasoning KR. [Richardson and Domingos 2006] Richardson, M., and Domingos, P Markov logic netorks. Machine Learning 621-2: