Counterexamples in Probabilistic LTL Model Checking for Markov Chains Matthias Schmalz 1 Daniele Varacca 2 Hagen Völzer 3 1 ETH Zurich, Switzerland 2 PPS - CNRS & Univ. Paris 7, France 3 IBM Research Zurich, Switzerland September 1st, 2009 Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 1 / 24
Probabilistic Model Checking Σ Φ P [Φ] > t? t Σ: discrete-time finite-state Markov chain Φ: linear-time temporal logic (LTL) formula Yes No One of the most important advantages of model checking... is its counterexample facility. (Clarke et al.) Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 2 / 24
Contributions a way of representing counterexamples in probabilistic LTL model checking a method supporting the user in finding the error algorithms for computing our counterexample representations Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 3 / 24
Terminology System (Markov chain) Σ: 0.7 0.4 s q p r 0.3 1 1 0.6 Notion: Example: Path x Property Y s q p r p r... spr (set of paths with prefix spr) Sat( r) (set of paths infinitely often visiting r) Transition probabilities are positive. Paths are infinite. Properties are sets of paths. Probability of a property: P [spr ] = 0.7 0.4. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 4 / 24
Quantitative and Qualitative Quantitative Probabilistic Model Checking: Σ Yes Φ P [Φ] > t? t No Qualitative Probabilistic Model Checking: Σ Yes P [Φ] = 1? Φ No Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 5 / 24
Outline Qualitative Counterexamples Other Results Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 6 / 24
Validity: Counterexample Specification: The model checker claims: Counterexample: AΦ Σ AΦ a path violating Φ The user finds the bug by inspecting the counterexample. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 7 / 24
Satisfiability: Simulation Specification: The model checker claims: Counterexample: E jackpot Σ E jackpot set of all paths of Σ (useless) How to find the bug? The user defined Σ and Φ. He has an idea how to reach the jackpot. The user tries to reach the jackpot. The user finds the bug by simulating the system. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 8 / 24
Probabilistic Correctness: Interaction Validity Probabilistic Satisfiability Correctness Σ AΦ P [Φ] = 1 Σ EΦ Counterexample: Interaction: Simulation: mc creates both create user creates a path. a path. a path. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 9 / 24
Our Approach Question: Why is P [Φ] < 1? Counterexample: a property Y with 1. Y Sat(Φ) =, 2. P [Y ] > 0. all paths Y Sat(Φ) Interaction: The user learns why 1 and 2 hold. Helps the user find a bug. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 10 / 24
An Example System Σ: 0.5 0.5 1 0.3 1 t... 0.4 0.3 s q p r 0.5 0.5 P [Φ] = 1 is independent of precise transition probabilities! only depends on which states are connected by a transition. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 11 / 24
An Example System Σ: t... s q p r P [Φ] = 1 is independent of precise transition probabilities! only depends on which states are connected by a transition. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 11 / 24
An Example System Σ: t... s q p r Bug: transition t q is missing Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 11 / 24
An Example System Σ: t... s q p r I will... give a specification Φ, give a counterexample Y in our representation, explain the interaction helping the user find the bug. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 11 / 24
Finitary Counterexamples Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r Try a finitary counterexample, e.g., Y := sp. Y Sat(Φ), as spp ω Y Sat(Φ). Y is no counterexample. Moreover: there is no finitary counterexample! Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 12 / 24
Beyond Finitary Counterexamples Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r Counterexample: Y := sp Sat( rr) Y Sat(Φ) sp Sat( q) =. rr belongs to a bscc reachable after sp. Hence, P [Y ] = P [sp ] > 0. Y is a counterexample. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 13 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r The model checker outputs Y := sp Sat( rr) and explains: 1. rr is in a bscc reachable after sp. 2. Y Sat(Φ) =. P [Φ] < 1. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r The model checker outputs Y := sp Sat( rr) and explains: 1. rr is in a bscc reachable after sp. 2. Y Sat(Φ) =. P [Φ] < 1. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r The model checker outputs Y := sp Sat( rr) and explains: 1. rr is in a bscc reachable after sp. 2. Y Sat(Φ) =. P [Φ] < 1. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r The model checker outputs Y := sp Sat( rr) and explains: 1. rr is in a bscc reachable after sp. 2. Y Sat(Φ) =.??? P [Φ] < 1. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r Y := sp Sat( rr) Why is Y Sat(Φ) =? User and MC create a path x. MC ensures x Y. User aims for x Φ. By failing the user finds the bug! Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r Y := sp Sat( rr) Why is Y Sat(Φ) =? User and MC create a path x. MC ensures x Y. User aims for x Φ. By failing the user finds the bug! s p Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r Y := sp Sat( rr) Why is Y Sat(Φ) =? User and MC create a path x. MC ensures x Y. User aims for x Φ. By failing the user finds the bug! s p t q Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finding the Bug Specification: Φ := rr q Question: Why is P [Φ] < 1? Σ: t... s q p r Y := sp Sat( rr) Why is Y Sat(Φ) =? User and MC create a path x. MC ensures x Y. User aims for x Φ. By failing the user finds the bug! s p t q Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 14 / 24
Finite Path Leading to a Recurrent Word Definition Recurrent word := finite path fragment belonging to a bscc A finite path α The bscc of γ is (almost surely) leads to the only bscc a recurrent word γ λ. reachable after α.... γ α... Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 15 / 24
Qualitative Counterexamples Question: Why is P [Φ] < 1? Counterexample: Y := α Sat( γ), where 1. γ recurrent 2. α (almost surely) leads to γ 3. Y Sat(Φ) = Theorem (Soundness) (a) 1, 2 = P [ γ α ] = 1 and hence P [Y ] > 0 (b) 1, 2, 3 = P [Φ α ] = 0 and hence P [Φ] 1 P [α ] < 1 α explains how much probability is lost. α explains where the probability is lost. γ explains why the probability is lost. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 16 / 24
Qualitative Counterexamples Question: Why is P [Φ] < 1? Counterexample: Y := α Sat( γ), where 1. γ recurrent 2. α (almost surely) leads to γ 3. Y Sat(Φ) = Theorem (Soundness) (a) 1, 2 = P [ γ α ] = 1 and hence P [Y ] > 0 (b) 1, 2, 3 = P [Φ α ] = 0 and hence P [Φ] 1 P [α ] < 1 α explains how much probability is lost. α explains where the probability is lost. γ explains why the probability is lost. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 16 / 24
Qualitative Counterexamples Question: Why is P [Φ] < 1? Counterexample: Y := α Sat( γ), where 1. γ recurrent 2. α (almost surely) leads to γ 3. Y Sat(Φ) = Theorem (Soundness) (a) 1, 2 = P [ γ α ] = 1 and hence P [Y ] > 0 (b) 1, 2, 3 = P [Φ α ] = 0 and hence P [Φ] 1 P [α ] < 1 α explains how much probability is lost. α explains where the probability is lost. γ explains why the probability is lost. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 16 / 24
Qualitative Counterexamples Question: Why is P [Φ] < 1? Counterexample: Y := α Sat( γ), where 1. γ recurrent 2. α (almost surely) leads to γ 3. Y Sat(Φ) = Theorem (Soundness) (a) 1, 2 = P [ γ α ] = 1 and hence P [Y ] > 0 (b) 1, 2, 3 = P [Φ α ] = 0 and hence P [Φ] 1 P [α ] < 1 α explains how much probability is lost. α explains where the probability is lost. γ explains why the probability is lost. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 16 / 24
Qualitative Counterexamples Question: Why is P [Φ] < 1? Counterexample: Y := α Sat( γ), where 1. γ recurrent 2. α (almost surely) leads to γ 3. Y Sat(Φ) = Theorem (Completeness) P [Φ] < 1 = there are α, γ such that 1, 2, 3 hold. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 16 / 24
Interaction Conditions 1, 2, 3 can be expressed in terms of path games between the user and the model checker. Condition i holds the model checker has a winning strategy in the respective path game. To understand why a condition holds, the user plays the respective path game against the model checker. By losing the user finds the error in the system. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 17 / 24
Interaction Disjointness Y Sat(Φ) = The path game: The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Interaction Disjointness Y Sat(Φ) = The path game: x = α The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Interaction Disjointness Y Sat(Φ) = The path game: x = α The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Interaction Disjointness Y Sat(Φ) = The path game: x = α γ The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Interaction Disjointness Y Sat(Φ) = The path game: x = α γ The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Interaction Disjointness Y Sat(Φ) = The path game: x = α γ γ γ The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Interaction Disjointness Y Sat(Φ) = The path game: x = α γ γ γ Φ The model checker ensures x Y. The user wins iff x Φ. The model checker has a winning strategy The user is unable to establish x Φ Y Sat(Φ) = The game corresponds to the Banach-Mazur game. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 18 / 24
Outline Qualitative Counterexamples Other Results Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 19 / 24
Quantitative Counterexamples Quantitative Counterexample: Y := W Fair Σ (R) W : set of finite paths R: set of recurrent words Y Sat(Φ) =, P [Y ] sufficiently large Theorem (Soundness) P [Φ] 1 P [W ] P [Φ W ] = 0 Theorem (Completeness) P [Φ] 1 t = There is a counterexample W Fair Σ (R), where R contains one rec. word per bscc, and W is regular. Interaction: as W is regular, various techniques from the literature can be applied for presenting W to the user. Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 20 / 24
Computing Counterexamples We have developed non-trivial extensions of an algorithm of Courcoubetis and Yannakakis (1995). Complexity in Σ Complexity in Φ α, γ Σ exonential α of max. probability Σ log Σ doubly exp. W Σ doubly exp. R Σ #bsccs exponential Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 21 / 24
Summary A qualitative counterexample can be represented as α Sat( γ). A quantitative counterexample can be represented as W Fair Σ (R), where W is regular. We describe an interactive game that supports the user in finding the error. We have developed algorithms computing our counterexample representations. Future directions: Generalize results for Markov Decision Processes. Case studies Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 22 / 24
Appendix Periodic Counterexamples 0.7 0.3 1 0.4 s q p r 1 0.6 Each periodic path has probability zero, e.g., P [{s(pr) ω }] = 0. The set of all periodic paths is countable. The set of all periodic paths has probability zero. Sets of periodic paths can in general not be used as counterexamples! Counterexamples in Probabilistic LTL Model Checking for Markov Chains Schmalz, Varacca, Völzer 24 / 24