On the Total Variation Distance of Labelled Markov Chains

Size: px

Start display at page:

Download "On the Total Variation Distance of Labelled Markov Chains"

Avis Wilkinson
5 years ago
Views:

1 On the Total Variation Distance of Labelled Markov Chains Taolue Chen Stefan Kiefer Middlesex University London, UK University of Oxford, UK CSL-LICS, Vienna 4 July 04

2 Labelled Markov Chains (LMCs) a c 4 c An LMC generates infinite words randomly. 4 b Pr({abacccc...}) = 4 4 = 64 Pr({a}Σ ω ) = Pr({b}Σ ω ) = 4 Pr( eventually only c ) = Pr assigns a measurable event E Σ ω a probability [0, ]. E Σ ω could be defined by an LTL formula.

3 Labelled Markov Chains (LMCs) a c 4 a c 4 c 4 c 4 b b Pr ({accc...}) = 4 = 8 Pr ({accc...}) = 4 4 = 6 The two LMCs are not equivalent and have positive distance. TV-distance d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) How large can Pr (E) Pr (E) get?

4 Labelled Markov Chains (LMCs) a c 4 a c 4 c 4 c 4 b b Pr ({accc...}) = 4 = 8 Pr ({accc...}) = 4 4 = 6 The two LMCs are not equivalent and have positive distance. TV-distance d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) How large can Pr (E) Pr (E) get? No larger than 5 6 <.

5 Digression: Total Variation Distance Pr Taolue Pr Stefan ({ }) ({ }) Pr Taolue Pr Stefan = 0. Pr Taolue Pr Stefan = 0. Pr Taolue, Pr Stefan, ({ }) ({ }) Pr Taolue Pr Stefan = 0. = 0.

6 What is the Maximising Event? a c 4 a c 4 c 4 c 4 b b d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) d(pr, Pr ) = Pr (E) Pr (E) holds for E = {wccc... w {a, b}, # a (w) # b (w)} E is a maximising event It s not clear that there is always a maximising event.

7 More Careful Definition d(pr, Pr ) := max E Σ ω Pr (E) Pr (E)

8 More Careful Definition d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) d(pr, Pr ) := sup E Σ ω Pr (E) Pr (E)

9 More Careful Definition d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) d(pr, Pr ) := sup E Σ ω Pr (E) Pr (E) Technically, E ranges only over the measurable subsets of Σ ω (still uncountably many such events E).

10 More Careful Definition d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) d(pr, Pr ) := sup E Σ ω Pr (E) Pr (E) Technically, E ranges only over the measurable subsets of Σ ω (still uncountably many such events E). Proposition (Existence of a Maximising Event) There is an event E Σ ω with d(pr, Pr ) = Pr (E) Pr (E). d(pr, Pr ) := max E Σ ω Pr (E) Pr (E)

11 The Distance Can Be Irrational. a c 4 a c 4 c 4 c 4 b b d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) d(pr, Pr ) = Pr (E) Pr (E) = /4 holds for E = {wccc... w {a, b}, # a (w) # b (w)}

12 Example for Distance 3 b 3 a 3 a 3 b The LMC is very symmetric. Both states enable all runs. But d(pr, Pr ) =. What is the maximising event? b a a b b a a b b a b a b a b b

13 Example for Distance 3 b 3 a 3 a 3 b The LMC is very symmetric. Both states enable all runs. But d(pr, Pr ) =. What is the maximising event? b a a b b a a b b a b a b a b b

14 Example for Distance 3 b 3 a 3 a 3 b The LMC is very symmetric. Both states enable all runs. But d(pr, Pr ) =. What is the maximising event? b a a b b a a b b a b a b a b b b b a b 0 Let E = this sequence converges to 3. Then: Pr (E) = and Pr (E) =

15 Example for Distance 3 b 3 a 3 a 3 b The LMC is very symmetric. Both states enable all runs. But d(pr, Pr ) =. What is the maximising event? b a a b b a a b b a b a b a b b b b a b 0 Let E = this sequence converges to 3. Then: Pr (E) = and Pr (E) = 0 There is no ω-regular maximising event

16 A Maximising Event: Intuition Pr Taolue ({ Pr Taolue Pr Stefan , }) Pr Stefan ({, }) = 0. For LMCs, define L(w) := Pr (w) Pr (w) Maybe E = {w Σ ω L(w) } is a maximising event?

A Maximising Event: Intuition Pr Taolue ({ Pr Taolue 0.3 0.6 0. Pr Stefan 0. 0.5 0.

17 A Maximising Event: Intuition Pr Taolue ({ Pr Taolue Pr Stefan , }) Pr Stefan ({, }) = 0. For LMCs, define L(w) := Pr (w) Pr (w) Maybe E = {w Σ ω L(w) } is a maximising event? Redefine L(w)...

18 A Maximising Event 3 b 3 a 3 a 3 b Fix a run w = a a a 3 Σ ω. For every k N define a nonnegative value: L k (w) := Pr (a a k Σ ω ) Pr (a a k Σ ω ) If the run w is produced randomly (by the first LMC), L, L,... is a sequence of random variables.

19 A Maximising Event The sequence L, L,... is a nonnegative martingale. = Martingale Convergence Theorem applies. = L := lim k L k exists almost surely. (L is a random variable.) Theorem (A Generic Maximising Event) Define E := {w Σ ω L(w) }. Then d(pr, Pr ) = Pr (E) Pr (E).

20 Approximation: Lower Bound a 4 a b b 3 4 b d(pr, Pr ) := max E Σ ω Pr (E) Pr (E) Fix k N. Idea: consider only events definable by the length-k prefix. I.e., define d k (Pr, Pr ) := max W Σ k Pr (W Σ ω ) Pr (W Σ ω ). Proposition For all k N: d k (Pr, Pr ) d k+ (Pr, Pr ) d (Pr, Pr ) = d(pr, Pr )

21 Approximation: Upper Bound a 4 a b b 3 4 b (, 0, 0) (0, 0, ) 0 =: con(0) a b (, 0, 0) (0, 0, 4 ) (0,, 0) (0, 3 4, 0) =: con() a b a b ( 4, 0, 0) (0, 0, 6 ) (0, 4, 0) (0, 3 6, 0) (0, 0, 0) (0, 0, 0) (0,, 0) (0, 3 4, 0) =: con()

22 Approximation: Upper Bound This defines an increasing sequence: 0 con(0) con()... con( ) = d(pr, Pr ) In general, define con(k) using equivalent distributions rather than equal states.

23 Approximation: Upper Bound This defines an increasing sequence: 0 con(0) con()... con( ) = d(pr, Pr ) In general, define con(k) using equivalent distributions rather than equal states. Theorem Given ε > 0, one can compute x Q with d(pr, Pr ) [x, x + ε]. Open: convergence speed

24 The Distance- Problem: Deniability Taolue: Stefan: Task for NSA: { distinguish those} guys! ω Is there E,, with Pr Taolue (E) = and Pr Stefan (E) = 0?

25 The Distance- Problem: Deniability Taolue: Stefan: Task for NSA: { distinguish those} guys! ω Is there E,, with Pr Taolue (E) = and Pr Stefan (E) = 0? If not, Taolue can plausibly deny that he is Taolue.

26 The Distance- Problem a 4 a b b 3 4 b (, 0, 0) (0, 0, ) 0 =: con(0) a b (, 0, 0) (0, 0, 4 ) (0,, 0) (0, 3 4, 0) =: con() a b a b ( 4, 0, 0) (0, 0, 6 ) (0, 4, 0) (0, 3 6, 0) (0, 0, 0) (0, 0, 0) (0,, 0) (0, 3 4, 0) =: con() The distance is < k N : con(k) > 0

27 The Distance- Problem: in PSPACE a (, 0, 0) (0, 0, ) b a ( 4, 0, 0) (0, 0, 6 ) (, 0, 0) (0, 0, 4 ) b (0, 4, 0) (0, 3 6, 0) a (0, 0, 0) (0, 0, 0) (0,, 0) (0, 3 4, 0) b (0,, 0) (0, 3 4, 0) = µ bb = µ bb The distance is < u Σ : µ u and µu overlap. It suffices to check words u of length at most Q. PSPACE algorithm: guess a word u Σ Q and check if µ u, µu overlap.

28 The Distance- Problem: in P To get a polynomial-time algorithm: Generalise distance between states to distance between state distributions. Exploit structural properties of the generalised notion: Lemma d(π, π ) = 0 = q supp(π ) : d(q, π ) < d(π, π ) < = q supp(π ) : d(q, π ) <

29 The Distance- Problem: in P To get a polynomial-time algorithm: Generalise distance between states to distance between state distributions. Exploit structural properties of the generalised notion: Lemma d(π, π ) = 0 = q supp(π ) : d(q, π ) < d(π, π ) < = q supp(π ) : d(q, π ) < 3 Exploit previous work on LMC equivalence and use linear programming. Theorem (Distance- Problem) One can decide in polynomial time whether the distance between two LMCs is.

30 Threshold Problem Threshold Problem Input: LMCs and threshold τ [0, ] Output: Is d(pr, Pr ) τ? Square-Root-Sum Problem Input: s,..., s n N and t N Output: Is n i= si t? The Square-Root-Sum problem is not known to be in NP.

31 Threshold Problem Threshold Problem Input: LMCs and threshold τ [0, ] Output: Is d(pr, Pr ) τ? Square-Root-Sum Problem Input: s,..., s n N and t N Output: Is n i= si t? The Square-Root-Sum problem is not known to be in NP. Theorem The Threshold Problem is NP-hard. The Threshold Problem is hard for the Square-Root-Sum Problem.

32 LMC with Parameters a b ( + x)b ( x)b a ( x)a ( + x)a ( θ )a θ a θ b ( θ )b b

33 Distance as Function in x θ = 3 d θ (x) x d θ (x)

34 Distance as Function in x θ = 3 d θ (x) x d θ (x)

35 Bernoulli-Convolutions d θ (x) (rescaled) is the cumulative distribution function of i=0 X i θ i with Pr(X i = ) = Pr(X i = +) = Bernoulli convolutions : studied since the 930s θ > : d θ is either absolutely continuous or singular. is the (ternary) Cantor function. d 3 For almost all θ (, ]: d θ is absolutely continuous. If θ is a Pisot number, then d θ is singular. [Erdős, 939] It is open, e.g., whether d 3/ is absolutely continuous.

36 Related Work: Bisimilarity Pseudometric b a b a c c a LMCs are (trace) equivalent, but not bisimilar. More precisely: TV-distance is 0, but bisimilarity distance is. [D. Chen, F. van Breugel, J. Worrell, FoSSaCS ]: TV-distance bisimilarity distance

37 Results and Open Problems Positive Results: There is a maximising event. The distance can be approximated within arbitrary precision. The distance- problem is in P. Negative Results: The maximising event may not be ω-regular. The threshold problem is NP-hard and hard for square-root-sum. The distance is related to Bernoulli convolutions. Open Questions: Efficient approximation? Is the threshold problem decidable?

38 Pisot Number: Definition A Pisot number is a real algebraic integer greater than such that all its Galois conjugates are less than in absolute value. Smallest Pisot number (.347): the real root of x 3 x Another one is the golden ratio

39 Motivation: Efficient Model Checking similar LMCs M,..., M n (ω-regular) events E,..., E m want: bounds on Pr i (E j ) for all i, j Assume d(pr, Pr i ) ε for all i n. Then by definition i j : Pr i (E j ) [Pr (E j ) ε, Pr (E j ) + ε]

Converging from Branching to Linear Metrics for Weighted Transition Systems

Converging from Branching to Linear Metrics for Weighted Transition Systems Giorgio Bacci, Giovanni Bacci, Kim G. Larsen, and Radu Mardare Dept. Computer Science, Aalborg University WATA 2016 1/24 Motivations