Automata-Based String Analysis

Transcription

1 1 / 46 Automata-Based String Analysis Model Counting Tevfik Bultan, Abdulbaki Aydin, Lucas Bang Verification Laboratory Department of Computer Science

2 Overview

3 Overview String Constraints

4 Overview String Constraints Model Counter

5 2 / 46 Overview String Constraints Model Counter Number of Solutions

6 Can you solve it, Will Hunting? 3 / 46

7 Can you solve it, Will Hunting? 4 / 46

8 5 / 46 Outline Motivation and Background Model Counting Boolean Formulas String Model Counting Automata-Based Methods Non-Automata-Based Method String Model Counting Benchmarks

9 6 / 46 A Motivating Example An adversary learns a password. User must select a new password.

10 6 / 46 A Motivating Example An adversary learns a password. User must select a new password. Policy for selecting a new password.

11 6 / 46 A Motivating Example An adversary learns a password. User must select a new password. Policy for selecting a new password. 1 public Boolean NewPWCheck(String new_p, old_p){ 2 if( old_p.contains(new_p)... 3 new_p.contains(old_p)... 4 old_p.reverse().contains(new_p))... 5 new_p.contains(old_p.reverse()) ){ 6 System.out.println("Too similar."); 7 return false; 8 } else 9 return true; 10 }

12 7 / 46 A Motivating Example Suppose an adversary knows old_p = "abc-16"

13 7 / 46 A Motivating Example Suppose an adversary knows old_p = "abc-16" and knows the policy.

14 7 / 46 A Motivating Example Suppose an adversary knows old_p = "abc-16" and knows the policy. Constraints on possible values of NEW_P (not (contains (tolower NEW_P) "abc-16")) (not (contains (tolower NEW_P) "61-cba")) (not (contains "abc-16" (tolower NEW_P))) (not (contains "61-cba" (tolower NEW_P)))

15 7 / 46 A Motivating Example Suppose an adversary knows old_p = "abc-16" and knows the policy. Constraints on possible values of NEW_P (not (contains (tolower NEW_P) "abc-16")) (not (contains (tolower NEW_P) "61-cba")) (not (contains "abc-16" (tolower NEW_P))) (not (contains "61-cba" (tolower NEW_P))) If password length = n, then there are Σ n possible passwords.

16 7 / 46 A Motivating Example Suppose an adversary knows old_p = "abc-16" and knows the policy. Constraints on possible values of NEW_P (not (contains (tolower NEW_P) "abc-16")) (not (contains (tolower NEW_P) "61-cba")) (not (contains "abc-16" (tolower NEW_P))) (not (contains "61-cba" (tolower NEW_P))) If password length = n, then there are Σ n possible passwords. If adversary knows old_p and the policy... how much is the reduction in search space? what is the probability of guessing the new password?

17 8 / 46 Motivation In general, we want to answer questions regarding probability of program behaviors, number of inputs that cause an error, amount of information flow, information leakage, other, as yet unforeseen, applications...

18 8 / 46 Motivation In general, we want to answer questions regarding probability of program behaviors, number of inputs that cause an error, amount of information flow, information leakage, other, as yet unforeseen, applications... These are quantitative questions which require model counting.

19 9 / 46 Motivation Techniques for model counting for other theories Boolean Logic Formulas DPLL Random sampling based Approximations

20 9 / 46 Motivation Techniques for model counting for other theories Boolean Logic Formulas DPLL Random sampling based Approximations Linear Integer Arithmetic: LattE Barvinok

21 10 / 46 Motivation String manipulating programs are pervasive security critical functions, server side sanitization functions, databases, dynamic code generation.

22 10 / 46 Motivation String manipulating programs are pervasive security critical functions, server side sanitization functions, databases, dynamic code generation. We need model counting for strings in order to make quantitative guarantees about these types of programs.

23 10 / 46 Motivation String manipulating programs are pervasive security critical functions, server side sanitization functions, databases, dynamic code generation. We need model counting for strings in order to make quantitative guarantees about these types of programs. Software for string constraint model counting Automata-Based Model Counter (ABC) [Aydin, et. al. CAV 2015] String Model Counter (SMC) [Luu, et. al. PLDI 2014 ] S3# [Trinh, et. al. CAV 2017]

24 11 / 46 Outline Motivation and Background Model Counting Boolean Formulas String Model Counting Automata-Based Methods Non-Automata-Based Method String Model Counting Benchmarks

25 12 / 46 Model Counting Recall the classic (boolean) SAT problem Given a formula φ from propositional logic, is it possible to assign all variables the values T (true) or F (false) so that the formula is true?

26 12 / 46 Model Counting Recall the classic (boolean) SAT problem Given a formula φ from propositional logic, is it possible to assign all variables the values T (true) or F (false) so that the formula is true? Example: φ = (x y) ( x z) (z w) x (y v)

27 12 / 46 Model Counting Recall the classic (boolean) SAT problem Given a formula φ from propositional logic, is it possible to assign all variables the values T (true) or F (false) so that the formula is true? Example: φ = (x y) ( x z) (z w) x (y v) φ is satisfiable by setting (x, y, z, w, v) = (T, F, T, F, T ).

28 12 / 46 Model Counting Recall the classic (boolean) SAT problem Given a formula φ from propositional logic, is it possible to assign all variables the values T (true) or F (false) so that the formula is true? Example: φ = (x y) ( x z) (z w) x (y v) φ is satisfiable by setting (x, y, z, w, v) = (T, F, T, F, T ). A satisfying assignment is called a model for φ.

29 13 / 46 Model Counting The model counting problem Given a formula φ over some theory (Boolean, LIA, Strings,... ) how many models are there for φ?

30 13 / 46 Model Counting The model counting problem Given a formula φ over some theory (Boolean, LIA, Strings,... ) how many models are there for φ? Difficulty of Model Counting Model counting is at least as hard as satisfiability check.

31 13 / 46 Model Counting The model counting problem Given a formula φ over some theory (Boolean, LIA, Strings,... ) how many models are there for φ? Difficulty of Model Counting Model counting is at least as hard as satisfiability check. φ > 0 φ is satisfiable

32 14 / 46 Model Counting Boolean SAT x y z w v F F F F F F F T F F T T F T F T F F F T F T F T T T F T T F F T F T T T T T T F F F F T T F F T F T T F T F F T T F T T F T T T F F T T T T F T T T T T T F T T T T T T T φ = (x y) ( x z) (z w) x (y v)

33 14 / 46 Model Counting Boolean SAT x y z w v F F F F F F F T F F T T F T F T F F F T F T F T T T F T T F F T F T T T T T T F F F F T T F F T F T T F T F F T T F T T F T T T F F T T T T F T T T T T T F T T T T T T T φ = (x y) ( x z) (z w) x (y v) φ has 6 models.

34 14 / 46 Model Counting Boolean SAT x y z w v F F F F F F F T F F T T F T F T F F F T F T F T T T F T T F F T F T T T T T T F F F F T T F F T F T T F T F F T T F T T F T T T F F T T T T F T T T T T T F T T T T T T T φ = (x y) ( x z) (z w) x (y v) φ has 6 models. Truth table method is θ(2 n ).

35 [1] Birnbaum, et. al. The good old Davis-Putnam procedure helps counting models. JAIR / 46 Model Counting Boolean SAT x y z w v F F F F F F F T F F T T F T F T F F F T F T F T T T F T T F F T F T T T T T T F F F F T T F F T F T T F T F F T T F T T F T T T F F T T T T F T T T T T T F T T T T T T T φ = (x y) ( x z) (z w) x (y v) φ has 6 models. Truth table method is θ(2 n ). DPLL method is O(2 n ), but is faster in practice. 1

36 15 / 46 Outline Motivation and Background Model Counting Boolean Formulas String Model Counting Automata-Based Methods Non-Automata-Based Method String Model Counting Benchmarks

37 16 / 46 Model Counting Strings A formula over the theory of strings can involve

38 16 / 46 Model Counting Strings A formula over the theory of strings can involve Word Equations: X U = Y Z

39 16 / 46 Model Counting Strings A formula over the theory of strings can involve Word Equations: X U = Y Z Length Constraints: 4 < Length(X) < 10

40 16 / 46 Model Counting Strings A formula over the theory of strings can involve Word Equations: X U = Y Z Length Constraints: 4 < Length(X) < 10 Regular Language Membership: X (a b)

41 16 / 46 Model Counting Strings A formula over the theory of strings can involve Word Equations: X U = Y Z Length Constraints: 4 < Length(X) < 10 Regular Language Membership: X (a b) and more complex constraints: (X = substring(y, i, j),... )

42 16 / 46 Model Counting Strings A formula over the theory of strings can involve Word Equations: X U = Y Z Length Constraints: 4 < Length(X) < 10 Regular Language Membership: X (a b) and more complex constraints: (X = substring(y, i, j),... )

43 17 / 46 Model Counting Strings Q: How many solutions for X? X (0 (1(01 0) 1))

44 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many!

45 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k?

46 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k}

47 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k}

48 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k} a 0 = 1 k X a k 0 ε 1

49 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k} a 0 = 1, a 1 = 1 k X a k 0 ε

50 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k} a 0 = 1, a 1 = 1, a 2 = 1 k X a k 0 ε

51 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k} a 0 = 1, a 1 = 1, a 2 = 1, a 3 = 1 k X a k 0 ε

52 17 / 46 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k} a 0 = 1, a 1 = 1, a 2 = 1, a 3 = 1, a 4 = 3 k X a k 0 ε , 1100,

53 Model Counting Strings X (0 (1(01 0) 1)) Q: How many solutions for X? A: Infinitely many! Q: How many solutions for X of length k? A counting sequence for language L encodes a k = {s : s L, len(s) = k} a 0 = 1, a 1 = 1, a 2 = 1, a 3 = 1, a 4 = 3, a 5 = 5,... k X a k 0 ε , 1100, , 10101, 11000, 11011, / 46

54 18 / 46 Outline Motivation and Background Model Counting Boolean Formulas String Model Counting Automata-Based Methods Non-Automata-Based Method String Model Counting Benchmarks

55 Deterministic Finite Automata 19 / 46

56 19 / 46 Deterministic Finite Automata X (0 (1(01 0) 1))

57 19 / 46 Deterministic Finite Automata X (0 (1(01 0) 1))

58 19 / 46 Deterministic Finite Automata X (0 (1(01 0) 1)) {s : s L, len(s) = k} {π : π is accepting path of length k}

59 19 / 46 Deterministic Finite Automata X (0 (1(01 0) 1)) {s : s L, len(s) = k} {π : π is accepting path of length k} String Counting Path Counting

60 20 / 46 Deterministic Finite Automata How to count paths of length k?

61 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming

62 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming a k

63 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s a k (s) =

64 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s s a k (s) = a k 1 (s )

65 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s s a k (s) = a k 1 (s )

66 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s s 3 a k (s) = a k 1 (s )

67 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s s 3 a k (s) = a k 1 (s ) s s

68 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s Initial Conditions s 3 a k (s) = a k 1 (s ) s s

69 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s 3 s Initial Conditions a 0 (0) = 1 a k (s) = a k 1 (s ) s s

70 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s 3 s Initial Conditions a 0 (0) = 1, a 0 (1) = 0, a 0 (2) = 0 a k (s) = a k 1 (s ) s s

71 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s 3 s Initial Conditions a 0 (0) = 1, a 0 (1) = 0, a 0 (2) = 0 a k (s) = a k 1 (s ) s s System of Recurrences a k (0) = a k 1 (0) + a k 1 (1)

72 20 / 46 Deterministic Finite Automata How to count paths of length k? Dynamic Programming s 1 s 2 s 3 s Initial Conditions a 0 (0) = 1, a 0 (1) = 0, a 0 (2) = 0 a k (s) = a k 1 (s ) s s System of Recurrences a k (0) = a k 1 (0) + a k 1 (1) a k (1) = a k 1 (0) + a k 1 (2) a k (2) = a k 1 (1) + a k 1 (2)

73 21 / 46 Deterministic Finite Automata How to count paths of length k?

74 21 / 46 Deterministic Finite Automata How to count paths of length k? System of Recurrences a k (0) = a k 1 (0) + a k 1 (1) a k (1) = a k 1 (0) + a k 1 (2) a k (2) = a k 1 (1) + a k 1 (2)

75 21 / 46 Deterministic Finite Automata How to count paths of length k? System of Recurrences a k (0) = a k 1 (0) + a k 1 (1) a k (1) = a k 1 (0) + a k 1 (2) a k (2) = a k 1 (1) + a k 1 (2) a 0(k) a 1 (k) = a 0(k 1) a 1 (k 1) a 2 (k) a 2 (k 1)

76 21 / 46 Deterministic Finite Automata How to count paths of length k? Matrix Exponentiation System of Recurrences a k (0) = a k 1 (0) + a k 1 (1) a k (1) = a k 1 (0) + a k 1 (2) a k (2) = a k 1 (1) + a k 1 (2) a 0(k) a 1 (k) = a 2 (k) k a 0(k) a 1 (k) = a 0(k 1) a 1 (k 1) a 2 (k) a 2 (k 1)

77 21 / 46 Deterministic Finite Automata How to count paths of length k? Matrix Exponentiation System of Recurrences a k (0) = a k 1 (0) + a k 1 (1) a k (1) = a k 1 (0) + a k 1 (2) a k (2) = a k 1 (1) + a k 1 (2) a 0(k) a 1 (k) = a 2 (k) k a 0(k) a 1 (k) = a 0(k 1) a 1 (k 1) a 2 (k) a 2 (k 1) a k = (A k ) 0,F

78 21 / 46 Deterministic Finite Automata How to count paths of length k? Matrix Exponentiation System of Recurrences a k (0) = a k 1 (0) + a k 1 (1) a k (1) = a k 1 (0) + a k 1 (2) a k (2) = a k 1 (1) + a k 1 (2) a 0(k) a 1 (k) = a 2 (k) k a 0(k) a 1 (k) = a 0(k 1) a 1 (k 1) a 2 (k) a 2 (k 1) a k = (A k ) 0,F a 4 = (A 4 ) 0,0 = 3

79 22 / 46

80 Generating functions are a way to compactly represent (possibly infinite) sequences. 22 / 46

81 22 / 46 Generating functions are a way to compactly represent (possibly infinite) sequences. g(z) = 1 (1 z) 3

82 22 / 46 Generating functions are a way to compactly represent (possibly infinite) sequences. 1 g(z) = (1 z) 3 = a k z k k=0

83 22 / 46 Generating functions are a way to compactly represent (possibly infinite) sequences. 1 g(z) = (1 z) 3 = a k z k k=0 g(z) = 1z 0 + 3z 1 + 6z z z

84 22 / 46 Generating functions are a way to compactly represent (possibly infinite) sequences. 1 g(z) = (1 z) 3 = a k z k k=0 g(z) = 1z 0 + 3z 1 + 6z z z g(z) = a 0 z 0 + a 1 z 1 + a 2 z 2 + a 3 z 3 + a 4 z

85 22 / 46 Generating functions are a way to compactly represent (possibly infinite) sequences. 1 g(z) = (1 z) 3 = a k z k k=0 g(z) = 1z 0 + 3z 1 + 6z z z g(z) = a 0 z 0 + a 1 z 1 + a 2 z 2 + a 3 z 3 + a 4 z Sequence element a k is the k th Taylor series coefficient of g(z).

86 23 / 46 The Taylor series of a function g(z) that is differentiable at 0 is the power series g(0) + g (0) 1! x + g (0) 2! x 2 + g (0) x !

87 23 / 46 The Taylor series of a function g(z) that is differentiable at 0 is the power series g(0) + g (0) 1! x + g (0) 2! x 2 + g (0) x ! which can be written in the more compact sigma notation as n=0 g (n) (a) n! (x a) n where n! denotes the factorial of n and g(n) (a) denotes the n-th derivative of f evaluated at the point a.

88 X (0 (1(01 0) 1)) 24 / 46

89 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k}

90 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) =

91 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) = 1z 0 k X a k 0 ε 1

92 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) = 1z 0 + 1z 1 k X a k 0 ε

93 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) = 1z 0 + 1z 1 + 1z 2 k X a k 0 ε

94 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) = 1z 0 + 1z 1 + 1z 2 + 1z 3 k X a k 0 ε

95 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) = 1z 0 + 1z 1 + 1z 2 + 1z 3 + 3z 4 k X a k 0 ε , 1100,

96 24 / 46 X (0 (1(01 0) 1)) A generating function for language L encodes a k = {s : s L, len(s) = k} g(z) = 1z 0 + 1z 1 + 1z 2 + 1z 3 + 3z 4 + 5z k X a k 0 ε , 1100, , 10101, 11000, 11011,

97 25 / 46 Deterministic Finite Automata How to count paths of length k?

98 25 / 46 Deterministic Finite Automata How to count paths of length k? Generating Functions A = g(z) = det(i za : i, j) ( 1) n det(i za)

99 25 / 46 Deterministic Finite Automata How to count paths of length k? Generating Functions A = g(z) = det(i za : i, j) ( 1) n det(i za) g(z) = 1 z z 2 (z 1) (2z 2 + z 1)

100 25 / 46 Deterministic Finite Automata How to count paths of length k? Generating Functions A = g(z) = det(i za : i, j) ( 1) n det(i za) g(z) = 1 z z 2 (z 1) (2z 2 + z 1) g(z) = 1z 0 + 1z 1 + 1z 2 + 1z 3 + 3z 4 + 5z

101 Good job, Will Hunting!!! 26 / 46

102 Automata-Based Model Counter (ABC) CAV 2015: Automata-Based Model Counting for String Constraints. Abdulbaki Aydin, Lucas Bang, Tevfik Bultan.

103 Automata-Based Model Counter (ABC) CAV 2015: Automata-Based Model Counting for String Constraints. Abdulbaki Aydin, Lucas Bang, Tevfik Bultan. String Constraints ABC Counting Function f (k)

104 Automata-Based Model Counter (ABC) CAV 2015: Automata-Based Model Counting for String Constraints. Abdulbaki Aydin, Lucas Bang, Tevfik Bultan. String Length, k String Constraints ABC Counting Function f (k)

105 27 / 46 Automata-Based Model Counter (ABC) CAV 2015: Automata-Based Model Counting for String Constraints. Abdulbaki Aydin, Lucas Bang, Tevfik Bultan. String Length, k String Constraints ABC Counting Function f (k) Number of solutions of length k

106 27 / 46 Automata-Based Model Counter (ABC) CAV 2015: Automata-Based Model Counting for String Constraints. Abdulbaki Aydin, Lucas Bang, Tevfik Bultan. String Length, k String Constraints ABC Counting Function f (k) Number of solutions of length k Idea: Convert string constraints to DFA. Count paths in DFA.

107 28 / 46 Password Changing Policy Constraint on NEW_P (declare-fun NEW_P () String) (not (contains (tolower NEW_P) "abc-16")) (not (contains "abc-16" (tolower NEW_P))) (not (contains (tolower NEW_P) "61-cba")) (not (contains "61-cba" (tolower NEW_P))) (check-sat) (model-count)

108 0 A, a 6 1 B, b 5 - C, c B, b 1 - C, c C, c - C, c - - C, c [NUL-,], [.-0], [2-5], [7-@] `, [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] 6 12 B, b A, a - [NUL-5], [7-@], `, C, c, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] 1 [NUL-0], [2-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] C, c [NUL-,], [.-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] B, b [NUL-5], [7-@], `, B, b, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] 28 [NUL-5], [7-@], `, B, b, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] [NUL-0], [2-5], [7-@], `, B b, [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] [NUL-,], [.-5], [7-@], `, C c, [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] 11 [NUL-,], [.-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] [NUL-5], [7-@], `, B, b, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] [NUL-,], [.-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] [NUL-,], [.-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] 6 [NUL-0], [2-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] [NUL-5], [7-@], `, B, b, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] [NUL-5], [7-@], `, C, c, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] B, b 6 6 A, a [NUL-0], [2-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] 14 [NUL-5], [7-@], `, C, c, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] [NUL-0], [2-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] 6 6 [NUL-5], [7-@], `, C, c, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] A, a A, a [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] A, a A, a A, a A, a A, a 6 1 [NUL-0], [2-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] A, a A, a A, a A, a A, a A, a [NUL-,], [.-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] 6 9 A, a A, a [NUL-5], [7-@], `, B, b, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] 1 6 A, a A, a A, a A, a [NUL-5], [7-@], `, C, c, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] 21 6 [NUL-5], [7-@], `, C, c, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] A, a - [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] 6 B, b [NUL-5], [7-@], `, B, b, [D-G] [d-g], [H-O], [h-o], [P-_] [p-253] A, a A, a C, c [NUL-,], [.-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] 6 [NUL-0], [2-5], [7-@], `, [B-C] [b-c], [D-G], [d-g], [H-O] [h-o], [P-_], [p-253] A, a 29 6 [NUL-5], [7-@], `, [B-C], [b-c] [D-G], [d-g], [H-O], [h-o] [P-_], [p-253] A, a B, b 6 A, a B, b A, a A, a 22 A, a C, c A, a / Password Changing Policy Figure: Solution DFA for all possible values of NEWP.

109 Password Changing Policy Figure: Transition matrix for DFA for all possible values of NEWP. 30 / 46

110 Password Changing Policy Figure: Transition matrix for DFA for all possible values of NEWP. 31 / 46

111 32 / 46 Password Changing Policy Generating function which enumerates NEW_P: g(z) = 8096z z z z 7 16z 6 256z z z z z z z z 3 247z 2

112 32 / 46 Password Changing Policy Generating function which enumerates NEW_P: g(z) = 8096z z z z 7 16z 6 256z z z z z z z z 3 247z 2 g(z) = 247z z z z z

113 32 / 46 Password Changing Policy Generating function which enumerates NEW_P: g(z) = 8096z z z z 7 16z 6 256z z z z z z z z 3 247z 2 g(z) = 247z z z z z To answer our quantitative question: Brute force searching for password length = 6: = 2 48 passwords.

114 32 / 46 Password Changing Policy Generating function which enumerates NEW_P: g(z) = 8096z z z z 7 16z 6 256z z z z z z z z 3 247z 2 g(z) = 247z z z z z To answer our quantitative question: Brute force searching for password length = 6: = 2 48 passwords. If adversary knows old_p and the policy: passwords.

115 32 / 46 Password Changing Policy Generating function which enumerates NEW_P: g(z) = 8096z z z z 7 16z 6 256z z z z z z z z 3 247z 2 g(z) = 247z z z z z To answer our quantitative question: Brute force searching for password length = 6: = 2 48 passwords. If adversary knows old_p and the policy: passwords. Reduces search space by about factor of

116 33 / 46 Outline Motivation and Background Model Counting Boolean Formulas String Model Counting Automata-Based Methods Non-Automata-Based Method String Model Counting Benchmarks

117 34 / 46 SMC Model Counting PLDI 2014: A Model Counter For Constraints Over Unbounded Strings. Luu, Shinde, Saxena, Demsky. SMC Tool Online: String Length, k String Constraints SMC Generating Function, g(z) Number of strings of length k

118 34 / 46 SMC Model Counting PLDI 2014: A Model Counter For Constraints Over Unbounded Strings. Luu, Shinde, Saxena, Demsky. SMC Tool Online: String Length, k String Constraints SMC Generating Function, g(z) Number of strings of length k Idea: go directly from constraints to g(z) using transformations.

119 35 / 46 SMC Model Counting For a regular expression constraint, generating function can be derived recursively.

120 35 / 46 SMC Model Counting For a regular expression constraint, generating function can be derived recursively. ε 1z 0

121 35 / 46 SMC Model Counting For a regular expression constraint, generating function can be derived recursively. ε 1z 0 c 1z 1

122 35 / 46 SMC Model Counting For a regular expression constraint, generating function can be derived recursively. ε 1z 0 c 1z 1 A B A(z) + B(z)

123 35 / 46 SMC Model Counting For a regular expression constraint, generating function can be derived recursively. ε 1z 0 c 1z 1 A B A(z) + B(z) A B A(z) B(z)

124 35 / 46 SMC Model Counting For a regular expression constraint, generating function can be derived recursively. ε 1z 0 c 1z 1 A B A(z) + B(z) A B A(z) B(z) A 1/(1 A(z))

125 Regular Expressions X (0 (1(01 0) 1))

126 Regular Expressions X (0 (1(01 0) 1)) z 1

127 Regular Expressions X (0 (1(01 0) 1)) z 0 1

128 Regular Expressions X (0 (1(01 0) 1)) z z 0 1

129 Regular Expressions X (0 (1(01 0) 1)) z 1 z 0 1

130 Regular Expressions X (0 (1(01 0) 1)) z 0 z 1 z 0 1

131 Regular Expressions X (0 (1(01 0) 1)) 0 1 z 2 1 z

132 Regular Expressions X (0 (1(01 0) 1)) z2 1 z 1 z 0 0 1

133 Regular Expressions X (0 (1(01 0) 1)) 0 z 1 z 1 z2 1 z

134 Regular Expressions X (0 (1(01 0) 1)) z 0 z 2 1 z2 1 z

135 Regular Expressions X (0 (1(01 0) 1)) z + z2 1 z2 1 z

136 36 / 46 Regular Expressions 1 1 z z2 1 1 z z2 X (0 (1(01 0) 1))

137 Regular Expressions X (0 (1(01 0) 1)) Generating Function: g(z) = 1 1 z z2 1 1 z z

138 Regular Expressions X (0 (1(01 0) 1)) Generating Function: g(z) = 1 1 z z2 1 1 z z2 = 1 z z 2 (z 1)(2z 2 +z 1) 0 1

139 Regular Expressions X (0 (1(01 0) 1)) Generating Function: g(z) = 1 1 z z2 1 1 z z2 = 1 z z 2 (z 1)(2z 2 +z 1) 0 1 g(z) = 1z 0 +1z 1 +1z 2 +1z 3 +3z 4 +5z

140 37 / 46 Other operations in SMC Specialized transformations for other operations

141 37 / 46 Other operations in SMC Specialized transformations for other operations contains(s 1, s 2 ) z n (1 Mz)(z n +(1 Mz)c(z))

142 37 / 46 Other operations in SMC Specialized transformations for other operations contains(s 1, s 2 ) z n (1 Mz)(z n +(1 Mz)c(z)) F 1 F 2 [max(l 1 (z), L 2 (z)), min(u 1 (z) + U 2 (z), G(z))]

143 37 / 46 Other operations in SMC Specialized transformations for other operations contains(s 1, s 2 ) z n (1 Mz)(z n +(1 Mz)c(z)) F 1 F 2 [max(l 1 (z), L 2 (z)), min(u 1 (z) + U 2 (z), G(z))] Also handle substring, length, negation, conjunction,..., with upper and lower bounds.

144 38 / 46 Outline Motivation and Background Model Counting Boolean Formulas String Model Counting Automata-Based Methods Non-Automata-Based Method String Model Counting Benchmarks

145 39 / 46 Experimental Comparison Table: Log scaled comparison between SMC and ABC bound SMC SMC ABC lower bound upper bound count nullhttpd ghttpd csplit grep wc obscure

146 40 / 46 Experimental Comparison JavaScript Benchmarks Kaluza benchmarks, extracted from JavaScript code via DSE, [Saxena, SSP 2010]

147 40 / 46 Experimental Comparison JavaScript Benchmarks Kaluza benchmarks, extracted from JavaScript code via DSE, [Saxena, SSP 2010] Small Constraints (19,731): ABC: 19,731 constraints, average 0.32 seconds per constraint SMC: 17,559 constraints, average 0.26 seconds per constraint.

148 40 / 46 Experimental Comparison JavaScript Benchmarks Kaluza benchmarks, extracted from JavaScript code via DSE, [Saxena, SSP 2010] Small Constraints (19,731): ABC: 19,731 constraints, average 0.32 seconds per constraint SMC: 17,559 constraints, average 0.26 seconds per constraint. Big Constraints (1,587): ABC: 1,587 constraints, average 0.34 seconds per constraint SMC: 1,342 constraints, average 5.29 seconds per constraint

149 ABC Bonus: Model Counting Linear Integer Arithmetic 41 / 46

150 41 / 46 ABC Bonus: Model Counting Linear Integer Arithmetic What is this language? X (0 (1(01 0) 1))

151 41 / 46 ABC Bonus: Model Counting Linear Integer Arithmetic What is this language? X (0 (1(01 0) 1)) L(X) = {s s is a binary number divisible by 3}

152 41 / 46 ABC Bonus: Model Counting Linear Integer Arithmetic What is this language? X (0 (1(01 0) 1)) L(X) = {s s is a binary number divisible by 3}

153 41 / 46 ABC Bonus: Model Counting Linear Integer Arithmetic What is this language? X (0 (1(01 0) 1)) L(X) = {s s is a binary number divisible by 3} Idea: DFA can represent (some) relations on sets of binary integers. We can use similar techniques that we used for #String to solve #LIA.

154 42 / 46 Model Counting Linear Integer Arithmetic Quantifier-Free Linear Integer Arithmetic (Z, +, <).

155 42 / 46 Model Counting Linear Integer Arithmetic Quantifier-Free Linear Integer Arithmetic (Z, +, <). Constraints of the form: Ax < B, x Z n

156 42 / 46 Model Counting Linear Integer Arithmetic Quantifier-Free Linear Integer Arithmetic (Z, +, <). Constraints of the form: Ax < B, x Z n It is possible to represent the solutions to a set of LIA constraints as a binary multi-track DFA.

157 43 / 46 Binary Multi-track DFA Solution DFA for LIA constraints. Read bits of x and y from most to least significant. ( ) bx Alphabet is a tuple of bits: b y Solution DFA for the constraint x > y. ( ( 0 1, 0) 1) ( 1 0) = > ( 0 1) < ( ( ( ( ,,, 0) 1) 0) 1) ( ( ( ( ,,, 0) 1) 0) 1)

158 43 / 46 Binary Multi-track DFA Solution DFA for LIA constraints. Read bits of x and y from most to least significant. ( ) bx Alphabet is a tuple of bits: b y Solution DFA for the constraint x > y. ( ( 0 1, 0) 1) ( 1 0) = > ( 0 1) < ( ( ( ( ,,, 0) 1) 0) 1) ( ( ( ( ,,, 0) 1) 0) 1) Solutions of length n solutions within bound 2 n

159 44 / 46 Model Counting Summary Counting Techniques for Different Theories Boolean

160 44 / 46 Model Counting Summary Counting Techniques for Different Theories Boolean Truth Table (Brute Force) DPLL

161 44 / 46 Model Counting Summary Counting Techniques for Different Theories Boolean Truth Table (Brute Force) DPLL Strings DFA with Dynamic Programming, Matrix Multiplication, GFs Regular Expression with GFs

162 44 / 46 Model Counting Summary Counting Techniques for Different Theories Boolean Truth Table (Brute Force) DPLL Strings DFA with Dynamic Programming, Matrix Multiplication, GFs Regular Expression with GFs Linear Integer Arithmetic Binary Multi-track DFA

163 45 / 46 Related work on model counting Stanley. Enumerative Combinatorics Chapter Sedgwick. Analytic Combinatorics Chapter 5: Generating Functions Biere. Handbook of Satisfiability. Chapter 20: Model Counting Pugh. Counting Solutions to Presburger Formulas: How and Why Parker. An Automata-Theoretic Algorithm for Counting Solutions to Presburger Formulas. Compiler Construction 2004 Boigelot. Counting the solutions of Presburger equations without enumerating them. TCS Barvinok. A polynomial time algorithm for counting integral points in polyhedra when the dimension is fixed. Mathematics of Operations Research 1994 De Loerab. Effective lattice point counting in rational convex polytopes. JSC 2004 Verdoolaege. Counting integer points in parametric polytopes using Barvinoks s Rational Functions Kopf Symbolic Polytopes for Quantitative Interpolation and Verification. CAV 2015 Luu. A Model Counter For Constraints Over Unbounded Strings. PLDI 2014 Ravikumara. Weak minimization of DFA - an algorithm and applications.implementation and Application of Automata 2004 Chomsky. The Algebraic Theory of Context-Free Languages Phan. Model Counting Modulo Theories. PhD Thesis Birnbaum. The good old Davis-Putnam procedure helps counting models. JAIR 1999

164 Thank you. 46 / 46