Hyper-Minimization. Lossy compression of deterministic automata. Andreas Maletti. Institute for Natural Language Processing University of Stuttgart

Hyper-Minimization Lossy compression of deterministic automata Andreas Maletti Institute for Natural Language Processing University of Stuttgart andreas.maletti@ims.uni-stuttgart.de Debrecen August 19, 2011 A. Maletti AFL 2011 1

Collaborators Reporting joint work with PAWEŁ GAWRYCHOWSKI MARKUS HOLZER ARTUR JEŻ DANIEL QUERNHEIM University of Wrocław University of Giessen University of Wrocław University of Stuttgart A. Maletti AFL 2011 2

Contents 1 History and Motivation 2 Hyper-Minimization 3 Hyper-Optimization 4 Restrictions and limitations A. Maletti AFL 2011 3

Problem definition Minimization given: DFA A return: minimal DFA B such that L(B) = L(A) Hyper-minimization given: DFA A return: minimal DFA B such that L(B) and L(A) differ finitely A. Maletti AFL 2011 4

AFL 2008 (Balatonfüred, Hungary) VILIAM GEFFERT spoke about: general problem and its structural characterization (inefficient) hyper-minimization [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 5

AFL 2008 (Balatonfüred, Hungary) VILIAM GEFFERT spoke about: general problem and its structural characterization (inefficient) hyper-minimization Unfortunately, I was not there [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 6

CIAA 2008 (San Francisco, CA, USA) ANDREW BADR spoke about: faster (yet still inefficient) hyper-minimization combination with cover automata minimization [BADR: Hyper-minimization in O(n 2 ). CIAA 2008] A. Maletti AFL 2011 7

CIAA 2008 (San Francisco, CA, USA) ANDREW BADR spoke about: faster (yet still inefficient) hyper-minimization combination with cover automata minimization MARKUS HOLZER and I were there [BADR: Hyper-minimization in O(n 2 ). CIAA 2008] A. Maletti AFL 2011 8

c David Iliff CIAA 2009 (Sydney, Australia) MARKUS HOLZER spoke about: efficient hyper-minimization [HOLZER, : An n log n algorithm for hyper-minimizing states in a (minimized) deterministic automaton. CIAA 2009] A. Maletti AFL 2011 9

c David Iliff CIAA 2009 (Sydney, Australia) MARKUS HOLZER spoke about: efficient hyper-minimization But at the same time... [HOLZER, : An n log n algorithm for hyper-minimizing states in a (minimized) deterministic automaton. CIAA 2009] A. Maletti AFL 2011 10

MFCS 2009 (Novy Smokovec, Slovakia) ARTUR JEŻ spoke about: efficient hyper-minimization (essentially the same algorithm) k-minimization [GAWRYCHOWSKI, JEŻ: Hyper-minimisation made efficient. MFCS 2009] Best student paper A. Maletti AFL 2011 11

CIAA 2010 (Winnipeg, Canada) I spoke about: error-optimal hyper-minimization [ : Better hyper-minimization not as fast, but fewer errors. CIAA 2010] A. Maletti AFL 2011 12

FSTTCS 2010 (Chennai, India) SVEN SCHEWE spoke about: hyper-minimization for DBA (deterministic Büchi automata) [SCHEWE: Beyond Hyper-Minimisation Minimising DBAs and DPAs is NP-Complete. FSTTCS 2010] A. Maletti AFL 2011 13

CIAA 2011 (Blois, France) ARTUR JEŻ spoke about: efficient combination with cover automata minimization [JEŻ, : Computing all l-cover automata fast. CIAA 2011] A. Maletti AFL 2011 14

AFL 2011 (Debrecen, Hungary) DANIEL QUERNHEIM I spoke about: hyper-minimization for weighted DFA [, QUERNHEIM: Hyper-minimisation of deterministic weighted finite automata over semifields. AFL 2011] A. Maletti AFL 2011 15

MFCS 2011 (Warsaw, Poland) PAWEŁ GAWRYCHOWSKI will speak about: efficient hyper-minimization for partial DFA limits of hyper-minimization [GAWRYCHOWSKI, JEŻ, : On minimising automata with errors. MFCS 2011] A. Maletti AFL 2011 16

Experiments on random NFA 250 200 150 100 50 0 size of mdfa 1 0.8 0.6 0.4 0.2 0 states saved 1 0.8 0.6 cyclicity 0.4 0.2 0 6 5 4 3 1 2 density 0 1 0.8 0.6 cyclicity 0.4 0.2 0 6 5 4 3 2 1 density 0 Left: Size of minimal DFA Right: Ratio of saved states in hyper-minimization Conclusion significant savings A. Maletti AFL 2011 17

Error analysis number of errors 6 4 2 0 0.5 0.4 0.3 0.2 0.1 0 errors avoided 1 0.8 0.6 3 cyclicity 0.4 4 0.2 5 0 6 0 1 1 2 density 0.8 0.6 3 cyclicity 0.4 4 0.2 5 0 6 0 1 2 density Left: Average number of errors (100 NFA per data point) Right: Ratio of avoided errors in optimal hyper-minimization [, QUERNHEIM: Optimal hyper-minimization. IJFCS 2011] [TABAKOV, VARDI: Experimental evaluation of classical automata constructions. LPAR 2005] A. Maletti AFL 2011 19

A. Maletti AFL 2011 20

Contents 1 History and Motivation 2 Hyper-Minimization 3 Hyper-Optimization 4 Restrictions and limitations A. Maletti AFL 2011 21

Basic definitions Definition Two languages L 1, L 2 are almost equal if L 1 L 2 is finite L 1 L 2 = (L 1 \ L 2 ) (L 2 \ L 1 ) Two DFA A 1, A 2 are almost equivalent if L(A 1 ) and L(A 2 ) are almost equal Example all finite languages are almost equal a and aaa are almost equal a and (aa) are not almost equal A. Maletti AFL 2011 22

Overview Common approach 1 Identify kernel states 2 Identify almost equivalent states 3 Merge states A. Maletti AFL 2011 23

Preamble and kernel states Definition preamble state: finitely many words lead to it kernel state: infinitely many words lead to it Words leading to state q: {w Σ δ(q 0, w) = q} A. Maletti AFL 2011 24

Preamble and kernel states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 25

Preamble and kernel states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 26

Computing kernel states Step 1 Compute strongly connected components (using TARJAN S algorithm) F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 27

Computing kernel states Step 1 Compute strongly connected components (using TARJAN S algorithm) F J M B E I L Q A D H R P 0 C G 2 Step 2 Mark all successors of nontrivial components F J M B E I L Q [TARJAN: Depth-first search and linear graph algorithms. SIAM J. Comput. 1972] A D H R P 0 C G A. Maletti AFL 2011 28 2

Computing kernel states Theorem We can compute the set of kernel states in linear time. A. Maletti AFL 2011 29

Almost equivalent states Definition Two states are almost equivalent if their right languages are almost equal Right language of state q: {w Σ δ(q, w) F} A. Maletti AFL 2011 30

Almost equivalent states Definition Two states are almost equivalent if their right languages are almost equal Right language of state q: {w Σ δ(q, w) F} Consequence For almost equivalent states p, q there is k N such that δ(p, w) = δ(q, w) for all w > k A. Maletti AFL 2011 31

Almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 32

Almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 33

Computing almost equivalent states Theorem Almost equivalence is a congruence. Theorem If p, q are different, but almost equivalent, then there are p, q such that δ(p, σ) = δ(q, σ) for all σ Σ. A. Maletti AFL 2011 34

Computing almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 35

Computing almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 36

Computing almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 37

Computing almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 38

Computing almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 39

Computing almost equivalent states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 40

Computing almost equivalent states Theorem The partition representing almost equivalent states can be computed in O(n log n) using O(n 2 ) space O(n log 2 n) using O(n) space [HOLZER, : An n log n algorithm for hyper-minimizing states in a (minimized) deterministic automaton. CIAA 2009] [GAWRYCHOWSKI, JEŻ: Hyper-minimisation made efficient. MFCS 2009] A. Maletti AFL 2011 41

Merging states Algorithm don t-care nondeterministic select representative of each block; kernel state if possible merge all preamble states into their representative [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 42

Merging states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 43

Merging states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 44

Merging states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 45

Merging states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 46

Merging states Definition A DFA is hyper-minimal if all almost equivalent DFA are larger. A. Maletti AFL 2011 47

Merging states Definition A DFA is hyper-minimal if all almost equivalent DFA are larger. Theorem A DFA is hyper-minimal if and only if it is minimal no preamble state is almost equivalent to another state [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 48

Merging states F J M merges: D into C B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 49

Merging states F J M merges: D into C G into I B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 50

Merging states F J M merges: D into C G into I H into J B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 51

Merging states F J M merges: D into C G into I H into J B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 52

Merging states Theorem Merging can be done in linear time. A. Maletti AFL 2011 53

Merging states Theorem Merging can be done in linear time. Theorem Hyper-minimization can be achieved in time O(n log n). A. Maletti AFL 2011 54

Contents 1 History and Motivation 2 Hyper-Minimization 3 Hyper-Optimization 4 Restrictions and limitations A. Maletti AFL 2011 55

Hyper-optimization Goal Obtain a DFA that 1 makes only finitely many mistakes 2 is as small as possible A. Maletti AFL 2011 56

Hyper-optimization Goal Obtain a DFA that 1 makes only finitely many mistakes 2 is as small as possible 3 additionally makes minimal number of mistakes Question Can it be done in polynomial time? [BADR, GEFFERT, SHIPMAN 2009] Can it be done in O(n log n)? A. Maletti AFL 2011 57

Comparison F J M F J M B E I L Q B E I L Q A D H R P A D H R P 0 C G 0 C G 2 2 A. Maletti AFL 2011 59

Comparison Theorem Two almost equivalent, hyper-minimal DFA are isomorphic up to 1 finality of preamble states 2 transitions from preamble to kernel states 3 initial state [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 60

Optimal merges F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 61

Finality of preamble states F J M B E I L Q A D H R P Question Which words lead to C? word w 2 2 w L 0 C G 2 A. Maletti AFL 2011 62

Finality of preamble states F J M B E I L Q A D H R P Question Which words lead to C? word w w L 2 2 0 C G 2 A. Maletti AFL 2011 63

Finality of preamble states F J M B E I L Q A D H R P Question Which words lead to C? word w w L 2 2 0 C G 2 make C final A. Maletti AFL 2011 64

Optimal merges F J M Errors B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 65

Transitions from preamble to kernel states F J M B E I L Q A D H R P Question On which words differ almost equivalent states? states words (number) P Q ε (1) 0 C G 2 A. Maletti AFL 2011 66

Transitions from preamble to kernel states F J M B E I L Q A D H R P Question On which words differ almost equivalent states? states words (number) P Q ε (1) L M (1) 0 C G 2 A. Maletti AFL 2011 67

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G Question On which words differ almost equivalent states? states words (number) P Q ε (1) L M (1) I J 2 (1) 2 A. Maletti AFL 2011 68

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G Question On which words differ almost equivalent states? states words (number) P Q ε (1) L M (1) I J 2 (1) H J ε, 2 (2) 2 A. Maletti AFL 2011 69

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G 2 Question On which words differ almost equivalent states? states words (number) P Q ε (1) L M (1) I J 2 (1) H J ε, 2 (2) H I ε, 2, 2 (3) A. Maletti AFL 2011 70

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 72

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 73

Transitions from preamble to kernel states F J M B E I L Q Errors u w u leads to C w error between H I A D H R P 0 C G 2 2 2 2 2 2 2 2 2 2 2 (6) A. Maletti AFL 2011 74

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 75

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G 2 A. Maletti AFL 2011 76

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G Errors u w u leads to C (2) w H J (2) or u leads to D (1) w I J (1) 2 A. Maletti AFL 2011 77

Transitions from preamble to kernel states F J M B E I L Q A D H R P 0 C G Errors u w u leads to C (2) w H J (2) or u leads to D (1) w I J (1) 2 only 2 2 + 1 1 = 5 errors A. Maletti AFL 2011 78

Optimal merges (cont d) F J M Errors B E I L Q A D H R P 0 C G 2 2 2 2 2 2 2 (6) 2 A. Maletti AFL 2011 79

Optimal merges (cont d) F J M Errors B E I L Q A D H R P 0 C G 2 2 2 2 2 2 2 2... (15) A. Maletti AFL 2011 80

Result Theorem Hyper-optimization can be achieved in O(n 2 ). [ : Better hyper-minimization not as fast, but fewer errors. CIAA 2010] A. Maletti AFL 2011 81

Result Theorem Hyper-optimization can be achieved in O(n 2 ). Theorem We can compute the number of errors of a hyper-minimal DFA relative to any almost equivalent DFA in time O(n 2 ). [ : Better hyper-minimization not as fast, but fewer errors. CIAA 2010] A. Maletti AFL 2011 82

Result Theorem Hyper-optimization can be achieved in O(n 2 ). Theorem We can compute the number of errors of a hyper-minimal DFA relative to any almost equivalent DFA in time O(n 2 ). Open question Can it also be done in O(n log n)? [ : Better hyper-minimization not as fast, but fewer errors. CIAA 2010] A. Maletti AFL 2011 83

Contents 1 History and Motivation 2 Hyper-Minimization 3 Hyper-Optimization 4 Restrictions and limitations A. Maletti AFL 2011 84

Canonical languages Definition Language L is canonical if recognized by a hyper-minimal DFA. Open question What are the closure properties of canonical languages? [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 85

Canonical languages class compl. conc. reversal regular canonical?? Table: Closure properties [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] [ : Notes on hyper-minimization. AFL 2011] A. Maletti AFL 2011 86

Canonical languages Input hyper-minimal DFA F J M F J M B E I L Q B E I L Q A D R P A C R P 0 2 0 2 A. Maletti AFL 2011 87

Canonical languages Minimal DFA recognizing union and intersection F J M F J M B E I L Q B E I L Q A C I L P A C I, J L, M P 0 2 R 0 2 R A. Maletti AFL 2011 88

k-minimization Definition Two languages L 1, L 2 are k-equivalent if L 1 L 2 Σ <k. Definition The gap of two states p, q is gap(p, q) = sup{ w δ(p, w) F xor δ(q, w) F} with sup = [GAWRYCHOWSKI, JEŻ: Hyper-minimisation made efficient. MFCS 2009] A. Maletti AFL 2011 89

k-minimization Definition Level of p is length of longest word leading to p level(p) = sup { w δ(q 0, w) = p} A. Maletti AFL 2011 90

k-minimization Definition Level of p is length of longest word leading to p level(p) = sup { w δ(q 0, w) = p} Definition Two states p, q are k-similar iff gap(p, q) + min(k, level(p), level(q)) < k A. Maletti AFL 2011 91

k-minimization Note k-similarity is not an equivalence! [GAWRYCHOWSKI, JEŻ: Hyper-minimisation made efficient. MFCS 2009] [GAWRYCHOWSKI, JEŻ, : On minimising automata with errors. MFCS 2011] A. Maletti AFL 2011 92

k-minimization Note k-similarity is not an equivalence! Theorem k-minimization can be done in time O(n log n). [GAWRYCHOWSKI, JEŻ: Hyper-minimisation made efficient. MFCS 2009] [GAWRYCHOWSKI, JEŻ, : On minimising automata with errors. MFCS 2011] A. Maletti AFL 2011 93

Optimize other criteria So far 1 make it as small as possible (allowing any finite number of errors) 2 minimize the number of errors A. Maletti AFL 2011 94

Optimize other criteria So far 1 make it as small as possible (allowing any finite number of errors) 2 minimize the number of errors More desirable optimize a ratio of saved states to committed errors A. Maletti AFL 2011 95

Error-bounded hyper-minimization Problem given DFA A, integers m, s construct a DFA with at most s states making at most m errors Note trivial with only one restriction A. Maletti AFL 2011 96

Bad news Theorem Error-bounded hyper-minimization is NP-complete. [GAWRYCHOWSKI, JEŻ, : On minimising automata with errors. MFCS 2011] A. Maletti AFL 2011 97

Bad news v e 1 1 2 3 e v 1. 1 2 3 v n e v n e 1 2 3 using reduction from 3-coloring problem redirection from middle to outside for 1 error redirection from outside to anywhere for 2 errors A. Maletti AFL 2011 98

Optimize other criteria (again) So far for k-minimization 1 make it as small as possible (allowing any number of errors of length at most k) A. Maletti AFL 2011 99

Optimize other criteria (again) So far for k-minimization 1 make it as small as possible (allowing any number of errors of length at most k) More desirable minimize the number of committed errors A. Maletti AFL 2011 100

Optimal k-minimization Problem given DFA A, integer k construct a DFA that is k-minimal for A commits the least number of errors for all such DFA Note Optimal hyper-minimization was possible in time O(n 2 ). A. Maletti AFL 2011 101

More bad news Theorem Optimal k-minimization is NP-complete. [GAWRYCHOWSKI, JEŻ, : On minimising automata with errors. MFCS 2011] A. Maletti AFL 2011 102

More bad news 30 3s 1 e 20 3s 1 21 k + 1 b l 1 2l k + l e 10 3s 1 11 k + 1 b l 1 1l k + l e v s + 1 e 0 k + 1 1 k + 2 {a, b} s 2 s 1 k + s s k + l + 1 s s v e... 3s {a, b} s 0 v v s + 1 e e k + 1 A. Maletti AFL 2011 103

Summary Open questions [BADR, GEFFERT, SHIPMAN] Properties of canonical languages Asymptotic state complexity Hyper-minimization of NFA, AFA, 2FA, WDFA Efficient hyper-minimization algorithm Minimize the number of errors Minimize length of longest error k-minimization [BADR, GEFFERT, SHIPMAN: Hyper-minimizing minimized deterministic finite state automata. ITA 2009] A. Maletti AFL 2011 104

That s all, folks! Thank you for your attention! A. Maletti AFL 2011 105