Special Factors and Suffix and Factor Automata

Size: px

Start display at page:

Download "Special Factors and Suffix and Factor Automata"

Alannah Cain
5 years ago
Views:

1 Special Factors and Suffix and Factor Automata LIAFA, Paris 5 November 2010

2 Finite Words Let Σ be a finite alphabet, e.g. Σ = {a, n, b, c}. A word over Σ is finite concatenation of symbols of Σ, that is, an element w Σ. If w = uv, u is called a prefix of w, and v a suffix of w. A factor of w is a prefix of a suffix of w (or, equiv. a suffix of a prefix). The empty word ε has length 0 and is factor of any word. A right infinite word is a non-ending concatenation of symbols of Σ.

3 Combinatorics of Finite Words Some examples of classes of finite words: palindromes: w R = w. Ex. radar, elle, ε.

4 Combinatorics of Finite Words Some examples of classes of finite words: palindromes: w R = w. Ex. radar, elle, ε. balanced words: for every a Σ, all the factors of the same length have the same number of a s up to 1. Ex. abaababaabaab.

5 Combinatorics of Finite Words Some examples of classes of finite words: palindromes: w R = w. Ex. radar, elle, ε. balanced words: for every a Σ, all the factors of the same length have the same number of a s up to 1. Ex. abaababaabaab. differentiable words: words over Σ such that their Run-Length Encoding is still a word over Σ. Ex over Σ = {1, 2}

6 Combinatorics of Finite Words Some examples of classes of finite words: palindromes: w R = w. Ex. radar, elle, ε. balanced words: for every a Σ, all the factors of the same length have the same number of a s up to 1. Ex. abaababaabaab. differentiable words: words over Σ such that their Run-Length Encoding is still a word over Σ. Ex over Σ = {1, 2} central, mechanical, rich, trapezoidal, etc.

7 Combinatorics of Finite Words Some examples of classes of finite words: palindromes: w R = w. Ex. radar, elle, ε. balanced words: for every a Σ, all the factors of the same length have the same number of a s up to 1. Ex. abaababaabaab. differentiable words: words over Σ such that their Run-Length Encoding is still a word over Σ. Ex over Σ = {1, 2} central, mechanical, rich, trapezoidal, etc. finite factors of infinite words: Sturmian and Episturmian words, smooth words, etc.

8 The Suffix Automaton Definition (A. Blumer et al. 85 M. Crochemore 86) The Suffix Automaton (or suffix DAWG) of the word w is the minimal deterministic automaton recognizing the suffixes of w. Example The SA of w = aabbabb: a a b b a b b b b 3 b 4 b a a 3

9 Algorithmic construction The SA allows the search of a pattern v in a text w in time and space O( v ). Theorem (A. Blumer et al. 85 M. Crochemore 86) The SA of a word w over a fixed alphabet Σ can be built in time and space O( w ).

10 One way to build the SA Build a naif non-deterministic automaton: w = aabbabb a a b b a b b

11 One way to build the SA Build a naif non-deterministic automaton: w = aabbabb a a b b a b b Determinize by subset construction: a a b b a b b {0, 1, 2,...,7} {1, 2, 5} {2} {3} {4} {5} {6} {7} b b b {3, 6} {4, 7} b a a {3, 4, 6, 7}

12 Ending Positions We associate to each factor v of w the set of ending positions of v in w. It is noted Endset w (v). Example w = a a b b a b b Endset w (ba) = {5}, Endset w (abb) = Endset w (bb) = {4, 7}.

13 Ending Positions We associate to each factor v of w the set of ending positions of v in w. It is noted Endset w (v). Example w = a a b b a b b Endset w (ba) = {5}, Endset w (abb) = Endset w (bb) = {4, 7}. Define on Fact(w) the equivalence: u SA v Endset w (u) = Endset w (v)

14 Ending Positions u SA v Endset w (u) = Endset w (v) Remark u SA v if and only if for any z Σ one has uz Suff(w) vz Suff(w) Remark Fact(w)/ SA is in bijection with the set of states of the SA of w.

15 The number of states of SA The number of states (classes) of the SA of w is therefore SA(w) = Fact(w)/ SA

16 The number of states of SA The number of states (classes) of the SA of w is therefore SA(w) = Fact(w)/ SA The bounds are well known: w + 1 SA(w) 2 w 1

17 The number of states of SA The number of states (classes) of the SA of w is therefore SA(w) = Fact(w)/ SA The bounds are well known: w + 1 SA(w) 2 w 1 The upper bound is reached for w = ab w 1, with a b.

18 The number of states of SA The number of states (classes) of the SA of w is therefore SA(w) = Fact(w)/ SA The bounds are well known: w + 1 SA(w) 2 w 1 The upper bound is reached for w = ab w 1, with a b. And for the lower bound?

19 The number of states of SA The number of states (classes) of the SA of w is therefore SA(w) = Fact(w)/ SA The bounds are well known: w + 1 SA(w) 2 w 1 The upper bound is reached for w = ab w 1, with a b. And for the lower bound? Problem (J. Berstel and M. Crochemore) Characterize the language L SA of words such that SA(w) = w + 1.

20 Special Factors Definition v is a left special factor of w if there exist a b such that av and bv are factors of w. v is a right special factor of w if there exist a b such that va and vb are factors of w. v is a bispecial factor of w if it is both left and right special.

21 Special Factors Definition v is a left special factor of w if there exist a b such that av and bv are factors of w. v is a right special factor of w if there exist a b such that va and vb are factors of w. v is a bispecial factor of w if it is both left and right special. Example ab is left special w = aabbabb

22 Special Factors Definition v is a left special factor of w if there exist a b such that av and bv are factors of w. v is a right special factor of w if there exist a b such that va and vb are factors of w. v is a bispecial factor of w if it is both left and right special. Example ab is left special b is right special w = aabbabb

23 Special Factors Definition v is a left special factor of w if there exist a b such that av and bv are factors of w. v is a right special factor of w if there exist a b such that va and vb are factors of w. v is a bispecial factor of w if it is both left and right special. Example ab is left special b is right special a and b are bispecial w = aabbabb

24 The number of states of SA Theorem (G. Fici 09) SA(w) = w S L w P w S L w = number of left special factors of w P w = length of the shortest prefix of w which is not left special

25 The number of states of SA Theorem (G. Fici 09) SA(w) = w S L w P w S L w = number of left special factors of w P w = length of the shortest prefix of w which is not left special Corollary (M. Sciortino and L.Q. Zamboni 07 G. Fici 09) w L SA if and only if every special factor of w is a prefix of w.

26 Example Example The SA of w = aabbabb: a a b b a b b b b 3 b 4 b a a 3 S L w = 5 since the left special factors of w are ɛ, a, b, ab, abb. P w = 2 since a is left special in w. SA(w) = w S L w P w = = 11.

27 Binary words Let f w denote the factor complexity of w, i.e. the function counting the number of distinct factors of w of each length. A binary word is a word w such that f w (1) = 2, i.e. having 2 distinct factors of length 1. Lemma Let w be a binary word. Then S L w = w H w. S L w = number of left special factors of w H w = length of the shortest unrepeated prefix of w

28 Binary words For binary words we thus have the formula: SA(w) = 2 w + 1 (H w + P w ) H w = length of the shortest unrepeated prefix of w P w = length of the shortest prefix of w which is not left special

29 Binary words For binary words we thus have the formula: SA(w) = 2 w + 1 (H w + P w ) H w = length of the shortest unrepeated prefix of w P w = length of the shortest prefix of w which is not left special As a corollary we obtain a new characterization of the set of prefixes of standard Sturmian words: Corollary A binary word w is a prefix of a standard Sturmian word if and only if w = H w + P w.

30 Example Example (w = aabbabb) a a b b a b b b b 3 b 4 b a a 3 H w = 2 since aa occurs only once in w. P w = 2 since a is left special in w. SA(w) = (2 + 2) = 11

31 The number of edges What about the number of edges E w?

32 The number of edges What about the number of edges E w? The bounds on E w are well known: w E w 3 w 4

33 The number of edges What about the number of edges E w? The bounds on E w are well known: w E w 3 w 4 For binary words we have the formula: Lemma E w = SA(w) + G(w) 1 G(w) is the union of the sets of bispecial factors and right special prefixes of w.

34 Example Example (w = aabbabb) a a b b a b b b b 3 b 4 b a a 3 G(w) = BIS(w) (Pref (w) RS(w)) = {ε, a, b} {ε, a} E w = SA(w) + G(w) 1 = = 13.

35 Standard Sturmian words A standard Sturmian word is the cutting sequence of a straight line of irrational slope and starting from the origin on the discrete plane. Lemma A right infinite word w is a standard Sturmian word if and only if the left special factors of w are prefixes of w.

36 The class of LSP words w L SA if and only if every special factor of w is a prefix of w.

37 The class of LSP words w L SA if and only if every special factor of w is a prefix of w. Corollary If Σ = 2, then w L SA if and only if w is prefix of a standard Sturmian word.

38 The class of LSP words w L SA if and only if every special factor of w is a prefix of w. Corollary If Σ = 2, then w L SA if and only if w is prefix of a standard Sturmian word. Corollary If Σ > 2, then: w is prefix of a standard Episturmian word w L SA. w is prefix of a standard ϑ-episturmian word w L SA (ϑ being any involutory antimorphism of Σ, i.e. such that ϑ(uv) = ϑ(v)ϑ(u) and ϑ ϑ = id).

39 The class of LSP words Definition A right infinite word w is LSP if the special factors of w are prefixes of w. So, if Σ = 2, LSP is the class of standard Sturmian words.

40 The class of LSP words Definition A right infinite word w is LSP if the special factors of w are prefixes of w. So, if Σ = 2, LSP is the class of standard Sturmian words. Problem Characterize the class of LSP words, over an arbitrary fixed alphabet Σ.

41 The class of LSP words Remark LSP words are, in general, not closed under reversal. Example Let φ be the morphism defined by a abcaab, b abc, and let w = φ(f ), the image of the Fibonacci word under φ. Then, for each n > 0, w has 1 l.s.f. but more than 1 r.s.f. of length n. Thus, the class of standard Episturmian words is strictly included in the class of LSP words.

42 The Factor Automaton Definition (A. Blumer et al. 85 M. Crochemore 86) The Factor Automaton (or factor DAWG) of the word w is the minimal deterministic automaton recognizing the factors of w. Example The FA of w = aabbabb: b a a b b a b b b 3 b a

43 Comparison between SA and FA Example (w=aabbabb) a a b b a b b b b 3 b 4 b a a 3 b a a b b a b b b 3 b a States 3 and 3 and states 4 and 4 have been identified.

44 Future Definition The future of v in w is what follows, in w, the occurrences of v: Fut w (v) = {z Σ : vz Fact(w)}

45 Future Definition The future of v in w is what follows, in w, the occurrences of v: Fut w (v) = {z Σ : vz Fact(w)} Example w = abbaabab Fut w (ba) = {ɛ, a, ab, aba, abab, b}.

46 Future Definition The future of v in w is what follows, in w, the occurrences of v: Fut w (v) = {z Σ : vz Fact(w)} Example w = abbaabab Fut w (ba) = {ɛ, a, ab, aba, abab, b}. Define on Fact(w) the equivalence: u FA v Fut w (u) = Fut w (v)

47 Future u FA v Fut w (u) = Fut w (v) Remark u FA v if and only if for any z Σ one has uz Fact(w) vz Fact(w) Remark Fact(w)/ FA is in bijection with the set of states of the FA of w.

48 The number of states of FA The number of states (classes) of the FA of w is therefore FA(w) = Fact(w)/ FA

49 The number of states of FA The number of states (classes) of the FA of w is therefore FA(w) = Fact(w)/ FA The bounds are well known: w + 1 FA(w) 2 w 2

50 The number of states of FA The number of states (classes) of the FA of w is therefore FA(w) = Fact(w)/ FA The bounds are well known: w + 1 FA(w) 2 w 2 The upper bound is reached for w = ab w 2 c, with a b c.

51 The number of states of FA The number of states (classes) of the FA of w is therefore FA(w) = Fact(w)/ FA The bounds are well known: w + 1 FA(w) 2 w 2 The upper bound is reached for w = ab w 2 c, with a b c. And for the lower bound?

52 The number of states of FA The number of states (classes) of the FA of w is therefore FA(w) = Fact(w)/ FA The bounds are well known: w + 1 FA(w) 2 w 2 The upper bound is reached for w = ab w 2 c, with a b c. And for the lower bound? Problem (J. Berstel and M. Crochemore) Characterize the language L FA of words such that FA(w) = w + 1.

53 Inclusion Remark If u SA v then u FA v. The converse is not true: Example w = abcaca Fut w (bc) = Fut w (c) but c appears also in position 5.

54 Inclusion Remark If u SA v then u FA v. The converse is not true: Example w = abcaca Fut w (bc) = Fut w (c) but c appears also in position 5. Clearly: FA(w) SA(w) and so L SA L FA.

55 Inclusion L SA L FA Example w = abcc We have SA(w) = 6 > w + 1, so w / L SA. Nevertheless FA(w) = 5 = w + 1, so w L FA.

56 The number of states of FA Does a formula for FA(w) exist?

57 The number of states of FA Does a formula for FA(w) exist? Definition (A. Blumer et al. 84) The stem of w is the shortest non-empty prefix v of the longest repeated suffix k of w such that v appears as prefix of k preceded by letter b and all other occurrences of v in w are preceded by letter a b, whenever such a prefix exists; otherwise it is undefined.

58 The number of states of FA Does a formula for FA(w) exist? Definition (A. Blumer et al. 84) The stem of w is the shortest non-empty prefix v of the longest repeated suffix k of w such that v appears as prefix of k preceded by letter b and all other occurrences of v in w are preceded by letter a b, whenever such a prefix exists; otherwise it is undefined. Example stem(aabbab) = ab stem(abacbb) is undefined.

59 The number of states of FA Lemma (A. Blumer et al. 84) The SA-classes that are identified by the FA-equivalence correspond to the prefixes x of the longest repeated suffix of w such that x stem(w).

60 The number of states of FA So we can define a new parameter: Definition stem(w) SK w = K w if stem(w) is defined otherwise K w = length of the shortest unrepeated suffix of w

61 The number of states of FA So we can define a new parameter: Definition stem(w) SK w = K w if stem(w) is defined otherwise K w = length of the shortest unrepeated suffix of w This allows us to derive a formula for FA(w) : Theorem FA(w) = w Sw L P w K w + SK w Sw L = number of left special factors of w P w = length of the shortest prefix of w which is not left special

62 The language L FA Let k = uv be the longest repeated suffix of w, where u is the longest prefix of k that is also prefix of w. Then v is the characteristic suffix of w. Any word w can be uniquely factorized as w = w v.

63 The language L FA Let k = uv be the longest repeated suffix of w, where u is the longest prefix of k that is also prefix of w. Then v is the characteristic suffix of w. Any word w can be uniquely factorized as w = w v. w u k k u v u v w

64 The language L FA Let k = uv be the longest repeated suffix of w, where u is the longest prefix of k that is also prefix of w. Then v is the characteristic suffix of w. Any word w can be uniquely factorized as w = w v. w u k k u v u v w Theorem The word w L FA if and only if its prefix w L SA.

65 Examples Example Let w = abaacaaa. The longest repeated suffix of w is v = aa, and the longest prefix of aa which is also a prefix of w is a. Then v = a and w = abaacaa. We have w / L FA, and w / L SA.

66 Examples Example Let w = abaacaaa. The longest repeated suffix of w is v = aa, and the longest prefix of aa which is also a prefix of w is a. Then v = a and w = abaacaa. We have w / L FA, and w / L SA. Example Let w = abaababbaa. The longest repeated suffix of w is v = baa. Then w = abaabab L SA, so w L FA.

67 Remarks Take Σ = 2. Definition A word w is trapezoidal if the factor complexity of w verifies f w (n + 1) f w (n) + 1 for every n 0. Definition A word w is rich if it contains w + 1 palindromic factors. We have: Proposition (A. de Luca 99) w balanced w trapezoidal Proposition (A. de Luca, A. Glen and L.Q. Zamboni 08) w trapezoidal w rich

68 Remarks Example Let w = abaababbaa. The longest repeated suffix of w is v = baa. Then w = abaabab L SA, so w L FA. Remarks: w is not balanced, since aa, bb Fact(w), w is not trapezoidal, since it has four factors of length 2, w is not rich, since it has only w palindromic factors: ε, a, b, aa, bb, aba, bab, abba, baab, abaaba.

69 Conclusions and future work We characterized the words in L SA and L FA, solving thus the problem of Berstel and Crochemore. In agenda: Characterize the balanced (or the trapezoidal, or the rich) words in L FA. Investigate LSP words. Apply an analogous approach with other data structures, e.g. suffix tree, suffix array, etc.

70 Thank you!

Combinatorics on Finite Words and Data Structures

Combinatorics on Finite Words and Data Structures Dipartimento di Informatica ed Applicazioni Università di Salerno (Italy) Laboratoire I3S - Université de Nice-Sophia Antipolis 13 March 2009 Combinatorics