Solving the String Statistics Problem in Time O(n log n)

Transcription

1 Alcom-FT Technicl Report Series ALCOMFT-TR Solving the String Sttistics Prolem in Time O(n log n) Gerth Stølting Brodl 1,,, Rune B. Lyngsø 3, Ann Östlin1,, nd Christin N. S. Pedersen 1,2, 1 BRICS, Deprtment of Computer Science, University of Arhus, Ny Munkegde, DK-8000 Århus C, Denmrk. E-mil: {gerth,nno,cstorm}@rics.dk 2 BiRC, University of Arhus, Ny Munkegde, DK-8000 Århus C, Denmrk. 3 Deprtment of Sttistics, Oxford University, Oxford OX1 3TG, UK. E-mil: lyngsoe@stts.ox.c.uk Astrct The string sttistics prolem consists of preprocessing string of length n such tht given query pttern of length m, the mximum numer of non-overlpping occurrences of the query pttern in the string cn e reported efficiently. Apostolico nd Preprt introduced the miniml ugmented suffix tree (MAST) s dt structure for the string sttistics prolem, nd showed how to construct the MAST in time O(nlog 2 n) nd how it supports queries in time O(m) for constnt sized lphets. A susequent theorem y Frenkel nd Simpson stting tht string hs t most liner numer of distinct squres implies tht the MAST requires spce O(n). In this pper we improve the construction time for the MAST to O(nlog n) y extending the lgorithm of Apostolico nd Preprt to exploit properties of efficient joining nd splitting of serch trees together with refined nlysis. 1 Introduction The string sttistics prolem consists of preprocessing string S of length n such tht given query pttern α of length m, the mximum numer of nonoverlpping occurrences of α in S cn e reported efficiently. Without preprocessing the mximum numer of non-overlpping occurrences of α in S cn e found in time O(n), y using liner time string mtching lgorithm to find ll occurrences of α in S, e.g. the lgorithm y Knuth, Morris, nd Prtt [14], nd then in greedy fshion from left-to-right compute the mximl numer of non-overlpping occurrences. Prtilly supported y the Future nd Emerging Technologies progrmme of the EU under contrct numer IST (ALCOM-FT). Supported y the Crlserg Foundtion (contrct numer ANS-0257/20). Bsic Reserch in Computer Science (BRICS), funded y the Dnish Ntionl Reserch Foundtion. Bioinformtics Reserch Center (BiRC), funded y Arhus University Reserch Fundtion.

2 Apostolico nd Preprt in [3] descried dt structure for the string sttistics prolem, the miniml ugmented suffix tree MAST(S), with preprocessing time O(n log 2 n) nd with query time O(m) for constnt sized lphets. In this pper we present n improved lgorithm for constructing MAST(S) with preprocessing time O(n log n), nd prove tht MAST(S) requires spce O(n), which follows from recent theorem of Frenkel nd Simpson [9]. The sic ide of the lgorithm of Apostolico nd Preprt nd our lgorithm for constructing MAST(S), is to perform trversl of the suffix tree of S while mintining the lef-lists of the nodes visited in pproprite dt structures (see Section 1.1 for definition detils). Trversing the suffix tree of string to construct nd exmine the lef-lists t ech node is generl technique for finding regulrities in string, e.g. for finding squres in string (or tndem repets) [2,17], for finding mximl qusi-periodic sustrings, i.e. sustrings tht cn e covered y shorter sustring, [1,6], nd for finding mximl pirs with ounded gp [4]. All these prolems cn e solved using this technique in time O(n log n). Other pplictions re listed y Gusfield in [10, Chpter 7]. A crucil component of our lgorithm is the representtion of lef list y collection of serch trees, such tht the lef-list of node in the suffix tree of S cn e constructed from the lef-lists of the children y efficient merging. Hwng nd Lin [13] descried how to optimlly merge two sorted lists of length n 1 nd n 2, where n 1 n 2, with O(n 1 log n1+n2 n 1 ) comprisons. Brown nd Trjn [7] descried how to chieve the sme numer of comprisons for merging two AVL-trees in time O(n 1 log n1+n2 n 1 ), nd Huddleston nd Mehlhorn [12] showed similr result for level-linked (2,4)-trees. In our lgorithm we will use slightly extended version of level-linked (2,4)-trees where ech element hs n ssocited weight. Due to lck of spce proofs hve een omitted. The omitted detils cn e found in [5]. 1.1 Preliminries Some of the terminology nd nottion used in the following origintes from [3], ut with minor modifictions. We let Σ denote finite lphet, nd for string S Σ we let S denote the length of S, S[i] the ith chrcter in S, for 1 i S, nd S[i.. j] = S[i]S[i + 1] S[j] the sustring of S from the ith to the jth chrcter, for 1 i j S. The suffix S[i.. S ] of S strting t position i will e denoted S[i.. ]. An integer p, for 1 p S, is denoted period of S if nd only if the suffix S[p ] of S is lso prefix of S, i.e. S[p ] = S[1.. S p]. The shortest period p of S is denoted the period of S, nd the string S is sid to e periodic if nd only if p S /2. A nonempty string S is squre, if S = αα for some string α. In the rest of this pper S denotes the input string with length n nd α sustring of S. A non-empty string α is sid to occur in S t position i if α = S[i.. i+ α 1] nd 1 i n α +1. E.g. in the string the sustring occurs t positions 1 nd 8. The mximum numer of nonoverlpping occurrences of string α in string S, is the mximum numer of

3 occurrences of α where no two occurrences overlp. E.g. the mximum numer of non-overlpping occurrences of in is three, since the occurrences t positions 1, 5 nd 9 do not overlp. The suffix tree ST(S) of the string S is the compressed trie storing ll suffixes of the string S where / Σ. Ech lef in ST(S) represents suffix S[i.. ] of S nd is nnotted with the index i. Ech edge in ST(S) is leled with nonempty sustring of S, represented y the strt nd end positions in S, such tht the pth from the root to the lef nnotted with index i spells the suffix S[i.. ]. We refer to the sustring of S spelled y the pth from the root to node v s the pth-lel of v nd denote it L(v). We refer to the set of indices stored t the leves of the sutree rooted t v s the lef-list of v nd denote it LL(v). Since LL(v) is exctly the set of strt positions i where L(v) is prefix of the suffix S[i.. ], we hve Fct 1 elow. Fct 1 If v is n internl node of ST(S), then LL(v) = c child of v LL(c), nd i LL(v) if nd only if L(v) occurs t position i in S. The prolem of constructing ST(S) hs een studied intensively nd severl lgorithms hve een developed which for constnt sized lphets cn construct ST(S) in time nd spce O( S ) [8,15,18,19]. For non-constnt lphet sizes the running time of the lgorithms ecome O( S log Σ ). In the following we let the height of tree T e denoted h(t) nd e defined s the mximum numer of edges in root-to-lef pth in T, nd let the size of T e denoted T nd e defined s the numer of leves of T. For node v in T we let T v denote the sutree of T rooted t node v, nd let v = T v nd h(v) = h(t v ). Finlly, for node v in inry tree we let smll(v) denote the child of v with smller size (ties re roken ritrrily). The sic ide of our lgorithm in Section 5 is to process the suffix tree of the input string ottom-up, such tht we t ech node v spend mortized time O( smll(v) log( v / smll(v) )). Lemm 1 then sttes tht the totl time ecomes O(n log n) [16, Exercise 35]. Lemm 1. Let T e inry tree with n leves. If for every internl node v, c v = smll(v) log( v / smll(v) ), nd for every lef v, c v = 0, then v T c v n log n. 2 The String Sttistics Prolem Given string S of length n nd pttern α of length m the following greedy lgorithm will compute the mximum numer of non-overlpping occurrences of α in S. Find ll occurrences of α in S y using n exct string mtching lgorithm. Choose the leftmost occurrence. Continue to choose greedily the leftmost occurrence not overlpping with ny so fr chosen occurrence. This greedy lgorithm will compute the mximum numer of occurrences of α in S in time O(n), since ll mtchings cn e found in time O(n), e.g. y the lgorithm y Knuth, Morris, nd Prtt [14].

4 11 9 v Figure1. To the left is the suffix tree ST(S) of the string S =. The node v hs pth-lel L(v) = nd lef-list LL(v) = {1, 3,6, 9}. To the right is the miniml ugmented suffix tree MAST(S) for the string S =. Numers in the internl nodes re the c-vlues In the string sttistics prolem we wnt to preprocess string S such tht queries of the following form re supported efficiently: Given query string α, wht is the mximum numer of non-overlpping occurrences of α in S? The mximum numer of non-overlpping occurrences of α is clled the c-vlue of α, denoted c(α). The preprocessing will e to compute the miniml ugmented suffix tree descried elow. Given the miniml ugmented suffix tree, string sttistics queries cn e nswered in time O(m). For ny sustring, α, of S there is exctly one pth from the root of ST(S) ending in node or on n edge of ST(S) spelling out the string α. This node or edge is clled the locus of α. In suffix tree ST(S) the numer of leves in the sutree elow the locus of α in ST(S) tells us the numer of occurrences of α in S. These occurrences my overlp, hence the suffix tree is not immeditely suitle for the string sttistics prolem. The miniml ugmented suffix tree for S, denoted MAST(S) cn e constructed from the suffix tree ST(S) s follows. A minimum numer of new uxiliry nodes re inserted into ST(S) in such wy tht the c-vlue for ll sustrings with locus on n edge (u, v), where u is the prent of v, hve c-vlue equl to c(l(v)), i.e. the c-vlue only chnges t internl nodes long pth from lef to the root. Ech internl node v in the ugmented tree is then leled y c(l(v)) to get the miniml ugmented suffix tree. Figure 1 shows the suffix tree nd the miniml ugmented suffix tree for the string. Frenkel nd Simpson in [9] prove tht string S contins less thn 2 S distinct squres, which implies the following lemm. Lemm 2. The miniml ugmented suffix tree for string S hs t most 3 S internl nodes.

5 Figure 2. The grouping of occurrences in string into chunks nd necklces. Occurrences re shown elow the string. Thick lines re occurrences in chunks. The grouping into chunks nd necklces is shown ove the string. Necklces re shown using dshed lines. Note tht necklce cn consist of single occurrence. It follows tht the spce needed to store MAST(S) is O(n). 3 String Properties The lemm elow gives chrcteriztion of how the occurrences of string α cn pper in S (proof omitted). Lemm 3. Let S e string nd α sustring of S. If the occurrences of α in S re t positions i 1 < < i k, then for ll 1 j < k either i j+1 i j = p or i j+1 i j > mx{ α p, p}, where p denotes the period of α. A consequence of Lemm 3 is tht if p α /2, then n occurrence of α in S t position i j cn only overlp with the occurrences t positions i j 1 nd i j+1. If p < α /2, then two consecutive occurrences i j nd i j+1, either stisfy i j+1 i j = p or i j+1 i j > α p. Corollry 1. If i j+1 i j α /2, then i j+1 i j = p where p is the period of α. Motivted y the ove oservtions we group the occurrences of α in S into chunks nd necklces. Let p denote the period of α. Chunks cn only pper if p < α /2. A chunk is mximl sequence of occurrences contining t lest two occurrences nd where ll consecutive occurrences hve distnce p. The remining occurrences re grouped into necklces. A necklce is mximl sequence of overlpping occurrences, i.e. only two consecutive occurrences overlp t given position nd the overlp of two occurrences is etween one nd p 1 positions long. Figure 2 shows the occurrences of the string in string of length 55 grouped into chunks nd necklces. By definition two necklces cnnot overlp, ut chunk cn overlp with nother chunk or necklce t oth ends. By Lemm 3 the overlp is t most p 1 positions. We now turn to the contriution of chunks nd necklces to the c-vlues. We first consider the cse where chunks nd necklces do not overlp. An isolted necklce or chunk is necklce or chunk tht does not overlp with other necklces nd chunks. Figure 3 gives n exmple of the contriution to the c-vlues y n isolted necklce nd chunk. More formlly, we hve the following lemm, which we stte without proof. Lemm 4. An isolted necklce of k occurrences of α contriutes to the c-vlue of α with k/2. An isolted chunk of k occurrences of α contriutes to the c-vlue of α with k/ α /p, where p is the period of α.

6 Figure 3. Exmples of the contriution to the c-vlues y n isolted necklce (left; α = nd the contriution is 5 = 9/2 ) nd n isolted chunk (right; α =, p = 2, nd the contriution is 3 = 8/ 5/2 ) Motivted y Lemm 4, we define the nominl contriution of necklce of k occurrences of α to e k/2 nd the nominl contriution of chunk of k occurrences of α to e k/ α /p. The nominl contriution of necklce or chunk of α s is the contriution to the c-vlue of α if the necklce or chunk pperes isolted. If the necklce of chunk does not pper isolted, i.e. it overlps with neighoring necklce or chunk, then its ctul contriution to the c-vlue of α is t most e one less thn its nominl contriution to the c-vlue of α. We define the excess of necklce of k occurrences to e (k 1) mod 2, nd the excess of chunk of k occurrences to e (k 1) mod α /p. The excess descries the numer of occurrences of α[1.. p] which re covered y the necklce or chunk, ut not covered y the mximl sequence of non-overlpping occurences. We group the chunks nd necklces into collection of chins C y the following two rules: 1. A chunk with excess t lest two is chin y itself. 2. A mximl sequence of overlpping necklces nd chunks with excess zero or one is chin. For chin c C we define # 0 (c) to e the numer of chunks nd necklces with excess zero in the chin. We re now redy to stte our min lemm enling the efficient computtion of the c-vlues. The lemm gives n lterntive to the chrcteriztion in [3, Proposition 2] (proof omitted). Lemm 5. The mximum numer of non-overlpping occurrences of α in S equls the sum of the nominl contriutions of ll necklces nd chunks minus c C # 0(c)/2. 4 Level-Linked (2,4)-Trees In this section we consider how to mintin set of sorted lists of elements s collection of level-linked (2,4)-trees where the elements re stored t the leves in sorted order from left-to-right, nd ech element cn hve n ssocited rel vlued weight. For detiled tretment of level-linked (2,4)-trees see [12] nd [16, Section III.5]. The opertions we consider supported re: NewTree(e, w): Cretes new tree T contining the element e with ssocited weight w.

7 Serch(p, e): Serch for the element e strting the serch t the lef of tree T tht p points to. Returns reference to the lef in T contining e or the immedite predecessor or successor of e. Insert(p, e, w): Cretes new lef contining the element e with ssocited weight w nd inserts the new lef immedite next to the lef pointed to y p in tree T, provided tht the sorted order is mintined. Delete(p): Deletes the lef nd element tht p is pointer to in tree T. Join(T 1, T 2 ): Conctentes two trees T 1 nd T 2 nd returns reference to the resulting tree. It is required tht ll elements in T 1 re smller thn the elements in T 2 w.r.t. the totl order. Split(T, e): Splits the tree T into two trees T 1 nd T 2, such tht e is lrger thn ll elements in T 1 nd smller thn or equl to ll elements in T 2. Returns references to the two trees T 1 nd T 2. Weight(T): Returns the sum of the weights of the elements in the tree T. Theorem 1 (Hoffmnn et l. [11, Section 3]). Level-linked (2,4)-trees support NewTree, Insert nd Delete in mortized constnt time, Serch in time O(log d) where d is the numer of elements in T etween e nd p, nd Join nd Split in mortized time O(log min{ T 1, T 2 }). To llow ech element to hve n ssocited weight we extend the construction from [11, Section 3] such tht we for ll nodes v in tree store the sum of the weights of the leves in the sutree T v, except for the nodes on the pths to the leftmost nd rightmost leves. These sums re strightforwrd to mintin while relncing (2,4)-tree under node splittings nd fusions, since the sum t node is the sum of the weights t the children of the node. For ech tree we lso store the totl weight of the tree. Theorem 2. Weighted level-linked (2,4)-trees support NewTree nd Weight in mortized constnt time, Insert nd Delete in mortized time O(log T ), Serch in time O(log d) where d is the numer of elements in T etween e nd p, nd Join nd Split in mortized time O(log min{ T 1, T 2 }). 5 The Algorithm In this section we descrie the lgorithm for constructing the miniml ugmented suffix tree for string S of length n. Algorithm ide: The lgorithm strts y constructing the suffix tree, ST(S), for S. The suffix tree is then ugmented with extr nodes nd c-vlues for ll nodes to get the miniml ugmented suffix tree, MAST(S), for S. The ugmenttion of ST(S) to MAST(S) strts t the leves nd the tree is processed in ottom-up fshion. At ech node v encountered on the wy up the tree the c-vlue for the pth-lel L(v) is dded to the tree, nd t ech edge new nodes nd their c-vlues re dded if there is chnge in the c-vlue long the edge. To e le to efficiently compute the c-vlues nd decide if new nodes should

8 e dded long edges the indices in the lef-list of v, LL(v), re stored in dt structure tht keeps trck of necklces, chunks, nd chins, s defined in Section 3. Dt structure: Let α e sustring of S. The dt structure D(α) is serch tree for the indices of the occurrences of α in S. The leves in D(α) re the leves in LL(v), where v is the node in ST(S) such tht the locus of α is the edge directly ove v or the node v. The serch tree, D(α), will e orgnized into three levels to keep trck of chins, chunks, nd necklces. The top level in the serch tree stores chins, the middle level chunks nd necklces, nd the ottom level occurrences. Top level: Unweighted (2,4)-tree (cf. Theorem 1) with the chins s leves. The leftmost indices in ech chin re the keys. Middle level: One weighted (2,4)-tree (cf. Theorem 2) for ech chin, with the chunks nd necklces s leves. The leftmost indices in ech chunk or necklce re the keys. The weight of lef is 1 if the excess of the chunk or necklce is zero, otherwise the weight is 0. The totl weight of tree on the middle level is # 0 (c), where c denotes the chin represented y the tree. Bottom level: One weighted (2,4)-tree for ech chunk nd necklce, with the occurrences in the chunk or necklce s the leves. The weight of lef is one. The totl weight of tree is the numer of occurrences in the chunk or the necklce. Together with ech of the 3-level serch trees, D(α), some vriles re stored. NCS(α) stores the sum of the nominl contriution for ll chunks nd necklces, ZS(α) stores the sum c C # 0(c)/2, where C is the set of chins. By Lemm 5 the mximum numer of non-overlpping occurrences of α is NCS(α) ZS(α). We lso store the totl numer of indices in D(α) nd list of ll chunks denoted CHUNKLIST(α). Finlly we store, p(α), which is the smllest difference etween the indices of two consecutive occurrences in D(α). Note tht, y Corollry 1, p(α) is the period of α if there is t lest one chunk. To mke our presenttion more redle we will sometimes refer to the tree for chin, chunk, or necklce just s the chin, chunk, or necklce. For the top level tree in D(α) we will use level-linked (2,4)-trees, ccording to Theorem 1, nd for the middle nd ottom level trees in D(α) we will use weighted level-linked (2,4)-trees, ccording to Theorem 2. In these trees predecessor nd successor queries re supported in constnt time. We denote y l(e) nd r(e) the indices to the left nd right of index e. To e le to check fst if there re overlps etween two consecutive trees on the middle nd ottom levels we store the first nd lst index in ech tree in the root of the tree. This cn esily e kept updted when the trees re joined nd split. We will now descrie how the suffix tree is processed nd how the dt structures re mintined during this process. Processing events: We wnt to process edges in the tree ottom-up, i.e. for decresing length of α, so tht new nodes re inserted if the c-vlue chnges long

9 the edge, the c-vlues for nodes re dded to the tree, nd the dt structure is kept updted. The following events cn cuse chnges in the c-vlue nd the chin, chunk, nd necklce structure. 1. Excess chnge: When α ecomes i p(α), for i = 2, 3, 4,... the excess nd nominl contriution of chunks chnges nd we hve to updte the dt structure nd possily dd node to the suffix tree. 2. Chunks ecome necklces: When α decreses nd ecomes 2p chunk degenertes into necklce. At this point we join ll overlpping chunks nd necklces into one necklce nd possily dd node to the suffix tree. 3. Necklce nd chin rek-up: When α decreses two consecutive occurrences t some point no longer overlp. The result is tht necklce or chin my split, nd we hve to updte the necklce nd chin structure nd possily dd node to the suffix tree. 4. Merging t internl nodes: At internl nodes in the tree the dt structures for the sutrees elow the node re merged into one dt structure nd the c-vlue for the node is dded to the tree. To keep trck of the events we use n event queue, denoted EQ, tht is common priority queue of events for the whole suffix tree. The priority of n event in EQ is equl to the length of the string α when the event hs to e processed. Events of type 1 nd 2 store pointer to ny lef in D(α). Events of type 3, i.e. tht two consecutive overlpping occurrences with index e 1 nd e 2, e 1 < e 2, terminte to overlp, store pointer to the lef e 1 in the suffix tree. For the lef e 1 in the suffix tree lso pointer to the event in EQ is stored. Events of type 4 stores pointer to the internl node in the suffix tree involved in the event. When the suffix tree is constructed ll events of type 4 re inserted into EQ. For node v in ST(S) the event hs priority L(v) nd stores pointer to v. The pointers re used to e le to decide which dt structure to updte. The priority queue EQ is implemented s tle with entries EQ[1]... EQ[ S ]. All events with priority x re stored in linked list in entry EQ[x]. Since the priorities of the events considered re monotonic decresing, it is sufficient to consider the entries of EQ in single scn strting t EQ[ S ]. The events re processed in order of the priority nd for events with the sme priority they re processed in the order s ove. Events of the sme type nd with the sme priority re processed in ritrry order. In the following we only look t one edge t the time when events of type 1, 2, nd 3 re tken cre of. Due to spce limittions mny lgorithmic detils re left out in the following. See [5] for detiled description of the lgorithm. 1. Excess chnge. The excess chnges for ll chunks t the sme time, nmely when α = i p(α) for i = 2, 3, 4,.... For ech chunk in CHUNKLIST(α) we will remove the chunk from D(α), recompute the excess nd nominl contriution sed on the numer of occurrences in the chunk, updte NCS(α), reinsert the chunk with the new excess nd finlly updte ZS(α). This is done s follows: First decide which chin ech chunk elongs to y serching the tree. Remove ech chunk from its chin y splitting the tree for the chin. Recompute the

10 excess for ech chunk nd reconstruct the tree. In the new tree the chin structure my hve chnged. Chunks for which the excess increses to two will e seprte chins, while chunks where the excess ecome less thn two my join two or three chins into one chin. NCS(α) nd ZS(α) re lwys kept updted during the processing of the event. If α = 2p(α) then insert n event of type 2 with priority 2p(α) into EQ, with pointer to ny lef in D(α). If α = ip(α) > 2p(α), then insert n event of type 1 with priority (i 1)p(α) into EQ, with pointer to ny lef in D(α). 2. Chunks ecome necklces. When α decreses to 2p ll chunks ecome necklces t the sme time. At this point ll chunks nd necklces tht overlp shll e joined into one necklce. Note tht ll chunks hve excess 0 or 1 when α = 2p nd since we first recompute the excess ll overlpping chunks nd necklces re in the sme chin. Hence, wht we hve to do is to join ll chunks nd necklces from left to right, in ech chin. This is done y first deciding for ech chunk which chin it elongs to. Next, for ech chin contining t lest one chunk, join ll chunks nd necklces from left to right. Updte NCS(α) nd ZS(α). 3. Necklce nd chin rek-up. When two consecutive occurrences of α with indices e 1 nd e 2 terminte to overlp this my cuse necklce or chin to rek up into two necklces or chins. If e 1 nd e 2 elong to the sme chin then the chin reks up in two chins. If e 1 nd e 2 elongs to the sme necklce then split oth the necklce nd the chin etween e 1 nd e 2. If e 1 nd e 2 elong to different necklces or chunks in the chin then split the chin etween the two sutrees including e 1 nd e 2 respectively. Updte NCS(α) nd ZS(α). 4. Merging t internl nodes. Let α e sustring such tht the locus of α is node v in the suffix tree. Then the lef-list, LL(v) for v is the union of the lef-lists for the sutrees elow v, hence t the nodes in the suffix tree the dt structures for the sutrees should e merged into one. We ssume tht the edges elow v re processed for α s descried ove. Let T 1,..., T t e the sutrees elow v in the suffix tree. We never merge more thn two dt structures t the time. If there re more thn two sutrees the merging is done in the following order: T = Merge(T, T i ), for i = 2,...,t, where T = T 1 to strt with. This cn lso e viewed s if the suffix tree is mde inry y replcing ll nodes of degree lrger thn 2 y inry tree with edges without lels. From now on we will descrie how to merge the dt structures for two sutrees. The merging will e done y inserting ll indices from the smller of the two lef-lists into the dt structure for the lrger one. Let T denote the 3-level serch tree to insert new indices in nd denote y e 1,..., e m the indices to insert, where e i < e i+1. The insertion is done y first splitting the tree T t ll positions e i for i = 1,...,m. The tree is then reconstructed from left to right t the sme time s the new indices re inserted in incresing order. Assume tht the tree is

11 reconstructed for ll indices, in oth trees, smller thn e i. The next step is to insert e i nd ll indices etween e i nd e i+1. This is done s follows: Check if the occurrence with index e i overlps ny occurrences to the left, i.e. n occurrence in the tree reconstructed so fr. Insert e i into the tree. If e i overlps with n occurrence lredy in the tree then check in wht wy this ffects the chin, chunk, nd necklce structure nd do the pproprite updtes. Do the corresponding check nd updtes when the tree to the right of e i (the tree for indices etween e i nd e i+1 ) is incorported, i.e. check if e i will cuse ny further chnges in the chin, chunk, nd necklce structure due to overlps to the right. Updte NCS(α) nd ZS(α). Every time, during the ove descried procedure, when two overlpping occurrences with indices e i nd e j, e i < e j, from different sutrees re encountered the event (e i, e j ) with priority e j e i is inserted into the event queue EQ nd the previous event, if ny, with pointer to e i is removed from EQ. Updte p(α) to e j e i if this is smller thn the current p(α) vlue. If α > 2p(α) then insert n event of type 1 with priority α /p(α) p(α) into EQ, with pointer to ny lef in D(α). 6 Anlysis Theorem 3. The miniml ugmented suffix tree, MAST(S), for string S of length n cn e constructed in time O(n log n) nd spce O(n). In the full version of the pper [5] we show tht the running time of the lgorithm in Section 5 is O(n log n). Here we only stte the min steps of the proof. The proof uses n mortiztion rgument, llowing ech edge to e processed in mortized constnt time, nd ech inry merge t node (in the inry version) of ST(S) of two lef-lists of sizes n 1 nd n 2, with n 1 n 2, in mortized time O(n 2 log n1+n2 n 2 ). From Lemm 1 it then follows tht the totl time for processing the internl nodes nd edges of ST(S) is O(n log n). Using Theorem 1 nd 2 we cn prove tht: Processing events of types 1 nd 2 tke time O(m log LL(v) m ), where m = CHUNKLIST(α). Processing n event of type 3 tkes time O(log c ), where c is the chin eing split. An event of type 4 hs processing time O(n 1 log n1+n2 n 1 ). Let v e node in the suffix tree nd let α e string with locus v or locus on the edge immeditely ove v. For the dt structure D(α) we define potentil Φ(D(α)). Let C e the set of chins stored in D(α), nd for chin c let c denote the numer of occurrences of α in c. We define the potentil of D(α) y Φ(D(α)) = Φ 1 (α) + Φ 2 (α) + c C Φ 3(c), where the rôle of Φ 1, Φ 2, nd Φ 3 is to ccount for the potentil required to e le to process events of type 1, 2, nd 3 respectively. For chunk, with leftmost occurrence of α t position i, consider the sustring S[i.. j] with mximl j nd S[i.. j] hving period p, where p = p(α) is the period of α. We denote the chunk green if nd only if α mod p j i+1 mod p. Otherwise the chunk is red. Let k denote the numer of chunks in D(α) nd let g denote the numer of green chunks in D(α).

12 We define Φ 1 (α) = 7g log v e g, Φ 2(α) = k log v e k, nd Φ 3(c) = 2 c log c 2, with the exceptions tht Φ 1 (α) = 0 if g = 0, nd Φ 2 (α) = 0 if k = 0. We cn prove tht processing events of type 1, 2, nd 3 relese sufficient potentil to py for the processing, while processing n event of type 4 increses the potentil y O(n 1 log n1+n2 n 1 ). By Lemm 1 the totl mortized time for hndling ll events is O(n log n). References 1. A. Apostolico nd A. Ehrenfeucht. Efficient detection of qusiperiodicities in strings. Theoreticl Computer Science, 119: , A. Apostolico nd F. P. Preprt. Optiml off-line detection of repetitions in string. Theoreticl Computer Science, 22: , A. Apostolico nd F. P. Preprt. Dt structures nd lgorithms for the string sttistics prolem. Algorithmic, 15: , G. S. Brodl, R. Lyngsø, C. N. S. Pedersen, nd J. Stoye. Finding mximl pirs with ounded gp. Journl of Discrete Algorithms, Specil Issue of Mtching Ptterns, 1(1):77 104, G. S. Brodl, R. B. Lyngsø, A. Östlin, nd C. N. S. Pedersen. Solving the string sttistics prolem in time O(n log n). Technicl Report RS-02-13, BRICS, Deprtment of Computer Science, University of Arhus, G. S. Brodl nd C. N. S. Pedersen. Finding mximl qusiperiodicities in strings. In Proc. 11th Comintoril Pttern Mtching, volume 1848 of Lecture Notes in Computer Science, pges Springer Verlg, Berlin, M. R. Brown nd R. E. Trjn. A fst merging lgorithm. Journl of the ACM, 26(2): , M. Frch. Optiml suffix tree construction with lrge lphets. In Proc. 38th Ann. Symp. on Foundtions of Computer Science (FOCS), pges , A. S. Frenkel nd J. Simpson. How mny squres cn string contin? Journl of Comintoril Theory, Series A, 82(1): , D. Gusfield. Algorithms on Strings, Trees nd Sequences: Computer Science nd Computtionl Biology. Cmridge University Press, K. Hoffmnn, K. Mehlhorn, P. Rosenstiehl, nd R. E. Trjn. Sorting Jordn sequences in liner time using level-linked serch trees. Informtion nd Control, 86(1-3): , S. Huddleston nd K. Mehlhorn. A new dt structure for representing sorted lists. Act Informtic, 17: , F. K. Hwng nd S. Lin. A simple lgorithm for merging two disjoint linerly ordered sets. SIAM Journl of Computing, 1(1):31 39, D. E. Knuth, J. H. Morris, nd V. R. Prtt. Fst pttern mtching in strings. SIAM Journl of Computing, 6: , E. M. McCreight. A spce-economicl suffix tree construction lgorithm. Journl of the ACM, 23(2): , K. Mehlhorn. Sorting nd Serching, volume 1 of Dt Structures nd Algorithms. Springer Verlg, Berlin, J. Stoye nd D. Gusfield. Simple nd flexile detection of contiguous repets using suffix tree. Theoreticl Computer Science, 270: , E. Ukkonen. On-line construction of suffix trees. Algorithmic, 14: , P. Weiner. Liner pttern mtching lgorithms. In Proc. 14th Symposium on Switching nd Automt Theory, pges 1 11, 1973.