Negative Selection Algorithms on Strings with Efficient Training and Linear-Time Classification

Transcription

1 Negtive Selection Algorithms on Strings with Efficient Trining nd Liner-Time Clssifiction Michel Elerfeld, Johnnes Textor Institut für Theoretische Informtik, Universität zu Lüeck, Lüeck, Germny Astrct A string-sed negtive selection lgorithm is n immune-inspired clssifier tht infers iprtitioning of string spce Σ l from trining set S contining smples from only one of the prtitions. The lgorithm genertes set of ptterns, clled detectors, to cover regions of the string spce contining none of the trining smples. These ptterns re then used for clssifiction. A mjor prolem with ll existing implementtions of this pproch is tht the detector genertion step suffers from exponentil worst-cse time complexity. Hence, reserchers hve found negtive selection to e of limited use for rel-world prolems such s network intrusion detection. Here we show tht for the two most widely used kinds of detectors, the r-chunk nd r-contiguous detectors sed on prtil mtching to sustrings of length r, negtive selection cn e implemented efficiently y voiding generting detectors ltogether: For ech detector type, trining set S Σ l nd prmeter r l one cn construct n utomton with n cceptnce ehviour tht is equivlent to the lgorithm s clssifiction outcome. The resulting runtime is O( S lr Σ ) for constructing the utomton in the trining phse nd O(l) for clssifying string. Key words: negtive selection, r-chunk detectors, r-contiguous detectors, rtificil immune systems, nomly detection 1. Introduction The dptive immune system successfully protects verterte species, including us humn eings, from ecoming extinguished y pthogens. According to current textook immunology, the immune system ccomplishes this without ctully knowing wht pthogen is. Insted, it is trined during infncy to tolerte the tissues, cells nd molecules tht re norml components of its host orgnism the self nd to ttck everything else the nonself. While nonself includes potentilly dngerous things such s viruses, cteri nd fungi, this implies tht enign intruders such s donted kidney or liver re lso ttcked y the immune system. The prdigm of self-nonself-discrimintion is nturl source of inspirtion for computer security: Computer systems nd networks re lso continuously ttcked y worms nd other mlwre, nd computer progrm tht discrimintes with perfect ccurcy etween enign nd mlign softwre cnnot exist. Thus, cn we enefit from trnsferring the immune system s nice hck [17] to the computer security domin? A populr pproch to designing such computer immune systems is to mimic how the immune system s T cells re generted nd trined to detect nonself entities. This process is known s negtive selection [14, 15]: T cell receptors re generted y rndom ssemly of gene frgments. In n orgn clled the thymus, neworn T cells re exposed to proteins from self. Every cell whose receptor mtches self protein is destroyed. Only the cells tht survive negtive selection leve from the thymus nd strt to continuously circulte through the orgnism, screening for nonself entities. A negtive selection lgorithm is essentilly n strction of this process. The negtive selection lgorithms tht we consider in this pper re inry clssifiers operting on string spce Σ l. The clssifiction prolem is posed s follows (Figure 1): Σ l is ssumed to e pre-prtitioned in two pirwise disjoint susets S (self) nd N (nonself). The strings cn represent, for exmple, dt pckets in computer network Emil ddresses: elerfeld@tcs.uni-lueeck.de (Michel Elerfeld), textor@tcs.uni-lueeck.de (Johnnes Textor) Preprint sumitted to Theoreticl Computer Science - C August 24, 2010

2 true iprtitioning trining phse inferred iprtitioning S N nonself region self region detector d D strings mtched y d smple s S inferred nonself inferred self Figure 1: The clssifiction prolem solved y negtive selection lgorithm. The string universe Σ l is preprtitioned in two regions S (self) nd N (nonself). The clssifier is given trining set S S (lrge dots) nd genertes detector set D (smll dots) to cover regions of the universe contining none of the trining exmples (circles). This detector set induces clssifiction oundry tht pproximtes the prtitions S nd N. [3] or sequences of system clls from UNIX processes [13], where the self nd nonself prtitions would correspond to norml nd nomlous ehviour, respectively. The lgorithm is given smple S S of self strings, clled self-set, nd set M Σ l of strings to clssify, clled monitor set. It then genertes set D of ptterns clled detectors. In nlogy to the T cells in the immune system, this is typiclly done y generting the detectors rndomly nd discrding those tht mtch ny string in the self-set. Consequently, ech string m M is clssified y lelling m s non-self if it is mtched y ny detector, nd self otherwise. In prticulr, m is never leled non-self if it lso occurs in the self-set. From roder mchine lerning perspective, negtive selection is usully descried s n nomly detection technique [16, 6]. The following two importnt properties distinguish negtive selection from mny well-known clssifiers: (1) The trining dt consists of exmples from only one clss. Other techniques with this property include clssifiers sed on kernel density estimtion [4, 20] nd the one-clss support vector mchine [22]. (2) Clssifiction is sed on negtive representtion of trining dt, typiclly on short sustrings (r-grms) tht do not occur in the self-set. While positive representtions such s the r-grm frequency distriution used e.g. for identifiction of lnguge [11] nd text ctegoriztion [5] re more common in the mchine lerning domin, similr negtive representtions hve een studied in string theory. For exmple, certin sets of non-occurring sustrings (foridden words) cn e used to descrie the complexity of lnguge [8] Contriution of this pper This pper presents two lgorithms tht implement negtive selection with r-chunk nd r-contiguous detectors y generting compressed representtions of the respective detector sets, from which utomt re constructed tht simulte the clssifiction outcome through their cceptnce ehviour. Both lgorithms use time O( S lr Σ ) to construct n utomton for given self-set S nd prmeter r, which is equivlent to the trining phse of the simulted negtive selection lgorithm. Upon construction, the utomton clssifies ech string in liner time O(l). This improves upon the exponentil worst-cse complexity of existing lgorithms, nd thus removes one mjor ostcle for pplying negtive selection to rel-world prolems [28, 23, 27]. In comprison to our preliminry conference version [12], the lgorithms presented in this pper re sed on prefix trees insted of ptterns. This reduces the overll runtime significntly (Tle 1), generlizes to higher lphets, nd llows for simpler nd more concise presenttion. In ddition to the clssifiction itself, the utomt cn lso e used to efficiently count the detectors nd, if necessry, enumerte them explicitly. The r-chunk nd r-contiguous detectors considered here re mong the most common ones in the rtificil immune systems literture [16]: (1) An r-contiguous detector is string of length l nd mtches ll strings to which it is identicl in t lest r contiguous positions. (2) An r-chunk detector is string of length r (or r-grm) with position 2

3 index nd mtches ll strings in which the r-grm occurs t tht position. Figure 2 shows n exmple self-set S {, } 5 long with the complete sets of 3-chunk nd 3-contiguous detectors tht do not mtch ny string in S, s well s the prtitioning of {, } 5 induced y these detector sets. The r-contiguous detectors re directly sed on model of ntigen recognition y T cell receptors [21, 14], nd r-chunk detectors were lter introduced to chieve etter results on dt where different regions of the input strings hve highly different menings, such s network dt pckets [3]. (,1) (,1) (,2) (,2) (,3) (,3) (,1) (,2) (,2) (,3) self-set S 3-chunk detectors iprtitioning of {, } 5 3-contiguous detectors iprtitioning of {, } 5 Figure 2: An exmple self-set S {, } 5 long with ll 3-chunk detectors nd 3-contiguous detectors tht do not mtch ny string in S is shown. For oth detector types, the iprtitionings of the shpe spce {, } 5 re illustrted with strings tht re clssified s nonself hving gry ckground nd strings tht re clssified s self hving white ckground. Bold strings re memers of the self-set. The generliztion region of the negtive selection clssifier consists of the strings tht re clssified s self lthough they do not occur in the self-set. These strings re lso clled holes in the negtive selection literture [16, 25] Relted Work on String-Bsed Negtive Selection The question whether negtive selection with r-contiguous nd r-chunk detectors cn e implemented in polynomil worst-cse time ws open for severl yers. The complexity issues cused y the vertim strction of negtive selection s performed y the immune system re two-fold: On one hnd, if the self prtition is only smll frction of Σ l, then there is n exponentil numer of potentil detectors, nd it is uncler how mny of these hve to e generted to chieve n cceptle detection rte. The erly work of D heseleer nd others [10, 9] ddressed these prolems y proving lower ounds on the numer of required detectors, nd presenting lgorithms tht generte detectors y structured exhustive serch. However, these lgorithms still hve runtime exponentil in r. Similr lgorithms nd heuristics were lter proposed y Wierzchoń [30], Ayr et l. [2], nd Stior et l. [26]. In n effort to clrify the computtionl complexity of negtive selection, Stior nd coworkers studied the ssocited decision prolem [24, 25]: Given self-set S, cn n r-contiguous detector e generted tht does not mtch ny string in S? It ws recently suggested tht this decision prolem might e NP-complete [28], lthough completeness proof ws not shown. The ongoing difficulties led some in the field to conclude tht negtive selection is computtionlly too expensive for rel-world dtsets [23, 1]. This issue ws settled y the preliminry version of the present pper [12]. Most recently, Liśkiewicz nd Textor discussed the ide of negtive selection without explicit detector genertion from lerning theoreticl perspective [19] Orgniztion of This Pper We strt out y defining the forml underpinnings of our lgorithms in the upcoming section. Afterwrds, in Section 3, we sketch the construction of n utomton consisting of prefix trees nd filure links tht cn e used to simulte negtive selection with r-chunk detectors. This rther strightforwrd construction is used s sis for the more involved one in Section 4, where we trnsform the utomton into one tht llows liner-time clssifiction with respect to r-contiguous detectors. 2. Preliminries In this section, we define the forml ckground of our work. First we review some sic terms relted to strings nd pttern mtching techniques like utomt. Then we define r-chunk detectors, r-contiguous detectors, nd the corresponding clssifiction pproches. 3

4 r-chunk detector- symptotic runtime sed lgorithms trining phse clssifiction phse Stior et l. [26] (2 r + S )(l r + 1) D l Elerfeld, Textor [12] S (l r + 1)r 2 S l 2 r Present pper S lr l r-contiguous detector- symptotic runtime sed lgorithms trining phse clssifiction phse D heseleer et l. [10] (liner) (2 r + S )(l r) D l D heseleer et l. [10] (greedy) 2 r S (l r) D l Wierzchón [30] 2 r ( D (l r) + S ) D l Elerfeld, Textor [12] S 3 l 3 r 3 S 2 l 3 r 3 Present pper S lr l Tle 1: Comprison of our results with the runtimes of previously pulished lgorithms. All runtimes re given for inry lphet ( Σ = 2) since not ll lgorithms re pplicle to ritrry lphets. The prmeter D, the desired numer of detectors, is only pplicle to lgorithms tht generte detectors explicitly our lgorithms produce the results tht would e otined with the mximl numer of generted detectors Strings, Sustrings nd Lnguges An lphet Σ is nonempty nd finite set of symols. A string s Σ is sequence of symols from Σ, nd its length is denoted y s. Given n index i {1,..., s }, then s[i] is the symol t position i in s. Given two indices i nd j, whenever j i, then s[i... j] is the sustring of s with length j i + 1 tht strts t position i nd whenever j < i, then s[i... j] is the empty string. If i = 1, then s[i... j] is prefix of s nd, if j = s, then s[i... j] is suffix of s. Given string s Σ l nd nother string d Σ j with 1 j l nd n index i {1,..., l j + 1}, we sy tht d occurs in s t position i if s[i... i + j 1] = d. A set of strings S Σ is clled lnguge. For two indices i nd j, we define S [i... j] = {s[i... j] s S }. We sy tht S voids string d t position i if d occurs in no s S t position i. Alterntively, we sy tht S voids the tuple (d, i) Prefix Trees, Prefix DAGs, nd Automt A prefix tree T such s is rooted directed tree with edge lels from Σ where for ll σ Σ, every node hs t most one outgoing edge leled with σ. For string s, we write s T if there is pth (which my contin cycles) from the root of T to lef such tht s is the conctention of the lels on this pth. The lnguge L(T) descried y T is defined s the set of ll strings with prefix s T. For exmple, for T ove we hve L(T) since T nd L(T) since no prefix of lies in T. A prefix dg D such s is directed cyclic grph with edge lels from Σ, where gin for ll σ Σ, every node hs t most one outgoing edge leled with σ. In nlogy to prefix trees, we will use the terms root nd lef to refer to node without incoming nd outgoing edges, respectively. We write s D if there is root node n r nd lef node n l in D with pth from n r to n l tht is leled y s. Given n D, the lnguge L(D, n) contins ll strings tht hve prefix tht lels pth from n to some lef. For instnce, if D is the dg ove nd n is its upper left node, then L(D, n) consists of ll strings strting with. Moreover, we define L(D) = n is root of D L(D, n). We will construct finite utomt to decide the memership of strings in lnguges. Formlly, finite utomton is tuple M = (Q, q i, Q, Σ, ), where Q is set of sttes with distinguished initil stte q i Q, Q Q the set of ccepting sttes, Σ the lphet of M, nd Q Σ Q the trnsition reltion. Furthermore, we ssume tht the trnsition reltion is unmiguous: for every q Q nd every σ Σ there is t most one q Q with (q, σ, q ). It is common to represent the trnsition reltion s grph with nodes Q (with the initil stte nd the ccepting sttes highlighted properly) nd leled edges (with σ-leled edge from q to q if (q, σ, q ).) An utomton M is sid to ccept string s if its grph contins pth from q i to some q Q whose conctented edge lels equl s. The lnguge L(M) contins ll strings ccepted y M. Note tht every prefix dg D cn e turned into 4

5 finite utomton M with L(D) = L(M). For more detiled discussion of utomt-sed string processing, we refer to the textook of Crochemore, Hncrt nd Lecroq [7] Detectors nd Self-Nonself-Discrimintion We fix n lphet Σ, string length l, self-set S Σ l, nd mtching prmeter r {1,..., l}. Definition 2.1 (r-chunk detector). An r-chunk detector (d, i) is tuple of string d Σ r nd n index i {1,..., l r + 1}. It mtches string s if d occurs in s t position i. The set of r-chunk detectors for S, denoted y chunk(s, r), contins exctly the r-chunk detectors (d, i) tht do not mtch ny string in S. Let m Σ l. The string m is nonself with respect to S nd r-chunk detectors if m mtches n r-chunk detector from chunk(s, r) nd self, otherwise. The set chunk-nonself(s, r) contins exctly the strings of length l over Σ tht re nonself with respect to S nd r-chunk detectors. Definition 2.2 (r-contiguous detector). An r-contiguous detector is string d Σ l. It mtches string s Σ l if there is n index i {1,..., l r + 1} where d[i... i + r 1] occurs in s. Similrly to the chunk detector cse, we define the set of r-contiguous detectors for S nd r, cont(s, r), s the set of ll r-contiguous detectors tht do not mtch ny string in S. Let m Σ l. The string m is nonself with respect to S nd r-contiguous detectors if m mtches n r-contiguous detector from cont(s, r) nd self, otherwise. The set cont-nonself(s, r) contins exctly the strings tht re nonself with respect to S nd r-contiguous detectors. Figure 2 from the introduction shows n exmple of self-set S, the corresponding detector sets chunk(s, 3) nd cont(s, 3), nd the corresponding prtitions of the shpe spce into self nd nonself. 3. Negtive Selection with Chunk Detectors In this section, we discuss how to construct utomt for chunk-nonself(s, r). The construction is rther strightforwrd comintion of two stndrd string processing tools: prefix trees nd filure links. We present the construction here for ske of completeness nd ecuse we use it s uilding lock for the more intricte one in the next section. Theorem 3.1. There exists n lgorithm tht, given ny S Σ l nd r {1,..., l}, constructs finite utomton M with L(M) Σ l = chunk-nonself(s, r) in time O( S lr Σ ). Proof. By definition, string m Σ l lies in the set chunk-nonself(s, r) exctly if S voids (m[i... i + r 1], i) for some index i {1,..., l r + 1}. Therefore, to clssify m in time O(lr), it suffices to construct, for every position i {1,..., l r + 1} independently, prefix tree T i with L(T i ) Σ r = Σ r \ S [i,..., i + r 1]. The prefix tree T i cn e constructed s follows: Strt with n empty prefix tree nd insert every s S [i,..., i + r 1] into it. Next, for every non-lef node n nd every σ Σ where no edge with lel σ strts t n, crete new lef n nd n edge (n, n ) leled with σ. Finlly, delete every node from which none of the newly creted leves is rechle. For the resulting prefix tree we hve L(T i ) Σ r = Σ r \ S [i,..., i + r 1]. It is redily seen tht for every s T i, ll of its prefixes s with s < s occur in S t position i. To enle clssifiction in time O(l), we construct n utomton y inserting filure links etween the prefix trees of djcent levels, similr to the well-known lgorithm of Knuth, Morris nd Prtt [18]. Briefly, the ide of our filure link method is s follows: If mismtch occurs in prefix tree T i t position k, then we need not restrt from the root of tree T i+1, ut cn go directly to the node in T i+1 tht corresponds to the lst k 1 symols red. By inserting the filure links from right to left, turning the prefix trees into prefix dg, we cn inductively ensure tht either such node exists or there is no mtch t ll. We strt y letting D e the disjoint union of T 1,..., T l r+1. Then we process the levels from i = l r down to 1 itertively s follows: Consider every node n from T i nd every symol σ Σ where T i hs no outgoing edge with lel σ. Let s e the string on the pth from the root of T i to n. Let s = sσ nd let n e the end node of the pth from the root of T i+1 tht is leled y s [2... s ]. If this n exists, we insert n edge from n to n with lel σ. By induction one cn show tht fter every itertion i we hve L(T i ) Σ l i+1 = chunk-nonself(s [i,..., l], r). Finlly, we 5

6 q i,,,,,,, Figure 3: The constructed utomton M with L(M) {, } 5 = chunk-nonself(s, 3) where S is the self-set from Figure 2. The solid lines re from the prefix trees T i, the dshed lines re filure links. turn D into finite utomton with the climed property y mking ll leves ccepting sttes with self-loops for ll σ Σ nd setting the initil stte to the root of T 1. An exmple of this construction is shown in Figure 3. Ech prefix tree T i cn e constructed in time O( S r Σ ). The filure links etween ech pir of djcent levels i nd i + 1 cn e inserted in time S r Σ y simultneous recursive trversl of T i nd T i+1. Since the numer of levels is l r + 1, we otin the climed runtime. 4. Negtive Selection with Contiguous Detectors In this section, we show how to efficiently construct utomt for the lnguges cont(s, r) nd cont-nonself(s, r), respectively. We first discuss the construction of n utomton for cont(s, r), which will prove the following theorem: Theorem 4.1. There exists n lgorithm tht, given ny S Σ l nd r {1,..., l}, constructs finite utomton M with L(M) Σ l = cont(s, r) in time O( S lr Σ ). The construction in this section is more complex thn the one in the previous section since, in order to ccept cont(s, r), it does not suffice to determine non-occuring length-r sustrings for the levels independently. Insted, we need to determine non-occuring sustrings tht cn e extended y non-occuring sustrings from other levels to form strings of length l the r-contiguous detectors. Let S Σ l, d Σ l, r {1,..., l}, nd (d, i) Σ r {1,..., l r + 1}. The string d is n (S, r)-voiding rightcompletion of (d, i) if (1) d occurs in d t position i nd S voids (d, i), nd (2) for ll j {i + 1,..., l r + 1}, there is string d Σ r such tht d occurs in d t position j nd S voids (d, j). If property (2) is phrsed with j rnging from 1 to i 1, then d is n (S, r)-voiding left-completion of (d, i). With this definition we hve d cont(s, r) iff there exists (d, i) Σ r {1,..., l r + 1} such tht d is oth n (S, r)-voiding left-completion nd n (S, r)-voiding right-completion of (d, i). To prove Theorem 4.1, we first prove the following Lemm: Lemm 4.2. There exists n lgorithm tht, given ny S Σ l nd r {1,..., l}, constructs prefix dg D with roots ρ 1,..., ρ l r+1 such tht L(D, ρ i ) Σ l i+1 = cont(s, r)[i... l] for every i {1,..., l r + 1} in time O( S lr Σ ). Proof. The construction of D is done in four phses, presented nd discussed in the next four prgrphs. While the following proof text explins the sic ides nd their correctness, the detiled computtionl steps re shown y the pseudocode in Figure 4, which lso provides n implementtion lueprint. For ll i {1,..., l r + 1}, let T i e prefix trees with L(T i ) Σ r = Σ r \ S [i... i + r 1] from the proof of Theorem 3.1; y definition we know tht every r-contiguous detector contins string t position i tht occurs in T i. However, there re still strings in T i tht do not occur in ny r-contiguous detector t position i. Those re precisely the strings tht hve no (S, r)-voiding left-completion or no (S, r)-voiding right-completion. We trim the trees T 1,...,T l r+1 to otin new trees T1 R,...,T l r+1 R where every T i R contins exctly the strings from T i tht hve (S, r)-voiding right-completions. This holds directly for ll strings from the rightmost level, so Tl r+1 R = T l r+1. We trim the other trees in right-to-left pss from i = l r down to 1. Ech time we initilize Ti R to 6

7 Procedure construct-detector-dg(s, r) 1 for i = 1 to l r + 1 do construct prefix trees 2 T i prefix tree with L(T i ) Σ r = Σ r \ S [i... i + r 1] 3 T R l r+1 T l r+1 trim the trees in right-to-left pss 4 for i = l r down to 1 do 5 T R i empty prefix tree 6 for ech string s T i do 7 if there exists s T R i+1 such tht s[2... s ] is prefix of s then insert s into T R i 8 T L 1 T R 1 trim the trees in left-to-right pss 9 for i = 2 to l r + 1 do 10 T L i empty prefix tree 11 for ech string s T R i do if there exists s Ti 1 L such tht s [2... s ] is prefix of s then insert s into Ti L D l r+1 Tl r+1 L weve the trees together into prefix dg 14 for i = l r down to 1 do 15 D i disjoint union of D i+1 nd Ti L; ρ i root of Ti L 16 for ech string s Ti L do 17 (n, n, σ) lst edge on the s-pth from ρ i in Ti L, nd its lel 18 n end node of the s[2... s ]-pth from ρ i+1 in D i+1 19 delete edge (n, n, σ) from D i nd insert edge (n, n, σ) 20 output D D 1 finl prefix dg with roots ρ 1,...,ρ l r+1 Figure 4: For given self-set S Σ l nd numer r {1,..., l}, this procedure constructs prefix dg D with roots ρ 1,..., ρ l r+1 such tht L(D, ρ i ) Σ l i+1 = cont(s, r)[i... l] for every i {1,..., l r + 1} in time O( S lr Σ ). Thus, in prticulr we hve L(D, ρ 1 ) Σ l = cont(s, r). e the empty tree. Then we consider every s T i nd insert it into Ti R if s[2... s ] is prefix of some s Ti+1 R. There re two potentil resons for string s T i not to e contined in Ti R : (1) It my e the cse tht lredy no string from T i+1 strts with s[2... s ], which, in turn, implies tht proper prefix of s[2... s ] lies in T i+1. Since s T i, ll of its proper prefixes occur in S t position i nd, thus, ll proper prefixes of s[2... s ] occur in S t position i + 1. This is contrdiction nd, thus, this cse cn never occur. (2) The second possiility is tht there is string tht strts with s[2... s ] in T i+1, ut not in Ti+1 R. By induction, one cn prove tht this is due to the fct tht (s[2... s ], i + 1) hs no (S, r)-voiding right-completion nd, therefore, lso (s, i) hs none. Next, we construct set of trees T1 L... T l r+1 L contining only the strings tht hve oth left- nd right-completions y n nlogous left-to-right pss. Thus, L(Ti L) Σr = cont(s, r)[i,..., i + r 1] holds. Finlly, we weve the trees together into prefix dg s follows: For the rightmost level i = l r + 1, we set D l r+1 = Tl r+1 L, since y construction L(T l r+1 L ) Σr = cont(s, r)[l r l]. Now we prove the lemm y decresing induction on i going from i = l r down to 1. For the induction step, suppose we hve prefix dg D i+1 with L(D i+1 ) Σ l i = cont(s, r)[i l]. For ll s Ti L, let n denote the corresponding lef in T i L. Let n denote the end node on the pth from the root of Ti+1 L with lel s, which exists y induction ssumption ecuse s[2... s ] is prefix of some d cont(s, r)[i l]. Crete new edge from the prent of n to n nd delete the lef n long with ll nodes nd edges from which only n cn e reched. After ll leves hve een iterted through, let D i e the resulting grph. Let d cont(s, r)[i... l]. Then d strts with prefix from Ti L nd, thus, d[2... d ] L(D i+1 ). Hence, d L (D i ) y construction. Conversely, let d L (D i ) with d = l i+1. Then d strts with nonempty prefix tht hs oth n (S, r)-voiding right-completion nd n (S, r)-voiding left-completion. Furthermore, d[2... d ] L (D i+1 ). Hence d cont(s, r)[i... l]. Now y setting D = D 1 we otin dg with the properties climed y the Lemm. The runtime of the construction cn e esily determined from the pseudocode given in Figure 4. As stted in Theorem 3.1, constructing the prefix trees in lines 1 nd 2 tkes time O( S lr Σ ). The inner loops in the right-to-left psses in lines 3 7 nd s well s in the left-to-right pss in lines 8 12 cn e implemented y simultneous recursion through the trees on djcent levels. This yields worst-cse runtime of O( S lr Σ ) for ech of the psses nd, hence, of the overll lgorithm. 7

8 Proof (Theorem 4.1). Let D with roots ρ 1,...,ρ l r+1 e the prefix dg from Lemm 4.2. We trnsform D into n utomton M = (Q, q i, Q, Σ, ) with L(M) Σ l = cont(s, r): For every lef n of D nd σ Σ we ppend self-loop with lel σ to n. Then Q nd re the set of nodes nd set of leled edges, respectively, Q contins ll former lefs, nd q = ρ 1. Figure 5 shows n exmple of such n utomton. q i, Figure 5: The constructed utomton M with L(M) {, } 5 = cont(s, 3) where S is the self-set from Figure 2. In ddition to descriing the lnguge cont(s, r), the prefix dg D cn lredy e used to clssify string m Σ l in time O(lr): Consider every position i {1,..., l r + 1} nd test whether m[i... i + r 1] L (D, ρ i ). If there exists position where this is true, then m is non-self nd self, otherwise. At the end of this section we will speed up the clssifiction to time O(l). But first let us show how to use the prefix dg D for counting the numer of detectors. Corollry 4.3. There exists n lgorithm tht, given S Σ l nd r {1,..., l}, outputs cont(s, r) in time O( S lr Σ ). Proof. Our tsk is simply to count the numer of strings of length l in L(D, ρ 1 ), where D is the prefix dg constructed in Lemm 4.2. First, for ech node n D, compute the numer of different pths leding from ρ 1 to n. Denote this quntity y P[n], nd let δ(ρ 1, n) denote the distnce etween ρ 1 nd n in D (note tht y construction, ll pths leding from ρ 1 to n in D hve the sme length). Then cont(s, r) = n is lef of D P[n] Σ l δ(ρ1,n). Since D is cyclic, computing P[n] cn e done y dynmic progrm tht trverses D in redth-first order from ρ 1. For the desired time ound note tht the numer of nodes nd edges in D is ounded y O( S lr Σ ). Finlly, let us discuss how to clssify single string in time O(l). Agin the solution is kin to filure links, however this time it is less simple thn in Section 3 since the set of potentil prtil mtches is no longer descried s set of prefix trees. Our pproch is to ugment the utomton constructed y Theorem 4.1 with edge outputs. The outputs will e numers nd their prtil sums will equl the lengths of mximl prtil mtches to r-contiguous detectors. Formlly, we use Mely utomt tht output numers nd define proper lnguge sed on these outputs. Definition 4.4. A Mely utomton is tuple M = (Q, q i, Q, Σ,, Ω, ω) where (Q, q i, Q, Σ, ) is finite utomton, Ω is the output lphet, nd ω : Ω is the output function. Let m Σ nd t 1,..., t m e the sequence of trnsitions mde y M for input m, then the output of M on input m is the string ω(m, m) = ω(t 1 )... ω(t m ) Ω. If Ω is set of numers, we define the r-threshold lnguge L(M, r) to e the set of strings m Σ where there exists n i m with i j=1 ω(m)[ j] r. Similr to finite utomton, Mely utomton cn e represented y grph where every edge lel represents oth the symol tht triggers the corresponding trnsition nd the output of the trnsition. For exmple, for the Mely utomton M = we hve L(M, 2) nd L(M, 1), ut L(M, 2) Theorem 4.5. There exists n lgorithm tht, given ny S Σ l nd r {1,..., l}, constructs Mely utomton M with output lphet Ω = { r,..., r} such tht L(M, r) Σ l = cont-nonself(s, r) in time O( S lr Σ ). Proof. Let M e the finite utomton constructed in the proof of Theorem 4.1 nd let ρ 1,..., ρ l r+1 e the roots of its underlying grph. We turn M into Mely utomton with output lphet Ω = { r,..., r} such tht L(M, r) Σ l = 8

9 Procedure construct-nonself-mely-utomton(s, r) 1 M Finite utomton from Theorem 4.1 with output 1 for ll trnsitions 2 ρ 1,..., ρ l r+1 root nodes of M s grph 3 for i = l r down to 1 do insert filure links with outputs in right-to-left pss 4 for ech node n rechle from ρ i ut not from ρ i+1 do 5 for ech σ Σ where n hs no outgoing σ-edge do 6 p pth from ρ i to n ; s string on p ; s sσ 7 if there exists pth p for s [2... s ] from ρ i+1 then 8 w sum of outputs on p ; w sum of outputs on p ; n end node of p 9 crete trnsition (n, n, σ) with output w w 10 output M Figure 6: The procedure sketched in the proof of Theorem 4.5, which trnsforms the finite utomton M constructed y Theorem 4.1 into Mely utomton with L(M, r) Σ l = cont-nonself(s, r). Note tht the lnguge L(M, r), formlized in Definition 4.4, depends solely on the output of M, regrdless its ccepting sttes. q i , 1 Figure 7: The Mely utomton M with L(M, 3) = cont(s, 3) where S is the self-set from Figure 2. The solid stright edges re the ones tht remin from the initil prefix trees. The dshed lines re filure links, inserted to dmit liner time clssifiction of given string. Every edge is leled with oth the symol tht triggers the corresponding trnsition, nd the numer output of the trnsition. cont-nonself(s, r) holds. We descrie the min ides of the construction nd discuss its correctness. For presenttion of the detiled computtion steps, we refer to the pseudocode in Figure 6. An exmple of the constructed utomton is shown in Figure 7. We strt y ssigning to ll existing trnsitions of M the output 1. Our im is to trnsform M in right-to-left pss tht inductively ensures the following property: Let m Σ l nd let 1 i j l. Let k 0 denote the length of the longest suffix of m[i... j] tht is lso suffix of some d cont(s, r)[i... j]. If k r l + j, then there exists pth from ρ i for m[i... j], nd the sum of outputs on this pth is equl to k. Otherwise there is no such pth. Hence, if such pth exists nd we hve k r, then m cont-nonself(s, r); otherwise, k is the length of the longest prtil mtch etween m[i... j] nd some d cont(s, r)[i... j] tht cn still e extended to length r. The property lredy holds for i = l r + 1. For i decresing from l r to 1, we itertively trnsform the grph of M s follows: For every node n in M tht is rechle from ρ i, ut not from ρ i+1, consider ll σ Σ where n hs no outgoing edge leled with σ. Let s e the string on the pth p from ρ i to n, w e the totl weight on p, nd s = sσ. If there exists pth p leled with s [2... s ] from ρ i+1, let w denote the sum of weights on this pth. Crete n edge from n to the lst node of p nd lel it with w w. Now there is pth from ρ i leled with s with weight w, fulfilling the required property. Agin, the correctness of this procedure is esily proved y induction, nd we otin Mely utomton with the desired property. Similrly s in Lemm 4.2, the descried trnsformtion cn e implemented in time O( S lr Σ ) y simultneous recursion from ρ i nd ρ i+1. Assuming tht we cn compute the sum of integers in unit time, we cn compute the memership test for the r-threshold lnguge L(M, r) in time O(l) nd thus otin negtive selection lgorithm with time O( S lr Σ ) for the trining phse nd time O(l) for clssifying one string. However, it is possile to get rid of the unit cost ssumption 9

10 y using finite utomton whose sttes store the vlue of the prtil sums. For this construction, we would need to invest n dditionl runtime fctor r in the trining phse. Corollry 4.6. There exists n lgorithm tht, given ny S Σ l nd r {1,..., l}, constructs finite utomton M with L(M) Σ l = cont-nonself(s, r) in time O( S lr 2 Σ ). 5. Conclusions We hve shown how to construct utomt tht simulte the clssifiction results of negtive selection lgorithms with r-contiguous nd r-chunk detectors. The constructions tke time O( S lr Σ ) nd enle susequent clssifiction of ech string in liner time O(l). Tle 1 in the introduction compres the runtimes of previously pulished lgorithms with those from the present pper. As corollry, our result estlishes tht the question if ny r-contiguous detectors cn e generted for given self-set [28] cn e nswered in polynomil time. We leve it s n open prolem whether the symptotic time nd spce complexities of our constructions re optiml. It is conceivle, for exmple, tht y pplying techniques similr to those used in on-line construction of suffix trees [29] the runtime of the construction could e further improved. References [1] Uwe Aickelin. Specil issue on rtificil immune systems editoril. Evolutionry Intelligence, 1(2):83 84, [2] Modupe Ayr, Jon Timmis, Rogério de Lemos, Lendro N. de Cstro, nd Ross Duncn. Negtive selection: How to generte detectors. In Jon Timmis nd Peter J. Bentley, editors, 1st Interntionl Conference on Artificil Immune Systems, pges 89 98, University of Kent t Cnterury, Septemer Unversity of Kent t Cnterury Printing Unit. [3] Justin Blthrop, Fernndo Espond, Stephnie Forrest, nd Mtthew Glickmn. Coverge nd generliztion in n rtificil immune system. In Proceedings of the Genetic nd Evolutionry Computtion Conference (GECCO 2002), pges 3 10, [4] Christopher M. Bishop. Novelty detection nd neurl network vlidtion. IEE Proceedings on Vision nd Imge Signl Processing, 141: , [5] Willim Cvnr nd John M. Trenkle. N-grm-sed text ctegoriztion. In Proceedings of SDAIR-94, 3rd Annul Symposium on Document Anlysis nd Informtion Retrievl, pges , [6] Vrun Chndol, Arindm Bnerjee, nd Vipin Kumr. Anomly detection: A survey. ACM Computing Surveys, 41(3):1 58, [7] Mxime Crochemore, Christophe Hncrt, nd Thierry Lecroq. Algorithms on Strings. Cmridge University Press, 1 edition, April [8] Mxime Crochemore, Filippo Mignosi, nd Antonio Restivo. Automt nd foridden words. Informtion Processing Letters, 67: , [9] Ptrick D heseleer. An immunologicl pproch to chnge detection: Theoreticl results. In Proceedings of the 9th IEEE Computer Security Foundtions, pges IEEE Computer Society, [10] Ptrick D heseleer, Stephnie Forrest, nd Pul Helmn. An immunologicl pproch to chnge detection: Algorithms, nlysis, nd implictions. In Proceedings of the IEEE Symposium on Security nd Privcy, pges IEEE Computer Society, [11] Ted Dunning. Sttisticl identifiction of lnguge. Technicl report, New Mexico Stte University, [12] Michel Elerfeld nd Johnnes Textor. Efficient lgorithms for string-sed negtive selection. In Proceedings of the 8th Interntionl Conference on Artificil Immune Systems (ICARIS 2009), volume 5666 of Lecture Notes in Computer Science, pges Springer, [13] Stephnie Forrest, Steven A. Hofmeyr, Anil Somyji, nd Thoms A. Longstff. A sense of self for unix processes. In Proceedings of the IEEE Symposium on Security nd Privcy, pges , Wshington, DC, USA, IEEE Computer Society. [14] Stephnie Forrest, Aln S. Perelson, Lwrence Allen, nd Rjesh Cherukuri. Self-nonself discrimintion in computer. In Proceedings of the IEEE Symposium on Reserch in Security nd Privcy, pges IEEE Computer Society Press, [15] Chrles Jnewy, Pul Trvers, Mrk Wlport, nd Mrk Shlomchick. Immunoiology. Grlnd Science, [16] Zhou Ji nd Dipnkr Dsgupt. Revisiting negtive selection lgorithms. Evolutionry Computtion, 15(2): , [17] Jeffrey O. Kephrt. A iologiclly inspired immune system for computers. In Artificil Life IV: Proceedings of the Fourth Interntionl Workshop on the Synthesis nd Simultion of Living Systems, pges MIT Press, [18] Donld E. Knuth, Jr. Morris, nd Vughn R. Prtt. Fst pttern mtching in strings. SIAM Journl on Computing, 6(2): , [19] Mciej Liśkiewicz nd Johnnes Textor. Negtive selection lgorithms without generting detectors. In Proceedings of Genetic nd Evolutionry Computtion Conference (GECCO 10). ACM, [20] Emnuel Przen. On the estimtion of proility density function nd mode. Annls of Mthemticl Sttistics, 33: , [21] Jerome K. Percus, Or E. Percus, nd Aln S. Perelson. Predicting the size of the T-cell receptor nd ntiody comining region from considertion of efficient self-nonself discrimintion. Proceedings of the Ntionl Acdemy of Sciences of the United Sttes of Americ, 90(5): , Mrch [22] Bernhrd Schölkopf, John C. Pltt, John Shwe-Tylor, Alex J. Smol, nd Roert C. Willimson. Estimting the support of highdimensionl distriution. Neurl Computtion, 13(7): , [23] Thoms Stior. On the Appropriteness of Negtive Selection for Anomly Detection nd Network Intrusion Detection. PhD thesis, Drmstdt University of Technology,

11 [24] Thoms Stior. Phse trnsition nd the computtionl complexity of generting r-contiguous detectors. In Proceedings of the 6th Interntionl Conference on Artificil Immune Systems (ICARIS 2007), volume 4628 of Lecture Notes in Computer Science, pges Springer, [25] Thoms Stior. Foundtions of r-contiguous mtching in negtive selection for nomly detection. Nturl Computing, 8: , [26] Thoms Stior, Kptsch M. Byrou, nd Cludi Eckert. An investigtion of r-chunk detector genertion on higher lphets. In Proceedings of the Genetic nd Evolutionry Computtion Conference (GECCO 2004), volume 3102 of Lecture Notes in Computer Science, pges Springer, [27] Thoms Stior, Philipp Mohr, Jonthn Timmis, nd Cludi Eckert. Is negtive selection pproprite for nomly detection? In Proceedings of the Genetic And Evolutionry Computtion Conference (GECCO 2005), pges , [28] Jonthn Timmis, Andrew Hone, Thoms Stior, nd Edwrd Clrk. Theoreticl dvnces in rtificil immune systems. Theoreticl Computer Science, 403:11 32, [29] Esko Ukkonen. On-line construction of suffix-trees. Algorithmic, 14(3): , [30] Slwomir T. Wierzchoń. Generting optiml repertoire of ntiody strings in n rtificil immune system. In Intelligent Informtion Systems, Advnces in Soft Computing, pges Physic-Verlg,