Announcements. Introduction to String Matching. String Matching. String Matching. String Matching. String Matching

Size: px

Start display at page:

Download "Announcements. Introduction to String Matching. String Matching. String Matching. String Matching. String Matching"

Antony Norris
5 years ago
Views:

1 Itroductio to Fudametal Data Structures ad Algorithms Peter Lee April 22, 2003 Aoucemets Quiz #4 available after class today! available util Wedesday midight Homework 6 is out! Due o May??, 11:59pm Touramet will be ru o May 7 details to come Fial exam o May 8 8:30am11:30am, UC McCoomy Review o May 4, details TBA Text strig T[0..N1] T = abacaabaccaabb Patter strig P[0..M1] P = Where is the first istace of P i T? T[10..15] = P[0..5] abacaabaccaabb abacaabaccaabb The brute force algorithm 22+6=28 comparisos. 1

2 , v.1 static it match(char[] T, char[] P){ it = T.legth; for (it i=0; i<=m; i++) { it j = 0; while (j<m && T[i+j]==P[j]) j++; if (j==m) retur i; retur 1; Quiz Break Rewrite this i oe loop static it match(char[] T, char[] P){ it = T.legth; it i = 0; it j = 0; Text strig T[0..N1] T = abacaabaccaabb Patter strig P[0..M1] P = Where is the first istace of P i T? T[10..15] = P[0..5] I geeral, how may comparisos T[i] = P[j]? are eeded to do the search? Worst case: O(NM) A bad case Typical text matchig = 65 comparisos are eeded How may of them could be avoided? match.java This is a sample setece s s s sete 20+5=25 comparisos are eeded (The match is ear the same poit i the target strig as the previous example.) I practice, 0 j 2 match.java 2

3 Brute force worst case O(MN) Expesive for log patters i repetitive text How to improve o this? Ituitio: Do t look at the text more tha oce. Remember what is leared from previous matches A bit of history Cook published a abstract result about machie models Match i O(N+M) vs. O(MN)?! Kuth ad Pratt studied it ad refied it ito a simple algorithm. Morris, aoyed at a desig problem i implemetig a text editor, discovered the same algorithm. How to avoid decremetig i? published together i Meawhile. Boyer ad Moore discovered aother algorithm that is eve faster (for some uses) i the average case. Gosper idepedetly discovered the same algorithm. Boyer ad Moore published i Kuth Morris Pratt The idea example Take advatage of what we already kow durig the match process. Suppose P = Suppose P[0..5] matches T[10..15] Suppose P[6] T[16] The we kow P[0] ay of T[10..15] Ad the ext possible match is P[0]? T[16] Match fails: T[i] P[j] i = 6 j = 6 Next match attempt i = 6 j = 0 T P

4 A worse case example: = 210 comparisos = 42 comparisos bc bc 21 comparisos bc bc 19 comparisos 5 preparatio comparisos The Big Idea Retai iformatio from prior attempts. Compute i advace how far to jump i P whe a match fails. Suppose the match fails at P[j] T[i+j]. The we kow P[0.. j1] = T[i.. i+j1]. We must ext try P[0]? T[i+1]. But we kow T[i+1]=P[1] There is therefore aother way to compare: P[1]?P[0] If so, icremet j by 1. No eed to look at T. What if P[1]=P[0] ad P[2]=P[1]? The icremet j by 2. Agai, o eed to look at T. I geeral, we ca determie how far to jump without ay kowledge of T! Implemetig Never decremet i, ever. Comparig T[i] with P[j]. Compute a table f of how far to jump j forward whe a match fails. The ext match will compare T[i] with P[f[j1]] Do this by matchig P agaist itself i all positios. Buildig the Table for f What f meas P = Fid selfoverlaps If f is zero, there is o selfmatch. This is good ews: Set j=0 Do ot chage i. The ext match is T[i]? P[0] f ozero implies there is a selfmatch. This is bad ews: E.g., f=2 meas P[0..1] = P[j2..j1] Hece must start ew compariso at j2, sice we kow T[i2..i1] = P[0..1] I geeral: Set j=f[j1] Do ot chage i. The ext match is T[i]? P[f[j1]] 4

5 Favorable coditios P = Fid selfoverlaps Mixed coditios P = Fid selfoverlaps Poor coditios P = Fid selfoverlaps matcher static it match(char[] T, char[] P) { it = T.legth; it[] f = computef(p); it i = 0; it j = 0; while(i<) { if(p[j]==t[i]) { if (j==m1) retur im+1; i++; j++; else if (j>0) j=f[j1]; else i++; retur 1; Use f to determie ext value for j. preprocess static it[] computef(char[] P) { it[] f = ew it[m]; f[0] = 0; it i = 1; it j = 0; while(i<m) { if(p[j]==p[i]) { f[i] = j+1; i++; j++; else if (j>0) j=f[j1]; else {f[i] = 0; i++; retur f; Use previous values of f Specializig the matcher

6 Performace At each iteratio, oe of three cases: T[i] = P[j] i icreases T[i] <> P[j] ad j>0 ij icreases T[I] <> P[j] ad j=0 i icreases ad ij icreases Hece, maximum of 2N iteratios. Costructig f[] eeds 2M iteratios. Thus worst case performace is O(N+M). Boyer Moore BM bc bc bc bc bc bc g f d e c b 21 comparisos 19 comparisos = 21 comparisos = 8 comparisos Boyer Moore BM Ideas Sca patter from right to left (ad target from left to right) Allows for bigger jumps o early failures Could use a table similar to. But follow a better idea: Use iformatio about T as well as P i decidig what to do ext. This strig is textual t textual = 23 comparisos This strig is textual l a u t x e t = 10 comparisos 6

7 This is a sample setece 25 comparisos BM This is a sample setece foobar 5 comparisos Boyer Moore Ideas Sca patter from right to left (ad target from left to right) Allows for bigger jumps o early failures Could use a table similar to. But follow a better idea: Use iformatio about T as well as P i decidig what to do ext. If T[i] does ot appear i the patter, skip forward beyod the ed of the patter. Boyer Moore matcher static it match(char[] T, char[] P) { it[] last = buildlast(p); it = T.legth; it i = m1; it j = m1; if (i > 1) retur 1; do { if (P[j]==T[i]) if (j==0) retur i; else { i; j; else { i = i + m Math.mi(j, 1 + last[t[i]]); j = m 1; while (i <= 1); retur 1; Use last to determie ext value for i. Boyer Moore matcher static it[] buildlast(char[] P) { it[] last = ew it[128]; for (it i=0; i<128; i++) last[i] = 1; for (it j=0; j<p.legth; j++) last[p[j]] = j; retur last; Mismatch char is owhere i the patter (default). last says jump the distace Mismatch is a patter char. last says jump to alig patter with last istace of this char BM BM This is a strig This is a strig g i r rig rig 13 comparisos 1 compariso 16 comparisos 7 comparisos 7

8 This is a strig trig BM This is a strig trig g i r t Matchig Summary 16 comparisos 8 comparisos KuthMorrisPratt Summary Ituitio: Aalyze the patter Aalog with a Matchig FSM. Steambased: Never decremet i. Works well: For selfrepetitive patters i selfrepetitive text But: For text, performace similar to brute force Possibly slower, due to precomputatio BoyerMoore Summary Ituitio: Aalyze the target ad the patter Work backwards from ed of patter Jump forward i target whe failig Works well: For large alphabets The last table for {0,1? For text, i practice But: Streams? Must be able to decremet i. A possiblytrue story: A programmer, reluctat to lear the tricky preprocessig i Morris s algorithm, i fact, implemeted it usig the brute force algorithm istead. I 1980, Karp ad Rabi discovered a simpler algorithm. Uses hashig ideas: quickly compute hashes for all Mlegth substrigs i T, ad compare with the hash for P. Karp ad Rabi Uses hashig ideas: quickly compute hashes for all Mlegth substrigs i T, ad compare with the has for P. Compute the hashes i a cumulative way, so each T[i] eeds to be see oly oce. Average case time is O(M+N). Worst case is ulikely (all collisios) at O(MN). 8

OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES

OPTIMAL ALGORITHMS -- SUPPLEMENTAL NOTES Peter M. Maurer Why Hashig is θ(). As i biary search, hashig assumes that keys are stored i a array which is idexed by a iteger. However, hashig attempts to bypass