Interes'ng- Phrase Mining for Ad- Hoc Text Analy'cs

Transcription

1 Interes'ng- Phrase Mining for Ad- Hoc Text Analy'cs Klaus Berberich inf.mpg.de) Joint work with: Srikanta Bedathur, Jens DiIrich, Nikos Mamoulis, and Gerhard Weikum (originally presented at VLDB 2010)

2 Mo'va'on Con'nuously growing wealth of unstructured text data, e.g.: e- mail and instant messaging social media content (e.g., twiier and facebook) customer reports and product reviews general Web (e.g., news portals) Search on this data is understood and under control (e.g., find all news ar'cles that men'on barack obama) Gaining insights is tedious and a task mostly lew to users (e.g., iden'fy characteris'c quota'ons by barack obama) 2 / 27

3 Mo'va'on Tools available today include Tag clouds Search- result snippets Limita4ons: focus on single terms no en'ty names, quota'ons, etc. frequency- based not necessarily interes'ng (e.g., stopwords) global analysis no ad- hoc document sets (e.g., determined by query) Our idea: Iden'fy interes'ng phrases that dis'nguish an ad- hoc document set from the document collec'on as a whole 3 / 27

4 Mo'va'on Tools available today include Tag clouds Search- result snippets Limita4ons: focus on single terms no en'ty names, quota'ons, etc. frequency- based not necessarily interes'ng (e.g., stopwords) global analysis no ad- hoc document sets (e.g., determined by query) Our idea: Iden'fy interes'ng phrases that dis'nguish an ad- hoc document set from the document collec'on as a whole 3 / 27

5 Mo'va'on Tools available today include Tag clouds Search- result snippets Limita4ons: focus on single terms no en'ty names, quota'ons, etc. frequency- based not necessarily interes'ng (e.g., stopwords) global analysis no ad- hoc document sets (e.g., determined by query) Our idea: Iden'fy interes'ng phrases that dis'nguish an ad- hoc document set from the document collec'on as a whole 3 / 27

6 Outline Mo'va'on Problem Statement Our Approach Prior Art Experimental Evalua'on Conclusion & Ongoing Research 4 / 27

7 Model & Problem Statement Document collec'on D Documents are sequences of terms (e.g., d 12 = < a x z b l k a q x > ) D Phrases are sequences of terms (e.g., < a x z >, < a x z b >, < z b l k >, ) a x z a x z a x z a x b zl k a x a x b z b zl k a x a x l a k a x b zl q x b z b z l a x a k a k a x l k a x q x b zl q x b z b z l a k a k a x l a k z x q x b l q x b z b z q x l a k a k k x l a k q x b zl q x q x d l a e a kq x q x d a kq x a q x s q x q a y d 12 d 37 d 42 Input: Ad- hoc document set D D (e.g., all documents that contain barack obama) Output: k most interes'ng phrases in D 5 / 27

8 Model & Problem Statement Document collec'on D Documents are sequences of terms (e.g., d 12 = < a x z b l k a q x > ) D Phrases are sequences of terms (e.g., < a x z >, < a x z b >, < z b l k >, ) D a x z a x z a x z a x b zl k a x a x b z b zl k a x l a k a x a x b z q x b z b z l l a k a k a x l k a x a x q x b z b z q x b z l l a k a k a x l a k z x q x b z q x b z q x l b l a k a k k x l a k q x b zl q x q x d l a e q x a k d a q x k q x a q x s q x q a y d 12 d 37 d 42 Input: Ad- hoc document set D D (e.g., all documents that contain barack obama) Output: k most interes'ng phrases in D 5 / 27

9 Model & Problem Statement Document collec'on D Documents are sequences of terms (e.g., d 12 = < a x z b l k a q x > ) D Phrases are sequences of terms (e.g., < a x z >, < a x z b >, < z b l k >, ) D a x z a x z a x z a x b zl k a x a x b z b zl k a x l a k a x a x b z q x b z b z l l a k a k a x l k a x a x q x b z b z q x b z l l a k a k a x l a k z x q x b z q x b z q x l b l a k a k k x l a k q x b zl q x q x d l a e q x a k d a q x k q x a q x s q x q a y d 12 d 37 d 42 Input: Ad- hoc document set D D (e.g., all documents that contain barack obama) Output: k most interes'ng phrases in D 1. < z x z > 2. < e s q x > k. < d l e s q > 5 / 27

10 When Do We Consider a Phrase Interes'ng? We dis'nguish for a phrase p its local frequency freq(p, D ) in the ad- hoc document set D global frequency freq(p, D) in the document collec'on D The interes'ngness of phrase p is defined as I(p, D )= freq(p, D ) freq(p, D) We consider only phrases as relevant that are globally frequent, i.e., occur in at least τ documents from D have at most length m (e.g., 5 in our experiments) Note: Our methods adapt to other defini'ons of interes'ngness (e.g., based on log- likelihood ra'os or PMI) 6 / 27

11 Outline Mo'va'on Problem Statement Our Approach Prior Art Experimental Evalua'on Conclusion & Ongoing Research 7 / 27

12 Overview We precompute the global frequency freq(p, D) for any relevant phrase p using an apriori- style algorithm and keep a phrase dic'onary that maps p freq(p, D) Our methods rely on forward indexes that map each document to a representa'on of its content d 12 d 37 d 42 representa'on of d 12 s content representa'on of d 37 s content representa'on of d 42 s content For a given ad- hoc document set D our methods access the forward index for each d D merge the D content representa'ons output the k most interes'ng phrases 8 / 27

13 Document Content Idea: Keep document content explicitly as a sequence of contained terms (or, term iden'fiers) d 12 d 37 d 42 < a x z b l k a q x > < z x z d l e s q x > < k x z d a k q a y > Benefit: Space- efficient Drawbacks: Enumera'on of phrases needed (including globally- infrequent ones) Requires phrase dic'onary with global frequencies freq(p, D) 9 / 27

14 Phrases Idea: Keep globally- frequent phrases contained in document in a consistent (e.g., lexicographical) order d 12 d 37 d 42 < a > < a x > < a x z > < b > < b l > < d > < d l > < d l e > < e > < e s > < a > < a k > < a y > < d > < d a > Benefits: Considers only globally- frequent phrases Consistent sort order facilitates merging Drawbacks: Space- inefficient Requires phrase dic'onary with global frequencies freq(p, D) 10/ 27

15 Frequency- Ordered Phrases Idea: Keep contained globally- frequent phrases in ascending order of their embedded global frequencies freq(p, D) d 12 d 37 d 42 5 : < x z b > < z b > 6 : < q > < x > < x z > 7 : < z > 5 : < e s q > < s q x > 6 : < q > < s > < x > < x z > 5 : < a k q a > < k q a > 6 : < q > < x > < x z > Interes'ngness of any unseen phrase is upper- bounded by min 1, D freq(p, D) where p is the last phrase encountered 11/ 27

16 Frequency- Ordered Phrases Idea: Keep contained globally- frequent phrases in ascending order of their embedded global frequencies freq(p, D) d 12 d 37 d 42 5 : < x z b > < z b > 6 : < q > < x > < x z > 7 : < z > 5 : < e s q > < s q x > 6 : < q > < s > < x > < x z > 5 : < a k q a > < k q a > 6 : < q > < x > < x z > Interes'ngness of any unseen phrase is upper- bounded by min 1, D freq(p, D) where p is the last phrase encountered 11/ 27

17 Frequency- Ordered Phrases Idea: Keep contained globally- frequent phrases in ascending order of their embedded global frequencies freq(p, D) d 12 d 37 d 42 5 : < x z b > < z b > 6 : < q > < x > < x z > 7 : < z > 5 : < e s q > < s q x > 6 : < q > < s > < x > < x z > 5 : < a k q a > < k q a > 6 : < q > < x > < x z > Interes'ngness of any unseen phrase is upper- bounded by min 1, D freq(p, D) where p is the last phrase encountered 11/ 27

18 Frequency- Ordered Phrases Idea: Keep contained globally- frequent phrases in ascending order of their embedded global frequencies freq(p, D) d 12 d 37 d 42 5 : < x z b > < z b > 6 : < q > < x > < x z > 7 : < z > 5 : < e s q > < s q x > 6 : < q > < s > < x > < x z > 5 : < a k q a > < k q a > 6 : < q > < x > < x z > Interes'ngness of any unseen phrase is upper- bounded by min 1, D freq(p, D) D freq(< s >, D) = 3 6 where p is the last phrase encountered 11/ 27

19 Frequency- Ordered Phrases Idea: Keep contained globally- frequent phrases in ascending order of their embedded global frequencies freq(p, D) d 12 d 37 d 42 5 : < x z b > < z b > 6 : < q > < x > < x z > 7 : < z > 5 : < e s q > < s q x > 6 : < q > < s > < x > < x z > 5 : < a k q a > < k q a > 6 : < q > < x > < x z > Benefits: Early termina'on possible when no unseen phrase can make it into the top- k of most interes'ng phrases Self- contained (i.e., no phrase dic'onary needed) Drawback: Space- inefficient 12/ 27

20 Prefix- Maximal Phrases Observa4on: Globally- frequent phrases are owen redundant and therefore do not need to be kept explicitly d 12 d 37 d 42 < a > < a x > < a x z > < b > < b l > < d > < d l > < d l e > < e > < e s > < a > < a k > < a y > < d > < d a > Defini4on: A phrase p is prefix- maximal in document d if p is globally- frequent d does not contain a globally- frequent phrase p of which p is a prefix A prefix- maximal phrase (e.g., < a x z >) can represent all its prefixes (i.e., < a > and < a x >) and it s guaranteed that they re globally- frequent and contained in d 13/ 27

21 Prefix- Maximal Phrases Observa4on: Globally- frequent phrases are owen redundant and therefore do not need to be kept explicitly d 12 d 37 d 42 < a > < a x > < a x z > < b > < b l > < d > < d l > < d l e > < e > < e s > < a > < a k > < a y > < d > < d a > Defini4on: A phrase p is prefix- maximal in document d if p is globally- frequent d does not contain a globally- frequent phrase p of which p is a prefix A prefix- maximal phrase (e.g., < a x z >) can represent all its prefixes (i.e., < a > and < a x >) and it s guaranteed that they re globally- frequent and contained in d 13/ 27

22 Prefix- Maximal Phrases Idea: Keep contained prefix- maximal phrases in lexicographical order and extract prefixes on- the- fly d d 12 d < a x z > < b l > < d l e > < e s > < a k > < a y > < d a > Benefit: Space- efficient Drawbacks: Extrac'on of prefixes entails addi'onal bookkeeping Requires phrase dic'onary with global frequencies freq(p, D) 14/ 27

23 Prefix- Maximal Phrases Idea: Keep contained prefix- maximal phrases in lexicographical order and extract prefixes on- the- fly d d 12 d < a > < a x > < a x z > < b l > < d l e > < e s > < a k > < a y > < d a > Benefit: Space- efficient Drawbacks: Extrac'on of prefixes entails addi'onal bookkeeping Requires phrase dic'onary with global frequencies freq(p, D) 14/ 27

24 Prefix- Maximal Phrases Idea: Keep contained prefix- maximal phrases in lexicographical order and extract prefixes on- the- fly d d 12 d < a > < a x > < a x z > < b l > < d > < d l > < d l e > < e s > < a k > < a y > < d a > Benefit: Space- efficient Drawbacks: Extrac'on of prefixes entails addi'onal bookkeeping Requires phrase dic'onary with global frequencies freq(p, D) 14/ 27

25 Prefix- Maximal Phrases Idea: Keep contained prefix- maximal phrases in lexicographical order and extract prefixes on- the- fly d d 12 d < a > < a x > < a x z > < b l > < d > < d l > < d l e > < e s > < a > < a k > < a y > < d a > Benefit: Space- efficient Drawbacks: Extrac'on of prefixes entails addi'onal bookkeeping Requires phrase dic'onary with global frequencies freq(p, D) 14/ 27

26 Outline Mo'va'on Problem Statement Our Approach Prior Art Experimental Evalua'on Conclusion & Ongoing Research 15/ 27

27 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 < x z z > d 12 16/ 27

28 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 D d 12, d 52, d 7, d < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 < x z z > d 12 16/ 27

29 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 D d 12, d 52, d 7, d < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 < x z z > d 12 16/ 27

30 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 D d 12, d 52, d 7, d < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 < x z z > d 12 16/ 27

31 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 D d 12, d 52, d 7, d < x z k > d 2737, d 57 < x z z > d 12 16/ 27

32 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 D d 12, d 52, d 7, d < x z z > d 12 16/ 27

33 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 < x z z > d 12 D d 12, d 52, d 7, d / 27

34 MCX: Mul'dimensional Content explora'on Objec4ve: Iden'fy k most frequent relevant phrases in a given ad- hoc document set D Idea: Build inverted index that maps p { d d contains p } and sorts phrases in descending order of freq(p, D) < a > d 12, d 2, d 89, d 122, d 7, d 128, d 876, d 4 < b > d 22, d 5, d 19, d 272, d 8, d 218, d 926 < c > d 6, d 51, d 29, d 92, d 41, d 928 < a i > d 12, d 52, d 7, d 218 < x z k > d 2737, d 57 < x z z > d 12 16/ 27

35 MCX: Mul'dimensional Content explora'on Op4miza4on: Approximate fast set intersec'ons through order randomiza'on Benefits: Early termina'on possible when no unseen phrase can make it into the top- k of most frequent phrases performs well for very large ad- hoc document sets D Drawback: considers only frequency as a coarse- grained pre- filter; re- ranks iden'fied frequent phrases based on interes'ngness Reference: A. Simitsis, A. Baid, Y. Sismanis, and B. Reinwald. Mul$dimensional Content explora$on. In VLDB 08 17/ 27

36 Outline Mo'va'on Problem Statement Our Approach Prior Art Experimental Evalua'on Conclusion & Ongoing Research 18/ 27

37 Experimental Evalua'on Dataset: The New York Times Annotated Corpus consis'ng of 1.8M newspaper ar'cles published between 1987 and 2007 Queries to determine ad- hoc document sets based on person- related (e.g., steve jobs, hillary clinton, ) news- related (e.g., world trade center, world cup, ) based on metadata (e.g., /travel/des4na4ons/europe, ) Implementa4on in Java represents data compactly (e.g., variable- length encoding) System: SUN server- class machine (4 CPUs / 16Gb RAM / RAID- 5 / Windows 2003 Server) 19/ 27

38 Examples of Interes'ng Phrases Query: john lennon 1) since john lennon was assassinated 2) lennon s childhood 3) post beatles work Query: bob marley 1) music of bob marley 2) marley the jamaican musician 3) i shot the sheriff Query: john mccain 1) to beat al gore like 2) 2000 campaign in arizona 3) the senior senator from virginia 20/ 27

39 Index Sizes for Different Values of τ Index Size (bytes) 1.4e e+10 1e+10 8e+09 6e+09 4e+09 Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX 2e Minimum Support Threshold! 21/ 27

40 Index Sizes for Different Values of τ Index Size (bytes) 1.4e e+10 1e+10 8e+09 6e+09 4e+09 Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX 2e Gb Minimum Support Threshold! 21/ 27

41 Index Sizes for Different Values of τ Index Size (bytes) 1.4e e+10 1e+10 8e+09 6e+09 4e+09 Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX 4.41 Gb 2e Minimum Support Threshold! 1.80 Gb 21/ 27

42 Index Sizes for Different Values of τ Index Size (bytes) 1.4e e+10 1e+10 8e+09 6e+09 4e+09 Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX 5.64 Gb 4.41 Gb 2e Minimum Support Threshold! 1.80 Gb 21/ 27

43 Index Sizes for Different Values of τ Index Size (bytes) 1.4e e+10 1e+10 8e+09 6e+09 4e+09 Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX Gb 5.64 Gb 4.41 Gb 2e Minimum Support Threshold! 1.80 Gb 21/ 27

44 Index Sizes for Different Values of τ Index Size (bytes) 1.4e e+10 1e+10 8e+09 6e+09 4e+09 Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX Gb 5.64 Gb 4.41 Gb 2e Minimum Support Threshold! 1.80 Gb 21/ 27

45 Wall- Clock Times for Different Values of D 1e+06 Wallclock Time (ms) Document Content 100 Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX k = 100 τ = 10 Input Cardinality D 22/ 27

46 Wall- Clock Times for Different Values of D 1e+06 Wallclock Time (ms) Document Content 100 Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX ,030 ms k = 100 τ = 10 Input Cardinality D 22/ 27

47 Wall- Clock Times for Different Values of D 1e+06 Wallclock Time (ms) Document Content 100 Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX ,500 ms 1,030 ms k = 100 τ = 10 Input Cardinality D 22/ 27

48 Wall- Clock Times for Different Values of D 1e+06 Wallclock Time (ms) Document Content 100 Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX ,779 ms 3,500 ms 1,030 ms k = 100 τ = 10 Input Cardinality D 22/ 27

49 Wall- Clock Times for Different Values of D 1e+06 Wallclock Time (ms) Document Content 100 Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX ,575 ms 14,779 ms 3,500 ms 1,030 ms k = 100 τ = 10 Input Cardinality D 22/ 27

50 Wall- Clock Times for Different Values of D 1e+06 Wallclock Time (ms) Document Content 100 Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases MCX ,575 ms 14,779 ms 3,500 ms 1,030 ms k = 100 τ = 10 Input Cardinality D 22/ 27

51 Wall- Clock Times for Different Values of k Wallclock Time (ms) Frequency-Ordered Phrases Frequency-Ordered Phrases w/ EarlyTermination τ = 10 D = 500 Result Size k 23/ 27

52 Wall- Clock Times for Different Values of k Wallclock Time (ms) τ = 10 D = Frequency-Ordered Phrases Frequency-Ordered Phrases w/ EarlyTermination Result Size k 357 ms 23/ 27

53 Wall- Clock Times for Different Values of k ms Wallclock Time (ms) τ = 10 D = Frequency-Ordered Phrases Frequency-Ordered Phrases w/ EarlyTermination Result Size k 357 ms 23/ 27

54 Wall- Clock Times for Different Values of k ms Wallclock Time (ms) τ = 10 D = Frequency-Ordered Phrases Frequency-Ordered Phrases w/ EarlyTermination Result Size k 357 ms 23/ 27

55 Outline Mo'va'on Problem Statement Our Approach Prior Art Experimental Evalua'on Conclusion & Ongoing Research 24/ 27

56 Conclusion Efficient iden'fica'on of interes'ng phrases on ad- hoc document sets is a challenging research problem Our methods to tackle this problem are based on forward indexes differ in how they represent document contents Frequency- Ordered Phrases allow for early termina'on Prefix- Maximal Phrases result in a very compact index Experiments on real- world dataset show that our methods work in prac'ce and outperform the state- of- the- art method substan'ally on realis'c inputs 25/ 27

57 Ongoing Research Alterna4ve scenario: Iden'fy phrases that best dis'nguish two ad- hoc document sets (e.g., barack obama vs. george bush or recently published vs. all documents on UEFA) Implement our index structures and pre- computa'ons using Hadoop (or another MapReduce implementa'on) Support grouping of almost- iden'cal phrases (e.g., george bush said vs. george w. bush said) Make use of NLP techniques such as part- of- speech tagging (e.g., to abstract from concrete pronouns) 26/ 27

58 Thank you! Ques'ons?

59 Wall- Clock Times for Different Values of τ Wallclock Time (ms) k = 100 D = Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases Minimum Support Threshold! 28/ 27

60 Wall- Clock Times for Different Values of τ Wallclock Time (ms) Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases ms k = 100 D = 500 Minimum Support Threshold! 28/ 27

61 Wall- Clock Times for Different Values of τ Wallclock Time (ms) Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases 784 ms 360 ms k = 100 D = 500 Minimum Support Threshold! 28/ 27

62 Wall- Clock Times for Different Values of τ Wallclock Time (ms) k = 100 D = Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases Minimum Support Threshold! 1,125 ms 784 ms 360 ms 28/ 27

63 Wall- Clock Times for Different Values of τ Wallclock Time (ms) k = 100 D = Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases Minimum Support Threshold! 4,075 ms 1,125 ms 784 ms 360 ms 28/ 27

64 Wall- Clock Times for Different Values of τ Wallclock Time (ms) k = 100 D = Document Content Phrases Frequency-Ordered Phrases Prefix-Maximal Phrases Minimum Support Threshold! 4,075 ms 1,125 ms 784 ms 360 ms 28/ 27