Frequent Pattern Mining. Toon Calders University of Antwerp

Transcription

1 Frequent Pattern Mining Toon alders University of ntwerp

2 Summary Frequent Itemset Mining lgorithms onstraint ased Mining ondensed Representations

3 Frequent Itemset Mining Market-asket nalysis transaction identifier TI items transaction

4 Frequent Itemset Mining support(i): number of transactions containing I TI Support() = 3 2 Support() =

5 Frequent Itemset Mining Problem Given, minsup Find all sets I with support(i) minsup TI minsup= {},,,,,,,,,, 5

6 Why? Important component in mining algorithms Sufficient statistics for interestingness measures onfidence X Y : Support(XY)/Support(X) ontingency tables (correlation, X 2 ) Y Y X s(xy) s(x) - s(xy) X s(y) - s(xy) s({}) - s(x) -s(y) + s(xy)

7 Summary Frequent Itemset Mining lgorithms onstraint ased Mining ondensed Representations

8 lgorithms There exist hundreds of algorithms that solve FIM (or related problems) IS, priori, prioriti, priorihybrid, FPGrowth, FPGrowth*, Eclat, declat, Pincersearch, S, I, ki, LM, IM, PIE, RMOR, FOPT, OFI, Patricia, MXMINER, MFI, NI-LL,

9 lgorithms There exist hundreds of algorithms that solve FIM (or related problems) oncentrate on the most important pruning principle: Monotonicity and the two main search strategies: readth-first epth-first

10 Monotonicity Principle If I J, then support(i) support(j) Therefore, if I is infrequent, then all its supersets are infrequent as well. ll FIM algorithms rely heavily on this principle to prune large parts of the search space.

11 Search Space infrequent {}

12 Levelwise lgorithm Exploits monotonicity as much as possible. Search Space is traversed bottom-up, level by level Support of an itemset is only counted in the database if all its subsets were frequent.

13 TI 2 3 priori 4 5 minsup=2 andidates {}

14 TI 2 3 priori 4 5 minsup=2 {}

15 TI 2 3 priori 4 5 minsup=2 2 2 {}

16 TI 2 3 priori 4 5 minsup=2 2 3 {}

17 TI 2 3 priori 4 5 minsup= {}

18 TI 2 3 priori 4 5 minsup= {}

19 TI 2 3 priori 4 5 minsup=2 andidates {}

20 TI 2 3 priori 4 5 minsup= {}

21 TI 2 3 priori 4 5 minsup=2 andidates {}

22 TI 2 3 priori 4 5 minsup= {}

23 epth-first lgorithms TI TI TI Find all frequent itemsets Find all frequent itemsets, with Find all frequent itemsets, without

24 epth-first lgorithm TI TI 3 4 TI [] [] TI [] TI [] TI [] TI [],,,,,,

25 readth-first vs epth-first epth-first outperformes breadth-first Number of frequent itemsets is very high atabase is relatively small readth-first outperformes depth-first Number of frequent sets is small atabase is large ifferences usually very small

26 Summary Frequent Itemset Mining lgorithms onstraint ased Mining ondensed Representations

27 Mining With onstraints Reduce output size, user sets focus itemsets of size > 5 sets of products with cost less than EUR sets that contain,, or. sets that are frequent in dataset, but infrequent in 2

28 Mining With onstraints Types of constraints (nti-)monotone, Succinct onvertible Two pproaches Pushing constraints into the mining algorithm hanging the atabase

29 Types of onstraints nti-monotone Support, size <,

30 Types of onstraints Monotone ost >EUR, ontains,, or,

31 Types of onstraints Succinct an be expressed using minus and union on a fixed number of powersets E.g., ontains or, but not : 2 I- 2 I- an be generated efficiently onvertible anti-monotone nti-monotone w.r.t. prefix-order E.g. avg(i.price)< EUR when ordered ascending by price.

32 Mining With onstraints Two approaches: Pushing constraints deep in data mining algorithm hanging database such that Support of itemsets satisfying the constraint does not change The support of itemsets that do not satisfy the constraint decreases

33 Pushing onstraints Monotone Frequency nti-monotone

34 Pushing onstraints Trade-off Pushing monotone constraints vs. anti-monotone pruning Not always better to push monotone constraints E.g. Size >

35 hanging the atabase Exnte lgorithm Exploit Monotone and nti-monotone constraints transaction that does not satisfy a monotone constraint will not contribute to any itemset satisfying the constraints E.g. constraint size > : every transaction of size < can be thrown away!

36 hanging the atabase minsup = 3 anti-mon. size 4 monotone I H G F E I

37 Summary Frequent Itemset Mining lgorithms onstraint ased Mining ondensed Representations

38 ondensed Representations Sometimes, the output of frequent set mining remains too large: Huge number of items Highly correlated High support items Hence, instead of mining all itemsets ondensed representation

39 ondensed Representations losed sets ivide frequent itemsets into equivalence classes Two itemsets are equivalent if they occur in the same transactions losed set: maximal element in an equivalence class

40 losed Itemsets ll sets in the same equivalence class have the same support Occur in the same transactions Maximal element in an equivalence class is unique If two itemsets occur in the same transactions, then so does their union

41 TI 2 3 losed Itemsets 4 5 {}

42 losed Itemsets Has nice mathematical properties losed sets form a lattice Galois connection Efficient algorithms to find them ased on the closed sets, it is easy to find the support of the other itemsets.

43 losed Itemsets Interesting class of patterns Maximal frequent itemsets are closed sets Highest correlation between items Strongest association rules Significant reduction of number of itemsets Especially with small number of large transactions

44 Non-erivable Itemsets ased on redundancies How do supports interact? What information about unknown supports can we derive from known supports? oncise representation: only store relevant part of the supports

45 Redundancies grawal et al. Supp(X) Supp() (Monotonicity) oulicaut et al., Lakhal et al. (Free sets) If Supp() = Supp() Then Supp(X) = Supp(X) (losed sets)

46 Redundancies ayardo (MXMINER) Supp(X) Supp(X) (Supp(X)-Supp(X)) ykowski, Rigotti drop (X, ) (isjunction-free sets) if Supp() = Supp() + Supp() Supp(), then Supp(X) can be derived from X, X, X

47 The Inclusion Exclusion Principle =

48 eduction Rules via Inclusion- Exclusion Let,,, be items Let correspond with the set { transaction t t contains } = Then: Supp() =

49 eduction Rules via Inclusion-Exclusion Inclusion-exclusion principle: = Thus, since n, Supp() s() + s() + s() - s() s() s() + n

50 omplete Set for Supp() 2 3 s s s s s Monotonicity s s Free, losed s s + s s s s + s s isjunction-free s s + s s s s + s + s s s s + n

51 erivable Itemsets Given: Supp(I) for all I J Lower bound on Supp(J) = l Upper bound on Supp(J) = u Without counting : Supp(J) [l,u] J is a derivable itemset (I) iff l = u We know Supp(J) exactly without counting!

52 erivable Itemsets J derivable itemset: No need to count Supp(J) No need to store Supp(J) We can use the deduction rules oncise representation: = { ( J, Supp(J) ) J not derivable from Supp(I), I J }

53 erivable Itemsets Theorem (Monotonicity) If J K, J derivable, then K derivable. Moreover: The width of the interval for J {} is at most half the size of the interval for J!

54 IV. Evaluation --- Theoretical Interval widths decrease exponentially Half each step Non-derivable itemset can never be larger than log( atabase ) Independent of sparse, dense,...

55 Evaluation --- Empirically Size NI vs. frequent itemsets omparison with Other oncise Reps

56 PUMS

57 PUMS

58 Evaluation Number of frequent NIs considerable smaller than number of frequent itemsets lgorithm is efficient alculating NI + deducing Is often outperforms priori

59 ondensed Representations Many other representations Free sets isjunction-free sets Generalized disjunction-free sets losed sets and NIs provable the smallest ones

60 onclusion epth-first vs readth-first algorithms for FIM onstraint mining to incorporate user focus Pushing constraints vs changing database ondensed Representations losed sets Non-erivable Itemsets

61 Topics Not overed Parallel algorithms for FIM Incremental FIM Generalized, Quantitative, Multi-level, Fuzzy Rs oupling FIM with RMS Privacy Preserving RM omputational omplexity Results Inverse mining problem Emerging Patterns, jumping emerging patterns ependency value, X 2 Lift, gain lock support, tilings,