Frequent Itemset Mining

Transcription

1 ì 1 Frequent Itemset Mining Nadjib LAZAAR LIRMM- UM COCONUT Team (PART I) IMAGINA 17/18 Webpage: lazaar@lirmm.fr

2 2 Data Mining ì Data Mining (DM) or Knowledge Discovery in Databases (KDD) revolves around the investigation and creation of knowledge, processes, algorithms, and the mechanisms for retrieving potential knowledge from data collections.

3 3 Game Data Mining ì Data about players behavior, server performance, system functionality ì How to convert these data into something meaningful? ì How to move from raw data to actionable insights? è Game data mining is the answer

4 4 Frequent Itemset Mining: Motivations Frequent Itemset Mining is a method for market basket analysis. It aims at finding regularities in the shopping behavior of customers of supermarkets, mail-order companies, on-line shops etc. ì ì ì More specifically: Find sets of products that are frequently bought together. Possible applications of found frequent itemsets: ì Improve arrangement of products in shelves, on a catalog s pages etc. ì Support cross-selling (suggestion of other products), product bundling. ì Fraud detection, technical dependence analysis, fault localization etc. Often found patterns are expressed as association rules, for example: ì If a customer buys bread and wine, then she/he will probably also buy cheese.

5 5 Frequent Itemset Mining: Basic notions ì Items: ì Itemset: ì Transactions: ì Language of itemsets: ì Transactional dataset: ì Cover of an itemset: ì (absolute) Frequency: I = {i 1,...i n } P I T = {t 1,...t m } L I =2 I D I T S(P )={t i 2 T P t i } freq(p )= S(P)

6 6 Absolute/relative frequency ì Absolute Frequency: freq(p )= S(P ) ì Relative Frequency: freq(p )= 1 m S(P )

7 7 Frequent Itemset Mining: Definition ì Given: ì ì ì A set of items I = {i 1,...i n } A set of transactions overs the items A minimum support T = {t 1,...t m } ì The need: ì The set of itemset P s.t.: freq(p )

8 8 Example (1) I = {a, b, c, d, e} T = {1...1} H D V D M D S(bc) ={2, 7, 9} freq(bc) =3

9 9 Example (1) ;

10 1 Example (1) ;

11 11 Example (1) Frequent itemset? ;

12 12 Example (1) Frequent itemset with minimum support θ=3? ;

13 13 Searching for Frequent Itemsets ì A naïve search that consists of enumerating and testing the frequency of itemset candidates in a given dataset is usually infeasible. ì Why? Number of items (n) Search space (2 n ) (atoms in the universe)

14 14 Anti-monotonicity property ì Given a transaction database D over items I and two itemsets X, Y: ì That is, X Y ) S(Y ) S(X) X Y ) freq(y ) apple freq(x)

15 15 Example (2) S(ade) ={1, 4, 8, 1} freq(ade) =4 S(acde) ={4, 8} freq(acde) =2 7 3 ; ade 2 2 acde

16 16 Apriori property ì Given a transaction database D over items I, a minsup θ and two itemsets X, Y: X Y ) freq(y ) apple freq(x) ì It follows: X Y ) (freq(y ) ) freq(x) ) All subsets of a frequent itemset are frequent! ì Contraposition: X Y ) (freq(x) < ) freq(y ) < ) All supersets of an infrequent itemset are infrequent!

17 17 Example (3) All subsets of a frequent itemset are frequent! = a c d e ; ac ad ae cd ce de acd ace ade 2 cde 2 acde

18 18 Example (3) All supersets of an infrequent itemset are infrequent! =2 7 3 ; be abe bce bde 2 abce abde 2 bcde abcde

19 19 Partially ordered sets ì A partial order is a binary relation R over a set S : 8x, y, z 2 S x R x x R y ^ y R x ) x = y x R y ^ y R z ) x R z (reflexivity) (anti-symmetry) (transitivity) ; S =? R =?

20 2 Poset (2 I, ) ì Comparable itemsets: ì Incomparable itemsets: x y _ y x x 6 y ^ y 6 x ;

21 21 Apriori Algorithm [Agrawal and Srikant 1994] ì Determine the support of the one-element item sets (i.e. singletons) and discard the infrequent items. ì Form candidate itemsets with two items (both items must be frequent), determine their support, and discard the infrequent itemsets. ì Form candidate item sets with three items (all contained pairs must be frequent), determine their support, and discard the infrequent itemsets. ì And so on! Based on candidate generation and pruning

22 Apriori Algorithm [Agrawal and Srikant 1994] 22

23 23 Apriori candidates generation apriori-gen(l k ) E ; 8P i,p j 2 L k s.t. : (P i = {i 1,...,i k 1,i k }) ^ (P i = {i 1,...,i k 1,i k }) ^ (i k <i k ) P P 1 [ P 2 //{i 1,...,i k 1,i k,i k } if 8i 2 P : P \{i} 2 L k then E E [ {P } return E

24 24 Improving candidates generation ì Using apriori-gen function, an item of k+1 size can be generated in a j possible ways: j = k(k+1) 2 ì Need: Generate itemset candidate at most once. ì How: Assign to each itemset a unique parent itemset, from which this itemset is to be generated

25 25 Improving candidates generation ì Assigning unique parents turns the poset lattice into a tree:

26 26 Canonical form for itemsets ì An itemset can be represented as a word over an alphabet I ì Q: how many words of 3 items can we have? Of 4 items? Of k items? k! ì An arbitrary order (e.g., lexicography order) on items can give a canonical form, a unique representation of itemsets by breaking symmetries. ì Lex on items : abc < acb < bac < bca...

27 Recursive processing with Canonical 27 forms ì Foreach P of a given level, generate all possible extension of P by one item such that: child(p )={P :(i/2 P ) ^ (P = P [ {i}) ^(c(p ).last < i) ^ (P is frequent)} ì Foreach P, process it recursively.

28 28 Example (4) Q: what are the children of: child(p )={P :(i/2 P ) ^ (P = P [ {i}) ^(c(p ).last < i) ^ (P is frequent)}

29 29 Items Ordering ì Any order can be used, that is, the order is arbitrary ì The search space differs considerably depending on the order ì Thus, the efficiency of the Frequent Itemset Mining algorithms can differ considerably depending on the item order ì Advanced methods even adapt the order of the items during the search: use different, but compatible orders in different branches

30 3 Items Ordering (heuristics) ì Frequent itemsets consist of frequent items ì Sort the items w.r.t. their frequency. (decreasing/increasing) ì The sum of transaction sizes, transaction containing a given item, which captures implicitly the frequency of pairs, triplets etc. ì Sort items w.r.t. the sum of the sizes of the transactions that cover them.

31 ì 31 Tutorials link: