Zijian Zheng, Ron Kohavi, and Llew Mason. Blue Martini Software San Mateo, California
|
|
|
- John Jacobs
- 5 years ago
- Views:
Transcription
1 1 Zijian Zheng, Ron Kohavi, and Llew Mason Blue Martini Software San Mateo, California August 11, 2001 Copyright , Blue Martini Software. San Mateo California, USA 1
2 Performance Improvement Irrelevant 2 Very narrow min-sup range of interest Super exponential growth in number of rules on real world data 0% Impossible Super exponential growth in # Rules >1,000,000,000 rules Fast, but incorrect results Apriori sufficient <1,000,000 rules < 10 minutes for all alg. Interesting Range as narrow as 0.02% Copyright , Blue Martini Software. San Mateo California, USA 2
3 3 Performance improvements on artificial data did not generalize to real world data Improvement over Apriori: Artificial Real-POS Closet 1.9 x 1.2 x Charm 2.8 x 1.1 x FP-growth 12.1 x 0.8 x Avg 3.1 x 1.0 x Are algorithms overfitting the artificial data? Copyright , Blue Martini Software. San Mateo California, USA 3 Frequence (%) Artificial Real-Web-1 Real-Web-2 Real-POS Transaction size
4 4 Association rule discovery: Active research area Good application potential Many promising new algorithms Each new algorithm has significant performance improvements mainly based on results on IBM Almaden artificial datasets How do these algorithms perform on realworld data? Copyright , Blue Martini Software. San Mateo California, USA 4
5 5 Transactions Distinct Items Maximum Transaction Size Average Transaction Size IBM-Artificial 100, BMS-POS 515,597 1, BMS-WebView-1 59, BMS-WebView-2 77,512 3, Copyright , Blue Martini Software. San Mateo California, USA 5
6 6 IBM artificial dataset is very different from the real-world datasets Frequence (%) IBM-Artificial BMS-WebView-1 BMS-WebView-2 BMS-POS Transaction size Copyright , Blue Martini Software. San Mateo California, USA 6
7 7 1,000,000,000 #Frequent Itemsets 1,000,000,000 #Frequent Itemsets 100,000,000 #Association Rules 100,000,000 #Association Rules 10,000,000 #Closed Frequent Itemsets 10,000,000 #Closed Frequent Itemsets 1,000,000 1,000, ,000 Count 10, ,000 Count 10,000 1,000 1, Minimum support (%) IBM-Artificial Minimum support (%) BMS-POS Copyright , Blue Martini Software. San Mateo California, USA 7
8 8 100,000,000 #Frequent Itemsets 100,000,000 #Frequent Itemsets 10,000,000 #Association Rules 10,000,000 #Association Rules 1,000,000 #Closed Frequent Itemsets 1,000,000 #Closed Frequent Itemsets 100, ,000 10,000 10,000 Count Count 1,000 1, Minimum support (%) Minimum support (%) BMS-WebView-1 BMS-WebView-2 Copyright , Blue Martini Software. San Mateo California, USA 8
9 Ratio of #Freq itemsets over #Closed freq itemsets Ratio Ratio Minimum support (%) Minimum support (%) BMS-POS BMS-WebView-2 Copyright , Blue Martini Software. San Mateo California, USA 9
10 10 Dual 550MHz Pentium III with 1 GB of memory Windows NT 4.0 Measures: time (seconds) for generating frequent itemsets/association rules Minimum support: 1.00%, 0.80%, 0.60%, 0.40%, 0.20%, 0.10%, 0.08%, 0.06%, 0.04%, 0.02%, and 0.01% Minimum confidence: 0% Copyright , Blue Martini Software. San Mateo California, USA 10
11 11 Apriori: C. Borgelt s implementation Charm: M. Zaki FP-growth: J. Han s research group Closet: J. Han s research group MagnumOpus : G. Webb Copyright , Blue Martini Software. San Mateo California, USA 11
12 12 Rankings (left is better) of the algorithms for generating frequent itemsets on the four datasets with high minimum supports and low minimum supports (Ap: Apriori, FP: FP-growth, Ch: Charm, Cl: Closet): Ranking High Min-Support Low Min-Support IBM-Artificial Ap > FP > Ch > Cl FP > Ch > Cl > Ap BMS-POS Ap > Cl > FP > Ch Ch > FP > Ap > Cl BMS-WebView-1 Ap > FP > Cl > Ch Ch > FP > Ap > Cl BMS-WebView-2 Ap > FP > Ch > Cl Ch > FP > Ap > Cl Copyright , Blue Martini Software. San Mateo California, USA 12
13 13 TIME (Seconds) 1, Charm FP-growth Apriori Closet Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 13
14 14 TIME (Seconds) 1, Closet Charm FP-growth Apriori Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 14
15 15 TIME (Seconds) 100, , , Charm FP-growth Apriori Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 15
16 16 TIME (Seconds) 100, , , Closet Charm FP-growth Apriori Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 16
17 (frequent itemsets) 17 1,000.0 TIME (Seconds) Charm FP-growth Apriori Closet Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 17
18 (frequent itemsets) 18 TIME (Seconds) 1, Closet Charm FP-growth Apriori Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 18
19 (frequent itemsets) 19 TIME (Seconds) 100, , , Charm FP-growth Apriori Closet Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 19
20 (frequent itemsets) 20 TIME (Seconds) 100, , , Closet Charm FP-growth Apriori Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 20
21 21 On some real-world datasets, when the minimum support is small, the number of frequent itemsets increases super-exponentially, thus no algorithm can handle it. E.g. BMS-WebView-1: Minimum support(%) Frequent itemsets *: estimated Copyright , Blue Martini Software. San Mateo California, USA 21
22 22 TIME (Seconds) 10,000 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 22
23 23 TIME (Seconds) 10,000 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 23
24 24 TIME (Seconds) 1,000, ,000 10,000 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 24
25 25 TIME (Seconds) 1,000, ,000 10,000 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 25
26 (association rules) 26 TIME (Seconds) 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 26
27 (association rules) 27 TIME (Seconds) 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 27
28 (association rules) 28 TIME (Seconds) 10,000 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 28
29 (association rules) 29 TIME (Seconds) 10,000 1, Apriori MO MO Minimum Support (%) Copyright , Blue Martini Software. San Mateo California, USA 29
30 30 MO could be a solution when the number of association rules is very large only the top-n rules are needed (based on some criteria such as lift or confidence) Copyright , Blue Martini Software. San Mateo California, USA 30
31 31 First objective evaluation and comparison of association rule algorithms on real-world e- commerce and retail datasets. Donated one e-commerce datasets for use in the research community. Copyright , Blue Martini Software. San Mateo California, USA 31
32 32 Artificial datasets have very different characteristics from the real-world datasets. On real world data: Very narrow min-sup range of interest Super exponential growth in number of rules Performance improvements on artificial data did not generalize to real world data Optimizing algorithms for these artificial datasets can mislead research effort Copyright , Blue Martini Software. San Mateo California, USA 32
An Efficient Algorithm for Enumerating Closed Patterns in Transaction Databases
An Efficient Algorithm for Enumerating Closed Patterns in Transaction Databases Takeaki Uno, Tatsuya Asai 2 3, Yuzo Uchida 2, and Hiroki Arimura 2 National Institute of Informatics, 2--2, Hitotsubashi,
A New Concise and Lossless Representation of Frequent Itemsets Using Generators and A Positive Border
A New Concise and Lossless Representation of Frequent Itemsets Using Generators and A Positive Border Guimei Liu a,b Jinyan Li a Limsoon Wong b a Institute for Infocomm Research, Singapore b School of
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Mining Frequent Patterns and Associations: Basic Concepts (Chapter 6) Huan Sun, CSE@The Ohio State University Slides adapted from Prof. Jiawei Han @UIUC, Prof. Srinivasan
Data Mining. Dr. Raed Ibraheem Hamed. University of Human Development, College of Science and Technology Department of Computer Science
Data Mining Dr. Raed Ibraheem Hamed University of Human Development, College of Science and Technology Department of Computer Science 2016 2017 Road map The Apriori algorithm Step 1: Mining all frequent
1 Frequent Pattern Mining
Decision Support Systems MEIC - Alameda 2010/2011 Homework #5 Due date: 31.Oct.2011 1 Frequent Pattern Mining 1. The Apriori algorithm uses prior knowledge about subset support properties. In particular,
Assignment 7 (Sol.) Introduction to Data Analytics Prof. Nandan Sudarsanam & Prof. B. Ravindran
Assignment 7 (Sol.) Introduction to Data Analytics Prof. Nandan Sudarsanam & Prof. B. Ravindran 1. Let X, Y be two itemsets, and let denote the support of itemset X. Then the confidence of the rule X Y,
Positive Borders or Negative Borders: How to Make Lossless Generator Based Representations Concise
Positive Borders or Negative Borders: How to Make Lossless Generator Based Representations Concise Guimei Liu 1,2 Jinyan Li 1 Limsoon Wong 2 Wynne Hsu 2 1 Institute for Infocomm Research, Singapore 2 School
An Efficient Algorithm for Enumerating Closed Patterns in Transaction Databases
An Efficient Algorithm for Enumerating Closed Patterns in Transaction Databases Takeaki Uno 1, Tatsuya Asai 2,,YuzoUchida 2, and Hiroki Arimura 2 1 National Institute of Informatics, 2-1-2, Hitotsubashi,
Chapter 6. Frequent Pattern Mining: Concepts and Apriori. Meng Jiang CSE 40647/60647 Data Science Fall 2017 Introduction to Data Mining
Chapter 6. Frequent Pattern Mining: Concepts and Apriori Meng Jiang CSE 40647/60647 Data Science Fall 2017 Introduction to Data Mining Pattern Discovery: Definition What are patterns? Patterns: A set of
Encyclopedia of Machine Learning Chapter Number Book CopyRight - Year 2010 Frequent Pattern. Given Name Hannu Family Name Toivonen
Book Title Encyclopedia of Machine Learning Chapter Number 00403 Book CopyRight - Year 2010 Title Frequent Pattern Author Particle Given Name Hannu Family Name Toivonen Suffix Email hannu.toivonen@cs.helsinki.fi
CSE 5243 INTRO. TO DATA MINING
CSE 5243 INTRO. TO DATA MINING Mining Frequent Patterns and Associations: Basic Concepts (Chapter 6) Huan Sun, CSE@The Ohio State University 10/17/2017 Slides adapted from Prof. Jiawei Han @UIUC, Prof.
.. Cal Poly CSC 466: Knowledge Discovery from Data Alexander Dekhtyar..
.. Cal Poly CSC 4: Knowledge Discovery from Data Alexander Dekhtyar.. Data Mining: Mining Association Rules Examples Course Enrollments Itemset. I = { CSC3, CSC3, CSC40, CSC40, CSC4, CSC44, CSC4, CSC44,
Selecting a Right Interestingness Measure for Rare Association Rules
Selecting a Right Interestingness Measure for Rare Association Rules Akshat Surana R. Uday Kiran P. Krishna Reddy Center for Data Engineering International Institute of Information Technology-Hyderabad
CS 584 Data Mining. Association Rule Mining 2
CS 584 Data Mining Association Rule Mining 2 Recall from last time: Frequent Itemset Generation Strategies Reduce the number of candidates (M) Complete search: M=2 d Use pruning techniques to reduce M
Un nouvel algorithme de génération des itemsets fermés fréquents
Un nouvel algorithme de génération des itemsets fermés fréquents Huaiguo Fu CRIL-CNRS FRE2499, Université d Artois - IUT de Lens Rue de l université SP 16, 62307 Lens cedex. France. E-mail: fu@cril.univ-artois.fr
Removing trivial associations in association rule discovery
Removing trivial associations in association rule discovery Geoffrey I. Webb and Songmao Zhang School of Computing and Mathematics, Deakin University Geelong, Victoria 3217, Australia Abstract Association
Data Mining: Concepts and Techniques. (3 rd ed.) Chapter 6
Data Mining: Concepts and Techniques (3 rd ed.) Chapter 6 Jiawei Han, Micheline Kamber, and Jian Pei University of Illinois at Urbana-Champaign & Simon Fraser University 2013 Han, Kamber & Pei. All rights
Approach for Rule Pruning in Association Rule Mining for Removing Redundancy
Approach for Rule Pruning in Association Rule Mining for Removing Redundancy Ashwini Batbarai 1, Devishree Naidu 2 P.G. Student, Department of Computer science and engineering, Ramdeobaba College of engineering
From statistics to data science. BAE 815 (Fall 2017) Dr. Zifei Liu
From statistics to data science BAE 815 (Fall 2017) Dr. Zifei Liu Zifeiliu@ksu.edu Why? How? What? How much? How many? Individual facts (quantities, characters, or symbols) The Data-Information-Knowledge-Wisdom
Data Mining Concepts & Techniques
Data Mining Concepts & Techniques Lecture No. 04 Association Analysis Naeem Ahmed Email: naeemmahoto@gmail.com Department of Software Engineering Mehran Univeristy of Engineering and Technology Jamshoro
Data Analytics Beyond OLAP. Prof. Yanlei Diao
Data Analytics Beyond OLAP Prof. Yanlei Diao OPERATIONAL DBs DB 1 DB 2 DB 3 EXTRACT TRANSFORM LOAD (ETL) METADATA STORE DATA WAREHOUSE SUPPORTS OLAP DATA MINING INTERACTIVE DATA EXPLORATION Overview of
Approximating a Collection of Frequent Sets
Approximating a Collection of Frequent Sets Foto Afrati National Technical University of Athens Greece afrati@softlab.ece.ntua.gr Aristides Gionis HIIT Basic Research Unit Dept. of Computer Science University
CS 412 Intro. to Data Mining
CS 412 Intro. to Data Mining Chapter 6. Mining Frequent Patterns, Association and Correlations: Basic Concepts and Methods Jiawei Han, Computer Science, Univ. Illinois at Urbana -Champaign, 2017 1 2 3
D B M G Data Base and Data Mining Group of Politecnico di Torino
Data Base and Data Mining Group of Politecnico di Torino Politecnico di Torino Association rules Objective extraction of frequent correlations or pattern from a transactional database Tickets at a supermarket
Association Rules. Fundamentals
Politecnico di Torino Politecnico di Torino 1 Association rules Objective extraction of frequent correlations or pattern from a transactional database Tickets at a supermarket counter Association rule
Unsupervised Learning. k-means Algorithm
Unsupervised Learning Supervised Learning: Learn to predict y from x from examples of (x, y). Performance is measured by error rate. Unsupervised Learning: Learn a representation from exs. of x. Learn
D B M G. Association Rules. Fundamentals. Fundamentals. Elena Baralis, Silvia Chiusano. Politecnico di Torino 1. Definitions.
Definitions Data Base and Data Mining Group of Politecnico di Torino Politecnico di Torino Itemset is a set including one or more items Example: {Beer, Diapers} k-itemset is an itemset that contains k
D B M G. Association Rules. Fundamentals. Fundamentals. Association rules. Association rule mining. Definitions. Rule quality metrics: example
Association rules Data Base and Data Mining Group of Politecnico di Torino Politecnico di Torino Objective extraction of frequent correlations or pattern from a transactional database Tickets at a supermarket
Lars Schmidt-Thieme, Information Systems and Machine Learning Lab (ISMLL), University of Hildesheim, Germany
Syllabus Fri. 21.10. (1) 0. Introduction A. Supervised Learning: Linear Models & Fundamentals Fri. 27.10. (2) A.1 Linear Regression Fri. 3.11. (3) A.2 Linear Classification Fri. 10.11. (4) A.3 Regularization
Distributed Mining of Frequent Closed Itemsets: Some Preliminary Results
Distributed Mining of Frequent Closed Itemsets: Some Preliminary Results Claudio Lucchese Ca Foscari University of Venice clucches@dsi.unive.it Raffaele Perego ISTI-CNR of Pisa perego@isti.cnr.it Salvatore
FP-growth and PrefixSpan
FP-growth and PrefixSpan n Challenges of Frequent Pattern Mining n Improving Apriori n Fp-growth n Fp-tree n Mining frequent patterns with FP-tree n PrefixSpan Challenges of Frequent Pattern Mining n Challenges
Chapters 6 & 7, Frequent Pattern Mining
CSI 4352, Introduction to Data Mining Chapters 6 & 7, Frequent Pattern Mining Young-Rae Cho Associate Professor Department of Computer Science Baylor University CSI 4352, Introduction to Data Mining Chapters
Mining Class-Dependent Rules Using the Concept of Generalization/Specialization Hierarchies
Mining Class-Dependent Rules Using the Concept of Generalization/Specialization Hierarchies Juliano Brito da Justa Neves 1 Marina Teresa Pires Vieira {juliano,marina}@dc.ufscar.br Computer Science Department
CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu 2/26/2013 Jure Leskovec, Stanford CS246: Mining Massive Datasets, http://cs246.stanford.edu 2 More algorithms
Data Mining and Analysis: Fundamental Concepts and Algorithms
Data Mining and Analysis: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA
CS 484 Data Mining. Association Rule Mining 2
CS 484 Data Mining Association Rule Mining 2 Review: Reducing Number of Candidates Apriori principle: If an itemset is frequent, then all of its subsets must also be frequent Apriori principle holds due
CS5112: Algorithms and Data Structures for Applications
CS5112: Algorithms and Data Structures for Applications Lecture 19: Association rules Ramin Zabih Some content from: Wikipedia/Google image search; Harrington; J. Leskovec, A. Rajaraman, J. Ullman: Mining
10/19/2017 MIST.6060 Business Intelligence and Data Mining 1. Association Rules
10/19/2017 MIST6060 Business Intelligence and Data Mining 1 Examples of Association Rules Association Rules Sixty percent of customers who buy sheets and pillowcases order a comforter next, followed by
Unit II Association Rules
Unit II Association Rules Basic Concepts Frequent Pattern Analysis Frequent pattern: a pattern (a set of items, subsequences, substructures, etc.) that occurs frequently in a data set Frequent Itemset
CS4445 Data Mining and Knowledge Discovery in Databases. B Term 2014 Solutions Exam 2 - December 15, 2014
CS4445 Data Mining and Knowledge Discovery in Databases. B Term 2014 Solutions Exam 2 - December 15, 2014 Prof. Carolina Ruiz Department of Computer Science Worcester Polytechnic Institute NAME: Prof.
Association Analysis Part 2. FP Growth (Pei et al 2000)
Association Analysis art 2 Sanjay Ranka rofessor Computer and Information Science and Engineering University of Florida F Growth ei et al 2 Use a compressed representation of the database using an F-tree
Discovering Threshold-based Frequent Closed Itemsets over Probabilistic Data
Discovering Threshold-based Frequent Closed Itemsets over Probabilistic Data Yongxin Tong #1, Lei Chen #, Bolin Ding *3 # Department of Computer Science and Engineering, Hong Kong Univeristy of Science
ASSOCIATION ANALYSIS FREQUENT ITEMSETS MINING. Alexandre Termier, LIG
ASSOCIATION ANALYSIS FREQUENT ITEMSETS MINING, LIG M2 SIF DMV course 207/208 Market basket analysis Analyse supermarket s transaction data Transaction = «market basket» of a customer Find which items are
Association Rules. Acknowledgements. Some parts of these slides are modified from. n C. Clifton & W. Aref, Purdue University
Association Rules CS 5331 by Rattikorn Hewett Texas Tech University 1 Acknowledgements Some parts of these slides are modified from n C. Clifton & W. Aref, Purdue University 2 1 Outline n Association Rule
CS341 info session is on Thu 3/1 5pm in Gates415. CS246: Mining Massive Datasets Jure Leskovec, Stanford University
CS341 info session is on Thu 3/1 5pm in Gates415 CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu 2/28/18 Jure Leskovec, Stanford CS246: Mining Massive Datasets,
COMP 5331: Knowledge Discovery and Data Mining
COMP 5331: Knowledge Discovery and Data Mining Acknowledgement: Slides modified by Dr. Lei Chen based on the slides provided by Jiawei Han, Micheline Kamber, and Jian Pei And slides provide by Raymond
Handling a Concept Hierarchy
Food Electronics Handling a Concept Hierarchy Bread Milk Computers Home Wheat White Skim 2% Desktop Laptop Accessory TV DVD Foremost Kemps Printer Scanner Data Mining: Association Rules 5 Why should we
Association Rule. Lecturer: Dr. Bo Yuan. LOGO
Association Rule Lecturer: Dr. Bo Yuan LOGO E-mail: yuanb@sz.tsinghua.edu.cn Overview Frequent Itemsets Association Rules Sequential Patterns 2 A Real Example 3 Market-Based Problems Finding associations
CSE-4412(M) Midterm. There are five major questions, each worth 10 points, for a total of 50 points. Points for each sub-question are as indicated.
22 February 2007 CSE-4412(M) Midterm p. 1 of 12 CSE-4412(M) Midterm Sur / Last Name: Given / First Name: Student ID: Instructor: Parke Godfrey Exam Duration: 75 minutes Term: Winter 2007 Answer the following
Pushing Tougher Constraints in Frequent Pattern Mining
Pushing Tougher Constraints in Frequent Pattern Mining Francesco Bonchi 1 and Claudio Lucchese 2 1 Pisa KDD Laboratory, ISTI - C.N.R., Area della Ricerca di Pisa, Italy 2 Department of Computer Science,
OPPA European Social Fund Prague & EU: We invest in your future.
OPPA European Social Fund Prague & EU: We invest in your future. Frequent itemsets, association rules Jiří Kléma Department of Cybernetics, Czech Technical University in Prague http://ida.felk.cvut.cz
A Global Constraint for Closed Frequent Pattern Mining
A Global Constraint for Closed Frequent Pattern Mining N. Lazaar 1, Y. Lebbah 2, S. Loudni 3, M. Maamar 1,2, V. Lemière 3, C. Bessiere 1, P. Boizumault 3 1 LIRMM, University of Montpellier, France 2 LITIO,
FREQUENT PATTERN SPACE MAINTENANCE: THEORIES & ALGORITHMS
FREQUENT PATTERN SPACE MAINTENANCE: THEORIES & ALGORITHMS FENG MENGLING SCHOOL OF ELECTRICAL & ELECTRONIC ENGINEERING NANYANG TECHNOLOGICAL UNIVERSITY 2009 FREQUENT PATTERN SPACE MAINTENANCE: THEORIES
Apriori algorithm. Seminar of Popular Algorithms in Data Mining and Machine Learning, TKK. Presentation Lauri Lahti
Apriori algorithm Seminar of Popular Algorithms in Data Mining and Machine Learning, TKK Presentation 12.3.2008 Lauri Lahti Association rules Techniques for data mining and knowledge discovery in databases
Layered critical values: a powerful direct-adjustment approach to discovering significant patterns
Mach Learn (2008) 71: 307 323 DOI 10.1007/s10994-008-5046-x TECHNICAL NOTE Layered critical values: a powerful direct-adjustment approach to discovering significant patterns Geoffrey I. Webb Received:
Data Mining and Knowledge Discovery. Petra Kralj Novak. 2011/11/29
Data Mining and Knowledge Discovery Petra Kralj Novak Petra.Kralj.Novak@ijs.si 2011/11/29 1 Practice plan 2011/11/08: Predictive data mining 1 Decision trees Evaluating classifiers 1: separate test set,
Mining Data Streams. The Stream Model. The Stream Model Sliding Windows Counting 1 s
Mining Data Streams The Stream Model Sliding Windows Counting 1 s 1 The Stream Model Data enters at a rapid rate from one or more input ports. The system cannot store the entire stream. How do you make
Mining Free Itemsets under Constraints
Mining Free Itemsets under Constraints Jean-François Boulicaut Baptiste Jeudy Institut National des Sciences Appliquées de Lyon Laboratoire d Ingénierie des Systèmes d Information Bâtiment 501 F-69621
1 [15 points] Frequent Itemsets Generation With Map-Reduce
![1 [15 points] Frequent Itemsets Generation With Map-Reduce](/thumbs/72/67172903.jpg "1 [15 points] Frequent Itemsets Generation With Map-Reduce")
Data Mining Learning from Large Data Sets Final Exam Date: 15 August 2013 Time limit: 120 minutes Number of pages: 11 Maximum score: 100 points You can use the back of the pages if you run out of space.
BAYESIAN MIXTURE MODELS FOR FREQUENT ITEMSET MINING
BAYESIAN MIXTURE MODELS FOR FREQUENT ITEMSET MINING A thesis submitted to the University of Manchester for the degree of Doctor of Philosophy in the Faculty of Engineering and Physical Sciences 2012 By
Regression and Correlation Analysis of Different Interestingness Measures for Mining Association Rules
International Journal of Innovative Research in Computer Scien & Technology (IJIRCST) Regression and Correlation Analysis of Different Interestingness Measures for Mining Association Rules Mir Md Jahangir
Introduction to Data Mining
Introduction to Data Mining Lecture #12: Frequent Itemsets Seoul National University 1 In This Lecture Motivation of association rule mining Important concepts of association rules Naïve approaches for
Machine Learning: Pattern Mining
Machine Learning: Pattern Mining Information Systems and Machine Learning Lab (ISMLL) University of Hildesheim Wintersemester 2007 / 2008 Pattern Mining Overview Itemsets Task Naive Algorithm Apriori Algorithm
Association Rule Mining on Web
Association Rule Mining on Web What Is Association Rule Mining? Association rule mining: Finding interesting relationships among items (or objects, events) in a given data set. Example: Basket data analysis
732A61/TDDD41 Data Mining - Clustering and Association Analysis
732A61/TDDD41 Data Mining - Clustering and Association Analysis Lecture 6: Association Analysis I Jose M. Peña IDA, Linköping University, Sweden 1/14 Outline Content Association Rules Frequent Itemsets
Frequent Pattern Mining: Exercises
Frequent Pattern Mining: Exercises Christian Borgelt School of Computer Science tto-von-guericke-university of Magdeburg Universitätsplatz 2, 39106 Magdeburg, Germany christian@borgelt.net http://www.borgelt.net/
On Condensed Representations of Constrained Frequent Patterns
Under consideration for publication in Knowledge and Information Systems On Condensed Representations of Constrained Frequent Patterns Francesco Bonchi 1 and Claudio Lucchese 2 1 KDD Laboratory, ISTI Area
Pattern Space Maintenance for Data Updates. and Interactive Mining
Pattern Space Maintenance for Data Updates and Interactive Mining Mengling Feng, 1,3,4 Guozhu Dong, 2 Jinyan Li, 1 Yap-Peng Tan, 1 Limsoon Wong 3 1 Nanyang Technological University, 2 Wright State University
Meelis Kull Autumn Meelis Kull - Autumn MTAT Data Mining - Lecture 05
Meelis Kull meelis.kull@ut.ee Autumn 2017 1 Sample vs population Example task with red and black cards Statistical terminology Permutation test and hypergeometric test Histogram on a sample vs population
Privacy Preserving Frequent Itemset Mining. Workshop on Privacy, Security, and Data Mining ICDM - Maebashi City, Japan December 9, 2002
Privacy Preserving Frequent Itemset Mining Stanley R. M. Oliveira 1,2 Osmar R. Zaïane 2 1 oliveira@cs.ualberta.ca zaiane@cs.ualberta.ca Embrapa Information Technology Database Systems Laboratory Andre
Correlation Preserving Unsupervised Discretization. Outline
Correlation Preserving Unsupervised Discretization Jee Vang Outline Paper References What is discretization? Motivation Principal Component Analysis (PCA) Association Mining Correlation Preserving Discretization
Outline. Fast Algorithms for Mining Association Rules. Applications of Data Mining. Data Mining. Association Rule. Discussion
Outline Fast Algorithms for Mining Association Rules Rakesh Agrawal Ramakrishnan Srikant Introduction Algorithm Apriori Algorithm AprioriTid Comparison of Algorithms Conclusion Presenter: Dan Li Discussion:
Mining Molecular Fragments: Finding Relevant Substructures of Molecules
Mining Molecular Fragments: Finding Relevant Substructures of Molecules Christian Borgelt, Michael R. Berthold Proc. IEEE International Conference on Data Mining, 2002. ICDM 2002. Lecturers: Carlo Cagli
Free-sets : a Condensed Representation of Boolean Data for the Approximation of Frequency Queries
Free-sets : a Condensed Representation of Boolean Data for the Approximation of Frequency Queries To appear in Data Mining and Knowledge Discovery, an International Journal c Kluwer Academic Publishers
Chapter 5-2: Clustering
Chapter 5-2: Clustering Jilles Vreeken Revision 1, November 20 th typo s fixed: dendrogram Revision 2, December 10 th clarified: we do consider a point x as a member of its own ε-neighborhood 12 Nov 2015
COMP 5331: Knowledge Discovery and Data Mining
COMP 5331: Knowledge Discovery and Data Mining Acknowledgement: Slides modified by Dr. Lei Chen based on the slides provided by Tan, Steinbach, Kumar And Jiawei Han, Micheline Kamber, and Jian Pei 1 10
Mining Non-Redundant Association Rules
Data Mining and Knowledge Discovery, 9, 223 248, 2004 c 2004 Kluwer Academic Publishers. Manufactured in The Netherlands. Mining Non-Redundant Association Rules MOHAMMED J. ZAKI Computer Science Department,
Quantitative Association Rule Mining on Weighted Transactional Data
Quantitative Association Rule Mining on Weighted Transactional Data D. Sujatha and Naveen C. H. Abstract In this paper we have proposed an approach for mining quantitative association rules. The aim of
Lecture Notes for Chapter 6. Introduction to Data Mining
Data Mining Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004
Association Analysis: Basic Concepts. and Algorithms. Lecture Notes for Chapter 6. Introduction to Data Mining
Association Analysis: Basic Concepts and Algorithms Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Association
DATA MINING - 1DL360
DATA MINING - 1DL36 Fall 212" An introductory class in data mining http://www.it.uu.se/edu/course/homepage/infoutv/ht12 Kjell Orsborn Uppsala Database Laboratory Department of Information Technology, Uppsala
Lecture Notes for Chapter 6. Introduction to Data Mining. (modified by Predrag Radivojac, 2017)
Lecture Notes for Chapter 6 Introduction to Data Mining by Tan, Steinbach, Kumar (modified by Predrag Radivojac, 27) Association Rule Mining Given a set of transactions, find rules that will predict the
Alternative Approach to Mining Association Rules
Alternative Approach to Mining Association Rules Jan Rauch 1, Milan Šimůnek 1 2 1 Faculty of Informatics and Statistics, University of Economics Prague, Czech Republic 2 Institute of Computer Sciences,
Statistical Privacy For Privacy Preserving Information Sharing
Statistical Privacy For Privacy Preserving Information Sharing Johannes Gehrke Cornell University http://www.cs.cornell.edu/johannes Joint work with: Alexandre Evfimievski, Ramakrishnan Srikant, Rakesh
Data Mining and Analysis: Fundamental Concepts and Algorithms
Data Mining and Analysis: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA
Constraint-Based Rule Mining in Large, Dense Databases
Appears in Proc of the 5th Int l Conf on Data Engineering, 88-97, 999 Constraint-Based Rule Mining in Large, Dense Databases Roberto J Bayardo Jr IBM Almaden Research Center bayardo@alummitedu Rakesh Agrawal
On the Complexity of Mining Itemsets from the Crowd Using Taxonomies
On the Complexity of Mining Itemsets from the Crowd Using Taxonomies Antoine Amarilli 1,2 Yael Amsterdamer 1 Tova Milo 1 1 Tel Aviv University, Tel Aviv, Israel 2 École normale supérieure, Paris, France
Processing Count Queries over Event Streams at Multiple Time Granularities
Processing Count Queries over Event Streams at Multiple Time Granularities Aykut Ünal, Yücel Saygın, Özgür Ulusoy Department of Computer Engineering, Bilkent University, Ankara, Turkey. Faculty of Engineering
Free-Sets: A Condensed Representation of Boolean Data for the Approximation of Frequency Queries
Data Mining and Knowledge Discovery, 7, 5 22, 2003 c 2003 Kluwer Academic Publishers. Manufactured in The Netherlands. Free-Sets: A Condensed Representation of Boolean Data for the Approximation of Frequency
DATA MINING LECTURE 3. Frequent Itemsets Association Rules
DATA MINING LECTURE 3 Frequent Itemsets Association Rules This is how it all started Rakesh Agrawal, Tomasz Imielinski, Arun N. Swami: Mining Association Rules between Sets of Items in Large Databases.
SPATIO-TEMPORAL PATTERN MINING ON TRAJECTORY DATA USING ARM
SPATIO-TEMPORAL PATTERN MINING ON TRAJECTORY DATA USING ARM S. Khoshahval a *, M. Farnaghi a, M. Taleai a a Faculty of Geodesy and Geomatics Engineering, K.N. Toosi University of Technology, Tehran, Iran
Department of Computer Science, University of Waikato, Hamilton, New Zealand. 2
Predicting Apple Bruising using Machine Learning G.Holmes 1, S.J.Cunningham 1, B.T. Dela Rue 2 and A.F. Bollen 2 1 Department of Computer Science, University of Waikato, Hamilton, New Zealand. 2 Lincoln
Detecting Anomalous and Exceptional Behaviour on Credit Data by means of Association Rules. M. Delgado, M.D. Ruiz, M.J. Martin-Bautista, D.
Detecting Anomalous and Exceptional Behaviour on Credit Data by means of Association Rules M. Delgado, M.D. Ruiz, M.J. Martin-Bautista, D. Sánchez 18th September 2013 Detecting Anom and Exc Behaviour on
Frequent Itemsets and Association Rule Mining. Vinay Setty Slides credit:
Frequent Itemsets and Association Rule Mining Vinay Setty vinay.j.setty@uis.no Slides credit: http://www.mmds.org/ Association Rule Discovery Supermarket shelf management Market-basket model: Goal: Identify
DATA MINING - 1DL105, 1DL111
1 DATA MINING - 1DL105, 1DL111 Fall 2007 An introductory class in data mining http://user.it.uu.se/~udbl/dut-ht2007/ alt. http://www.it.uu.se/edu/course/homepage/infoutv/ht07 Kjell Orsborn Uppsala Database
Slide source: Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University.
Slide source: Mining of Massive Datasets Jure Leskovec, Anand Rajaraman, Jeff Ullman Stanford University http://www.mmds.org #1: C4.5 Decision Tree - Classification (61 votes) #2: K-Means - Clustering
Realistic synthetic data for testing association rule mining algorithms for market basket databases
Realistic synthetic data for testing association rule mining algorithms for market basket databases ABSTRACT Association rule mining (ARM) is an important subtask in Knowledge Discovery in Databases. Existing
Interesting Patterns. Jilles Vreeken. 15 May 2015
Interesting Patterns Jilles Vreeken 15 May 2015 Questions of the Day What is interestingness? what is a pattern? and how can we mine interesting patterns? What is a pattern? Data Pattern y = x - 1 What
On Minimal Infrequent Itemset Mining
On Minimal Infrequent Itemset Mining David J. Haglin and Anna M. Manning Abstract A new algorithm for minimal infrequent itemset mining is presented. Potential applications of finding infrequent itemsets
Fundamentals of Mathematics (MATH 1510)
Fundamentals of Mathematics () Instructor: Email: shenlili@yorku.ca Department of Mathematics and Statistics York University February 3-5, 2016 Outline 1 growth (doubling time) Suppose a single bacterium