Belief Classification Approach based on Dynamic Core for Web Mining database

Size: px

Start display at page:

Download "Belief Classification Approach based on Dynamic Core for Web Mining database"

Elfrieda Bradley
5 years ago
Views:

1 Third international workshop on Rough Set Theory RST 11 Milano, Italy September 14 16, 2010 Belief Classification Approach based on Dynamic Core for Web Mining database Salsabil Trabelsi Zied Elouedi Larodec, Institut Supérieur de gestion de Tunis, Tunisie Pawan Lingras Saint Mary's University, Halifax, Canada 1

2 Plan Introduction Rough set theory Belief function theory BRSC-GDT based on dynamic core Experimentations on web usage database Conclusion 2

3 Introduction The World Wide Web is a fertile area for data mining research. The Web usage mining has seen a rapid increase in interest. The Rough Sets proposed by Pawlak (1982) constitutes a sound basis for decision support system and data mining. J The models generated by rough sets take the form of "IF-THEN". J This classifier performs feature selection before generation rules. J This method can avoid many iterations. J It is a efficient technique that try to produce a minimal and a significant set of decision rules. It is a successful technique of classification applied in several real-world applications like web usage area. 3

4 Introduction The web usage databases may be characterized by incomplete or uncertain data. We have applied a new approach of classification denoted Belief Rough Set Classifier based on generalization distribution table (BRSC-GDT) to discover hidden patterns from uncertain and real web usage database. This classifier was able to learn decision rules from uncertain data. The uncertainty appears only in decision attributes and is handled by the belief function theory. The web usage mining dataset was obtained from web access logs of the introductory computing science course at Saint Mary's University. To better handle the noise and the uncertain issues existing in the chosen web usage mining database, the feature selection step used to construct our BRSC GDT will be based on dynamic core. Decision rules induced by means of dynamic core are more appropriate to classify new objects. 4

5 Rough set theory Basic concepts Decision Table (DT) Indiscernibility relation Set approximation Positive region & Dependency of attributes Reduct & Core Value reduct & Value Core Dynamic reduct & Dynamic Core 5

6 Belief function theory Basic concepts Frame of discernment (Θ) Basic belief assignment (bba) Belief function (bel) Plausibility functions (pl) The Rules of combination Discounting The pignistic transformation 6

7 BRSC-GDT based on dynamic core New belief classification technique based on the hyprid system GDT-RS (combination of Generalization distribution Table (GDT) and Rough Set (RS) methodology).

8 BRSC-GDT based on dynamic core based on the redefined basic concepts of Rough Set theory under Belief function framework

9 BRSC-GDT based on dynamic core Basic concepts of rough sets under uncertainty 1) Uncertain Decision Table (UDT) 2) Tolerance relation 3) Set approximation 4) Positive region & Dependency of attributes 5) Reduct & Core 6) Value Reduct & Value Core 7) Dynamic reduct & Dynamic core 9

10 Basic concepts of rough sets under uncertainty Uncertain decision table Our uncertain decision system A= (U, C {ud}) U contains n objects o j C = {c 1,c 2,.c n } set of certain condition attributes ud uncertain decision attribute represented by a bba m j for the object o j Patients Headache Muscle - pain Temperature Flu o 1 yes yes very high m 1 ({yes})=0.95 m 1 (Θ)=0.05 o 2 yes no high m 2 ({no})=1 o 3 yes yes normal m 3 ({yes})=0.5 m 3 (Θ)=0.5 o 4 no yes normal m 4 ({no})=0.4 m 4 (Θ)=0.6 o 5 no yes normal m 5 ({no})=1 o 6 yes no high m 6 ({no})=0.95 m 6 (Θ)=0.05 o 7 no yes very high m 7 ({yes})=1 o 8 no yes high m 8 ({yes})=0.9 m 8 (Θ)=0.1 For o 3, 0.5 of beliefs are exactly committed to decision ud 1 =yes, whereas 0.5 of beliefs is assigned to Θ. 10

11 Basic concepts of rough sets under uncertainty Tolerance relation J Indiscernibility relation for the condition attributes is the same as in the certain case. L Indiscernibility relation for the decision attribute is not the same. The decision is represented by a bba. We need to assign each object characterized by a bba m j to the right equivalence class ud i The idea is to use the distance between two bba s. (m j and a certain bba m({ud i })=1) We choose the distance measure proposed by (Bosse et al, 2001) which satisfies some properties such as non-negativity, non-degeneracy and symmetry. For every uncertain decision value ud i X i = {o j dist(m, m j ) <1- threshold} such that m({ud i })=1 The tolerance relation based on decision: IND {ud} = U/{ud} ={X i ud i Θ } 11

12 Basic concepts of rough sets under uncertainty Tolerance relation For the uncertain decision value ud 1 =yes Dist(m, m 1 ) < 1-threshold Dist(m, m 3 ) < 1-threshold Dist(m, m 4 ) < 1-threshold Dist(m, m 7 ) < 1-threshold Dist(m, m 8 ) < 1-threshold So, X 1 ={o 1, o 3, o 4, o 7, o 8 } For the uncertain decision value ud 2 =no Dist(m, m 2 ) < 1-threshold Dist(m, m 3 ) < 1-threshold Dist(m, m 4 ) < 1-threshold Dist(m, m 5 ) < 1-threshold Dist(m, m 6 ) < 1-threshold So, X 2 ={o 2,o 3,o 4,o 5,o 6 } U/{ud}={{o 1, o 3, o 4, o 7, o 8 }, {o 2, o 3, o 4, o 5, o 6 }} 12

13 Basic concepts of rough sets under uncertainty Set approximation To compute the new lower and upper approximations, we follow two steps: 1. For each equivalence class from U/C, combined their bba using operator mean to obtain m [oj]c 2. For each equivalence class X i from U/{ud}, compute the lower and upper approximation: CX i ={o j [o j ] C X and dist( m, m[o j ] C ) threshold} Such that m({ud i })=1 C X i ={o j [o j ] C X } 13

14 Basic concepts of rough sets under uncertainty Set approximation 1. For each equivalence class from U/C ={{o 1 }, {o 2, o 6 }, {o 3 }, {o 4, o 5 }, {o 7 }, {o 8 }}, combined their bba of using the mean operator: The combined bba for {o 4, o 5 } Patient m({yes}) m({no}) m(θ) o O.6 o m 4, For each equivalence class from U/{ud}={{o 1,o 3, o 4,o 7, o 8 }, {o 2,o 3,o 4,o 5,o 6 }}, compute the lower and upper approximation ud 1 =yes Let X 1 ={o 1, o 3, o 4, o 7, o 8 } ud 2 =no Let X 2 ={o 2, o 3, o 4, o 5, o 6 } CX 1 ={o 1,o 7, o 8 } CX 2 ={o 2,o 6 } CX 1 ={o 1, o 3, o 4, o 5, o 7,o 8 } CX 2 ={o 2, o 3, o 4, o 5, o 6 } NB C ={o 3, o 4, o 5 } 14

15 Basic concepts of rough sets under uncertainty Positive region & Dependency of attributes Positive region: UPos C (A, {ud})= CX X U/{ud} UPos C (A,{ud})={o 1,o 2,o 6,o 7,o 8 } Dependency of attributes k=γ C (A, {ud})= UPos C (A, {ud}) U k=γ C (A,{ud})=5/8 15

16 Basic concepts of rough sets under uncertainty Reduct & Core Reduct: Using the new formalism of positive region, we can find the reduct of C as a minimal set of attributes B C such that: UPos B (A, {ud}) = UPos C (A, {ud}) Core: The set of indispensable attributes It is included in every reduct. UCore C (A,{ud}) = URed C (A,{ud}) 16

17 Basic concepts of rough sets under uncertainty UPos C ({ud})={o 1,o 2,o 6,o 7,o 8 } UPos {Headache,Muscle-pain} ({ud})= {o 2,o 6 } UPos {Headache,Temperature} ({ud})= {o 1,o 2,o 6,o 7,o 8 } UPos {Muscle-pain,Temperature} ({ud})= {o 1,o 2,o 6,o 7,o 8 } UPos {Headache} ({ud})= UPos {Muscle-pain} ({ud})= UPos {Temperature} ({ud})= {o 1,o 7 } Reduct & Core There are two reducts: 1. {Headache, Temperature} 2. {Muscle-pain, Temperature} The Core {Headache, Temperature} {Muscle-pain, Temperature} = {Temperature} 17

18 Basic concepts of rough sets under uncertainty Value Reduct & Value Core Value Reduct: We also need to redefine the concept of value reduct follows: for each belief decision rule of the form: If C(o j ) then m j For all B C, let X= {o k B(o j )=B(o k )} If Max(dist(m j, m[o k ] B )) < threshold then B is a reduct value of o j Value Core: The set of indispensable attributes values It is included in every reduct. UCore j C (A,{ud}) = URedj C (A,{ud}) 18

19 Basic concepts of rough sets under uncertainty Dynamic reduct & dynamic core If A=(U, C {ud}) is an uncertain decision table, then any system A =(U, C {ud}) such that U U is called a sub-table of A. Let F be a family of sub-tables of A UDR(A, F)= URED (A, {ud}) RED (A, {ud}) B F Any element of UDR(A,F) is called an F-dynamic reduct of A. A reduct of A is dynamic if it is also a reduct of all sub-tables from a given family F 19

20 Basic concepts of rough sets under uncertainty Dynamic reduct & dynamic core If A=(U, C {ud}) is an uncertain decision table, then any system A =(U, C {ud}) such that U U is called a sub-table of A. Let F be a family of sub-tables of A. DCore(A, F)= UCore (A, {ud}) UCore (A, {ud}) A F Any element of DCore(A, F) is called an F-dynamic core of A. A core of A is dynamic if it is also a core of all sub-tables from a given family F. 20

21 Basic concepts of rough sets under uncertainty Dynamic reduct & dynamic core We divide our uncertain decision table to two sub-tables A and A to obtain a family F of sub-decision table. A contains the objects o 1, o 2, o 3, o 4 A contains the objects o 5, o 6, o 7, o 8 The two sub-tables have the same reducts as the whole decision table A. So, the sub-sets {Headache,Temperature} and {Musclepain,Temperature} are dynamic reducts relative to the chosen family F. The core Temperature is also the dymainc core relative to the chosen family F. 21

22 Construction procedure of BRSC-GDT based on dynamic core Step 1: Creation of the GDT: This step can be omitted because the prior distribution of a generalization can be calculated Step 2: Feature selection based on dynamic Core: We remove the superfluous condition attributes that are not in reducts. Therefore it is important to search stable reduct containing dynamic core (if it is also a core of all subtables from a given family F). Step 3: Definition of the compound object: considering the indiscernibily classes with respect to the subset B of the condition attribute as one object, called the compound object. Where B is the reduct containning the dynamic core 22

23 Construction procedure of BRSC-GDT based on dynamic core Step 4: Elimination of the contradictory compound objects: Removing the compound objects where the decision of their composing objects are not very similar. Step 5: Minimal description length of decision rule: Determining the sets of all reduct values of the compound object. Step 6: Selection of the best rules: Constructing the decision rules obtained from the local reducts for each compound object. Then, we select only the best rule having the best strength. 23

24 Experimentations on web usage database

25 Data description The study data were obtained from web access logs of the introductory computing science course at Saint Mary's University. The course is ``Introduction to Computing Science and Programming" offered in the first term of the first year. The initial number of students in the course was 180. The number reduced over the course of the semester to 140 students. The students in the course come from a wide variety of backgrounds. Lingras and West (2004) showed that visits from students attending the first course could fall into one of the following three categories (decision values): 1. Studious 2. Crammers 3. Workers 25

26 Data description Based on some observations, it was decided to use the following attributes for representing each visitor to the log web site: 1. On campus/off campus access 2. Day time/night time access 3. Access during lab/class days or non-lab/class days 4. Number of hits 5. Number of class-notes downloads Total visits were The visits where no class-notes were downloaded were eliminated, since these visits correspond to either casual visitors or workers. Elimination of outliers and visits from the search engines further reduced the sizes of the data set to Instead of representing an object as belonging to a cluster ud i (decision value), we also associated a degree of belief m j for the object o j. 26

27 Evaluation criteria & Experimental results The relevant criteria used to judge the performance of our classifier on the web usage database: 1. The accuracy represents the percent of correct classification (PCC). 2. The size represents the number of the generated belief decision rules. 3. The time requirement represents the number of seconds for the construction procedure. The experimental results relative to BRSC-GDT based on dynamic core Approach PCC Certain Uncertain Size Time requirement BRSC-GDT

28 Conclusion We have introduced the basic concept of rough set theory We have introduced the basic concept of belief function theory We have described the adaptation the basic concepts of rough sets under the belief function framework We have detailed the main steps to construct Belief classification approach based on dynamic core We have described our real and uncertain web usage database We have reported the experimental results on the real web user database. Based on three evaluation criteria : time requirement, size and accuracy. 28

A novel k-nn approach for data with uncertain attribute values

A novel k-nn approach for data with uncertain attribute values A novel -NN approach for data with uncertain attribute values Asma Trabelsi 1,2, Zied Elouedi 1, and Eric Lefevre 2 1 Université de Tunis, Institut Supérieur de Gestion de Tunis, LARODEC, Tunisia trabelsyasma@gmail.com,zied.elouedi@gmx.fr