Investigating Measures of Association by Graphs and Tables of Critical Frequencies

Transcription

1 Investigating Measures of Association by Graphs Investigating and Tables Measures of Critical of Association Frequencies by Graphs and Tables of Critical Frequencies Martin Ralbovský, Jan Rauch University Martin Ralbovský, of Economics, Jan Prague Rauch W. Churchill Sq. 4, Praha 3, Czech Republic University of Economics, Prague, W. Churchill Sq. 4, Praha 3, Czech Republic Abstract. There is lot of effort to find suitable measures of interestingness of association rules. The most known measures are confidence and support but there are tens of additional ones. Each association measure can be understood as a function of four independent variables. These variables correspond to frequencies from a fourfold contingency table of antecedent and succedent. A natural way to investigate functions is to study their graphs. However it is hard to deal with graphs of functions of four independent variables; thus graphs are generally not suitable to study association measures. We show that tables of critical frequencies can be used to overcome this difficulty for some important classes of association measures. We give an overview of important classes of association measures and then we show how the graphs of tables of critical frequencies describe behavior of corresponding association measures in a reasonable way. 1 Introduction There are lot of papers dealing with various aspects of association measures; see e.g. [5 7]. The goal is to find optimal criterion of truthfulness of the association rule ϕ ψ expressing a relation of Boolean attributes ϕ and ψ in given data. We suppose to have data matrix M with two Boolean columns corresponding to ϕ and ψ. Both ϕ and ψ can be derived from the other, usually non Boolean, columns. The whole situation is fully described by the four-fold contingency table 4ft(ϕ, ψ, M) (the 4ft table for short) of ϕ and ψ in the data matrix M. It is the quadruple a, b, c, d of natural numbers such that a is the number of rows of M satisfying both ϕ and ψ, b is the number of rows of M satisfying ϕ and not satisfying ψ etc.; see Table 1. Table 1. 4ft table 4ft(ϕ, ψ, M) of ϕ and ψ in M M ψ ψ ϕ a b ϕ c d c Václav Snášel (Ed.): Znalosti 2008, pp , ISBN FIIT STU Bratislava, Ústav informatiky a softvérového inžinierstva, 2008.

2 172 Martin Ralbovský, Jan Rauch There are various requirements concerning relation of ϕ and ψ. We can try to express the classical association rule with confidence and support, some relation described by a simple condition concerning frequencies a, b, c, d from the 4ft table 4ft(ϕ, ψ, M) or even a relation corresponding to a statistical hypothesis test. We call the symbol as 4ft-quantifier [11]. The truthfulness of the rule ϕ ψ in data matrix M is often defined such that ϕ ψ is true if and only if F (a, b, c, d) p or F (a, b, c, d) p where a, b, c, d = 4ft(ϕ, ψ, M). The function F and the parameter p are given by the 4ft-quantifier. We call the function F the evaluation function of 4ft-quantifier. Examples of 4ftquantifiers and their evaluation functions are in section 2. Properties of the 4ft-quantifier are fully described by behavior of its evaluation function F. A natural way to investigate functions is to study their graphs. Some results in this direction are presented in section 2. The evaluation function is the function of four independent variables; thus graphs are not suitable to study association measures. This problem can be partly solved by graphs of tables of critical frequencies that describe behavior of important 4ft-quantifiers in a comprehensive way; see section 4. Graphs of tables of critical frequencies are used in section 5 to describe behavior of several important 4ft-quantifiers. Section 6 compares our approach to related work. Conclusions and further research are presented in section 7. 2 Evaluation Functions of 4ft-quantifiers The association rule is the expression ϕ ψ where ϕ and ψ are Boolean attributes and is the 4ft-quantifier. The rule ϕ ψ is true in the analyzed data matrix M if the condition related to the 4ft-quantifier is satisfied in the 4ft table 4ft(ϕ, ψ, M) of ϕ and ψ in M; see Tab. 1 and Section 1. Some important 4ft-quantifiers are presented below. We use a, b, c, d see Tab. 1. In addition we use n = a + b + c + d. We also define the evaluation function F introduced in Section 1 for each presented 4ft-quantifier. The 4ft-quantifier p of founded implication is for 0 < p 1 defined in [3] by the condition a a+b p. The rule ϕ p ψ says that at least 100p per cent of objects of M satisfying ϕ satisfy also ψ. The evaluation function F p of p is defined such that F p (a, b, c, d) = a a+b, thus we write only F p (a, b) instead of F p (a, b, c, d) Remark: The 4ft-quantifier of founded implication is actually defined by the condition a a+b p a B where 0 < p 1, and B > 0. We omit the parameter B because of we are interested in the graph of the function F p (a, b). The same is true for the additional 4ft-quantifiers in this paper. The 4ft-quantifier! of lower critical implication is for 0 < p 1 and 0 < α < 0.5 defined in [3] by the condition a+b ( a+b ) i=a i p i (1 p) a+b i α. The rule ϕ! ψ can be derived from the statistical binomial test (on the significance level of α) of the null hypotheses H 0 : P (ψ ϕ) p against the alternative one H 1 : P (ψ ϕ) > p. The rule ϕ! ψ is true in the data matrix M exactly in those cases when H 0 is rejected by the test in favour of H 1. Here P (ψ ϕ) is the

3 Investigating Measures of Association by Graphs and Tables conditional probability of the validity of ψ under the condition ϕ. The evaluation function F! is defined such that F! (a, b, c, d) = a+b ( a+b ) i=a i p i (1 p) a+b i, thus we write only F! (a, b) instead of F! (a, b, c, d). The 4ft-quantifier? of upper critical implication is for 0 < p 1 and 0 < α < 0.5 defined in [3] by the condition a ( a+b ) i=0 i p i (1 p) a+b i > α. The rule ϕ! ψ can be derived from the statistical binomial test (on the significance level of α) of the null hypotheses H 0 : P (ψ ϕ) p against the alternative one H 1 : P (ψ ϕ) < p. The evaluation function F? is defined such that F? (a, b, c, d) = a ( a+b ) i=0 i p i (1 p) a+b i thus we write only F? (a, b) instead of F? (a, b, c, d). The 4ft-quantifier p of founded equivalence is for 0 < p 1 defined in [2] by the condition a+d n p. The rule ϕ p ψ means that ϕ and ψ have the same value (either true or false) for at least 100p per cent of all objects of M. The evaluation function F p of p is defined such that F p (a, b, c, d) = a+d a+b+c+d. The 4ft-quantifier δ of simple deviation is for 0 < δ defined in [3] by the condition ad > e δ bc. The rule ϕ δ ψ can be interpreted as the logarithmic interaction of ϕ and ψ is estimated to be greater than δ. The evaluation function F δ is defined such that F δ (a, b, c, d) = ln( ad bc ), the verification condition is F δ (a, b, c, d) > δ. The Fisher s quantifier α is for 0 < α < 0.5 defined in [3] by the condition min(a+b,a+c) i=a ( a+c i )( a+b+c+d i a+b i ) ( a+b+c+d a+b ) α ad > bc The rule ϕ α ψ can be derived from the statistical one-sided Fisher s test (on the level of α ) of the null hypothesis H 0 : ϕ and ψ are independent against the alternative one H 1 : the logarithmic interaction of ϕ and ψ is positive. The evaluation function F α of α is here defined such that F α (a, b, c, d) = min(a+b,a+c) i=a ( a+c i )( a+b+c+d i a+b i ) ( a+b+c+d a+b ) if ad > bc and F α (a, b, c, d) = 0.5 if ad bc. The verification condition is F α (a, b, c, d) α (for 0 < α < 0.5). The 4ft-quantifier + p of above average dependence is for 0 < p defined in [12] by the condition a a+c a+b (1 + p) n. The rule ϕ + p ψ means that among the objects satisfying ϕ is at least 100p per cent more objects satisfying ψ than among all objects. The evaluation function F + p of + p is defined such that an F + p (a, b, c, d) = (a+b)(a+c) 1, the verification condition is F + (a, b, c, d) = p. p The last presented quantifier is the pairing quantifier. The quantifier was introduced in [4] by condition 2 a2 d 2 a 4 +d p and should measure the level of 4 pairing of tuple of examined items. The evaluation function for the quantifier is F (a, b, c, d) = 2 a2 d 2 a 4 +d. The function is independent on b and c and can be 4 written as F (a, d). 3 Graphs of Evaluation Functions The method described in this article tries to learn evaluation functions of association measures by graphical means. The first attempt to explore the quantifiers

4 174 Martin Ralbovský, Jan Rauch was to plot them with suitable graphical tools. One can only draw graphs of functions that contain two or three independent variables. In case of three independent variables, we chose one of the variables (the least significant) as parameter and draw graphs of two remaining variables while changing the parameter. Although work described in this section was done mainly to get familiarized with functions and did not contain any special theory, we obtained one interesting result. It is the comparison of the founded equivalence and pairing quantifiers. The pairing quantifier is defined only by a and d, founded equivalence uses additionally n, size of the contingency table. We used n as parameter for the graph. The two graphs can be seen in figure 1. Fig. 1. Founded equivalence and Pairing quantifiers graphs Semantics of the quantifier is stated by its evaluation function. However, the formula for pairing quantifier is too complex to be comprehended. Therefore it is suitable to examine graph of this quantifier and compare it to the graph of the founded equivalence, because these two quantifiers use same variables (a and d). The founded equivalence graph is a plane that increases with a and d and its slope is determined by n. Contrary, the pairing quantifier s graph is a ridge above the ad diagonal. Then, the characterizing property of the founded equivalence is the bigger a+d, the better 1 and for pairing quantifier the closer a is to d, the better. Both quantifiers have disadvantages: founded equivalence is unable to distinguish between a and d. This fact reveals common misunderstanding of equivalence quantifiers: although it may seem that equivalence is a stronger implication, for implicational and equivalence classes of quantifiers, it is not the case. The pairing quantifier on the other hand is unable to consider b and c, thus resulting rule may be weakly supported. 1 This was apparent also from the evaluation function.

5 Investigating Measures of Association by Graphs and Tables The analysis shows us how to use both quantifiers in a more profound way. When using founded equivalence, we should also look at the ratio of a/d to see how good the rule is in terms of positive and negative examples. We should use pairing quantifier to find balanced all-positive, all-negative examples ratio; it is preferable to aid with the founded equivalence evaluation function. This new perception of the two quantifiers was made possible mainly by examining their graphs. One of the initial motivations was to somehow express the founded implication, lower critical implication and upper critical implication from the same point of view. Graph of functions do not help, because the only way to graph critical implications is to set p as parameter, which makes them incomparable to the founded implication, where the p is value of two-dimensional function. This problem is solved in section 5 by displaying graphs of critical frequencies of quantifiers. 4 Tables of Critical Frequencies Tables of critical frequencies are closely related to classes of 4ft-quantifiers. Examples of classes of 4ft-quantifiers are: implicational quantifiers [3], double implicational quantifiers [11] or Σ- equivalency quantifiers [11]. There are both theoretically interesting and practically important results that are related to classes of 4ft-quantifiers. These results concern namely deduction rules of the where both ϕ ψ and ϕ ψ are association rules; see [11], dealing with missing information; see [3, 9] and definability of association rules in classical predicate calculi with equality [10]. There are also results concerning tables of critical frequencies that can be used to optimize verification of particular association rules. We use tables of critical frequencies to simplify graphical description of behavior of some of quantifiers introduced in section 2. We deal with the class of implicational quantifiers and with the class of quantifiers with F-property. The class of implicational quantifiers is defined in [3] such that the 4ft quantifier is implicational if it satisfies the condition form ϕ ψ ϕ ψ (a, b, c, d) = 1 a a b b implies (a, b, c, d) = 1 for all the 4ft tables a, b, c, d and a, b, c, d. It is proved in [3] that the quantifiers p of founded implication,! of lower critical implication and? of upper critical implication (see section 2) are implicational. It is easy to prove that for each implicational quantifier there is a non-negative and nondecreasing function T b with value T b (a) {0, 1, 2,...} { } such that it is (a, b) = 1 if and only if b < T b (a) for all integers a 0 and b 0. We call the function T b a table of maximal b for implicational quantifier [3, 9]. It is important that precomputed

6 176 Martin Ralbovský, Jan Rauch function T b makes it possible to use a simple test of inequality instead of a rather complex computation. E.g., we can use inequality b < T b! (a) instead of condition a+b i=a (a+b)! i!(a+b i)! pi (1 p) a+b i α for quantifier! of lower critical implication. The class of 4ft-quantifiers with F-property is defined in [8] (see also [13]) such that the 4ft quantifier has the F-property if it satisfies: 1. If (a, b, c, d) = 1 and b c 1 0 then (a, b + 1, c 1, d) = If (a, b, c, d) = 1 and c b 1 0 then (a, b 1, c + 1, d) = 1. We say that the quantifier is symmetrical [3] if it satisfies (a, b, c, d) = (a, c, b, d). It is proved in [8] that for the symmetrical 4ft-quantifier with the F-property there is a function T that assigns to each triple a, d, n of natural numbers satisfying a + d n the number T (a, d, n) such that for each b 0 and c 0 where a + b + c + d = n it is (a, b, c, d) = 1 iff b c T (a, d, n). The function T (a, d, n) can be used in the same way as the function T b (a) for the implicational quantifier, see above. The function T (a, d, n) is called table of minimal b c. The functions T b and T (a, d, n) are called tables of critical frequencies. Note that there are also tables of critical frequencies for additional classes of 4ft-quantifiers; see e.g. [13]. The above stated theory allows us to compare quantifiers with reduced dimensionality. For given implicational quantifiers 1 and 2 and for given a, we say that 1 is I-stronger than 2, if T b 1 (a) > T b 2 (a). Similarly, for given quantifiers with the F-property 1, 2 and for given a, d and n, we say that 1 is F-stronger than 2, if T 1 (a, d, n) < T 2 (a, d, n). 5 Graphs of Tables of Critical Frequencies 5.1 Implicational quantifiers By the means stated in section 3 we examined the three most used implicational quatifiers: founded implication, lower and upper critical implication. Figure 2 displays graphs of maximal b s for p = 0.8, α = 0.05 and for a = The graph shows, that we cannot use (in the examined range of a) computationally simple founded implication instead of statistically sound but computationally demanding lower and upper critical implications. This is an important result that could not be obtained without construction of tables of maximal b s. We can think of tables of maximal b as another definition of the quantifier. It is function from a to b. For founded implication we know exact definition of the function. It is T b p = a(1 p) p + 1 and can be obtained by basic arithmetic operations, x means upper integer part of x. To our best knowledge, arithmetic extraction of the function for critical implications is impossible. For construction

7 Investigating Measures of Association by Graphs and Tables Fig. 2. Tables of maximal b s for implicational quantifiers of the table, we programmed iterations over a and b and checked, when the evaluation function stops being valid. This is computationally demanding because of the factorials in binomial coefficient and cannot be done effectively for very high numbers. Thus the method cannot be used to calculate values of functions close to infinite values. To get a better idea about the functions, we examined their slopes. It is constant for founded implication and equals the 1 p p. Slopes of the critical implication in chosen points (a values) are shown in table 2. Table 2. Slopes of critical frequencies p = 0.8, α = Lower Critical Impl Upper Critical Impl The table reveals interesting facts. Note, that the difference between slopes of critical implications and slope of founded implication remains the same. This means that the critical implications maximal b s tables are symmetric with respect to the founded implication maximal b table. Our working hypothesis is, that T b! lim (a) T b? = lim (a) T b p (a) = lim = 1 p a a a a a a p.

8 178 Martin Ralbovský, Jan Rauch However we are currently unable to prove it. If we combine the hypothesis with results from figure 2 (for fixed p, α the lower critical implication is I-stronger than founded implication, which is I-stronger than upper critical implication, than for all natural a, T b! (a) < T b p (a) < T b? (a). 5.2 Quantifiers with F-property For quantifiers with the F property, we constructed tables of minimal b c s. The construction algorithm worked in two steps: the first step was to find a, b, c, d quadruples that satisfied the quantifier. The second step was to find for given a and d the minimal b c for the quadruples. This way a matrix indexed with a and d containing minimal valid b c was obtained. The matrix subsequently transformed to a graph. We compared the simple deviation and Fisher s quantifiers. The results are displayed in figure 3, where graphs of tables of critical frequencies for n = 10 and n = 1000 are shown, α = 0.05 and δ = 0. Fig. 3. Tables of minimal b c for Fisher and Simple Deviation quantifiers The graphs show, that Fisher s quantifier behaves differently for n = 10 and for n = In the latter case, graphs of two quantifiers resemble each other,

9 Investigating Measures of Association by Graphs and Tables which was expcected. We did not compute values of the graph in all the possible points, we chose 100 representative points instead. Therefore the graphs only estimate real tables of minimal b c. For larger sizes of contingency tables, it is preferable to use the simple deviation 2. We also examined the above average dependence graph as shown in figure 4, with n = 1000 and p = 0 (this corresponds to δ = 0 for simple deviation). Note, that all the graphs displayed have a similar feature: the quantifiers are F-strongest along the inverse ad diagonal. This is the most interesting result of experiments with tables of minimal b c. It is a promising important characteristics of quantifiers with F-property, and should receive further attention. Fig. 4. Tables of minimal b c for Above average dependence quantifier The points along the inverse ad diagonal have a common property, that is their sum is equal or close to n. This means that there is not much left for b and c (from n, the size of the contingency table). This explains, why values of minimal b c are so low along the inverse ad diagonal, but it does not explain the fact, that the all three examined quantifiers are valid along this diagonal 3. The validity of quantifiers with F-property along the inverse ad diagonal remains an open question. 6 Related Work One of the most known methods for graphical comparison of classification functions are ROC curves [1]. The theory dealing with ROC curves originates from 2 Or χ 2 quantifier, which was not introduced in this paper. 3 This is rather obvious for simple deviation quantifier, but not obvious for quantifiers with the F-property at all

10 180 Martin Ralbovský, Jan Rauch construction of confusion matrices, which are similar to 4ft-tables presented in this paper. The theory gives meaning to points in the ROC space, to ROC curves and the area under ROC curves. However, the curves are used mainly for measuring quality of classification functions. In our approach, we are not interested in the fact, how good the evaluation functions were if used as classifiers. Therefore, the ROC curves are not object of our interest. 7 Conclusions Understanding of association measures is one of the key factors in association mining[5 7]. The subject has been gaining lasting scientific attention. In the paper, we studied graphs of evaluation functions of association measures (quantifiers). This task was difficult mainly because the evaluation functions are in general functions of four independent variables. However, for some quantifiers this was possible. We constructed graphs of founded equivalence and pairing quantifiers, the graphs enabled comparison of the two quantifiers and showed some properties, which were not apparent without the graphical representation. For other quantifiers, we used existing theory concerning classes of quantifiers and tables of critical frequencies. Tables of critical frequencies help to reduce the dimensionality of the quantifier evaluation function while preserving properties of the quantifier. We compared founded implication with lower and upper critical implication from the implicational class by constructing tables of maximal b s. For symmetrical quantifiers with F-property one can construct tables of minimal b c. We constructed the tables for Fisher s quantifier, above average dependence and simple deviation and observed interesting properties, the most significant being the F-strength of all observed quantifiers along the inverse ad diagonal. To summarize, graphing functions of quantifiers combined with theoretical result is a strong and promising tool that broadens our knowledge about quantifiers. The paper is an initiatory work in the field, majority of quantifiers and possibilities to graph them were unemployed. The work also showed further direction of research in the area. Issues such as construction of graphs of critical frequencies for other classes of quantifiers, effects of changes of the parameters on the graphs or various combinations of several graphs will be subjected to further research. Acknowledgements This work was supported by the project MSM of the Ministry of Education of the Czech Republic and by the project 201/05/0325 of the Czech Science Foundation.

11 Investigating Measures of Association by Graphs and Tables References 1. Fawcett T.: ROC Graphs: Notes and Practical Considerations for Data Mining Researchers. Technical report HPL HP Labs, Hájek P, Havránek T, Chytil M (1983) GUHA Method. Academia, Prague (in Czech) 3. Hájek P, Havránek T (1978) Mechanising Hypothesis Formation - Mathematical Foundations for a General Theory. Springer, Berlin Heidelberg New York 4. Kupka D.: User support 4ft-miner procedure for data mining. Master Thesis, Faculty of Mathematics and Physics, Charles University, Prague 2007 (in Czech) 5. Wu T., Chen Y., Han J.: Association Mining in Large Databases: A Reexamination of Its Measures. In: Kok J. (Edt.): Knowledge Discovery in Databases: PKDD 2007, conference proceedings. Springer Verlag, 2007, ISBN: Céline H. Crémilleux B.: A Unified View of Objective Interestingness Measures. In.: Perner P.(Edt.): Machine Learning and Data Mining in Pattern Recognition. MLDM 2007, conference proceedings, Springer, Tan P., Kumar V. Srivastava J.: Selecting the right interestingness measure for association patterns. In Proceedings of the Eight A CM SIGKDD International Conference on Knowledge Discovery and Data Mining, July Rauch J (1986) Logical Foundations of Hypothesis Formation from Databases. Mathematical Institute of the Czechoslovak Academy of Sciences, Prague, Dissertation (in Czech) 9. Rauch J (1998) Classes of Four-Fold Table Quantifiers. In: Zytkow J, Quafafou M (Eds.): Principles of Data Mining and Knowledge Discovery. Springer, Berlin Heidelberg New York 10. Rauch J (2004) Definability of Association Rules and Tables of Critical Frequencies. In: Lin T Y et al. (Eds.): Foundations of Data Mining. Brighton, IEEE Computer Society 11. Rauch J (2005) Logic of Association Rules. Applied Intelligence, 22, pp Rauch J., Šimůnek M.: (2005) An Alternative Approach to Mining Association Rules. In: Lin T Y et al. (Eds.) Data Mining: Foundations, Methods, and Applications. Springer, Berlin Heidelberg New York 13. Rauch, J.: Observational Calculi, Classes of Association Rules and F-property. San Francisco In: Granular Computing Los Alamitos: IEEE Computer Society Press, 2007, s ISBN X 14. Ralbovský M, Kuchař T (2007) Using Disjunctions in Association Mining. In: Perner P (Ed.): Advances in Data Mining - Theoretical Aspects and Applications. Springer, Berlin Heidelberg New York