Generalization of the Principal Components Analysis to Histogram Data

Transcription

1 Generalization of the Principal Components Analysis to Histogram Data Oldemar Rodríguez 1, Edwin Diday 1, and Suzanne Winsberg 2 1 University Paris 9 Dauphine, Ceremade Pl Du Ml de L de Tassigny orodrigu@ceremadedauphinefr diday@ceremadedauphinefr 2 IRCAM, 1 Place Igor Stravinsky F 75004, Paris, FRANCE SuzanneWinsberg@ircamfr Abstract In this article we propose an algorithm for Principal Components Analysis when the variables are histogram type This algorithm also works if the data table has variables of interval type and histogram type mixed If all the variables are interval type it produces the same output as the one produced by the algorithm of the Centers Method propose in [5, Cazes, Chouakria, Diday and Schektman (1997)] 1 The algorithm In this algorithm we use the idea proposed in [9, Diday (1998)] We represent each histogram individual by a succession of k interval individuals (the first one included in the second one, the second one included in the third one and so on) where k is the maximum number of modalities taken by some variable in the input symbolic data table Instead of representing the histograms in the factorial plane, we are going to represent the Empirical Distribution Function F Y defined, in [3, Bock and Diday (2000)] associated with each histogram In other words if we have an histogram variable Y on a set E = {a 1, a 2, } of objects with domain Y represented by the mapping Y (a) = (U(a), π a ), for a E, where π a is frequency distribution, then in the algorithm we will use the function F (x) = i / π i x π i instead of the histogram Definition 1 Let X = (x ij ) i=1,2,,m, j=1,2,,n be a symbolic data table with variables type continuous, interval and histogram, and let be k = max{s, where s is the number of modalities of Y j, j = 1, 2,, n} where Y j is a variable of histogram type 1 We define the vector succession of intervals associated with each cell of X as: 1 If all the variables are interval type then k = 1

2 1 if x ij = [a, b] then the vector succession of intervals associated is: [a, b] x ij = [a, b] [a, b] 2 If x ij = (1(p 1 ), 2(p 2 ),, s(p s )) with s k (histogram) then the vector succession of intervals associated is: [0, p 1 ] x ij = [0, p 1 + p 2 ] [0, s w=1 p w] k 1 k 1 3 If x ij = a then the vector succession of intervals associated is: [a, a] x ij = [a, a] [a, a] Definition 2 Let X = (x ij ) i=1,2,,m, j=1,2,,n be a symbolic data table with variables type continuous, interval and histogram We define the matrix X = (x ij ) for i = 1, 2,, m and j = 1, 2,, n It is important to note that X has m k rows 2 and n columns [ ] (1(01), 2(04), 3(05)) (1(02), 2(03), 3(05)) Example 1 If X = then (1(07), 2(02), 3(01)) (1(08), 2(01), 3(01)) [00000, 01000] [00000, 02000] [00000, 05000] [00000, 05000] X = [00000, 10000] [00000, 10000] [00000, 07000] [00000, 08000] [00000, 09000] [00000, 09000] [00000, 10000] [00000, 10000] The idea is to apply the algorithm 3 proposed in [13, Rodríguez (2000)] to the matrix X With this Principal Components Analysis we can find the shape of the individual histogram in the principal plane However because all the individual histogram will be projected almost in the same position around the origin So we have to apply another principal components analysis in order to find a good cluster structure to the individual histogram Therefore we will apply a classical principal components analysis to the matrix presented in the followings definitions 2 k like in the previous definition k 1

3 Definition 3 Let X = (x ij ) i=1,2,,m, j=1,2,,n be a symbolic data table with variables type continuous, interval and histogram We define the row vector associated with each cell of X as: 1 If x ij = [a, b] then the row vector associated is: [ ] a + b x ij = If x ij = (1(p 1 ), 2(p 2 ),, s(p s )) where s is number of modalities of the j th variable, then the row vector associated is: x ij = [p 1, p 2,, p s ] 1 s 3 If x ij = a then the row vector associated is: x ij = [a] 1 1 Definition 4 Let X = (x ij ) i=1,2,,m, j=1,2,,n be a symbolic data table with variables type continuos, interval and histogram We define the matrix X = (x ij ) of m rows and p = n j=1 s j columns, where number of modalities of the variable If the variable j is histogram type, s j = 1 If the variable j is interval type, 1 If the variable j is continue type [ ] (1(01), 2(04), 3(05)) (1(02), 2(03), 3(05)) Example 2 If X = then (1(07), 2(02), 3(01)) (1(08), 2(01), 3(01)) [ ] X = The idea of the algorithm is to apply a principal components analysis to the matrix X to find the shape of the individual histogram, and then to apply another principal components analysis to the matrix X Using this last principal components, we will translate the individual histogram to find the cluster structure of individual histogram in the principal plane Algorithm 1: Histogram Principal Components Analysis Input : m =number of symbolic objects n =number of symbolic variables The symbolic data table X = x 11 x 12 x 1n x 21 x 22 x 2n x m1 x m2 x mn

4 Output : The symbolic matrix with the first q principal components: y 11 y 12 y 1q y 21 Y = y 22 y 2q, y m1 y m2 y mq where (k like in definition 1): [ ] yij 1, y1 ij [ ] y ij = y 2 ij, y2 ij [ ] yij k, yk ij Step 1: Compute the matrix X of the definition 2 Step 2: Apply the algorithm 3 proposed in [13, Rodríguez (2000)] taking as input X It will produce the matrix: ŷ 11 ŷ 12 ŷ 1q 1 ŷ Ŷ 21 = ŷ 22 ŷ 2q 1, ŷ m1 ŷ m2 ŷ mq 1 where (k like in definition 1): [ ] ŷij 1, ŷ1 ij [ ] ŷ ij = ŷ 2 ij, ŷ2 ij [ ] ŷij k, ŷk ij for i = 1, 2,, n and j = 1, 2,, q 1 with q 1 n Step 3: Compute the matrix X of the definition 4 Step 4: Apply a classical principal components analysis to the matrix X It will produce the matrix: Ỹ = ỹ 11 ỹ 21 ỹ 12 ỹ 1q2 ỹ 22 ỹ 2q2 ỹ m1 ỹ m2 ỹ mq2 where q 2 p = n j=1 s j (s j like in definition 4):,

5 Step 5: q = min(q 1, q 2 ) Step 6: Compute the first q principal components: using the translation: Step 7: End of the algorithm y 11 y 12 y 1q y 21 Y = y 22 y 2q, y m1 y m2 y mq [ ] [ ] yij 1, y1 ij ŷij 1 [ ] + ỹ ij, ŷ1 ij + ỹ ij [ ] y ij = yij 2, y2 ij ŷij 2 = + ỹ ij, ŷ2 ij + ỹ ij [ ] [ ] yij k, yk ij ŷij k + ỹ ij, ŷk ij + ỹ ij 2 Examples To illustrate how the algorithm works in this section we present two examples Example 3 In this example we present the execution of the algorithm 1 with the symbolic data table presented in (1) This matrix has five variables, the first one is interval type, the second one is a variable quantitative discrete, and the last three variables are histogram type (the values are truncated) X = [1, 4] 2 (1(04), 2(01), 3(02), 4(007), 5(002)) (1(01), 2(09)) (1(07), 2(02)) [1, 4] 3 (1(06), 2(01), 3(01), 5(00)) (1(01), 2(09)) (1(07), 2(02)) [1, 5] 2 (1(07), 2(02)) (1(00), 2(09)) (1(07), 2(02)) [1, 4] 1 (1(07), 2(00), 3(01), 4(00), 5(00), 6(00), 7(00)) (1(00), 2(09)) (1(07), 2(02)) [1, 4] 1 (1(04), 3(04), 4(00), 5(00)) (1(00), 2(09)) (1(08), 2(01)) [1, 6] 2 (2(04), 3(01), 4(03), 5(00), 6(00), 7(00)) (1(00), 2(09)) (1(07), 2(02)) (1) Applying the algorithm 1 proposed above, we get the principal plane of Figure 1 If we plot the pyramid (see Figure 2) associated with the matrix (1) we get the same cluster structure as the one obteined it in the principal plane of Figure 1 The individuals East midlands non-metropolitan and Northern Ireland are isolated and the individuals North non-metropolitan, Yorks and Humberside metropoli, Yorks and Humberside non-metro and East midlands non-metropolitan are grouped

6 Fig 1 Principal plane with data of continuous, interval and histogram type Fig 2 Pyramid with data of continuous, interval and histogram type 3 The interpretation To explain how to interpret the Histogram Principal Components Analysis we will use one small example The interpretation of the position of the histogram individual in the principal plane is the same as in the classical principal components analysis situation We shall explain the interpretation of the succession of rectangles that represents each individual

7 Example 4 Let be VAR-1 VAR-2 X = IND-1 (1(01), 2(04), 3(05)) (1(02), 2(03), 3(05)) IND-2 (1(07), 2(02), 3(01)) (1(08), 2(01), 3(01)) This matrix can be also represent like we show in the Figure 3 Fig 3 Data table with two individus and two histogram variables If we apply the Histogram Principal Components Analysis to the previous data table we get the principal plane that we show in the Figure 4 The smallest rectangle of the projection of the individual 1(Ind1) represents the probability that individual 1 takes the modality 1 for the variable 1 or the modality 1 for the variable 2 The size of the rectangle agrees with the representation of the individual 1 in the Figure 3, because the value of the modality 1 for the variable 1 is 01 and the value of the modality 1 for the variable 2 is 02, ie the mean for the modality 1 is 015 The second rectangle of the projection of the individual 1 represents the probability that individual 1 takes the modality 1 or the modality 2 for the variable 1, or the probability that individual 1 takes the modality 1 or modality 2 for the variable 2 The size of the second rectangle also agrees with the representation of individual 1 in the Figure 3, because the value of the empirical distribution function for the modality 2 of the variable 1 is 05 and the value of the empirical distribution function for the modality 2 of the variable 2 is also 05 The third rectangle of individual 1 represents the probability 1, that is the probability that individual 1 takes any of the modalities

8 The smallest rectangle of the projection of individual 2 (Ind2) is bigger than the smallest rectangle of the projection of the individual 1 (see Figure 4); it is consistent with the interpretation, because the probability for individual 2 to take the modality 1 for the variable 1 is 07 and the probability for individual 2 to take the modality 1 for the variable 2 is 08, ie the mean of taken the modality 1 is 075 This value is bigger than the same value for individual 1 that is 15; that s why, the smallest rectangle of the projection of Ind1 is smaller than the smallest rectangle of the projection of Ind2 For the same reasons the second rectangle of the projection of Ind1 is smaller than the second rectangle of the projection of Ind2 Fig 4 Histogram Principal Component Plane References 1 Bertrand P et Goupil F Descriptive statistics for symbolic data, In Symbolic official data analysis, Springer, , Billard L and Diday E Regression analysis for interval value data, In data analysis,classification and related methods, Eds Kiers H, Rasson J, Groenen P and Schader M, IFCS Bock H-H and Diday E (eds) Analysis of Symbolic Data Exploratory methods for extracting statistical information from complex data Springer Verlag, Heidelberg, 425 pages, ISBN , Brito P Analyse de donnees symboliques: Pyramides d heritage, Thèse de doctorat, Université Paris IX Dauphine, 1991

9 5 Cazes P, Chouakria A, Diday E et Schektman Y Extension de l analyse en composantes principales à des données de type intervalle, Rev Statistique Appliquée, Vol XLV Num 3 pag 5-24, Francia, Chouakria A Extension des méthodes d analysis factorialle à des données de type intervalle, Thèse de doctorat, Université Paris IX Dauphine, Diday E Lemaire J, Pouget J, Testu F Eléments d Analyse des données Dunod, Paris, Diday E Introduction l approche symbolique en Analyse des Donnés Première Journées Symbolique-Numérique Université Paris IX Dauphine Décembre Diday E L Analyse des Donnes Symboliques: un cadre théorique et des outils Cahiers du CEREMADE, Diday E An Introduction to symbolic data analysis ans its application to the sodas project: purpose, history and perspective, Paris IX University Dauphine, Paris, Gettler Summa M Factorial axis interpretation by symbolic objects, Actes des Journées Symbolique-Numérique, Ed E Diday, Y Kodratoff, S Pinson Editeurs Univ Paris IX Dauphine 12 Rodríguez O, Symbolic correlation circle in principal components analysis, IFCS Rodríguez O, The Duality Problem in Symbolic Principal Components Analysis Also in this book 14 Pollaillon G Organisation et interprétation par les treillis de Galois de données de type multivalué, intervalle ou histogramme Thése de doctorat, Université Paris IX Dauphine, Saporta, G, L Analyse des Données Que sais-je? Presses Universitaires de France, Paris, 1980