ON THE ANALYSIS OF SYMBOLIC DATA

Transcription

1 ON THE ANAYSIS OF SYMBOIC DATA Paula Brio FEP / IACC-NIAAD niv. of Poro, Porugal

2 Ouline From classical o symbolic daa Symbolic variables Conceual versus disance-based clusering Dynamical clusering Sandardizaion A regression model Disersion, associaion and linear combinaions of inerval variables Discriminan analysis Inerval ime-series Conclusions and Persecives 2

3 From classical o symbolic daa Classical daa analysis : Daa is reresened in a n x marix each of n individuals (in row) ake one single value for each of variables (in column) Too simle model o reresen variabiliy uncerainy 3

4 From classical o symbolic daa Symbolic daa new variable yes: Se-valued variables : variable values are subses of an underlying se Inerval variables Caegorical muli-valued variables Modal variables : variable values are disribuions on an underlying se Hisogram variables 4

5 From classical o symbolic daa : Inerval Variables Ω {ω,..., ω n } Y wih underlying domain O R I se of inervals of O Y : Ω I ω i [l i, u i ] 5

6 Inerval Daa Y Y j... Y ω [l, u ]... [l j, u j ]... [l, u ] ω i [l i, u i ]... [l ij, u ij ] [l i, u i ] ω n [l n, u n ] [l nj, u nj ] [l n, u n ] 6

7 Modal variables Ω {ω,..., ω n } Y wih underlying domain O{m,, m k } Y : Ω D ω i ω ω k {m ( i ),, m ( i k )} 7

8 The key quesion Mulivariae mehods ofen rely on measures of disersion How o evaluae disersion of symbolic daa? Disersion around a cenral oin versus generaliy measures : Are we measuring he same or differen hings? 8

9 Clusering Disance based mehods vs conceual mehods Conceual mehods : usually based on generalizaion measures: if I merge he wo iems, how much of he descriion sace do I cover? Disance based mehods : usually generalizaions of corresonding mehods for sandard daa. Sill 9

10 Hierarchical and Pyramidal Clusering Numerical disance based Clusering Maximum, Minimum, Average, Diameer inkage Ward: sum of squares (Hardy, 2006) Symbolic Clusering : clusers are conces Minimum Generaliy Minimum Increase in Generaliy Generaliy is evaluaed according o he generalizaion mehod 0

11 Hierarchical and Pyramidal Clusering Generaliy evaluaion: Generaliy degree Proorion of he descriion sace covered by he objec a [ Y ] j R j Vj e j j j G (a) j m (V j ) m (O j ) G (e j ) j

12 Generaliy evaluaion: Inerval variables m(vi) max Vi min Vi (range) Consider a grou described by a symbolic objec s [age [ 20, 45]] [salary [000, 3000]] e e G (e ) 0, G (e 2 ) G (s ) 0,55* 0,2 0, 0,2 2

13 Generaliy evaluaion: Modal variables When generalising by he Maximum G (a) k j which uses for each variable j he affiniy coefficien (Mausia, 95) beween ( j,, k jj) and he uniform disribuion G(a) is maximum () when ij /k j, i, k : uniform This means ha we consider an objec he more general he more similar i is o he uniform disribuion Corresonding mehod for generalizaion by he Minimum j k j i ij 3

14 Algorihm for symbolic (conceual) clusering Saring wih he one-objec clusers A each se, form a cluser (,s) union of (, s ) and ( 2, s 2 ) such ha, 2 can be merged ogeher s s s 2 (comlee) ex E s G(s) G(s s 2 ) is minimum 4

15 Which are he more dissimilar??? If O [0, 00] e I [0, 20], I2 [30, 40] e I3 [0, 00], I4 [9, 99] 30 I I2 [0, 40] G (I I2 ) 0,3 00 I3 I4 [9, 00] 9 G(I3 I4) 0,9 00 [ ] 800 [ ] / 2 2 (I I2) (30 0) + (40 20) 2 2 / 2 2 (I3 I4) (00 99) + (0 9) 5

16 Which are he more dissimilar??? No such roblem for classical daa! and d (0, 30) > d (0, 20) G([0, 30]) > G([0, 20]) 6

17 Dynamical clusering of inerval daa De Carvalho, Brio & Bock : Pariioning clusering mehod 2 disance beween inervals Dynamic clusering aroach Assignmen funcion Reresenaion funcion nil convergence 7

18 Assignmen Funcion : Given reresenaives (l,, l k ) he ariion P {P,, P k } is defined by: P h {ω Ω : D({ω}, l h ) D ({ω},l m ), m k} Reresenaion Funcion : Given a ariion {P,, P k }, he reresenaives (l,, l k ) are defined by : l h : D(P h, l h ) minimizes D(P h, ) l ( l,u ], K,[l,u ] ) h [ h h h h 8

19 Dynamical clusering of inerval daa Alying ieraively he assignemen funcion followed by he reresenaion funcion in urn decreases seadily he values of W(,P) k D(P, l ) h h h) unil a local miminum is aained. The mehod hence minimizes W(,(P)) k ([l h i P j h ij l hj ] 2 + [u ij u hj ] 2 ) wih resec o ariion P. 9

20 Sandardizaion Dissimilariy values and clusering resuls are srongly affeced when modifying he scales of variables. Some sandardizaion mus be erformed rior o he clusering rocess in order o aain an 'objecive' or 'scale-invarian' resul. The same ransformaion mus be alied o boh he lower and he uer limis of each inerval. 20

21 Sandardizaion sing he disersion of he inerval ceners The firs mehod considers he mean and he disersion of he inerval ceners and sandardizes such ha he resuling ransformed midoins have zero mean and disersion in each dimension. I ij [l ij, u ij ] I ij [l ij, u ij ] 2

22 Sandardizaion 2 sing he disersion of he inerval boundaries Evaluae he disersion of an inerval variable by he disersion of inerval boundaries: I ij [l ij, u ij ] I ij [l ij, u ij ] 22

23 Sandardizaion 3 sing he global range The hird sandardizaion mehod ransforms, for a given variable, he inervals I ij [l ij, u ij ] (i,...,n) such ha he range of he n rescaled inervals is he uni inerval [0,]. Min j Min { l ij, u ij, i,,n } Min { l ij, i,,n } Max j Max { l ij, u ij, i,,n } Max { u ij, i,,n } I ij [l ij, u ij ] I ij [l ij, u ij ] 23

24 Exerimenal resuls Simulaion sudies showed ha sandardizaion grealy imroves he qualiy of he cluser resuls (recover of an imosed srucure). Sandardizaion 2 slighly beer for illsearaed clusers where inervals have large ranges 24

25 Sandardizaion 2 : Consequences Covariance Measure Correlaion 25

26 ooking for he regression model Y deenden variable X indeenden variable 26

27 Regression Model i,..., n α and β minimize () Model obained indeendenly by Neo and De Carvalho by direc minimizaion of he crierion (). 27

28 Exerimenal resuls Mone-Carlo exeriences Simulaing inerval daa wih differen degrees of lineariy Differen degrees of variabiliy (inerval ranges) and qualiy of adjusmen Performance similar o ha of mehod based on midoins (Billard and Diday) (MSE on lower and uer bounds, R 2 of lower and uer bounds) 28

29 Disersion, associaion and linear combinaions (Duare Silva & Brio) I[I ij ] i,...,n, j,..., I ij [l ij, u ij ] β[β ij ] i,...,, j,...,r Z[Z ij ] i,...,n, j,...,r Z [ z, zij] ij ij Z I β 29

30 Disersion, associaion and linear combinaions β β β S S (P2) ij j j I i β I β l l (P) 30 β β j ij j i j ij j i u z l z l l l l (C) β + β β + β < β > β < β > β 0 ij j 0 ij j i 0 ij j 0 ij j i j j j j l u z u l z l l l l l l l l l l (C2) β β β I S I S (P2)

31 Disersion, associaion and linear combinaions C is aroriae when lower (resec. uer) bounds of differen variables end o occur simulaneously: Posiive Inner Correlaion C does no saisfy P C2 saisfies P and is aroriae in he absence of inner correlaion. 3

32 Disersion, associaion and linear combinaions Disersion s j2 and associaion s jj measures deend on l ij and u ij symmerically C and C2 saisfy P2 Variances of linear combinaions are given by quadraic forms : raios are maximized by a radiional eigenanalysis. 32

33 Discriminan funcions. Disribuional Aroach Equidisribuional hyohesis (Berrand, Gouil, 2000) : An uniform disribuion is assumed for each observed inerval Emirical Dis. of each inerval variable is a mixure of n uniform laws K grous : mixure of K mixures 33

34 Discriminan funcions. Disribuional Aroach m j n n i l ij + 2 u ij s n j ij ij ij ij 2 ij uij 3n i 4n i s jj' ( n l + l u + u ) ( l + u ) 4n 2 4n i n n i ( l + u )( l + u ) ( ) ( ) l + u l + u ij ij ij ij ij' n i ij' ij' ij' 34

35 Discriminan funcions. Disribuional Aroach From hese measures a decomosiion in w w jj ' wihin-grous and (j j ) and beween-grous comonens is obained. jj b jj' Discriminan linear funcions are hen given by he eigenvecors of W - B. 35

36 2. Verices aroach Each individual is reresened by he corresonding hyercube verices : he original marix is exanded ino a (n 2 ) marix A classical analysis of he verices marix is erformed. imis for he l-h discriminan funcion on individual ω i : z il Min{z ql,q Q i } z il Max{z ql,q Q i } 36

37 3. Midoins and ranges aroach Each inerval is reresened by is midoin c ij and range r ij Two classical analysis are hen erformed : o Searaely for C[c ij ] and R[r ij ] o Joinly for he marix [C R] 37

38 Classificaion rules Classificaion rules are derived from discriminan sace reresenaions Poinwise Inervalwise Euclidean Dis. Mahalanobis Dis. Hausdorff Dis. δ ( z, z ) Max{ z z, zil z jl il jl Oher inerval disances il jl } 38

39 Exerimenal resuls Searaion by midoins only: Mehods which ake ranges exlicily ino accoun have wors erformance Searaion by midoins and ranges: Mehods which ake ranges exlicily ino accoun have bes erformance Mehods based on inerval aroaches caure range informaion in a limied exen 39

40 Modelling Inerval Time Series Daa (Teles, Brio) Inerval-valued daa colleced as an ordered sequence hrough ime (or any oher dimension) Inerval ime series X lower limis X uer limis of he observed inervals wihx X, K,N Inerval ime series : [ X,X ], [ X,X ], K, [ X,X ]. 2 2 N N Aroriae mehodology o sudy and model i: ime series analysis ARMA models 40

41 Main model Assumion : boh limis are realizaions of second-order saionary sochasic rocesses. Assumion 2: he limis of he inerval ime series follow ARMA models wih he same orders and ARMA arameers. Main assumion, basis of our aroach 4

42 Proosed model for he inerval ime series X X where C C +φ X +φ X [ X,X ],, K, N + K+φ + K+φ X X + a + a θa θ a K θ K θ q q a a q q () C µ ( φ K φ ) C µ ( φ K φ ) and accoun for he differen means of X and X wih µ > µ C C ; > φ, K φare he auoregressive arameers; θ, K θ are he moving average arameers;,, q 42

43 Proosed model for he inerval ime series a a and are indeenden whie noise sequences wih 2 zero mean and variances σ and. 2 a σ a I is also assumed ha a and a are indeenden of each oher. Thus, i is assumed ha X and have differen means and variances and follow ARMA(,q) rocesses wih he same ARMA arameers. X 43

44 Proosed model for he inerval ime series Model () may also be wrien as ( ) ( q ) φb K φb X C + θ θ B K qb ( φ φ ) + ( q B K B X C θ B K θ B ) j where B is he backshif oeraor, BX X j. In a more comac form, φ φ ( B) X C + θ( ) ( B) X C + θ( ) B a B a q a a (2) (3) where φ( B) φ K φ and ( ) θ B θ K θq are he auoregressive and moving average olynomials, resecively. 44

45 Proosed model for he inerval ime series We assume ha o he olynomials have no roos in common, ( B ) o he roos of θ are ouside he uni circle so ha hese rocesses are inverible, φ ( B) o he roos of are ouside he uni circle so ha hese rocesses are saionary. The rocedures we roose o model inerval ime series will be based on model (). 45

46 Model for boh inerval limis e X & X µ and & wih E ( X & ) EX (& ) 0. X X µ From (), we have X& X& φ X & + K +φ X & + a θ a K θ a φ X& + K+φ X& + a θ a K θ q q a q q (4) and esimaion of he model arameers is required. 46

47 Since boh inerval limis follow ARMA(,q) rocesses wih he same ARMA coefficiens, i is ossible o base he esimaion mehod on a ime series ha joins all he uer and lower observaions, i.e., This doubles he samle size used in esimaion..,x,,x,x, X N N K K 47 This doubles he samle size used in esimaion. Rewriing (4) as and assuming he iniial condiions (5) φ φ + + θ + θ φ φ + + θ + θ q q q q X X X a a a X X X a a a & K & & K & K & & K q q a a a 0 a a a + + K K

48 The condiional sum of squares funcion is S N [ ] ( ) 2 ( ) 2 + ( ) ( 2 2 φ, θ a φ, θ a φ, θ a a )( φ θ) + N +, + N + (6) φ and θ ( θ K, θ ). where ( φ, K φ ),, q Therefore, condiional leas squares esimaors of are obained from minimizing S. φ and θ 48

49 Afer obaining he arameer esimaes and θ ˆ ( θˆ,, ˆ K θ q ), 2 ( φˆ,, φˆ ) φ ˆ K i is necessary o esimae he whie 2 noise variances σ a and σ a. This is accomlished wih he uer and he lower ime series searaely: σ ˆ σˆ 2 a 2 a S N S N N ( ) a ( ˆ, ˆ ) 2 ˆ, ˆ φ θ φ θ + ( 2 + q) N ( 2 + q) ( φˆ, θˆ ) ( 2 + q) N + N a 2 ( φˆ, θˆ ) ( 2 + q) where S and S are he uer and lower condiional sums of squares resecively and he denominaors are he degrees of freedom, i.e., N (+q) N (2 + q). 49

50 Finally, he esimaors of he means μ and μ are he samle means of and resecively: X X X N X X ˆ N N µ µ 50 The esimaion rocess is now comlee. N X ˆ µ and ( ) ( ). ˆ ˆ X Ĉ ˆ ˆ X Ĉ φ φ φ φ K K

51 Forecasing fuure values of he inerval ime series is based on he esimaed model (), i.e., he uer and he lower limis are forecas searaely from he resecive models, which is a sandard rocedure in ime series analysis. 5

52 Model for he differences of he inerval limis A simler aroach is based on he differences of X & andx & in model (): D X& X& This rocedure leads o a roblem of modeling a single ime series D follows an ARMA(,q) rocess wih he same ARMA arameers as model (): D + ( ) ( ) X& X& φ X& X& + K+φ X& X& ( ) ( ) ( ) a a θ a a K θ a a φd + K+φ D +ε θ ε q K θ q q ε q q sandard ime series analysis 52

53 This simle rocedure reduces he roblem of modeling an inerval ime series o he sandard analysis of a single ime series sraighforward and very easy o imlemen. 53

54 Exerimenal resuls The roosed rocedures erform very well in erms of esimaion accuracy in erms of forecasing erformance. Esimaion accuracy: The mehod for boh inerval limis doubles he number of observaions used in esimaion I shows higher accuracy and clearly ouerforms he oher mehod, being more efficien. 54

55 Exerimenal resuls Forecasing erformance: The wo mehods show he same forecasing erformance as single ime series models. In sie of he beer esimaion accuracy of he mehod for boh inerval limis, he forecasing erformance is he same. 55

56 Conclusions The exension of classical mehodologies o he analysis of inerval daa raises new roblems: How o evaluae disersion? How o define linear combinaions? Which roeries remain valid?... 56

57 Conclusions Reresenaions in lower dimensional saces may assume differen forms : Inervals, emhasize variabiliy inheren o each observaion Poins, can dislay disinc conribuions o he searaion beween grous 57

58 Persecives Need of models inear modelisaion is no sraighforward choices mus be made Temoral modelisaion : Two series? One serie? Saisical models allowing o make esimaion and es hyohesis 58