A clustering view on ESS measures of political interest:

Transcription

1 A clustering view on ESS measures of political interest: An EM-MML approach Cláudia Silvestre Margarida Cardoso Mário Figueiredo Escola Superior de Comunicação Social - IPL BRU_UNIDE, ISCTE-IUL Instituto de Telecomunicações, Inst. Sup. Técnico Portugal

2 Outline Objective Model Finite Mixture Models Selection Criterion Minimum Message Length Algorithm EM-MML Results Conclusions

3 Objective Clustering the regions in the European Social Survey based on attitudes towards politics Voted last national election ( Yes; No; Not eligible) Contacted politician or government official in last 12 months Worked in political party or action group in last 12 months Worked in another organisation or association in last 12 months Worn or displayed campaign badge/sticker in last 12 months Signed petition in last 12 months Taken part in lawful public demonstration in last 12 months Boycotted certain products last in 12 months Feel closer to a particular party than all other parties (Y/N)

4 K f y i θ = α k f y i θ k Model: Finite Mixture Models k=1 K is the number of segments y i is regarded as incomplete data, the allocation to segments (z i ) being missing Complete data: y i, z i

5 The log of complete likelihood Model: Finite Mixture Models f y i, z i θ z i α f y i θ i uknown

6 How to select the number of segments? Information criteria such as BIC, AIC, CAIC, AIC3 or ICL can be used Selection Criterion We adopt Minimum Message Length criterion embedded in the model estimation (Figueiredo and Jain, 2002), which: Provides estimates of all the model parameters including the number of segments Is less sensitive to initialization than EM Avoids the boundary of the parameters space

7 Shannon s Information Theory: optimally transmitting a random variable Y with probability f y requires about 2 f y bits of information. Selection Criterion: MML to encode y: y θ 2 f y, θ to encode y and θ the total message length is: y, θ y θ + θ

8 Algorithm: EM-MML EM is a popular algorithm for finding ML parameter estimates, when unobserved (missing) data is considered in the model. The EM-MML A mixture of multinomials is adopted and the MML estimates are obtained via an EM-type algorithm.

9 Categorical variables: Y Y,, Y i, Y Algorithm: EM-MML Y i Y i,, Y id where variable d (d 1 D) has C d categories θ θ,, θ, α,, α, α are the clusters weights or mixing probabilities θ the multinomials parameters

10 log f y θ i log f y i θ Algorithm: EM-MML Mixture of multinomials: D C d θ dc y idc f y i θ α d n! c y idc!

11 Assuming that: The segments have independent priors independent from the mixing probabilities Algorithm: EM-MML A noninformative Jeffreys prior for θ y, θ y θ + θ M 2,αk >0 n α 12 + k z n 12 + k z M log f y θ M is the number of parameters specifying each segment k z is the number of segments with non-zero probability

12 E-step E Z i y i ; θ t P Z i 1 y i ; θ t Algorithm: EM-MML α t f y i ; θ t α t f y i ; θ t where D C d t θ dc y idc f y i ; θ t d n! c y idc!

13 M-step Update the estimates of mixing probabilities α t+ max 0, i P Z i 1 y i ; θ t M 2 Algorithm: EM-MML max 0, i P Z i 1 y i ; θ t M 2 Update the estimates of multinomial parameters θ dc t+ i P Z i 1 y i ; θ t y idc n! i P Z i 1 y i ; θ t

14 compute P Z i α 1 Y i, θ (t) Algorithm: EM-MML α = 0 K:=K-1 > 0 compute θ

15 Results The clustering of Regions in the European Social Survey based on attitudes towards politics, using EM-MML, yields 2 clusters

16 Results: cohesion-separation stability computation time BIC; CAIC; ICL AIC; AIC3 EM-MML 2012 Number of clusters Silhouette index Calinski-Harabasz Computation time (seconds) Number of clusters Silhouette index Calinski-Harabasz Computation time (seconds) vs 2014 Adjusted Rand Normalized mutual information

17 160 number of regions 160 number of regions Results: round 6 vs round ESS6 CLU 1 ESS6 CLU 2 0 ESS7 CLU 1 ESS7 CLU 2

18 8 76% 7 6 Regions in cluster 2 share a more active role in politics (Yes %) 57% % 32% 28% 37% % 2% 6% 4% 12% 13% 6% 9% 9% 8% 9% Contacted politician or government official last 12 months Worked in political party or action group last 12 months Worked in another organisation or association last 12 months Worn or displayed campaign badge/sticker last 12 months Signed petition last 12 months Taken part in lawful public demonstration last 12 months Boycotted certain products last 12 months Feel closer to a particular party than all other parties Voted last national election Not eligible to vote ESS6 CLU 1 ESS6 CLU 2

19 8 73% Regions in cluster 1 share a more passive role in politics (Yes %) 58% 6 41% 4 34% % % 18% 3% 6% 12% 12% 9% 7% 8% 1 Contacted politician or government official last 12 months Worked in political party or action group last 12 months Worked in another organisation or association last 12 months Worn or displayed campaign badge/sticker last 12 months Signed petition last 12 months Taken part in lawful public demonstration last 12 months Boycotted certain products last 12 months Feel closer to a particular party than all other parties Voted last national election Not eligible to vote ESS7 CLU 1 ESS7 CLU 2

20 Regions in cluster 2 are clearly more interested in politics (as expected ) 41% 4 37% 32% % 2 42% 36% 29% 3 27% Results % 14% 12% % 1 12% How interested in politics - very interested How interested in politics - not at all interested How interested in politics - very interested How interested in politics - not at all interested ESS6 CLU 1 ESS6 CLU 2 ESS7 CLU 1 ESS7 CLU 2

21 2 Most respondents in Cluster 1 do not trust politicians 21% 2 19% Results % 1 13% 13% 9% 13% 13% 11% 14% 11% 7% 2% 4% 1% 1% Trust in politicians - Not at all Trust in politicians - Completely ESS6 CLU 1 ESS6 CLU 2

22 or political parties Results 1 12% 1 14% 14% 13% 11% 13% 14% 11% 1 1 9% 7% 4% 4% 2% 1% 1% Trust in political parties - Not at all Trust in political parties - Completely ESS6 CLU 1 ESS6 CLU 2

23 2 Regions in cluster 2 share a more positive view of other people ESS6 and ESS7 results being very similar 21% 21% 2 18% % 11% 11% 12% 13% 12% 1 9% 1 9% 2% 4% 2% 3% 3% 2% 2% Mostly looking out for themselves Most of the time people helpful ESS6 CLU 1 ESS6 CLU 2

24 All regions in Sweden, Norway, Finland, Denmark and Germany belong to cluster 2 Slovenia Sweden Portugal Poland Norway Netherlands Lithuania Israel Ireland Hungary United Kingdom France Finland Spain Estonia Denmark Germany Czech Republic Switzerland Belgium 3% 4% 5,7% 6% 8,7% 9% 9% 8% 1 9% 1 9,6% 9,2% 9,2% 9,8% 11,4% 10,9% 12% 1,8% 3,9% 9,4% 6,8% 16% 0, 1,2% 0, 1,9% 2% 4% 6% 8% 1 12% 14% 16% 18% ESS6 CLU 2 ESS6 CLU 1 0, 0, 0, 0,

25 All regions in Sweden, Norway, Finland, Denmark and Germany belong to cluster 2 25 regions change to cluster 2, e.g. Lisbon (in Portugal ) Jihoceský kraj (in Czech Republic) 4 regions change to cluster 1: Prov. West-Vlaanderen (in Belgium ), Principado de Asturias, La Rioja (in Spain) and Drenthe (in Netherlands) Slovenia Sweden Portugal Poland Norway Netherlands Lithuania Israel Ireland Hungary United Kingdom France Finland Spain Estonia Denmark Germany Czech Republic Switzerland Belgium 1% 1% 4% 7% 7,4% 7% 7% 6% 8% 5,8% 8% 9,8% 10,3% 1 9% 1 8% 0, 12,4% 13,6% 0, 14% 12,2% 0, 15, 2,3% 0, 0,4% 9, 0, 0,4% 2% 4% 6% 8% 1 12% 14% 16% 18% ESS7 CLU 2 ESS7 CLU 1 0, 0,4% 0,

26 Conclusions A new EM variant the EM-MML was used to cluster categorical aggregated data and estimate the number of clusters simultaneously. It estimates parameters of a finite mixture of multinomials, using a Minimum Message Length criterion. EM-MML shows better performance when compared with traditional EM-ML combined with BIC, AIC and ICL: more parsimonious and robust solutions; better cohesion-separation and stability Abrief profiling of the segments showing that the main changes occurred between rounds 6 and 7

27 References Biernacki, C., Celeux, G. and Govaert, G., Assessing a Mixture model for Clustering with the integrated Completed Likelihood. IEEE Transactions on Pattern analysis and Machine Intelligence, 22: Figueiredo, M. A. T., and Jain, A. K., Unsupervised learning of finite mixture models", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, pp Fonseca, J. R. & Cardoso, M. G. (2007). Mixture-model cluster analysis using information theoretical criteria. Intelligent Data Analysis 11(2): Silvestre, C., Cardoso, M. and Figueiredo, M., Clustering with finite mixture models and categorical variables. Contributed Papers to the International Conference on Computacional Statistics, Porto, Portugal, pp Silvestre, C., Cardoso, M., and Figueiredo, M., Categorical Data Clustering Using a Minimum Message Length Criterion. IDA The eleventh International Symposium on Intelligent Data Analysis. Helsínquia, Finlândia, de outubro, Silvestre, C., Cardoso, M., and Figueiredo, M., Determining the Number of Groups while Clustering Categorical Data. IFCS 2013 The International Federation os Classification Societies. Tilburg, the Netherlands, July (Book of Abstracts p. 158)