Visualisation of Indicators in the AMELI Project

Transcription

1 Die Informationsmanager Visualisation of Indicators in the AMELI Project Matthias Templ, Andreas Alfons, Alexander Kowarik, Bernhard Meindl, Peter Filzmoser, Beat Hulliger, Daniela Lussmann Pooda STA TISTIK AUSTRIA Advanced Methodology for European Laeken Indicators NTTS 2011, Feb 23, 2011 Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

2 Disclaimer Disclaimer Statement for this presentation: Some graphics presented at this presentation can cause eye cancer. For complaining we refer to contact the corresponding commercial software providers. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

3 Disclaimer Disclaimer Statement for this presentation: Some graphics presented at this presentation can cause eye cancer. For complaining we refer to contact the corresponding commercial software providers. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

4 Disclaimer Disclaimer Statement for this presentation: Some graphics presented at this presentation can cause eye cancer. For complaining we refer to contact the corresponding commercial software providers. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

5 How We Do NOT Want to Present Statistical Information Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

6 What We Do Want To Visualise BUT NOT within the AMELI Project Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

7 Visualisation in the AMELI project Visualisation Topics in AMELI The aim is to research and implement visualization tools (in R): for visualising indicators in order to support policy decisions, for highlighting selected/special data (e.g. outlying and inuential observations, non-response, imputed values), for the visualisation of simulation results, for visualizing regional indicators in maps, as well as for visualisation for a better understanding by the end user of the indicator values including their quality. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

8 Visualisation in the AMELI project Visualisation Topics in AMELI The aim is to research and implement visualization tools (in R): for visualising indicators in order to support policy decisions, for highlighting selected/special data (e.g. outlying and inuential observations, non-response, imputed values), for the visualisation of simulation results, for visualizing regional indicators in maps, as well as for visualisation for a better understanding by the end user of the indicator values including their quality. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

9 Visualisation in the AMELI project Visualisation Topics in AMELI The aim is to research and implement visualization tools (in R): for visualising indicators in order to support policy decisions, for highlighting selected/special data (e.g. outlying and inuential observations, non-response, imputed values), for the visualisation of simulation results, for visualizing regional indicators in maps, as well as for visualisation for a better understanding by the end user of the indicator values including their quality. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

10 Visualisation in the AMELI project Visualisation Topics in AMELI The aim is to research and implement visualization tools (in R): for visualising indicators in order to support policy decisions, for highlighting selected/special data (e.g. outlying and inuential observations, non-response, imputed values), for the visualisation of simulation results, for visualizing regional indicators in maps, as well as for visualisation for a better understanding by the end user of the indicator values including their quality. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

11 Visualisation in the AMELI project Visualisation Topics in AMELI The aim is to research and implement visualization tools (in R): for visualising indicators in order to support policy decisions, for highlighting selected/special data (e.g. outlying and inuential observations, non-response, imputed values), for the visualisation of simulation results, for visualizing regional indicators in maps, as well as for visualisation for a better understanding by the end user of the indicator values including their quality. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

12 Visualisation in the AMELI project Visualisation Topics in AMELI The aim is to research and implement visualization tools (in R): for visualising indicators in order to support policy decisions, for highlighting selected/special data (e.g. outlying and inuential observations, non-response, imputed values), for the visualisation of simulation results, for visualizing regional indicators in maps, as well as for visualisation for a better understanding by the end user of the indicator values including their quality. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

13 Visualisation in the AMELI project Done in AMELI but not shown VIM Package for analysing the structure of missing values + many side products from our work Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

14 Visualisation in the AMELI project Done in AMELI but not shown VIM Package for analysing the structure of missing values + many side products from our work Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

15 Visualisation in the AMELI project Selection of Graphics Due to time contraints we only show certain plots, namely... Checkerplots Thematic Maps Sparktables (Tufte 2006, Alfons, Filzmoser, Hulliger, Meindl, Schoch, and Templ 2009, Kowarik, Meindl, and Zechner 2010) extensions to maps and sparkevals Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

16 Visualisation in the AMELI project Checkerplots Traditional Representation... hard to determine a speci c country and hard to see the trend. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

17 Visualisation in the AMELI project Checkerplots From Thematic Maps to Checkerplots... and here is to less space available to visualise complex statistical information for small countries. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

18 Visualisation in the AMELI project Checkerplots Checkerplots To display social indicators for several countries including geographical information. In thematic maps each country has unequal space which do not allow to display complex statistical information. Each country gets the same space in a (trellis panel) grid whereas the position of the countries maintained as much as possible. For 34 countries in a map e + 38 dierent positions of countries are possible. All complex sorting rules for arranging the countries give worse results. a better - an optimal solution - is required Checkerplots can be applied very easily to a broad kind of data. Before showing the problem, let's see visually how a checkerplot looks like... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

19 Visualisation in the AMELI project Checkerplots Checkerplots To display social indicators for several countries including geographical information. In thematic maps each country has unequal space which do not allow to display complex statistical information. Each country gets the same space in a (trellis panel) grid whereas the position of the countries maintained as much as possible. For 34 countries in a map e + 38 dierent positions of countries are possible. All complex sorting rules for arranging the countries give worse results. a better - an optimal solution - is required Checkerplots can be applied very easily to a broad kind of data. Before showing the problem, let's see visually how a checkerplot looks like... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

20 Visualisation in the AMELI project Checkerplots Checkerplots To display social indicators for several countries including geographical information. In thematic maps each country has unequal space which do not allow to display complex statistical information. Each country gets the same space in a (trellis panel) grid whereas the position of the countries maintained as much as possible. For 34 countries in a map e + 38 dierent positions of countries are possible. All complex sorting rules for arranging the countries give worse results. a better - an optimal solution - is required Checkerplots can be applied very easily to a broad kind of data. Before showing the problem, let's see visually how a checkerplot looks like... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

21 Visualisation in the AMELI project Checkerplots Checkerplots To display social indicators for several countries including geographical information. In thematic maps each country has unequal space which do not allow to display complex statistical information. Each country gets the same space in a (trellis panel) grid whereas the position of the countries maintained as much as possible. For 34 countries in a map e + 38 dierent positions of countries are possible. All complex sorting rules for arranging the countries give worse results. a better - an optimal solution - is required Checkerplots can be applied very easily to a broad kind of data. Before showing the problem, let's see visually how a checkerplot looks like... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

22 Visualisation in the AMELI project Checkerplots Checkerplots To display social indicators for several countries including geographical information. In thematic maps each country has unequal space which do not allow to display complex statistical information. Each country gets the same space in a (trellis panel) grid whereas the position of the countries maintained as much as possible. For 34 countries in a map e + 38 dierent positions of countries are possible. All complex sorting rules for arranging the countries give worse results. a better - an optimal solution - is required Checkerplots can be applied very easily to a broad kind of data. Before showing the problem, let's see visually how a checkerplot looks like... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

23 Visualisation in the AMELI project Checkerplots Checkerplots To display social indicators for several countries including geographical information. In thematic maps each country has unequal space which do not allow to display complex statistical information. Each country gets the same space in a (trellis panel) grid whereas the position of the countries maintained as much as possible. For 34 countries in a map e + 38 dierent positions of countries are possible. All complex sorting rules for arranging the countries give worse results. a better - an optimal solution - is required Checkerplots can be applied very easily to a broad kind of data. Before showing the problem, let's see visually how a checkerplot looks like... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

24 Visualisation in the AMELI project Checkerplots Checkerplot. Example: Heuristic Arrangement Employment rate of older workers 2003 and IS IE UK CH ES PT NO NL FR LI IT MT SE DK BE SI MK GR LV DE LU HR BG CY EE PL CZ SK RO EU27 FI LT AT HU TR men 2003 women 2003 men 2008 women 2008 Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

25 Visualisation in the AMELI project Checkerplots Checkerplot. Example: Heuristic Arrangement IS Unemployment of females ES IT MK BG RO PT MT GR CY EU27 TR NA or using packe 'x' is missing NO SE LV EE FI IE NL DK DE PL LT UK FR BE LU CZ CH LI SI HR SK HU AT year Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

26 Visualisation in the AMELI project Checkerplots Problem Let both X and Y be a two-dimensional data set with n observations. The pairwise distances between X and Y are calculated by d ij = x i y j 2 2. (1) The objective function in the optimization problem is then given by these distances, sticked row-wise together and resulting in a vector of length n 1 n 2. Let this vector denoted by z = (d 11 d d 1n 2 d 21d d 2n 2... d i1d i2... d in 2... d n1 n 2 ). (2) Appropriate contraints have to be formulated (skipped in the slides) to solve a linear program which results in one optimal solution. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

27 Visualisation in the AMELI project Checkerplots Assignment Problem: The Grid Versus the Centers of Countries in Lat./Long. IS SE FI NO EE DK LT LV IE UK NL DE BE LU CHLI FR PL CZ SK AT HU SI HR RO IT BG PT ES GR TR MT CY Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

28 Visualisation in the AMELI project Checkerplots Optimal Assignment of Countries to the Grid IS SE FI NO DK LT EE LV IE UK NL PL DE BE LU CZ CH LI FR SK AT HU SI HR RO IT BG PT ES GR TR MT CY Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

29 Visualisation in the AMELI project Checkerplots Optimal Assignment 1:1 match (33 places in the grid) versus 36 available places to put 33 countries IS NO SE UK NL DK PL EE FI IE BE DE SK LT LV LI LU AT CZ HU RO CH FR SI HR BG TR PT ES IT MT GR CY IS UK LU PT NL IE CH FR ES NO DE SE DK FI LT EE LV BE CZ PL SK LI AT HU SI RO HR BG TR IT MT GR CY Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

30 Visualisation in the AMELI project Checkerplots Result (1:1 Match) Employment rate of older workers 2003 and IS UK IE LI CH PT NO NL BE LU FR ES SE DK DE AT SI IT PL SK CZ HR MT EE LT HU BG GR FI LV RO TR CY men 2003 women 2003 men 2008 women 2008 Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

31 Some Issues in Mapping Projections From Distorted Maps to Thematic Maps. The Projection Problem. Map of Europe from Eurostat is in long/lat. representation. Projection to other coordinate systems is absolutely necessary otherwise distorted maps results. Aim: projections should be made interactively when selecting countries for plotting. Problem: Every selected region needs other parameters for the projection (solved). Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

32 Some Issues in Mapping Projections From Distorted Maps to Thematic Maps. The Projection Problem. Map of Europe from Eurostat is in long/lat. representation. Projection to other coordinate systems is absolutely necessary otherwise distorted maps results. Aim: projections should be made interactively when selecting countries for plotting. Problem: Every selected region needs other parameters for the projection (solved). Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

33 Some Issues in Mapping Projections Map of Europe in Lat./Long. as we Used to it Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

34 Some Issues in Mapping Projections Long./Lat. Representation of Austria Looks Quite Distorted Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

35 Some Issues in Mapping Projections Lambert Equal Area Presentation of Austria Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

36 Some Issues in Mapping Projections Long./Lat. Presentation of Ireland Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

37 Some Issues in Mapping Projections Lambert Equal Area Presentation of Ireland Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

38 Some Issues in Mapping Projections Useful Functionality For projections we made use of package rgdal (Keitt, Bivand, Pebesma, and Rowlingson 2010) and wrote additional functionality which deal with the special structure of spatial dataframes. subsetnuts(): selects regions. projection(): computes the optimal projection parameters, after plot.subsetnuts() is applied. plus functions for interactive selection, maps in continuous color scale, etc. are developed. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

39 Graphical Tables in Web and Publications Sparklines Visualization for Policy Support Exemplarely: Seasonally adjusted production indices by branch, monthly. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

40 Graphical Tables in Web and Publications Sparklines Example (production indices enhanced) Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

41 Graphical Tables in Web and Publications Sparklines Example (developments of the Austrian population) Bevölkerung zu Jahresbeginn seit 1981 nach Geschlecht bzw. breiten Altersgruppen (Absolutwerte) Nach Geschlecht Nach Altersgruppen Jahr Insgesamt Männer auf Männer Frauen Frauen 0 bis 19 Jahre 20 bis 64 Jahre 65 Jahre und älter dar.: 75 Jahre und älter Q: STATISTIK AUSTRIA, Statistik des Bevölkerungsstandes.- Revidierte Ergebnisse für 2002 bis Erstellt am: Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

42 Graphical Tables in Web and Publications Sparklines Example (developments of the Austrian population) Bevölkerung zu Jahresbeginn seit 1981 nach Geschlecht bzw. breiten Altersgruppen (Überblick) Minimum Maximum Insgesamt Männer Frauen Männer auf Frauen Jahre Jahre Jahre Jahre Bevölkerung zu Jahresbeginn seit 1981 nach Geschlecht bzw. breiten Altersgruppen (Absolutwerte) Geschlecht Altersgruppen Männer auf Jahr Insgesamt Männer Frauen Frauen 0-19 Jahre Jahre 65+ Jahre 75+ Jahre Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

43 Graphical Tables in Web and Publications Sparklines Graphical representations Creation of customized time-series, box- and barplots Time series plots Bar plots Box plots Graphical options include (among others) Highlighting of specic values in time series plots Choice of colors Choice of boundaries Inclusion of interquartile-range in time-series plots Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

44 Correlation of Time Series Correlation over time of one country to the others Cross correlation between countries should only be estimated after prewhitening. Prewhitening generally applies the following procedure: 1 First an ARIMA (autoregressive integrated moving average) model is t on the rst time series. 2 Transformation of the correlated input series x t to the uncorrelated white noise series α t, which consists in fact of the residuals of the tted time series. 3 The same transformation is applied on the second time series using the tted parameters from modelling the rst time series, which leads to the second process β t. 4 Estimate the cross correlation between the two new series. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

45 Correlation of Time Series Correlation of the Gini indicator over time between AT and EU after prewhitening and weighting the time series Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

46 Correlation of Time Series Maps and Sparklines From Sparklines to Maps to Sparkevals The idea is to put sparklines into thematic maps. The trend and the actual value of an indicator should be visible. Interactivity should be provided, e.g. to zoom an area or to present other plots. Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

47 Correlation of Time Series Maps and Sparklines From Sparklines to Maps Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

48 Correlation of Time Series Maps and Sparklines From Maps to Sparkevals A modied plot is presented when clicking on one region - the evaluation plot (Hulliger and Lussmann 2008, Zechner 2010)... Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

49 Conclusion Conclusion We presented only a (very) small selection of our work in visualization, namely Checkerplots they easy to apply and we provide an optimal solution for the arrangement of countries in a any grid. Scalable sparklines can be integrated almost everywhere Correlations between short time series needs special treatment before they presented in maps. A lot of aspects where not shown (automatic output for websites, visual perception, automatic selection of plot methods depending on the input structure,... ) Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

50 Conclusion Further Work Done in AMELI We covered those aspects in the AMELI project: Visualisation as diagnostic tools in the production prozess Visualisation of indicators for policy makers Mapping for end-users Visualisation of simulation results... should be denitly continued in form of other research projects! ;-) Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

51 References A. Alfons, P. Filzmoser, B. Hulliger, B. Meindl, T. Schoch, and M. Templ. State-of-the-art in visualization of indicators in survey statistics. Technical report, Vienna University of Technology, A. Alfons, M. Templ, and P. Filzmoser. An object-oriented framework for statistical simulation: The R Package simframe. Journal of Statistical Software, 37(3):136, B. Hulliger and D. Lussmann. Bewertung der Nachhaltigkeits- und Umwelt-Indikatoren. Technical report, Institute for Competitiveness and Communication, University of Applied Sciences, Northwestern Switzerland, Timothy H. Keitt, Roger Bivand, Edzer Pebesma, and Barry Rowlingson. rgdal: Bindings for the Geospatial Data Abstraction Library, URL R package version Alexander Kowarik, Bernhard Meindl, and Stefan Zechner. sparktable: Sparklines and graphical tables for tex and html, URL R package version Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41

52 Conclusion E. Tufte. Beatiful Evidence. Graphics Press, Cheshire, S. Zechner. Visualization of indicators. Master's thesis, Department of Statistics and Probability Theory, Vienna University of Technology, Vienna, Austria, Templ, et al. (AMELI) Visualisation of Indicators Feb 23, / 41