Sources and Methods for the Analysis of International Data

Transcription

1 UNIVERSITA' DEGLI STUDI DI NAPOLI FEDERICO II Master s Degree in International Relations Sources and Methods for the Analysis of International Data F. Di Iorio

2 Couse Outline Population and statistical unit. Statistical variables. Data Collection. Data sources. Official statistics sources and online databases: ISTAT, Eurostat, World Bank, UN, WHO, etc. Exploratory data analysis: representing and synthesizing a distribution. Graphical representations. Measures of location and their properties (mean, median, mode, etc). Measure of variability and their properties (variance, standard deviation, interquartile range, etc). Measures of skewness and the boxplot. Concentration. Analysis of relationships between variables: covariance, correlation and simple regression. Chi square test for independence. Basics of the Analysis of Variance. Data manipulation and data analysis with Excel.

3 What is statistics? Statistics consists of a body of methods for collecting and analyzing data. (Agresti & Finlay, 1997) Statistics is the methodology which scientists and mathematicians have developed for interpreting and drawing conclusions from collected data. Statistical methods can be used to find answers to the questions like: What kind and how much data need to be collected? How should we organize and summarize the data? How can we analyse the data and draw conclusions from it? How can we assess the strength of the conclusions and evaluate their uncertainty?

4 What is statistics? Statistics provides methods for: Design: Planning and carrying out research studies. Description: Summarizing and exploring data. Inference: Making predictions and generalizing about phenomena represented by the data.

5 Population and statistical unit Population and sample are two basic concepts of statistics. Population is the set of individual persons or objects in which an investigator is primarily interested. Population is the collection of all individuals or items under consideration in a statistical study (Weiss, 1999). A (statistical) population is the set of measurements corresponding to the entire collection of units for which inferences are to be made (Johnson & Bhattacharyya, 1992). Sample is that part of the population from which information is collected. (Weiss, 1999). A sample from statistical population is the set of measurements that are actually collected in the course of an investigation (Johnson & Bhattacharyya, 1992).

6 Population and statistical unit The source of each measurement as sampling unit. A statistical unit is the unit of observation or measurement for which data are collected or derived. An important feature of the Statistical unit is the fact that it concerns the "results" side of the statistical process ; it is the elementary building block for the calculation of statistical aggregates. The statistical unit may be distinct from the collection unit: for example, it is possible to collect information about the "salaried employees" statistical unit by selecting a sample of establishments and obtaining the required information about all or part of the salaried employees working in these establishments

7 Statistical variables A characteristic that varies from one person or thing to another is called a variable, i.e, a variable is any characteristic that varies from one individual member of the population to another. Examples of variables: height, weight, numbers of children in family, sex, marital status, and eye color. First three of these variables yield numerical information and are examples of quantitative variables (continuous or discrete) Last three yield non-numerical information and are examples of qualitative (or categorical) variables.

8 Scales for Qualitative Variables Qualitative Variables can be described according to the scale on which they are defined. The categories into which a qualitative variable falls may or may not have a natural ordering E.g. occupational status (employed- non employed) have no natural ordering. E.g. Education (primary, high school, university) have natural ordering. Qualitative variable with unordered categories are defined on a nominal scale (categories are merely names). Qualitative variable with ordered categories are defined in ordinal scale. Based on what scale a qualitative variable is defined, the variable can be called as a nominal variable or an ordinal variable. Examples of ordinal variables are education (classified e.g. as low, high) and "strength of opinion" on some proposal (classified according to whether the individual favors the proposal, is indifferent towards it, or opposites it), and position at the end of race (first, second, etc.).

9 The W s of a Data Set Who the observations (population set of all objects you are interested in obtaining the value of some parameter for since we usually can t observe all objects, we take a sample of objects a subset of the overall population of objects to observe) What the variables Why why was the data collected How how was the data collected When/Where more information that could be relevant

10 Organization of the data Observing the values of the variables for one or more people or things yield data. Each individual piece of data is called an observation and the collection of all observations for particular variables is called a data set or data matrix. Data set are the values of variables recorded for a set of sampling units. For manipulating (recording and sorting) the values of the qualitative variable, they are often coded by assigning numbers to the different categories, and thus converting the categorical data to numerical data in a trivial sense. For example, marital status might be coded by letting 1,2,3, and 4 denote a person s being single, married, widowed, or divorced but still coded 11 data still continues to be nominal data. Coded numerical data do not share any of the properties of the numbers we deal with ordinary arithmetic. With recards to the codes for marital status, we cannot write 3 > 1 or 2 < 4, and we cannot write 2 1 = 4 3 or = 4. This illustrates how important it is always check whether the mathematical treatment of statistical data is really legimatitelegimatite.

11 Organization of the data Data is presented in a matrix form (data matrix). All the values of particular variable is organized to the same column; the values of variable forms the column in a data matrix. Observation, i.e. measurements collected from sampling unit, forms a row in a data matrix. E.g. k variables and n numbers of observations (sample size is n).

12 Data Collection Who the observations (population set of all objects you are interested in obtaining the value of some parameter for since we usually can t observe all objects, we take a sample of objects a subset of the overall population of objects to observe) Note: There is NO such thing as a population sample Data Collection or sample population. What the variables Why why was the data collected How how was the data collected (related to design/sampling in chapters 12-13) When/Where more information that could be relevant

13 Data sources Official statistics sources Corporate data Sample survey

14 Official statistics sources and online databases Official statistics are Statistics published by government agencies or other public agency such as international organizations. They provide quantitative or qualitative data on all major aspects of citizens' lives: economic and social situations, living conditions,health,education, and so on. National Statistical Office (ISTAT, INSEE, INE, etc) EUROSTAT, NBER Statistical departments of ministries (es. Ministero della Giustizia- Direzione Generale di Statistica e Analisi Organizzativa) OCSE-OECD, IMF, ONU

15 Official statistics sources and online databases Autentica utente

16 Official statistics sources and online databases Autentica utente

17 Official statistics sources and online databases Autentica utente

18 Official statistics sources and online databases Autentica utente

19

20

21

22 Exploratory data analysis Exploratory data analysis is an approach to analyzing data sets to summarize their main characteristics. Synthesizing a distribution by Frequency Tables and by Measures of location and their properties (mean, median, mode, etc) and variability. There are a number of graphical tools that are useful Typical graphical techniques: Histogram, Box Plot, Pie chart

23 Frequency table Suppose 30 students in a statistics class took a test and made the following scores: It is natural to group the scores on the standard ten-point scale, and count the number of scores in each group

24 Frequency table Then we have Class Absolute frequency Relative frequency Percentage Total

25 Histogram

26 Frequency and histogram The same procedure can be applied to any collection of numerical data. Observations are grouped into several classes and the frequency (the number of observations) of each class is noted. These classes are arranged and indicated in order on the horizontal axis (called the x-axis), and for each group a vertical bar, whose length is the number of observations in that group, is drawn. The relative frequency table or histogram for the data are exactly the same as the frequency table or histogram except that the vertical axis in the relative frequency, that is the ratio: frequency of a class i f i = Total number of observations

27 Frequency Tables

28 Frequency Tables

29 How to Table unit of measurement and time period self-explanatory title categories Total

30 How to Table self-explanatory title unit of measurement and time period Time series categories of which

31 Grouping variable

32 Combo! Total by rows Time series categories categories Age groups Total by column

33 Pie chart

34 Bar Chart

35 Pictogramme

36 Histogram

37 Time series

38 Time series

39 Time series

40 Infographics (good for a website, not for a repot)

41

42 NO!!!!! (impressive for a website, totally illegible)

43 NO!!!!! (impressive for a website, totally illegible)

44 Statistical Indeces Statistical Indeces location variability shape Mean Median Mode Quantiles Variance Stand. Deviation Range IQ Range Asimmetry Kurtosis

45 Mean, Median, Mode Mean (average): The mean is found by adding up all of the given data and dividing by the number of data. Example: the grade 10 math class recently had a mathematics test: 464 / 6 = Hence, 77.3 is the mean average of the class.

46 Mean, Median, Mode Median: The median is the middle number. First you arrange the numbers in order from lowest to highest, then you find the middle number by crossing off the numbers until you reach the middle. Stupid example: if data are 1,2,3,4,5 the median is 3 Example: Consider First arrange the numbers as you can see we have two numbers in the middle, there is no middle number. Solution: take the two middle numbers and find the average, ( or mean ) = / 2 = 76.5 Hence, the median is 76.5.

47 Mean, Median, Mode Mode: this is the number that occurs most often. Example: Given the following data: The mode is 78.

48 Mean, Median, Mode

49 Properties of Arithmetic Mean: 1. The sum of deviations of the items from the arithmetic mean is always zero i.e. (X m) =0. 2. The Sum of the squared deviations of the items from arithmetic mean is minimum, which is less than the sum of the squared deviations of the items from any other values. 3. If each item in the series is replaced by the mean, then the sum of these substitutions will be equal to the sum of the individual items. 4. The arithmetic mean is between the minimum and maximum values of the sample

50 Demerits of Arithmetic Mean It is affected by extreme items i.e., very small and very large items (Variance is everything!!). Es. 1,2,3,4,5 3 1,2,3,4,50 15 In some cases A.M. does not represent the actual item. For example, average patients admitted in a hospital is 10.7 per day. A.M. is not suitable in extremely asymmetrical distributions.

51 Variability Consider the mark of the following three students A={ } B={ } C={ } The 3 students heve the same mean: 25 but we can say that this value has the same meaning in all cases? C s marks have no variability, B have just 22 and 28 (and one 25) VARIANCE IS EVERYTHING

52 Variance Variability is a measurement of the spread between numbers in a data set. The spread can be measured with respect different point The VARIANCE measures how far each number in the set is from the MEAN. Variance is defined as the average of the squared of the deviations of the numbers from the arithmetic mean (remember property 2 of the Mean) In the previous example: Var(A)=4.9 Var(B)=8.2 Var(C) =0

53 Mean, variance and standard devation

54 Mean, variance and standard devation for grouped data

55 Mean, Median, Mode

56 =C5*D5 =Sum(C4:C11) =Sum(D4:D11) =D13/C13

57

58 Variance

59 Quantiles In statistics and probability quantiles are cut points dividing the range of a probability distribution into continuous intervals with equal probabilities, or dividing the observations (frequency distribution) in a sample in the groups of same dimension. 2-quantile is the median 4-quantiles are called quartiles (denoted as Q) the difference between upper and lower quartiles is the interquartile range IQR = Q 3 Q 1 Example: 0,3,3,4,5,6,7,7,7,8,9,11,13,14,15,17,17,17,20 Q1 Me Q3

60 Quartiles Q1 and Q3

61 Interquartile range (IQR) The interquartile range (IQR) is a measure of the spread of a distribution of a single quantitative variable. The IQR is a rather simple calculation and is merely the difference between (hence range ) the upper quartile (Q3) and the lower quartile (Q1) (hence inter and quartile ). Unlike total range, the interquartile range has a breakdown point of 25%, and is thus often preferred to the total range since give a measure of dispersion robust to extreme values. as a tool to determine possible outliers For a symmetric distribution (where the median equals the mean and the average of the first and third quartiles), half the IQR equals the median absolute deviation (MAD).

62 Box plot box plot or boxplot is a method for graphically depicting groups of numerical data through their quartiles. Box plots may also have lines extending vertically from the boxes (whiskers) indicating variability outside the upper and lower quartiles, hence the terms box-and-whisker plot and box-andwhisker diagram. Outliers, defined as observations that fall below Q1 1.5*IQR or above Q *IQR, may be plotted as individual points.

63 Box plot

64 Box plot: compare distributions

65 Simmetric distribution For symmetric distributions: mean approximately equal to median. The tail is the part where the counts in the histogram become smaller. For a symmetric distribution, the left and right tails are equally balanced, meaning that they have about the same length.

66 Asimmetry Right skewed distribution: the mean greater than the median. Tail of the distribution on the right hand (positive) side is longer than on the left hand side. From the box and whisker diagram we see that the median is closer to the Q1 than Q3.

67 Asimmetry

68 Kurtosis

69 Gini Index and Concentration curve The Gini Index (or coefficient) measures the inequality among values of a frequency distribution. Notable examples are the levels of income or wealth. The coefficient is in [0;1] zero represents perfect equality (same values in the distribution: everyone has the same income) and 1 represents maximal inequality among values (e.g., only one person hasall the income or consumption, and all others have none) Index often used in economic reports (see IMF, OECD etc)

70

71 Gini Index and Concentration curve The index is half of the relative mean absolute difference that is the average absolute difference of all pairs of values of the population divided by the average The graphical representation is the concentration curve or the Lorenz curve

72 Concentration curve The Lorenz curve is a function where the cumulative proportion of ordered individuals is on the horizontal axis and the corresponding cumulative proportion portion of the variable (wealth or income) is on the vertical axis

73 Gini Index and Concentration curve

74 Composite indicators

75 Indicators A statistical indicator is the representation of statistical data for a specified time, place or any other relevant characteristic, corrected for at least one dimension (usually size) so as to allow for meaningful comparisons. It is a summary measure related to a key issue or phenomenon and derived from a series of observed facts. Indicators can be used to reveal relative positions or show positive or negative change By themselves, indicators do not necessarily contain all aspects of development or change, but they hugely contribute to explaining them. They allow comparisons over time between, for instance, countries and regions, and in this way assist in gathering evidence for decision making.

76 Indicators Accident at work Age-specific fertility rate Agricultural area (AA), Agricultural income Divorce rate, marriage rate, mortality rate, death rate, Fertility rate, birth rate, CPI, Death rate of enterprise, Deficit, Deflator of sales Degree of defoliation Disposable income, GDP, Concentration index, Average labour cost per hour, Average monthly labour cost, Employment rate, Gross electricity consumption Etc etc

77 Indicators A single indicator described a single aspect, a single phenomenon but our world can be more complex E.g. the GPD per capita describes the state of an economy but it doesn t take in to account other possible aspects such as the status of the labour market, Dependence on foreign raw materials (energy in particular), the environment conditions and so on. More a Comparison between States, or institutions or systems in general become a difficult task when considering a set of single indcators (who s the more relevant indicators? How define a ranking?) From a set of indicators to a COMPOSITE INDICATOR

78 Composite indcators Composite indicators are tools for assessing and ranking countries and institutions in terms of environmental performance, sustainability, and other complex concepts that are not directly measurable A composite indicator is formed when individual indicators are compiled into a single index, on the basis of an underlying model of the multi-dimensional concept that is being measured.

79 Composite indicators A well designed composite indicators can provide a comprehensive vision of a multidimensional phenomenon allows for the setting of national benchmarks a for further international comparisons is a starting point for analysis and discussion

80 Composite indicatos The number of CIs in existence around the world is growing year after year (more than 160 composite indicators). Such composite indicators provide simple comparisons of countries that can be used to illustrate complex and sometimes elusive issues in wide-ranging fields, e.g., environment, economy, society or technological development. It is easier interpret CIs than to identify common trends across many separate indicators, and they are useful in benchmarking country performance. However, CIs can send misleading policy messages if they are poorly constructed or misinterpreted

81 Composite indicators

82 Composite indicators

83 How build a CI 3. Input missing data

84 How build a CI

85 How build a CI

86 How build a CI

87 Then: Have a clear idea of what you want to measure Choose the elementary indicators that best fir to the research question Identify the best way to synthesize indicators (simple average? Weighted average? when weighted, what weights? Find strengths and weaknesses of each selected indicator Normalize: obtain a CI possible in the range [0,1]

88

89

90 Weighting and aggregation Weights are essentially value judgements. Weights can be based on statistical methods, Weights can be chosen to reward (or punish) components that are deemed more (or less) influential, depending on expert opinion, to better reflect policy priorities or theoretical factors.

91 Weighting and aggregation Most composite indicators rely on equal weighting (EW) (simple average) Weight can be based on statistical models: Principal components analysis, Data envelopment analysis, Regression analysis Based on public/expert opinions

92 Robustness and sensitivity Uncertainty analysis focuses on how uncertainty in the input factors propagates through the structure of the composite indicator and affects the composite indicator values. Sensitivity analysis assesses the contribution of the individual source of uncertainty to the output variance. Possible actions: 1. Inclusion and exclusion of individual indicators. 2. Using alternative data normalisation schemes, such as Mni-Max, standardisation, use of rankings. 3. Using different weighting schemes

93 Presentation and dissemination The way composite indicators are presented is not a trivial issue. Tables provide the complete information, but sometimes can be obscure or long to read Graphical representation can help A tabular format is the simplest presentation, in which the composite indicator is presented for example for each country as a table of values. Usually countries are displayed in descending rank order. Rankings can be used to track changes in country performance over time. Composite indicators can be expressed via a simple bar chart; ex. countries are on the vertical axis and the values of the composite on the horizontal. The top bar can indicate the average performance of all countries

94 Some CI examples Environmental Sustainability Index(WEF) Air Quality Index (WEF) Human Development Index (United Nations) Health System Achievement Index (WHO) Corruption Perceptions Index (Transparency International) World Income Inequality Database:GiniIndex (United Nations) Economic Sentiment Indicator (EC) Composite Leading Indicators (OECD) Innovative Capacity Index(Porter and Stern) Investment/Performance in the knowledge based economy (EC) World Competitiveness Index (IMD)

95

96

97