FROM BIG DATA TO INFORMATION: STATISTICAL ISSUES THROUGH EXAMPLES

Transcription

1 FROM BIG DATA TO INFORMATION: STATISTICAL ISSUES THROUGH EXAMPLES Silvia Biffignandi Serena Signorelli University of Bergamo

2 SUMMARY Introduction on Big Data Statistical issues Some empirical studies Case study

3 INTRODUCTION ON BIG DATA Different type of data (AAPOR, 2015): Social media data Personal data Sensor data Transactional data Administrative data

4 INTRODUCTION ON BIG DATA Three main characteristics (Laney, 2001): Volume Velocity Variety Other features (AAPOR, 2015): Variability Veracity Complexity

5 STATISTICAL ISSUES Big Data useful for two main different purposes: 1. Operational 2. Statistical Main issues: quality and representativeness

6 STATISTICAL ISSUES Big Data not collected and designed to a specific statistical purpose Traditional statistical tools not immediately applicable ERRORS In literature, many experiences focus on the potentiality of Big Data but most of them are not focused on their statistical properties.

7 VOLATILITY AND INSTABILITY Social networks providers introduce recurring changes in order to improve structure and user experience Transactional and administrative data could change their structure and the way they are collected for operational and efficiency reasons Data could become incomparable from one day to the next

8 BIG DIMENSIONALITY Big Data often used to detect correlations among variables: suitable to study correlations but not to explain them (causal relationships) (Fan et al., 2014) Spurious relationship Noise accumulation Uncorrelation of model covariates with the residual error Big dimensionality BUT represent only a specific population: REPRESENTATIVENESS

9 REPRESENTATIVENESS Different definitions exist in literature, we consider it as the attempt to generalize the result to the whole population. Problem: Big Data collected through a variety of formats catch units of phenomena that differ from the units or phenomena that are not collected. Proposed solution: attempt of generalization by Elliott (2009) who built pseudoweights in order to combine probability and non-probability samples.

10 QUALITY Causes of poor Big Data Quality (Bellmore, 2014): i. Initial data loading ii. iii. Application integration Data maintenance Problem: Big Data are more vulnerable to statistical errors than traditional data sources (Saha et al., 2014): User entry errors Redundancy Corruption

11 QUALITY Proposed solution: 1. Introduction of a Total Error Framework specific for Big Data (AAPOR, 2015) based on the Total Survey Error framework (Biemer, 2010). Biemer (2014) has created the Big Data process map; three phases: I. Generation II. Extraction / Transformation / Loading III. Analysis For each phase Biemer individuates which kind of errors arise

12 QUALITY Proposed solutions: 2. Introduction of a framework to assess the quality of Big Data at three stages (UNECE Big Data Quality Task Team, 2015): 1. Input 2. Throughput 3. Output It focuses on the specific quality requirements and challenges for the use of Big Data in official statistics

13 BIG DATA AND OFFICIAL STATISTICS Opportunities of Big Data (Kitchin, 2015): Possibility for nowcasting Rich source of granular data to complement and extend micro-level and small area analysis Potentially ensure comparability of phenomena across countries

14 BIG DATA AND OFFICIAL STATISTICS Challenges of Big Data (Kitchin, 2015): Representativeness, both of phenomena and populations Big Data not generally representative of an entire population, they only relate to whom ever uses a service Data quality dimensions (OECD, 2011) largely unknown with respect to various forms of Big Data Big Data generators reluctant to share methodological transparency in how they were produced and processed Frames within Big Data are generated are mutable, changing over times

15 SOME EMPIRICAL STUDIES Analysis of Big Data (in some cases in combination with traditional survey data) in many fields: Medical Marketing Social media Traffic data Mobile data Web scraping

16 MEDICAL Self-report of Diabetes and Claims-based Identification of Diabetes Among Medicare Beneficiaries (Day, Parker, 2013) National Health Interview Survey (NHIS) + Medicare Chronic Condition (CC) Summary file Aim: verify the correspondence of the self-reported answers about diabetes in the survey with the diabetes indicators in the CC file

17 MARKETING Behavioral Data as a Complement to Mobile Survey Data in Measuring Effectiveness of Mobile Ad Campaign (Duong, Millman, 2015) Adding Big Data Booster Packs to Survey Data (Porter, Lazaro, 2014) Big Data + traditional survey data Aim: check the effectiveness of mobile ads and brands

18 SOCIAL MEDIA Using Social Media to Measure Labor Market Flows (Antenucci et al., 2014) Twitter data Aim: monitoring of the unemployment trend through the construction of Social Media Job Loss Index

19 SOCIAL MEDIA Big Data and Official Statistics (Daas et al., 2013) Twitter data Aim: sentiment analysis through the main social networks

20 TRAFFIC DATA Big Data and Official Statistics (Daas et al., 2013) Traffic sensors Aim: predict traffic intensity

21 MOBILE DATA Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data (Bogomolov et al., 2014) Demographics + mobile data Aim: predict crime

22 WEB SCRAPING Measuring the UK s digital economy with Big Data (Nathan et al., 2013) Web data Aim: using all Web available data on English digital enterprises, rewrite the official classification of English enterprises

23 CASE STUDY Aim: trying to put in an unique interpretative framework one traditional statistics source (2011 ISTAT Origin/destination matrix from the 15 th Population and housing census) and one typical kind of Big Data ( st Telecom Big Data Challenge datasets) in order to evaluate some informative potentialities of this approach.

24 CASE STUDY 2011 ISTAT Origin/destination matrix from the 15 th Population and housing census It contains data on the number of persons that commute between municipalities or inside the same municipality classified by gender, mean of transportation, departure timeslot and journey duration. Spatial Aggregation: Italian municipalities. Temporal aggregation: census was carried out on October 9 th 2011, the question regarding commuting pattern referred to last Wednesday or a typical working/studying day.

25 CASE STUDY st Telecom Big Data Challenge datasets It provides information regarding the level of interaction between the areas of the city of Milan and the Italian provinces. The level of interaction between an area of Milan and a province is given as a pair of decimal numbers proportional to the number of calls issued from the area to the province and vice versa. Spatial Aggregation: the Milano GRID squares and the Italian provinces. Temporal aggregation: the values are aggregated in timeslots of ten minutes Our analysis is limited to Lombardy region, divided into twelve administrative provinces. We built commuting patterns regarding the city of Milan and all provinces (city of Milan excluded).

26 OUTFLOW FROM MILAN TO PROVINCES - ISTAT Maps were created using CartoDB. Provinces were ranked considering the outflow from the municipality of Milan to Lombardy provinces (city of Milan excluded). Then the twelve provinces were splitted into seven buckets and coloured from dark to pale green. Different colours represent a ranking. It is possible to filter results by purpose of commuting: WORK or STUDY.

27 OUTFLOW FROM MILAN TO PROVINCES - ISTAT Outflow for STUDY Outflow for WORK

28 OUTFLOW FROM MILAN TO PROVINCES - ISTAT It is also possible to split results into four departure timeslots: Timeslot 1: before 7,15 Timeslot 2: from 7,15 to 8,14 Timeslot 3: from 8,15 to 9,14 Timeslot 4: after 9,14

29 OUTFLOW FROM MILAN TO PROVINCES - ISTAT

30 OUTFLOW FROM MILANO TO PROVINCES - ISTAT Outflow by CAR Outflow by OTHER TRANSPORT

31 OUTFLOW FROM MILAN TO PROVINCES -TELECOM Telecom dataset was provided with data of November and December We used only weekdays of November and computed the average over 20 days in order to have a mean value comparable to the one from ISTAT. Also in this case different colours represent the ranking. It is not possible here to separate calls by commuting purpose.

32 OUTFLOW FROM MILAN TO PROVINCES -TELECOM It is possible to split into timeslots as similar as possible as ISTAT ones. As the data were collected every ten minutes, we built the following: Timeslot 1: 6,20 7,10 Timeslot 2: 7,20 8,10 Timeslot 3: 8,20 9,10 Timeslot 4: 9,20 10,10

33 OUTFLOW FROM MILAN TO PROVINCES -TELECOM

34 PROVINCES CODES Province ISTAT code Province ISTAT code Bergamo 16 Mantova 20 Brescia 17 Milano 15 Como 13 Monza e della Brianza 108 Cremona 19 Pavia 18 Lecco 97 Sondrio 14 Lodi 98 Varese 12

35 OUTFLOW COMPARISON METHOD Telecom ISTAT T.1 T.2 T.3 T.4 T.1 T.2 T.3 T

36 OUTFLOW COMPARISON METHOD Timeslot 1 Timeslot 2 Timeslot 3 Timeslot 4 Telecom Istat Telecom Istat Telecom Istat Telecom Istat

37 OUTFLOW COMPARISON METHOD Timeslot 1 Timeslot 2 Timeslot 3 Timeslot 4 Telecom Istat Telecom Istat Telecom Istat Telecom Istat

38 OUTFLOW COMPARISON METHOD TELECOM VS ISTAT T.1 T.2 T.3 T.4

39 OUTFLOW COMPARISON METHOD TELECOM VS ISTAT T.1 T.2 T.3 T.4 Cell comparison

40 OUTFLOW COMPARISON METHOD TELECOM VS ISTAT TELECOM VS ISTAT T.1 T.2 T.3 T.4 T.1 T.2 T.3 T.4 Cell comparison Column comparison

41 OUTFLOW COMPARISON METHOD TELECOM VS ISTAT T.1 T.2 T.3 T.4 Row comparison

42 OUTFLOW COMPARISON METHOD TELECOM VS ISTAT TELECOM VS ISTAT T.1 T.2 T.3 T.4 T.1 T.2 T.3 T.4 = = = / = = = / = / = = = / = = Row comparison Partial row comparison

43 OUTFLOW COMPARISON RESULTS Comparison between Telecom dataset and ISTAT dataset divided into timeslots Telecom dataset: general outflow ISTAT dataset: different filters applied General outflow Work outflow Study outflow Outflow by car Outflow by other transport Work outflow by car Work outflow by other transport Four kind of comparison: cell, column, row, partial row

44 OUTFLOW COMPARISON RESULTS Telecom vs ISTAT Cell (48) Column (4) Row (12) Partial row (12) Outflow Work outflow Study outflow Outflow by car Outflow by other transport Work outflow by car Work outflow by other transport

45 OUTFLOW COMPARISON RESULTS Telecom vs ISTAT Cell (48) Column (4) Row (12) Partial row (12) Outflow Work outflow Study outflow Outflow by car Outflow by other transport Work outflow by car Work outflow by other transport

46 OUTFLOW COMPARISON RESULTS Telecom vs ISTAT % Cell (48) Column (4) Row (12) Partial row (12) Outflow 100,00 100,00 100,00 100,00 Work outflow 91,67 50,00 66,67 100,00 Study outflow 33,33 0,00 8,33 16,67 Outflow by car 83,33 0,00 58,33 50,00 Outflow by other transport 58,33 0,00 16,67 25,00 Work outflow by car 64,58 0,00 33,33 83,33 Work outflow by other transport 45,83 0,00 16,67 33,33

47 INFLOW COMPARISON RESULTS Telecom vs ISTAT Cell (48) Column (4) Row (12) Partial row (12) Inflow Work inflow Study inflow Inflow by car Inflow by other transport Work inflow by car 23 1* 1 4 Work inflow by other transport *Two missing matches

48 INFLOW COMPARISON RESULTS Telecom vs ISTAT Cell (48) Column (4) Row (12) Partial row (12) Inflow Work inflow Study inflow Inflow by car Inflow by other transport Work inflow by car 23 1* 1 4 Work inflow by other transport *Two missing matches

49 INFLOW COMPARISON RESULTS Telecom vs ISTAT % Cell (48) Column (4) Row (12) Partial row (12) Inflow 37,50 0,00 25,00 41,67 Work inflow 45,83 0,00 8,33 50,00 Study inflow 27,08 0,00 8,33 16,67 Inflow by car 47,92 0,00 8,33 33,33 Inflow by other transport 43,75 0,00 25,00 33,33 Work inflow by car 47,92 8,33* 8,33 33,33 Work inflow by other transport 43,75 0,00 8,33 41,67 *Two missing matches

50 OPPORTUNITIES Increased information on the social exchanges between provinces. A vision of the exchanges in a physical perspective and in a communication (measured through mobiles calls) perspective If the matching is sufficiently satisfactory, alternative source for Official Statistics: cheaper and up-to-date could be evaluated for integrating official statistics source

51 LIMITS Different purposes of the flows: Telecom calls for every purpose, ISTAT only flows for work and study Only Telecom Italia Mobile users, those from other providers could behave differently Only traditional calls, other type of calls (i.e. Skype, Whatsapp) not considered Telecom dataset contains only province reference, it would be useful to have municipalities in order to do a better match with ISTAT and to map commuting patterns

52 CONCLUDING REMARKS Big Data potentialities for official statistics need a huge amount of experimentation and of economic statistics studies to set up a suitable metadata framework and to evaluate quality of the considered Big Data The overview of potentiality and problems presented in our paper highlights most critical research points The present case study shows some ideas on how to go through the tentative use of Big Data in official statistics and some potentialities seem to be expected

53 FUTURE RESEARCH More detailed analyses on similarities and differences between the two datasets Search for more possible data to be considered in the comparison Identification of alternative case studies on Big Data analysis

54 THANK YOU! Silvia Biffignandi Serena Signorelli

55 REFERENCES AAPOR Report on Big Data, AAPOR (American Association For Public Opinion Research) Big Data Task Force, February 12, ANTENUCCI, D., CAFARELLA, M., LEVENSTEIN, M.C., RE, C. & SHAPIRO, M.D Using Social Media to Measure Labor Market Flows. NBER Working Papers 20010, National Bureau of Economic Research, Inc. BELLMORE, C The three root causes of poor Big Data quality, BackOffice Associates Website, March 24th. BIEMER, P Dropping the S from TSE: Applying the Paradigm to Big Data International Total Survey Error Workshop, October 1-3, BIEMER, P Total Survey Error: Design, Implementation, and Evaluation. Public Opinion Quarterly, 74(5), BOGOMOLOV, A., LEPRI, B., STAIANO, J., OLIVER, N., PIANESI, F., & PENTLAND, A Once Upon a Crime: Towards Crime Prediction from Demographics and Mobile Data. Proceedings of the 16th International Conference on Multimodal Interaction, DAAS, P.J.H., PUTS, M.J., BUELENS, B., VAN DEN HURK, P.A.M Big Data and Official Statistics. Paper for the 2013 New Techniques and Technologies for Statistics conference. Brussels, Belgium. DAY, H.R., PARKER, J.D Self-report of Diabetes and Claims-based Identification of Diabetes Among Medicare Beneficiaries. National Health Statistics Reports, 69. DUONG, T., MILLMAN, S Behavioral Data as a Complement to Mobile Survey Data in Measuring Effectiveness of Mobile Ad Campaign. Presented at the CASRO Digital Research Conference.

56 REFERENCES ELLIOTT, M.R Combining Data from Probability and Non-Probability Samples Using Pseudo-Weights, Survey Practice, 2(6). FAN, J., HAN, F., & LIU, H Challenges of Big Data analysis. National Science Review, 1(2), KITCHIN, R Big Data and Official Statistics: Opportunities, Challenges and Risks, Statistical Journal of the IAOS, 31(3), LANEY, D D Data Management: Controlling Data Volume, Velocity and Variety. META Group Research Note. NATHAN, M., ROSSO, A., GATTEN, T., MAJMUDAR, P., MITCHELL, A Measuring the UK s digital economy with Big Data. National Institute of Economic and Social Research (NIESR). PORTER, S., LAZARO, C.G Adding Big Data Booster Packs to Survey Data. Presented at the CASRO Digital Research Conference. SAHA, B., SRIVASTAVA, D Data quality: the other face of Big Data, 2014 IEEE 30th International Conference on Data Engineering (ICDE). TASK TEAM ON BIG DATA QUALITY A Suggested Framework for National Statistical Offices for assessing the Quality of Big Data. Paper for the 2015 New Techniques and Technologies for Statistics conference. Brussels, Belgium.