Spatial Crowdsourcing: Challenges and Applications

Size: px

Start display at page:

Download "Spatial Crowdsourcing: Challenges and Applications"

Godwin Parrish
5 years ago
Views:

1 Spatial Crowdsourcing: Challenges and Applications Liu Qiyu, Zeng Yuxiang 2018/3/25 We get some pictures and materials from tutorial spatial crowdsourcing: Challenges, Techniques,and Applications in VLDB

2 2 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

3 3 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

4 4 Crowdsourcing: Concept Crowdsourcing Organizing the crowd (Internet workers) to do micro-tasks in order to solve human-intrinsic problems.

5 5 Crowdsourcing: Example recaptcha, 2007 Object differentiating Human from Robot digitizing old books one word at a time Human task: input the blurry words Luis von Ahn, CMU Supervisor: Manuel Blum

6 6 Crowdsourcing: Application Wikipedia Collecting Knowledge ImageNet Labeling Image recaptcha Digitializing newspaper Amazon Mechanical Turk General purpose crowdsourcing

7 7 Crowdsourcing: An example of AMT Crowdsourcing connects requesters and workers on the Internet

8 8 Spatial Crowdsourcing: Concept Spatial Crowdsourcing Organizing the crowd (Mobile Internet workers) to do spatial tasks by physically moving to other locations a.k.a, mobile crowdsourcing, mobile crowdsensing, participatory sensing, location-based crowdsourcing

9 9 Spatial Crowdsourcing: Example Open Street Map, 2004 collecting data from GPS devices, editing from users, answers from users. Crowdsourced map Steve Coast University College London Guide/Query Route

10 10 Spatial Crowdsourcing: Application Open Street Map Constructing map Waze Real-time traffic Uber Smart transportation Gigwalk General purpose spatial crowdsourcing

11 11 Crowdsourcing vs Spatial Crowdsourcing Crowdsourcing Spatial Crowdsourcing Information Collection Passive Participation Specialized Platform General Purpose Platform

12 Spatial Crowdsourcing: Other Apps 12

13 13 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

14 14 Spatial Crowdsourcing: Workflow Requesters Submit tasks Platform Manage tasks Workers Perform tasks

15 15 Spatial Crowdsourcing: Workflow Requesters Submit tasks Platform Manage tasks Workers Perform tasks

16 16 Spatial Crowsourcing: Task General Spatial Task Inventory identification Placement checking Data collection

17 17 Spatial Crowdsourcing: Task Specific spatial task Taxi calling service Food delivery service

18 18 Spatial Crowdsourcing: Workflow Requesters Submit tasks Platform Manage tasks Workers Perform tasks

19 19 Spatial Crowdsourcing: Platform Mangement Mode Worker Selected Tasks(WST) workers actively select tasks Leyla Kazemi, Cyrus Shahabi. GeoCrowd: Enabling Query Answering with Spatial Crowdsourcing.GIS 2012

20 Spatial Crowdsourcing: Platform Mangement Mode Worker Selected Tasks(WST) workers actively select tasks Server Assigned Tasks(SAT) workers

20 20 Spatial Crowdsourcing: Platform Mangement Mode Worker Selected Tasks(WST) workers actively select tasks Server Assigned Tasks(SAT) workers passively wait for the platform to assign tasks Leyla Kazemi, Cyrus Shahabi. GeoCrowd: Enabling Query Answering with Spatial Crowdsourcing.GIS 2012

21 21 Spatial Crowdsourcing: Platform Mangement Mode Worker Selected Tasks(WST) workers actively select tasks Server Assigned Tasks(SAT) workers passively wait for the platform to assign tasks Most studies focus on SAT mode because more optimization techniques can be designed by the platform Leyla Kazemi, Cyrus Shahabi. GeoCrowd: Enabling Query Answering with Spatial Crowdsourcing.GIS 2012

22 22 Spatial Crowdsourcing: Workflow Requesters Submit tasks Platform Manage tasks Workers Perform tasks

23 23 Spatial Crowdsourcing: Worker Influence factor Distance Workers tend to perform nearby tasks Socioeconomic status G. Quattrone, A. J. Mashhadi, L. Capra. Mind the map: the impact of culture and economic affluence on crowd-mapping behaviours. CSCW 2014.

24 24 Spatial Crowdsourcing: Worker Influence factor Distance Socioeconomic status Workers tend to perform tasks in high income regions J. Thebault-Spieker, L. G. Terveen, B. J. Hecht. Toward a Geographic Understanding of the Sharing Economy: Systemic Biases in UberX and TaskRabbit. TOCHI 2017.

25 25 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

26 26 Core issues in Spatial Crowdsourcing Quality Control Spatial Crowdsourcing Task Assignment Incentive Mechanism

27 27 Core issues in Spatial Crowdsourcing Quality Control Crowd might err/ Spatiotemporal factors might influence quality. Spatial Crowdsourcing Task Assignment Incentive Mechanism

28 28 Core issues in Spatial Crowdsourcing Quality Control Crowd might err/ Spatiotemporal factors might influence quality. Spatial Crowdsourcing Task Assignment Incentive Mechanism Assignment of tasks to appropriate workers is non-trivial.

29 29 Core issues in Spatial Crowdsourcing Quality Control Crowd might err/ Spatiotemporal factors might influence quality. Spatial Crowdsourcing Task Assignment Incentive Mechanism Assignment of tasks to appropriate workers is non-trivial. Recruitment/Payment of mobile crowd workers is not free.

30 A spatial Crowdsourcing System 30

31 31 Existing works How many papers? 7 SIGMOD VLDB ICDE GIS

32 32 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

33 33 Quality Control: Example An Example Task Where is the capital of the Switzerland? A. Zurich A.Zurich B. Bern B.Bern I support A. Zurich! Can I trust your answer?

34 34 Quality Control: Example An Example Task Where is the capital of the Switzerland? A. Zurich A.Zurich B. Bern B.Bern A. Zurich! B. Bern! B. Bern! How to infer the truth of the task?

35 35 Quality Control: Definition Definition: given different tasks s answers collected from workers, the target is to infer the truth of the each task. tasks answers workers tasks truth? truth inference truth? truth?

36 36 Quality Control: Solution Majority voting [1] Take the answer that is aggregated by the majority (or most) voting of the workers Bayesian voting [2] Take the answer that is aggregated by the Bayesian voting strategy of the workers [1] C.C. Cao, J. She, Y. Tong, L. Chen. Whom to ask?: jury selection for decision making tasks on micro-blog services. VLDB [2] Y. Zheng, R. Cheng, S. Mainu, L. Mo. On optimality of jury selection in crowdsourcing. EDBT 2015.

37 37 Quality Control in Spatial Crowdsourcing Quality Control Traditional crowdsourcing Workers only need to complete tasks on the Internet Spatial crowdsourcing Workers need to physically move to the location of the tasks Similarity: The aggregation method is also applied to quality control in spatial crowdsourcing but with spatiotemporal constraint.

38 Quality Control in Spatial Crowdsourcing Definition: given a set of tasks and a set of workers with their reputation, the problem is to maximize the total number of assigned tasks such that

38 38 Quality Control in Spatial Crowdsourcing Definition: given a set of tasks and a set of workers with their reputation, the problem is to maximize the total number of assigned tasks such that Quality requirement of each task is satisfied Spatio constraint: tasks should locate in the service range of the assigned workers L. Kazemi,C. Shahabi,L. Chen.GeoTruCrowd:trustworthy query answering with spatial crowdsourcing.gis 2013.

39 39 Quality Control in Spatial Crowdsourcing Quality requirement depends on quality control methods, e.g. Majority voting L. Kazemi,C. Shahabi,L. Chen.GeoTruCrowd:trustworthy query answering with spatial crowdsourcing.gis 2013.

40 40 Quality Control in Spatial Crowdsourcing Quality Control Traditional crowdsourcing Workers only need to complete tasks on the Internet Spatial crowdsourcing Workers need to physically move to the location of the tasks Similarity: The aggregation method is also applied to quality control in spatial crowdsourcing but with spatiotemporal constraint. Difference: Diversified answers may yield good quality in spatial crowdsourcing.

41 P. Cheng, X. Lian, Z. Chen, R. Fu, L. Chen, J. Han, J. Zhao. Reliable DiversityBased Spatial Crowdsourcing by Moving Workers. VLDB Quality Control in Spatial Crowdsourcing Spatial Diversity

42 P. Cheng, X. Lian, Z. Chen, R. Fu, L. Chen, J. Han, J. Zhao. Reliable DiversityBased Spatial Crowdsourcing by Moving Workers. VLDB Quality Control in Spatial Crowdsourcing Temporal Diversity

43 P. Cheng, X. Lian, Z. Chen, R. Fu, L. Chen, J. Han, J. Zhao. Reliable DiversityBased Spatial Crowdsourcing by Moving Workers. VLDB Quality Control in Spatial Crowdsourcing Spatialtemporal Diversity

44 44 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

Assignment is to assign tasks to proper workers.

45 Task Assignment Objective: Given a set of workers and a set of tasks distributed in 2D space, the objective of Task Assignment is to assign tasks to proper workers. Depends on different specific objectives: e.g., minimum total travel distance

46 Classification of Previous Researches Arrival Scenarios: Static vs. Dynamic Problem Formulation: Matching-based vs. Planning-based Static Dynamic Matching Static Matching Dynamic Matching Planning Static Planning Dynamic Planning

47 Static vs. Dynamic Static Scenario: the platform is assumed to know all the spatial-temporal information at the beginning. Dynamic Scenario: the platform does not know the information of subsequent tasks/workers when it performs task assignment dynamically. Assign tasks based on partial information!

48 Matching vs. Planning Matching-based Model: Solve task assignment as Weighted Bipartite Graph Matching. One worker can be assigned exactly one task.

49 Matching vs. Planning Planning-based Model: Formulate task assignment problems as route planning for workers to complete a number of tasks. One worker can perform multiple tasks.

50 Classification of Previous Researches Arrival Scenarios: Static vs. Dynamic Problem Formulation: Matching-based vs. Planning-based Static Dynamic Matching Static Matching Dynamic Matching Planning Static Planning Dynamic Planning

51 Static Matching: MaxSum Objective 1: Maximizing the total utility of the matching, which can be formulated as a Maximum Weighted Bipartite Graph Matching problem (MWBM) and can be solved by classical Max-Flow algorithms like Fold-Fulkerson algorithm. Kazemi, L., & Shahabi, C. Geocrowd: enabling query answering with spatial crowdsourcing. GIS 2012.

52 Yiu, M. L., Mouratidis, K., & Mamoulis, N. Capacity constrained assignment in spatial databases. SIGMOD Static Matching: MinSum Objective 2: Minimizing the total distance of the maximum-cardinality matching.

53 Static Matching: Stable Matching Objective 3: Given preference list of n tasks and n workers in 2D spatial space, the objective is to find a perfect matching such that there is no unstable pairs. A worker-task pair (w, t) is unstable if Dist(w, t) < Dist(t s partner, t) in the given matching Dist(w, t) < Dist(w, w s partner) in the given matching Wong, R. C. W., Tao, Y., Fu, A. W. C., & Xiao, X. On efficient spatial matching. VLDB 2007.

54 Static Matching: Stable Matching A worker-task pair (w, t) is unstable if Dist(w, t) < Dist(t s partner, t) in the given matching Dist(w, t) < Dist(w, w s partner) in the given matching Wong, R. C. W., Tao, Y., Fu, A. W. C., & Xiao, X. On efficient spatial matching. VLDB 2007.

55 Static Matching: Stable Matching A worker-task pair (w, t) is unstable if Dist(w, t) < Dist(t s partner, t) in the given matching Dist(w, t) < Dist(w, w s partner) in the given matching Wong, R. C. W., Tao, Y., Fu, A. W. C., & Xiao, X. On efficient spatial matching. VLDB 2007.

56 Dynamic Matching: Batch-based Matching Batch-based Matching: Perform static matching for new arrival tasks/workers per time slot (e.g., 4 min). Kazemi, L., & Shahabi, C. Geocrowd: enabling query answering with spatial crowdsourcing. GIS 2012.

57 1. Tong, Y., She, J., Ding, B., Wang, L., Chen, L. Online mobile micro-task allocation in spatial crowdsourcing. ICDE Tong, Y., She, J., Ding, B., Chen, L., Wo, T., Xu, K. Online minimum matching in real-time spatial data: experiments and analysis. VLDB Dynamic Matching: Online Matching Online Matching: Tasks/workers dynamically appear one by one and the online matching performs assignment immediately and irrevocably when a new task or worker arrives. Evaluation Measurement: Competitive Ratio (CR) Adversary Model (worst case): CR A = min any input sequence f(alg) f(opt) Random Order Model (average case): CR A = min E any input sequence[f ALG ] f(opt)

58 References Kazemi, L., & Shahabi, C. Geocrowd: enabling query answering with spatial crowdsourcing. GIS Yiu, M. L., Mouratidis, K., & Mamoulis, N. Capacity constrained assignment in spatial databases. SIGMOD Wong, R. C. W., Tao, Y., Fu, A. W. C., & Xiao, X. On efficient spatial matching. VLDB Kazemi, L., & Shahabi, C. Geocrowd: enabling query answering with spatial crowdsourcing. GIS Tong, Y., She, J., Ding, B., Wang, L., Chen, L. Online mobile micro-task allocation in spatial crowdsourcing. ICDE Tong, Y., She, J., Ding, B., Chen, L., Wo, T., Xu, K. Online minimum matching in real-time spatial data: experiments and analysis. VLDB 2016.

59 59 Outline Overview of Spatial Crowdsourcing (15 min) Motivation Workflow Core issues Fundamental Challenges (25 min) Quality Control Task Assignment Incentive Mechanism

60 Incentive Mechanism Objective: Stimulate workers to finish tasks efficiently with proper reward (e.g., money cost). Trade-off between workers and task requesters:

61 Incentive Mechanism Design

62 Yang, D., Xue, G., Fang, X., & Tang, J. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. MobiCom 12. Monetary Incentive: Game Theory-based Platform-centric Model: The platform determines a total reward for a set of tasks and a set of workers determine their working time to gain a fraction of the total reward based on their contribution. Objective: Under the above assumption, the platform determines an optimal total reward R to maximize its total utility. Problem Formulation: Model the optimal reward determining problem as a Stackelberg game.

63 Yang, D., Xue, G., Fang, X., & Tang, J. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. MobiCom 12. Monetary Incentive: Game Theory-based Problem Formulation: Model the optimal reward determining problem as a Stackelberg game. leader non-cooperative game follows

64 Yang, D., Xue, G., Fang, X., & Tang, J. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. MobiCom 12. Monetary Incentive: Auction-based Auction: Buyers decides to obtain goods or services by offering increasingly higher prices. Reverse Auction: Sellers decides to compete a transaction according to buyers prices (bids), which will typically decrease during the auction process.

65 Yang, D., Xue, G., Fang, X., & Tang, J. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. MobiCom 12. Monetary Incentive: Auction-based User-centric Model: The platform posts a set of tasks T = {t 1,, t n } and a user i selects a subset of tasks T i to do with a price b i that he is willing to sell his service. Objective: After receiving all the pairs T i, b i, the platform selects a subset of workers to accomplish all the tasks T and decides the reward for these selected workers such that the total utility of the platform is maximized. T 1, b 1 T 3, b 3 T 2, b 2 T 4, b 4

66 References Yang, Dejun, et al. Crowdsourcing to smartphones: Incentive mechanism design for mobile phone sensing. MobiCom Singla, Adish, and Andreas Krause. Truthful incentives in crowdsourcing tasks using regret minimization mechanisms. WWW Zhang, Xinglin, et al. Incentives for mobile crowd sensing: A survey. IEEE Communications Surveys & Tutorials 2016.

67 Q & A 67

Spatial Crowdsourcing: Challenges, Techniques, and Applications

Spatial Crowdsourcing: Challenges, Techniques, and Applications Yongxin Tong Lei Chen Cyrus Shahabi Beihang University Hong Kong University of Science and Technology University of Southern California We