The Livehoods Project: Utilizing social media to understand the dynamics of a city. Trung Phan

Transcription

1 The Livehoods Project: Utilizing social media to understand the dynamics of a city Trung Phan {tphan@idiap.ch} 1

2 Content 1 Introduction 2 Clustering 3 Interview 4 Result and Discussion 5 Conclusion 5/26/16 2

3 Introduction Motivation Pittsburgh at night Src: 5/26/16 3

4 Introduction Related work v Observation and interview based Urban studies and urban design (Lynch 1992; Suttles 1973; Milgram 1977;Putnam 2000; Oldenburg 1989) Disadvantages: long time spans and extend resources v Location sharing system based Large scale urban dynamics (Cranshaw and Yano 2010; Noulas et al. 2011; Chang and Sun 2011) The effects location technologies have over people s behavior (Lindqvist et al. 2011; Cramer et al. 2011; Schwartz 2012) 5/26/16 4

5 Goals 5/27/16 Introduction v Utilizing social media Check-ins Users Venues v To understand the dynamics of cities Present a clustering model for discovering the clusters of the distinct geographic areas of the city, reflecting movement pattern of people. Introduce semi-structured interviews for exploring and validating the result clusters. Public web-based tool for visualizing the clusters. 5

6 Introduction Data Collections v Foursquare and Twitter 18M check-ins Twitter public tweets = 11M (from Chen et al. 2011) + 7M (collected from Jun-Dec 2011) Each check-in has 4sq check-ins links with userid, time, lat/lng, venue name and venue category. Filtered check-ins in Pittsburgh 42,787 Check-ins 3,840 Users 5,349 Venues They want to cluster venues into cluster. They call cluster as Livehood! 5/27/16 6

7 Content 1 Introduction 2 Clustering 3 Interview 4 Result and Discussion 5 Conclusion 5/26/16 7

8 Clustering Clustering algorithm 5/27/16 i v Geophraphic distance: d(i,j): geography distance between venue i and venue j using lat/lng. v Social distance: Set V contains n " = 5349 venues Set U contains n ( = 3840 users Set C contains check-ins of n ( users at n " venues At each venue v, build vector c - having n ( dimensions, each dimension value is # of check-ins of user u /0 at venue s i, j = with c or c : as vector of a venue 8 s(i,j): cosine distance between venue i and venue j j 8

9 Clustering Clustering algorithm v Building graph of venues: At venue i, choose N m i = m closest venues according to geographic distance d(i,.). i d(i,.) a 9,: =? s i, j + α if j N E i or i N E j 0 othervise Where α is a constant and s i, j as social distance Building graph G(A) with matrix size n " x n " n " n " (a 9,: ) j 5/27/16 9

10 Clustering Clustering algorithm v Using Spectral Clustering for Livehoods Input Output n " n " (a 9,: ) Spectral Clustering for Livehoods 5/26/16 10

11 Clustering Implementation v Implementation and building website They implement clustering model on their data collections From output clusters, building Livehoods website. They call cluster as Livehood (LH) 5/26/16 11

12 Content 1 Introduction 2 Clustering 3 Interview 4 Result and Discussion 5 Conclusion 5/26/16 12

13 Interview Interviewee Stat v Stats 27 interviewees as residents of Pittsburgh (Nov 17 Dec 17 5/26/ ) >=18 years-old, live in their neighborhood at least one year. Paid 10$ in one hour interview. v 22/27 interviewees 12 females and 10 males Age: Mean 35, min 23, max 62, standard deviation 11. Education: 1 high school, 2 college, 10 bachelor, 2 graduated school, 7 master. v 5/27 interviewees 1 senior planner from Pittsburgh Office of city planning,1 realestate developer, 3 partners of 1 large real estate development firm 13

14 Interview Interview Protocol v Goal Validating clusters discovered by their algorithm. Exploring similarities and differences between their result clusters (Livehoods) and official municipal neighborhood boundaries in 3 dispersion patterns: LH1 Municipal LH2 (1) split: a municipal neighborhood contains >= one Livehood (2) spilled: when 1 Livehood cluster spills over the boundaries of a municipal borders. (3) coressponding: the Livehood cluster and municipal borders coincide 5/27/16 LH2 Municipal LH7 Municipal LH8 14

15 Interview Interview Procedure Open discussion Draw neighborhood boundary Ask shift in feel Show municipal map Ask for feedbacks Explain Livehood clusters Show Livehood website Ask comments and shifting border Reveal algorithm works Show related area 5/26/16 15

16 Content 1 Introduction 2 Clustering 3 Interview 4 Result and Discussion 5 Conclusion 5/26/16 16

17 Result and Discussion Shadyside and East Liberty Market Market 5/26/16 17

18 Result and Discussion Shadyside and East Liberty - Comments from interviewees: - Kelly, 29 years-old eastern Shadyside resident - Erin, 24 years-old, graphic designer - Donna, 62 year old, East Liberty resident 5/26/16 18

19 Result and Discussion Shadyside and East Liberty - Discussion - Municipal neighborhood border: LH1: split ; LH2: Spilled - Developments and resources: LH2 spilled -> economic development. 5/27/16 19

20 Result and Discussion Lawrenceville and Polish Hill 5/26/16 20

21 Result and Discussion Lawrenceville and Polish Hill - General information - Comments from interviewees: - Daniel, 43 years-old Lawrenceville resident - Claudia, 54 yearsold, journalist - Roger, 47 year old, Polish Hill resident - Jessica, LH3 resident 5/26/16 21

22 Result and Discussion Lawrenceville and Polish Hill - Discussion - Municipal neighborhood border: - LH3-LH4-LH5: corresponding - LH3-Polish Hill: spilling. 5/27/16 22

23 Result and Discussion The South Side 5/26/16 23

24 Result and Discussion The South Side - General information - Comments from interviewees: - Ashley, 25 years-old - Sara, 30 years-old, designer - Kara, resident between LH7-LH8 - Zack, 30 years-old, technology consultant 5/27/16 24

25 Result and Discussion The South Side 5/27/16 - Discussion - Municipal neighborhood border: LH6- LH7LH8- LH9 : split. - Demographics: LH6 LH8 LH9. - Development and Resources: LH6 - Geography and Architecture: building density in LH7 and LH8 25

26 Content 1 Introduction 2 Clustering 3 Interview 4 Result and Discussion 5 Conclusion 5/26/16 26

27 Conclusion Contribution v Dynamics of a city based on social media Livehood mappings show divisions and subtle changes in local social pattern. Various factors effect on dynamics of a city: municipal borders, demographics, development, resources, geography and architecture. Each Livehood manifests itself v Dynamics of a city based on social media Visualizing Livehoods 5/26/16 27

28 Conclusion Limitations v Data aggregation majority bias Hide behaviors of minority v Data collection Sample limitation (42,787 Check-ins; 3,840 Users; 5,349 Venues) 4sq users: years-old 5/27/16 28

29 Thank You 5/26/16 29