IMPROVING INFRASTRUCTURE FOR TRANSPORTATION SYSTEMS USING CLUSTERING

Transcription

1 IMPROVING INFRASTRUCTURE FOR TRANSPORTATION SYSTEMS USING CLUSTERING Presented By: Apeksha Aggarwal Research Scholar, C.S.E. Department, IIT Roorkee Supervisor: Dr. Durga Toshniwal Associate Professor, C.S.E. Department, IIT Roorkee The Fifth International Conference on Big Data Analytics (BDA 2017), December, 2017 Hyderabad, India

2 Contents Introduction Literature Survey Literature Gaps Proposed Framework Methodology Results Conclusion References

3 Introduction Transportation systems are called the lifeline of any urban area. One of the major problem urbanization has caused is the traffic congestion, which is to be addressed for smart transportation. Owing to the adversities due to traffic most of the countries [2] around the world are taking productive steps for human health and environment by encouraging the use of public transport rather than the use of private diesel cars. This work encourages the more use of public transport over private vehicles. Public transport system includes bus, metro, trams, ferries etc.

4 Literature Survey In the past many problems related to smart transportation or smart mobility have been addressed. Finding traffic patterns, recommendation systems, parking etc. Improving experiences of driver traversing over the roads. Taxi transportation system: Taxis are an important part of both public and private transportation system of a city for providing door to door services. Bus transportation systems: Primary means of transport in many countries.

5 Literature: Improving driver s experiences Thiagarajan et al. [3] suggested VTrack, a hidden markov model (HMM) based technique to calculate the travel time and to suggest the paths most likely to be visited by people. T-Drive [4] is a system that provide driving directions according to traffic, driving habits of drivers, weather conditions etc. This system also recommends optimal paths to drivers to traverse on empty roads. Wang et al. [5] proposed a model which is real time and city wide for estimating travelling time on any road segments using trajectory data.

6 Literature: Taxi Transport System In cities people wait for non-trivial time for empty taxis while at other end cabs drivers wait deliberately to get commuters. Efficiently connecting vacant taxi with waiting passengers is important for saving time of the commuters, reducing energy consumption, reducing traffic, increasing profits etc. The main issue in this system is the uncertainty of the movement of the taxi while searching for the taxi and managing them is a further induced problem to be addressed. The algorithms used for finding free cabs are flooding and probabilistic [6] In flooding routing algorithm client request is broadcasted to all neighbors recursively until request reaches the free cab or TTL (time to live) becomes zero. Ge et al [7] developed a recommendation system to recommend the pick-up points and parking lots to the taxi drivers. This system aims at minimizing the energy wastage and maximizing profit of taxis. T-Finder [8] system gives the taxi drivers routes to the location where chances of getting passengers are high. This system also suggests nearby location points where empty cabs may be found out.

7 Literature: Bus Transport System Bus transportation is very important public transport system on which most people rely for commuting. In order to improve the bus transportation system, it is required to make bus services reliable and more frequent. Watkins et al. [9] studied the influence of the giving real-time information about bus arrival not only reducing the waiting time of people waiting at busstops but also of people who plan their journey on basis of such information. In case if GPS systems are not installed on the buses then alternative solution was given in [10]. Zimmerman et al. [10] proposed to use the GPS traces of the commuters of the bus. Then collected traces were processed and real-time prediction were made. Bastani et al. [11] proposed a system called flexi in which bus routes were found out by analyzing the trip data of very large set of the taxis.

8 Literature Gaps Identified The systems that have developed so far either focusses on improving driving experiences of drivers by estimating time, providing directions to destination or on improving search experience of taxis or bus services. There is not any significant work done on the finding out the new places in the cities where large number of people commute, using the GPS trajectories of large set of taxis and then checking whether bus routes are provided to that place or not. Clustering algorithms have been used in our work to solve this problem. Density based clustering algorithm and partitioning based clustering algorithms k-means and k-means++ are employed in this work.

9 Proposed Framework Threefold objectives of this work are given further in order to improve the bus transportation: Problem 1. First is to find out the interesting patterns in the traffic volume data of taxis and generating heat maps to identify the areas with more number of commuters in the city. Problem 2. Second is to find out the most traffic congested areas and among these areas select areas that do not have bus stops around them. Problem 3. Third is to provide recommendations about areas where new bus stops needs to be established.

10 Methodology: Data Mining Fig 1: Data mining steps 1. Data Preprocessing and Cleaning: In this step, missing values are treated and taxi data is transformed to preprocessed data. 2. Temporal Data Segmentation: Dividing data into time zones as traffic in the cities is variable with respect to time. 3. Clustering: Finding areas in the city contributing to high pick-ups by taxis. 4. Geocoding: Geocoding of existing bus stops with respect to their actual locations. 5. Recommendation: Finally recommendations for new bus stops is given.

11 Methodology: Data Mining Fig 1: Data mining steps DATA PREPROCESSING AND CLEANING In the data pre-processing step data cleaning is done. Removal of missing Values. Attribute selection -pick-up latitude, pick-up longitude, drop-off latitude and drop-off longitude, pick-up time.

12 Methodology: Data Mining TEMPORAL DATA SEGMENTATION TABLE 1: TEMPORAL DATA SEGMENTATION Time Zone Number Time Interval Time Zone 1 Time Zone 2 Time Zone 3 Time Zone 4 0:00 to 5:59 hours 6:00 to 11:59 hours 12:00 to 17:59 hours 18:00 to 23:00 hours Fig 1: Data mining steps

13 Methodology: Data Mining Fig 1: Data mining steps HEAT MAPS Heat maps are a great tool for representation of the data. They show the intensity of the data plotted on geographical maps. Heat maps are superimposed on google-maps to show the traffic intensity on different places.

14 Clustering Maximize the inter-cluster distance and minimize the intra-cluster distance. In k-means clustering [12] centroid of cluster is used to represents the cluster. Time complexity is O(nkt) where n is total data points, k is total clusters and t is number of iterations. DBSCAN: Density-based spatial clustering of applications with noise (DBSCAN) [12] is a density based clustering algorithm. Initially all points are marked as unvisited. Algorithm selects one of the point p, all those points which are in ε neighbor are added to N. Cluster C is expanded until it can t be further expanded. Time complexity of DBSCAN is O(n 2 ) where n is total data points. K-means++ Clustering: K-means clustering [13] have some limitations. In this clustering, initial selection of data points is not random unlike k-means. They are selected intelligently. Let the smallest distance between initial cluster center and any other data point be D i. Let there are n data points {x1, x2,...xn}. In k-means++ clustering number of clusters must be decided beforehand. Approximate number of clusters, k is decided by k = sqrt( n 2 ) Where k is number of clusters and n is the number of records.

15 Methodology: Data Mining CLUSTERING After merging, new clusters are represented as point (latitude, longitude) by the mean of latitudes and longitudes merged cluster centers. Latitude of new merged cluster = latitude m Longitude of new merged cluster = longitude m Algorithms used: K-Means, DBSCAN, K-Means++ clustering algorithms Fig 1: Data mining steps

16 Methodology: Data Mining Fig 1: Data mining steps GEOCODING Data of bus stops is in the form of postal addresses. Conversion into respective latitude and longitude points. Scikit-learn tool provided by Python has been used for geocoding [14]. Tool provides library that accurately converts addresses to the latitude-longitude point [15] in order to fetch bus stops in the form of latitude, longitude points.

17 Methodology: Data Mining BUS STOP RECOMMENDATION STEPS: Traverse cluster centers. Traverse bus stops. If cluster center is not close to any bus stop then recommend cluster center for new stop. Fig 1: Data mining steps

18 Dataset and Implementation TABLE 2: NUMBER OF RECORDS IN EACH TIME ZONE Time Zone Number of records Time Zone 1 11,33,806 Time Zone 2 14,61,442 Time Zone 3 16,03,335 Time Zone 4 17,72,738 Two datasets have been used in this work. First data is taxi volume data of New York City, USA for the month of June 2016 [17]. This data contains 18 attributes. Some of the attributes are pickup_datetime, dropoff_datetime, pickup_longitude, pickup_latitude, dropoff_longitude, dropoff_latitude. Total number of records in taxi volume data is 62,49,304. [17]. Second dataset is of addresses of the existing bus stops in the New York City, USA. Bus stops were given in the form of postal addresses. [18].

19 Results Fig 2: Variation of number of passengers in each hour Figure shows that during morning hours there is much less traffic compared to other times of the day. Pick-ups of passengers increases as day passes till 18:00 hours. In the evening number of pick-ups decreases slightly. Again, during night traffic increases. Hence the reason for division into four time zones each of six hour duration respectively.

20 Results: Heat Maps 3 a) 0:00 to 5:59 hours 3 b) 6:00 to 11:59 hours Fig 3: Heat Map superimposed on Google maps showing density of pick-up and drop-off points in different time zones 13 c) 12:00 to 17:59 hours 3 d) 18:00 to 23:59 hours

21 Results: Clustering 4 a) 0:00 to 5:59 hours 4 b) 6:00 to 11:59 hours Fig 4: Google Map showing center of k- means++ clustering over data of different time zones 4 c) 12:00 to 17:59 hours 4 d) 18:00 to 23:59 hours

22 Results Table 5: Comparison between results of 3 clustering algorithms Algorithm: K-Means++ K-means DBSCAN Time Zone: Time Zone 1 Time Zone 2 Time Zone 3 Time Zone 4 Time Zone 1 Time Zone 2 Time Zone 3 Time Zone 4 Time Zone 1 Time Zone 2 Time Zone 3 Time Zone 4 Number of clusters Number of new bus stops found 7 2 0

23 Results: Recommendation TABLE 4: LATITUDE AND LONGITUDE OF NEW RECOMMENDED STOPS Stop Number Latitude Longitude In order to make sure that area around bus stops have generally high traffic and high number of pick-ups by taxis, sample of taxi data set has been taken. Clustering was applied on this sample data set. Total clusters that formed are 443. These clusters are within the 400 meters distance from the nearest bus stop. Error percentage is calculated using: Error = Number of clusters in sample data Total number of bus stops Total number of bus stops Error came out to be 11.5% which is quite less. 100%

24 Results: Recommendation Fig 5: Google map showing existing bus stops (blue points) and recommended bus stops (red markers)

25 Ground Truth Figure 6 showing the ground truth. Figure 6A) shows that New York Department of Transportation is searching for new plans to reduce traffic in Woodhaven Boulevard (Stop number 2 in Table 4). Figure 6C) shows congestion in Borough Park (Stop number 7 in Table 4) due to opening of new schools. 6A) Traffic Congestion in Woodhaven 6B) New ferry service in Brooklyn 6C) Traffic congestion in Borough Park.

26 Conclusion This work encourages the more use of public transport over private vehicles so as to save environment, energy and resources by suggesting improvement in infrastructure of bus services. Experimental work on data collected from city of New York is presented in this work. This data collected from taxis is used to analyze the presence of traffic in the area. Temporal data segmentation with respect to different time zones is performed considering the dynamic patterns of urban traffic. Next, popular data mining techniques of clustering are applied on this segmented data to form clusters for each time zone so as to identify areas of high traffic. Further, another data set of bus stops is used to identify places with no bus stops and high traffic congestions. Henceforth new bus stops are suggested on places with high traffic density and no bus stops. Thus, a comparative study over baseline is done to recommend places that require bus stops.

27 References 1. Aggarwal, D. Toshniwal, Data Mining Techniques for Smart Mobility - A Survey Accepted in Proceedings of the 5th International Conference on Advanced Computing, Networking, and Informatics, June Thiagarajan, L. Ravindranath, K. LaCurts, S. Madden, H. Balakrishnan, S. Toledo, and J. Eriksson, VTrack: Accurate, energy-aware road traffic delay estimation using mobile phones, In Proceedings of the 7th ACM Conference on Embedded Networked Sensor Systems, Pages 85-98, J. Yuan, Y. Zheng, C. Zhang, W. Xie, X. Xie, G. Sun, and Y. Huang, T-Drive: Driving directions based on taxi trajectories, In Proceedings of ACM SIGSPATIAL Conference on Advances in Geographical Information Systems, ACM, Pages , L.Wang, Y. Zheng, and Y. Xue., Travel time estimation of a path using sparse trajectories, In Proceedings of the 20th SIGKDD Conference on Knowledge Discovery and Data Mining, ACM, P. Zhou. EZCab: A Cab Booking Application Using Short-Range Wireless Communication. In Proceedings Third IEEE International Conference on Pervasive Computing and Communications. PerCom, Pages Y. Ge, H. Xiong, A. Tuzhilin, K. Xiao, M. Gruteser, and M. Pazzani, An energy-efficient mobile recommender system, In Proceedings of 16th SIGKDD Conference on Knowledge Discovery and Data Mining, ACM, Pages , N. J. Yuan, Y. Zheng, L. Zhang, and X. Xie, T-Finder: A recommender system for finding passengers and vacant taxis, IEEE Transactions on Knowledge and Data Engineering, Pages , K. Watkins, B. Ferris, A. Borning, S. Rutherford, and D. Layton, Where is my bus? Impact of mobile real-time information on the perceived and actual wait time of transit riders, Transportation Research Part A 45, Pages , J. Zimmerman, A. Tomasic, C. Garrod, D. Yoo, C. Hiruncharoenvate, R. Aziz, N. R. Thiruvengadam, Y. Huang, and A. Steinfeld., Field trial of Tiramisu: Crowd sourcing bus arrival times to spur co-design, In Proceedings of the Annual Conference on Human Factors in Computing Systems, ACM, Pages , 2011.

28 References 11. F. Bastani, Y. Huang, X. Xie, and J.W. Powell, A greener transportation mode: flexible routes discovery from GPS trajectory data, In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, ACM, Pages , Jiawei Han, Micheline Kamber, Cluster Analysis: Basic Concepts and Methods, in Data Mining: Concepts and Techniques, 3rd edition, India: Academic Press, Arthur, David, and Sergei Vassilvitskii, "k-means++: The advantages of careful seeding", In the Proceedings of the Eighteenth annual ACM-SIAM symposium on Discrete algorithms, Society for Industrial and Applied Mathematics, Pages , Scikit-learn: Machine Learning in Python, Pedregosa, F. and Varoquaux, G. and Gramfort, A. and Michel, V. and Thirion, B. and Grisel, O. and Blondel, M. and Prettenhofer, P. and Weiss, R. and Dubourg, V. and Vanderplas, J. and Passos, A. and Cournapeau, D. and Brucher, M. and Perrot, M. and Duchesnay, E., In Proceedings of Journal of Machine Learning Research, Pages , Geopy , Walker Jarrett, "Purpose-driven public transport: creating a clear conversation about public transport goals", In the proceedings of Journal of transport geography 16.6, Pages , NYC Taxi and Limousine Commission-Trip Record Data, Metropolitan Transportation Authority (MTA) Bus Time, Vincenty, Thaddeus. "Direct and inverse solutions of geodesics on the ellipsoid with application of nested equations." Survey review (1975):

29 Thank you