quan col... Probabilistic Forecasts of Bike-Sharing Systems for Journey Planning

Transcription

1 Probabilistic Forecasts of Bike-Sharing Systems for Journey Planning Nicolas Gast 1 Guillaume Massonnet 1 Daniël Reijsbergen 2 Mirco Tribastone 3 1 INRIA Grenoble 2 University of Edinburgh 3 Institute for Advanced Studies Lucca CIKM, 21 October / 27

2 Preamble Goal: provide users with information about bicycle/slot availability at a bicycle station in the future. Two types of predictions/forecasts: point forecast e.g., there will be 2 bikes available in fifteen minutes probabilistic forecast e.g., probabilities of finding 0, 1, 2 or 3 bikes in fifteen minutes are 25%, 25%, 30% and 20% respectively. Focus of literature about bike-sharing system analysis has been on point predictions. (Example: MoReBikeS Challenge.) Our paper: methodology for evaluating accuracy of forecasting methods, with focus on trip feasibility. 2 / 27

3 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 3 / 27

4 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 4 / 27

5 Bike Sharing Systems quan.....col Bike-sharing system: users rent bicycles for trips between stations. There are over 700 bike sharing systems across the world. Biggest systems worldwide: Wuhan and Hangzhou, with 90,000 and 60,600 bicycles respectively. Melbourne has 600. Biggest in Europe: Ve lib system in Paris, with 18,000 bicycles and image source: Wikimedia Commons 1,230 stations. 5 / 27

6 Data Some early experiments (1965 Amsterdam, 1993 Cambridge) failed because too many bikes went missing, others (1974 La Rochelle) worked because tourists had to hand in passport. Modern systems rely on IT-driven customer identification data The data can be used for multiple purposes, including: 1. Analysis of historical system performance, mostly in terms of the feasibility of trips (inhibited by empty or full stations). 2. Impact of bike redistribution strategies. 3. Improvement of the user experience: data of the current system state can be used to make predictions about trip feasibility. We will focus on Item 3. 6 / 27

7 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 7 / 27

8 Basic Bike Model I Known bicycle network topology: we focus on a single station. Y t,i : number of bikes at the station at time t on day i, t T, i I. T is the set of times in a single day, could be interval on R +, in practice discrete either up to seconds, or set of polling times (e.g., every 20 seconds). I is the set of days being considered. 8 / 27

9 Basic Bike Model II Assume that for some subset J I, probability measure governing Y t,i, i J, is the same; e.g., all weekdays the same, or all weekend days, and within the same season. Let Y t be the occupancy process on any day in J. 9 / 27

10 Research Question Let the bike station have fixed capacity c (in reality, this fluctuates in time). We are interested in probabilistic forecasts of future bike availability: P(Y t+h = y Y t = x), x, y {0,..., c}, t, h R +. Particular interest: y = 0, y = c. Alternatively, one could use a single-value or point prediction Ŷ t+h (x, t, h) of future bike availability. Literature focus has been on point predictions. 10 / 27

11 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 11 / 27

12 Classical Predictors Last Value Predictor: Does not depend on h. Historical Predictor: P(Y t+h = y Y t = x) = { 1 if y = x, 0 otherwise. P(Y t+h = y Y t = x) = 1 I {i I : y t+h,i = y}. Does not depend on x. 12 / 27

13 Queueing Model Predictor Idea: model a bike station as a single queue. Poisson arrival process with intensity λ(t) if non-full, departure intensity µ(t) if non-empty. This results in a time-inhomogeneous Continuous-Time Markov Chain with generator Q(t): ( h ) P(Y t+h = y Y t = x) = exp Q(t + s)ds. 0 x,y Assumption: λ(t) and µ(t) are piecewise constant functions of t. 13 / 27

14 Parameter Estimation We use the data to learn λ(t) and µ(t). Goodness-of-fit to Poisson distribution gives an idea about performance of queueing model predictor. CDFs CDFs # arrivals # arrivals (a) Station Pecqueur (b) Station Gare du Nord 14 / 27

15 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 15 / 27

16 Root Mean Square Error Given the fact that various prediction algorithms exist, the question is how to compare their performance. Literature benchmark: the Root Mean Square Error (RMSE), which can only be applied to point predictions. Given a vector x t,i of predictions and y t,i of observations for station n at time t on day i, with A the set of prediction/observation pairs, the RMSE is defined as 1 (x t,i y t,i ) A 2 (t,i) A for a given prediction method and time window h. 16 / 27

17 Remarks about the RMSE RMSE may depend more strongly on the variance inherent in the bike availability process than the performance of the prediction algorithm. RMSE only works for point predictions. Not suitable for evaluating trip feasibility predictions. Can we construct better performance metrics? 17 / 27

18 Proper Scoring Rules A scoring rule S(P, i) assigns to a combination of a probabilistic forecast P and an outcome i a score in R { }. The idea is that better prediction methods give higher scores. A scoring rule is said to be proper when the true probability distribution Q is the one that maximises the expected score. That is, for all forecasts P, E(S(Q, X )) E(S(P, X )). If the inequality is strict when P Q, S is said to be strictly proper. 18 / 27

19 Proper Scoring Rules: Example (I) Example: given a prediction, the user is advised to head to a station if the probability of finding an available bike is higher than 80%. Idea: we assign a score of 1 to correct predictions and a score of 0 to wrong predictions. 1 if P(X > 0) > 0.8 and x > 0, 0 if P(X > 0) > 0.8 and x = 0, Formally: S(P, x) = 0 if P(X > 0) 0.8 and x > 0, 1 if P(X > 0) 0.8 and x = 0. Is this a proper scoring rule? 19 / 27

20 Proper Scoring Rules: Example (II) Example: given a prediction, the user is advised to head to a station if the probability of finding an available bike is higher than 80%. Idea: we assign a score of 1 to correct predictions and a score of 0 to wrong predictions. This is not a proper scoring rule. For example, if the true probability of finding an available bike is always 70%, then a perfect predictor would be wrong 70% of the time, and yield an average score of 0.3. A predictor that incorrectly forecasts the probability of finding an available bike to be over 80% would be right 70% of the time, resulting in an average score of / 27

21 Proper Scoring Rules: Example (III) Say that given a prediction, the user is advised to head to a station if the probability of finding an available bike is higher than 80%. Proper scoring rule: 1 if P(X > 0) > 0.8 and x > 0, 4 if P(X > 0) > 0.8 and x = 0, S(P, x) = 1 4 if P(X > 0) 0.8 and x > 0, 1 if P(X > 0) 0.8 and x = 0. Includes a penalty of 4 for incorrectly recommending to go. 21 / 27

22 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 22 / 27

23 Classical Scoring Rules: Brier score. Spherical score. Logarithmic score. Brier-score (availability) Queue (QMP) historic (HP) last value (LVP) Spherical-score (availability) Queue (QMP) historic (HP) last value (LVP) prediction horizon (in hours) (a) Brier score prediction horizon (in hours) (b) Spherical score 23 / 27

24 Trip Feasibility Scoring Rules Proper scoring rules can be defined for different penalties U of incorrectly recommending to go. score of predictor (for the success) Queue QMP Historic HP Last-value LVP always go prediction horizon (in hours) (a) U=0 score of predictor (for the success) Queue QMP Historic HP Last-value LVP always go prediction horizon (in hours) (b) U= 5 score of predictor (for the success) Queue QMP Historic HP 0.0 Last-value LVP always go prediction horizon (in hours) (c) U= / 27

25 Probabilistic Forecasts of Bike-Sharing Systems 1 Motivation 2 Probabilistic Model 3 Prediction Algorithms Last Value Historical Mean Queueing Model 4 Evaluating Predictor Performance 5 Results 6 Conclusions 25 / 27

26 Conclusions & Future Work From the perspective of the user, probabilistic forecasts are more informative than point predictions. To compare forecasting algorithms, a good evaluation methodology is required. We introduced an evaluation methodology based on proper scoring rules. Particularly suited for trip feasibility predictions. Future work: include state-of-the-art prediction algorithms (based on time series analysis) in comparison. 26 / 27

27 Thank you for your attention. 27 / 27