Are You The Beatles Or The Rolling Stones?

Transcription

1 N. Ortelli, G. Lederrey, A. Alahi, M. Bierlaire Transport and Mobility Laboratory School of Architecture, Civil, and Environmental Engineering ENAC École Polytechnique Fédérale de Lausanne 13th Workshop on Discrete Choice Models June 21-23, 2018

2

3 The new and good ideas come from having a very broad and multidisciplinary range of interests. Robin Chase

4 Discrete choice models Understand and predict individual choice behavior using mathematical models 4 / 51

5 Discrete choice models Understand and predict individual choice behavior using mathematical models 4 / 51

6 Discrete choice models U in = V in + ε in 5 / 51

7 Discrete choice models U in = V in + ε in P in = e V in j Cn ev jn 5 / 51

8 Discrete choice models U in = V in + ε in P in = e V in j Cn ev jn V in =... + β c Cost in + β t Time in / 51

9 Discrete choice models U in = V in + ε in P in = e V in j Cn ev jn V in =... + β c Cost in + β t Time in +... }{{} linear in parameters 5 / 51

10 Discrete choice models U in = V in + ε in P in = e V in j Cn ev jn V in =... + β c Cost in + β t Time in +... }{{} linear in parameters Prediction accuracy + Interpretability 5 / 51

11 Machine learning Neural networks 6 / 51

12 Machine learning Neural networks Very good prediction accuracy 6 / 51

13 Machine learning Neural networks Very good prediction accuracy 6 / 51

14 Machine learning 7 / 51

15 Machine learning 7 / 51

16 Enhancing Discrete Choice Models with Neural Networks

17 General idea Bringing the predictive strength of Neural Networks, a powerful machine learning-based technique, to the field of Discrete Choice Models without compromising the interpretability of these models 9 / 51

18 Multinomial Logit as Convolution NN 10 / 51

19 Multinomial Logit as Convolution NN Activation Function: Softmax Loss: Categorical Cross-Entropy 11 / 51

20 Multinomial Logit as Convolution NN A single convolution gives us the utility functions 12 / 51

21 Model and Validation on Swissmetro Stated Preference Survey Simple utilities for validation purposes: V car = ASC car + β cost Cost car + β time Time car V train = ASC train + β cost Cost train + β time Time train V SM = ASC SM + β cost Cost SM + β time Time SM Biogeme betas Convolution Kernel weights 13 / 51

22 Neural Network enhanced DCM Case 1: same inputs The new term from the NN overruns the MNL linear parameters 14 / 51

23 Neural Network enhanced DCM Case 2: different inputs New term in the utility specification coming from a NN using all unused features 15 / 51

24 Neural Network enhanced DCM Case 2: different inputs U in = ASC i + β cost Cost in + β time Time in + β NN NN in + ε in Interpretation: NN in = uncaptured information of MNL model MNL estimates keep their significance Increase in log-likelihood 16 / 51

25 Swissmetro case Utility Functions 17 / 51

26 Swissmetro case Unused Features 18 / 51

27 Results Benchmark - standard MNL 19 / 51

28 Results Benchmark - standard MNL 19 / 51

29 Results Hybrid model ++ Increased Likelihood 20 / 51

30 Results Hybrid model ++ Increased likelihood 20 / 51

31 Results Hybrid model ++ Keeps important parameters significant 20 / 51

32 Results Hybrid model Some parameters loose significance 20 / 51

33 Results Simple hybrid model - only key parameters ++ Keeps important parameters significant 21 / 51

34 Results Simple hybrid model - only key parameters Increased likelihood + significant parameters 21 / 51

35 Results Comparison 22 / 51

36 Conclusions Enhancing DCM greatly increases likelihood Same input or correlated input - Breaks original parameter significance Independent Input - Best performances for likelihood - All parameters have good sign and significance 23 / 51

37 Future work Interpretability: what is really happening? 24 / 51

38 What do you think?

39 Automatic Utility Specification Using Machine Learning Techniques

40 Credit Nicola Ortelli 27 / 51

41 Are You The Beatles Or The Rolling Stones? Motivation Determining the appropriate utility specification for a particular application is a difficult task 28 / 51

42 Motivation Determining the appropriate utility specification for a particular application is a difficult task Expertise + Intuition 28 / 51

43 Motivation Determining the appropriate utility specification for a particular application is a difficult task Inspiration + Experience 28 / 51

44 Motivation Determining the appropriate utility specification for a particular application is a difficult task Trial-and-error 28 / 51

45 Motivation Determining the appropriate utility specification for a particular application is a difficult task Time consuming 28 / 51

46 Objective Build a procedure that automatically finds a good utility specification based on the data Define neighborhood relations between specifications and use classical local search algorithms 29 / 51

47 Automatic utility specification framework 30 / 51

48 Data partition 31 / 51

49 Data partition Data set separated into train set and test set Train set separated into K folds 32 / 51

50 Specifications and neighborhood structure 33 / 51

51 Specifications and neighborhood structure Current assumptions: Only continuous and binary variables Each continuous variable is included either on its own or in interaction with one binary variable All parameters are alternative specific Example: V =... + β x x +... V =... + β x,b1 =0 x (1 b 1) + β x,b1 =1 x b V =... + β x,b2 =0 x (1 b 2) + β x,b2 =1 x b V =... + β x,b1 =0,b 2 =0 x (1 b 1) (1 b 2) + β x,b1 =1,b 2 =0 x b 1 (1 b 2) + β x,b1 =0,b 2 =1 x (1 b 1) b 2 + β x,b1 =1,b 2 =1 x b 1 b / 51

52 Specifications and neighborhood structure Neighbors of a particular specification = all specifications that are a single change away from it Four possible changes: add, remove, interact, un-interact 35 / 51

53 Measure of performance 36 / 51

54 Measure of performance Current assumptions: Performance of a specification is measured as the log-likelihood it yields on new data Cross-validation is used to avoid overfitting or favorizing models with a large number of parameters The global performance P m of model m is defined as the sum of the log-likelihoods obtained on each fold after estimation on the K-1 others: K P m = k=1 L mk 37 / 51

55 Neighbor validation 38 / 51

56 Neighbor validation Conditions of acceptance of a model m : A neighbor that performs better than the baseline is always accepted P(accept m) = 1 if P m P mb One that performs worse might still get accepted, depending on the difference of performance P m P mb and the current iteration number P(accept m) = exp(pm Pm b ) T t if P m < P mb The algorithm ends when all neighbors of a model are rejected 39 / 51

57 Case study 1: Optima

58 Case study: Optima Revealed preference survey conducted in Switzerland for CarPostal: Three alternatives: public transports, car and soft modes observations left after discarding incomplete ones Two binary and six continuous variables are considered different specifications can be obtained 41 / 51

59 Best specification 42 / 51

60 Comparison with an existing model 43 / 51

61 Case study 2: SwissMetro

62 Case study: SwissMetro Stated choice survey to analyze the impact of the Swissmetro: Three alternatives: train, Swissmetro and car Nine different situations for each of the respondents Two binary and eight continuous variables are considered different specifications can be obtained 45 / 51

63 Best specification 46 / 51

64 Comparison with an existing model 47 / 51

65 Conclusions

66 Conclusions Despite a certain number of restrictions, results show that this topic is worth further investigation The procedure reaches very good specifications in a matter of minutes In some cases, the obtained model outperforms the benchmark with less variables under consideration 49 / 51

67 Future work VNS > neighborhood structures + relax assumptions What is a good utility specification? Predictability Parameters significance Behavioral interpretation / 51

68 Thank you! The fact is I ve always loved both bands :-)