Nitzan Carmeli. MSc Advisors: Prof. Haya Kaspi, Prof. Avishai Mandelbaum Empirical Support: Dr. Galit Yom-Tov

Transcription

1 Nitzan Carmeli MSc Advisors: Prof. Haya Kaspi, Prof. Avishai Mandelbaum Empirical Support: Dr. Galit Yom-Tov Industrial Engineering and Management Technion

2 Introduction o Interactive Voice Response (Voice Response Unit) 2

3 Introduction o Agents salaries are typically 60%-80% of the overall operating budget in call centers o Properly designed IVR system increase customer satisfaction increase loyalty decrease staffing costs Customers self-serve Increase profit 3

4 Introduction Literature Review Research Goals Search Model MDP Case Study Model Implications Summary Introduction o Call Center with an IVR system IVR? Success? Abandonment P servers Returns 4

5 Introduction Literature Review Research Goals Search Model MDP Case Study Model Implications Summary Literature Review o Methodologies for evaluating IVRs o Quantifying IVR usability and cost-effectiveness o Agent time being saved by handling the call, or part of the call, in the IVR o Original reasons for calling vs. experience (by analyzing end-to-end calls) 30% completed service in IVR Suhm B. and Peterson P. (2002, 2009) Only 1.6% completed it successfully o Designing and optimizing IVRs o Human-Factor-Engineering o Mainly comparing broad (shallow) designs and narrow (deep) designs. Examples: Schumacher R. et al. (1995), Commarford P. et al. (2008) and many more. 5

6 Literature Review Broad (shallow) design: Narrow (deep) design:

7 Introduction Literature Review Research Goals Search Model MDP Case Study Model Implications Summary Literature Review o Reducing IVR service times o The IVR menu as a service tree reducing the average time to reach a desired service Minimize f ( T ) Salcedo-Sanz S. et al. (2010) M = i= 1 t p i i o Stochastic search in a Forest o Models of stochastic search in the context of R&D projects o Optimality of index policy. Denardo E.V., Rotblum U.G., Van der Heyden L. (2004) Granot D. and Zuckerman D. (1991) 7

8 Research Goals o Modeling customer flow within an IVR system o Using EDA to estimate model parameters and identify usability problems: has actually inspired our theoretical model o Using optimal search solutions and empirical analysis to compare alternative designs and infer design implications 8

9 Search Model Adapted from an actual IVR of a commercial enterprise 9

10 Search Model o Notations G = ( V, E) - rooted tree representing the IVR system M V - set of tree leaves, representing IVR services E - Tree edges, representing IVR menu options s - root vertex, representing the IVR main menu A( i) - set of immediate successors of vertex i pre( i) - immediate predecessor of vertex i s pre( c) = a b a e d c g f 10

11 Search Model o Customer search o Customers pay for each unit of time they spend in the IVR o Customers receive reward when reaching a desired service o Customers may look for more than one service o Customers have finite patience 11

12 Search Model o Customer search o At each stage the customer can choose: 1. Terminate the search and leave the system 2. Terminate the search and opt-out to an agent 3. Move forward to one of i s immediate successors, j A( i) 4. Move backward to i s immediate predecessor, 5. Move backward to the root vertex, s s pre( i) opt-out b a opt-out e d c opt-out g f 12

13 Search Model o IVR flow animation 13

14 Search Model o States Sn = ( i, T, N ) Where i = current vertex T = Total time spent in the serach N : V 1 vector representing repeated visits to each vertex 14

15 Search Model o Model Parameters - Edges N ( i) t i, j - exploration time of edge e = ( i, j), given that vertex i was visited N( i) times before Where N( i) - number of previous visits to vertex i Sn ( i, T, N ) = ( N ( i S = ) ) n 1 j, T + ti, j, N + δ + ( i) 15

16 Search Model ID time before Play Account Balance Total for November2008 to June2009,All days Relative frequencies % :05 00:11 00:17 00:23 00:29 00:35 00:41 00:47 00:53 00:59 01:05 01:11 01:17 Time(mm:ss) (1 sec. resolution) Total [5-10) [10-15) [15-20) [20-30) [30-40) [40-50) [50-100) 16

17 Search Model o Model Parameters - Edges τ - customer patience τ exp( θ ), assumed ( N ( i) ) ( N ( i) τ τ τ ) Pi, j = P > T + ti, j > T = P > ti, j (memoryless) 17

18 Search Model o Model Parameters - Leaves P i r i tser t F - success probability of vertex i - reward earned from successful completion of vertex i ( i) - time spent in vertex i given that we have been successful in visiting it ( i) - time spent in vertex i given that we have not been successful in visiting it T i tser ( i) w. p. Pi = tf ( i) w. p. 1 Pi o For non-leaves vertices, i V \ M : P =1 i r =0, t (i)=0, t (i)=0 i ser F 18

19 Search Model o Search candidates A sequence U = ( i,..., i ) of vertices in V will be called a candidate 1 n Definition 1. A candidate U will be called proper if it starts with with s, and if every two sequential vertices is U : ik, i k + 1, satisfies one of the following conditions: b s a i i k + 1 k + 1 A( i ) pre( i ) i s, i = s k = k k + 1 k e d c Example: U = s a c f c a d g f 19

20 Search Model o Search candidates A sequence U = ( i,..., i ) of vertices in V will be called a candidate 1 n Definition 1. A candidate U will be called proper if it starts with with s, and if every two sequential vertices is U : ik, i k + 1, satisfies one of the following conditions: i i k + 1 k + 1 A( i ) pre( i ) i s, i = s k = k k + 1 k Λ Λ- The - The set set of of all all admissible proper candidates 20

21 Search Model o Search candidates X ( U ) - Net revenue earned by fathoming candidate U [ ] R( U ) = E X ( U ) ( Expectation over: patience, success probabilities, time in vertices and edges) * Our goal is to find a candidate U that maximizes R( U ) ; U Λ 21

22 Search Model o Search candidates Proposition 1. A proper candidate U could be dominated by another candidate (could not be optimal) if it has at least one of the following properties: 1. There is at least one leaf i M which appears more than once in U 2. The candidate U has a subsequence pre( i) i pre( i), where i M 3. The candidate U has a subsequence i s, where i M 4. The candidate U ends with i M 5. The candidate U has a subsequence i pre( i) i 6. The candidate U has a subsequence i s direct sequence to j, where L( i, j) = pre( i) 22

23 Search Model o Search candidates D - The set of all prope r candidates with at least one of the properties of Propositi on 1 Λ ' = Λ \ D The set of all admissible candidates Proposition 2. The set Λ ' is finite and each of its candidates is a finite path 23

24 Search Model Maximal Tree Algorithm T T 0. Initialization: T = ( V, E ) G 1. Spanning the tree with backward transitions 2. Spanning the tree with forward transitions 3. Updating T with the set of new vertices, M if M T = Stop, else, return to step 1. T 24

25 Search Model o Maximal Tree Algorithm U T : s a c f ( i T N ) Recall: S =,, n ( f ) c u U base( u) = current vertex index( u) = ordered vector representing former backward transitions ( f, c ) a d ( f, c ) 25

26 Search Model Theorem. A proper candidate U is admissible (in Λ') iff it is represented T T ( V E ) by a path starting from the root, U T =, The set Λ' = Λ\D and the output of Maximal Tree Algorithm are equivalent 26

27 MDP o Backward induction Index calculation r( v) - discounted revenue earned from vertex v V T r( v) = payment for time spent in vertex r e ce ds w. p. P discounted reward from vertex tser ( v ) α tser ( v ) α s v 0 t F ( v ) ce α s ds 0 payment for time spent in vertex v w. p. 1 Pv z 27

28 MDP o Backward induction Index calculation * T r ( v) - discounted revenue earned from the tree with root vertex v V ( ) = ( ) * * R v E r v T * In the special case where v M : r ( v) = r( v) 28

29 MDP o Backward induction Index calculation T r - discounted revenue earned earned from edge ( u, v) E, u, v given that we successfully reached vertex u after t units of time u R = E r = u, v u, v tu + tu, v α ( tu + tu, v ) * α s E Ι{, } e R ( v) ce ds τ tu + t u v t u + Ι u { tu + tu, v > τ tu } τ α s ce ds τ tu t 29

30 MDP o Backward induction Index calculation We get: t * c c u Ru, v e α θ θ = Lt ( α + θ ) R ( v) 1 1 u, v + α α + θ α α + θ While in u, it is enough to calculate: R c θ c θ = L ( α + θ ) R ( v) α α + θ α α + θ * *, u v t u, v 30

31 MDP o Backward induction Index calculation v A( u) v A( u) * * u, v * Set σ ( u) [ ] if max R 0 : Set R ( u) = E r( u) tser ( u) ( * ) αtser ( u) α s ru + max R u, v e ce ds w. p. Pu v A( u) * * 0 u, v > = tf ( u) ( * ) αtf ( u) α s max u, v.. 1 v A( u) 0 if max R 0 : Set r ( u) = Stop R e ce ds w p P u * * Set R ( u) =E r ( u) v A( u) {, } * * Set σ ( u) = arg max R u v 31

32 MDP G = ( V, E) IVR original tree T T T = ( V, E ) Extended tree representing possible customer actions within the IVR system Calculating indices using backward induction to find optimal search policy 32

33 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Case Study Description o A medium bank o About 450 working agents per day (~200 during peak hours) o Call by call data from May 1 st, 2008 to June 30 th, 2009 o Overall Summary of calls IVR Calls Total % Out of total Average per weekday # Served only by IVR 27,709, % 50,937 # Requesting also agent service 17,280, % 31,765 10% of IVR calls Identification only! 33

34 Case Study Requested by less than 2% of the calls 34

35 Case Study o Estimating abandonments Threshold time Threshold time Threshold time 35

36 Case Study o Estimating abandonments ILBank IVR subevent duration, Query, Current Account - Recent Account Activity Total for May2008 to March2009,All days Fitting Mixtures of Distributions Relative frequencies % Components Mixing Proportions (%) Parameter Estimates Scale Shape Mean Standard Deviation 1. Lognormal Lognormal Lognormal Lognormal Lognormal Lognormal :02 00:11 00:21 00:31 00:41 00:51 01:01 01:11 01:21 Time(mm:ss) (1 sec. resolution) Empirical Total Lognormal Lognormal Lognormal Lognormal Lognormal Lognormal 36

37 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o Example - Comparing different designs (based on frequent services in our database) 37

38 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o Example - Comparing different designs c = i, j 0.05 NIS/sec α =0.05 θ =0.004 average customer patience = 250 sec t, t exponentially distributed ser F t exponentially distributed for every i, j Service r Type 1 tser (mean, sec) tf (mean, sec) Play account balance 200c Account Summary Recent Account Activity 400c Account Activity Today Selected Securities Stock Exchange Israel Transfer between Account Credit Card Vouchers 600c P 38

39 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o Example - Comparing different designs o Design A - original: Optimal candidate : U: Main menu Play Account Balance Main Menu Checking account Recent Account Activity Main menu General Activities Credit Card Vouchers Customer revenue: R( U ) =

40 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o Example - Comparing different designs o Design B - deep: Optimal candidate: U: Main Menu Play Account Balance Main Menu M1 M2 Recent Account Activity M2 M3 M4 M5 M6 Credit Card Vouchers Customer revenue: R( U ) =

41 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o Example - Comparing different designs o Design C - shallow: Optimal candidate: U : Main Menu Play Account Balance Main Menu Main Menu Credit Card Vouchers Recent Account Activity Customer revenue: R( U ) =

42 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o Comparing different designs Organization point of view Example, assume: o 10,000 calls per day o Cost per unit of time of communication = NIS/sec o Cost per second of agent call = 0.03 NIS/sec (based on UG project) Design A - Original Design B Deep Design C - Shallow Customer revenue Cost of communication Revenue from saved agent time Total organization profit 4,736 5,606 4,826 30,564 30,564 30,564 25,828 24,958 25,738 42

43 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Model Implications Description o How to design an IVR tree to encourage the use of certain services? o Customer learning? o Do customers navigate optimally? o Services with low demand lack of interest or lack of patience? o Sensitivity to customer patience? o Anticipating long waiting in agent queue does it affect customer behavior? o Using IVR and state information to control customers behavior examples: return later, use IVR for service 43

44 Introduction Literature Review Research Theoretical Goals Model Search Empirical Model Analysis MDP Case Model Study Implications Model Implications Summary and Summary Discussion System Summary Description and Future research o Stochastic models of customer flow within the IVR o Comparing IVR designs o Insights for better designs o Supplementing previous knowledge in HFE, CS fields o Data analysis as a basis for theoretical models o Can be easily modified to other self-service systems Example: Web search engines Hassan A. et al. (2010) o Interesting open questions: o How does one recognize an abandonment from self-service when seeing one? o Interrelation between customer-ivr interaction and customer-agent interaction? 44

45