Interactive Preference Measurement for Consumer Decision Support

Transcription

1 Interactive Preference Measurement for Consumer Decision Support John Liechty Srikant Vadali Arvind Rangaswamy Penn State University June 7, Our Corporate Sponsors: IBM, Unisys, Xerox, AT&T Wireless, Delphi Ventures, SAP America, Cigna, Tyco International, HP Pennsylvania State University 2004

2 Consumer Decision Support Systems Decision Theory Framework: Objective Function, Individual Level Utility Recommend best product Need an Individual Utility Function Attribute based product space Specifically understand high utility region Data: Experimental Sequence of Questions Timing: Real-time processing 6/11/2004 2

3 Helping Customers Make Good Choices in Crowded Markets Consumers face an increasing array of choices: Over 8,200 mutual funds Over 500 models of cars Over 30,000 products in a grocery store Over 100,000 prescription drugs 6/11/2004 3

4 Choosing is not so Easy! Finding the right one makes my headache worse! 6/11/2004 4

5 Active Sales Assistant: An example commercial service that helps customers makes choices in an online environment.

6

7

8

9

10

11

12

13 More Headaches for Customers! Filling out all these ratings also gives me a headache! 6/11/

14 Desirable Characteristics of Simplified Preference Assessment Methods Interactive Adaptive Focused/brief Real-time (< 5 seconds response latency) Share information across customers Have memory of past purchases of customer Useability Generate customer confidence 6/11/

15 General Strategy (sample) Model Development Strategy for sequential questions: i. Predictive error minimization ii. Treed models iii. Probabilistic fast polyhedral Prior Measurement And Analysis Obtain data either from a survey, or from past purchases. Generate prior distribution for model. Generate optimal questions (Treed only) Online Measurement And Analysis Based on ex ante modeling, dynamically determine best sequence of questions to ask (except with Treed approach). OPTIMAL RECOMMENDATIONS Ex ante conjoint model building Dialog at Web site 6/11/

16 Three Modeling Approaches Study 1: Sequential questioning to minimize predictive error around the most desirable option. Study 2: Making recommendations to impatient customers using demographic information. Study 3: Probabilistic fast polyhedral: Estimation and sequential design. 6/11/

17 Study 1: Ratings and the Predictive Error Pennsylvania State University 2004

18 Study 1: Ratings and the Predictive Error Model Development Strategy for sequential questions: i. Predictive error minimization Ex ante conjoint model building 6/11/

19 Study 1: Ratings and the Predictive Error Predictive Squared Error Loss Function L ( T y c β ) 2 n+ 1 = n+ 1 n+ 1 y n+1 c β n+1 : next rating to be given : element of design space (a product profile) : next estimate of partworths 6/11/

20 Study 1: Ratings and the Predictive Error Preposterior risk R n [ L ] = E yn+ 1, β + 1 Dn, X n, c n n X n+1 n : next profile to be rated (decision task at hand) { X X, y y } D =,...,,..., 1 n 1 n 6/11/

21 Study 1: Ratings and the Predictive Error Predictive Distribution Prior y ( T 2 X β ) n+ 1 X n+ 1, β n+ 1 = d N n+ 1 n+ 1, β n B = Σ + 2 σ i= 1 β σ ( 1 1 B b B ) 2 n+ 1 Dn, β 0, Σ, σ = d N, ( ) = N,Σ 1 d β 0 T X i X i b = Σ 1 β σ i= 1 : informative starting prior n X i y i 6/11/

22 Study 1: Ratings and the Predictive Error Minimizing preposterior risk R R n+ [ L ] = E n 1 yn+ 1, β n 1, 1, + + D n X n + c ( ) T ( 1 T )( ) 2 n+ 1 = σ + X n+ 1 c Σ + β nβ n X n+ 1 1 c positive X n = c +1 positive definite : minimizes preposterior risk What part of the design space does the decision maker care about? 6/11/

23 Study 1: Ratings and the Predictive Error Minimizing risk based on what is important to the individual: µ n () c :density on design space reflects areas of interest to decision maker Ω :design space, set of possible products 6/11/

24 Study 1: Ratings and the Predictive Error Minimizing expected risk based on what is important to the individual: Possible densities [ ] [ R ] E R X E [ c] min Eµ n+ 1 = µ n+ 1 n+ 1 = µ X n Ω Singleton: mass on design point with largest utility 2. Proportional: mass proportional to utility 6/11/

25 Study 1: Ratings and the Predictive Error Possible densities 1. Singleton: mass on design point with largest utility µ n { } ( ) T { * T c = I c β = max c β } 2. Proportional: mass proportional to utility 6/11/ n c * Ω ( ) T { } µ n c c β I c Ω n min c { } 0 { * T } min{ * T c β I c β } * n < * n Ω c Ω

26 Study 1: Ratings and the Predictive Error (sample) Model Development Strategy for sequential questions: i. Predictive error minimization Prior Measurement And Analysis Obtain data either from a survey or past purchases. Generate prior distribution for model. Chinese Dinner Study Ex ante conjoint model building 6/11/

27 Study 1: Ratings and the Predictive Error Study context: Chinese Dinners 8 attributes, 2, 3, or 4 options each 4 profiles for predictive validation Phase I -- Obtain Prior Distribution: 24 subjects β 0,Σ 27 profiles to rate (Orthogonal design) 6/11/

28 Study 1: Ratings and the Predictive Error (sample) Model Development Strategy for sequential questions: i. Predictive error minimization Prior Measurement And Analysis Obtain data either from a survey or past purchases. Generate prior distribution for model. Online Measurement And Analysis Based on ex ante modeling, dynamically determine best sequence of questions to ask. OPTIMAL RECOMMENDATIONS Ex ante conjoint model building Dialog at Web site 6/11/

29 Study 1: Ratings and the Predictive Error Phase II Dynamic Approach 20 students Dynamically generated product profiles using singleton density 6/11/

30 Study 1: Ratings and the Predictive Error Software Implementation Pennsylvania State University 2004

31 6/11/

32 6/11/

33 6/11/

34 6/11/

35 6/11/

36 6/11/

37 6/11/

38 6/11/

39 Study 1: Ratings and the Predictive Error Expectations Profiles being rated have a higher preferred score, on average, as number of profiles rated increases Calculate average rating, across all participants, each time a profile was rated y n = IND i= 1 y i, n IND 6/11/

40 Study 1: Ratings and the Predictive Error Expectations Profiles being rated have a higher preferred score, on average, as number of profiles rated increases Ability to forecast rating of most preferred profile increases over time (MSE gets smaller) Consider how well dynamic estimates predict the rating of the last profile Calculate MSE for last profile (across all participants) based on dynamic estimates of partworths MSE n = 1 IND IND ( T y ) i, N β n X i, N i= 1 2 6/11/

41 Study 1: Ratings and the Predictive Error Expectations Profiles being rated have a higher preferred score, on average, as number of profiles rated increases Ability to forecast rating of most preferred profile increases over time (MSE gets smaller) Ability to forecast over other parts of the design space decreases over time (MSE gets larger) Calculate the MSE (across all participants) for the hold-out profiles, based on dynamic estimates 6/11/

42 y n = IND i = 1 y i, n IND Average Preference Score (Across 20 individuals) Order of dynamic questions 6/11/

43 MSE IND 1 = n i, N n X IND = i 1 ( T y β ) i, N MSE of last dynamic product, using preference estimates after each question Order of dynamic questions 6/11/

44 MSE of Holdout Predictions 1500 MSE of hold out profiles Order of dynamic questions 6/11/

45 Study 3: Probabilistic Fast Polyhedral Model Pennsylvania State University 2004

46 Study 3: Probabilistic Fast Polyhedral Model Model Development Strategy for sequential questions: iii. Probabilistic fast polyhedral Ex ante conjoint model building 6/11/

47 Study 3: Motivation Polyhedral Approaches for Conjoint Analysis (Toubia et al. Marketing Science, 2003). Question sequence & Estimation of partworth utilities Key idea: Represent feasible values of partworths as a bounded polyhedron. Reduce polyhedron rapidly with optimal questions. Center of polyhedron is the partworth estimate. However, response errors (i.e., responses that lead to infeasible regions) are treated in a theoretically appealing manner. Our aim: Extend polyhedral method to incorporate response error using a well-defined probability model. 6/11/

48 Study 3:FASTPACE: Two-attribute Laptop Example Partworth for Size β Bounded Polyhedron Partworth for Price β 1 6/11/

49 Study 3: FASTPACE example cont d Ask respondent: How much do you prefer product 1 over product 2? Description of Product 1 Description of Product 2 Attribute Product 1 Product 2 Price $ 91 $ 70 Size Large Medium 6/11/

50 Study 3: FASTPACE example cont d Set least desirable level of attributes to 0. Each response results in a reduction of the dimensionality of the polyhedron. Partworth for Size 0 Partworth for Price β 1 β 2 = a Partworth estimate 6/11/

51 Study 3: FASTPACE example cont d Ask another question and refine our estimates. Partworth for Size Partworth estimate 0 Partworth for Price 6/11/

52 Study 3: FASTPACE Example cont d Response error is allowed only when inconsistent responses are detected. Allow for error Partworth for Size Partworth for Size 0 Partworth for Price 0 Partworth for Price 6/11/

53 Study 3: Extending FASTPACE Partworth for Size Incorporate response error within FASTPACE framework using a general probability model. 0 Feasible space is a line without error Partworth for Price Partworth for Size 0 Feasible space is an area if errors are allowed. a + δσ a δσ Partworth for Price 6/11/

54 Study 3: Extending FASTPACE After next question, the space shrinks further. Procedure ensures that there is always some probability mass in the feasible region. Feasible space after 2 questions Partworth for Size 0 Partworth for Price 6/11/

55 Study 3: Extending FASTPACE Enhanced HB Regression Framework a i ~ 2 N( X β, σ ) I( X β δσ 1 < a < Xβ + δσ 1) i i i i i i Partworth for Size 0 Xβ i + δσ Partworth for Price Xβ i δσ X β i i 6/11/

56 Study 3: Extending FASTPACE Probability Model for the parameters i β ~ N( β, Σ ) I( β 0) I( β 100) i i σ 2 ~ IG( shape, scale) Assume prior distributions for β ~ N(.,.), Σ ~ IW (.,.) 6/11/

57 Study 3: Preliminary Results Initial test of model done using laptop bag study of Toubia et al (2003). Details: 88 respondents. Respondent answered self-explicated questions followed by 20 paired comparison questions. Each paired comparison question had product descriptions consisting of 3 attributes that were chosen from 10 attributes. Each attribute had 2 levels. Hold-out task: Respondents rank-ordered 5 bags (selected randomly out of 16 available bags). Goal of test: To examine how well FASTPACE and our model predict the ranking of the bags. 6/11/

58 Study 3: Preliminary Results cont d Preliminary results show that our model performs on par with FASTPACE on holdout task (their laptop bags data). Our rank order correlation with actual choice: 0.68 FASTPACE rank order correlation: 0.68 Their Sawtooth software HB model s rank order correlation: /11/

59 Study 3: Current Research Underway Implement question design FASTPACE based on structure of polyhedron. Our approach: Based on probability model. Fully nest FASTPACE within our HB model Currently, FASTPACE is conceptually nested within our model. However, operationally, FASTPACE uses min-max criteria whereas we use min sum of squared errors criteria (i.e., OLS-type minimization). Develop implementation techniques for question sequencing and estimation in real-time web environments. 6/11/

60 Study 3: Conclusion We extend FASTPACE by incorporating response errors in a theoretically appealing manner. With test data (and without full nesting of FASTPACE within our approach), HB does as well as FASTPACE. Further research in progress to establish domains of applicability of our HB model (especially with reference to FASTPACE). 6/11/

61 Your thoughts/suggestions for future research? 6/11/