An Axiomatic Framework for Multi-channel Attribution in Online Advertising

Transcription

1 An Axiomatic Framework for Multi-channel Attribution in Online Advertising Garud Iyengar Columbia University Workshop on Marketplace Innovation 2017 Joint work with Antoine Desir, Vineet Goyal and Omar Besbes Partially support by a grant from Adobe 1

2 Optimization Cycle in Online Advertising Payments Ad Actions Channels Conversion Value Optimize Media Mix 2

3 Optimization Cycle in Online Advertising Payments Ad Actions Channels Conversion Value Optimize Media Mix A lot of academic effort and progress on media mix optimization... relatively less progress on the measurement and attribution steps 2

4 Multi-Channel Attribution in Online Advertising Display Sponsored Search... Conversion Question: How to give credit to all channels contributing to a conversion? 3

5 Common current attribution methods None 44% First or Last touch 30% Even weights 17% Customized weights 17% Algorithmic 11% Other 1% Source: Beyond Last Touch: Understanding Campaign Effectiveness, 2013 Quantcast 4

6 Goals for this work Framework for attributing credit to channels Evaluate heuristic procedures Address limitations of current heuristics and propose prescriptions 5

7 Related work Heuristics for multi-touch attribution Shapley value: [Shao and Li, 2011], [Dalessandro et al., 2012], [Berman, 2013], [Li and Kannan, 2014] Incremental impact: [Abhishek et al., 2012], [Anderl et al., 2014] Empirical work [Kireyev et al., 2013], [Lewis and Rao, 2014] Other related work [Jordan et al., 2011], [Archak et al., 2010] 6

8 Model elements: channels and customer states Display Sponsored Search... Unaware Interested... Considering Conversion 7

9 Model elements: channels and customer states Action Display Sponsored Search... Unaware Interested... Considering Conversion Two possible effects of an action: 7

10 Model elements: channels and customer states Action Display Sponsored Search... Unaware Interested... Considering Conversion Two possible effects of an action: Network effect 7

11 Model elements: channels and customer states Action Display Sponsored Search... Unaware Interested... Considering Conversion Two possible effects of an action: Network effect and Funnel effect 7

12 Approach Observational inference Each channel c shows the ad to β c fraction of customers Observe one-step browsing behavior Attribution Payment per impression Payment independent of the customer state or browsing behavior 8

13 Model Markov chain consisting two state variables σ = (c, s) c = identity of the channel, e.g. particular website s = latent state of the visitor Will ignore latent state in the talk. All results go through! 9

14 Model Markov chain consisting two state variables σ = (c, s) c = identity of the channel, e.g. particular website s = latent state of the visitor Will ignore latent state in the talk. All results go through! Transition probability P (1) c,c P (0) c,c = probability c c when a visitor exposed to an ad on channel c = probability c c when a visitor not exposed to an ad on channel c Transition when β fraction see an ad P β = diag(β)p (1) + diag(1 β)p (0) 9

15 Model (contd) Two absorbing states c = conversion. Conversion only on advertising. q = quit π c = conversion probability from channel c 10

16 Model (contd) Two absorbing states c = conversion. Conversion only on advertising. q = quit π c = conversion probability from channel c λ c = external traffic to channel c and µ β c = effective traffic at c µ β = λ + (P β ) µ β h β c = conversion probability from c h β = P β h β + π 10

17 Payment per impression Channel c is paid z c per impression Payment not a function of the latent state Payment not a function of the downstream browsing 11

18 Payment per impression Channel c is paid z c per impression Payment not a function of the latent state Payment not a function of the downstream browsing Desirable properties Channel rationality: z 0 Advertiser rationality: global budget balance λ h β }{{} Total value generated (µ β ) diag(β)z }{{} Total payments Fairness: hard to quantity! 11

19 Last touch Pay each channel one unit for each conversion Not quite allowed in our formulation but admits equivalent representation through payment per impression Trivially satisfies global budget balance. But, very unfair! 12

20 Last touch Pay each channel one unit for each conversion Not quite allowed in our formulation but admits equivalent representation through payment per impression Trivially satisfies global budget balance. But, very unfair! Example P (1) i,i+1 = 1 P (0) = 0 λ 1 = 1, λ i = 0, i 1 β i c q q q q All value goes to channel 4 although all traffic brought in by channel 1! 12

21 Last touch Pay each channel one unit for each conversion Not quite allowed in our formulation but admits equivalent representation through payment per impression Trivially satisfies global budget balance. But, very unfair! Example P (1) i,i+1 = 1 P (0) = 0 λ 1 = 1, λ i = 0, i 1 β i Channel rationality Global budget? Fairness q q q q c All value goes to channel 4 although all traffic brought in by channel 1! 12

22 Incremental value heuristic (IVH) Incremental value of the advertising action Advertise: P (1) h β + π Do not advertise: P (0) h β Incremental value: (P (1) P (0) ) h β + π }{{} := 13

23 Incremental value heuristic (IVH) Incremental value of the advertising action Advertise: P (1) h β + π Do not advertise: P (0) h β Incremental value: (P (1) P (0) ) h β + π }{{} := Incremental value heuristic: z = h β + π. Pays for both immediate conversion: π network effect: h β 13

24 Incremental value heuristic (IVH) Incremental value of the advertising action Advertise: P (1) h β + π Do not advertise: P (0) h β Incremental value: (P (1) P (0) ) h β + π }{{} := Incremental value heuristic: z = h β + π. Pays for both immediate conversion: π network effect: h β Example: z c 1 fair! But c µ β c β c z c = n c λ c h β c = 1! c q q q q 13

25 Incremental value heuristic (IVH) Incremental value of the advertising action Advertise: P (1) h β + π Do not advertise: P (0) h β Incremental value: (P (1) P (0) ) h β + π }{{} := Incremental value heuristic: z = h β + π. Pays for both immediate conversion: π network effect: h β Example: z c 1 fair! But c µ β c β c z c = n c λ c h β c = 1! Channel rationality? Global budget Fairness q q q q c 13

26 IVH: generically overbudget Proposition. Let δ = µ T diag(β) h(β). Then λ h β (µ β ) diag(β)z = δ δ > 0 when advertising generates downstream indirect conversions 14

27 IVH: generically overbudget Proposition. Let δ = µ T diag(β) h(β). Then λ h β (µ β ) diag(β)z = δ δ > 0 when advertising generates downstream indirect conversions Over payment: Every conversion gets counted several times pay for both for direct conversion, and eventual conversion 14

28 IVH with past payments Need to account for past payments! Proposal: z c = ( h) c + π c E[past payments c] 15

29 IVH with past payments Need to account for past payments! Proposal: z c = ( h) c + πc E[past payments c] Example 1 ɛ λ ɛ ɛ a b c Channel b: past payment = z a and ( h) b + π b = ɛ. Thus, z a ɛ. 15

30 IVH with past payments Need to account for past payments! Proposal: z c = ( h) c + πc E[past payments c] Example 1 ɛ λ ɛ ɛ a b c Channel b: past payment = z a and ( h) b + π b = ɛ. Thus, z a ɛ. Channel a: cannot be compensated for the 1 ɛ fraction it converts! 15

31 Outcome based cost accounting Requirements discriminate between outcomes when allocating payments local budget balance must ensure global budget balance 16

32 Outcome based cost accounting Requirements discriminate between outcomes when allocating payments local budget balance must ensure global budget balance Cost per visitor at channel c: p c + β c z c past payment p c payment per impression z c 16

33 Outcome based cost accounting Requirements discriminate between outcomes when allocating payments local budget balance must ensure global budget balance Cost per visitor at channel c: p c + β c z c past payment p c payment per impression z c Allocate cost x a,b along the arc (a, b) Axiom 1: Cost proportional to outcome value: x a,b h β b 16

34 Outcome based cost accounting Requirements discriminate between outcomes when allocating payments local budget balance must ensure global budget balance Cost per visitor at channel c: p c + β c z c past payment p c payment per impression z c Allocate cost x a,b along the arc (a, b) Axiom 1: Cost proportional to outcome value: x a,b h β b Axiom 2: Forward payment conservation: E[x a,b ] = p a + β a z a 16

35 Outcome based cost accounting Requirements discriminate between outcomes when allocating payments local budget balance must ensure global budget balance Cost per visitor at channel c: p c + β c z c past payment p c payment per impression z c Allocate cost x a,b along the arc (a, b) Axiom 1: Cost proportional to outcome value: x a,b h β b Axiom 2: Forward payment conservation: E[x a,b ] = p a + β a z a Axiom 3: Backward payment conservation: p c = b P(b c)x b,c 16

36 Outcome based accounting (contd) Axioms 1+2 Axiom 3 x a,b = (p a + β a z a ) h β b c P β a,ch β c + π a = (p a + β a z a ) hβ b h β a p c = b = b P(b c)x b,c ( µb P β bc µ c ) hc h b (p b + β b z b ) Definition. (Local budget balance) p c + β c z c h β c 17

37 Revisit previous example 1 ɛ λ ɛ ɛ a b c 18

38 Revisit previous example 1 ɛ λ ɛ ɛ a b c Past payments p a = 0 p b = z a hb h a = z a ɛ 1 ɛ + ɛ 2 18

39 Revisit previous example 1 ɛ λ ɛ ɛ a b c Past payments p a = 0 p b = z a hb h a = z a ɛ 1 ɛ + ɛ 2 Budget balance conditions z a h a = 1 ɛ + ɛ 2 ɛz a z b + 1 ɛ + ɛ 2 h b = ɛ 18

40 Revisit previous example 1 ɛ λ ɛ ɛ a b c Past payments p a = 0 p b = z a hb h a = z a ɛ 1 ɛ + ɛ 2 Budget balance conditions z a h a = 1 ɛ + ɛ 2 ɛz a z b + 1 ɛ + ɛ 2 h b = ɛ z a = 1 2ɛ + 2ɛ 2 ɛ 3 and z b = ɛ 2 feasible! 18

41 Revisit earlier example c q q q q Past payments p 1 = 0 p k = (p k 1 + z k 1 ) h k = p k 1 + z k 1 = z j h k 1 j<k 19

42 Revisit earlier example c q q q q Past payments p 1 = 0 p k = (p k 1 + z k 1 ) Local budget balance h k = p k 1 + z k 1 = z j h k 1 j<k Global budget balance! p k + z k h k = 1, k n z k 1 k=1 19

43 Local balance implies global balance! Theorem. Suppose p c + β c z c h β c, c (Local budget balance) Then (µ β ) diag(β)z λ h β. (Global budget balance) Proof. Use definition of p c Sum all local budget balance constraint Careful accounting of slacks 20

44 Feasible per impression payments z Channel rationality: z 0 Advertiser rationality: global budget balance implied by Outcome based cost allocation Local budget balance Fairness 21

45 Feasible per impression payments z Channel rationality: z 0 Advertiser rationality: global budget balance implied by Outcome based cost allocation Local budget balance Fairness IVH with payments: Value V c at channel c satisfies V = P β V + π diag(β)z Reduces to h β when z 0 z V + π [No lift = No payment] Does not identify z uniquely! Introduce an objective function. 21

46 Maximize payments Example c q q q q Max payment any payment satisfying global budget constraint k z k 1 22

47 Maximize payments Example c q q q q Max payment any payment satisfying global budget constraint k z k 1 Very fragile! π 1 = ɛ: unique optimal solution: z 1 = h 1 and z j = 0 for j > 1 22

48 Nash bargaining solution Let Z denote a set of feasible payments for a given network. Z = z p c = b V = P β V + π diag(β)z z V + π p + diag(β)z h β ( µb P β ) bc hc (p b + β b z b ) µ c h b 23

49 Nash bargaining solution Let Z denote a set of feasible payments for a given network. Let f denote a function that maps Z to an element z Z that satisfies Pareto efficiency Symmetry Invariance to scaling payoffs Independence of Irrelevant Alternatives Then f(z) = max z s.t. n i=1 z Z ln(z i ) 23

50 Sensitivity analysis: Path c q q q q Nash bargaining solution: µ i β i z i = µ i h i /n z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH

51 Sensitivity analysis: Path c q q q q Nash bargaining solution: µ i β i z i = µ i h i /n z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH Change π 1 = 0.1 z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH

52 Sensitivity analysis: Path c q q q q Nash bargaining solution: µ i β i z i = µ i h i /n z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH Set P (0) (3, 4) = P (1) (3, 4) = 0.5 z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH

53 Sensitivity analysis: Cycle q q q c z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH

54 Sensitivity analysis: Cycle q q q c Change π 1 = 0.2 z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH

55 Sensitivity analysis: Cycle q q q c z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH Set π 3 = 0 and P (1) (3, 4) = P (0) (3, 4) = 0.5 z 1 z 2 z 3 z 4 Total Max Nash Nash + IVH

56 Concluding remarks Principled framework for attribution cost approach build on accounting principles captures conversion funnel and network effect Current status obtain set of feasible costs per impressions avoids limitations of last touch and incremental value heuristics Further questions Alternative criteria for identifying unique z Empirical tests 26

57 Abhishek, V., Fader, P., and Hosanagar, K. (2012). Media exposure through the funnel: A model of multi-stage attribution. Available at SSRN Anderl, E., Becker, I., V Wangenheim, F., and Schumann, J. H. (2014). Mapping the customer journey: A graph-based framework for online attribution modeling. Archak, N., Mirrokni, V. S., and Muthukrishnan, S. (2010). Mining advertiser-specific user behavior using adfactors. In Proceedings of the 19th international conference on World wide web, pages ACM. Berman, R. (2013). Beyond the last touch: Attribution in online advertising. Available at SSRN Dalessandro, B., Perlich, C., Stitelman, O., and Provost, F. (2012). Causally motivated attribution for online advertising. In Proceedings of the Sixth International Workshop on Data Mining for Online Advertising and Internet Economy, page 7. ACM. 27

58 Jordan, P., Mahdian, M., Vassilvitskii, S., and Vee, E. (2011). The multiple attribution problem in pay-per-conversion advertising. In Algorithmic Game Theory, pages Springer. Kireyev, P., Pauwels, K., and Gupta, S. (2013). Do display ads influence search? attribution and dynamics in online advertising. Harvard Business School, Boston, MA. Lewis, R. A. and Rao, J. M. (2014). The unfavorable economics of measuring the returns to advertising. Available at SSRN Li, H. and Kannan, P. (2014). Attributing conversions in a multichannel online marketing environment: An empirical model and a field experiment. Journal of Marketing Research, 51(1): Shao, X. and Li, L. (2011). Data-driven multi-touch attribution models. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages ACM. 28