Online Appendix. Online Appendix A: MCMC Algorithm. The model can be written in the hierarchical form: , Ω. V b {b k }, z, b, ν, S

Size: px

Start display at page:

Download "Online Appendix. Online Appendix A: MCMC Algorithm. The model can be written in the hierarchical form: , Ω. V b {b k }, z, b, ν, S"

Jemimah Charleen Sherman
5 years ago
Views:

1 Online Appendix Online Appendix A: MCMC Algorithm The model can be written in the hierarchical form: U CONV β k β, β, X k, X β, Ω U CTR θ k θ, θ, X k, X θ, Ω b k {U CONV }, {U CTR b }, X k, X b, b, z, b, V b, Ω b {U CONV }, {U CTR b }, {b k }, X k, X b, V b, Ω, b, V Ω {U CONV }, {U CTR }, {b k b }, {b}, X k, X b, ν Ω, S Ω V b {b k }, z, b, ν, S b {b k }, z, A, b where b k = [β k θ k γ k α k ], b = [β θ γ α], V b = [V β V θ V γ V α ], b = [Δ β Δ θ Δ γ Δ α ], b = [β θ γ α ], = b [Δ β Δ θ Δ γ Δ ] α and where b X k are independent variables with keyword specific coefficients and X b are independent variables with common coefficients in equations 2,4,5 and 6. We have used as the initial value for elements of b k,b, b, b, and an identity matrix as an initial value for elements of V. Further, V = 1I

2 ν Ω = 1, S Ω = 1I ν = 1, S = 1I Δ α =, A =.1I The MCMC algorithm is described below. Step I: DrawU CONV CTR & U We use a data augmentation approach and random walk Metropolis-Hastings algorithm for sampling U = (U CONV, U CTR ) (Rossi & Allenby, 25) CTR U new CTR = U old + δ CTR where δ CTR ~N(,.2I) CONV U new CONV = U old + δ CONV where δ CONV ~N(,.2I) The draws are accepted with a probability α where α = min [ exp[ 1 2 (U new X E ) new new A(U X E )]l(u ), 1] exp[ 1 2 (U old X E ) A(U old X E )]l(u old ) and where l(u ) is the likelihood of orders and clicks K T l(u ) = (Λ CONV Λ CTR ) Orders ((1 Λ CONV ) Λ CTR ) Clicks Orders (1 k=1 and X = [ βk β X k β + βx θ k θ X k + θx t=1 Λ CTR ) Impressions Clicks θ ] e = [ e 1 2 e ] where e 1 = ln(adpos ) γ k γ X k γ γ X & 2 e = ln(organic_comp ) α k α X k α α X E = W 12 W 1 22 e A 1 = W 11 W 12 W 1 22 W 21

3 W 11 = [ Ω 11 Ω 12 Ω 21 Ω 22 ], W 22 = [ Ω 33 Ω 34 Ω 43 Ω 44 ], W 12 = W 21 = [ Ω 13 Ω 14 Ω 23 Ω 24 ] Step II: Draw b k = [β k θ k γ k α k ] We define x k = X k β θ X k γ X k α [ X k ] U CONV β βx V β U y k = CTR θ θx, V = [ V θ γ ln(adpos ) γx V γ ], b k = [ ln(organic_comp ) αx α V ] α β z k θ z k γ z k [ α z k ] Q k = [(x k Ωx k ) 1 + V 1 ] 1 & b k = Q k [x k Ω 1 y k + V 1 ] b k Then b k ~N(b, k Q k ) Step III: Draw b = [β θ γ α] We define X β x = X θ X γ [ X α ] y = U CONV β k β X k U CTR θ k θ X k ln(adpos ) γ k γ X k α k ] [ ln(organic_comp ) α k X, V = 1I, b = [ ] Q = [(x Ωx) 1 + V 1 ] 1 & b = Q[x Ω 1 y + V 1 b ] Then b~n(b, Q) Step IV: Draw Ω K T Y Ω~IW(ν Ω + N, k=1 t=1 Y + S Ω ) where Y = observations, ν Ω = 1, S Ω = 1I U CONV β k β X k β βx U CTR θ k θ X k θ θx ln(adpos ) γ k γ X k γ γx [ ln(organic_comp ) α k α X k αx α ], N = No of

4 If a latent variable is used as an instrument for Organic_Comp, execute Steps V and VI. Otherwise skip to step VII Step V: Draw π π belongs to class c with a probability p C π We draw π as a binary variable with a posterior probability given by p C π = L({U CONV CTR },{U },{b k },{b},ν Ω,S Ω,π c ) p c C i=1 L({U CONV },{U CTR },{b k },{b},ν Ω,S Ω,π i ) p i where π c denotes that keyword k at time t is assigned to class c, L({U CONV }, {U CTR }, {b k }, {b}, ν Ω, S Ω, π c ) is the likelihood of the model evaluated at π c and p c is the prior probability of class c membership Step VI: Draw p c Draw new class probability p based on p ~ Dirichlet(1 + K, 1 + K 1 ) where K c denotes the sum of all c π i.e. K c = c k t π Step VII: Draw V β V θ V γ V α V β K ~IW(ν + N, (β k Δ β z k ) k=1 (β k Δ β z k ) + S) where N = No of keywords, ν = 1, S = 1I V θ K ~IW(ν + N, (θ k Δ θ z k ) k=1 (θ k Δ θ z k ) + S) where N = No of keywords, ν = 1, S = 1I V γ K ~IW(ν + N, (γ k Δ γ z k ) k=1 (γ k Δ γ z k ) + S) where N = No of keywords, ν = 1, S = 1I V α K ~IW(ν + N, (α k Δ α z k ) k=1 (α k Δ α z k ) + S) where N = No of keywords, ν = 1, S = 1I Step VIII: Draw Δ β Δ θ Δ γ Δ α

5 Then Δ β ~N(Δ, β q β ) where q β = [(z k z k ) 1 + A ] 1 & Δ β = q β [z k β k + A Δ ] β Δ β =, A =.1I Δ θ ~N(Δ, θ q θ ) where q θ = [(z k z k ) 1 + A ] 1 & Δ θ = q θ [z k θ k + A Δ ] θ Δ θ =, A =.1I Δ γ ~N(Δ, γ q γ ) where q γ = [(z k z k ) 1 + A ] 1 & Δ γ = q γ [z k γ k + A Δ ] γ Δ γ =, A =.1I Δ α ~N(Δ, α q α ) where q α = [(z k z k ) 1 + A ] 1 & Δ α = q α [z k α k + A Δ ] α Δ α =, A =.1I

6 Online Appendix B: Robustness of Results In this section we outline several steps we have taken to evaluate the robustness of our results. without Keyword heterogeneity: We evaluate an alternate model without keyword heterogeneity and compare it with our original model using Bayes Factor. We also compare our main model with another model where only the position variable has the random coefficient. We use the harmonic mean (Newton and Raftery, 1994) to calculate the log-marginal density based on the MCMC output. We report log-marginal densities and the Bayes factors in Table B1. Based on Bayes Factor we find strong evidence supporting our model with keyword heterogeneity. Holdout Sample Analysis As one test of robustness, we have attempted to verify the prediction accuracy of our results using a holdout sample. To do this, we consider data for the first 4 weeks for each keyword as the estimation sample and data for the same keywords for the remaining two weeks as the holdout sample. We use mean absolute percentage error (MAPE) for daily CTR and CONV values at the aggregate level and at the keyword level. The error values are reported in Table B2 and show that the model prediction accuracy is similar for both the estimation and holdout samples. This suggests that our model estimates are robust. with No Endogeneity Correction We also estimate a model without any endogeneity correction (Tables B3 and B4). While our results are qualitatively similar to the main model, the magnitude of coefficients is different from our main results. This suggests that there can be a bias in the estimates due to time varying unobservables. with Alternate Instruments for Organic Competition Lagged Organic Competition: We followed Villas-Boas and Winer (1999), Archak et al. (211) and Ghose, Ipeirotis, and Li (212) by using the lagged value of organic competition as an instrument for organic competition in conjunction with Google Trends data specifying search volume for each keyword. The current value will be correlated with the lagged value due to common cost components. A potential issue with the lagged value is that it can still be correlated with the error term for click and conversion

7 equations as there can be time trends in consumer demand and organic competition is correlated with this time trend. Controlling for trends through our use of search volume data for different keywords should alleviate most, if not all, such concerns. The corresponding results for CTR and CONV are shown in Tables B3 and B4 and are qualitatively similar to our main result. Latent Instrumental Variable: We also apply the latent instrumental variable (LIV) approach developed by Ebbes et al. (25, 29). In this approach, a binary unobserved IV partitions the endogenous variable (in our case organic competition) into two components, one uncorrelated with the error terms in the main model and the other correlated with the error terms in the main model (the models for click through rate and conversion rate). Ebbs et al. (25) show that all model parameters are identifiable, that one discrete instrument is sufficient in most cases, and that this specification is robust under various non-normal distributions of the true instrument. However, there is some loss of efficiency compared with a situation in which a true discrete instrument is used (see Ebbes et al. 29). This approach has been adopted by Zhang et al. (29) to address the endogeneity issues related to the impact of ad characteristics on sales. Similarly, Rutz and Trusov (211) and Rutz et al. (212) use the LIV approach to address the endogeneity of position in sponsored search ads. Using this approach organic competition can be specified as a function of the LIV variable as follows α (8) Organic_Comp = απ + ε where π is a the latent instrument which follows a c-dimensional multinomial distribution with probabilities p π and where p π C is the probability that the cth latent instrument is one. Following Rutz and Trusov (211) we assume that there are two categories (c=2). The specific estimation approach is described in the online appendix A. The corresponding results for CTR and CONV are shown in Tables B3 and B4 and are qualitatively similar to our main result. The posterior mean values for α are 1.33 (.4)*** and.65 (.2)*** and that of p π is.18 (.1)***.

8 with Instrument for Sponsored Competition We use the position of the competing ads as an input to calculate sponsored competition. It is possible that the position of competing ads is correlated with unobservable promotions run by these advertisers; however, the randomization of position through bids of our focal advertiser makes this less likely. Nonetheless, the order in which the competing ads appear could still be correlated with their relative promotions, which could potentially bias the estimate for sponsored competition. In order to correct for this bias, we follow the same approach to determine a suitable instrument as we did in the case of organic competition. For every keyword in our sample, we consider the competing firms in the sponsored listings and determine the average sponsored position for each competitor for other non-related keywords. We use this average sponsored position for each sponsored competitor for a keyword to compute the instrument for its sponsored competition as shown below: 1 IV = C 1 where Pos kc t is the sponsored position for a non-related keyword k c at time t n Pos kc kc k c t of the competitor C associated with keyword k and n kc is the number of non-related keywords for competitor C follows Using this approach, sponsored competition can be specified as a function of the IV variable as (9) Sponsored_Comp = δ k + δ k δ 1 IVSponsored + δ Time Time + ε with δ k = δ z k + u k δ and u k δ ~N(, V δ ) The corresponding results for CTR and CONV are shown in Tables B3 and B4 and are qualitatively similar to our main result. with Fixed Effect for Keywords We also use a fixed effects approach to control for keyword specific effects. This allows us to address potential correlation across keywords. Corresponding CTR and CONV results are shown in Tables B3 and B4 respectively. Higher values of organic competition result in a decrease in CTR and in an increase in conversion performance.

9 Effect of Organic vs Sponsored Competition We found that organic competition has greater impact than sponsored competition on consumer click and conversion performance. However, it is possible that the impact of sponsored competition is dampened because we separately control for ad position (which obviously depends on competition). We re-evaluate the effect of organic and sponsored competition without controlling for position in order to capture the full effect of the sponsored competition. We also account for the endogeneity of sponsored competition. The results for CTR and CONV are shown in Tables B3 and B4 respectively. We find that sponsored competition now has a negative effect on CTR but the effect is lower as compared to organic competition (Table B3). This suggests that consumers are more influenced by the relative position and the perceived quality of the organic competition as compared to that of sponsored competition. Our CONV model continues to find that organic competition has a positive impact on conversion rate and that sponsored competition has no effect on CONV performance (Table B4). This again suggests that buyers are more influenced by organic competition than sponsored competition. with Fixed Time Effects We also use a fixed effects approach to control for time specific effects. Corresponding CTR and CONV results are shown in Tables B3 and B4 respectively. Higher values of organic competition result in a decrease in CTR and in an increase in conversion performance. with Additional Keyword Characteristics We also test the validity of results with additional keyword characteristics such as keyword length and keyword popularity. We obtain the search volume data from Google Trends for each of our sample keyword and use the average search volume for each keyword during the panel period as a measure of its popularity. Corresponding CTR and CONV results are shown in Tables B3 and B4 respectively and are qualitatively similar to our main results. without sponsored competition

10 Finally, we validate our results using CTR and CONV models without sponsored competition to ensure that our results are not driven by high correlation between position and sponsored competition. Results are shown in Tables B3 and B4 and are qualitatively similar to our main results. References Archak, N., A. Ghose and P.G. Ipeirotis (211), Deriving The Pricing Power Of Product Features By Mining Consumer Reviews, Management Science, 57(8), Ebbes, P., M. Wedel, U. Böckenholt (29), Frugal IV alternatives to identify the parameter for an endogenous regressor, J. Appl. Econometrics 24(3) Ebbes, P., M. Wedel, U. Böckenholt, T. Steerneman (25), Solving and testing for regressor-error (in)dependence when no instrumental variables are available: With new evidence for the effect of education on income, Quantitative Marketing Economics, 3(4), Newton, Michael A and Adrian E. Raftery (1994), Approximate Bayesian Inference with the Weighted Likelihood Bootstrap, Journal of the Royal Statistical Society, Series b, 56, Rutz, O. and M. Trusov (211), Zooming In on Paid Search Ads - A Consumer-level Calibrated on Aggregated Data, Marketing Science, 3(5), Rutz, O., R. Bucklin,G. P. Sonnier (212), A Latent Instrumental Variables Approach to ing Keyword Conversion in Paid Search Advertising, Journal of Marketing Research, 49(3), Villas-Boas, J. Miguel, Russell S. Winer Endogeneity in Brand Choice s. Management Science 45(1) Zhang, J., M. Wedel, and R. Pieters (29), Sales Effects of Visual Attention to Feature Ads: A Bayesian Mediation Analysis, Journal of Marketing Research, 46 (1),

11 Table B1: Fit for Different s s Marginal Density Log-Bayes Factor Main vs. Main with random coefficients for only ad position without Keyword Heterogeneity Table B2: Prediction Accuracy for Estimation & Holdout Samples s CTR Fit (MAPE) CONV Fit (MAPE) Aggregate Keyword Aggregate Keyword Estimation Sample Holdout Sample Aggregate MAPE is the average MAPE across all datapoints. Keyword MAPE is the average of the average MAPE for different keywords

12 Table B3: Parameter estimates for CTR for Different s Const AdPos Organic_ Comp without endogeneity correction (.63)*** (.16)*** -.35 (.15)** Sponsored _Comp -.14 (.13) Organic Pos -.3 (.2) Quality Score Time.16 (.4)*** -.9 (.3)*** Using Lagged values as instrument (.44)*** (.15)*** -.45 (.17)*** -.6 (.12) -.15(.1).28 (.4)*** -.8 (.2)*** Brand.5 (.54).2 (.46) Specificity.4 (.68).26 (.58) Organic_ Comp x Brand.16(.33).9 (.34) Using Latent Instrument Variable (.29)*** (.21)*** -.3 (.14)** Using instruments for both organic and sponsored competition (.41)*** (.17)*** -.34 (.16)** -.2 (.11) -.6 (.13).1 (.1)*.25 (.3)*** -.8 (.2)*** -.2 (.1)*.18 (.3)*** -.1 (.4)** -.26 (.43) -.63 (.48) -.15 (.51) -.39 (.62) Fixed Effects (3.16) (.2)*** -.2 (.11)** -.1 (.6)* -.2 (.2).33 (.8)*** -.1 (.)*** with no control for the effect of ad position -5.5 (.48)*** -.67 (.14)*** -.46 (.14)*** -.4 (.2)**.34 (.7)*** -.9 (.3)*** (3.13) (.49)** -.54 (4.39) -.92 (.64) *,**,*** Statistically significant at 1%, 5%, and 1% respectively with Time Fixed Effects (.55)*** (.18)*** -.36 (.15)** -.16 (.11) -.2 (.2).27 (.4)*** with additional keyword attributes (.57)*** -1.2 (.18)*** -.4 (.15)*** -.9 (.12) -.2 (.2).22 (.5)*** -.1 (.3)** * without Sponsored Competitio n (.65)*** (.16)*** -.3 (.15)** -.4 (.2)**.34 (.1)*** -.7 (.2)***.22 (.45).15 (.51) -.2 (.47) -.14 (.58).77 (.7).23 (.6) -.13 (.32) -.5 (.36).24 (.35).2(.31).32(.37).42(.29) Sponsored _Comp x Brand.3(.26).18 (.27).1 (.24).33 (.28).29 (.31) Organic_ Comp x Specficity.46(.39).16 (.4) -.6(.27).1(.29) -.14 (.41). (.39).13 (.35).54(.36).44(.42).2(.3) Sponsored _Comp x Specificity.42(.32).15 (.33).15 (.3).35 (.37).3 (.44).23(.32) Keyword Length Popularity -.1(.47).72 (.26)***.3 (.13)**

13 Table B4: Parameter estimates for CONV for Different s Const without endogeneity correction (.68)*** Using Lagged values as instrument (.63)*** AdPos.31 (.31).65 (.21)*** Organic_ Comp 1.8 (.25)***.62 (.3)** Using Latent Instrument Variable (.39)*** 1.19 (.23)***.53 (.15)*** Using instruments for both organic and sponsored competition (.67)***.63 (.26)** 1.2 (.24)*** Sponsored_ Comp.24 (.18) -.27 (.14)*.3 (.12) -.2 (.19) Organic Pos -.1 (.2) -.4 (.2)** Quality Score -.5 (.6) -.11 (.4)** Time -.21 (.4)***.2 (.2) Brand.23 (.69) 1.62 (.56)*** Specificity 3.37 (.79)***.64 (.69).6 (.1)***.2 (.2) -.12 (.3)*** -.12 (.2)*** -.29 (.6)*** -.18 (.3)*** (.65)**.77 (.67) 1.65 (.75)** 2.92 (.81)*** Fixed Effects -1.8 (3.24).81 (.23)*** 1.16 (.19)*** -.49 (3.11) -.1 (.2) -.31 (.12)*** -.1 (.)***.31 (4.44) with no control for the effect of ad position (.52)***.69 (.18)*** with Time Fixed Effects (.58)*** 1.17 (.19)***.78 (.3)*** -.13 (.18) -.6 (.15).5 (.2)*** -.31 (.6)*** -.19 (.3)***.16 (.6)*** 1.2 (.88).3 (.2)** -.24 (.4)*** 1.47 (.74)**.91 (.61) 1.52 (.71)** with additional keyword attributes such as length and keyword popularity -.63 (.52) 1.27 (.26)***.99 (.23)*** -.4 (.13) -.4 (.2)** -.33 (.6)*** -.6 (.6) -1.1 (.69) -.91 (.93) without Sponso red Compet ition (.45)* **.84 (.22)* **.45 (.17)* **.7 (.3)* -.34 (.1)** * -.12 (.3) ***.29 (.53).66 (.85) Organic_ Comp x Brand.88(.55).6 (.4).48 (.38).12 (.55).76 (.56) 1.36(.34)**.52(.61) Sponsored_ Comp x Brand -.32(.34) (.34)*** -.12 (.27) (.36)*** (.34)*** -.7(.36).31(.33) Organic_ Comp x Specficity.5(.49) -.65 (.42) -.66 (.43) -.15 (.6).1 (.6).19(.57) -.67(.6) Sponsored_ Comp x Specificity Keyword Length Popularity (.41)** * -1.1 (.35)*** -1. (.34)*** (.43)*** -.3 (.42) -.55(.4) *,**,*** Statistically significant at 1%, 5%, and 1% respectively -.19(.48) (.32)*** -.8 (.18)***.74(. 48).41(. 46)

14 % Change in Profitability % Change in CONV % Change in CTR Online Appendix C: Keyword Budget Allocation Figure C1: Change in keyword performance estimates by considering organic competition % -5% -1% -15% -2% -25% -3% -35% -4% -45% -5% 16% 14% 12% 1% 8% 6% 4% 2% % -2% -4% 4% 3% 2% 1% % -1% -2% -3% -4% -5%

15 Change in Budget Allocation for all keywords Figure C2: Change in Budget Allocation by considering organic competition 6% 5% 4% 3% 2% 1% % -1% -2% -3% -4%

Index. Pagenumbersfollowedbyf indicate figures; pagenumbersfollowedbyt indicate tables.

Index. Pagenumbersfollowedbyf indicate figures; pagenumbersfollowedbyt indicate tables. Index Pagenumbersfollowedbyf indicate figures; pagenumbersfollowedbyt indicate tables. Adaptive rejection metropolis sampling (ARMS), 98 Adaptive shrinkage, 132 Advanced Photo System (APS), 255 Aggregation