arxiv: v2 [stat.ap] 28 Mar 2016

Transcription

1 Bayesian modeing of networks in compex business inteigence probems Daniee Durante, Say Paganin, Bruno Scarpa and David B. Dunson arxiv:0.00v [stat.ap] Mar 0 University of Padua, Department of Statistica Sciences, Via Cesare Battisti, Padua, Itay e-mai: saypaganin@gmai.com, e-mai: durante@stat.unipd.it, e-mai: scarpa@stat.unipd.it. Introduction Duke University, Department of Statistica Science, Box 0, Durham, NC 0, USA. e-mai: dunson@duke.edu Abstract: Compex network data probems are increasingy common in many fieds of appication. Our motivation is drawn from strategic marketing studies monitoring customer choices of specific products, aong with co-subscription networks encoding mutipe purchasing behavior. Data are avaiabe for severa agencies within the same insurance company, and our goa is to efficienty expoit co-subscription networks to inform targeted advertising of cross-se strategies to currenty mono-product customers. We address this goa by deveoping a Bayesian hierarchica mode, which custers agencies according to common mono-product customer choices and co-subscription networks. Within each custer, we efficienty mode customer behavior via a custer-dependent mixture of atent eigenmodes. This formuation provides key information on mono-product customer choices and mutipe purchasing behavior within each custer, informing targeted cross-se strategies. We deveop simpe agorithms for tractabe inference, and assess performance in simuations and an appication to business inteigence. Keywords and phrases: Business Inteigence, Chinese Restaurant Process, Co-subscription Networks, Cross-se Marketing Strategies, Mixture of Latent Eigenmodes, Mono-product Choices. Increasing business competition and market saturation have ed companies to progressivey shift the focus of their marketing strategies from the acquisition of new customers to an increased penetration of their customer base. Targeting existing customers via cross-se campaigns instead of attracting new ones provides a more effective strategy for the growth of the company and enhances customer retention by increasing the switching costs (Kamakura et a., ). Therefore, mono-product customers purchasing a singe product from a company represent a key segment of the customer base, and companies are naturay interested in expanding these customers purchases to additiona products. Business statistics currenty offers a wide set of procedures for cross-se reying on shared acquisition patterns of products, based on customer ownership data. A first effort in addressing this aim can be found in the atent trait mode deveoped by Kamakura et a. () to estimate the propensity of a customer towards a particuar product, based on its ownership of other products. This procedure has been ater improved by Kamakura et a. (00) combining information from customer databases with survey data. Verhoef and Donkers (00) focus instead on predicting the potentia vaue of a current customer via a mutivariate probit mode, and propose a two-by-two segmentation to improve targeting. Finay, Thuring (0) deveops a mutivariate credibiity method to identify a profitabe set of customers for cross-seing. This is accompished by estimating a atent risk profie for each customer, expoiting information imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

2 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing Agency : Co subscription network Agency : Co subscription network Agency : Co subscription network Agency : Mono product portfoio 0 Agency : Mono product portfoio 0 Agency : Mono product portfoio Fig : For three seected agencies. Upper pane: observed co-subscription networks A i. Back refers to an edge, white to a non-edge. Lower pane: tota number of mono-product customers for each product v =,..., based on choice data y is, s =,..., n i. on caims. Refer aso to Thuring et a. (0) and Kaishev et a. (0) for recenty deveoped cross-se strategies and for an overview on avaiabe methodoogies. Previous procedures expoit different sources, incuding customer demographics and survey data, to estimate co-subscription probabiities among pairs of products for each customer in a singe agency. Differenty from this setting, we do not observe customer demographic data for a singe agency, but monitor mono-product customer choices aong with co-subscription networks among V = products for n = 0 agencies operating in the Itaian insurance market. Customer reationship management is becoming increasingy important to effectivey operate in the insurance market. This sector is mosty stabe in deveoped countries, and rising customer expectations, aong with tight competition among top corporations and ow growth potentias, force companies to efficienty expoit their databases to create, manage and maintain their portfoio of profitabe customers (Matiş and Iieş, 0). We observe choice data y is {,..., V } denoting the product subscribed to by mono-product customer s =,..., n i, within agency i =,..., n. The co-subscription network for each agency i =,..., n is avaiabe via a V V symmetric adjacency matrix A i, with A i[vu] = A i[uv] = if in agency i the number of customers subscribed to both products v and u, exceeds 0% of the tota amount of muti-product customers subscribed to at east one of the two, for every v =,..., V and u =,..., v. A presence of an edge between two products suggests a preference of customers in agency i for that specific pair, controing for the tota number of muti-product customers subscribed to at east one of the two products. Refer to Figure for an exampe of avaiabe data. imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

3 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing Athough the 0% threshod can be adapted to the company requirements, we found the resuting adjacency matrices A,..., A 0 robust to moderate changes in this vaue. In fact, when the threshod is set at.% instead of 0%, the resuting adjacency matrices differ from the previous ones for ony % of the pairs. This gap remains on simiary ow vaues % and % when the threshod is set at.% and %, respectivey. Each agency can define appropriate cross-se strategies by expoiting its co-subscription network A i to estimate the propensity of a customer who subscribed to product v =,..., V to additionay buy u v. This eads to V different cross-se strategies u i,..., u iv, with u iv defining which additiona product u v is the best offer to currenty mono-product customers subscribed to v in agency i. Hence, u iv = argmax u {pr(a i[vu] = ) : u v}, with A i[vu] the random variabe characterizing the presence or absence of an edge between products v and u in agency i. Efficienty targeting advertising by offering customers the product mosty compementary to their current choice can substantiay improve performance reative to untargeted advertising, whie increasing the satisfaction and reducing churn effects due to frequent and pointess cross-seing attempts (Azzaini and Scarpa, 0; Kamakura et a., 00). Satisfied customers are a key to enhancing positive word-of-mouth communication and are ess sensitive to competing brands and price (Matiş and Iieş, 0). The effectiveness at increasing the number of muti-product customers in agency i depends not ony on the propensity of customers with v to aso subscribe to u, measured by pr(a i[vu] = ), but aso on the proportion of mono-product customers with v, defined by p iv = pr(y i = v), with Y i the random variabe denoting the choices of mono-product customers in agency i. If p iv is ow, then strategy u iv targets a sma portion of the customer base of agency i, and hence has a ow ceiing on effectiveness. To take into account the roe of p iv, we associate each strategy u iv with a performance indicator e iv = p iv max{pr(a i[vu] = ) : u v}, for each v =,..., V and i =,..., n. Strategies with a high e iv wi target a sizabe proportion of the avaiabe customers for that agency with advertising for a new product ikey to be appeaing to them. In defining and evauating cross-se strategies, there are two important issues to take into consideration. Firsty, we are faced with statistica error in estimating the components underying strategies u iv and indicators e iv, for each v =,..., V and i =,..., n. This is a particuar probem in estimating pr(a i[vu] = ) due to data sparsity. The second issue is that it is important to take into account the fact that administrative overhead can be reduced by considering the same strategy for different agencies within the same company. For groups of agencies having sufficienty simiar customer bases, an identica strategy can be adopted to reduce administrative costs without decreasing effectiveness. Motivated by this notion, we propose to address both the statistica error and administrative overhead issue through custering of agencies according to the parameters characterizing their customer bases, and then administrating the same strategy to a agencies within a custer. As suggested by Figure, it is reasonabe to expect agencies operating in simiar markets to exhibit custers, corresponding to common patterns in the composition of their mono-product customer choices and co-subscription networks. Efficient detection of such custers aows adaptive reduction of the tota number of strategies to be devised from u i,..., u iv, i =,..., n to u k,..., u kv, k =,..., K < n, with each custer-specific set of strategies u k,..., u kv maintaining its effectiveness in targeting simiar agencies. This higher eve targeting and profiing represents a key to baance the need of the company to reduce costs and the importance of providing agencies with effective strategies that account for their specific structure. Providing agencies with sets of strategies suitaby reated with their structure is further important to increase their trust in the company and improve synergy. We address this goa by deveoping a Bayesian hierarchica mode, which adaptivey associates shared sets of strategies u k,..., u kv and performance indicators e k,..., e kv to custers imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

4 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing of agencies characterized by mono- and muti-product customers with simiar purchasing behavior. Each custer-specific set of cross-se strategies u k,..., u kv is devised by earning custerspecific co-subscription patterns among pairs of products, expoiting data on muti-product customers. Joining this information with the estimated custer-specific distribution of monoproduct customers across products, performance indicators e k,..., e kv are constructed. To our knowedge, this is the first approach in the iterature that considers a two-eve cross-se segmentation of the customer base, which custers agencies with simiar customer preferences, and profies mono- and muti-product purchasing behavior within each custer to define cross-se strategies and reated performance indicators... Joint modeing of mixed domain data As a step towards our goa of designing efficient cross-se strategies, we first deveop a joint mode for the data {y i = (y i,..., y ini ), A i }, i =,..., n, which characterizes the distribution across agencies of the mono-product customer subscriptions aong with the co-subscription network for muti-product customers. The mode is chosen to be fexibe whie automaticay custering different agencies that have simiar mono-product customer choices and co-subscription network profies. This custering is usefu for borrowing information across agencies in efficienty and effectivey earning the joint distribution of the mono- and muti-product purchasing behavior of the customer bases. In addition, custering provides a simpification, which is usefu in design of interpretabe strategies. There is an increasing statistica iterature on joint modeing and co-custering of mixed domain data. Avaiabe procedures focus on earning the association between a univariate response variabe and an object predictor, typicay characterized by a function. Bigeow and Dunson (00) favor custering among predictor trajectories, with each custer associated to a specific offset in a generaized inear mode for the response variabe. Athough providing an appeaing procedure for the sake of interpretabiity and inference, their mode acks fexibiity in constraining predictor and response custers to be the same. This may require the introduction of many custers to appropriatey characterize the joint distribution of the mixed domain data, reducing the performance in estimating custer-specific components and providing a biased overview of the underying custering structure. Dunson et a. (00) address the previous issue by modeing the conditiona distribution of the response via a custer-dependent mixture representation, rather than considering ony a custer-specific offset in the conditiona expectation. In improving fexibiity via dependent mixture modeing, their procedure can better identify the underying custering structure; see aso Banerjee et a. (0) for a recent overview of this topic. Athough we are simiar to the previous methods in ooking for fexibe and accurate joint modeing and co-custering procedures for mixed domain data, our motivating data set is substantiay different in considering categorica mono-product customer choices and network-vaued co-subscription data. Fexibe modeing of the conditiona distribution of a network-vaued random variabe is sti an ongoing issue, which requires carefu representations in order to borrow information across edges, reduce the dimensionaity and maintain fexibiity. We propose a custer-dependent mixture of atent eigenmodes, which aows the distribution of the co-subscription networks to fexiby change across custers of agencies via custer-specific mixing probabiities, whie borrowing information among these agencies in earning the shared mixture components. Considering custer dependence ony in the mixing probabiities aows further dimensionaity reduction, whie providing simpe and efficient computationa methods. Differenty from Dunson et a. (00), we additionay avoid fixing the tota number of custers, but earn this key quantity from our data via a Chinese Restaurant Process prior for the custer imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

5 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing assignments. The paper is organized as foows. In Section, we carefuy describe our hierarchica formuation for joint modeing and co-custering of co-subscription networks and mono-product choice data. Prior specification and detaied steps for posterior computation are outined in Section. Section considers simuation studies to evauate the performance of our mode in a scenario mimicking our motivating appication. Resuts of the appication to our business inteigence probem in an insurance company are presented and discussed in Section. Finay, Section contains concuding remarks and suggests other possibe fieds of appication aong with further research directions.. Joint modeing of mono-product choices and co-subscription networks Let C = (C,..., C n ) denote a vector of custer assignments, with C i {,..., K} indicating the custer membership of agency i. Agencies within the same custer are characterized by a simiar composition of their mono-product customer choices as we as a comparabe cosubscription network. To compete a specification of the joint mode, we need to define a custerspecific probabiistic representation of the mono-product customer choices, as we as a custerspecific probabiistic generative mechanism underying the co-subscription networks. The atter is a key to define cross-se strategies u k,..., u kv in each custer k =,..., K, whie the former provides the additiona information to define the performance indicators e k,..., e kv. As a mono-product customer can be associated with ony one subscription v =,..., V, it is straightforward to define a probabiistic representation for the mono-product customer choices within each custer. Let Y k denote the categorica random variabe characterizing the choices of the mono-product customers in agencies within custer k, for each k =,..., K. To define the probabiity mass function associated to each Y k, we introduce a custer-specific vector p k = (p k,..., p kv ), with p kv indicating the probabiity that a mono-product customer in an agency within custer k subscribes to product v, for each v =,..., V. Assuming independence of the mono-product customer choices, the joint probabiity for data y i in agency i given its membership to custer k is pr(y k = y i )pr(y k = y i ) pr(y k = y ini ) = V v= p niv kv, () with n iv the number of mono-product customers in agency i who subscribed to product v. Within each custer k, co-subscription networks are reaizations from a network-vaued random variabe. As our network data are undirected and sef-reations are not of interest, each symmetric adjacency matrix A i, i =,..., n, is uniquey characterized by its ower trianguar eements comprising L(A i ) = (A i[], A i[],..., A i[v ], A i[],..., A i[v ],..., A i[v (V )] ). Hence, in defining a probabiistic generative mechanism for the co-subscription networks within each custer k, we can focus on the mutivariate random variabe L(A k ) with binary eements L(A k ) {0, }, measuring the presence or absence of an edge among each pair of products =,..., V (V )/ for agencies in custer k. Note that in our notation L( ) is an operator vectorizing the ower trianguar eements of a given symmetric matrix. As there are V (V )/ possibe configurations of co-subscription networks among V products, we cannot estimate the probabiity mass function associated to each L(A k ), k =,..., K nonparametricay without dimensionaity reduction. To reduce dimension whie maintaining fexibiity, we mode each L(A k ) via a custer-dependent mixture of atent eigenmodes. This eads to the foowing probabiity for the co-subscription network A i in agency i given its memimsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

6 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing bership to custer k: pr{l(a k ) = L(A i )} = H h= V (V )/ ν hk = { π (h) } L(Ai) { π (h) } L(Ai), () V (V )/ with each component-specific edge probabiity vector π (h) = (π (h),..., π(h) V (V )/ )T (0, ) defined as a function of a shared simiarity vector Z R V (V )/ and a component-specific one D (h) R V (V )/ characterized via matrix factorization representations. In particuar, we et [, π (h) = + exp{ Z D }] (h) D (h) = L(X (h) Λ (h) X (h)t ), h =,..., H, () with the ogistic mapping in () appied eement-wise. Equations () () carefuy incorporate custer dependence via custer-specific mixing probabiities ν k = (ν k,..., ν Hk ), k =,..., K as we as network information by considering a atent eigenmode for each mixture component. Focusing on the mixture component h, the atent eigenmode (Hoff, 00) defines the undirected edges as reaizations from conditionay independent Bernoui random variabes given their corresponding edge probabiities π (h) (0, ), =,..., V (V )/, and then borrows network information across these edge probabiities via ower dimensiona representations. In particuar, according to () and etting correspond to the pair of products v and u, v > u each π (h) is constructed as a function of the pairwise simiarity among products v and u in a atent space, with this simiarity arising from the dot product of the products atent coordinate vectors X (h) v = {X (h) v,..., X(h) vr R R, v =,..., V, with X (h)t }T v the vth row of X (h). Hence, products having coordinates in the same direction are more ikey to be co-subscribed than products with coordinates in opposite directions, with the R R matrix Λ (h) = diag(λ (h) ) = diag(λ (h),..., λ(h) R ) weighting the simiarity in each dimension r by a non-negative parameter λ (h) r. Note that, consistenty with our notation, the vector L(X (h) Λ (h) X (h)t ) in () is equa to (X (h)t Λ (h) X (h), X(h)T Λ (h) X (h),..., X(h)T Λ (h) X (h), X(h)T Λ (h) X (h),..., X(h)T Λ (h) X (h),..., X(h)T Λ (h) X (h) V The atent eigenmode provides an appeaing choice in reducing the dimensionaity from V (V )/ edge probabiities to V R atent coordinates and R weights typicay R V and has been shown to provide a more fexibe characterization of the connectivity patterns and network structures than stochastic bock modes (Nowicki and Snijders, 00), atent distance modes (Hoff et a., 00) and mixed membership stochastic bock modes (Airodi et a., 00). However, as discussed in Durante et a. (0), a singe atent eigenmode fais in fexiby characterizing the probabiistic generative mechanism underying a network-vaued random variabe. To improve fexibiity and maintain computationa tractabiity, equations () () mix together H atent eigenmodes, whie adding a common simiarity vector Z R V (V )/ shared among a the co-subscription networks and centering the different mixture components to improve computationa and custering performance. According to Durante and Dunson (0), this characterization guarantees fu fexibiity in approximating the custer-specific probabiity mass functions for the co-subscription networks. Our goa is to expoit representation () () in order to deveop shared cross-se strategies. Figure provides an exampe of the output from our mode for decision making in business inteigence when there are n = agencies and K = custers. According to Figure, agencies, and share common profies of mono-product customer choices and co-subscription networks as C = C = C =. In Figure, mono-product choices in custer are characterized by the vector p, whie the co-subscription behavior is summarized by the expectation of the networkvaued random variabe in custer, E{L(A )} = π = L(A) A V L(A)pr{L(A ) = L(A)}, with V V V )T. imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

7 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing Custer Custer Custer π π π p p p Agencies in custer Agencies in custer Agencies in custer Fig : Exampe of a possibe output from our mode for cross-seing. Upper panes: custer-specific edge probabiity vectors π, π and π rearranged in adjacency matrix form representing co-subscription profies in each custer. Coor goes from white to back as the probabiity goes from 0 to. Lower panes: custer-specific probabiity vectors p, p and p representing mono-product customer choices in each custer. Quantities in the upper panes are required to define the cross-se strategies in each custer. Quantities in the ower panes provide the additiona information to define the associated performance indicators. A V denoting the sampe space of a the possibe network configurations among V products. As discussed in Durante and Dunson (0), under representation (), π = H h= ν hπ (h), which coincides with the vector of co-subscription probabiities for pairs of products in custer. Hence, π can be used to define the set of cross-se strategies for agencies, and. The same description hods for custers and. These quantities are estimated from our mode and considered when defining the custerspecific cross-se strategies u k,..., u kv and computing their performance indicators e k,..., e kv, for each k =,..., K. Adapting the initia discussion in Section to the output of our mode, cross-se strategy u kv offers to mono-product customers who subscribed to v in agencies within custer k, the additiona product u kv = argmax u {pr(a k[vu] = ) : u v}. The probabiity of a co-subscription between products v and u for agencies in custer k, pr(a k[vu] = ), is easiy avaiabe from our mode as π k, with the index denoting the pair v and u in the vectorized representation of the adjacency matrix. The performance indicator associated to u kv is instead e kv = p kv max{pr(a k[vu] = ) : u v}. As motivated above, the main purpose of our anaysis is to custer agencies having customers with simiar mono-product and muti-product purchasing behavior, whie providing accurate estimates of the performance indicators for the different strategies. In order to deveop procedures imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

8 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing Z λ (h) π (h) X (h) h =,..., H ν k G i L(A i ) k =,..., K C i p k y i k =,..., K Fig : Graphica representation of the mechanism to generate {y i, L(A i )}, under representation () and (). for tractabe estimation and inference, it is usefu to rey on a hierarchica specification equivaent to () (), which introduces an additiona cass index for each agency i, G i {,..., H}, as foows: indep L(A i ) π i Bern(π i ), =,..., V (V )/, [ π i G i = h, π (h) = π (h), π (h) = + exp{ Z L(X (h) Λ (h) X )}] (h)t () pr(g i = h C i = k) = ν hk, h =,..., H, independenty for i =,..., n. Recaing that C i is the custer index for agency i, representation () shows that agencies in the same custer have a common set of probabiity weights over the components in the mixture mode for the co-subscription network. Moreover, athough agencies in different custers have different probabiities to be aocated in each mixture component, we are not forcing the cass indices to be necessariy different for agencies beonging to different custers. In fact, under (), two agencies i and j in different custers C i C j are aowed to share the same component G i = G j in the mixture mode for the co-subscription networks. This faciitates efficient borrowing of information across custers in modeing the mixture components characterizing the custer-dependent mixture of atent eigenmodes. Refer to Figure for a graphica representation of our hierarchica mode. The next section deveops a Bayesian approach for inference under the proposed mode. Athough mode () () is reminiscent of recent proposas from Durante et a. (0) and Durante and Dunson (0), the procedure deveoped in this contribution has key differences. Durante et a. (0) deveop a nonparametric mixture of atent eigenmodes to characterize the probabiity mass function of a network-vaued random variabe. Durante and Dunson (0) test imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

9 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing for changes in this probabiity mass function across the vaues of a categorica predictor. Our current focus is instead on using the fexibe and parsimonious joint mode presented in Figure to deveop shared cross-se strategies. In our business inteigence probem, the conditiona distribution of the co-subscription networks varies as a function of a atent custering variabe, which is endogenousy determined by mono-product choices and co-subscription patterns shared across subsets of agencies. These custer assignments are key unknown quantities, requiring carefu priors and adapted computationa methods.. Prior specification and posterior computation.. Prior specification The assignment vector C = (C,..., C n ) of the agencies and the tota number of custers K are key unknowns in our anaysis. There is a rich iterature on prior probabiity modes for C and K. A widey used approach is the Chinese Restaurant Process (CRP) (Adous, ), in which each custer attracts new units in proportion to its size. In particuar, etting C = (C,..., C n ) CRP(α c ), the prior distribution over custers for the ith agency, conditioned on the membership of the others C,..., C i, C i+,..., C n, is pr(c i = k C,..., C i, C i+,..., C n ) = { nk, i n +α c for k =,..., K i, α c n +α c for k = K i +, () where K i is the tota number of nonempty custers after removing C i, n k, i is the tota number of agencies aocated to custer k, excuding the ith one, and α c > 0 is a concentration parameter controing the expected number of occupied custers E(K) = O(α c og n). High vaues of α c favor more custers a priori, with E(K) growing as sampe size n increases. Each custer characterizes a specific joint profie of mono- and muti-product customer behaviors, with these profies potentiay arising from specific preferences of customers in different regions or economic areas where each agency operates. Differences in profies may be aso associated to particuar business administration choices of each agency. Hence, we expect a growing K as the number of agencies in the sampe increases, with the subinear reation between E(K) and n favoring parsimony in the number of cross-se strategies to be defined. Equation () defines the conditiona distribution of C i given the other assignments, after rearranging indices so that,..., K i custers are nonempty after removing C i. These operations are possibe since the custer abes are arbitrary and observations are exchangeabe based on the CRP prior; refer to Griffiths and Ghahramani (0) and Gershman and Bei (0) for an introductory overview. Athough we focus on the CRP, our mode can easiy accommodate other commony used priors for random partitions, such as the Pitman-Yor process or the Kingman paintbox among others. Refer to Hjort et a. (00) for a genera overview. Accurate custering of agencies aso reies on carefu modeing of the sequence of custerspecific mono-product choices defined in () and the coection of custer-specific probabiistic generative mechanisms associated to the co-subscription networks via () (). Efficient estimation of these quantities is fundamenta to deveop cross-se strategies u k,..., u kv in each custer k =,..., K and to quantify their performance via e k,..., e kv. Hence, we define arge support priors, which do not rue out a priori any generative mechanism whie maintaining tractabe computations. As p k represents the probabiity mass function for the categorica random variabe Y k, we et p k = (p k,..., p kv ) Dirichet(α,..., α V ), k =,..., K. () imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

10 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing 0 The prior for the probabiistic generative mechanism of the co-subscription networks in each custer is defined by choosing independent priors for the quantities in () (). To maintain computationa tractabiity, we consider independent Gaussian priors for the common simiarities Z N(µ, σ ), =,..., V (V )/ and standard Gaussians for the atent coordinates X (h) vr N(0, ), for each v =,..., V, r =,..., R and h =,..., H. Adapting Rousseau and Mengersen (0), we choose independent Dirichet priors with sma hyperparameters for each custer-specific mixing probabiity vector ν k = (ν k,..., ν Hk ) Dirichet(/H,..., /H), k =,..., K to favor deetion of redundant mixture components not required to characterize the co-subscription networks. Finay, as the dimension of the atent spaces in the eigenmodes is unknown, we consider a shrinkage prior on the weights vector λ (h), h =,..., H which adaptivey deetes redundant atent space dimensions not required to characterize the co-subscription probabiities. This is accompished by choosing mutipicative inverse gamma MIG(a, a ) priors (Bhattacharya and Dunson, 0) for vectors λ (h), h =,..., H, which favor shrinkage effects by inducing priors on eements λ (h) r around 0 as r increases, for appropriate a. Hence λ (h) r = r m= ϑ (h) m for r =,..., R, that are increasingy concentrated, ϑ (h) Ga(a, ), ϑ (h) m> Ga(a, ), r =,..., R, h =,..., H. () Beside providing simpe agorithms for posterior inference, adapting resuts in Durante and Dunson (0), the above specification induces a prior on the probabiistic generative mechanism for the co-subscription networks in each custer having fu support. Fu prior support is a key property to ensure good performance in defining accurate cross-se strategies, because without prior support about the true data generating process, the posterior cannot possiby concentrate around the truth... Posterior computation Posterior computation is avaiabe via a simpe Gibbs samper outined in Agorithms and, which expoits resuts in Nea (000) to aocate agencies to custers under the CRP prior and steps in Durante et a. (0) to update the quantities in () (). Durante et a. (0) join the standard data augmentation procedure for inference in mixture modes outined in (), with a recent Póya-gamma data augmentation for conjugate inference in Bayesian ogistic regression (Poson et a., 0). Agorithm Part I of the Gibbs samper for joint modeing of mixed domain data Conditionay on (C,..., C n ) update priors for quantities in equations (), () and () as foows [] Update the custer-specific mono-product choice probabiities in p k, k =,..., K for k =,..., K do Update p k from (p k,..., p kv ) Dirichet ( α + i:c n i=k i,..., α V + i:c n ) i=k iv end for [] Aocate each co-subscription network to one of the mixture components for i =,..., n do imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

11 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing Sampe the cass indicator G i from the discrete distribution with probabiities V (V )/ ν hci pr(g i = h ) = H q= ν V (V )/ qc i = {π (h) = {π (q) } L(Ai) { π (h) } L(Ai) } L(Ai) { π (q) } L(Ai) for each h =,..., H, with π (h) defined in (). end for [] Update the custer-specific vectors of mixing probabiities ν k, k =,..., K for k =,..., K do Update each ν k from (ν k,..., ν Hk ) Dirichet(/H + n k,..., /H + n Hk ) with n hk denoting the number of agencies in custer k, currenty associated to mixture component h. end for [] Update quantities Z, X (h) and λ (h), h =,..., H via Poyá-gamma data augmentation expoiting derivations in Durante et a. (0) as foows for h =,..., H do [.] For each =,..., V (V )/ update augmented data ω (h) from the Póya-gamma } PG {n h, Z + L(X (h) Λ (h) X (h)t ), ω (h) with PG(b, c) denoting the Póya-gamma distribution with parameters b > 0 and c R, and n h the tota number of agencies in mixture component h. [.] Update the shared simiarity vector Z from Z N V (V )/ {µ Z, diag(σ Z,..., σ Z V (V )/ )}, with µ Z having eements µ Z = σz [σ µ + H h= {L(A(h) ) n h / ω (h) L(X (h) Λ (h) X (h)t ) }], where σz = /(σ + H h= ω(h) ), for each =,..., V (V )/ and L(A (h) ) = i:g L(A i=h i). [.] As D (h) = L(X (h) Λ (h) X (h)t ) = L( X (h) X(h)T ), with X (h) = X (h) Λ (h)/, update eements in X (h), knowing that, under our prior specification (h) X vr λ r (h), N(0, λ (h) r ). Therefore for v =,..., V do Bock-sampe the vth row of X (h) (h) (h) (h) (h). Define X v = ( X v,..., X vr )T and et X ( v) denote the (V ) R matrix obtained by removing the vth row in X (h). Consider the ogistic regression L(A (h) ) (v) Binom(n h, π (h) (v) ), ogit(π(h) (v) ) = Z (v) + (h) (h) X ( v) X v, with L(A (h) ) (v) and Z (v) obtained by stacking eements L(A (h) ) and Z, respectivey, for a corresponding to pairs having v as a one of the two nodes, and ordered consistenty with the inear predictor. Expoiting previous ogistic regression, and etting Ω (h) (v) be the diagona matrix with the corresponding Póya-gamma augmented data, the fu conditiona is X (h) v { ( N R X(h)T ( v) Ω(h) (h) (v) X ( v) + Λ(h) ) ( } η (h) v, X(h)T ( v) Ω(h) (h) (v) X ( v) + Λ(h) ), imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

12 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing with η v (h) (h)t = X ( v) {L(A(h) ) (v) V n h / Ω (h) (v) Z (v)} end for [.] Sampe the gamma quantities defining the shrinkage weights λ (h),..., λ(h) R where θ ( r) m ϑ (h) Ga { ϑ (h) r> Ga { a + a + V R, + = m t=,t r ϑ(h) t for r =,..., R. R m= θ ( ) m V (R r + ), + V ( v= R m=r X (h) vm) }, θ ( r) m V ( v= X (h) vm) }, end for [] Update the edge probabiity vectors π (h), in each mixture component h =,..., H for h =,..., H do Compute π (h) as π (h) = [ + exp{ Z L( X (h) X(h)T )} ] end for Agorithm provides detaied steps to update the custer-specific probabiistic representation of the mono-product customer choices and the custer-specific probabiistic generative mechanism underying co-subscription networks, given the custer assignments C,..., C n. In performing these steps, the number of mixture components H and the dimension of the atent spaces R are set at conservative upper bounds, aowing the shrinkage priors for these quantities to adaptivey empty redundant components that are not required to characterize the observed are not concentrated around 0 for any h, this suggests that H or R shoud be increased, respectivey. A the previous steps are straightforward to compute, expoiting the data augmentation strategy described in () for updating quantities in () (). The atter provides key computationa benefits aso when samping from the fu conditiona of the custer assignments described in Agorithm. data. If a the mixture components are occupied or the posteriors for some weights λ (h) r Agorithm Part II of the Gibbs samper for joint modeing of mixed domain data Conditionay on sampes for the quantities in (), () and (), update the custer indices in C. [] Sampe the custer assignments (C,..., C n ) via a sequentia re-seating procedure for i =,..., n do Update C i conditionay on C i = (C,..., C i, C i+,..., C n ) [.] Remove agency i since we are going to sampe its custer membership C i. [.] If no other agencies are in the same custer of i, this custer becomes empty and is removed aong with its associated mono-product choice probabiities and co-subscription network probabiity mass function. [.] Re-order custer indices so that,..., K i are nonempty. imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

13 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing [.] Update the custer of i from the fu conditiona categorica variabe with custer probabiities { nk, i n +α pr(c i = k ) c pr { } L(A i ), y i, G i C i = k, π (Gi), p k, ν k for k =,..., K i, α c n +α c pr { L(A i ), y i, G i C i = K i +, π (Gi)} for k = K i +, () [.] If i is assigned a new custer K i +, add a new custer and sampe a new p K i+ and ν K i+ conditionay on y i and G i according to steps [] and [], respectivey. end for To perform steps in Agorithm one needs to compute the conditiona probabiities in equation () at each MCMC iteration. Athough this is apparenty a cumbersome task, our mode formuation () () aong with its hierarchica representation in () aows key simpifications, substantiay improving computationa tractabiity. Specificay, under our mode, the conditiona probabiity pr { } L(A i ), y i, G i C i = k, π (Gi), p k, ν k for augmented data incuding Gi can be factorized as V pr{l(a i) G i, π (G i) }pr(y i C i = k, p k )pr(g i C i = k, ν k ) = pr{l(a i) G i, π (G i) } v= p n iv kv H h= ν (h) (G i) hk () with (h) (G i ) = if G i = h and (h) (G i ) = 0, otherwise. According to () inducing custerdependence through the mixing probabiities ν k, whie considering custer-independent mixture components in () (), has the key benefit of maintaining pr{l(a i ) G i, π (Gi) } constant across the custer assignments. Recaing formuation (), conditionay on G i = h, the probabiity of observing L(A i ) is pr{l(a i ) G i = h, π (h) } = V (V )/ = { π (h) } L(Ai) { π (h) } L(Ai), and does not depend on the custer assignment C i. As a resut pr{l(a i ) G i, π (Gi) } acts as a mutipicative constant in (), aowing () to be further simpified as { nk, i V H pr(c i = k ) n +α c v= pniv kv h= ν (h)(g i) hk for k =,..., K i, α c n +α c pr(y i C i = K i + )pr(g i C i = K i + ) for k = K i +, (0) where the two margina probabiities corresponding to a newy occupied custer are easiy avaiabe expoiting the mutinomia-dirichet conjugacy. In particuar, it is easy to show that V pr(y i C i = K i + ) = p niv K dπ(p i+,v K i+) = Γ( V v= α V v) v= Γ(α v + n iv ) V v= v= Γ(α v) Γ{ V v= (α v + n iv )}, () for the mono-product choices, and H pr(g i C i = K i + ) = ν (h)(g i) h,k dπ(ν i+ K i+) = Γ( H h= /H) H h= Γ{/H + (h)(g i )} H h= Γ(/H) Γ[ H h= {/H + (h)(g i )}], () h= for the cass indicator variabe in the custer-dependent mixture of atent eigenmodes. Hence, considering ony custer dependence in the mixing probabiity vectors ν k, k =,..., K and expoiting the augmented data G i, i =,..., n, in the mixture representation for the generative mechanism underying co-subscription networks, aows a massive gain in computationa tractabiity for step [] in Agorithm. Whie () and () can be easiy derived in cosed form, the margina probabiity of the co-subscription networks with respect to the edge probabiity vectors arising from the construction in () is not anayticay avaiabe. imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

14 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing. Simuation studies We consider a simuation study to evauate the performance of our method in accuratey recovering custers of agencies and in efficienty estimating the key quantities required to define the set of cross-se strategies for each custer and their associated performance indicators. In simuating data, we define a scenario mimicking the structure of our appication. We focus on n = 00 agencies equay divided in K = atent custers and consider a tota number of V = products. Graphica anayses of our data highighted in Figure show that mono-product customers typicay concentrate on a sma subset of the avaiabe products with high probabiity, whie choosing the remaining set with very ow frequency. We maintain this behavior in constructing p 0 k, k =,...,, whie defining a chaenging scenario with sma changes in p 0 k across custers. In particuar, we set p0 v and p 0 v to be equa for a products v except for permuting and, so pr(y 0 = ) = pr(y 0 = ) and pr(y 0 = ) = pr(y 0 = ). We adopt a simiar strategy for custers and by considering p 0 v and p 0 v equa for a products v except and, where we et pr(y 0 = ) = pr(y 0 = ) and pr(y 0 = ) = pr(y 0 = ). We simuate mono-product subscription data y is, i =,..., 00 and s =,..., 00, from the categorica random variabe Yk 0 with probabiity mass function p0 k, where k = for agencies i =,..., 0, k = for i =,..., 00, k = for i = 0,..., 0 and k = for i =,..., 00. Athough agencies in our appication have at east,000 mono-product customers, we consider a smaer number n i = 00 for each agency i =,..., 00 to evauate the performance when there is ess data. Co-subscription networks are simuated expoiting our constructive representation of the mixture mode in (). We consider H = mixture components with each edge probabiity vector π 0(h) generated to mimic a possibe co-subscription scenario. Specificay, π 0() is characterized by one dense community among 0 possiby highy reated products, whie assigning ow probabiity to the remaining pairs of products. Vector π 0() represents the case of hub products, which occur with high probabiity in muti-product customer choices, whie fixing the remaining co-subscription probabiities at ow vaues. To reduce separation among mixture components and provide a more chaenging scenario, the edge probabiity vector π 0() is very simiar to π 0() with exception of product v = which is hed out from the hub products. In avoiding the eigenmode construction () in the definition of π 0(h), h =,...,, we additionay aim to evauate the performance of representation () in accuratey characterizing the co-subscription probabiities for each component h. In simuating vectors L(A i ), i =,..., 00, we consider custer-specific mixing probabiity vectors ν 0 = ν 0 = (0., 0.0, 0.0), ν 0 = (0.0, 0., 0.0) and ν 0 = (0.0, 0.0, 0.). This choice aows the first co-subscription scenario defined by π 0() to be very ikey in agencies beonging to custers and. Scenarios characterized by π 0() and π 0() are instead more ikey in custers and, respectivey. Letting ν 0 = ν 0 further reduces separation among custers and. These two custers have very simiar mono-product choice probabiities and equa generative processes for the co-subscription networks, providing a chaenging scenario to evauate custering performance. We anayze the simuated data under our mode () (). As in Durante et a. (0), we set a =., a =. and σ = 0, =,..., V (V )/. Quantities µ, =,..., V (V )/ are defined as µ = ogit{ n i= L(A i) /n} in order to center the mixture representation for the co-subscription networks around a structure shared by a the agencies in the company. We adopt a simiar strategy for the hyperparameters of the Dirichet prior () by setting α v = n i= n iv/n for each v =,..., V, in order to center () around the averaged preferences of mono-product customers in the entire company. This empirica choice improves performance reative to symmetric Dirichet priors, which can be sensitive to the concentration parameter, imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

15 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing π π π π p p p p Fig : For the four nonempty custers. Upper panes: posterior mean ˆ π k of the co-subscription probabiities among pairs of products in each custer k =,..., (ower trianguar) and true π k 0, k =,..., (upper trianguar). Coor goes from white to back as the probabiity goes from 0 to. Lower panes: posterior mean ˆp k of the mono-product choices in each custer k =,..., (dark gray) and true p 0 k, k =,..., (ight gray). overy penaizing the addition of new custers when concentration is sma. Finay, we set the concentration parameter α c = in the CRP prior for the custer assignments, according to standard practice. Our approach can be easiy generaized to earn α c from the data as in Escobar and West (). However, we obtain the same resuts in term of custering, cross-se strategies and performance indicators when using α c = 0., α c =, α c = 0, α c = and α c = 0. We coect,000 Gibbs iterations and set H = and R = 0 as upper bounds for the number of mixture components and the dimension of the atent spaces, respectivey. These upper bounds provide a good choice, with the sparse Dirichet prior for the mixing probabiities in ν k, k =,..., K, and the mutipicative inverse gamma for the weights in λ (h), h =,..., H, adaptivey removing redundant components. Trace pots suggest convergence is reached after a burn-in of,000 and mixing for the quantities of interest for inference is good. As inference focuses on custer-specific structures, it is important to first check for abe switching issues, and reabe the custers at each MCMC iteration using, for exampe, Stephens (000) as needed. Trace pots suggest abe switching is not an issue in our simuation. We initiaize our MCMC agorithm by assigning a agencies to a singe custer. Then, using sampes after burn-in, we estimate Ĉ as corresponding to the partition of agencies observed in the maximum proportion of draws, providing an estimate of the MAP. We found this simpe approach to have good performance and stabiity in our simuations, but as the number of agencies increase using more refined procedures, such as Medvedovic et a. (00) and Lau and Green (00), may be preferabe. We estimate ˆK =, correcty grouping a the simuated agencies, incuding those in custers and, which are characterized by very subte differences in their generating process. Accurate custering further aows for efficient estimation of the custer-specific components required for cross-seing. According to Figure, we correcty estimate the probabiity mass imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0

16 Durante, Paganin, Scarpa and Dunson/Modeing of compex network data in marketing Custers 0 Products Fig : Summary of the estimated custer-specific cross-se strategies û k,..., û kv aong with their performance indicators ê k,..., ê kv. Each ce [k, v] of the matrix defines the cross-se strategy for mono-product customers subscribed to product v in agencies beonging to custer k, aong with its performance. The number in ce [k, v] corresponds to the best offer û kv = argmax u { ˆpr(A k[vu] = ) : u v}. The dimension of each square is proportiona to max{ ˆpr(A k[vu] = ) : u v}, whie the gray scae of the coor is proportiona to the corresponding estimated performance ê kv = ˆp kv max{ ˆpr(A k[vu] = ) : u v}. The estimated probabiity of a co-subscription for each pair of products v and u in agencies within custer k, ˆpr(A k[vu] = ), is easiy avaiabe from the posterior mean ˆ π k of π k for each k =,..., K and =,..., V (V )/. functions p 0 k characterizing the mono-product choices in each custer, with a simiar performance in recovering the vectors of co-subscription probabiities π k 0. These estimates are a key to define the custer-specific cross-se strategies u k,..., u kv and reated performance indicators e k,..., e kv, as shown in Figure. Consistenty with resuts in Figure, cross-se strategies are the same in custers and as ˆ π ˆ π, whie performance indicators differ ony for strategies targeting mono-product customers subscribed to v = or v =. The first segment is more profitabe in custer whie the second is in custer. This is consistent with our estimates in Figure highighting ˆpr(Y = ) > ˆpr(Y = ) and ˆpr(Y = ) > ˆpr(Y = ). Mono-product customers subscribed to v = and v = are highy profitabe in custer in being highy represented and having high co-subscription probabiity with at east one further product. Athough v = is highy popuated in custer, according to ˆ π it is not possibe to find another product u having high co-subscription probabiity with v =, so such customers are not profitabe in terms of targeting of cross-se campaigns. Conditionay on Ĉi = k, agency i is associated with the cross-se strategies suggested by ˆ π k. Hence, in ensuring accurate targeting, it is important to assess to what extent the estimated custer-specific co-subscription probabiities adequatey characterize the observed networks. Figure dispays the box pot of the areas under the ROC curve (AUC), with each AUC i, i =,..., n obtained using the simuated L(A i ) and the estimated co-subscription probabiity vector ˆ π k of its corresponding custer. Most of the co-subscription networks are adequatey characterized by the co-subscription probabiity vectors specific to their custers, with amost a the AUCs greater than 0.. Decreased predictive performance is obtained for a sma set of agencies highighted by dots. For such agencies, the company may devise ad-hoc cross-se strategies based on the posterior mean of their agency-specific edge probabiity vector π i rather than considering the cross-seing suggested to the custers they beong to. To evauate the fit with respect to the mono-product data, we consider the standardized L distance between observed and estimated product frequencies ε i = V v= n iv/n i ˆp kv /V with k the custer to which agency i is aocated. In our simuation the maximum of these quantities is max(ε,..., ε n ) = 0.0 meaning that mono-product choices in each custer k are adequatey imsart-generic ver. 0/0/0 fie: ARXIV_dd.tex date: Juy, 0