Parameter identification in Markov chain choice models

Transcription

1 Poceedings of Machine Leaning Reseach 76:1 11, 2017 Algoithmic Leaning Theoy 2017 Paamete identification in Makov chain choice models Aushi Gupta Daniel Hsu Compute Science Depatment Columbia Univesity New Yok, NY 10027, USA Editos: Steve Hanneke and Lev Reyzin Abstact This wok studies the paamete identification poblem fo the Makov chain choice model of Blanchet, Gallego, and Goyal used in assotment planning. In this model, the poduct selected by a custome is detemined by a Makov chain ove the poducts, whee the poducts in the offeed assotment ae absobing states. The undelying paametes of the model wee peviously shown to be identifiable fom the choice pobabilities fo the all-poducts assotment, togethe with choice pobabilities fo assotments of all-but-one poducts. Obtaining and estimating choice pobabilities fo such lage assotments is not desiable in many settings. The main esult of this wok is that the paametes may be identified fom assotments of sizes two and thee, egadless of the total numbe of poducts. The esult is obtained via a simple and efficient paamete ecovey algoithm. Keywods: discete choice modeling, paamete identification, Makov chains 1. Intoduction In assotment planning, the selle s goal is to select a subset of poducts (called an assotment to offe to a custome so as to maximize the expected evenue. This task can be fomulated as an optimization poblem given the evenue geneated fom selling each poduct, along with a pobabilistic model of the custome s pefeences fo the poducts. Such a discete choice model must captue the custome s substitution behavio when, fo instance, the offeed assotment does not contain the custome s most pefeed poduct. Ou focus in this pape is the Makov chain choice model (MCCM poposed by Blanchet et al. (2016. In this model, the poduct selected by the custome is detemined by a Makov chain ove poducts whee the poducts in the offeed assotment ae absobing states. The cuent state epesents the desied poduct; if that poduct is not offeed, the custome tansitions to anothe poduct accoding to the Makov chain pobabilities, and the pocess continues until the desied poduct is offeed o the custome leaves. MCCM genealizes widely-used discete choice models such as the multinomial logit model (Luce, 1959; Plackett, 1975, as well as othe genealized attaction models (Gallego et al., 2014; it also wellappoximates othe andom utility models found in the liteatue such as mixed multinomial logit models (McFadden and Tain, At the same time, the MCCM pemits computationally efficient unconstained assotment optimization as well as efficient appoximation algoithms in the constained case (Blanchet et al., 2016; Dési et al., 2015; this stands in c 2017 A. Gupta & D. Hsu.

2 contast to some iche models such as mixed multinomial logit models (Rusmevichientong et al., 2010 and the nested logit model (Davis et al., 2014 fo which assotment optimization is geneally intactable. This combination of expessiveness and computational tactability makes MCCM vey attactive fo use in assotment planning. A cucial step in this oveall entepise e.g., befoe assotment optimization may take place is the estimation of the choice model s paametes fom obsevational data. Paamete estimation fo MCCM is only biefly consideed in the oiginal wok of Blanchet et al. (2016. In that wok, it is shown that the paametes can be detemined fom the choice pobabilities fo the all-poducts assotment, togethe with the assotments compised of all-but-one poduct. This is not satisfactoy because it may be unealistic o unpofitable to offe assotments of such lage cadinality. Theefoe, it is desiable to be able to detemine the paametes fom choice pobabilities fo smalle cadinality assotments. We note that this is indeed possible fo simple choice models such as the multinomial logit model (see, e.g., Tain, 2009, but these simple models ae limited in expessiveness fo example, they cannot expess heteogeneous substitution behavio. In this pape, we show that the MCCM paametes can be identified fom the choice pobabilities fo assotments of sizes as small as two and thee, independent of the total numbe of poducts. 1 We also give a simple and efficient algoithm fo econstucting the paametes fom these choice pobabilities. 2. Model and notation In this section, we descibe the Makov chain choice model (MCCM of Blanchet et al. (2016, along with notations used fo choice pobabilities and model paametes. The set of n poducts in the system is denoted by N := {1, 2,..., n}. The no puchase option is denoted by poduct 0. Upon offeing an assotment S N, the set of possible outcomes is S + := S {0}: eithe some poduct in S is puchased, o no poduct is puchased. Undelying the MCCM is a Makov chain with state space N +. The (tue paametes of the model ae the initial state pobabilities λ = (λ i i N+ and the tansition pobabilities ρ = (ρ i,j (i,j N+ N + (a ow stochastic matix. The tansition pobabilities satisfy the following popeties: 1. ρ 0,0 = 1 and ρ 0,j = 0 fo j N (i.e., the no puchase state is absobing. 2. ρ i,i = 0 fo i N (i.e., no self-loops in poduct states. 3. The submatix ρ := (ρ i,j (i,j N N is ieducible. We use ρ i = (ρ i,j j N+ to denote the i-th ow of ρ. In MCCM, the custome aives at a andom initial state X 1 chosen accoding to λ. At time t = 1, 2,... : If X t = 0, the custome leaves the system without puchasing a poduct. 1. We focus on identifiability because estimation of choice pobabilities fom obsevational data is faily staightfowad, especially when the assotments have small cadinality. Howeve, this issue is evisited in Section 5 in the context of sample complexity. 2

3 If the poduct X t is offeed (i.e., X t S, the custome puchases X t and leaves. If the poduct X t is not offeed (i.e., X t / S, the custome tansitions to a new andom state X t+1 chosen accoding to ρ Xt and the pocess continues in time step t + 1 as if the custome had initially aived at X t+1. Anothe way to descibe this pocess is that the Makov chain distibution is tempoaily modified so that the states S + ae absobing, and the custome puchases the poduct upon eaching such a state (o makes no puchase if the state is 0. The ieducibility of ρ ensues that the custome eventually leaves the system (i.e., an absobing state is eached. Note that only the identity of the final (absobing state is obseved, as it coesponds to eithe a puchase o non-puchase. The (X t t=1,2,..., themselves do not coespond to obsevable custome behavio, and hence the model paametes λ and ρ cannot be diectly estimated. The choice pobabilities ae denoted by π(j, S fo S N and j S + : this is the pobability that j is the final state in the afoementioned pocess. Blanchet et al. (2016 elate the choice pobabilities and the paametes λ and ρ as follows: 1 if i = 0 and j = 0, π(j, N \ {i} π(j, N λ j = π(j, N, ρ i,j = if i N, j N +, and i j, (1 π(i, N 0 othewise. The elations in Equation (1 show that the paametes may be identified fom choice pobabilities fo the assotments S = N and S = N \ {i} fo i N. These choice pobabilities may be diectly estimated fom obsevations upon offeing such assotments to customes. 3. Main esult The following theoem establishes identifiability of the MCCM paametes fom choice pobabilities fo assotments of sizes as small as two and thee. Theoem 1 Thee is an efficient algoithm that, fo any {2, 3,..., n 1}, when given as input the choice pobabilities (π(j, S j S+ fo all assotments S N of cadinality and + 1 fo a Makov chain choice model, etuns the paametes λ and ρ of the model. The numbe of assotments fo which the algoithm actually equies choice pobabilities is O(n 2 when n/2, which is fa fewe than ( ( n + n +1, the total numbes of assotments of sizes and + 1. The details of this bound ae shown following the poof of Theoem 1. Howeve, to simplify the pesentation, we descibe ou paamete ecovey algoithm as using choice pobabilities fo all assotments of sizes and + 1. The main steps of ou algoithm, shown as Algoithm 1, involve setting up and then solving systems of linea equations that (as we will pove detemine the unknown paametes λ and (ρ i i N. (Note that ρ 0 is aleady known. The coefficients of the linea equations ae detemined by the given choice pobabilities via conditional choice pobabilities π(j, S i fo S N and i, j N +, defined as follows: π(j, S i := P ( state j is eached befoe any state in S + \ {j} initial state is i. (2 3

4 Algoithm 1 Paamete ecovey algoithm fo Makov chain choice model input Fo some {2, 3,..., n 1}, choice pobabilities (π(j, S j S+ fo all assotments S N of sizes and + 1. output Paametes ˆλ and ˆρ. 1: fo i N do 2: Solve the following system of linea equations fo ˆρ i = (ˆρ i,k k N+ : k N + π(j, S k ˆρ i,k = π(j, S i fo all S ( N s.t. i / S and j S +, (3 whee ( N denotes the family of subsets of N of size, and π(j, S k is defined in Equation (5. 3: end fo 4: Solve the following system of linea equations fo ˆλ = (ˆλ i i N+ : 5: etun ˆλ and ˆρ. k N + π(j, S k ˆλ k = π(j, S fo all S ( N and j S + (4 Note that the initial state in the MCCM is not obseved, so these conditional pobabilities cannot be diectly estimated. Nevetheless, they can be indiectly estimated via the following elationship between the conditional choice pobabilities and the (unconditional choice pobabilities. Lemma 2 Fo any S N and i, j S +, 1 if i = j, π(j, S π(j, S {i} π(j, S i = if i N \ S, π(i, S {i} 0 if i S + \ {j}. (5 Poof The cases whee i = j ( π(j, S i = 1 and i S + \ {j} ( π(j, S i = 0 ae clea fom the definition in Equation (2. It emains to handle the case whee i N \ S. 4

5 Fix such a poduct i, and obseve that π(j, S = P ( j is eached befoe S + \ {j} = P ( j is eached befoe S + \ {j} i is not eached befoe S + + P ( j is eached befoe S + \ {j} i is eached befoe S + = P ( j is eached befoe (S + {i} \ {j} + P ( j is eached befoe S + \ {j} i is eached befoe S + P (i is eached befoe S + = P ( j is eached befoe (S {i} + \ {j} + P ( j is eached befoe S + \ {j} initial state is i P ( i is eached befoe (S {i} + \ {i} = π(j, S {i} + π(j, S i π(i, S. The penultimate step uses the Makov popety and the case condition that i N \ S. Reaanging the equation gives the elation claimed by the lemma in this case. Lemma 2 shows that the conditional choice pobabilities fo assotments S of size can be detemined fom the unconditional choice pobabilities of assotments of size and + 1. The systems of linea equations used in Algoithm 1 (Equations (3 and (4 ae defined in tems of these conditional choice pobabilities and hence ae ultimately defined in tems of the unconditional choice pobabilities povided as input to Algoithm 1. It is clea that the tue MCCM paametes λ and ρ satisfy the systems of linea equations in Equations (3 and (4. Howeve, what needs to be poved is that they ae uniquely detemined by these linea equations; this is the main content of the poof of Theoem Poof of Theoem 1 In this section, we give the poof of Theoem The case without the no puchase option Fo sake of claity, we fist give the poof in the case whee the no puchase option is absent. This can be egaded as the special case whee λ 0 = 0 and ρ i,0 = 0 fo all i N. So hee we just egad λ = (λ j j N and each ρ i = (ρ i,j j N as pobability distibutions on N. The geneal case will easily follow fom the same aguments with mino modification Poof stategy We make use of the following esult about M-matices, i.e., the class of matices A that can be expessed as A = si B fo some s > 0 and non-negative matix B with spectal adius at most s. (Hee, I denotes the identity matix of appopiate dimensions. In paticula, the matix I ρ is a (singula M-matix that is also ieducible. 5

6 Lemma 3 (See, e.g., Theoems & in Beman and Plemmons, 1994 If A R p p is an ieducible M-matix (possibly singula, then evey pincipal submatix 2 of A, othe than A itself, is non-singula. If A is also singula, then it has ank p 1. Fo each S ( N and j S, define the vecto h j,s := (π(j, S k k N. Fo each i N, the collection of the vectos {h j,s : S i j S} povide the left-hand side coefficients in Equation (3 fo ˆρ i. We ll show that the span of these vectos (in fact, a paticula subset of them has dimension at least n 1. This is sufficient to conclude that ρ i is the unique solution to the system of equations in Equation (3 because it has at most n 1 unknown vaiables, and it is clea that ρ i satisfies the system of equations. (In fact, thee ae eally only n 2 unknown vaiables, because we can foce ˆρ i,i = 0 and ˆρ i,n = 1 n 1 k=1 ˆρ i,k. Fo the same eason, it is also sufficient to conclude that λ is the unique solution to the system of equations in Equation (4 (whee, in fact, we may use all vectos {h j,s : S ( N j S} Rank of linea equations fom a single assotment We begin by chaacteizing the space spanned by {h j,s : j S} fo a fixed S ( N. We claim, by Lemma 2, that the vectos in {h j,s : j S} ae linealy independent. Indeed, if this collection of vectos is aanged in a matix [h j,s : j S], then the submatix obtained by selecting ows coesponding to j S is the S S identity matix. Thus we have poved Lemma 4 Fo any S ( N, dim ( span{hj,s : j S} = S =. Note that in the case = n 1, we ae done. But when < n 1, the linea equations given by the {h j,s : j S} may not uniquely detemine the ρ i fo i N \ S. To ovecome this, we need to be able to combine linea equations deived fom multiple assotments. Howeve, fo a sum of subspaces V and W, dim(v + W dim(v + dim(w unless V and W ae othogonal. In ou case, the subspaces span{h j,s : j S} and span{h j,s : j S } fo diffeent assotments S and S ae not necessaily othogonal (even if S and S ae disjoint. So a diffeent agument is needed Rank of linea equations fom multiple assotments Ou aim is to show that the intesection of subspaces V := span{h j,s : j S} span{h j,s : j S } fo diffeent assotments S and S cannot have high dimension. We do this by showing that the intesection is othogonal to a subspace of high dimension. 2. Recall that a pincipal submatix of a p p matix A is a submatix obtained by emoving fom A the ows and columns indexed by some set I [p]. 6

7 Fo each i N, let a i denote the i-th ow of the matix A := I ρ (which is an M-matix. That is, a i := e i ρ i, whee e i is the i-th coodinate basis vecto. Recall that if i N \ S, then ρ i satisfies Equation (3. This fact can be witten in ou new notation as In othe wods, Lemma 5 h j,se i h j,sρ i = a i h j,s = 0, j S. Fo any S ( N, span{hj,s : j S} span{a i : i N \ S}. Now conside two assotments S and S, and the intesection of thei espective subspaces. It follows fom Lemma 5 that span{h j,s : j S} span{h j,s : j S } span{a i : i N \ (S S }. This othogonality is the key to lowe-bounding the dimension of the sum of these subspaces, which we captue in the following geneal lemma. Lemma 6 Let S be a family of subsets of N, S be a subset of N, and S := S {S }. Define the subspaces V S := span{h j,s : S S, j S}, V S := span{h j,s : j S }, V S := V S + V S. Then { dim (V S dim (V S + S max 1, ( } S S S S. Poof Let S 0 := S S S. Fix any v V S V S. Then, by Lemma 5, a i v = 0 fo all i (N \ S 0 (N \ S = N \ (S 0 S. In othe wods, V S V S W, whee W := span{a i : i N \ (S 0 S }, and dim (V S V S dim(w = n dim(w. To detemine dim(w, obseve that W is the span of ows of cetain ows of the M-matix A. By Lemma 3, the pincipal submatix of A coesponding to N \ (S 0 S is eithe nonsingula (when S 0 S o is A itself; in eithe case, it has ank n max{1, S 0 S }. Hence, dim(w = n max{1, S 0 S } as well. Combining the dimension fomula with the last two equation displays gives dim(v S = dim(v S + V S The claim now follows fom Lemma 4. = dim(v S + dim(v S dim(v S V S dim(v S + dim(v S max{1, S 0 S }. 7

8 Choice of assotments We now choose a collection of assotments and ague, via Lemma 4 and Lemma 6, that they define linea equations of sufficiently high ank. Specifically, fo each i N, we need a collection S ( N such that each S S does not contain i, and dim span {h j,s : j S} n 1. (6 S S Lemma 7 Suppose the assotments S 1, S 2,..., S T ( N have a paiwise common intesection S = S t S t fo all t t, and S = 1. Then dim (span T t=1 {h j,s t : j S t } T + 1. Poof Let d τ := dim (span τ t=1 {h j,s t : j S t } fo τ {1, 2,..., T }. By Lemma 4, we know that d 1 =. Now, assume d τ τ + 1, and use the fact 2 and Lemma 6 to conclude that d τ+1 d τ + ( 1 = d τ + 1 τ +. The claim now follows by induction. Fix any i N and S ( N \{i} 1, and obseve that N \ (S {i} = n. Conside the collection of size- assotments given by S := { S {k} : k N \ (S {i} }. (7 These assotments do not contain i, they have the common intesection S, with S = 1, and thee ae n assotments in total. So by Lemma 7, the collection S satisfies the dimension bound in Equation (6. As was aleady agued in the poof stategy, this suffices to establish the uniqueness of the ρ i and λ as solutions to the espective systems of linea equations in Equation (3 and Equation (4. This concludes the poof of Theoem 1 without the no puchase option The geneal case with the no puchase option We now conside the geneal case, whee the no puchase option is pesent. The main diffeence elative to the pevious subsection is that ρ is no longe ieducible, as the no puchase state 0 is absobing. Howeve, the submatix ρ = (ρ i,j (i,j N N is ieducible, so I ρ is an ieducible M-matix. The definition of h j,s, fo S N and j S +, is now taken to be h j,s := (π(j, S k k N+. Because the indexing stats at 0, we still define a i to be the i-th ow of A = I ρ, so a i = e i ρ i. (In paticula, a 0 is the all-zeos vecto. With these definitions, we have the following analogue of Lemma 4 and Lemma 5: Lemma 8 Fo any S ( N, dim(span{h j,s : j S + } = S + = + 1, span{h j,s : j S + } span{a i : i N \ S}. 8

9 Hee, the key diffeence is that the dimension is + 1, athe than just. We now establish an analogue of Lemma 6 (which is typogaphically nealy identical. Lemma 9 Let S be a family of subsets of N, S be a subset of N, and S := S {S }. Define the subspaces V S := span{h j,s : S S, j S + }, V S := span{h j,s : j S +}, V S := V S + V S. Then { dim (V S dim (V S + S max 1, ( } S S S S. Poof The poof is nealy the same as that of Lemma 6. Define S 0 := S S S and take v V S V S. By Lemma 8, V S V S W, whee W := span{a i : i N \ (S 0 S }, and dim(v S V S dim(w = n + 1 dim(w. We now use the fact that I ρ, which is a submatix of A, is an ieducible M-matix. By Lemma 3, the pincipal submatix of A coesponding to N \(S 0 S is eithe non-singula (when S 0 S o is I ρ; in eithe case, it has ank n max{1, S 0 S }. So we have dim(w = n max{1, S 0 S } and dim(w = 1 + max{1, S 0 S }. Finishing the poof as in Lemma 6, we have dim(v S dim(v S + dim(v S 1 max{1, S 0 S } dim(v S + max{1, S 0 S } whee the second inequality uses Lemma 8 (instead of Lemma 4. The choice of assotments demonstating the subspace of equied dimension is the same as befoe, except now we show that the dimension is at least n. Again, fix some i N, and choose the collection of n assotments S ( N as befoe (descibed in and diectly befoe Equation (7. Following the inductive agument in the poof of Lemma 7, but now using Lemma 8 and Lemma 9 (instead of Lemma 4 and Lemma 6, we have dim span {h j,s : j S} ( (n 1 ( max{1, 1} = n. S S Since each of the systems of linea equations fom Equation (3 and Equation (4 have (at most n unknown vaiables, we conclude that the ρ i and λ ae unique as solutions to thei espective systems of linea equations. This concludes the poof of Theoem 1. 9

10 4.3. Total numbe of assotments equied We now show that the numbe of assotments fo which we need the choice pobabilities is O(n 2 fo n/2. Indeed, the constuction given above based on Lemma 7 can be used to avoid using all assotments of size (and + 1 in Algoithm 1. We choose two sets S, S ( N 1, which shall seve as common intesection sets (in the sense used in Section 4.1.4, as follows. The fist set S ( N 1 is chosen abitaily; it seves as the common intesection set fo all i N \ S. The second set S ( N \S 1 is chosen abitaily as long as it is disjoint fom S (which is possible because n/2; it seves as the common intesection set fo i S. Fo each i N \ S, we need the equations fo the assotments S {k} fo all k N \ (S {i}. Obtaining the equations fo one such S {k} equies choice pobabilities fo assotments S {k} and S {k} {j} fo j N \(S {k} as pe Lemma 2. In total, fo all i N \ S, we need choice pobabilities fo O(n 2 assotments. Fo the emaining i S, we use the same agument fo the disjoint common intesection set S, and thus equie the choice pobabilities fo at most anothe O(n 2 assotments. 5. Discussion Ou main esult establishes the identifiability of MCCM paametes fom choice pobabilities fo assotments of sizes diffeent fom n 1 and n. This is impotant because eal systems often have cadinality constaints on the assotment sizes. While such constaints ae typically consideed in the context of assotment optimization (see, e.g., Dési et al., 2015, it is also impotant in the context of paamete estimation. One complication of using small size assotments to estimate the MCCM paametes is that the numbe of diffeent assotments equied may be as lage as O(n 2. In contast, only n + 1 assotments ae needed when the sizes ae n 1 and n. On the othe hand, the statistical difficulty of estimating choice pobabilities fo lage assotments may be highe than the same task fo smalle assotments. So the possible tade-offs in sample complexity is not staightfowad fom this analysis. This is an inteesting question that we leave to futue wok. Acknowledgments We ae gateful to Shipa Agawal and Vineet Goyal fo helpful discussions, and to Vineet fo oiginally suggesting this poblem. This wok was suppoted in pat by NSF awads DMR and IIS , a Bloombeg Data Science Reseach Gant, a Sloan Reseach Fellowship, and the Reseach Oppotunities and Appoaches to Data Science (ROADS gant fom the Data Science Institute at Columbia Univesity. Refeences Abaham Beman and Robet J Plemmons. sciences. SIAM, Nonnegative matices in the mathematical 10

11 Jose Blanchet, Guillemo Gallego, and Vineet Goyal. A makov chain appoximation to choice modeling. Opeations Reseach, 64(4: , James M Davis, Guillemo Gallego, and Huseyin Topaloglu. Assotment optimization unde vaiants of the nested logit model. Opeations Reseach, 62(2: , Antoine Dési, Vineet Goyal, Danny Segev, and Chun Ye. Capacity constained assotment optimization unde the makov chain based choice model Guillemo Gallego, Richad Ratliff, and Segey Shebalov. A geneal attaction model and sales-based linea pogam fo netwok evenue management unde custome choice. Opeations Reseach, 63(1: , R Duncan Luce. Individual choice behavio: A theoetical analysis. Wiley, Daniel McFadden and Kenneth Tain. Mixed mnl models fo discete esponse. Jounal of applied Econometics, pages , Robin L Plackett. The analysis of pemutations. Jounal of the Royal Statistical Society. Seies C (Applied Statistics, pages , Paat Rusmevichientong, David Shmoys, and Huseyin Topaloglu. Assotment optimization with mixtues of logits. Technical epot, Tech. ep., School of IEOR, Conell Univesity, Kenneth E Tain. Discete choice methods with simulation. Cambidge univesity pess,