Generating a Random Collection of Discrete Joint Probability Distributions Subject to Partial Information

Transcription

1 Methodol Comput Appl Pobab DOI / Geneating a Random Collection of Dicete Joint Pobability Ditibution Subject to Patial Infomation Lui V. Montiel J. Eic Bickel Received: 4 Octobe 011 / Accepted: 7 May 01 Spinge ScienceBuine Media, LLC 01 Abtact In thi pape, we develop a pactical and flexible methodology fo geneating a andom collection of dicete joint pobability ditibution, ubject to a pecified infomation et, which can be expeed a a et of linea containt (e.g., maginal aement, moment, o paiwie coelation). Ou appoach begin with the contuction of a polytope uing thi et of linea containt. Thi polytope define the et of all joint ditibution that match the given infomation; we efe to thi et a the tuth et. We then implement a Monte Calo pocedue, the Hit-and- Run algoithm, to ample point unifomly fom the tuth et. Each ampled point i a joint ditibution that matche the pecified infomation. We povide guideline to detemine the quality of thi ampled collection. The ampled point can be ued to olve optimization model and to imulate ytem unde diffeent uncetainty cenaio. Keywod Joint pobability ditibution Simulation Hit and un Polytope AMS 010 Subject Claification 90C99 65C05 60H99 1 Intoduction Many deciion amenable to quantitative method include ignificant uncetainty. In thee cae, we often aume the undelying pobability ditibution i known. Howeve, in many ituation we have only patial infomation with which to contain L. V. Montiel J. E. Bickel (B) Gaduate Pogam in Opeation Reeach/Indutial Engineeing, The Univeity of Texa at Autin, 1 Univeity Station C00, Autin, TX 7871, USA ebickel@mail.utexa.edu L. V. Montiel lvmontiel@gmail.com, lvm9@mail.utexa.edu

2 Methodol Comput Appl Pobab thi ditibution. Fo example, we may know only the maginal ditibution and ome paiwie coelation coefficient. In thee cae, the pobability ditibution i not unique, but athe i a membe of a et of ditibution with known popetie. Thi uncetainty about the undelying ditibution poe a ignificant challenge and olution fall into two categoie. Fit, obut optimization method, eek a olution that i in ome ene good ove a wide ange of poible cenaio (Ben-Tal and Nemiovki 00). Second, appoximation method have been popoed, which poduce a ingle joint ditibution given patial infomation. The mot popula of thee appoache i maximum entopy (Jayne 1957, 1968). In thi pape, we peent a imulation pocedue to ceate not one, but a collection of joint ditibution unifomly ampled fom a finite dimenional et conitent with the given infomation. Specifically, ou pocedue geneate a collection of finitedimenional, dicete, joint pobability ditibution whoe maginal have finite uppot. Thi pocedue can be ued in conjunction with o a an altenative to the appoximation and obut optimization method dicued above. A an example, conide a andom vecto X ={X 1, X,...,X n },withpecified maginal ditibution F i (X i ) and coelation matix X. Thee ae an infinite numbe of joint ditibution G(X) that match thee containt. We efe to thi et of ditibution a the tuth et (T). By tuth we mean that any ditibution within thi et i conitent with the tated containt and theefoe could be the tue joint ditibution. Ou goal i to geneate a collection of joint ditibution G i (X),i= 1toN, that ae conitent with the given infomation, whee N i the numbe of ample in ou collection. A we detail below, we ue the Hit-and-Run (HR) ample to poduce a collection of ample unifomly ditibuted in T (Smith 1984). It i impotant to emphaize that the method we ugget hee i fundamentally diffeent fom othe method of andom vaiate geneation uch a NORTA (Ghoh and Hendeon 003) and cheboad technique (Mackenzie 1994; Ghoh and Hendeon 001). Thee method poduce intantiation x of X baed on a ingle ditibution G(X) that i conitent with a et of pecified maginal ditibution, coelation matix, and in the cae of NORTA, the aumption that the undelying dependence tuctue can be modeled with a nomal copula. Thu, NORTA and the cheboad technique poduce andom vaiate baed on a ingle ditibution contained within T. A the eade will ee, in ou dicete etting, we enviion the ample pace of X a being fixed and theefoe eek to ceate a et of dicete pobabilitie that ae conitent with the given infomation. Befoe poceeding, we hould explain thi focu on geneating pobabilitie G(X) athe than outcome of X. Within the deciion analyi community, fo example, the poblem of pecifying a pobability ditibution given patial infomation i well known (Jayne 1968) and of geat pactical impotance (Abba 006; Bickel and Smith 006). Fo example, uppoe one know that the aveage numbe olled on a ix-ided die i 4.5. What pobability hould one aign to each of the ix face? A dicued above, one poible appoach i the application of maximum entopy (Jayne 1957, 1968). Maximum entopy would pecify the (unique) pobability ma function (pmf) that i cloet to unifom, while having a mean of 4.5. The pocedue decibed in thi pape wa oiginally developed to tet the accuacy of maximum entopy and othe appoximation. Hence, it exploe a lage numbe of pobability ditibution unifomly ampled fom T.

3 Methodol Comput Appl Pobab Liteatue Review HR i an effective method to ample the inteio of a polytope and i eay to implement. Howeve, it i not the only poible ampling pocedue. In the following, we peent a bief eview of altenative method and dicu thei hotcoming. The fit et of ampling pocedue ae acceptance-ejection method (von Newmann 1963). Thee method embed the egion of inteet S within a egion D fo which a unifom ampling algoithm i known. Fo example, one might embed S within the union of non-ovelapping hypeectangle o hypephee (Rubin 1984) and then unifomly ample fom D, ejecting point that ae not alo in S. A pointed by Smith (1984), thi method uffe fom two ignificant poblem a fa a ou wok i concened. Fit, embedding the egion of inteet within a uitable upeet may be vey difficult (Rubin 1984). Second, a the dimenion of S inceae, the numbe of ejection pe accepted ample (i.e., the ejection ate) gow exponentially. Fo example, Smith (1984) how that when S i a 100-dimenional hypecube and D i a cicumcibed hypephee, ample ae equied on aveage fo evey ample that i accepted. The polytope that we conide ae at leat thi lage and moe complex. The econd altenative, decibed by Devoye (1986), conit of geneating andom point within the polytope by taking andom convex combination of the polytope vetice. Thi method i clealy infeaible fo mot poblem of pactical inteet, ince it equie pecifying all of the polytope vetice in advance. Fo highdimenional polytope thi i vey difficult, if not impoible, on a eaonable time cale. Fo example, conide a imple joint pobability ditibution compied of eight binay andom vaiable, whoe maginal ditibution ae known. The polytope encoding thee containt could have up to vetice (McMullen 1970). While thi i an uppe bound, the numbe of vetice one i likely to encounte in eal poblem i till enomou (Schmidt and Matthei 1977). The final altenate method i baed on decompoition, in which the aea of inteet i divided into non-ovelapping egment fo which unifom ampling i eay to pefom. Again, thi method equie full knowledge of all the exteme point of T. Rubin (1984) povide a bief eview of uch method and note that they entail ignificant computational ovehead and ae pactical only fo low-dimenional polytope. HR i a andom walk though the egion of inteet. A uch, it avoid the poblem detailed above ince evey ampled point i feaible and it doe not equie knowledge of the polytope vetice. The dawback of thi method i that the ample ae only aymptotically unifomly ditibuted and it can take a lage numbe of ample befoe the ample et i acceptably cloe to unifom (Rubin 1984; Smith 1984). We deal with thi iue below. Thi pape i oganized a follow. Section 3 peent a motivational example to build the eade intuition egading the method we decibe. Section 4 decibethe geneal pocedue fo geneating collection of joint ditibution. Section 5 peent an illutative example of the ampling pocedue. Finally, in Section 6 we conclude and dicu futue eeach. 3 Motivational Example To illutate and motivate ou technique, uppoe we ae deciding whethe o not to maket a new poduct. At peent, we ae uncetain about ou poduction cot and

4 Methodol Comput Appl Pobab V1 V V1 1.0 P Pob. p1 0.5 Fequency q.5 p 1- P3 p3 1-1-p1-p-p q P1 1.0 (a) Pobability Tee. (b) Tuth Set L Nom (c) Euclidean Ditance Ditibution. Fig. 1 Two binay vaiable with unknown infomation whethe o not a competito will ente the maket. Let V1 epeent the uncetainty egading whethe the competito will ente the maket (V1 = 1) o not (V1 = 0), and let V epeent ou poduction cot being high (V = 1) o low (V = 0). Gaphically, we can epeent thee cenaio uing the binay pobability tee in Fig. 1a. We tat by auming we have no knowledge of the maginal ditibution of V1 and V no thei dependence tuctue. In thi cae, T conit of all joint ditibution of (V1, V ) with fou outcome n = 4 and pobabilitie p = ( p1, p, p3, p4 ). We can implify the joint ditibution uing only thee pobabilitie ( p1, p, p3 ) ince p4 = 1 p1 p p3. The tuth et, hown in Fig. 1b i a polytope in thee dimenion and it vetice epeent exteme joint ditibution. In thi cae, the cente of T i the joint pmf p = (0.5, 0.5, 0.5, 0.5), which aume the andom vaiable ae independent and with unifom maginal. The mall dot in Fig. 1b ae ample (complete pmf) geneated by the HR pocedue we decibe below. By meauing the Euclidean ditance (L-Nom) fom the cente of T to all othe joint ditibution in T (Fig. 1c), we ee that mot of the ample ae at 0.3 unit fom the cente. The ample, which coepond to the ditibution of volume in the tuth et, ae le concentated cloe to the cente and the cone. Now, uppoe we have infomation that thee i a 70% chance the competito will ente the maket and that thee i a 30% chance that poduction cot will be V1 V V1 Pob. p p V1 p1p=0.7 V p1p3= P3 0.5 p P p1-p-p p1-p=0.3 (a) Pobability Tee p1-p3=0.7 (b) Maginal Vaiable. Fig. Two binay vaiable with maginal pobability infomation P1 1.0 (c) New tuth et.

5 Methodol Comput Appl Pobab high (unconditioned on the enty of the competito). The new infomation modifie the pobability tee (Fig. a) and intoduce two new containt to etict the joint ditibution matching the maginal pobabilitie (Fig. b). Each containt i a hypeplane that cut T, educing it dimenion by one. A hown in Fig. c, T i now a line with exteme at (0.3, 0.4, 0.0) and (0.0, 0.7, 0.3). A a efeence, the ditibution that aume independence (i.e., the maximum entopy appoximation in thi cae) i located at (0.1, 0.49, 0.09) and i maked with a lage black dot. Late, we will expand thi example, inceaing it dimenion. 4 Geneal Pocedue 4.1 Poblem Statement The objective of ou pocedue i to ceate a collection of dicete joint pobability ditibution unifomly ampled fom a finite dimenional, continuou, convex and compact et that contain all poible ealization of the joint ditibution that ae conitent with given infomation. We aume the joint ditibution ae dicete with finite uppot, a ae the maginal ditibution. To enue that ou tuth et i convex, we only admit infomation that can be encoded with linea equality containt. While thi i cetainty a limitation, we ae till able to adde a lage cla of poblem that ae of pactical impotance (Bickel and Smith 006). 4. Notation In ode to decibe the ampling pocedue, we fit etablih the notation, which we illutate with a imple example uing a joint ditibution with two vaiable; the fit vaiable having High, Medium, and Low outcome, and the econd vaiable having Up and Down outcome. In thi cae, we equie one et including two andom vaiable V ={V 1, V }, plu two et fo the outcome O V1 ={H, M, L} and O V ={U, D}. Finally, we ceate a et including the cadinal poduct of all the outcome: U ={[H, U], [H, D], [M, U], [M, D], [L, U], [L, D]}. Additional et ae equied to include moe infomation. Fo example, et that include joint outcome whee V 1 i et to High. A fomal definition of the notation i peented next. Notation: Indice and et: I Set of available infomation. V Set of andom vaiable. V i V Random vaiable i in V. O V i Set of poible outcome fo andom vaiable V i. ω V i O V i Realization fo andom vaiable V i indexed by = 1,... O V i.

6 Methodol Comput Appl Pobab U Set of all joint outcome, U = O V 1 O V O V V. ω k U Joint outcome ω k ={ω V 1,ω V V,...,ωV z } indexed by k = 1,,... O V i. V i U V ω i Set of joint outcome fo which andom vaiable V i obtain the value ω V i. Set of joint outcome fo which andom U V ω i ω V j Data: q ω V i q V ω i ω V j vaiable V i and V j obtain value ω V i Pobability that V i = ω V i. Pobability that V i = ω V i and V j = ω V j. ρ Vi,V j Moment coelation between V i and V j. ρ V i,v j Rank coelation between V i and V j. σ Vi,V j Covaiance between V i and V j. m z V i The zth moment of andom vaiable V i. and ω V j. Deciion vaiable: p Vecto of deciion vaiable defining the joint pobability ma function. p ωk p Deciion vaiable decibing the pobability of the joint event ω k. Table 1 applie the et notation to ou example. The vaiable ae V 1 and V and thei epective maginal outcome ae O V1 ={H, M, L} and O V ={U, D}. ω V 1 1 = H i the fit poible ealization of V 1. The joint outcome ω k U ae defined a ω 1 =[H, U], ω =[H, D], ω 3 =[M, U],...,ω 6 =[L, D]. The pobabilitie of thee outcome ae p ω1 = P(V 1 = H, V = U), p ω = P(V 1 = H, V = D), p ω3 = P(V 1 = M, V = U),...,p ω6 = P(V 1 = L, V = D). We ue dot-notation to maginalize the andom vaiable. Fo example, U V ω 1 = 1 = U D,whee implie maginalization ove that andom vaiable. U H and U V ω Uing the ame index k fo ω k, we have U D ={ω,ω 4,ω 6 }. The et of available infomation, denoted a I {q V ω i, q V ω i include all the infomation to be included in the model. q V ω i ω V j,ρ Vi,V j,σ Vi,V j, m z }, V i P(V i = ω V i ) i Table 1 Notation example V ={V 1, V }, O V 1 ={H, M, L}, O V ={U, D}, U ={[H, U], [H, D], [M, U], [M, D], [L, U], [L, D]}, U H ={[H, U], [H, D]}, U U ={[H, U], [M, U], [L, U]}, U M ={[M, U], [M, D]}, U D ={[H, D], [M, D], [L, D]}, U L ={[L, U], [L, D]}, U HU ={[H, U]}, U HD ={[H, D]}, U MU ={[M, U]}, U MD ={[M, D]}, U LU ={[L, U]}, U LD ={[L, D]}.

7 Methodol Comput Appl Pobab the maginal ditibution fo vaiable V i, and q V ω i ω V j P(V i = ω V i, V j = ω V j ) i the paiwie joint ditibution fo vaiable V i, V j. When maginal infomation i available, it i poible to decibe the moment coelation ρ Vi,V j, the ank coelation ρ and the covaiance σ V i,v Vi,V j j fo the vaiable V i and V j. Additionally, if the maginal ae unknown, we can make ue of known moment m z to contain the V i tuth et. Ou notation can be extended to moe than two vaiable and to match thee-way o fou-way pobabilitie. 4.3 Containt We ae now in poition to contain the tuth et to match the infomation povided by I. In thi ection, we peent diffeent familie of equation that can be ued to contain T, ceating a ytem of m linea equation Ap = b and n non-negativity containt p 0, wheep ={p ω1, p ω,...,p ω U }. A R m n define the popetie we want to contain and b R m epeent the available infomation Matching Neceay Condition In all cae, the joint pmf mut um to one and each pobability mut be non-negative. We epeent thee containt with Eq. 1a and 1b. p ωk = 1, (1a) p ωk 0, ω k U. (1b) Equation 1a and 1b give the neceay and ufficient condition fo p to be a pmf and ae equied in all cae. Notice that Eq. 1a educe the dimenion of the polytope T fom n to n 1 and Eq. 1b limit T to poitive quadant. Thi containt alone aue that T i a compact et Matching Maginal and Paiwie Pobabilitie A econd et of equation i ued when we have infomation egading the maginal and paiwie pobabilitie. Equation a equie that the joint pobabilitie match the maginal aement. While, Eq. b equie that they match paiwie joint aement. ω V i ω V V i j ω p ωk p ωk = q ω V i V i V, ω V i O V i, (a) = q V ω i ω V j V i, V j V,(ω V i j,ωv ) i OV O V j. (b) Equation a and b can be extended to cove thee-way, fou-way, o highe-ode joint pobability infomation.

8 Methodol Comput Appl Pobab Matching Moment If the outcome can be epeented a numeical value, intead of categoical data, we can match moment infomation uing Eq. 3. (ω V i )z p ωk = m z V V i V. (3) i ω V i O V i ω V i Equation 3 matche the zth moment of vaiable V i. Fo z = 1 we can match the expected value of V i and fo z = we can match the econd aw moment. We note that z = 0 i imply a etatement of Eq. 1a and the equiement that the pobabilitie um to one. Recall the outcome ω V i ae known, o the containt i linea in the joint pobabilitie p ωk Matching Covaiance and Coelation If the fit moment fo vaiable V i and V j ae known, it i poible to etict the joint ditibution to match a given covaiance σ Vi,V j. Moeove, we can match the coelation if the vaiance fo V i and V j ae alo known. Equation 4a and 4b match the covaiance and moment coelation epectively. ω V i O V i ω V i O V i ω V j O V j ω V j O V j ω V i ω V i ω V j ω V j ω V i V j ω ω V V i j ω p ωk =σ Vi,V j m 1 V i m 1 V j V i, V j V, (4a) p ωk =ρ Vi,V j σ σ V i V j V i, V j V. (4b) whee σ V i i the vaiance of vaiable V i,andρ Vi,V j i the moment coelation of vaiable V i and V j Matching Speaman Coelation Coef f icient Anothe meaue of vaiation that equie le infomation, and can be ued with numeical a well a categoical infomation, i the ank coelation. The ank coelation i defined a ρ = V i,v j ( ( Cov P V i ω V i Va ( P ( V i ω V i ) (, P )) Va ( P V j ω V j ( )) V j ω V j )). (5) Unlike the Peaon poduct-moment coelation, ank coelation i invaiant with epect to the maginal outcome. Thi and othe chaacteitic make it a eliable meaue of aociation (fo moe on ank coelation and aement method ee Clemen and Reilly 1999 andclemenetal.000). Rank coelation only equie infomation egading the maginal pobabilitie fo V i and V j and can be decibed a a linea function a follow. Let H : R R be a two-place eal function, and let B = [x 1, x ] [ y 1, y ] be a ectangle whoe vetice ae in the domain of H. Accoding to Nelen (005), the H-volume i defined a V H [B] = H(x, y ) H(x, y 1 ) H(x 1, y ) H(x 1, y 1 ). (6)

9 Methodol Comput Appl Pobab Let ω k (V i) be the outcome ω Vi of vaiable V i at the joint outcome ω k and let ω k (V i) be the outcome ω Vi 1 of V i. Notice that ω V i 1 i the outcome that pecede ωv i in the maginal ditibution of V i. The cumulative pobabilitie that V i i le than the outcome ω k (V i) and ω k (V i) ae p k (V i) = P(V i ω k (V i)) and p k (V i) = P(V i ω k (V i)), epectively. Thee cumulative pobabilitie define the inteval I ωk (V i ) a follow I ωk (V i ) [ p k (V i), p k (V i) ]. (7) Uing the inteval I ωk (V i ) and I ωk (V j ), we can define a ectangula aea I ωk (V i ) I ωk (V j ) equivalent to B. Then, uing the H-volume we can define the ank coelation between V i and V j a [ V x p y Iωk (V i ) I ωk (V j ) ] ρ 3 V i,v j ωk =, (8) q ω k (Vi) q ω k (V j) 3 whee q ω k (Vi) = P(V i = ω k (V i)), which i the maginal pobability of vaiable V i having the outcome ω Vi at the joint outcome ω k. Additionally, the H-volume V H i a defined fo H = x y,wheex I ωk (V i ) and y I ωk (V j ). It i impotant to ecall that, the ank coelation ρ i bounded by a cala uch that a ˆm < 1, whee ˆm i the maximum numbe of poible outcome of vaiable V i and V j. The bound wee poven by Mackenzie (1994) fo unifom dicete ditibution. Mackenzie (1994) alo pove that lim ˆm a ˆm =1, meaning that uing moe outcome in each maginal ditibution povide a moe efined ank coelation bounded by [ 1, 1]. 4.4 Sampling Pocedue Afte chaacteizing the tuth et T, the next tep ue the HR ample (Smith 1984) to unifomly ample ditibution fom T. The HR pocedue i the fatet known algoithm to ample the inteio of an abitay polytope. The algoithm ha been poven to mix in O(h 3 ) time, whee h = (n m) i the dimenion of the polytope. Although the mixing time i polynomial, a dicued above, the numbe of ample equied to guaantee convegence to the unifom ditibution can be lage (Lovaz 1998). To ovecome thi poblem, in the following ection we popoe a pactical definition fo convegence that educe the numbe of ample equied to ceate a dicete epeentation of the tuth et Hit-and-Run Sample The algoithm i decibed below and illutated in two dimenion in Fig. 3. Step 1: Set i = 0 and elect an abitay point x i T. Step : Geneate a et D R n of diection. Step 3: Chooe a andom diection d i unifomly ditibuted ove D. Step 4: Find the line et L = T {x x = x i λd i,λa eal cala}. Step 5: Geneate a andom point unifomly ditibuted ove x i1 L. Step 6: If i = N, top. Othewie, et i = i 1 andetuntostep.

10 Methodol Comput Appl Pobab Step 1: x i. Step : Geneate et D. Step 3: Chooe d i D. Step 4: Ue d i to et L. Step 5: Select x i1 L. Step 6: Retun to Step. Fig. 3 Hit and un ample. Illutation of the algoithm in d The HR wa deigned fo full-dimenional polytope, howeve, with mino modification it can be adapted to ample efficiently fom non-full-dimenional et. Thee modification ae peented in the following ection Sampling Non-full-dimenional Polytope A we noted in Section 4.3, the chaacteization of T decibe the polytope a a ytem of m linea equation and n non-negative vaiable: Ap = b, p 0. TheHR ample i deigned to ample point in full-dimenional polytope. Howeve, the polytope T i not full-dimenional ince h = n m < n. To ovecome thi poblem, we find the pojection of p R n into the hypeplane Ap = b uing Eq. 9, wheei epeent the identity matix. p = (I A T (AA T ) 1 A) p A T (AA T ) 1 b (9) Then, we can ceate an hypephee D R n in the full-dimenional pace by ampling independent vecto of ize n fom the multivaiate tandad nomal and nomalizing them o that each vecto ha equal magnitude. Uing Eq. 9, wecan poject the diection et D into T. With the pope caling, the final eult i a et of diection D T fom which we can elect diection unifomly ditibuted. The line L i ceated by extending the diection ±d i D until p 0 i violated. The et of the implementation i taightfowad. Thi tep emove the non full dimenional poblem by educing the dimenion fom n to n m fo all the tep that equie it. It i now poible to teat T a a full-dimenional polytope in n m dimenion. 4.5 Stopping Time HR guaantee that the ampled collection eventually convege to the unifom ditibution ove T (Smith 1984). Howeve, a pointed by Rubin (1984), the theoetical numbe of equied ample to each thi convegence can be lage. Yet, a we how

11 Methodol Comput Appl Pobab in thi ection, the numbe of ample equied to achieve eaonable pefomance in pactical application in geneally much malle. Meauing the ate of convegence to the unifom ditibution, even in lowdimenional polytope, i vey difficult. Unifomity would imply that any poible patition of T contain a faction of ample that i popotional to that patition volume divided by the volume of the polytope. Computing the volume of abitay polytope i a difficult tak (Báány and Füedi 1987). In fact, in many cae, the volume of the polytope can only be appoximated by a andom walk though the polytope (Dye et al. 1991; Kannanetal.1996), a pocedue imila to HR. Theefoe, we popoe a meaue of convegence that doe not diectly ely on global popetie of the polytope and i eay to compute. We begin by noting that fo p i, a andom vecto ampled fom T uing HR, thee exit unique vecto μ ={μ 1,...,μ n } and σ ={σ1,...,σ n } uch that lim N p i N i=1 N = μ and lim N (p i μ) N i=1 = σ, whee all calculation ove p N 1 i ae pefomed element-wie. Recall that ince p i ha bounded uppot and the HR aue convegence in ditibution, all the moment mut convege (Caella and Bege 00, p. 65). A dicued below, we meaue convegence of HR by meauing the convegence of the ample mean and vaiance. Thee moment ae of paticula inteet due to thei intuitive intepetation. The ample mean decibe how cloe the cente of the collection i to the cente of T. The vaiance decibe how the dipeion of the ample matche the dipeion of the T volume. Hence, we now popoe the following definition fo what we tem fai-convegence. Definition 1 A collection of joint ditibution of ize N i called fai-in-mean, ifthe aveage vecto of the joint ditibution in a collection fo the fit N and N ample of the HR algoithm ae within an ε-ball of diamete α. Definition A collection of joint ditibution of ize N i called fai-in-dipeion, if the tandad deviation vecto of the joint ditibution in a collection fo the fit and N ample of the HR algoithm ae within an ε-ball of diamete β. N Definition 3 A collection of joint ditibution of ize N i called fai, if it i fai-inmean and fai-in-dipeion fo elected mall paamete α, β > 0. We implemented thee definition uing Eq. 10 and 11, wheep i i the ith ampled dicete pobability ditibution with n joint element. To make notation eaie we ue p i and aume all calculation ae pefomed element-wie except fo. Equation 10 compute the aveage of the collection ampled afte N iteation ( N p i i=1 N ) and compae it to the aveage afte N iteation. If afte N iteation the vecto of aveage i within an ε-ball if diamete α of the peviou vecto (fo ome mall α>0), we aume the ample i fai-in-mean. Equation 11 i the equivalent veion fo the vaiance, whee ( N j=1 p i p j ) = ( Np i N j=1 p j ) ( N = N j=1 p i p ) j = N (p i μ), N and whee μ i the vecto of aveage fo each joint element of the ample. In a imila way, if afte N iteation the new vecto of vaiance i within a ε-ball of

12 Methodol Comput Appl Pobab diamete β of the peviou vecto (fo ome mall β>0), we aume the ample i fai-in-dipeion. N p i N i=1 i=1 N i=1 p i N ( N ) N j=1 p i p j N 1 α (10) N i=1 4 ( N j=1 p i p j ) ( N 1) β N (11) The implementation of thi topping time fo the mean can be pefomed by keeping tack of N i=1 p i at each iteation and dividing it by the numbe of ample at each check point. Additionally, uing the ecuion in Eq. 1, we can alo keep tack of the vaiance of each joint element at each iteation. ( ) f (i) (i 1)σw,(i) = (i )σ w,(i 1) f (i), (1a) i i 1 f (i) = i 1 j=1 p j w (i 1)pi w i =, 3,...,N. σ w,(1) = 0, w {1,,...,n}. (1b) Ou expeience ugget that the numbe of ample equied fo fai convegence i conideably malle (eduction ae in the ode of 10 9 ) than the Lovaz (1998) theoetical lowe bound. A an example, Fig. 4 povide illutative eult fo fai convegence in ix uncontained polytope (Eq. 1 only) of diffeent dimenion. If T i uncontained (h = n 1), the tuth et i ymmetic and the cente of ma of T i known to be the dicete unifom ditibution. Theefoe, we can tet the convegence of HR by tating the algoithm at a point cloe to a cone and monito the numbe of ample needed fo the mean to aive within an ε-ball of adiu α>0 with cente at the dicete unifom ditibution. Fo thee collection, the algoithm will top once the ample i fai-in-mean, and uing Eq. 11 we check fo fai-in-dipeion. Thi i a tong tet becaue we ae electing the wot poible point to initialize the algoithm. Fig. 4 Minimum equied numbe of ample (N)to geneate a fai ample v the numbe of event (n) inthe ditibution fo ix uncontained polytope. A olid line connect the empiical data and the dahed line peent the bet fit Requied numbe of ample to enue fai convegence N 100,000,000 10,000,000 1,000, ,000 10,000 1, y=.845x R = n Numbe of joint event

13 Methodol Comput Appl Pobab In paticula, we initialize the algoithm by meauing the ditance fom the cente n 1 to a cone of T: n.wethenueδ = 1 τ and τ =.9 to define the initial point n n 1 p 0 ={1 δ (n 1),δ,...,δ} whee p 0 p = τ. Afte the initial point i n et, we look fo the mallet N uch that N p i i=1 N p n 1 <α= ϕ fo ϕ = n Finally, we check fo convegence evey K = 100 iteation. Fo the ample ize popoed the collection ae alo fai-in-dipeion. 5 Illutative Example We now demontate the pocedue by applying it to an extenion of the motivational example intoduced ealie. Recall that we ae unue if a competito will ente the maket (V 1 ) and about ou poduction cot (V ). Additionally, aume we ae in negotiation to acquie anothe company that would allow u to add a new Fig. 5 Poible pofit of the economic model V 1 Competito V Cot V 3 Acquiition V 4 Demand Pofit In million ente high cot (hc) ucceful () unucceful (u) high (h) med (m) low (l) h m l h m $40 $30 -$10 $10 -$10 -$50 $80 $40 low cot (lc) u l h m $10 $0 $0 l h m -$0 $95 $70 do not ente hc lc u u l h m l h m l h m $50 $90 $60 $30 $100 $80 $70 $95 $75 l $65

14 Methodol Comput Appl Pobab Fig. 6 Euclidean ditance fom each ampled point to cente poduct featue, booting ale. The acquiition will eithe be ucceful (V 3 = ) o unucceful (V 3 = u). In addition, we ae uncetain about the demand fo ou poduct (V 4 ), which could be high, medium, o low. In thi cae, thee ae 4 poible cenaio. Aume the company ha etimated the pofit aociated with each cenaio, which i hown in Fig. 5. We imulate 10 million joint pmf, unde an aumption of no infomation egading the likelihood of the vaiou cenaio. Fo each cenaio, we meaue the Euclidean ditance (i.e., the L-Nom) fom the ample to the cente of T,whichwe take to be the mean of all ampled ditibution. Figue 6 how a catteplot fo a potion of the imulation eult, whee each point i a poible tue joint pmf given the tate of infomation. Figue 7a peent the ditibution of the L-Nom, whee the olid line mak the exteme in the ample. A in Section 3, we obeve that the ample ae le concentated nea the cente and the cone of T and moe concentated at ditance whee the volume i moe abundant. Fo each ampled joint ditibution, we can calculate the expected pofit of ou model. Figue 7b peent the ditibution of the expected pofit when no infomation i available. Now, uppoe ou eeach team ha aeed the maginal pobabilitie fo the fou andom vaiable. Specifically, aume the pobability that the competito will ente i 70%, the pobability that poduction cot will be high i 30%, the pobability that the acquiition will be ucceful i 35%, and the pobabilitie fo the demand Fequency Fequency (a) Euclidean ditance pdf. L Nom (b) Expected pofit pdf (in million). Expected Pofit Fig. 7 Simulation eult fo a et with no patial infomation

15 Methodol Comput Appl Pobab Fequency Fequency (a) Euclidean ditance pdf. L Nom (b) Expected pofit pdf (in million). Expected Pofit Fig. 8 Simulation eult fo a et with maginal aement being high, medium, and low ae 50, 0 and 30%, epectively. The new eult fo the ditibution of the Euclidean ditance (L-Nom) and the expected pofit ae howninfig.8a and b, epectively. Again, the dahed line mak the exteme of the ample. Figue 8a how a light eduction in the ditance, compaed to Fig. 7a, fom the cente to the ampled point. Thi i the eult of containing the tuth et with new infomation egading the maginal pobabilitie. Figue 8b how a ubtantial eduction in the poible ange fo the expected pofit. Taking ou example one tep futhe, aume now that ou eeach team ha detemined that the ank coelation between V 1 (enty of the competito) and V (podution cot), V 1 and V 4 (demand), and V and V 4 ae 0., 0.3, and0., epectively. Figue 9a and b how the eult of the ditibution fo the Euclidean ditance and the expected pofit fo the thee cenaio: no infomation (black), only maginal aement (gey), and maginal aement and thee ank coelation (white). The exteme of each cae ae hown with vetical olid (no infomation), dahed (maginal only), and dotted line (maginal with thee paiwie coelation). We ee that additional infomation futhe contain the tuth et and educe the ange fo the poible expected pofit. Fequency Fequency (a) Euclidean ditance pdf. L Nom (b) Expected pofit pdf (in million). Expected Pofit Fig. 9 Simulation eult fo thee infomation cenaio

16 Methodol Comput Appl Pobab 6 Concluion The geneal imulation pocedue we have decibed povide a flexible and poweful tool to analyze tochatic model when the joint ditibution i incompletely pecified. The methodology i eay to implement, develop a collection of joint ditibution, and epeent an altenative to peviou appoache uch a obut optimization and appoximation uch a maximum entopy. We demontated the pocedue with a imple example baed on maginal and paiwie ank coelation coefficient. The methodology can be extended to highe numbe of andom vaiable, andom vaiable with moe than thee poible outcome, and higheode conditioning uch a thee-way aement. Futue eeach will exploe diffeent application of thi ampling methodology and tet the accuacy of exiting ditibution appoximation uch a maximum entopy, among othe. Appendix A: Speaman Coelation The deivation of Eq. 8 tat fom baic pinciple a follow: ρ V i,v j 3 = Cov(F Vi, F V j ) Va(FVi ) Va(F V j ) 3 = E(F V i, F V j ) = 1 E(F Vi, F V j ) (13a) = V i V j c(v i, V j ) dv i dv j = 1 p c ωk (V i, V j ) k (Vi) p k (Vi) p k (V j) p k (V j) V i V j dv i dv j (13b) = 3 [ [p c ωk (V i, V j ) k (V i)p k (V j) ] [ p k (V i)p k (V j) ] [ p k (V i)p k (V j) ] [ p k (V i)p k (V j) ] ] = 3 [ c ωk (V i, V j )V x y Iωk (V i ) I ωk (V j ) ] (13c) = 3 [ V x p y Iωk (V i ) I ωk (V j ) ] ωk. (13d) q ω ω k U k (Vi) q ω k (V j) In Eq. 13a Tanition ae given by F Vi U[0, 1], fo which mean and vaiance ae well known. In Eq. 13b We expand the expectation, patition the integal in ectangle, and et c ωk (V i, V j ) a contant inide each ectangle (Mackenzie 1994). In Eq. 13c We olve the integal and evaluate each ectangle aea. In Eq. 13d We ue the definition fo H-Volume and take c ωk (V i, V j ) a in Eq. 14.

17 Methodol Comput Appl Pobab Fo each ectangle aea in B, the pobability p ωk i the volume of a body with bae aea q ω k (Vi) q ω k (V j) and height c ωk (V i, V j ). Hence, we have: c ωk (V i, V j ) = p ωk q ω k (Vi) q ω k (V j). (14) Acknowledgement Thi mateial i baed upon wok uppoted by the National Science Foundation unde CAREER Gant No. SES Refeence Abba AE (006) Entopy method fo joint ditibution in deciion analyi. IEEE Tan Eng Manage 53(1): Báány I, Füedi Z (1987) Computing the volume i difficult. Dicete Comput Geom (1): Ben-Tal A, Nemiovki A (00) Robut optimization methodology and application. Math Pogam 9: Bickel JE, Smith JE (006) Optimal equential exploation: a binay leaning model. Deciion Anal 3(1):16 3 Caella G, Bege RL (00) Statitical infeence, nd edn. Duxbuy Advanced Seie, Thomon Leaning, Califonia Clemen RT, Reilly T (1999) Coelation and copula fo deciion and ik analyi. Manage Sci 45():08 4 Clemen RT, Fiche GW, Winkle RL (000) Aeing dependence: ome expeimental eult. Manage Sci 46(8): Devoye L (1986) Non-unifom andom vaiate geneation. Spinge-Velag, New Yok Dye M, Fieze A, Kannan R (1991) A andom polynomial time algoithm fo appoximating the volume of convex bodie. Jounal of the ACM 38(1):1 17 Ghoh S, Hendeon G (001) Cheboad ditibution and andom vecto with pecified maginal and covaiance matix. Ope Re 50(5): Ghoh S, Hendeon SG (003) Behavio of the NORTA method fo coelated andom vectogeneation a the dimenion inceae. ACM Tan Model Comput Simul 13(3):76 94 Jayne ET (1957) Infom. theoy and tatit. mechanic. pat 1. Phy Rev 106(4): Jayne ET (1968) Pio pobabilitie. IEEE Tan Syt Sci Cyben 4(3):7 41 Kannan R, Lovz L, Simonovit M (1996) Random walk and an O*(n 5 ) volume algoithm fo convex bodie. Random Stuct Algoithm 11(1):1 50 Lovaz L (1998) Hit-and-un mixe fat. Math Pogam 86(3): Mackenzie GR (1994) Appoximately maximum entopy multivaiate ditibution with pecified maginal and paiwie coelation. Dietation, Univ. of Oegon, Oklahoma McMullen P (1970) The maximum numbe of face of a convex polytope. Mathematika 17(): Nelen RB (005) An intoduction to Copula, nd edn. Spinge Seie in Statit., Spinge von Newmann J (1963) Vaiou technique ued in connection with andom digit. Collected Wok, vol 5. Pegamon Pe, New Yok, NY, pp Rubin PA (1984) Geneating andom point in a polytope. Commun Stat Simul Comput 13(3): Schmidt BK, Matthei TH (1977) The pobability that a andom polytope i bounded. Math Ope Re (3):9 96 Smith RL (1984) Efficient Monte Calo pocedue fo geneating point unifomly ditibuted ove bounded egion. Ope Re 3(6):