An efficient method for computing single parameter partial expected value of perfect information

Transcription

1 An effcent metho for computng sngle parameter partal expecte value of perfect nformaton Mark Strong,, Jeremy E. Oakley 2. School of Health an Relate Research ScHARR, Unversty of Sheffel, UK. 2. School of Mathematcs an Statstcs, Unversty of Sheffel, UK. Corresponng author August 202 Abstract The value of learnng an uncertan nput n a ecson moel can be quantfe by ts partal expecte value of perfect nformaton EVPI. Ths s commonly estmate va a two level neste Monte Carlo proceure n whch the parameter of nterest s sample n an outer loop, an then contonal on ths sample value the remanng parameters are sample n an nner loop. Ths two level metho can be ffcult to mplement f the jont strbuton of the nner loop parameters contonal on the parameter of nterest s not easy to sample from. We present an smple alternatve one level metho for calculatng partal EVPI that avos the nee to sample rectly from the potentally problematc contonal strbutons. We erve the upwar bas an varance of our estmator. Introucton The value of learnng an nput to a ecson analytc moel can be quantfe by ts partal expecte value of perfect nformaton partal EVPI Raffa, 968; Claxton an Posnett, 996; Fell an Hazen, 998, The stanar two level Monte Carlo approach to calculatng partal EVPI s to sample a value of the nput parameter of nterest n an outer loop, an then to sample values from the jont contonal strbuton of the remanng parameters an run the moel n an nner loop Brennan et al., 2007; Koerkamp et al., Suffcent numbers of runs of both the outer an nner loops are requre to nsure that the partal EVPI s estmate wth suffcent precson, an wth an acceptable level of bas Oakley et al., 200. We recognse two mportant practcal lmtatons to the stanar two level Monte Carlo approach to calculatng partal EVPI. Frstly, the neste two level nature of the algorthm wth a moel run at each nner loop step can be hghly

2 computatonally emanng for all but very small loop szes f the moel s expensve to run. Seconly, we requre a metho of samplng from the jont strbuton of the nputs exclung the parameter of nterest contonal on the nput parameter of nterest. If the nput parameter of nterest s nepenent of the remanng parameters then we can smply sample from the uncontonal jont strbuton of the remanng parameters. Inee, Aes et al show that n certan classes of moel, most notably ecson tree moels wth nepenent nputs, the Monte Carlo nner loop s unnecessary snce the target nner expectaton has a close form soluton. However, f nputs are not nepenent we may nee to resort to Markov chan Monte Carlo MCMC methos f there s no close form analytc soluton to the jont contonal strbuton. Inclung an MCMC step n the algorthm s lkely to ncrease the computatonal buren conserably, as well as requrng atonal programmng. We present here a smple one level orere nput algorthm for calculatng partal EVPI that takes nto account any epenency n the nputs. The metho avos the nee to sample rectly from the contonal strbutons of the nputs, an nstea requres only a sngle set of the sample nputs an corresponng outputs n orer to calculate partal EVPI values for all nput parameters. We erve an expresson for the samplng varaton of the estmator. 2 Metho We assume we are face wth ecson optons, nexe =,...,, an have bult a computer moel y = f, x that ams to prect the net beneft of ecson opton gven a vector of nput parameter values x. We enote the true unknown values of the nputs X = {X,..., X p }, an the uncertan net beneft uner ecson opton as Y. We enote the parameter for whch we wsh to calculate the partal EVPI as X an the remanng parameters as X = {X,..., X, X +,..., X p }. We enote the expectaton over the full jont strbuton of X as E X, over the margnal strbuton of X as E X, an over the contonal strbuton of X X as E X X. The partal EVPI for nput X s ] EV P IX = E X [max E X X {f, X, X } max E X {f, X}. We wsh to evaluate the partal EVPI for each nput X wthout samplng rectly from the contonal strbuton X X, snce ths may requre computatonally ntensve numercal methos f nputs are correlate. Brefly, the ea s as follows. We assume we have a set of samples from the jont strbuton of the moel nput parameters, an a corresponng set of moel outputs.e. net benefts. The net benefts for each ecson opton are orere wth respect to the nput of nterest, an then parttone nto subsets of equal sze. Wthn each subset we calculate the mean of the net benefts for each ecson opton, an take the maxmum across the ecson optons. The average of these maxma s taken as an approxmaton to the 2

3 frst term n Equaton. The secon term n Equaton s compute usng stanar Monte Carlo samplng,.e. for each ecson opton we calculate the mean of the net benefts corresponng to the whole set of nput samples, an then take the maxmum of these means. 2. Algorthm In the followng subsectons we ntrouce notaton an escrbe the algorthm n etal n a seres of stages. Coe for mplementng the algorthm n R R evelopment Core Team, 20 s shown n appenx A an s avalable for ownloa from Stage We efne the Monte Carlo sample of moel nputs an corresponng moel outputs as {x s, y s, s =,..., S, =,..., }, where the xs are rawn from the jont strbuton of the nputs, px, an y s = f, xs s the evaluaton of the moel output at x s for ecson opton. Note the use of superscrpts to nex the ranomly rawn sample sets. We let M be the matrx of nputs an corresponng outputs x... x p y... y x 2... x 2 p y 2... y 2 M = x S... x S p y S... y S 2..2 Stage 2 For parameter of nterest, we extract the x an y,..., y columns an reorer wth respect to x, gvng M = x y... y x 2 y 2... y 2. x S..., 3 y S... y S where x x 2... x S. Note the use of brackete superscrpts to enote the sample set orere wth respect to the nput of nterest Stage 3 We partton the resultng matrx nto k =,..., K sub matrces M k of J rows each, M k = x,k x 2,k. x J,k y,k... y,k y 2,k... y 2,k..., 4 y J,k... y J,k 3

4 retanng the orerng wth respect to x, an where the row nexe j, k n Equaton 4 s the row nexe j + k J n Equaton 3. Note that J K must equal the total sample sze S Stage 4 For each M k we estmate for each ecson opton the contonal expectaton µ k = E X X =x {f, X k, X } by averagng over j =,..., J,.e. ˆµ k = J J j= y j,k, 5 where x k = J j= xj,k /J. The justfcaton for ths rests on recognsng that f J s small compare { to S, then the orere values of the nput of nterest x,k,..., x J,k } wll all be close to ther { mean value, x} k, an the corresponng values of the remanng nputs x,k,..., x J,k wll be approxmately a sample from the strbuton of X X = x k. See appenx B for a more formal justfcaton. The maxmum m k = max E X X =x {f, X k, X } s then estmate by ˆm k = max ˆµ k, 6 an fnally we estmate the frst term n the rght han se of Equaton by averagng over k =,..., K,.e Stage 5 ˆm = K ˆm k. 7 k= We estmate the secon term n the rght han se of Equaton usng smple Monte Carlo samplng,.e. max E X {f, X} max S S y n. 8 where the orer of the x n s rrelevant. Stages 2 to 4 are repeate for each parameter of nterest, notng that only a sngle set of moel runs stage s requre. n= 2.2 Choosng values for J an K We assume that we have a fxe number of moel evaluatons S an wsh to choose values for J an K subject to the constrant J K = S. Frstly we note that for small values of J the EVPI estmator s upwarly base ue to the maxmsaton n Equaton 6 Oakley et al., 200. Inee for 4

5 J = an K = S our orere nput estmator for the frst term n the rght han se of Equaton reuces to S S s= max ys, 9 whch s the Monte Carlo estmator for the frst term n the expresson for the overall EVPI, } EV P I = E X {max f, X max E X {f, X}. 0 Seconly we note that for very large values of J, an hence small values of K, the EVPI estmator s ownwarly base, an converges to zero when J = S. In ths case our orere nput estmator for the frst term n the rght han se of Equaton reuces to max S S y, s whch s the Monte Carlo estmator for the secon term n the rght han se of Equaton. The precson of the partal EVPI estmate only epens on S an not on J an K see Secton 2.4 for the ervaton of an expresson for the varance of the estmator. We therefore only nee to conser the mnmsaton of bas n our choce of J an K when S s fxe. Because the upwar bas ue to small J converges to zero as J ncreases, a sensble choce of J s that whch s just large enough such that the estmate bas ˆb s smaller than some constant c. Any choce of J larger than ths wll rsk ntroucng a ownwar bas whch becomes apparent at small values of K. s= 2.3 Estmaton of the upwar bas n the frst term of the partal EVPI estmator We estmate the upwar bas n the followng manner, usng the metho propose by Oakley et al Frstly, we wrte the vector of Monte Carlo estmators for the contonal expecte net benefts from Equaton 5 as ˆµ k =. ˆµ k,..., ˆµk If we can etermne the samplng strbuton of ths vector of estmators then we can quantfy the upwar bas n ˆm, an hence the upwar bas n the partal EVPI. Unless J s very small, ˆµ k wll follow a multvarate Normal strbuton wth mensons. Thus we have ˆµ k N µ k, J V k, 2 where µ k = by, µ k,..., µk an where each element p, q of V k s estmate ˆV k p,q = cov ˆµ k p, ˆµ k q. 3 5

6 In orer to estmate the bas n ˆm we frst raw, for each k =,..., K, a set of N samples from a multvarate Normal strbuton wth mean vector ˆµ k an varance matrx k J ˆV p,q. We choose N to be large, say,000. Let us enote these samples µ k n = µ k,n,..., µk,n for n =,..., N an k =,..., K. The bas n ˆm k s estmate by ˆbk = N N n= an the expecte bas n ˆm as, { } { } max µ k,n,..., µk,n max ˆµ k,..., ˆµk, 4 ˆb = K ˆbk. 5 k= 2.4 Estmaton of the varance of the frst term of the partal EVPI estmator Here we erve an expresson for the varance of ˆm, the frst term n the estmator for the partal EVPI Equaton. If we enote k = arg max we can rewrte Equaton 7 as The varance of ˆm s snce the y j,k k ˆµ k Ê X ˆm k = ˆm = K k= k= ˆm k = ˆµ k K k k= = J K J = S var ˆm = var S = S 2 J k= j= J k= j= J k= j= var j= y j,k k y j,k. 6 k y j,k k y j,k k, 7 are nepenent. The estmator for var ˆm s therefore smply var ˆm = SS J k= j= ˆm 2 y j,k. 8 k We see therefore that the precson of the frst term n the partal EVPI estmator oes not epen on the nvual choces of J an K, but only on S = J K. 6

7 Appenx A: R coe for mplementng the algorthm The partal.evp.functon functon as wrtten below takes as nputs the costs an effects rather than the net benefts. Ths allows the partal EVPI to be calculate at any value of wllngness to pay, λ. partal.evp.functon<-functonnputs,nput.of.nterest,costs,effects,lamba,j,k { S <- nrownputs # number of samples fj*k!=s stop"the number of samples oes not equal J tmes K" <- ncolcosts # number of ecson optons nb <- lamba*effects-costs baselne <- maxcolmeansnb perfect.nfo <- meanapplynb,,max evp <- perfect.nfo-baselne sort.orer <- orernputs[,nput.of.nterest] sort.nb <- nb[sort.orer,] nb.array <- arraysort.nb,m=cj,k, mean.k <- applynb.array,c2,3,mean partal.nfo <- meanapplymean.k,,max partal.evp <- partal.nfo-baselne partal.evp.nex <- partal.evp/evp returnlst baselne = baselne, perfect.nfo = perfect.nfo, evp = evp, partal.nfo = partal.nfo, partal.evp = partal.evp, partal.evp.nex = partal.evp.nex } Appenx B: Theoretcal justfcaton for the algorthm The orere algorthm s a metho for effcently computng the nner expectaton n the frst term of the rght han se n Equaton. roppng the ecson opton nex for clarty but wthout loss of generalty, our target s E X X =x {fx, X } where x s a realse value of the parameter of nterest, an X are the remanng uncertan parameters wth jont contonal strbuton px X{ = x. } Gven a sample x,..., xj from px X = x, the Monte Carlo estmator for E X X =x {fx, X } s Ê X X =x {fx, X } = J J f j= x, x j. 9 7

8 In our orere approxmaton metho we replace Equaton 9 wth Ê X X =x {fx, X } = J J f j= x + ε j, x j, 20 where {x + ε,..., x + ε J} = {x px X [x,..., x J } s an orere sample from s a ± ζ] for some small ζ an therefore ε 0, an xj sample from px X = x + ε j Equaton 20 s an unbase Monte Carlo estmator of { E X [x ±ζ] EX X fx, X } = fx, X px X px X [x ± ζ]x X, 2 X X whch we can rewrte by ntroucng an mportance samplng rato as X = = X X fx, X px X px X [x ± ζ]x X X fx, X px X px X [x ± ζ] X px X px X = x px X px X = x X X px X fx, X X px X = x px X [x ± ζ]x px X = x X. wthn the nner ntegral as a func- We wrte the terms fx, X ton g,.e. px X px X =x px X fx, X px X = x = gx, x, X. If g s approxmately lnear n the small nterval X [x ±ζ] then we can express gx, x, X as a frst orer Taylor seres expanson about gx, x, X, gvng px X fx, X px X = x = gx, x, X, g x, x, X + X x g X, x, X X = fx, X + X x g X, x, X X X=x X=x Substtutng back nto Equaton 22 wth c = gx,x,x X gves X=x px X fx, X X X px X = x px X [x ± ζ]x px X = x X {fx, X + cx x } px X [x ± ζ] X px X = x X. X X Snce X cx x px X [x ± ζ]x = E X [x ±ζ] {cx x } 0 an. 22 8

9 X px X [x ± ζ] X =, then {fx, X + cx x } px X [x ± ζ] X px X = x X, X X = fx, X px X = x X, X = E X X =x {fx, X }. px X Hence, we have shown that as long as gx, x, X = fx, X px X =x s suffcently smooth such that t s approxmately lnear n some small nterval X [x ± ζ], the orere approxmaton metho Equaton 20 wll prove a goo estmate of our target contonal expectaton E X X =x {fx, X }. Acknowlegements MS was fune by UK Mecal Research Councl fellowshp grant G06072 urng the course of ths work. 9

10 References Aes, A. E., Lu, G. an Claxton, K Expecte value of sample nformaton calculatons n mecal ecson moelng, Mecal ecson Makng, 24 2: Brennan, A., Kharroub, S., O Hagan, A. an Chlcott, J Calculatng partal expecte value of perfect nformaton va Monte Carlo samplng algorthms, Mecal ecson Makng, 27 4: Claxton, K. an Posnett, J An economc approach to clncal tral esgn an research prorty-settng, Health Economcs, 5 6: Fell, J. C. an Hazen, G. B Senstvty analyss an the expecte value of perfect nformaton, Mecal ecson Makng, 8 : Fell, J. C. an Hazen, G. B Erratum: Correcton: Senstvty analyss an the expecte value of perfect nformaton, Mecal ecson Makng, 23 : 97. Koerkamp, B. G., Myram Hunnk, M. G., Stjnen, T. an Wensten, M. C Ientfyng key parameters n cost-effectveness analyss usng value of nformaton: a comparson of methos, Health Economcs, 5 4: Oakley, J. E., Brennan, A., Tappenen, P. an Chlcott, J Smulaton sample szes for Monte Carlo partal EVPI calculatons, Journal of Health Economcs, 29 3: R evelopment Core Team 20. R: A Language an Envronment for Statstcal Computng, R Founaton for Statstcal Computng, Venna, Austra, ISBN Raffa, H ecson Analyss. Introuctory Lectures on Choces Uner Uncertanty, Reang, Massachusetts: Ason-Wesley. 0