Proceedgs of the 007 Wter Smulato Coferece S G Hederso, B Bller, M-H Hseh, J Shortle, J D Tew, ad R R Barto, eds DERIVATIVE ESTIMATION WITH KNOWN CONTROL-VARIATE VARIANCES Jame R Welad Bruce W Schmeser School of Idustral Egeerg 35 North Grat Street Purdue Uversty West Lafayette, IN 7907, USA ABSTRACT We vestgate the cocepto that the sample varace of the cotrol varate (CV should be used for estmatg the optmal lear CV weght, eve whe the CV varace s kow A mxed estmator, whch uses a estmate of the correlato of the performace measure ( ad the cotrol ( s evaluated Results dcate that the mxed estmator has most potetal beeft whe o formato o the correlato of ad s avalable, especally whe sample szes are small Ths work s preseted terms of CV for famlarty, but ts prmary applcato s dervatve estmato I ths cotext, ulke CV, ad are ot assumed to be correlated INTRODUCTION I smulato expermets cotrol varate (CV estmators are used for varace reducto Much work has bee doe developg ad aalyzg CV estmators, cludg Laveberg ad Welch (98, Rubste ad Marcus (985, Nelso (989, Nelso ad Rchards (99, ad Szechtma ad Gly (00 I a expermet wth a objectve of estmatg the expected value of performace measure, the lear CV ( E ( estmator s α, where s the sample mea of the performace measure, s the sample mea of the cotrol, ad α s the CV weght The choce of α that mmzes the varace of the CV estmator s α, where s the covarace of ad, s the varace of the cotrol, ad α s referred to as the optmal CV weght (Law ad Kelto 000 Assumg depedet samplg, the varace of the CV estmator usg α s ( ρ, ( whch s less tha the varace of whe the correlato betwee ad, deoted ρ, does ot equate to ze- ro To use the optmal CV weght ad must be kow Otherwse, these quattes are estmated, resultg a varace reducto less tha that acheved ( (Bauer 987; ad Bauer, Vekatrama, ad Wlso 987 We cosder the case where α must be estmated, but the cotrol varace,, s kow I ths case, there s a choce of usg ether the kow varace,, or the sample varace, ˆ, estmatg α Such a case may occur, for example, whe the cotrol s a put varable wth a user-specfed dstrbuto Cheg ad Feast (980 ote that the majorty of cotrols suggested the lterature do ot have kow varace; usg stadardzed sums, they develop a method for covertg a cotrol wth ukow varace to oe wth kow varace Let ˆα be the estmate of α usg least-squares estmates for ad based o sample szes of ob- servatos, ˆ ˆ α ˆ ( ( ( ad ˆKV α be the estmate of α usg the least squares estmate of wth observatos ad kow varace, ˆ α ˆ ( ( ( KV Commo cocepto s that eve whe s kow, ˆα, rather tha ˆKV α, should be used to estmate α Ths cocepto could be attrbuted to results from the aalyss of rato estmators For example, usg the frst order terms of a Taylor seres expaso aroud the meas to approxmate the varaces of ˆα ad ˆKV α (see Appedx A, we fd that ˆα has lower varace tha α whe, ˆKV --306-0/07/$500 007 IEEE 560
Welad ad Schmeser ( ˆ Var Cov ( ˆ, ˆ ( Ths expresso dcates that postve correlato betwee the umerator ad deomator of a rato estmator ca reduce varace Due to commoalty of terms ˆ ˆ ad, these quattes ted to have postve covarace, whch s why t ca be preferable to use the sample varace, rather tha the kow varace estmatg α The applcablty of expresso ( s lmted, however, whe ad Var ( ˆ are ukow I ths work we develop a model, assumg that ad have a bvarate ormal dstrbuto, to quatfy the decso of whether to use ˆα or ˆKV α for estmatg α Results of our aalyss dcate that the correlato betwee ˆ ad ˆ s almost equvalet to ρ Ths s costructve because we have solved for a upper boud o ρ for whch ˆKV α s the preferred estmator Furthermore, wthout much addtoal computatoal effort, whe ρ s ukow, t ca be estmated ad used to dcate whether ˆα or ˆKV α s preferred Usg a estmate of ρ, deoted ˆ ρ, we evaluate a mxed estmator for α Compared to ˆα ad ˆα KV, the mxed estmator s performace s more robust evaluated across all possble values of ρ As a result, the mxed estmator wll provde the most beeft cotexts where ρ s ukow Despte ths work beg preseted terms of CV, ts applcablty ths cotext s lmted because cotrols are chose such that ρ s hgh order to maxmze the reducto varace acheved Our results are more useful, however, the cotext of gradet estmato, whch s the prmary motvato for ths work Welad ad Schmeser (006 propose us- g ˆα to estmate the dervatve of the expected value of the performace measure wth respect to the expected value of put, de ( de ( I ths cotext, ulke that of CV, the objectve s ot to reduce varace relatve to the sample mea, but oly to obta a pot estmate of the dervatve Furthermore, ad are ot assumed to be correlated as they are CV For example, f de ( / d E( s close to zero, the ρ may also be close to zero Thus, there s more potetal beeft usg ˆ ρ to dcate whether or should be used whe estmatg α ˆ both ad, deoted (,, for,,, ; we aalyze the problem of whether to use ˆα or ˆKV α for estmatg α There are, of course, other estmators that could be cosdered, oe of whch s evaluated Secto 5 The metrc used to compare estmators s E (( ˆ α α α + We refer to ths metrc as relatve error, whch s geeralzed MSE stadardzed by α + I the deomator of ths metrc, the absolute value of α s used to elmate dffereces estmatg postve ad egatve weghts Furthermore, oe s added to α prevetg dvso by zero for the cases where α 0 3 PREVIOUS WORK Bauer (987b frst proposed usg kow CV varaces for estmatg α Assumg a multvarate ormal model, he shows that kow varace ca yeld better estmators depedg o ρ Ths work dffers from our work that he does ot estmate ρ to dcate whch estmator s preferred Cheg ad Feast (980 use stadardzed sums to develop cotrols wth kow varaces They fd that these cotrols yeld better CV estmators Schmeser ad Taaffe (000 vestgate replacg the cotrol-smulato mea wth a approxmato The resultg cotrol-varate estmator s based COMPARING ESTIMATORS To compare ˆα ad ˆα KV, we assume that ad have a bvarate ormal (BVN dstrbuto wth parameters μ, μ,,, ad ρ The foudato of ths assumpto s that asymptotcally, as sample szes approach fty, most estmators follow a multvarate cetral lmt theorem Uder the BVN assumpto we fd that the correlato of ˆ ad ˆ, deoted ρ, s almost equvalet to ρ (See Fgure Sce these quattes capture the same effects, we proceed wth our aalyss terms of ρ rather tha Cov ( ˆ, ˆ, whch was the orgal cocept preseted Secto, because we have obtaed expressos for varaces of ˆα ad ˆα KV terms of ρ PROBLEM STATEMENT Gve performace measure E ( ; the expected value, μ, ad varace,, of cotrol/put ; sample sze ; ad the ablty to obta depedet observatos of 56
Welad ad Schmeser 0 09 08 07 06 05 0 03 0 0 00 00 0 0 03 0 05 06 07 08 09 0 Fgure : Estmated Mote Carlo results for ρ versus ρ Note that these two quattes are almost equvalet Uder the BVN assumpto, both ˆα ad ˆKV α are ubased estmators for α so we compare/cotrast oly the varaces of these estmators, whch assumg depedet samplg are ( ρ Var ( ˆ α 3 ad Var ( ˆ α KV ( ( + ρ ( Refer to Appedx B for detals The meas, μ ad μ, do ot affect the varaces Furthermore, ad oly re-scale the problem, affectg the varaces of both estmators equally The better estmator, as measured by lower varace, depeds o sample sze ad ρ, whch s typcally ukow As ρ, however, ˆα s the better estmator, because Var ( ˆ α 0 regardless of sample sze To further llustrate the depedece of the statstcal performace of these estmators o ρ ad sample sze, we plot the relatve error metrc versus ρ for ˆα ad ˆKV α See Fgure For smplcty ad wthout loss of geeralty, we fx ad cosder oly cases where ρ s postve because the graph s symmetrc alog the vertcal axs Fgure compares the relatve error for ˆα ad ˆα KV Error curves for ˆα KV are dsplayed wth dotted les, ad sold les are used to dsplay the curves for ˆα Curves for samples szes, 5,0 are show wth the darker gray les represetg larger sample szes The black curves dcate the lmtg cases as For a gve sample sze, the better estmator, as measure by lower relatve error, depeds o ρ As sample szes crease, ˆα s the better estmator across all ρ As stated prevously ths secto, the BVN assumpto s based o the cocept that asymptotcally, as sample szes approach fty, most estmators follow a multvarate cetral lmt theorem Despte the assumpto beg supported asymptotcally, we have preseted results that are depedet o sample sze These results should be terpreted the cotext of batch meas (Law 3 0 0 0 0 03 0 05 06 07 0 8 09 Fgure : Relatve error versus ρ for both ˆα ad ˆα KV Error curves for ˆα KV are dsplayed wth dotted les Sold les are used to dsplay the curves for ˆα Curves for samples szes, 5,0 are show wth the darker gray les represetg larger sample szes The black curves dcate the lmtg cases as 56
Welad ad Schmeser ad Kelto 000 We assume depedet samplg our aalyss, but smulato output data ca be autocorrelated I such a case the umber of depedet samples requred to equate to oe depedet sample s, + h h h ρ where ρh s the lag-h autocorrelato Ths expresso dcates that whe autocorrelato s preset output data, large samples of depedet data may be requred to obta the equvalet of oly a few depedet observatos Gve depedet observatos, the upper boud o ρ for whch the relatve error of ˆKV α s less tha that of ˆα s / ρ < (3 Usg ths expresso, f ρ were kow, we could determe whch estmator to use The decso rule would smply be to use ˆKV α f ρ s less tha ( / ad ˆα otherwse Whe ρ s ukow, whch s typcally the case, t ca be estmated, provdg formato as to whch s the preferred estmator for α 5 A MIED ESTIMATOR We ow exame the problem of developg a estmator for α gve ot oly the formato lsted the orgal problem statemet Secto, but also a estmate of ρ Assumg that we are gve ˆ ρ, we use expresso (3 to develop a ew estmator for α as ( / ˆ α, ˆ ρ ˆ α ˆ α KV, ow Ideally, the boudary betwee ˆα ad ˆKV α would be chose such that the relatve error of ˆα s mmzed Usg a boudary of ( /, however, s a reasoable approxmato because the error curves dsplayed Fgure for ˆα ad ˆKV α are somewhat lear Ths represets a costat loss fucto for mmzg the relatve error of ˆα Therefore, ay devato the optmal boudary away from ( / would be attrbuted ρ beg ukow Whe the loss fucto s ot costat, the optmal boudary leas towards the drecto of lower loss terms of relatve error 5 Expermetal Results Usg Mote Carlo results we estmate ( ˆ MSE α across sample szes, 5,0,00 ad correlatos ρ 0, 05, 05, 075, Fgure 3 dsplays the relatve error of ˆα versus ρ, whch s dcated by the mxed sold/dotted le The relatve error curves for ˆα ad ˆα KV are also dsplayed Fgure 3, wth sold curves for ˆα ad dotted curves for ˆα Two curves are dsplayed for each estmator The KV 3 0 0 0 0 03 0 05 06 07 08 09 Fgure 3: Relatve error versus ρ for ˆα, whch s dcated by the mxed sold/dotted le The relatve error curves for ˆα ad ˆα KV are also dsplayed, wth sold curves for ˆα ad dotted curves for ˆα KV Two curves are dsplayed for each estmator The lghter gray curves represet sample szes of The black curves represet sample szes of 5 563
Welad ad Schmeser gray curves represet sample szes of The black curves represet sample szes of 5 Curves for all three estmators for sample szes of 0 ad 00 are show Fgure Note that the scale of the vertcal axs has bee magfed so that dffereces betwee the estmators are vsble I Fgure, the gray curves represet sample szes of 0 The black curves represet sample szes of 00 Results dsplayed Fgures 3 ad show that ˆα has superor performace to ˆα for small sample szes ad almost equvalet performace for large sample szes Addtoally, lke the relatve error for ˆα, the error for ˆα s 0 whe ρ ad t creases as ρ 0 Oe reaso that error creases as ρ 0, especally for small sample szes, s that the varace of ˆ ρ creases as ρ 0 ad as sample sze decreases Because the relatve error of ˆα s ot lower tha that of ˆα KV across all values of ρ, a alteratve metrc for comparso gve sample sze would be area uder the relatve error curve Usg ths metrc, ˆα has slghtly better performace tha ˆα KV for, but much better performace as sample sze creases 6 CONCLUSIONS KV We proposed a mxed estmator that uses a estmate of ρ to decde betwee usg ad ˆ estmatg α Compared to ˆα ad ˆα, the mxed estmator s performace s more robust evaluated across all possble values of ρ Our recommedato s to use the mxed estmator for estmatg α cases where o pror formato about ρ s kow, regardless of sample sze Whe sample szes are small (e 0, ˆα should be used uless ρ s thought to be relatvely close to zero (e ρ 03 I that case ˆα KV should be used Whe sample szes are moderate to large (e > 0, there s relatvely lttle dfferece betwee ˆα ad ˆα Ether of these estmator are preferred to ˆα KV for all ρ Our results dcate that the cocepto that eve whe s kow ˆ should be used for estmatg α s true for large sample szes Whe sample szes are small, the decso as to whether to use or ˆ estmatg α s depedet o ρ 7 RELEVANCE AND FUTURE RESEARCH Ths work was preseted the cotext of cotrol varates, but ts relevace to ths area s lmted because cotrols are ofte chose such that ρ s hgh Therefore, mprovemet usg the proposed mxed estmator ( ˆα over the tradtoal estmator ( ˆα for the optmal cotrol weght s margal, especally for large sample szes The proposed mxed estmator has most potetal beeft whe lttle formato regardg ρ s kow Such a case commoly occurs gradet estmato, 5 05 0 0 0 0 03 0 05 06 07 08 09 Fgure : Relatve error versus ρ for ˆα, whch s dcated by the mxed sold/dotted le The relatve error curves for ˆα ad ˆα KV are also dsplayed, wth sold curves for ˆα ad dotted curves for ˆα KV Two curves are dsplayed for each estmator The lghter gray curves represet sample szes of 0 The black curves represet sample szes of 00 Note that the scale of the vertcal axs has bee magfed ths fgure, compared to that Fgures ad 3, so that dffereces betwee estmators are vsble 56
Welad ad Schmeser whch was the prmary motvato for ths work (refer to Welad ad Schmeser 006 for detals I ths cotext, ulke CV, there are typcally o pror assumptos regardg ρ We focused o aalyzg oly three estmators, but there are others that could be cosdered For example, the CV cotext t s assumed that the expected value of the cotrol s kow Ths formato could be used the estmator for the optmal CV weght by replacg the sample mea of the cotrol wth ts expected value as ( E ( ( ( E ( Compared to the tradtoal CV estmator, ˆα, ths estmator has a addtoal degree of freedom Aother alteratve would be to use a lear combato of ˆα ad ˆα KV, ταˆkv + ( τ ˆ α I ths estmator the weght, τ, should be chose to mmze relatve error Because ˆα ad ˆα KV are deped- et, the optmal weght s a fucto of Cov ( ˆ α, ˆKV α, whch would eed to be estmated Ths s a area of future research Aother area of future research s extedg these results to hgher dmesoal problems Such problems would corporate multple cotrols ad exted dervatve estmato to gradets APPENDI A Cosder two rato estmators R Z a ad R Z A, where a s E ( A The varace of R s Var ( Z a A approxmato of the varace of R s obtaed from usg the frst-order terms of a Taylor seres expaso aroud E ( Z ad E ( A, whch s Var ( Z E ( Z Var ( Z E ( Z Cov ( Z, A + 3 E( A E( A E( A Comparg the varace of R ad R, we fd that the varace of R s less tha that of R whe E( Z Var( Z Cov ( Z, A E A ( Puttg ths result terms of our otato for the optmal lear CV weght, we have Var ( ˆ Cov ( ˆ, ˆ Therefore, whe ths equalty holds, usg tha, results ˆα havg lower varace ˆ, rather APPENDI B ( + ρ Result : Var ( ˆ α KV ( Proof: Var ( ˆ αkv Var ( E ( ˆ α, KV x 3 Part A ( ( ˆ α x + E Var, KV 3 Part B Part A: Calculato for ˆKV α ( ( ˆ α x Var E, KV ( ( Var E, x ( Var ( x E (( x x ( Var ( x ( ( x α x x ( α Var ( ( ( ( ( ote that 0 ( Var ( α ( α α ( Var ( S Part B: Calculato for ˆKV α E Var ˆ α x, ( ( KV ( x x( ( ( ( E Var, x ( E Var ( ( x, ( assumg that are depedet of each other ote that 0 565
Welad ad Schmeser E ( E ( x ( ( x + ρ ( ( ( + ρ E ( ( ( ( + ρ + ρ ( E ( S ( ( x Var( x x ( Result : Var ( ˆ α ( ρ ( 3 Proof: Var ( ˆ α Var ( E ( ˆ α, x 3 Part A ( ( ˆ α x + E Var, 3 Part B Part A: Calculato for ˆα Var E ˆ α x, ( ( ( ( ( ( Var E, x ( Var Var ( x x E (( x ( x x ( x x ( x μ y0 + α E + ( y α( x μ 0 ( x x α ( x x( x x ( x x ( α Var Var 0 Part B: Calculato for ( ˆ α x Var, ˆα ( ( Var x, ( assume that are depedet of each other ( x x( ote that 0 ( x x Var ( x ( x x ( x x ( ( ρ ( x x ( ( ( x x ( ( ρ ρ ( x x ( x x Moreover, E Var α x, ( ( ˆ ( ( ρ ( ( ρ E E ( ( S ( ( ( ( ρ ρ E ( S 3 REFERENCES ( Bauer, K W 987 Improved batchg for cofdece terval costructo steady-state smulato PhD thess, School of Idustral Egeerg, Purdue Uversty, West Lafayette, Idaa Bauer, K W, Vekatrama, S, ad J R Wlso 987 Estmato procedures based o cotrol varates wth kow covarace matrx I Proceedgs of the 987 Wter Smulato Coferece, ed A These, H Grat, ad W D Kelto, 33 3 Cheg, RCH, ad G M Feast 980 Cotrol varables wth kow mea ad varace The Joural of the Operatoal Research Socety 3: 5 56 566
Welad ad Schmeser Laveberg, SS ad P D Welch 98 A perspectve o the use of cotrol varables to crease the effcecy of Mote Carlo smulatos Maagemet Scece 7: 3 335 Law, A M, ad W D Kelto 000 Smulato modelg & aalyss 3rd ed New ork: McGraw-Hll, Ic Nelso, B L 989 Batch sze effects o the effcecy of cotrol varates smulato Europea Joural of Operatoal Research 3:8 96 Nelso, B L, ad D Rchards 99 Cotrol varate remedes Operatos Research 38:97 99 Rubste, R, ad R Marcus 985 Effcecy of multvarate cotrol varates Mote Carlo smulato Operatos Research 33:66 677 Schmeser, BW, Taaffe, MR, ad J Wag 00 Based cotrol-varate estmato IIE Trasactos 33:9 8 Szechtma, R, ad P W Gly 00 Costraed Mote Carlo ad the method of cotrol varates I Proceedgs of the 00 Wter Smulato Coferece, ed B A Peters, J S Smth, D J Mederos, ad M W Rohrer, 39 00 Welad, J R, ad B W Schmeser 006 Stochastc gradet estmato usg a sgle desg pot I Proceedgs of the 006 Wter Smulato Coferece, ed L F Perroe, B G Lawso, J Lu, ad F P Welad, 390 397 AUTHOR BIOGRAPHIES JAMIE R WIELAND s a PhD studet the School of Idustral Egeerg at Purdue Uversty She receved a BS Idustral Egeerg & Maagemet Sceces from Northwester Uversty 00 ad a MS Idustral Egeerg from Purdue Uversty 003 Her research terests are stochastc operatos research ad ecoomcs Her emal address s <jwelad@purdueedu> BRUCE W SCHMEISER s a professor the School of Idustral Egeerg at Purdue Uversty Hs research terests ceter o developg methods for better smulato expermets He s a Fellow of INFORMS, s a Fellow of IIE, ad has bee actve wth the Wter Smulato Coferece for may years, cludg beg the 983 Program Char ad charg the Board of Drectors from 988 990 Hs emal address s <bruce@purdueedu> 567