Gradient Estimation Schemes for Noisy Functions Brekelmans, Ruud; Driessen, L.; Hamers, Herbert; den Hertog, Dick

Transcription

1 Tlburg Uverty Graet Etmato Scheme for Noy Fucto Brekelma, Ruu; Dree, L.; Hamer, Herbert; e Hertog, Dck Publcato ate: 003 Lk to publcato Ctato for publhe vero (APA: Brekelma, R. C. M., Dree, L., Hamer, H. J. M., & e Hertog, D. (003. Graet Etmato Scheme for Noy Fucto. (CetER Dcuo Paper; Vol Tlburg: Operato reearch. Geeral rght Copyrght a moral rght for the publcato mae acceble the publc portal are retae by the author a/or other copyrght ower a t a coto of acceg publcato that uer recoge a abe by the legal requremet aocate wth thee rght.? Uer may owloa a prt oe copy of ay publcato from the publc portal for the purpoe of prvate tuy or reearch? You may ot further trbute the materal or ue t for ay proft-makg actvty or commercal ga? You may freely trbute the URL etfyg the publcato the publc portal Take ow polcy If you beleve that th ocumet breache copyrght, pleae cotact u provg etal, a we wll remove acce to the work mmeately a vetgate your clam. Dowloa ate: 11. me. 016

2 No GRADIENT ESTIMATION SCHEMES FOR NOISY FUNCTIONS By Ruu Brekelma, Loeke Dree, Herbert Hamer a Dck e Hertog February 003 ISSN

3 Graet etmato cheme for oy fucto Ruu Brekelma 1, Loeke Dree, Herbert Hamer 3,DckeHertog 3,4 1 Tlburg Uverty, CetER Apple Reearch Cetre for Quattatve Metho BV, Ehove 3 Tlburg Uverty, Departmet of Ecoometrc a OR, P.O. Box 90153, 5000 LE Tlburg, The Netherla, e-mal: D.eHertog@uvt.l 4 Correpog author. February 7th, 003 Abtract I th paper we aalyze fferet cheme for obtag graet etmate whe the uerlyg fucto oy. Goo graet etmato e.g. mportat for olear programmg olver. A a error crtero we take the orm of the fferece betwee the real a etmate graet. Th error ca be plt up to a etermtc a a tochatc error. For three fte fferece cheme a two Deg of Expermet (DoE cheme we aalyze both the etermtc a the tochatc error. We alo erve optmal tep ze for each cheme, uch that the total error mmze. Some of the cheme have the ce property that th tep ze alo mmze the varace of the error. Bae o thee reult we how that to obta goo graet etmate for oy fucto t worthwhle to ue DoE cheme. We recomme to mplemet uch cheme NLP olver. Key wor: Deg of Expermet, fte fferece, graet etmato, oy fucto 1 Itroucto We are terete a fucto f : R R a more pecfcally t graet f(x. The fucto f ot explctly kow a we caot oberve t exactly. All obervato are the reult of fucto evaluato, whch are ubject to certa perturbato error.

4 Hece, for a fxe x R we oberve a approxmato g(x f(x+ε(x. (1 The error term ε(x repreet a raom compoet. We aume that the error term (1 are... raom error wth E[ε(x] 0 a V [ε(x] σ, hece the error term o ot epe o x. Note that g ca alo be a computer mulato moel. Eve etermtc mulato moel are ofte oy ue to all k of umercal error. I th paper we aalyze both fte fferece cheme a Deg of Expermet (DoE cheme for obtag graet etmato. I all thee cheme the graet etmate by obervg the fucto value everal pot the eghborhoo of x, ug fte tep ze h. We compare the reultg error mae the graet etmato ue to both the preece of oe a the etermtc approxmato error ( lack of ft. It wll appear that DoE cheme are worthwhle alteratve for fte fferece cheme the cae of oy fucto. Moreover, we wll erve effcet tep ze for the fferet cheme, uch that the total error (um of etermtc a tochatc error mmze. We wll compare thee tep ze to thoe whch mmze the varace of the total error. Graet play a mportat role all k of optmzato techque. I mot olear programmg (NLP coe, frt-orer or eve eco-orer ervatve are ue. Sometme thee ervatve ca be calculate ymbolcally: recet year automatc fferetato ha bee evelope; ee e.g. [7] a [3]. Although th becomg more a more popular, there are tll may optmzato techque whch fte fferecg ue to approxmate the ervatve. I almot every NLP coe uch fte fferece cheme are mplemete. Fte fferece cheme have alo bee apple to problem wth tochatc fucto. Kefer a Wolfowtz [8] were the frt to ecrbe the o-calle tochatc (qua graet; ee alo []. Metho bae o tochatc qua graet are tll ubject of much reearch; for a overvew ee [6]. So, although fte fferece cheme orgate from obtag graet etmato for etermtc fucto, they are alo apple to tochatc fucto. Alo the fel of Deg of Expermet (DoE, cheme are avalable for obtag graet etmato. Some popular cheme are full or fractoal factoral cheme, clug Plackett-Burma cheme. Cotrary to fte fferecg, thee cheme take

5 3 oe to accout. The cheme are uch that, for example, the varace of the etmator a mall a poble. However, mot DoE cheme aume a pecal form of the uerlyg moel, e.g. polyomal, a lack of ft uually ot take to accout. I [4] a [5] alo lack of ft take to accout bee the oe. I thoe paper t aalyze what happe whe the potulate lear (rep. quaratc moel mpecfe, ue to the true moel tructure beg of eco (rep. thr orer. I thee two paper ew DoE cheme are erve by mmzg the tegrate mea quare error for ether the prector or the graet. However, we thk that uch etmato are le valuable for optmzato purpoe ce the tegrate mea quare error ot a goo meaure for the graet oe pot. Moreover, the uerlyg aumpto thoe paper tll that the real moel quaratc ( [4] or thr orer ( [5] whch ot ecearly true. The remaer of th paper orgaze a follow. I Secto we aalyze three fte fferece cheme for obtag graet etmato. I Secto 3 we o the ame for two DoE cheme. I Secto 4 we compare the error of all the fve cheme. We e wth ome cocluo Secto 5. Graet etmato ug fte fferecg.1 Forwar fte fferecg Oe clacal approach to etmate the graet of f to apply forwar fte fferecg (FFD to the approxmatg fucto g, efe (1. I th cheme, a etmator of the partal ervatve, f(x x ( 1,...,, obtae by ˆβ FFD (h g(x + he g(x, h > 0, ( h where h the tep ze a e the -th ut vector. Ug (1 a Taylor formula, we ca rewrte the etmator a ˆβ FFD f(x + he f(x+ε(x + he ε(x h f(x x (3 + 1 het f(x + ζhe e + ε(x + he ε(x, (4 h whch 0 ζ 1. We are ow terete how goo th etmator. Note that E[ˆβ FFD ] f(x x + O(h (5

6 4 VAR[ˆβ FFD ] σ. (6 h The etmator error ε(x: ˆβ FFD a ˆβ FFD j Cov[ˆβ FFD FFD, ˆβ j ] E are correlate, becaue both epe o the raom ( (ˆβ FFD E[ˆβ FFD ](ˆβ FFD j E[ˆβ FFD j ] 1 h E ((ε(x + he ε(x(ε(x + he j ε(x 1 h E ( ε(x σ h, j. However, we are ot oly terete the error of the vual ervatve, but more the error mae the reultg etmate graet. A logcal meaure for the qualty of our graet etmato the mea quare error: E ( ˆβFFD f(x. Not oly the expectato mportat, but alo the varace VAR( ˆβFFD f(x, ce hgh varace mea that we ru the rk that the error a real tuato much hgher (or lower tha expecte. Suppoe for example that two mulato cheme have the ame expecte mea quare error, the we prefer the cheme wth the lowet varace. The varace ca alo be ue etermg the optmal tep ze h, a we wll ee Secto 4. By efg the etermtc error error FFD a the tochatc error we get error FFD E f(x+he 1 f(x ḥ. f(x+he f(x h ε(x+he 1 ε(x ḥ. ε(x+he ε(x h f(x ( ˆβFFD f(x ( error FFD + E error FFD.

7 5 From(3weealyervethat error FFD 1 4 h D, whch D the maxmal eco orer ervatve of f(x. Let u ow aalyze the tochatc error. The frt part of the followg theorem well-kow the lterature; ee ([10]. Theorem 1 For FFD we have ( E error FFD σ ( h VAR error FFD VAR( ˆβFFD f(x VAR h 4 [ (M4 σ 4 +M 4 +3σ 4] ( error FFD +σ D whch M 4 the fourth momet of ε(x (1,.e. M 4 E(ε(x 4. Proof. By efg ε ε(x + he, 1,...,, a ε 0 ε(x, we have for forwar fte fferecg E( error FFD 1 ( h E (ε ε 0 1 ( h E (ε + ε 0 ε ε 0 (7 σ h, (8 whch prove the frt part of the theorem. Coerg the eco part, we have VAR( error FFD E( error FFD 4 E ( error FFD. (9 Let u ow cocetrate o the frt term of the rght-ha e of (9: E( [ error FFD 4 1 h E (ε 4 + ε 0 ε ε 0 ] (ε j + ε 0 ε j ε 0 j 1 h [E ( ( ( ε 4 ε j + E ε ε 0 E ε εj ε 0 +E ( ( ( ε 0 ε j + E ε 0 ε 0 E ε 0 εj ε 0 E ( ( ε ε 0 ε j E ε ε 0 ε 0 +4E ( ε ε 0 εj ε 0 ] 1 h 4 [(M 4 + ( 1σ 4 + σ σ 4 + M σ 4 ] 1 h 4 [ (M 4 +3σ 4 +(M 4 +3σ 4 ] Subttutg th reult a the quare of (8 to (9, we have the eco part of the theorem. To prove the thr part, frt oberve that

8 6 ( ˆβFFD VAR f(x VAR( error FFD Further ote that a (error FFD E( error FFD error FFD + error FFD + error FFD 4 + E( error FFD 4 E ( error FFD + error FFD 4 + error FFD E( error FFD +4E((error FFD T error FFD ( +4E error FFD (error FFD T error FFD +4 error FFD E( ( error FFD T error FFD error FFD 4 error FFD E( error FFD E ( error FFD VAR( error FFD +4E((error FFD T error FFD + ( error FFD 4E (error FFD T error FFD VAR( error FFD +4 (error FFD E(error FFD + 4 (error FFD E(error FFD 3. (10 ( 1 E(error FFD 4 h D ( σ 1 σ D (11 h (error FFD E(error FFD 3 0, (1 ce E(error FFD 3 1 h E(ε 3 ε ( h E (ε 3 3ε ε 0 +3ε ε 0 ε (13 Fally, ubttutg (11 a (1 to (10 reult to the thr part of the theorem. Cetral fte fferecg A varat of the forwar fte fferecg (FFD the cetral fte fferecg (CFD approach. I th cheme, a etmato of the partal ervatve, f(x x ( 1,...,, obtae by ˆβ CFD (h g(x + he g(x he, h > 0, (14 h

9 7 where h the tep ze a e the -th ut vector. Ug (1 we ca rewrte the etmate a ˆβ CFD f(x + he f(x he +ε(x + he ε(x he h f(x x (15 + h 1 3 f(x + ζ 1 he [e,e,e ]+ h 1 3 f(x + ζ he [e,e,e ] (16 + ε(x + he ε(x he, (17 h where the lat equalty follow from Taylor formula f(x + he f(x+h f(x x whch 0 ζ 1 1, a + 1 h e T f(xe + h3 6 3 f(x + ζ 1 he [e,e,e ] f(x he f(x h f(x + 1 x h e T f(xe h3 6 3 f(x + ζ he [e,e,e ] whch 0 ζ 1. Let u frt aalyze the vual ervatve: a E[ˆβ CFD ] f(x + O(h (18 x VAR[ˆβ CFD ] σ h. (19 Cotrary to the FFD etmato, the etmato ˆβ CFD a ˆβ CFD j are ot correlate: ( Cov[ˆβ CFD CFD, ˆβ j ] E (ˆβ CFD E[ˆβ CFD ](ˆβ CFD j E[ˆβ CFD j ] 1 h E[(ε(x + he ε(x he (ε(x + he j ε(x he j ] 0, j. We ow aalyze the mea quare error crtero E ( ˆβCFD f(x, a t varace VAR( ˆβCFD f(x.

10 8 By efg error CFD f(x+he 1 f(x he 1 ḥ. f(x+he f(x he h f(x a error CFD ε(x+he 1 ε(x he 1 ḥ. ε(x+he ε(x he h we get ( ˆβCFD E f(x ( error CFD error CFD + E From (15 t eay to verfy that error CFD 1 36 h4 D3, whch D 3 the maxmal thr orer ervatve of f(x. Let u ow aalyze the tochatc error. The frt part of the followg theorem well-kow the lterature; ee ([10]. Theorem For CFD we have: VAR ( error CFD E σ ( h error CFD VAR [ M4 + σ 4] ( 8h 4 ˆβCFD f(x VAR( error CFD h σ D3. Proof. By efg ε ε(x + he, ε ε(x he,1,...,, a ε 0 ε(x we have for CFD E( error CFD 1 ( 4h E (ε ε 1 ( 4h E (ε + ε ε ε (0 σ h, (1 whch prove the frt part of the theorem. For the varace we have: VAR( error CFD E( error CFD 4 E ( error CFD (

11 9 Let u ow cocetrate o the frt term of the rght-ha e (: E( error CFD h E (ε 4 + ε ε ε (ε j + ε j ε j ε j j 1 16h [E ( ( ( ε 4 ε j + E ε ε j E ε εj ε j +E ( ( ( ε ε j + E ε ε j E ε εj ε j E ( ( ε ε ε j E ε ε ε j +4E ( ε ε εj ε j ] 1 16h [(M ( 1σ 4 + σ σ 4 +(M 4 + ( 1σ σ 4 ] 1 ( M4 + ( +1σ 4 8h 4 Subttuto of th reult a the quare of (1 to formula ( prove the eco part of the theorem. The lat part of the theorem follow mlar a the proof of the lat part of the prevou theorem: ( ˆβCFD VAR f(x VAR( error CFD +4 (error CFD E(error CFD + Th coclue the proof. 4 (error CFD VAR( error CFD VAR( error CFD E(error CFD 3 +4( 1 36 h4 D3( σ h h σ D3. The reult of th theorem ca be mply checke for a pecal cae. Suppoe that all ε(x are taar ormal trbute. The by ormalzg the tochatc error through the varace (ee (19, we kow that h error CFD σ (3 the um of quare tochatc ormally trbute varable, ce ( error CFD ε ε ormally trbute. Hece (3 χ ( trbute, wth expectato a varace. So,weget E( error CFD σ h, whch exactly the reult of the theorem. Furthermore, VAR( error CFD σ4 4h σ4 4 h, 4 whch alo agree wth the reult of the theorem, ce for a ormal trbuto we have M 4 3σ 4.

12 10.3 Replcate cetral fte fferecg To ecreae the tochatc error oe ca repeat cetral fte fferecg K tme. We call th replcate cetral fte fferecg (RCFD. Of coure the etermtc error wll ot chage by og replcato. The ext theorem how the expectato a varace of the reultg tochatc error. Theorem 3 For RCFD we have: VAR E ( error RCF D ( error RCF D σ h K [ M4 +(4K 3σ 4] 8h 4 K 3 VAR ( ˆβRCF D f(x VAR( error RCF D K h σ D 3. Proof. By efg ε k ε k (x + he, ε,k ε k (x he,1,...,, k 1,..., K a ε 0k ε k (x, where k eote the k-th replcate, we have for RCFD E( ( ( error RCF D 1 4h K E (ε k ε,k (4 k ( 1 4h K E (ε k ε l ε,k ε,l + ε,k ε,l ε,k ε,l (5,k,l σ h K, (6 whch prove the frt part of the theorem. For the varace we have: VAR( error RCF D E( error RCF D 4 E ( error RCF D (7

13 11 Let u ow cocetrate o the frt term of the rght-ha e (7: E( error RCF D 4 ( 1 16h 4 K E (ε 4 k ε l ε,k ε,l + ε,k ε,l ε,k ε,l,k,l ( ( 1 16h 4 K [E ε 4 k ε l + E,k,l ( ( +E ε,k ε,l + E,k,l ( +E ε,k ε,l ε k ε l + E,k,l,k,l ε k ε l ε,k ε,l,k,l,k,l ε,k ε,l ε,k ε,l,k,l,k,l ( ε,k ε,l,k,l +E ( ε,k ε,l ε,k ε,l + E,k,l,k,l ( ε,k ε,l ],k,l 1 16h 4 K 4 [[KM 4 + K ( 1σ 4 +3K(K 1σ 4 ]+[K σ 4 ] +[K σ 4 ]+[K σ 4 ] +[K σ 4 ]+[KM 4 + K ( 1σ 4 +3K(K 1σ 4 ] +[K σ 4 ]+[K σ 4 ]] 1 16h 4 K [KM 4 4 +(K ( 1 + 6K(K 1 + K +4K σ 4 ] Subttuto of th reult a the quare of formula (6 to formula (7 prove the eco part of the theorem. Fally, the thr part ca be erve almot etcal a the proof of the prevou theorem. 3 Graet etmato ug DoE 3.1 Plackett-Burma We ow aalyze Deg of Expermet (DOE cheme for etmatg the graet. Let u tart wth the Plackett-Burma cheme. Suppoe that we have a et of vector k R (k 1,...,N wth k 1a that we oberve g(x + h k for fxe x R a h>0. Defe the matrx 1 h T 1 X :... (8 1 h T N

14 1 Now uppoe that N, wth +1 N +4, a multple of four. The the Plackett- Burma cheme ca be wrtte a 1 p T 1 H.., 1 p T N where p k { 1, 1}. Th o-calle Haamar matrx ha the property H T H NI, where I the etty matrx. For more formato, ee [1] or [8] a for a example ee the Appex. Now let the vector k (8 be efe by k p k, k 1,...,N. It the follow that ( X T X ag N, h N,..., h N. The vector cotag the fucto value of f at x a the graet ca be etmate by the OLS etmator (ˆβPB g(x + h 1 0 (X T X 1 X T ˆβ PB. g(x + h N f(x + h 1 ε(x + h 1 (X T X 1 X T. +(X T X 1 X T.. f(x + h N ε(x + h N Frt ote that E[ˆβ PB 0 ]f(x+o(h, V[ˆβ PB 0 ] 1 +1 σ E[ˆβ PB ] f(x x + O( h, V[ˆβ PB ] σ, (+1h 1,...,. (9 Furthermore, ce the colum of X are orthogoal, we have Cov[ˆβ PB PB, ˆβ j ]0, j.

15 13 Now efg D a the X matrx exclug the frt colum, a f(x + h 1 error PB h N DT. f(x f(x + h N ε(x + h 1 error PB h N DT. ε(x + h N we have ( ˆβPB E f(x ( error PB error PB + E. Let u ow cocetrate o the etermtce error. Ug Taylor formula f(x + h k f(x+h T k f(x+ h T k f(x + ζh k k. whch 0 ζ 1, t eay to erve that error PB h D 4 whch D a overall upper bou for the eco orer ervatve. Cocerg the expectato a the varace of the tochatc error we have the followg theorem. Theorem 4 For Plackett-Burma eg we have: VAR E ( error PB σ Nh ( error PB VAR 4 N 3 h ( 4 ˆβPB f(x VAR( error PB Proof. For the Plackett-Burma cheme we have: error PB h N DT ν hn P T ν whch P the H matrx exclug the frt colum, a ε(x + h 1 ν.. ε(x + h N (M 4 +( N 3σ4 + 3 σ D N

16 14 We ca ow erve the followg: E( error PB h N E( P T ν (30 h N Nσ σ (31 Nh whch prove the frt part of the theorem. For the varace we have: VAR( error PB E( error PB 4 E ( error PB Let u ow cocetrate o the frt term of the rght-ha e of (3:. (3 E( error PB 4 h 4 N 4 E h 4 N 4 E h 4 N 4 E h 4 N [E 4 ( +E +E ( ( (P j ε j k ( P j P kj ε ε k,j,k ( (P kj ε k P j P kj P r P tr ε ε k ε ε t,j,k,r,,t ( P j P kj P r P kr ε ε k,j,k,r P j P j P r P r ε ε,j,,r ( P j P j P r P r ε 4,j,r ], where the lat equalty hol ce the expectato of the term ε ε k ε ε t, ε 3 ε k a ε ε k ε are zero. We ow cocetrate o the three term the lat equalty. For the frt term we have ( ( ( E P j P kj P r P kr ε ε k P j P r P kj P kr σ 4 + P j P j P kj P kj,j,k,r,j,r j k,,j k, P j P r ( P j P r +(N 1σ 4,j,r j ( 1N + (N 1σ 4 N(N σ 4. Moreover, for the eco term t hol ( E P j P j P r P r ε ε N(N 1σ 4.,j,,r σ 4

17 15 Forthethrtermwehave: ( E P j P j P r P r ε 4 NM 4.,j,r Subttutg thee reult a the quare of (31 to (3, we have prove the eco part of the theorem. The thr part of the theorem follow mlar a the proof of the lat part of Theorem 1: ( ˆβPB VAR f(x VAR( error BP +4 (error PB E(error PB + Th coclue the proof. 4 (error PB VAR( error PB E(error PB 3 +4( 1 4 h D( σ Nh +0 VAR( error PB + 3 σ D N. 3. Factoral eg Factoral eg are bae o the ame prcple a Plackett-Burma cheme, but ow N for full factoral eg a N p,p, for fractoal factoral eg; for more formato ee [1] or [8], a for a example ee the Appex. For the etermtc error we ca erve a better bou tha for Plackett-Burma cheme. Aga we have error FD f(x + h 1 h N DT. f(x. f(x + h N Now ug Taylor formula f(x + h k f(x+h T k f(x+ h T k f(x k + h3 6 3 f(x + ζh k [ k, k, k ], (33 whch 0 ζ 1, a ug the fact that factoral eg for each vector k there ext exactly oe other vector j the factoral eg cheme uch that k j,we obta by ag thee two vector: f(x + h k f(x + h j h T k f(x+ h3 3 D 3

18 16 whch D 3 a overall upper bou for the thr orer ervatve. Combg all N/ par of vector we get error FD f(x + h 1 h N DT. f(x f(x + h N h 4 D Cocerg the tochatg error we ca erve the followg reult. Theorem 5 For factoral eg we have: VAR E ( error FD σ Nh ( VAR error FD 4 N 3 h ( 4 ˆβFD f(x VAR( error FD (M 4 +( N 3σ N 3 h σ D3. Proof. Cocerg the frt a eco part we ca erve the ame reult a for Plackett-Burma eg the ame way. We therefore omt the proof of thee part. The thr part of the theorem follow mlar a the proof of the lat part of Theorem 1: ( ˆβFD VAR f(x VAR( error FD +4 (error FD E(error FD + 4 (error FD E(error FD 3. VAR( error FD +4( 1 36 h4 D3( σ Nh +0. VAR( error FD + 1 9N 3 h σ D3. 4 Comparo of the fve cheme I the prevou ecto we have erve both the etermtc a the tochatc etmato error for everal cheme; ee Table 1. The etermtc error are creag the tep ze h, whle the tochatc error are ecreag h. The expreo for the total error are covex fucto h. It traghtforwar to calculate the optmal tep

19 17 ze for each cheme uch that the total error mmze. The reult are metoe the lat colum of Table 1. Of coure, uually we o ot kow the value for σ,d a D 3. However, for a practcal problem we mght etmate thee value by amplg. Moreover, thee optmal tep ze gve ome cato; e.g., the tep ze are creag σ a ecreag N,D, a D 3, whch agree wth our tuto. #eval error E( error opt. h e Forwar FD h D Cetral FD 1 36 h4 D 3 Replcate CFD K 1 36 h4 D 3 σ h 4 σ h 6 σ h K 8 σ D 9 σ D σ KD 3 Plackett-Burma +1 N h D σ Nh 4 4 σ ND Factoral N p 1 36 h 4 D 3 σ Nh 18 6 σ ND 3 Table 1: Overvew of the umber of evaluato a the error for both fte fferece a DoE cheme, a the optmal tep ze uch that the total error mmze. Forwar FD Cetral FD Replcate CFD VAR( error VAR( error +error opt. h v h 4 [(M 4 σ4 +M 4 +3σ 4 ] 8h 4 (M 4 +σ4 8h 4 K 3 [M 4+(4K 3σ 4 ] h 4 [(M 4 σ4 +M 4 +3σ 4 ]+σ D 8h 4 (M 4 +σ h σ D 3 6 9(M 4 +σ 4 σ D 3 8h 4 K 3 [M 4+(4K 3σ 4 ]+ 18K 1 h σ D3 6 9(M 4 +(4K 3σ 4 K σ D 3 Plackett-Burma 4 N 3 h 4 (M 4 +( N 3σ4 4 N 3 h 4 (M 4 +( N 3σ4 + 3 σ D N Factoral 4 N 3 h 4 (M 4 +( N 3σ4 4 N 3 h 4 (M 4 +( N 3σ N 3 h σ D (M 4 +( N 3σ4 N σ D 3 Table : Overvew of the varace of the error vector for both fte fferece a DoE cheme a the optmal tep ze to mmze the varace. From the lterature we kow that CFD gve a much lower etermtc error tha FFD. Cocerg the tochatc error we ee from the table that the CFD cheme four tme better tha FFD. However, the umber of evaluato two tme more. To ave evaluato, we ca ue a Plackett-Burma eg: t umber of evaluato mlar to the FFD cheme, but the tochatc error two tme lower; the etermtc error, however, tme hgher. Full or fractoal factoral eg have a much

20 18 lower etermtc error tha Plackett-Burma cheme. The tochatc error mlar, but ce the umber of evaluato hgher tha for a Plackett-Burma cheme the tochatc error ca be mae much lower by creag N. However th reult more evaluato. Oberve alo that the etermtc error for Plackett-Burma a factoral cheme are epeet of the umber of evaluato, N. For the factoral cheme th alo mea that we ca ecreae the tochatc error by creag N, wthout affectg the etermtc error. Cocerg the varace of the tochatc error t appear that CFD, Plackett-Burma a factoral cheme are much better tha FFD. Whe comparg RCFD a factoral cheme t appear that the reult are mlar, ceforagoocomparowehavetotaken K. Note, however, that the cae of umercal oe, e.g. may etermtc mulato, RCFD ot applcable, ce replcate wll lea to the ame outcome. For uch cae factoral cheme are ueful. I Table we have lte the varace of the tochatc error a the total error. Note that the calculato for the optmal tep ze h e Table 1 the varace of the error are ot take to accout. Oe ca alo eterme a fferet tep ze by e.g. mmzg the expecte error plu a certa umber tme the taar evato. It ca ealy be verfe that th wll creae the optmal tep ze h. I the lat colum of Table we have calculate the optmal tep ze uch that the total varace mmze. Th calculato ot poble for FFD a Plackett-Burma ce thoe varace are ecreag fucto h. The optmal tepze h v for the other cheme reemble the correpog h e. Suppoe for example that all ε(x are taar ormal trbute, the t ca ealy be verfe that h v 6 h e 1.1h e, ce the M 4 3σ 4. Th mea that the tep ze h e whch mmze the total error equal approxmately the tep ze whch mmze the upper bou for the varace of the error. Th property a avatage of the cheme CFD, RCFD a FD above CFD a PB. I th paper we focu o the etmato of graet. However, ote that CFD, Plackett- Burma, a factoral cheme alo elver better etmato for the fucto value. Thee better etmato ca alo be valuable for NLP olver. Cocerg the amout of work eee to calculate the graet etmato, we emphaze that the etmato bae o the DoE cheme ee N ato/ubtracto a multplcato, whle FFD a CFD ee ato/ubtracto a multplcato a RCFD ee K ato/ubtracto a multplcato. So, the extra amout of work eee DOE cheme lmte

21 19 5 Cocluo I the prevou ecto we have cue everal metho for etmatg the graet of a fucto that ubject to... raom error. The error that we make whe etmatg the graet ca be plt to two part: a etermtc error a a tochatc error. The etermtc error are becaue we o ot oberve the fucto exactly at x, but the eghborhoo of x ug fte tep ze h. The tochatc error are becaue of the oe. We have erve upper bou for both the etermtc a tochatc error. Bae o thee upper bou we have cue the avatage a avatage of three fte fferece cheme a two DoE cheme. The cocluo that whe the uerlyg fucto ee oy the (fractoal or full factoral DoE cheme are ueful to reuce the tochatc error. Such cheme o ot vary the varable oe at a tme, but vary all varable multaeouly. The error for factoral cheme are exactly the ame a for replcate cetral fte fferece, but cae of umercal oe we ca ue factoral cheme whle replcate are meagle. Plackett-Burma cheme are ueful whe the evaluato are expeve. The tochatc error of thee cheme are two tme lower tha FFD, but the etermtc error hgher. Moreover, our error aaly cate how to chooe the tep ze h. Italo how that for CFD, RCFD a FD-cheme the tep ze whch mmze the total error, alo mmze the varace of the error. The DoE cheme ca be ealy clue the NLP olver to etmate graet. Ackowlegemet. We woul lke to thak our colleague Jack Kleje for h ueful remark o a earler vero of th paper, a Gul Gurka for provg u wth relevat lterature. Referece [1] Box, G.E.P., W.G. Huter, a J.S. Huter (1987, Stattc for Expermeter, Wley, New York. [] Blum, J.R. (1954, Multmeoal Stochatc Approxmato Metho, Aal of Mathematcal Stattc 5, [3] Dxo, L.C.W. (1994, O Automatc Dfferetato a Cotuou Optmzato, NATO Avace Stuy Ittute Sere 434,

22 0 [4] Doohue, J.M., E.C. Houck a R.H. Myer (1993, Smulato Deg for Cotrollg Seco-Orer Ba Frt-Orer Repoe Surface, Operato Reearch 41, [5] Doohue, J.M., E.C. Houck a R.H. Myer (1995, Smulato Deg for the Etmato of Quaratc Repoe Surface Graet the Preece of Moel Mpecfcato, Maagemet Scece 41 (, [6] Ermolev, Y. (1980 Stochatc Quagraet Metho, : Y. Ermolev a R.J- B. Wet, e., Numercal Techque for Stochatc Optmzato, Sprger Verlag, Chapter 6. [7] Grewak, A. (1989 O Automatc Dfferetato, : M. Ir a K. Taabe, e., Mathematcal Programmg, KTK Scetfc Publher, Tokyo, [8] Kefer, J. a J. Wolfowtz (195, Stochatc Etmato of a Regreo Fucto, Aal of Mathematcal Stattc 3, [9] Motgomery, D.C. (1984, Deg a Aaly of Expermet, eto, Wley, New York. [10] Zaza, M.A. a R. Sur (1993 Covergece Rate of Fte-Dfferece Setvty Etmate for Stochatc Sytem, Operato reearch 41 (4, Appex: DoE cheme I Table 3, four evaluato cheme are gve for 4. Note that for Plackett-Burma we have N 8, whch mea that 8 evaluato are eee. I th cae the umber of evalato for Plackett-Burma the ame a for CFD; geeral, however, the umber of evaluato eee by CFD more. Moreover, t eay to verfy that the orthogoalty property hol for th pecfc full factoral a Plackett-Burma cheme. I fact, Plackett-Burma cheme were evelope to reuce the umber of evaluato, but uch that the orthogoalty property tll hol. There o ee for tabulatg the DoE cheme, ce there a mple proceure for geeratg uch cheme.

23 1 FFD CFD Plackett-Burma Full factoral x 1 x x 3 x 4 x 1 x x 3 x 4 x 1 x x 3 x 4 x 1 x x 3 x Table 3: Evaluato cheme for 4factor.