Naonal Insue of Sandards and Technology (NIST) Informaon Technology Laboraory (ITL) Sascal Engneerng Dvson (SED) Robusness Expermens wh Two Varance Componens by Ana Ivelsse Avlés avles@ns.gov Conference on New Drecons n Expermen Desgn, Chcago May 6, 003 Jon work wh ruce E. Ankenman and José C. Pnhero.
OUTLINE I. Inroducon» Key Conceps II. Assembled Desgns» Noaon and Parameers» D-Opmaly III. Dsperson Effecs Esmaon (ML)» Heursc Algorhm» Analyss IV. Conclusons
I. Inroducon 3
Robus Expermenal Desgn Generally, each produc or process performance characersc wll have a arge. The objecve s o reduce he varably around hs arge. Knowledge of boh effecs of mean and effecs on varance allows he engneer o fnd condons, whch mnmze he varaon as well as se he average performance on arge. I 4
Illusrave Example Pharmaceucal Sudy Expermenal Objecve: o deermne he facor sengs so ha he able wegh s on arge and he varably of he able wegh from bach-o-bach and from sample-o-sample s mnmzed. Facor Level Level SPEED - PRETAMP - COMPRESS - Lamb, oos, and rowne (Technomercs, 996) I 5
Key Conceps CROSSED FACTORS ndependenly se Modeled as fxed effecs NESTED FACTORS dependen; effec changes a dfferen sengs of hgher level facor *RANDOM NESTED FACTORS E.g., sample whn bach *VARIANCE COMPONENTS he varance of he random effecs assocaed wh nesed facors. E.g., ach-o-ach varaon and Sample-o-Sample varaon. I 6
Robusness Expermens Crossed Facors Locaon Effecs (effecs on he response mean) Dsperson Effecs (effecs on he response varance) I 7
GOAL of Research To provde Expermenal Desgns Procedures and Analyss Technques for esmang: locaon effecs of crossed facors varance componens assocaed wh nesed facors dsperson effecs ha crossed facors may have on he varance componens. 8
Desgns for Example We need o learn abou he effecs of SPEED, PRETAMP, and COMPRESS n able wegh and abou he varably of able wegh from ATCH-TO-ATCH and from SAMPLE-TO-SAMPLE. Use Crossed Facor Desgns (e.g. FF, CCD) for LOCATION EFFECTS: Use Herarchcal Nesed Desgns (HND) for VARIANCE COMPONENTS: ach ach ach 3 Sample Sample Sample 3 Sample Sample Sample 3 Sample Sample Sample 3 I 9
II. Assembled Desgns 0
II r 8 s An Assembled Desgn A hybrd desgn ha places a herarchcal nesed desgn (HND) a each desgn pon n a crossed facor desgn. n n 6 4 3 Parameers: q COMPRESS - - SPEED - PRETAMP r # of desgn pons n he crossed facor desgn s # of dfferen HND s (srucures) used n he desgn n j # of observaons n he j h srucure j # baches n he j h srucure q # of varance componens (equals # of levels of nesng)
Noaon for Herarchcal Nesed Desgns (HNDs) Nesed Facors: bach number and sample number COMPRESS - PRETAMP - SPEED - Srucure ach ach ach 3 ach 4 (,,,) Sample Sample Sample Sample Sample Sample Srucure ach ach ach 3 (,,) Sample Sample Sample Sample Sample Sample II
s j Noaon for Assembled Desgns srucure j@{desgn pons wh srucure Mus have desgn pons ordered n some way. j} II Example: 3 desgn, Splng Generaor AC, r8, s, 4, 3, n6 (ASPEED, PRETAMP, and CCOMPRESS) Desgn Pon A C AC Srucure - - - - (,,,) - - (,,) 3 - - (,,) 4 - - (,,,) 5 - - (,,) 6 - - (,,,) 7 - - (,,,) 8 (,,) (,,,)@{,4,6,7} (,,)@{,3,5,8} 3
Mos alanced HND gven number of baches () and number of samples (n) The mos balanced HND, gven n and, s obaned when he dfference n he number of samples among baches s a mos one. n5; 3 (3,,) (,,) Algorhm o Generae Mos alanced HND: m * n /,, n / n /, n, / C n n / F C where C s he number of celngs and F s he number of floors. II For example: n C F n 5 and 3; n/ 5/3.667 C n / 5.667 3 ( ) (,, ).667,.667,.667 3 5 3 4
II Mos alanced Assembled Desgn gven n j and j, j,,s The mos balanced Assembled Desgn s found by placng he mos balanced HND, gven n j and j, a each desgn pon of he crossed facor desgn. If here are NO Dsperson Effecs, hen gven an allocaon for he degrees of freedom MOST ALANCED D-OPTIMAL for esmang Locaon Effecs and Varance Componens (Maxmum Lkelhood) as long as` σ σ τ j j n j j OR n and j < 3C j Ankenman, Avlés, and Pnhero (Sasca Snca, 003) 5
III. Dsperson Effecs Esmaon 6
III Model for Robusness Expermens Wolfnger and Tobas (Technomercs, 998) Locaon effecs y Xβ y s a vecor of observaons. q X s he locaon-effecs desgn marx. Dsperson effecs Z C C represens any dsperson effecs ha he crossed facors have on he h varance componen. β s a vecor of r unknown coeffcen (ncludng he consan erm). Z s an ndcaor marx assocaed wh he h varance componen. u s a vecor of normally dsrbued ndependen random effec assocaed wh he h varance componen u ~ N( 0, Iσ ). u Random effecs 7
Srucure of Dsperson-Effec Model γ j The dsperson-effec parameer on he h varance componen by he j h regressor. Vecor of Dsperson-effec parameers T C ( ) Z Z Z X [ exp( T )] dag γ Desgn Marx for Dsperson Effecs (Depends on he facor sengs) γ γ γ γ γ 3 r III 8
9 Informaon Marx (ML esmaes) Ankenman, Avlés, and Pnhero (ASA-Q&P Secon Proceedngs, 000) III ( ) ( ),,, ) (,,,, γ γ 0 0 β γ γ β σ σ σ σ Inf Inf Inf ( ) ( )( ) ( ) ( )( ) ( ) ( )( ) ( ) ( )( ) r r r r Inf ) (,,, a a V a a Z Z V a a Z Z V a a Z Z V γ γ σ σ ( ) ( ) γ D D D exp exp x γ x γ x a σ ( ) X X V β ) ( Inf ( ) ( ).,,,,,, r Inf Inf γ γ γ γ σ σ σ σ
III Near D-Opmaly by Emprcal Sudy D-creron-deermnan of he nformaon marx Expressons for he D-creron are analycally nracable when dsperson effecs are presen. V V Cov( y) Cov( y) q q σ Z σ Z Z dag (whou dsperson effecs) [ exp( Τ γ )] Z (wh dsperson effecs) However, he D-creron can be calculaed for a parcular desgn under a parcular rue model. σ ( ) ( ) r, τ, exp γ j, and exp γ j σ Thus, an emprcal sudy was run o fnd desgns ha are nearly D-opmal. 0
Emprcal Sudy Assume a crossed facor desgn. Assembled Desgns s and r r r/ Desgn udge (Only compare desgns of equal cos) alanced Desgns (Only compare mos balanced desgns) Grd of poenal varance raos: τ L τ H,.5,, 3, 4, 5, 6, 7, 8, 9,0,5,.5,, 3, 4, 5, 6, 7, 8, 9, 0, 5,30 a 0.5,.0,.5 τ L τ H σ σ exp a exp ( γ A γ A ) ( γ A γ A ) ( γ γ ) exp A A σ γ ( ) A A ln a γ σ τ H τ τ L H τ A ln L a γ σ III
Cos ass for Comparable Desgns Dffculy: Allocang degrees of freedom o each varance componen Whou dsperson effecs n he model, seemed reasonable o ask he praconer o choose he degrees of freedom for he esmaon of each varance componen. Wh dsperson effecs n he model, he praconer s unlkely o have accurae knowledge abou whch dsperson effecs are mporan. The praconer wll now only be requred o provde he number of desgn pons, r, and nformaon abou he cos of runnng he expermen. III
Cos Srucure M rφ r Φ s j r j j s j r j n j M expermen al budge cos o measure a sample Φ r cos o make anoher recpe cos o measure a sample Φ cos o make a bach cos o measure a sample M s he scaled expermenal cos Φ r s he recpe/sample cos rao Φ s he bach/sample cos rao r s he number of recpes (.e., desgn pons) j s he number of baches n srucure j n j s he number of samples n srucure j r j s he number of recpes ha conan srucure j III 3
4 Cos Srucure (s and r r r/) ( ) ( ) n n r r r M r Φ Φ Feasbly Condon ( ) ( ) n n n n r r r < < < < Sample/ach Observaon Rao III n n n m s j j s j j
Sample/ach Observaon Rao Plo mˆ ( ) Φ Opmal Sample/ach Observaon Rao * 6.00 5.00 4.56 4.00 3.00.00.00 0.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00.8.00.00 0.00 9.00 8.00 7.84 7.00 6.00 5.7 5.00 4.67 4.00 3.47 3.00.00.00 0.00 0.00 5.00 0.00 5.00 0.00 5.00 III Φ ach/sample Cos Rao 5
Heursc Algorhm for Fndng a Nearly D-Opmal Assembled Desgn for a Robusness Expermen STEP : Defne he cos srucure Process Facors: SPEED, PRETAMP, and COMPRESS r 3 8. udge: $5,000. Coss: $ per sample, $40 per bach, and $65 per recpe. M budge sample cos $5000 $ 500 Φ r recpe cos sample cos $65 $ 3.50 Φ bach cos sample cos $40 $ 0 III 6
Heursc Algorhm STEP : Oban he recommended sample/bach observaon rao ( ) mˆ * Φ 6.00 5.00 4.00 3.00.00.00 0.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00.00.00 0.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00.00.00 0.00 4.56 0.00 5.00 0.00 5.00 0.00 5.00 Φ ( ) m 0 4. 56 ˆ * Φ III For a bach/sample cos rao of 0, he recommended sample/bach observaon rao s 4.56. 7
Heursc Algorhm STEP 3: Defne se of feasble desgns III m s j s j n j j n n n n m 3 500 66.67 5 60 7.33 4 480 0.00 6 40 5.00 5 460 9.00 7 0.94 6 440 73.33 8 00. 7 40 60.00 9 80 9.47 8 400 50.00 0 60 8.00 9 380 4. 40 6.67 0 360 36.00 0 5.45 340 30.9 3 00 4.35 30 6.67 4 80 3.33 3 300 3.08 5 60.40 4 80 0.00 6 40.54 n n M Φ r Φ ( ) n n > 3 r or by S-PLUS funcon seeqcos m 8
Heursc Algorhm STEP 4: Fnd he (feasble) sample/bach observaon rao closes o he recommended sample/bach observaon rao. ˆ * m ( 0) 4. 56 Φ n n n n m 3 500 66.67 5 60 7.33 4 480 0.00 6 40 5.00 5 460 9.00 7 0.94 6 440 73.33 8 00. 7 40 60.00 9 80 9.47 8 400 50.00 0 60 8.00 9 380 4. 40 6.67 0 360 36.00 0 5.45 340 30.9 3 00 4.35 30 6.67 4 80 3.33 3 300 3.08 5 60.40 4 80 0.00 6 40.54 m III ( )* 3 ( )* n n 300 m * 3.08 9
Heursc Algorhm STEP 5: Consruc he Mos alanced Assembled Desgn (MAD) gven (n n )* and ( )* y S-PLUS funcon adde > adde(s3,ns300,cf3) [[]]: [[]]$sruc: [] 4 3 3 3 3 3 3 [[]]$runs: [] 4 6 7 [[]]: [[]]$sruc: [] 3 3 3 3 3 3 [[]]$runs: [] 3 5 8 Algorhm o Generae MAD: s odd, hen. If ( ). If ( ) and max{ n } Le R ( n n ) ( ) m If n : R, hen n m R and n m If. R >, hen n m and n m ( R ) s even, hen If ( ) n and mn{ n }. n : n s even, hen n n, else n n. 3. Defne he wo srucures of he mos balanced desgn (see slde 4). 4. Place each srucure a half of he desgn pons. > convername(adde(s3,ns300,cf3)) [] "(4,3,3,3,3,3,3)@{,4,6,7}(3,3,3,3,3,3)@{,3,5,8}" III 30
The Assembled Desgn (4,3,3,3,3,3,3)@{,4,6,7}(3,3,3,3,3,3)@{,3,5,8} Srucure Srucure Srucure Srucure COMPRESS - Srucure Srucure - SPEED Srucure PRETAMP - Srucure Srucure ach ach Sample Sample 4 Sample Sample 3 Srucure ach ach Sample Sample 3 Sample Sample 3 ach 7 Sample Sample 3 (4,3,3,3,3,3,3) ach 6 Sample Sample 3 (3,3,3,3,3,3) III 3
Analyss The Maxmum Lkelhood (ML) mehod s used o esmae he parameers. If all he locaon effecs and he man effecs of he k crossed facors on varance are of neres, hen here are rk parameers o esmae. The ML esmaes can be obaned wh he S-PLUS funcon ADMLes. > ADMLes(adadex, beabeaex,yyex) Log-lkelhood: -765.678 Parameer esmaes: bea bea bea3 bea4 bea5 bea6 bea7 bea8 9.8684 3.4577.87369-0.4333-3.45963-0.485979-0.64988 0.37 s s gamma. gamma. gamma.3 gamma. gamma. gamma.3 5.77887.068886.7365 0.0566436 0.004903.5877-0.079789-0.08338 Number of eraons: 87 III 3
ML Esmaes of Sgnfcan Effecs (ASPEED, PRETAMP, and CCOMPRESS) ˆ β I 9.9, ˆ β A 3. 5, ˆβ.8, ˆβ 3.4 A, ˆ σ 5.8, ˆ σ., γ., and. Effecs on Mean: ˆ A A effec on mean: 3.5 effec on mean:.8 A and neracon effec on Mean: -3.4 Sandard Devaon Rao s Effecs on Varance: ˆ A γ. ˆ σ ˆ σ 5.8. Scale rao for Facor A on bach-o-bach varaon: exp(.)3.0 Scale rao for Facor A on sample-o-sample varaon : exp(.)3.3.3 III 33
EXPERIMENTAL GOAL: Table wegh s on arge and varance s mnmzed. 0.5 0.08 0. 0.06 0.5 0. 0.04 0.5 0.05 0 0 30 40 0.08 0.0 0 0 30 40 0. 0.06 0.5 0. 0.05 0 0 0 30 40 0.04 0.0 0 0 30 40 COMPRESS 0.5 0. 0.08 0.06 III 0.5 0. - 0.5 0. 0.05 0 0 30 40-0.5 0. 0.05 0 0 30 40 SPEED 0.08 0.06 0.04 0.0 0.04 0.0 0 0 30 40 0 0 30 40 PRETAMP Se SPEED o low level and PRETAMP o hgh level. - 34
IV. Conclusons Assembled Desgns can be used for esmang locaon effecs, varance componens, and dsperson effecs. When here are wo levels of nesng, balancng he observaons among all he branches of he nesed desgn resuls n D-opmal desgns for ML esmaon excep n a few relavely uncommon suaons. If dsperson effecs are par of he model and here are wo levels of nesng, hen nearly D-opmal desgns for a gven budge are provded. Gudelnes and S-PLUS code for he desgn and analyss of robusness expermens are avalable upon reques (Emal: avles@ns.gov). IV Ths maeral s based upon work suppored under a Naonal Scence Foundaon Graduae Fellowshp and a Ford Foundaon Dsseraon Fellowshp. 35
Dscusson 36