Recent Advances on Computer Experiment

Transcription

1 All Chinese Look Alike? Why? Recent Advances on Comuter Exeriment Dennis Lin The Pennsylvania State University February, 008 OR Seminar Penn State (US) criteria for eole classification (as in your driver license): Height Weight Hair Color Eye Color Short Light Black Black You must simulate under the correct (right subject/model). Where have all the Data gone? No need for data (Theoretical Develoment) Survey Samling and Design of Exeriment (Physical data collection) Comuter Simulation (Exeriment) Statistical Simulation (Random Number generation) Engineering Simulation Data from Internet On-line auction Search Engine Comuter Exeriment What is Comuter Simulation? What for? And How?

2 Comuter Exeriment Simulation A device that enables the oerator to reroduce under test conditions henomena likely to occur in actual erformance Webster Stochastic Deterministic Exensive (really exensive) Inexensive (really chea) Under H o is true What to Simulate??? Model function Resonse Outut Deendent Variable y = f (x, θ ) + ε Inut Predicted Variable Indeendent Variable Parameter Parameter Constant c, c, Error Noise y = f (x, θ ) + ε You Could Simulate y Simulate f Simulate x Simulate θ Simulate ε

3 What do you mean by simulation?? A sequence of oints that follows a secific desirable distribution π Simle π Comlicated π geometrical roerties Equal sacing (Uniform) orthogonal Indeendent? Conditional Indeendent? What to Simulate??? More y = f (x, θ ) + ε You could also Simulate y x, Simulate θ x, Simulate {u, u,, u m } Take them all, or use reject-accet strategy; Simulate u t u t-, etc Did you use the correct simulation??? y = f (x, θ ) + ε Statistics vs. Engineering Models 3

4 A Tyical Engineering Model (age of 3, in Liao and Wang, 995) y = f (x, θ ) + ε Statistical Model, f y=β 0 +Σβ i x i +Σβ ij x i x j +ε In this Talk Distribution Theory: Coin Examle Distribution Theory: R Story Distribution Theory: Significance Bootstraing MCMC (Markov Chain Monte Carlo) Random Number Generation Orthogonal Latin Hyercube Design and more Uniform Design Strategy for Comuter Exeriments Secial Toics If time ermits Bootstraing (Re-samling) Samling from the samles Treat Samles as Poulation Alternatively, delete some data from the samles Evaluate statistic from the current samle obtain one value Reeat this many times Average these statistics as the estimate MCMC (Markov Chain Monte Carlo) OR-seminar by Dr. Murali Haran (Nov 007) 4

5 Distribution Theory Statistical Hyothesis Testing Coin Examle R Story Significance in Regression Model H 0 : Null Hyothesis vs H : Alternative Hyothesis Statistical Hyothesis Testing Statistical Hyothesis Testing Does Not Prove anything, but Could be owerful to Disarove something (H 0 ). Under the Null Hyothesis H o (tyically imlies the white noise), what will the test statistics behave? what is tyical and what is abnormal?. Now, comare your observed test statistic to the distribution in () If it is tyical accet Null Hyothesis (H 0 ) If it is abnormal Reject H o 5

6 Coin Examle Throw a fair coin 00 times and the results were recorded: One sequence is real and the other one is fake. (A) (B) Potential Indexes/Statistics Number of Heads ( s) Number of Sign Changes Maximal Length Comuter Simulation Assumtion: For a fair coin (50-50) Do the followings Toss the coin 00 times From this sequence, evaluate its statistic (say, number of s) Now, reeat this rocess for 000 times (say), and you will have 000 statistics. Put these statistics in a histogram (this is the distribution of such a statistic) 6

7 Number of s in 000 Simulations (97 vs 09) Frequency Histogram of Sucess Normal A B Mean 00.4 StDev 7.35 N 000 Numbers of Change in 000 Simulations Frequency A B Sucess 0 Number of changes Max-Length in 000 Simulations B A Frequency Regression Model: JMP Demo Significance R Longest Run 7

8 Statistical Simulation Research Random Number Generators Deng and Lin (997, 00, 007) Robustness of transformation (Sensitivity Analysis) From Uniform random numbers to other distributions Goodness of Random Number Generators Period Length Efficiency Portability Theoretical Justification: Uniformity Indeendence Emirical Performance Small & Big Crash Tests LCG: Linear Congruential Generator Classical Random Number Generators X t =(B X t- + A) mod m Length=m Lehmer (95); Knuth (98) With roer choice of A & B Length=m= 3 -= (=.x0 9 ) Deng & Lin (000) The American Statistician 8

9 Deendence: y t vs y t- Deendence: y t vs y t- Range=(0.70, 0.7) Range=(0.700, 0.70) Briefings & Udate We have found a system of random number generators breaking the current world record. (Recall = 3 - is about 0 9 ) Old world record: MT9937 (998) Period length = New record with = 3 -: DX-597 [Deng, 005] Period length: Longest Period found so far: Deng and Lin (007) A Penn State Patent Period= Survived from all (Small & Big Crash) Tests Normal Random Numbers: Examles Central Limit Theorem X i ~iid U(0,) Z=ΣX i -6 Box-Muller Transformation X i ~ ind U(0,), i= & Z = Z = Rejection Polar Method 9

10 Other Aroaches Box-Muller Transformation Kinderman and Ramage (976) Triangular Accetance/Rejection Method Traezoidal Method (Ahrens, 977) Ratio of Uniform (Kinderman & Monahan, 976) Rectangle/Wedge/Tail Method (Marsaglia, Maclaren & Bray, 964) Goals Comuter Exeriment Engineering Comuter Exeriments Mostly deterministic Many inut variables Time consuming Confirmation Sensitivity Analysis Emirical Model Building Otimization Model Validation High Dimension Integration 0

11 Verification Validation Sace Filling Design How to (otimally) ut n oints in d dimensional sace? Otimal=cover as much sace as ossible Mother Nature Physical Exeriments Comuter Simulative Model (Exensive) (Aroximate Model: ANN) Simlified Comuter Model (Inexensive) Inference A Structured Roadma for Verification and Validation-- Highlighting the Critical Role of Exeriment Design Comuter Exeriment Exensive simulation James J. Filliben National Institute of Standards and Technology Information Technology Division Statistical Engineering Division 004 Worksho on Verification & Validation of Comuter Models of High-Consequence Engineering Systems NIST Administration Building Lecture Room D 3:0-3:5, November 8, 004 When Monte Carlo study is infeasible, how to run simulation? Latin Hyercube

12 Irrelevant Issues What is a Latin Hyercube? Relicates Blocking Randomization Question: How can a comuter exeriment be run in an efficient manner? Lin (997) Why Latin Hyercube Designs? Relication is worthless in CEs Factor levels are easily changed in CEs (not so in PEs) Suose certain terms have little influence Factorial designs roduce relication when terms droed Can estimate high-order terms for other factors Provides seudo-randomness since CEs are deterministic Smaller variance than random samling or stratified random samling (McKay, Beckman, and Conover (979) A secial class of LHC x x τ τ τ 3 τ 4... τ 6 τ i : ermutation of {,, 6} 6! n! for size n & (n!) d- for d-dim

13 Bayesian Designs Maximin Distance Designs, Johnson, Moore,and Ylvisaker (990) Maximizes the Minimum Interoint Distance (MID) Moves design oints as far aart as ossible in design sace MID = min d( x, x) x, x D D* is a Maximin Distance Design if MID = min d( x, x) = max min d( x, x) x, x D* D x, x D Rotated Factorial Designs Comuter exeriments are gaining in oularity One main research area of the next 0 years Rotated factorial designs good factorial design roerties (orthogonality and structure) good Latin hyercube roerties (unique and equally-saced rojections) easy to construct comarable by Bayesian criteria very suitable for comuter exeriments Lin (997) 3

14 4 d M M M M M M M M d d 4 3 υ υ υ υ V X D = desirable design factorial design rotated matrix [ ] = = v v V For d = = )* ( ) * ( c c c c c V V V V V c c For d = c where the oerator ( )* works on any matrix with an even number of rows by multilying the entries in the to half of the matrix by - and leaving those in the bottom half unchanged.

15 Rotation Theorem for Mixed Level Design Rotated Factorial with other n ( k ) Points d = d = 4 d = 8 d = c R = q q + q + qrs q + q + qrs R = qrs + rs + qrs qrs rs + rs + qrs q + q + q r + q + q r qr rs + rs + q r qr + rs + q r + q + qrs qrs qrs q + q + qrs + rs + qrs rs + rs + qrs qr q + q + q r qr + q + q r rs + rs + q r + rs + q r Beattie & Lin (004) Beam Examle 5

16 Some Comments Comuter exeriments are gaining in oularity main research area of the next 0 years Rotated factorial designs good factorial design roerties (orthogonality and structure) good Latin hyercube roerties (unique and equally-saced rojections) easy to construct comarable by Bayesian criteria very suitable for comuter exeriments Extensions Tye U and Tye E designs extension to sizes other than higher dimensional extension romising D desirable design = X M M M M M M M M V factorial design d rotated matrix υ υ Basic Idea- υ3 υ 4 d d Beattie & Lin (998): Rotating Full Factorials D = X V Basic Idea- Basic Idea-3 desirable design Two-level fractional factorial design rotated matrix S S O S i O S m Bursztyn & Steinberg (00): Rotating in Grous n n Now, Put these two ideas together! Grouing all design columns into grous, each forms a full factorial design, then rotate each grou (in block). 6

17 Steinberg and Lin (006) A Construction Method for Orthogonal Latin Hyercube Designs (with -level) Pang, Liu and Lin (007) Uniform Design Uniform Designs Fang, Lin, Winker & Yang (Technometrics, 999) Fang and Lin (Handbook of Statistics, Vol, 003) A uniform design rovides uniformly scatter design oints in the exerimental domain. htt:// 7

18 Uniform Design Fˆ n ( x) = Emirical Cumulative Distribution Function F(x) = Uniform Cumulative Distribution Function Find x = ( x, x,..., xn ) such that Fˆ n ( x) is closest to F(x). Discreancy Wang & Fang (980) D = Fˆ n( x) F( x) Ω dx The centered L -discreancy is invariant under exchanging coordinates from x to -x. Esecially, the centered L -discreancy, denoted by CL, has the following comutation formula: ( CL (P )) s 3 n s = + xki xki n k = i = n n s + + xki + x ji xki x ji. n k = j = i = Recent Research on Samling Strategies for Comuter Exeriments: Design and Analysis Simson, T.W., Lin, D.K.J., and Chen, W. (00) Obtaining information which are not ossible, without modern technology Censor RFID Simulation How to (otimally) design these devices? How to analyze the outcomes (data)? 8

19 After all, simulation means not real Good for descrition, But Not necessary good for a solid roof! There are many tyes of simulations, they must be used with care! This talk is based on Deng and Lin (007) Patent on Random Number Generation. Steinberg and Lin (006) A Construction Method for Orthogonal Latin Hyercube Designs, Biometrika, 93, Fang and Lin (007) Uniform Design in Comuter and Physical Exeriments, The Grammar of Technology Develoment, ed. Shu Yamada, <htt:// STILL QUESTION? Send $500 to Dennis Lin University Distinguished Professor 483 Business Building Deartment of Suly Chain & Information Systems Penn State University (hone) (fax) DKL5@su.edu (Customer Satisfaction or your money back!) 9