Chapter Output Aalysis for a Sigle Model Baks, Carso, Nelso & Nicol Discrete-Evet System Simulatio
Error Estimatio If {,, } are ot statistically idepedet, the S / is a biased estimator of the true variace. V( ˆ ) Almost always the case whe {,, } is a sequece of output observatios from withi a sigle replicatio (autocorrelated sequece, time-series). Suppose the poit estimator q is the sample mea i i / Variace of is almost impossible to estimate. For system with steady state, produce a output process that is approximately covariace statioary (after passig the trasiet phase). The covariace betwee two radom variables i the time series depeds oly o the lag (the # of observatios betwee them).
3 Error Estimatio For a covariace statioary time series, {,, }: Lag-k autocovariace is: Lag-k autocorrelatio is: If a time series is covariace statioary, the the variace of is: The expected value of the variace estimator is: ), cov( ), cov( k i i k k k k ) ( k k k V / where ), ( c B BV S E c
Error Estimatio The expected value of the variace estimator is: E S BV ( ), where B / c ad V ( ) is the variace of If i are idepedet, the S / is a ubiased estimator of V ( ) If the autocorrelatio k are primarily positive, the S / is biased low as a estimator of V ( ). If the autocorrelatio k are primarily egative, the S / is biased high as a estimator of V ( ). 4
C.I. with Specified Precisio [Termiatig Simulatios] The half-legth H of a 00( a)% cofidece iterval for a mea q, based o the t distributio, is give by: S H t a /, R R R is the # of replicatios S is the sample variace Suppose that a error criterio e is specified with probability - a, a sufficietly large sample size should satisfy: P q e a.. 5
C.I. with Specified Precisio [Termiatig Simulatios] Assume that a iitial sample of size R 0 (idepedet) replicatios has bee observed. Obtai a iitial estimate S 0 of the populatio variace. The, choose sample size R such that R R 0 : Sice t a/, R- z a/, a iitial estimate of R: za / S0 R, za / is the stadard ormal distributio. e t /, 0 R is the smallest iteger satisfyig R R 0 ad a R S R e Collect R - R 0 additioal observatios. The 00(-a)% C.I. for q:.. t a /, R S R 6
C.I. with Specified Precisio [Termiatig Simulatios] Call Ceter Example: estimate the aget s utilizatio over the first hours of the workday. Iitial sample of size R 0 = 4 is take ad a iitial estimate of the populatio variace is S 0 = (0.07) = 0.0058. The error criterio is e= 0.04 ad cofidece coefficiet is -a = 0.95, hece, the fial sample size must be at least: 0 z.05s0.96 *0.0058 e 0.04 For the fial sample size:.4 R 3 4 5 t 0.05, R-.8.6.4 ta /, RS0 /e 5.39 5. 4.83 R = 5 is the smallest iteger satisfyig the error criterio, so R - R 0 = additioal replicatios are eeded. After obtaiig additioal outputs, half-width should be checked. 7
Output Aalysis for Steady-State Simulatio Cosider a sigle ru of a simulatio model to estimate a steady-state or log-ru characteristics of the system. The sigle ru produces observatios,,... (geerally the samples of a autocorrelated time series). Performace measure: q lim T i lim 0 T E E T, E for discrete measure Idepedet of the iitial coditios. i ( t) dt, for cotiuous measure (with probability ) (with probability ) 8
Output Aalysis for Steady-State Simulatio The sample size is a desig choice, with several cosideratios i mid: Ay bias i the poit estimator that is due to artificial or arbitrary iitial coditios (bias ca be severe if ru legth is too short). Desired precisio of the poit estimator. Budget costraits o computer resources. Notatio: the estimatio of q from a discrete-time output process. Oe replicatio (or ru), the output data:,, 3, With several replicatios, the output data for replicatio r: r, r, r3, 9
Iitializatio Bias Methods to reduce the poit-estimator bias caused by usig artificial ad urealistic iitial coditios: Itelliget iitializatio. Divide simulatio ito a iitializatio phase ad data-collectio phase. Itelliget iitializatio Iitialize the simulatio i a state that is more represetative of log-ru coditios. If the system exists, collect data o it ad use these data to specify more early typical iitial coditios. If the system ca be simplified eough to make it mathematically solvable, e.g. queueig models, solve the simplified model to fid log-ru expected or most likely coditios, use that to iitialize the simulatio. 0
Iitializatio Bias Divide each simulatio ito two phases: A iitializatio phase, from time 0 to time T 0. A data-collectio phase, from T 0 to the stoppig time T 0 +T E. The choice of T 0 is importat: After T 0, system should be more early represetative of steady-state behavior. System has reached steady state: the probability distributio of the system state is close to the steady-state probability distributio (bias of respose variable is egligible).
Iitializatio Bias M/G/ queueig example: A total of 0 idepedet replicatios were made. Each replicatio begiig i the empty ad idle state. Simulatio ru legth o each replicatio was T 0 +T E = 5,000 miutes. Respose variable: queue legth, L Q (t,r) (at time t of the rth replicatio). Batchig itervals of,000 miutes, batch meas Esemble averages: To idetify tred i the data due to iitializatio bias The average correspodig batch meas across replicatios:. j rj R r The preferred method to determie deletio poit. R R replicatios
Iitializatio Bias 3
Iitializatio Bias 4
Iitializatio Bias A plot of the esemble averages,..(, d), versus 000j, for j =,,,5. Illustrates the dowward bias of the iitial observatios. 5
Iitializatio Bias Cumulative average sample mea (after deletig d observatios):.. (, d). j d jd Not recommeded to determie the iitializatio phase. It is apparet that dowward bias is preset ad this bias ca be reduced by deletio of oe or more observatios. 6
Iitializatio Bias No widely accepted, objective ad prove techique to guide how much data to delete to reduce iitializatio bias to a egligible level. Plots ca, at times, be misleadig but they are still recommeded. Cumulative average becomes less variable as more data are averaged. The more correlatio preset, the loger it takes for approach steady state.. j to Differet performace measures could approach steady state at differet rates. 7