Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors
Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5 S 7 S S S 3 S 4 S 5 S 6 S 7 3 4 5 6 7 8 9 0 3 4 Average System me Let S k be the tme that customer k speds the queue, the, t Sˆ = S k = k A = Area uder x(t) IMPORA: hs s a Radom Varable Estmate of the average system tme over the frst customers Example: Sgle Server Queueg System x(t) 0 0 3 3 3 4 5 6 7 8 9 0 3 4 Probablty that x(t)= Let () be the total observed tme durg whch x(t)= Probablty Estmate Average queue legth ˆ Q ˆ = p () = () ˆ p () = = A = () = 0 Utlzato otal observato terval ( 0) ˆ ρ = () = = t
Parameter Estmato Let X,,X be depedet detcally dstrbuted radom varables wth mea θ ad varace σ. I geeral, θ ad σ are ukow determstc quattes whch we would lke to estmate. Sample Mea: Radom Varable! ˆ X = X E [ X ˆ ] = Var[ X ˆ ] = = X = = θ E = E[ X ] = = θ σ ( ) E Xˆ θ = E X θ = = = Τhe sample mea ca be used as a estmate of the ukow parameter θ. It has the same mea but less varace tha X. σ Estmator Propertes Ubasedess: A estmator θˆ s sad to be a ubased estmator of the parameter θ f t satsfes Bas: E θ ˆ θ = I geeral, a estmator s sad to be a based sce the followg holds E ˆ θ = θ + b where b s the bas of the estmator If X,,X are d wth mea θ, the the sample mea s a ubased estmator of θ.
Estmator Propertes Asymptotc Ubasedess: A estmator θˆ s sad to be a asymptotcally ubased f t satsfes E ˆ ad lm ˆ θ θ b E = + θ = θ Strog Cosstecy: A estmator s strogly cosstet f wth probablty lm E ˆ θ = θ If X,,X are d wth mea θ, the the sample mea s also strogly cosstet. Cosstecy of the Sample Mea he varace of the sample mea s σ Var[ Xˆ ] = f ( x) f ˆX Icreasg ˆX ( x) θ x But, σ s ukow, therefore we use the sample varace Also a Radom Varable! S ( X ˆ θ ) = = θ x
Recursve Form of Sample Mea ad Varace Let M ad S be the sample mea ad varace after the -th sample s observed. Also, let M 0 =S 0 =0. M X = = ( X ) M = he recursve form for geeratg M + ad S + s X + X M + = M + + M + = + X + M = M + S M + = S + + + S = ( )( M ) + Example: Let X be a sequece of d expoetally dstrbuted radom varables wth rate λ= 0.5 (sample.m). Iterval Estmato ad Cofdece Itervals Suppose that the estmator ˆ θ = θ the, the atural questo s how cofdet are we that the true parameter θ s wth the terval (θ -ε, θ +ε)? Recall the cetral lmt theorem ad let a ew radom varable ˆ θ ˆ E θ Z = var ˆ θ ˆ θ θ For the sample mea case Z = σ / he, the cdf of Z approaches the stadard ormal dstrbuto (0,) gve by x Φ ( x) = e π r / dr
Iterval Estmato ad Cofdece Itervals Let Z be a stadard ormal radom varable, the f Z (x) Area = -a Z a / Za / { } { } 0 Pr Z Z = Pr Z Z Z = a a/ a/ a/ hus, as creases, Z desty approaches the stadard ormal desty fucto, thus { } Pr Z Z Z a a/ a/ x Iterval Estmato ad Cofdece Itervals f Z (x) Substtutg for Z ˆ θ θ Pr Za/ Za/ a σ / 0 Z a / Za / { ˆ } θ ˆ a σ θ θ / a σ / Pr Z / + Z / a hus, for large, ths defes the terval where θ les wth probablty -a ad the followg quattes are eeded he sample mea he value of Z a/ whch ca be obtaed from tables gve a he varace of used. θˆ θˆ x whch s ukow ad so the sample varace s
Example Suppose that X,, X are d expoetally dstrbuted radom varables wth rate λ=. Estmate ther sample mea as well as the 95% cofdece terval. SOLUIO ˆ he sample mea s gve by θ = X = From the stadard ormal tables, a =0.05, mples z a/ Fally, the sample varace s gve by Sˆ ( ˆ ) = X θ = herefore, for large, { ˆ ˆ } ˆ ˆ θ S θ θ S Pr / + / 0.95 SampleIterval.m How Good s the Approxmato he stadard ormal (0,) approxmato s vald as log as s large eough, but how large s good eough? Alteratvely, the cofdece terval ca be evaluated based o the t-studet dstrbuto wth degrees of freedom A t-studet radom varable s obtaed by addg d Gaussa radom varables (Y ) each wth mea μ ad varace σ. = = Y σ μ
ermatg ad o-ermatg Smulato ermatg Smulato here s a specfc evet that determes whe the smulato wll termate E.g., processg M packets or Observg M evets, or smulate t tme uts,... Ital codtos are mportat! o-ermatg Smulato Iterested log term (steady-state) averages [ ] θ = lm E X k k ermatg Smulato Let X,,X M are data collected from a termatg smulato, e.g., the system tme a queue. X,,X M are O depedet sce X k =max{0, X k- -Y k }+Z k Y k, Z k are the kth terarrval ad servce tmes respectvely Defe a performace measure, say L = M X M = Ru smulatos to obta L,,L. Assumg depedet smulatos, the L,,L are depedet radom varables, thus we ca use the sample mea estmate M ˆ θ = L = X = = M =
Examples: ermatg Smulato Suppose that we are terested the average tme t wll take to process the frst 00 parts (gve some tal codto). Let 00, =,,M, deote the tme that the 00 th part s fshed durg the -th replcato, the the mea tme requred s gve by M ˆ L = 00, M = Suppose we are terested the fracto of customers that get delayed more tha mute betwee 9 ad 0 am at a certa AM mache. Let be the delay of the th customer durg the th replcato ad defe [D ]= f D >, 0 otherwse. he, L M = D > M = M ˆ L = D > 0 = M = o-ermatg Smulato Ay smulato wll termate at some pot m <, thus the tal traset (because we start from a specfc tal state) may cause some bas the smulato output. Replcato wth Deletos he suggesto here s to start the smulato ad let t ru for a whle wthout collectg ay statstcs. he reasog behd ths approach s that the smulato wll come closer to ts steady state ad as a result the collected data wll be more represetatve warm-up perod Data collecto perod 0 r m tme
o-ermatg Smulato Batch Meas Group the collected data to batches wth m samples each. Form the batch average m B = X = + ( ) m ake the average of all batches m B = B = X = m = = m+ ( ) For each batch, we ca also use the warm-up perods as before. o-ermatg Smulato Regeeratve Smulato Regeeratve process: It s a process that s characterzed by radom pots tme where the future of the process becomes depedet of ts past ( regeerates ) 0 Regeerato pots tme Regeerato pots dvde the sample path to tervals. Data from the same terval are grouped together We form the average over all such tervals. Example: Busy perods a sgle server queue detfy regeerato tervals (why?). I geeral, t s dffcult to fd such pots!
Emprcal Dstrbutos ad Bootstrappg Gve a set of measuremets X,,X whch are realzatos of d radom varables accordg to some ukow F X (x;θ), where θ s a parameter we would lke to estmate. We ca approxmate F X (x; θ) usg the data wth a pmf where all measuremets have equal probablty /. he approxmato becomes better as grows larger. Example Suppose we have the measuremets x,,x that came from a dstrbuto F X (x) wth ukow mea θ ad varace σ. We would lke to estmate θ usg the sample mea μ. Fd the Mea Square Error (MSE) of the estmator based o the emprcal data. / / x x x Emprcal dstrbuto X μ = xp = = = MSE ( ( ) ) e = E e g X μ x he emprcal mea s a ubased estmator of θ. Vector of RVs from the emprcal dstrbuto Based o emprcal dstrbuto
Example μ = x p = = = MSE ( ( ) ) e = E e g X μ = Ee X μ = = E e ( X μ ) = E X μ Var X = = = x X s a RV from the emprcal dstrbuto ( ) [ ] e e [ ] = ( ) = ( x μ ) Var X E X μ e e herefore MSE Var X [ ] e e = = = = ( x μ ) Compare ths wth the sample varace!