On Reaching Nirvana (a.k.a. on Reaching a Steady State) Moshe Pollak Joint work (in progress) with Tom Hope

Transcription

1 On Reaching Nirvana (a.k.a. on Reaching a Steady State) Moshe Pollak Joint work (in progress) with Tom Hope 1

2 What is Nirvana? "ביקש יעקב אבינו לישב בשלוה" רש"י, בראשית לז 2 the Pursuit of Happiness (US Declaration of Independence) The Big Mathematical Question: 2

3 and the crooked shall become level WHEN????? Isaiah

4 4 \ / Warm-up period? Where does it end?

5 Terms Warm-up Start-up Burn-in Transient state 5

6 Hoad 2008: 46 methods 1 Simple Time Series Inspection Graphical 48 2 Ensemble (Batch) Average Plots Graphical 51 3 Cumulative-Mean Rule Graphical 48, 35, 32, 16, 57, 6, 51, 37, 4, 45 4 Deleting-The-Cumulative-Mean Rule Graphical 57, 6 5 CUSUM Plots Graphical 16 6 Welch's Method Graphical 34, 7, 53, 52, 51, 36, 4, 1, 45 7 Variance Plots (or Gordon Rule) Graphical 48, 35, 32, 7 8 Statistical Process Control Method (SPC) Graphical 52, 1, 8 9 Ensemble (Batch) Average Plots with Schribner's Rule Heuristic 35, 13, 7 10 Conway Rule or Forward Data-Interval Rule Heuristic 40, 13, 32, 35, 50, 7, 23, 5, 1, Modified Conway Rule or Backward Data-Interval Rule Heuristic 35, 32, 5, Crossing-Of-The-Mean Rule Heuristic 35, 32, 13, 7, 5, 58, 1 13 Autocorrelation Estimator Rule Heuristic 24, 35, 7 14 Marginal Confidence Rule or Marginal Standard Error Rules (MSER) Heuristic 5, 28, Marginal Standard Error Rule m, (e.g. m=5, MSER-5) Heuristic 28, 1, Goodness-Of-Fit Test Statistical 7 17 Relaxation Heuristics Heuristic 46, 7, 57, 6, Kelton and Law Regression Method Statistical 11, 34, 46, 7, 57, 6, 47, 52, Randomisation Tests For Initialisation Bias Statistical 23, 1 20 Schruben's Maximum Test (STS) Initialisation Bias Tests 20, 34, 18, 23, 22, Schruben's Modified Test Initialisation Bias Tests 16, 34, 28, Optimal Test (Brownian Bridge Process) Initialisation Bias Tests 22 18, 46, 7, 31, Rank Test Initialisation Bias Tests 14, 31, Batch Means Based Tests - Max Test Initialisation Bias Tests 33, 42, 43, 52, Batch Means Based Tests - Batch Means Test Initialisation Bias Tests 33, 43, 22, 28, Batch Means Based Tests - Area Test Initialisation Bias Tests 33, 43, 22, Pawlikowski's Sequential Method Hybrid 7 28 Scale Invariant Truncation Point Method (SIT) Hybrid Exponentially Weighted Moving Average Control Charts Graphical 9 30 Algorithm for a Static Dataset (ASD) Statistical 4 31 Algorithm for a Dynamic Dataset (ADD) Statistical 4 34 Telephone Network Rule Heuristic Ockerman & Goldsman Students t-tests Method Initialisation Bias Tests Ockerman & Goldsman (t-test) Compound Tests Initialisation Bias Tests Glynn & Iglehart Bias Deletion Rule Statistical Wavelet-based spectral method (WASSP) Statistical 39, 54, Queueing approximations method (MSEASVT) Statistical Chaos Theory Methods (methods M1 and M2) Statistical Beck's Approach for Cyclic output Heuristic Tocher's Cycle Rule Heuristic 7 43 Kimbler's Double exponential smoothing method Heuristic Kalman Filter method Statistical 47, Euclidean Distance (ED) Method Heuristic M eth o d Ty pe Paper Refs ID 46 Neural Networks (NN) Method Heuristic 58 6

7 State of the Art "Analytically tractable models that adequately capture steady-state response also may be unavailable, or at least difficult to determine with any degree of confidence or fidelity" (White and Robinson, 2010). Special Case Glynn and Zhang (2010): Renewal Queues 7

8 A Simple Case X 1,X 2, ~ N(δ i,1) independent δ i δ Mathematically, Nirvana is never reached We have to make our compromise with life 8

9 The Compromise There exists ε>0 such that if we are within ε of δ we pretend to be in steady-state 9

10 A Model In warm-up: X 1,X 2, ~ N(δ i,1), δ i, δ i δ-ε In steady-state: X 1,X 2, ~ N(δ,1) X 1,X 2,,X ν-1 in warm-up period X ν, X ν+1, in steady-state 10

11 Approach: sequential test of hypotheses T = stopping time, at which we hope that we have already reached steady-state H 0 : at time T we are not in steady-state H 1 : we are 11

12 If δ i, δ were known Λ k n = likelihood ratio of ν=k vs. ν= Π i=1,,k-1 exp(-½(x i -δ i ) 2 Π i=k,,n exp(-½(x i -δ) 2 = Π i=1,,n exp(-½(x i -δ i ) 2 = exp(σ i=k,,n (δ-δ i )X i -½ Σ i=k,,n (δ 2- δ i2 )) = exp(σ i=k,,n (δ-δ i )(X i -½ (δ+δ i )) 12

13 A comment In some applications, δ i are assumed to have a parametric form: δ i = δ / [1+e η-γi ] In this case, estimating η,γ will yield an estimate of δ i, to be substituted for δ i 13

14 Alternatively Represent warm-up parameters δ i by δ-ε Resulting likelihood ratios are Λ n k = exp(ε [Σ i=k,,n (X i -½(2δ-ε)) ] ) 14

15 Estimation: MLE ν n = argmax 1 k n Λ k n Stopping time: CUSUM T A = min{n max 1 k n Λ n k A} = min{n max 1 k n εσ i=k,,n (X i -½(2δ-ε)) log A} = min{n max 1 k n Σ i=k,,n (X i -½(2δ-ε)) (log A)/ε} 15

16 Confidence Proposal: measure confidence of being in steady-state at time T by P steady-state (ν T will forever remain the MLE of ν I T) (if we were to continue taking observations) 16

17 Recall: T A = min{n max 1 k n Σ i=k,,n (X i -½(2δ-ε)) (log A)/ε} log A ε Σ i=1,,n (X i -½(2δ-ε)) n CUSUM goes by cycles ν T will not remain forever log Λ T+m will go below 0 for some m ν T for some m Λ T Λ T+m 1 1 /Λ T+m Λ T A ν T ν T ν T ν T 17

18 Meaning of 1 /Λ T+m Λ T A ν T ν T 1 /Λ T+m is the likelihood ratio of X T+1,,X T+m for ν= vs. ν T ν T When ν T, the probability that this likelihood ratio will ever exceed A is less than 1 / A Therefore, if we were to continue taking observations, P steady-state ( ν T will forever remain the MLE of ν I T) 1-1 / A Hence, if we set A= 1 / β, we can guarantee with confidence 1-β that we have reached steady-state Recall: H 0 : at time T we are not in steady-state H 1 : we are Note: the power of the test is P steady-state ( ν T will forever remain the MLE of ν I T) = P ν T (likelihood ratio sequence of future observations will never exceed A I T) 1-1 / A 18

19 Type I error Definition: Let T be a stopping time G(t)= sup{ P(T t) P is a pre-steady-state probability on {X 1,X 2, } } Set t so that G(t) = α The p-value associated with stopping before having reached steady-state is G(T) 19

20 Recall: if δ i = δ-ε Λ k n = exp(σ i=k,,n (δ-δ i )X i -½Σ i=k,,n (δ 2- δ i2 )) = exp(εσ i=k,,n X i - ½( δ 2 -(δ-ε) 2 )(n-k+1) ) = exp(ε(σ i=k,,n X i (δ-½ε)(n-k+1) ) ) T β = min{n max 1 k n Λ k n 1 / β } = min{n max 1 k n {Σ i=k,,n (X i -δ i )-Σ i=k,,n (δ-δ i )+½ε log( 1 / β )/ε} min{n max 1 k n {Σ i=k,,n ( Z i )-Σ i=k,,n ( ε )+½ε log( 1 / β )/ε} T β is stochastically smallest when δ i = δ-ε p-value = G β (T β ) where G β is the cdf of T β G β (T β ) 1 exp( T β / ARL2FA ) ARL2FA 2 / ε 2 ( 1 / β + log β -1 ) + o(1) 20

21 DELTA KNOWN * Gamma = 0.1 ; Eta = 1; delta = 50 * Epsilon = 0.1; Beta = 1/100 * Nu = 73; nu_hat = 81; Stopping Time= 207 * Epsilon at stopping time = 1.4*10-7

22 If δ is unknown Define Y i = X 2i X 2i-1, Y n t(ranspose) =(Y 1,,Y n ) Here exp{ -½ (Y n -µ (k) n ) t Σ -1 (Y n -µ (k) n ) } Λ n k = exp{ -½ (Y n -µ ( ) n ) t Σ -1 (Y n -µ ( ) n ) } where (µ n (k) ) t = (µ 1,,µ k-1,0,,0) are the expectations of Y i 's (µ n ( ) ) t = (µ 1,.,µ n ) Assumption: µ i 0 as i Representing µ i by ε (or by anything non-anticipating), here, too, P steady-state (Λ n 1 for some n>t A I T ) 1/A ν A T 22

23 Type I error Clearly Π k m n f ν=k (Y m Y m-1,, Y 1 ) Λ n k = Π k m n f ν= (Y m Y m-1,, Y 1 ) Can show for m k : log ( f ν=k (Y m Y m-1,, Y 1 ) / f ν= (Y m Y m-1,, Y 1 ) ) = ε[k(k-1) m(m+1)]{z m +( m-1 / m,, 1 / m )Z - ε[k(k-1)+m(m+1)]/(4m)}/(2(m+1)) + ε[k(k-1) m(m+1)] (µ m +m -1 Σ 1 i m-1 iµ i ) /(2(m+1)) This is decreasing (separately) in each µ i T β is stochastically smallest when µ i ε p-value = G β (T β ) where G β is the cdf of T β 23

24 DELTA UNKNOWN * Same sequence as in previous slide * Epsilon = 0.01; Beta = 1/100 * Nu = 73; nu_hat = 358; Stopping Time = 422 * Epsilon at stopping time

25 Non-normal data Suppose X i ~fθ i in the warm-up period and X i ~fθ 0 in steady-state where X i are stochastically increasing. If θ 0 is known, a procedure analogous to the case of normally distributed data can be constructed. If θ 0 is not known, a procedure analogous to the case of normally distributed data cannot be easily constructed. A sometimes solution: define Y i =h(x 2i )-h(x 2i-1 ) for a suitably chosen h such that Y i are stochastically decreasing. In steady-state, EY i =0. Proceed as in the case of known steady-state (based on {-Y i }). An alternative is to go non-parametric. 25

26 Nonparametrics Y i ~f i in warm-up and Y i ~f 0 in steady-state, such that in warm-up Y i are stochastically decreasing. f i and f 0 are unknown, though f 0 is symmetric about 0. Define: σ i =1(Y i >0) r in =(Σ m=1,,n 1(Y m Y i )) Z n =((r 11,σ 1 ), (r 22,σ 2 ),, (r nn,σ n )) We need: a likelihood ratio for Z n 26

27 {Z n } is invariant with respect to transformations of Y i that leave σ i and r n i intact Let F DE be the cdf of the double exponential dist and let F 0 be the cdf of the steady-state dist F -1 DE (F 0 (Y i )) is such a transformation of Y i w.l.g. assume that in steady-state X i ~double exp Represent (transformed) warm-up distributions by f(x)=p a exp(-ax)1(x>0)+(1-p) b exp(bx)1(x<0) where p>½, a<1<b 27

28 Lemma (Savage, 1956) Let X 1, X 2,, X n ~ iid Exponential(1) and let u 1, u 2,, u n be positive constants. Then P( X 1 /u 1 < X 2 /u2 < < X n /un ) = Π i=1,,n (u i /Σ m=i,,n u m ) 28

29 Example Consider: Z 5 = ( (1,1), (1,0), (2,0), (4,1), (4,1) ) This is equivalent to Y 2 < Y 3 < 0 < Y 1 < Y 5 < Y 4 If ν=3: then Y 1 ~Exp(a), -Y 2 ~Exp(b), -Y 3,Y 4,Y 5 ~Exp(1) P ν=3 (Y 2 <Y 3 <0<Y 1 <Y 5 <Y 4 ) = [ P ν=3 (-Y 2 >-Y 3 >0 Y 2,Y 3 <0<Y 1,Y 5,Y 4 ) P(0<Y 1 <Y 5 <Y 4 Y 2,Y 3 <0<Y 1,Y 5,Y 4 ) ] P(Y 2,Y 3 <0<Y 1,Y 5,Y 4 ) = [P(Exp(1)/1 < Exp(1)/b) P(Exp(1)/a<Exp(1)/1 <Exp(1)/1)] (1-p)½p½½ 29

30 Type I error Can show that if in warm-up Y i are stochastically decreasing then T β is stochastically smallest when f i (x) p a exp(-ax)1(x>0)+(1-p) b exp(bx)1(x<0) p-value = G β (T β ) where G β is the cdf of T β 30

31 General stochastically decreasing processes Data: a sequence X 1, X 2, that is stochastically decreasing towards steady-state. Regard these as fixed. Do not observe them. After the n th data point get someone to sample a point Y n from X 1, X 2,, X n (uniformly), independently of past Y i 's. Apply a nonparametric procedure to {Y n }. 31

32 32