A. Distribution of the test statistic

Transcription

1 A. Distribution of the test statistic In the sequentia test, we first compute the test statistic from a mini-batch of size m. If a decision cannot be made with this statistic, we keep increasing the mini-batch size by m datapoints unti we reach a decision. This procedure is guaranteed to terminate as expained in Section 4. The parameter contros the probabiity of making an error in a singe test and not the compete sequentia test. As the statistics across mutipe tests are correated with each other, we shoud first obtain the joint distribution of these statistics in order to estimate the error of the compete sequentia test. Let j and s,j be the sampe mean and standard deviation respectivey, computed using the first j mini-batches. Notice that when the size of a mini-batch is arge enough, e.g. n>, the centra imit theorem appies, and aso s,j is an accurate estimate of the popuation standard deviation. Additionay, since the degrees of freedom is high, the t-statistic in Eqn. 5 reduces to a z-statistic. Therefore, it is reasonabe to make the foowing assumptions: Assumption. The joint distribution of the sequence (,,...) foows a mutivariate norma distribution. Assumption. s, where std({ i }) Fig. 7 shows that when µ µ the empirica margina distribution of t j (or z j ) is we fitted by both a standard student-t and a standard norma distribution. Under these assumptions, we state and prove the foowing proposition about the joint distribution of the z-statistic z def (z,z,...), where z j ( j µ )/ t j, from different tests. Proposition. Given Assumption and, the sequence z foows a Gaussian random wak process: where P (z j z,...,z j )N (m j (z j ), z,j) (9) j j m j (z j )µ std p j j ( j ) s j j + z j () j j z,j j j j ( j ) () with µ std (µ µ)p N being the standardized mean, and j jm/n the proportion of data in the first j minibatches. Proof of Proposition. Denote by x j the average of m s in the j-th mini-batch. Taking into account the fact that the s are drawn without repacement, we can compute the mean and covariance of the x j s as: E[x j ]µ () 8 m ><, i j Cov(x i,x j ) m N (3) >:, i 6 j N It is trivia to derive the expression for the mean. For the covariance, we first derive the covariance matrix of singe data points as Cov( k, k )E k,k [ k k ] E k [ k ]E k [ k ] if k k k µ def if k 6 k E k6k[ k k] µ N(N ) ( k,k k k N N µ k N N k) µ k µ (4) Now, as x j can be written as a inear combination of the eements in j-th mini-batch as x j m T j, the expression for covariance in Eqn. 3 foows immediatey from: Cov(x i,x j )E[x i x j ] E[x i ]E[x j ] m T Cov( i T j ) (5) According to Assumption, the joint distribution of z j s is Gaussian because z j is a inear combination of j s. It is however easier to derive the mean and covariance matrix of z j s by considering the vector z as a inear function of x: z Q(x µ ) with where Q d d... dj p N d j q j N x jm jm (6) (7) The mean and covariance can be computed as E[z] Q(µ µ ) and Cov(z) QCov(x)Q T and the conditiona distribution P (z j z,...,z j ) foows straightforwardy. We concude the proof by pugging the definition of µ std and j into the distribution.

2 .4 Empirica Student t Norma.3.4 Empirica Student t Norma.3.4 Empirica Student t Norma (a) n5 5 5 (b) n5 5 5 (c) n Figure 7. Empirica distribution (bue bars) of the t-statistic under resamping n datapoints without repacement from a dataset composed of digits 7 and 9 from the MNIST dataset (tota N 4points, mean of s is removed). Aso shown are a standard norma (green dashed) and a student-t distribution with n degrees of freedom (red soid). Figure 8. An exampe of the random wak foowed by z with µ std >. Figure 9. Sequentia test with 3 mini-batches. Red dashed ine is the bound G. Fig. 8 shows the mean and 95% confidence interva of the random wak as a function of with a few reaizations of the z sequence. Notice that as the proportion of observed data j approaches, the mean of z j approaches infinity with a constant variance of. This is consistent with the fact that when we observe a the data, we wi aways make a correct decision. It is aso worth noting that given the standardized mean µ std and j, the process is independent of the actua size of a mini-batch m, popuation size N, or the variance of s. Thus, Eqns. and appy even if we use a different size for each mini-batch. This formuation aows us to study genera properties of the sequentia test, independent of any particuar dataset. Appying the individua tests?, z j? ( ) def G at the j-th mini-batch corresponds to threshoding the absoute vaue of z j at j with a bound G as shown in Fig. 9. Instead of m and, we wi use m/n and G as the parameters of the sequentia test in the suppementary. The probabiity of incorrecty deciding µ<µ when µ µ over the whoe sequentia test is computed as: E(µ std,,g) J P (z j < j G, z i appe G, 8i <j) (8) where J d/ e is the maximum number of tests. Simiary the probabiity of incorrecty deciding µ µ when µ<µ can be computed simiary by repacing z j < G with z j >Gin Eqn. 8. We can aso compute the expected proportion of data that wi be used in the sequentia test as: (µ std,,g)e z [ j ] J j P ( z j >G, z i appe G, 8i <j) (9) j where j denotes the time when the sequentia test terminates. Eqn. 8 and 9 can be efficienty approximated together using a dynamic programming agorithm by discretizing the vaue of z j between [ G, G]. The time compexity of this agorithm is O(L J) where L is the number of discretized vaues.

3 Average Data Usage Simuation Theoretica Worst Case Error, P a Average Abs Error Actua Error with u marginaized Upper Bound 3 Standardized Mean, µ std Exact P a Figure. Average data usage estimated using simuation (bue cross) and dynamic programming (red ine). The worst case scenario with µ std is aso shown (back dashed ine). Figure. Error in the acceptance probabiity (magenta circe) vs. exact acceptance probabiity P a. Bue crosses are the expected vaue of E w.r.t. the distribution of u. Back dashed ine shows the upper bound. the actua error in the acceptance probabiity as: The error and data usage as functions of µ std are maximum in the worst case scenario when µ std!, µ! µ. In this case we have: E(,,G) im E(µ std,,g)( P (j J))/ µ std! def E worst (,G) () Figs. and show respectivey that the theoretica vaue of the error (E) and the average data usage ( ) estimated using our dynamic programming agorithm match the simuated vaues. Aso, note that both error and data usage drop off very fast as µ moves away from µ. B. Error in One Metropois-Hastings Step In the approximate Metropois-Hasting test, one first draws a uniform random variabe u, and then conducts the sequentia test. As µ std is a function of u (and µ,, both of which depend on and ), E measures the probabiity that one wi make a wrong decision conditioned on u. One might expect that the average error in the accept/reject step of M-H using sequentia test is the expected vaue of E w.r.t. to the distribution of u. But in fact, we can usuay achieve a significanty smaer error than a typica vaue of E. This is because with a varying u, there is some probabiity that µ>µ (u) and aso some probabiity that µ<µ (u). Part of the error one wi make given a fixed u can be canceed when we marginaize out the distribution of u. Foowing the definition of µ (u) for M-H in Eqn., we can compute (µ(, ), (, ),,G)P a, Z Z P (µ>µ (u))du P a P (µ>µ (u))du Z P a E(µ µ (u))du Z Pa Z Pa Z Pa du P a ( P (µ>µ (u)))du E(µ µ (u))du () Therefore, it is often observed in experiments (see Fig. for exampe) that when P a.5, a typica vaue of µ std (u) is cose to, and the average vaue of the absoute error E can be arge. But due to the canceation of errors, the actua acceptance probabiity P a, can approximate P a very we. Fig. shows the approximate P a in one step of M-H. This resut aso suggests that making use of some (approximate) knowedge about µ and wi hep us obtain a much better estimate of the error than the worst case anaysis in Eqn.. C. Proof of Theorem C.. Upper Bound Based on One Step Error We first prove a emma that wi be used for the proof of Theorem. Lemma 3. Given two transition kernes, T and T, with respective stationary distributions, S and S, if T satisfies the foowing contraction condition with a constant [, ) for a probabiity distributions P : d v (P T, S ) appe d v (P, S ) () and the one step error between T and T is upper bounded uniformy with a constant > as: d v (P T,PT ) appe, 8P (3)

4 Approximate Probabiity, P a,ε Exact Probabiity, P a Figure. Approximate acceptance probabiity vs. true acceptance probabiity. then the distance between S and S is bounded as: d v (S, S ) appe (4) Proof. Consider a Markov chain with transition kerne T initiaized from an arbitrary distribution P. Denote the distribution after t steps by P P T t. At every time step, (t) def t, we appy the transition kerne T on P (t). According to the one step error bound in Eqn. 3, the distance between P (t+) and the distribution obtained by appying T to P (t) is upper bounded as: d v (P (t+),p (t) T )d v (P (t) T,P (t) T ) appe (5) Foowing the contraction condition of T in Eqn., the distance of P (t) T from its stationary distribution S is ess than P (t) as d v (P (t) T, S ) appe d v (P (t), S ) (6) Now et us use the triange inequaity to combine Eqn. 5 and 6 to obtain an upper bounded for the distance between P (t+) and S : d v (P (t+), S ) appe d v (P (t+),p (t) T )+d v (P (t) T, S ) appe + d v (P (t), S ) (7) Let r< be any positive constant and consider the ba B(S, r ) def {P : d v (P, S ) < r }. When P (t) is outside the ba, we have appe rd v (P (t),s). Pugging this into Eqn. 7, we can obtain a contraction condition for P (t) towards S : d v (P (t+), S ) appe (r + )d v (P (t), S ) (8) So if the initia distribution P is outside the ba, the Markov chain wi move monotonicay into the ba within a finite number of steps. Let us denote the first time it enters the ba as t r. If the initia distribution is aready inside the ba, we simpy et t r. We then show by induction that P (t) wi stay inside the ba for a t t r.. At t t r, P (t) B(S, r ) hods by the definition of t r.. Assume P (t) B(S, r ) for some t t r. Then, foowing Eqn. 7, we have d v (P (t+), S ) appe + r r + r < r ) P (t+) B(S, r ) (9) Therefore, P (t) B(S, r ) hods for a t P (t) converges to S, it foows that: Taking the imit r! C.. Proof of Theorem t r. Since d v (S, S ) < r, 8r < (3), we prove the emma: d v (S, S) appe (3) We first derive an upper bound for the one step error of the approximate Metropois-Hastings agorithm, and then use Lemma 3 to prove Theorem. The transition kerne of the exact Metropois-Hastings agorithm can be written as T (, )P a (, )q( )+( P a (, )) D ( ) (3) where D is the Dirac deta function. For the approximate agorithm proposed in this paper, we use an approximate MH test with acceptance probabiity P a, (, ) where the error, def P a P a, P a, is upper bounded as P a appe max. Now et us ook at the distance between the distributions generated by one step of the exact kerne T and the approximate kerne T. For any P, Z d ( ) (P T )( ) (P T )( ) Z Z d ( ) dp ( ) P a (, )(q( ) D( )) Z appe max d ( Z ) dp ( )(q( )+ D ( )) max Z d ( )(g Q ( )+g P ( )) max (33) where g Q ( ) def R dp ( )q( ) is the density that woud be obtained by appying one step of Metropois-Hastings

5 without rejection. So we get an upper bound for the tota variation distance as d v (P T,PT ) Z d ( ) P T P T appe max (34) Appy Lemma 3 with max and we prove Theorem. D. Optima Sequentia Test Design It is possibe to design optima tests that minimize the amount of data used whie keeping the error beow a given toerance. Ideay, we want to do this based on a toerance on the error in the stationary distribution S. Unfortunatey, this error depends on the contraction parameter,, of the exact transition kerne, which is difficut to compute. A more practica choice is a bound max on the error in the acceptance probabiity, since the error in S increases ineary with max. Given max, we want to minimize the average data usage over the parameters (or G) and/or m (or ) of the sequentia test. Unfortunatey, the error is a function of µ and which depend on and, and we cannot afford to change the test design at every iteration. One soution is to base the design on the upper bound of the worst case error in Eqn. which does not rey on µ std. But we have shown in Section B that this is a rather oose bound and wi ead to a very conservative design that wastes the power of the sequentia test. Therefore, we instead propose to design the test by bounding the expectation of the error w.r.t. the distribution P (µ, ). This eads to the foowing optimization probem: min E µ, E u (µ,,µ (u),,g),g s.t. E µ, (µ,,,g) appe max (35) The expectation w.r.t. u can be computed accuratey using one dimensiona quadrature. For the expectation w.r.t. µ and, we coect a set of parameter sampes (, ) during burn-in, compute the corresponding µ and for each sampe, and use them to empiricay estimate the expectation. We can aso consider coecting sampes periodicay and adapting the sequentia design over time. Once we obtain a set of sampes {(µ, )}, the optimization is carried out using grid search. We have been using a constant bound G across a the individua tests. This is known as the Pocock design (Pocock, 977). A more fexibe sequentia design can be obtained by aowing G to change as a function of. (Wang & Tsiatis, 987) proposed a bound sequence G j G.5 j where [.5, ] is a free parameter. When, it reduces to the Pocock design, and when, it reduces to O Brien-Feming design (O Brien & Feming, 979). We can adopt this more genera form in our optimization probem straightforwardy, and the grid search wi now be conducted over three parameters,, G, and. E. Reversibe Jump MCMC We give a more detaied description of the different transition moves used in experiment 6.3. The update move is the usua MCMC move which invoves changing the parameter vector without changing the mode. Specificay, we randomy pick an active component j : j and set j j + where N (, update). The birth move invoves (for k<d) randomy picking an inactive component j : j and setting j. We aso propose a new vaue for j N (, birth). The birth move is paired with a corresponding death move (for k > ) which invoves randomy picking an active component j : j and setting j. The corresponding j is discarded. The probabiities of picking these moves p(! ) is the same as in (Chen et a., ). The vaue of µ used in the MH test for different moves is given beow.. Update move: " # µ N og u k k k k k k (36). Birth move: " # µ N og u k k k p(! )N ( j, birth)(d k) k k (k+) p(! ) k (37). Death move: µ N " # og u k k k ) (k ) k (k ) k p(! ) N ( j, birth)(d k + ) (38) We used update. and birth.in this experiment. As mentioned in the main text, both the exact reversibe jump agorithm and our approximate version suffer from oca minima. But, when initiaized with the same vaues, we obtain simiar resuts with both agorithms. For exampe, we pot the margina posterior probabiity of incuding a feature in the mode, i.e. p( j N,y N, ) in figure 3. F. Appication to Gibbs Samping The same sequentia testing method can be appied to the Gibbs samping agorithm for discrete modes. We study a mode with binary variabes in this paper whie the extension to muti-vaued variabes is aso possibe. Consider

6 Incusion Probabiity Incusion Probabiity Ground Truth Feature ε Feature Figure 3. Margina probabiity of features to be incuded in the mode running a Gibbs samper on a probabiity distribution over D binary variabes P (,..., D ). At every iteration, it updates one variabe i using the foowing procedure:. Compute the conditiona probabiity: P ( i,x i ) P ( i x i ) P ( i,x i )+P( i,x i ) (39) where x i denotes the vaue of a variabes other than the i th one.. Draw u Uniform[, ]. If u<p( i x i ) set i, otherwise set i. The condition in step is equivaent to checking: og u og( u) < og P ( i,x i ) og P ( i,x i ) (4) When the joint distribution is expensive to compute but can be represented as a product over mutipe terms, P () Q N n f n(), we can appy our sequentia test to speed up the Gibbs samping agorithm. In this case the variabe µ and µ is given by µ og u N og( u) µ N og f n( i,x i ) N f n ( i,x i ) n (4) (4) Simiar to the Metropois-Hastings agorithm, given an upper bound in the error of the approximate conditiona probabiity max max i,x i P ( i is assigned x i ) P ( i x i ) we can prove the foowing theorem: Theorem 4. For a Gibbs samper with a Dobrushin coefficient [, ) (Brémaud, 999, 7.6.), the distance between the stationary distribution and that of the approximate Gibbs samper S is upper bounded by d v (S,S ) appe max Proof. The proof is simiar to that of Theorem. We first obtain an upper bound for the one step error and then pug it into Lemma 3. The exact transition kerne of the Gibbs samper for variabe i can be represented by a matrix T,i of size D D : if x T,i (x, y) i 6 y i (43) P (Y i y i y i ) otherwise where appe i appe N,x,y {, } D. The approximate transition kerne T,i can be represented simiary as if x T,i (x, y) i 6 y i (44) P (Y i y i y i ) otherwise where P is the approximate conditiona distribution. Define the approximation error T i (x, y) def T,i (x, y) T,i (x, y). We know that T i (x, y) if y i 6 x i and it is upper bounded by max from the premise of Theorem 4. Notice that the tota variation distance reduces to a haf of the L distance for discrete distributions. For any distribution P, the one step error is bounded as d v (P T,i,PT,i ) kp T,i P T,i k appe y P (x) T (x, y) x y x i{,} max y P (x i,y i ) P (x i y i ) P (Y i y i ) max (45) For a Gibbs samping agorithm, we have the contraction condition (Brémaud, 999, 7.6.): d v (P T,S) appe d v (P, S) (46)

7 Pug max and into Lemma 3 and we obtain the concusion. F.. Experiments on Markov Random Fieds We iustrate the performance of our approximate Gibbs samping agorithm on a synthetic Markov Random Fied. The mode under consideration has D binary variabes and they are densey connected by potentia functions of three variabes i,j,k ( i, j, k ), 8i 6 j 6 k. There are D(D )(D )/6 potentia functions in tota (we assume potentia functions with permuted indices in the argument are the same potentia function), and every function has 3 8vaues. The entries in the potentia function tabes are drawn randomy from a og-norma distribution, og i,j,k ( i, j, k ) N (,.). To draw a Gibbs sampe for one variabe i we have to compute (D )(D )/ 485 pairs of potentia functions as Q P ( i x i ) P ( i x i ) i6j6k i,j,k( i,x j,x k ) Qi6j6k i,j,k( i,x j,x k ) (47) The approximate methods use a mini-batches of 5 pairs of potentia functions at a time. We compare the exact Gibbs samping agorithm with approximate versions with {.,.5,.,.5,.,.5}. To measure the performance in approximating P () with sampes x t, the idea metric woud be a distance between the empirica joint distribution and P. Since it is impossibe to store a the probabiities, we instead repeatedy draw M 6 subsets of 5 variabes, {s m } M m,s m {,...,D}, s m 5, and compute the average L distance of the joint distribution on these subsets between the empirica distribution and P : Error M s m k ˆP ( sm ) P ( sm )k (48) The true P is estimated by running exact Gibbs chains for a ong time. We show the empirica conditiona probabiity obtained by our approximate agorithms (percentage of i being assigned ) for different in Fig. 4. It tends to underestimate arge probabiities and overestimate on the other end. When., the observed maximum error is within.. Fig. 5 shows the error for different as a function of the running time. For sma, we use fewer mini-batches per iteration and thus generate more sampes in the same amount of time than the exact Gibbs samper. So the error decays faster in the beginning. As more sampes are coected the variance is reduced. We see that these pots converge towards their bias foor whie the exact Gibbs samper outperforms a the approximate methods at around seconds. Empirica Probabiity True probabiity ε. ε.5 ε. ε.5 ε. ε True Conditiona Probabiity Figure 4. Empirica conditiona probabiity vs exact conditiona probabiity for different vaues of. The dotted back ine shows the resut for exact Gibbs samping. L error ε., T 349 ε.5, T 3979 ε., T 4494 ε.5, T 4889 ε., T 557 ε.5, T 5959 Gibbs, T Time (s) Figure 5. Average L error in the joint distribution over ciques of 5 variabes vs running time for different vaues of. The back ine shows the error of Gibbs samper with an exact acceptance probabiity. T in the egend indicates the number of sampes obtained after seconds.