SAMPLING LIPSCHITZ CONTINUOUS DENSITIES OLIVIER BINETTE Abstract. A simple ad efficiet algorithm for geeratig radom variates from the class of Lipschitz cotiuous desities is described. A MatLab implemetatio is freely available o GitHub (repository olivierbiette/lipsample). 1. Itroductio Let f be a measurable o-egative fuctio o the iterval [0, 1]. Give a upper boud M o f, the followig acceptace-rejectio algorithm geerates a radom variate followig the desity proportioal to f. Algorithm 0. 1. Geerate (x, y) uiformly distributed o the cube [0, 1] [0, M]. 2. If y > f(x), the go to 1; otherwise retur x. This is the simplest example of a uiversal or black box geerator. It requires a fuctio f, which may be evaluated at arbitrary poits ad a kow upper boud M o f, but otherwise is ot tailored to ay particular fuctioal form. Here we refie Algorithm 0 i the case where f is Lipschitz cotiuous of a kow order (e.g. whe a upper boud o the secod derivative of f is kow). The idea is to use the kow Lipschitz order ad the evaluatio of f o a grid of poits to costruct a first-degree splie evelope which is the applied to acceptace-rejectio samplig. Classes of desities with kow Lipschitz cotiuity order iclude polyomial ad trigoometric polyomial desities, log-polyomial desities such as the Vo Mises distributio ad Gaussia distributios, etc. This work was motivated by poor simulatio algorithms proposed i the litterature for particular classes of desities. For istace, Feradez-Dura (2014) proposed the use of Algorithm 0 to simulate from the class of trigoometric polyomial desities. Sice the first derivate of such desities is easily bouded, our method is a immediate (ad rather sigificative) improvemet. 1.1. Short litterature review. Hörma et al. (2004) describe uiversal geerators for differet classes of desities excludig Lipschitz cotiuous desities. Beliakov (2005) implemets a uiversal geerator for multivariate Lipschitz cotiuous desities based o zeroth degree splie approximate evelopes. Date: November 5, 2017. 1
2 OLIVIER BINETTE 1.2. Structure of the paper. The rather simple method, which we refer to as the LipSample algorithm, is preseted i sectio 2. Sectio 3 presets a efficiet vectorized implemetatio i MatLab. 2. The LipSample algorithm Recall that a fuctio f defied o the iterval [0, 1] is said to be of Lipschitz f(x) f(y) order M, deoted f Lip(M), if sup x,y [0,1] x y M. If f is cotiuously differetiable, the f Lip(sup f ). The followig algorithm geerates radom variables followig the desity proportioal to f Lip(M). Algorithm 1 (LipSample). 1. Costruct a splie evelope S of f. 2. Geerate x followig the desity proportioal to S ad geerate y uiformly o [0, S(x)]. 3. If y > f(x), the go to 2; otherwise retur x. We may also costruct a lower splie evelope of f i order to accelerate step 3. Next we describe how to perform steps 1 ad 2. 2.1. The splie evelope. For simplicity ad computatioal efficiecy, we use the first order splie evelope S = S give by ( (1) S = f( i ) + h i,) Pi,, i=1 where h i, R is to be determied, P i, (x) = c i, max{0, 1 i x }, c i, = 2 if i = 0 or i =, c i, = 1 otherwise. Hece S liearly iterpolates betwee the poits (i/, f(i/) + h i, ), i = 0, 1,...,. Figure 1 The costats hi, must ow be chose as to esure S f. The simplest choice would be hi, = L/(2), but we ca do better. Give two sample poits A = ( i 1 i 1, f( )) ad B = ( i, f( i )), the Lipschitz coditio o f yields a triagle ABC such that if x [ i 1, i ] ad f(x) is greater tha the value of the liear
SAMPLING LIPSCHITZ CONTINUOUS DENSITIES 3 iterpolat of A ad B at x, the (x, f(x)) is iside of ABC. Precisely, the slope of the lie AC is arcta(m) ad the slope of BC is arcta(m). If say C = (z, f(z)) the we deote the differece betwee f(z) ad the value of the liear iterpolat at z by l i, (see Figure 1). It ca be show by elemetary geometry that (2) l i, = r(m (f( i i 1 ) f( ) ), where r = 1 2 Hece takig A B si(π α β) si(α) (3) h i, = max{l i,, l i+1, }, α = arcta(m) ad β = arcta( (f( i i 1 ) f( ) ). esures that S f ad furthermore this boud is sharp over Lip(M). Note also that at ay poit x [0, 1], oly two terms of the sum S (x) are o-zero. I fact, lettig i = ( + 1)x, we fid S (x) = (x i)f( i i+1 ) + (i + 1 x)f( ). It is also quite easy to sample from the desity proportioal to S as it is a mixture of triagular desities. Lemma 1. Let p i = f( i )+h i, if i = 1, 2,..., 1, p i = (f( i )+h i,)/2 if i = 0,. Suppose I is distributed as P (I = i) p i ad that U 1, U 2 id. uif([0, 1]). Let Z = (U 1 + U 2 + I 1)/ ad X = Z + 2(1 Z) I Z>1. The X is distributed proportioally to S. Proof. Write S = i=0 p P i i,, where P i, = P i, if i = 1, 2,..., 1 ad P i, = 2P i, if i = 1, 2. Fially, remark that P [0,1] i, does ot deped o i ad that coditioally o I = i, X is distributed proportioally to P i. 3. Implemetatio Let the fuctio f defied o [0, 1] be give together with its Lipschitz costat M. The followig MatLab code produces radom variates followig the desity proportioal to f. Step 1. Costructig the splie evelope. 1 % Number of compoets for the splie. 2 = ceil(40*sqrt(m)); 3 4 % Evaluate f at +1 equidistat poits ad add the costats h_{i,} = M / (2) 5 x = lispace(0,1, +1); 6 y = arrayfu(f, x) + M/(2*); Step 2. Geerate radom variates from the evelope.
4 OLIVIER BINETTE 1 % Assume Prop is give. 2 U1 = rad(1, Prop); 3 U2 = rad(1, Prop); 4 5 % Simulate from the discrete distributio specified by the weights y. 6 y(1) = y(1)/2; 7 y(ed) = y(ed)/2; 8 I = discretesample(y, Prop); % Fuctio available o github.com/olivierbiette/libsample. 9 y(1) = 2*y(1); 10 y(ed) = 2*y(ed); 11 12 % The radom variates are give by U. 13 U = abs((u1 + U2 + I - 2)/); 14 U(U > 1) = 2 - U(U > 1); Step 3. Acceptace-rejectio. 1 V = rad(1, Prop); 2 sample = U(lt(V.* iterp1(x,y,u), B)); The precedig steps may be repeated if the size of the sample is smaller tha eeded. 3.1. Improvemets. The followig calculates the costats h i, as i sectio 2 ad may be used to replace Step 1. 1 x = lispace(0,1,+1); 2 y = arrayfu(f, x); 3 4 % Use the Lipschitz costat to locally adjust the splie. 5 alpha = ata(l); 6 d = diff(y); 7 beta = abs(ata(*d)); 8 r = 0.5*sqrt(^2 + d.^2).*si(pi-alpha-beta)./si(alpha); 9 ell = r.*(l - abs(*d)); 10 11 y(1) = y(1) + ell(1); 12 y(+1) = y(+1) + ell(); 13 for i = 2: 14 y(i) = y(i) + max(ell(i-1), ell(i)); 15 ed We also use a lower splie evelope to accelerate Step 3. A proper iterface to this algorithm is available o GitHub (repository olivierbiette/libsample). 3.2. Performace. For the geeratio of a large umber of ormal variates restricted to [ 5, 5], the LipSample procedure is oly about 50 times slower tha
SAMPLING LIPSCHITZ CONTINUOUS DENSITIES 5 2.5 2 1.5 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Figure 2. The desity f(x) = 1 + cos(2πx) (blue), a splie evelope S (orage), ad 100 000 radom variates obtaied usig the LipSample algorithm (grey histogram). MatLab s ow rad. We cosider this to be quite fast: rad has bee iterally optimized whereas our LipSample code is iterpreted by MatLab ad completely geeral. Refereces [1] Beliakov, G. (2005). Class library ralip for multivariate ouiform radom variate geeratio. Computer Physics Commuicatios, Volume 170, Issue 1. [2] Devroye, L. (1988). No-Uiform Radom Variate Geeratio. Spriger-Verlag. [3] Ferádez-Durá, J. ad Gregorio-Domíguez, M. (2014). Modelig agles i proteis ad circular geomes usig multivariate agular distributios based o multiple oegative trigoometric sums. Statistical Applicatios i Geetics ad Molecular Biology, 13(1), pp. 1-18. [4] Hörma, W., J. Leydold, J. ad Derfliger, G. (2004). Automatic Nouiform Radom Variate Geeratio. Spriger. [5] Robert, C. et Casella, G. (2004). Mote Carlo Statistical Methods. Spriger-Verlag.