TOTAL VARIANCE AS AN EXACT ANALYSIS OF THE SAMPLE VARIANCE*

Transcription

1 29th Annual Peise Time and Time Inteval (PTTI) Meeting V TOTAL VARIANCE AS AN EXACT ANALYSIS OF THE SAMPLE VARIANCE* b I F Donald B. Peival Applied Physis Laboatoy Univesity of Washington Box Seattle, WA USA dbpoapl. washington, edu David A. Howe Time and Fequeny Division National Institute of Standads and Tehnology 325 Boadway Boulde, CO USA dhowehist. gov I P Abstat Given a sequene of fational fequeny deviates, we investigute the elationship between the sumple vaiane of these deviates and the total vaiane (Totva) estimato of the Auan vaiane. We demonsiate that we un eove exutly twie the sample vaiane by enomalizing the Totva estimato and then summing it ove dyadi aveaging times, 2, 4,..., 2J along with one additional tem that epesents vaiations at all dyadi aveaging times geute than 2-?. This deomposition of the sample vaiane mimis a simila theoetial deomposition in whih summing the tue Allan vaiane ove all possible dyadi aveaging times yields twie the poess vaiane. We also establish a elationship between the Totva estimato of the AUan vuiune and a biased muximal ovelap estimato that uses a iulaized vesion of the oiginal fational fequeny deviates. INTRODUCTION The goal of this pape is to exploe the elationship between the sample vaiane of a sequene of fational fequeny deviates {g,, : n =,.. *, NY], namely, and a new estimato of the Allan vaiane alled Totva ( total vaiane - see the ompanion atile by Howe and Geenhall [l] in these Poeedings fo additional details). The Totva estimato is based upon the hypothesis that easonable suogates fo unobseved deviates yn, n < o n > A?!, an be fomed by taking on evesed vesions of {yn) at the beginning and end of the oigmal seies. The Totva estimato makes use of etain of these suogate values in ode to ome up with a new estimato of the Allan vaiane that has bette mean-squaed eo popeties than the usual Allan vaiane estimato at the vey lagest sampling times (Howe and Geenhall [l]). Hee we show that a enomalized vesion of the Totva estimato an be used to exatly deompose twie the sample vaiane. Exept fo the fato of two (an histoial atifat due to the oiginal definition of the Allan vaiane), this deomposition of the sample vaiane is vey muh simila to the one affoded by taditional spetal analysis Contibution of the U. S. Govenment, not subjet b wpyight. 97

2 estimatos, whih exatly deompose the sample vaiane aoss diffeent Fouie fequenies. By ompaison ou esults show that the (enomalized) Totva estimato deomposes the sample vaiane aoss dyadi aveaging times (ie., aveaging times of the fom 2j70, whee T~ is the sample peiod fo {u,,}). Ou esult thus says that the Allan vaiane an be egaded as an example of an analysis of vaiane tehnique, whih is one of the most widely used data analysis methods in moden statistis. The emainde of this pape is oganized as follows. In Setion 2 we eall that in fat a vey ealy estimato of the Allan vaiane (the nonovelapped estimato) exatly deomposes twie the sample vaiane fo the speial ases when ivy is a powe of two. Beause of its poo vaiane popeties, the nonovelapped estimato is vey seldom used, so we disuss in Setion 3 what is geneally onsideed to be the pefeed estimato, namely, the maximal ovelap estimato. The usual fomulation of this estimato does not yield a deomposition of twie the sample vaiane; howeve, if we view this estimato as the mean-squaed output of a iula filteing opeation, we an augment the estimato with additional tems (namely, ones that make expliit use of the iulaity assumption) and ome up with an biased vesion of the maximal ovelap estimato that does yield a deomposition of twie the sample vaiane fo any sample size N,. Beause of the potential mismath between ~ and y~,, this iulaity assumption an lead to seious biases. Thus, in Setion 4 we onside using the biased maximal ovelap estimato with the seies of length 2N, fomed by taking on a evesed vesion of {y,} at the end of the oiginal seies. This new estimato an be witten as a enomalized vesion of the Totva estimato. In Setion 5 we summaize ou esults and onlude with a few omments. 2 THE NONOVEFUAPPED ESTIMATOR OF THE ALLAN VARIANCE Fo this setion only we assume that the sample size is a powe of two ; i.e., we an wite N3, = 2J fo a positive intege J. Given a sequene of T0-aveage fational fequeny deviates {y, : n = I,...,Ny} with a sampling peiod between adjaent obsevations given by T~ also, let us define the mo-aveage fational fequeny deviate as. m-l If we egad {~j,(m) as a ealization of one potion of the stohasti poess {L,,(m) : n = 0, fi, f2,...}, the Allan vaiane fo aveaging time mi0 is defined as : n = m,...,ny} whee we assume that the stohasti poess is suh that the expetation above in fat depends on the aveaging time index n, but not on the time index n (this will be tue if the fist diffeene poess {L,(l)- F,+l(l)} is a stationay poess). Fo m = 2J fo j = 0,,..., J -, let us fom the so-alled nonovelapped estimato of the Allan vaiane: 98

3 Fo example, if j = 0 so that n =, the above edues to 5 so that eah yn ontibutes to exatly one tem in the sum of squaes above (hene the oigin of the name 44nonovelapped we have estimato ). At the othe exteme when j = J - so that m = 2, The nonovelapped estimato an be intepeted in tems of an othonomal tansfom of the olumn veto y whose elements ae given by (yn}. Fo & = 8, this tansfom is given by the following 8 x 8 mat& -* * o I fi & $ a 0 T Y $\I -- Ja.$ 7 3- (fa othe N,, the NY x Ny matix W is fomulated in an analogous manne and and is one vesion of the disete Haa wavelet tansfom - fo details, see, e.g., [4]). Letting w = Wy and letting (wn} denote the elements of w, it follows that w;+w;+w;+w; = 48=,,-*(> w,+w,~ 2 = 48:,-(2) w; = 42Z,-(4) w; Beause W is an othonomal tansfom, we must have llw2 = lly2, whee llxll is the usual Eulidean nom of the veto x. It follows that Fo geneal Ny = 2-,the oesponding esult is i.e., summing the nonovelapped Allan vaiane estimato ove all dyadi aveaging times less than o equal to 3 yields exatly twie the sample vaiane (fo additional details and some histoial bakgound, see Setion I of [3]). 99

4 3 MAXIMAL OVERLAP ESTIMATORS OF THE ALLAN VARIANCE The nonovelapped estimato of the Allan vaiane is aely used in patie beause it does not take advantage of etain infomation egading a:(n). To see in what sense this is tue, let us onside the fom of the maximal ovelap estimato fo m = : Note that, wheeas eah y, appeas exatly one in the nonovelapped estimato of Equation (l), the vaiables B,...,YN,-~ appea twie beause now, in addition to tems like [p2-yi2, [g., - y3i2 and [UE - 95j2 that appea in 6;,-(), the maximal ovelap estimato also inludes tems like [ys- yzi2 and b5 - g4i2. Fo geneal n the maximal ovelap estimato takes the fom -2 qt,-(m) = N, L [ Fkb> - Fk-n(774l2 * w, - 2m + ) t2m Even if we wee to estit the sample size N;. to be a powe of two 2J, it an be agued that in geneal J- j-0 so the usual maximal ovelap estimato does not onstitute an analysis of twie the sample vaiane. Thee is, howeve, an inteesting way to define a vaiation on the maximal ovelap estimato that in fat does yield an exat analysis of vaiane, as the following agument shows. We stat with two filtes {b,l} - { (in the wavelet liteatue, &J} and {a,!} defined as follows: i, =0; * =0; h0.l e -+, I =, and and GOJ = (a: I = ; and 0, <0o 22; <0o>_2 and {ao$) ae two vesions of what ae alled the Haa wavelet and saling filtes - fo details, see, e.g., [4]). Let &(.) and &,(*) be the tansfe funtions fo {L} and {God: OD G he- i2xfl = i- f si.(f) and Goy) gle-i2f = e- f os(?f); I=-m i.e., &(*) is the disete Fouie tansfom (DFT) of {hoj). Note that we have leo(f)2 + lg0(f)l2 = fo all f. (2) m I=-00 Y We want to iulaly filte {gn} sepaately using the filtes {ho,l} and {~o,l}. Fomally, we do SO by defining {&$ : = 0, +. *, N, - ) and {& : = 0,...,Ny- I}, whih ae said to be {ho~} and {GO,[} peiodized to length IVY. By definition, t-x, L

5 I, with a simila definition fo {j$}. If Ny 2 2, we have * =0; Z=O; h;,i -- 2, I = ; and and & = I: { = ; and 2' 25Ny-l; 0, 255Ny-; - I 0, if, howeve, Ny = so that {&,} and {&} eah have but a single tem, then &,o = o and G;,o =. It is an easy exeise to show that the DFT of {&} an be obtained by subsampling the DFT fo {&,$}; i.e., The finite sequenes {&,i} we will expess as Similaly we have {f& Nm-l C i=o e-i2xklin u = &( k), k = 0).* *, Ny - * 0. and {I&($-)} thus onstitute a Fouie tansfom pai, a elationship - : 6 = 0,...,Ny- ) {I&(*) : k = 0,.. *, N9 - ). Let us now define Nm- Nm- G,n %$!/+l)md Nu and G,n &y(,-l)mod N,, n =,... I N,, =0 l=o whee we define n mod Ny to be n if 5 n 5 Ny and to be n + FEN, othewise, whee k is the unique nonzeo intege suh that 5 n + kny 5 IVY (thus - mod Ny = N, - ; 0 mod Ny = IVY; mod Ny = ;. * ; Ny mod Nv = N,; Ny -t mod Ny = ; et.). By onstution we have (3). i.e., we have expessed the maximal ovelap estimato of the Allan vaiane fo m = in tems of a sum of squaes of the output fom iula filteing {yn} with {&,l}. An impotant point to note is that &:-(I) does not involve the entie output fom the filte: it is missing Go,l o YN, - yl, whih is the only tem that expliitly makes use of the iulaity assumption. Inlusion of this tem is one of the two keys to defining a vesion of the maximal ovelap estimato that onstitutes an analysis of vaiane. The othe key is to eognize that 8:,-(2j) fo j =,2,*.. an be obtained by futhe filteing of ($0,") so that, wheeas ontains infomation about the vaiations of {ynn) at TO aveaging times, the seies {Co,n} ontains infomation about vaiations of {y,,} at all dyadi aveaging times highe than T~ (ie., 270, hop et.). Aodingly, let GO be an N, dimensional veto whose elements ae {Go,.,}, and define Go to ontain {Zo,n}- Letting {&} be the DFT of {y,,}, we have (fom a standad theoem in filteing theoy) - - {GpI {?b(k)~,~ and {G,n} {ZO(~)Y~}*, 0

6 Paseval's theoem tells us that whih in tun yields whee we have made use of Equation (2) and a seond appliation of Paseval's theoem. Let us now define the following estimato of the Allan vaiane fo m = : We efe to this estimato as the biased maximal ovelap estimato of oi() based on {g,,}. It diffes fom the standad maximal ovelap estimato (Equation (3)) beause of an additional tem popotional to (YN,- ~ )~. Although this estimato is in geneal a biased estimato of the tue Allan vaiane, it is in fat unbiased when {gn} is a white noise poess. It satisfies the analysis of vaiane ondition 5:,-() + fii,-(2) = 28;, whee $,(2) = -~~70~~2 4 We an egad the seond piee of the deomposition Q,-(2) {gn} at dyadi aveaging times of 2 and geate. 2 - G2. as being elated to vaiations in Just as {gn} was split into the omponents (Goa} and {Gbp}, we now split {Gj,n} into two omponents, namely, {GI,?} and {Gl,n}. The fist omponent (iiil,n} will be used to onstut an estimato of a;@), while the seond omponent is elated to vaiations in {gn} at dyadi aveaging times of 4 and geate. The filtes that aomplish the desied split ae N,-peiodized vesions of ones whose tansfe funtions ae defined by &(2f) and 6&f) - the impulse esponse sequene fo these filtes an be fomed by taking the oiginal filtes {b,,} and {fio0,l} and inseting a single zeo afte eah element, a poedue that is known as upsampling in the engineeing liteatue [4]. Fo example, sine the I : 0,,2,3 and 4 values of the impulse esponse sequene fo &(f) ae given by i, -4, 0,O and 0, the oesponding values fo &2f) ae given by $, 0, -3, 0 and 0. We an also obtain {GI+} and {GI,,,} by dietly filteing {y,}: - N,- w.n = C gjg(n-l)motinn and G,n = C fi;jg(n-qd N,, n =, l=o l=o whee (K;} is the iula filte suh that N,- {%,I - {fio($~o($)h likewise, {G;,t} - {Go($ 2k Note that the impulse esponse sequene fo {QJ} is the iula onvolution of the impulse esponse sequene fo GO($ and go(%), i.e., a, $, 0,... onvolved with $, 0, -4, 0, whih o($). 02

7 yields $, 0,,..(as long as Ny > 4). This latte filte is seen to be popotional to the filteing opeation ommonly used in estimates of 0,(2). Define GI as the N, dimensional veto ontaining {G,n}, and let 5 ontain {Gl,l,n}. By a simple vaiation on the agument used to establish IIi?0~ + V0~ = y2, we have %2.f IlSII2 = ~o:02. Now define the following estimato of the Allan vaiane fo m = 2: This is the standad maximal ovelap estimato with thee additional tens - these ae popo- 2 tional to (Pi -t PvW - PN,-~_ P,-i), (Pi + Y2 - PN+ - PN,)~ and ( ~ 2 + ~3 - m - Y N,)~. h geneal this estimato is a biased estimato of a$). We have the analysis of vaiane ondition We an now state the esult fo geneal J, a poof of whih follows fom an easy indutive agument. We define - Nu- wj,n= $,ly(n-l)nodnv and Gj,n= ~~~jy(n-l)dnu, n= ---~Ny, 4 l=o - 2j-Ik - p -ak - ($4) m $$~o(+go(~) *-Go(*)> whee - and {$J} { Go(~)Go(~)a(~) - - *Go(&),. An indutive agument an be used to show that is the usual filte involved in estimating 4(2i). Letting G, and Ti be Ny dimensional vetos ontaining {fij,n} and {Zj,n}, define Nw- whih is the biased maximal ovelap estimato of 0329 based upon {yn} - it diffes fom the standad maximal ovelap estimato due to 2j+l- additional tems involving expliit iula use of {yn}. Fo any J, the biased maximal ovelap estimatos satisfy the analysis of vaiane ondition C whee the tem.?;,-(2j+) geate. epesents vaiations in {yn} at dyadi aveaging times of 2J+ and 03

8 4 ESTIMATION OF THE ALLAN VARIANCE USING A CIRCULARIZED SERIES Fo most models fo (9,) of inteest fo atual fequeny standads, a iulaity assumption an yield an unaeptably lage bias in the estimato 5:,-(29 due to the fat that yl and yn, an be quite diffeent. To solve this poblem, we onstut a seies {y;} of length 2Ny { ha 5 n 5 N,; and Ny + 5 n I 2Ny. $f2nw+l-n Fo example, if N = 3 SO that only yl, and ae obseved, the values of yi,..., & ae given, espetively, by y,y~,y~,ys,y2,y. Note that, by onstution, the sample mean and vaiane of {pa} and {y;} ae idential. We now apply the estimation poedue of Setion 3 to {y;} to obtain the following estimato of u,2(29: whee G; is a veto of length 2Ny fomed by iulaly filteing {&} with the iula filte of length 2 4 whose DFI is given by We efe to GE.,-($) as the biased maximal ovelap estimato of 7329 based on the iulaized seies {&} (note, howeve, that this estimato is in fat unbiased fo the speial ase whee {pa} is a white noise poess). This biased estimato satisfies the following analysis of vaiane ondition fo all J and all sample sizes Nu: Finally we note that thee is a vey simple elationship between E$,,-(29 (Geenhall, 997, pivate ommuniation): and Totva estimato whee Totva is defined as in Equation (4) of Howe and Geenhall [l]. 5 SUMMARY AND COMMENTS We have developed a elationship between the sample vaiane &: and the Totva estimato Totva(m, Nyl TO) of the Allan vaiane af(m), whee m sets the aveaging time mo, N, is the numbe of.o-aveage fational fequeny deviates {pj, and T~ is the basi sampling and aveaging time of the obseved deviates. Fo any sample size N, and any positive intege J, we have demonstated that 04

9 whee +i.-(2j+) an be intepeted in tems of vaiations in {pa} at all dyadi aveaging times geate than 2-. We have also shown that the Totva estimato is elated to a biased maximal ovelap estimato of the Allan vaiane that is based upon {y;}, whih is a sequene of length 2N, fomed by taking onto {gn} a evesed vesion of itself. In losing, we make the following omments about ou esults, some of whih will be expanded upon in futue eseah. It an be shown that, if {yn} is a potion of a ealization of a stationay o nonstationay poess {Y,,} fo whih the Allan vaiane is well-defined, then we have 4 29 = 24, j=o whee ui is the poess vaiane of {K} (this is taken to be infinite if {Y,} is nonstationay). The ifi,,-(29 estimato is the fist moden estimato of the Allan vaiane to mimi this impotant popety. Beause highe ode Daubehies wavelet filtes also satisfy Equation (2), the above development extends tivially to highe ode wavelet vaianes (the Allan vaiane is essentially twie the Haa wavelet vaiane). These highe ode wavelet vaianes ae suitable substitutes fo some of the vaiations on the Allan vaiane that have been poposed and studied in the liteatue (an example is the modified Allan vaiane). Fo details, see [2]. In addition to plotting i$.,-(2j> vesus 2iO on a log-log sale, we suggest be plotted (with a sepaate symbol) vesus 2- +*~~ - this will indiate how muh of 26; has not been aounted fo by estimates of the Allan vaiane, In theoy J an be made as lage as desied, but thee will be seious biases in 5iy2.,m(25) fo any J suh that > Beause of its lose elationship to Totva, the esults of Howe and Geenhall [l] indiate that 5:.,-(29 outpefoms taditional estimatos of the Allan vaiane fo aveaging times lose to $. Refeenes h [l] D. A. Howe and C. A. Geenhall 998, Total Vaiane: a Pogess Repot on a New Fequeny Stability Chaateization, 29th PTTI Meeting, these Poeedings. [2] D. A. Howe and D. B. Peival 995, Wavelet Vaiane, Allan Vaiane, and Leakage, IEEE Tans. Instum. Meas. IM-44, [3] D. B. Peival 99, Chaateization of Fkequeny Stability: Requeny-Domain Estimation of Stability Measues, Po. IEEE 79, [4] M. Vetteli and J. Kovakvie 995, Wavelets and Subband Coding (Englewood Cliffs, New Jesey: Pentie Hall). 0306