Optimal estimation of clock values and trends from finite data

Transcription

1 Opimal esimaion of clock values and rends from finie daa Charles Greenhall Je Propulsion Laboraory, California Insiue of Technology Pasadena, CA USA Absrac We show how o solve wo problems of opimal linear esimaion from a finie se of phase daa. Clock noise is modeled as a sochasic process wih saionary dh incremens. The covariance properies of such a process are conained in he generalized auocovariance funcion (GACV). We se up wo principles for opimal esimaion; hese principles lead o a se of linear equaions for he regression coefficiens and some auxiliary parameers. The mean square errors of he esimaors are easily calculaed. The mehod can be used o check he resuls of oher mehods and o find good subopimal esimaors based on a small subse of he available daa. I. INTRODUCTION Suppose ha he phase residual x () of a clock (or difference of clocks) is given a a finie se of imes, T = { 1,..., n }, which don have o be equally spaced. We consider wo esimaion arges: 1) he phase value x ( ) a some / T ; 2) he overall rend coefficien of x (). Depending on he model for x () and wha else is known, his can mean he long-erm average value of phase or of is dh ime derivaive: frequency, drif rae, or aging rae (d =1, 2, or P 3). For each arge, we wan o calculae he linear esimaor n i=1 a ix ( i ) ha is opimal in he mean-square sense while saisfying an invariance condiion ha will be explained below. To carry ou his program, we need a sochasic model for x (). The general model used here is he class of sochasic processes wih saionary dh incremens, he subjec of a monograph of Yaglom [1]. These include saionary processes, indefinie inegrals of saionary processes, and all he powerlaw processes familiar o he ime and frequency field: flicker PM, whie FM, flicker FM, and so on. Yaglom showed how o solve problems of opimal predicion and filering for hese processes from heir values on unbounded or bounded ime inervals. For pure power-law processes, oher auhors have derived he mean square error (MSE) of he predicor ha is based on he infinie pas [2] [4]; Boulanger and Douglas [5] calculaed he MSE of wo-poin linear exrapolaion. Vernoe e al. [6] calculaed predicors and heir MSEs based on exrapolaion of leas-squares linear and quadraic fis of equally spaced finie daa ses. For finie-sae clock models, a recursive opimal predicor and is MSE, based on all discreeime pas measuremens, can be calculaed from a Kalman filer [7]. The MSE of various subopimal drif-rae esimaors has been calculaed for power-law noises [8] [11]. Here we show how o calculae he regression coefficiens a i and he MSE of he opimal linear esimaors of boh arges, using sysems of linear equaions ha generalize he equaions of orhogonal projecion for saionary x (). II. CLOCK NOISE MODELS A real-valued, mean-square coninuous 1 sochasic process x () is said o have saionary dh incremens (d 1) iffor each τ he process d µ d d τ x () = ( 1) k x ( kτ) (1) k k=0 is saionary. I is convenien o le SI (d) denoe he class of all such processes, and o le SI (0) denoe he saionary processes. Then SI (d) SI (d +1), and we define he degree of x () as he leas d such ha x SI (d). Everyhing we know abou a process x SI (d) is wrapped up in he dh incremens (1), which do no change if we add a polynomial of degree d 1 o x (). In his sense, a process in SI (d) is ambiguous. Any use of hese processes mus ake accoun of his ambiguiy. For any x SI (d), Yaglom esablished a nonnegaive specral densiy funcion 2 S x (f) ha exends he noion of specral densiy for saionary processes. Here, we are using he wo-sided, even version: S x ( f) =S x (f). Ford 1, S x (f) can diverge as f 0 bu obeys he resricions Z Z f 2d S x (f) df <, S x (f) df <. (2) f <1 f >1 The process also has an average rend coefficien, denoed by c d, ha can be defined as he infinie-ime average of he saionary process d 1x (): c d = lim Z 2 1 d 1x () d, he limi being aken in he mean-square sense. In his reamen, c d can be a random variable. We ofen wan o ge rid of c d, and here are wo ways o do i. Firs, if we know c d,hen we can consider he process x 0 () =x () c d d /d!, whichis also in SI (d) bu has rend coefficien zero. Second, if we don know c d, hen we can rea x () as a member of SI (d +1); as such, is rend coefficien c d+1 is always zero. For example, a saionary process wih an unknown mean can be reaed as 1 This means ha E[x (u) x ()] 2 0 as u. 2 Acually, a measure on he puncured frequency axis f 6= /05/$ IEEE. 377

2 amemberofsi (1). If a random walk of phase (whie FM), which is in SI (1), has an unknown slope (frequency) added o i, we can rea i as a member of SI (2). Inhiswaywe can ge resuls ha are invarian o he unknown rend. Yaglom s heory of SI (d) was based on he specral densiy. For saionary processes, we also have he auocovariance (ACV) funcion which saisfies s x () = Z e i2πf S x (f) df, (3) s x ( u) =Ex () x (u). (4) The esimaion mehods given here for SI (d) are based on a generalized ACV (GACV) funcion [12][13], also wrien as s x (), ha can be obained from S x (f) and c d by a generalized Fourier inegral as follows: Z " # 2d 1 s x () = e i2πf (i2πf) k S x (f) df f 1 k! k=0 Z + e i2πf S x (f) df + E c 2 2 d d. (5) f >1 (2d)! One may add a polynomial of degree 2d 1 o s x () wihou changing he value of any formula in which s x () is properly used. Wih his ambiguiy undersood, Table I gives he GACV for power-law componens of clock noise, specified by he onesided specral densiy of frequency, S y + (f) =2(2πf) 2 S x (f). For any saionary phase componen, he GACV is he ACV. The flicker PM enry is obained by passing pure 1/f noise hrough a moving-average filer of widh τ o saisfy he second condiion in (2). The rend coefficiens are zero. The degree of a sum of noises is he maximum degree of he summands, and he GACV of he sum of orhogonal noises is he sum of he GACVs. By (5) we may and will assume ha he GACV is an even funcion: s x ( ) =s x (). TABLE I GENERALIZED AUTOCOVARIANCE OF POWER-LAW NOISES Name degree S + y (f) s x () Flicker PM 1 h 1 f sin2 (πfτ) (πfτ) 2 h 1 3 4π 2 2 ln τ, =0 (lowpass) h 1 4π 2 ln, À τ Whie FM 1 h 0 h 0 4 Flicker FM 2 h 1 f 1 h 1 2 ln 2 Random walk FM 2 h 2 f 2 h 2 π Flicker walk FM 3 h 3 f 3 h 3π 2 4 ln 6 Random run FM 3 h 4 f 4 h 4π 4 5 Now we have o say how he GACV is used. In he following discussion, a () and b () denoe funcions ha are zero excep on some unspecified finie se of imes; by using his noaion 30 we can freely perform and combine sums over wihou worrying abou he range of he summaions. Such a funcion a () is said o saisfy he momen condiion of order d [6][14] if a () j =0, j =0,...,d 1. (6) In oher words, if x () in P a () x () is replaced by a polynomial p () of degree d 1, he resul is zero. The coefficiens of he dh incremen (1) saisfy his condiion. All he covariances needed for he esimaion problems can be calculaed by he following heorem. Theorem 1: Le s x () be he GACV of x (), a process wih saionary dh incremens, where d 1. Ifa () and b () saisfy he momen condiion of order d, hen Ã!Ã! E a () x () b () x () = a () b (u) s x ( u). (7),u For saionary processes, (7) follows from (4), and no momen condiions are needed. For d 1, we are only allowed o ake he covariance of linear combinaions of x () whose coefficiens saisfy he momen condiion. According o he heorem, we may do so as if (4) were rue. In realiy, s x () is no an ACV, and (4) does no hold. Neverheless, he enire formula (7) is correc, even hough he corresponding erms of he expansions of is lef and righ sides are no equal. P We define M d (x) as he se of random variables a () x () whose coefficiens a () saisfy he momen condiion of order d. ThenM d (x) is a linear subspace, and Theorem 1 ells us how o calculae he covariance of wo members of M d (x). I can be shown ha any member of M d (x) is a mean-square limi of linear combinaions of dh incremens of x (); for his reason, if c d =0hen he members of M d (x) have zero expecaion. We also define M d (x, T ) as hose members of M d (x) whose coefficiens are suppored on he finie se T. III. CLOCK PREDICTION Le x () have saionary dh incremens. The esimaion arge is x ( ), and he esimaors are of form ˆx ( )= a () x (), (8) where T = { 1,..., n } wih n d. (Alhough his problem is called predicion, here is no need o insis ha > max i.). We now have o make he problem invarian o he ambiguiy of x () wih respec o polynomials p () of degree d 1. We do so by insising ha he esimaion error x ( ) ˆx ( ) should no change if x () is replaced by x ()+p (). Thus he expression x ( )+p ( ) a ()[x ()+p ()] 378

3 should no depend on p (). Consequenly, a () p () =p ( ) (9) for any polynomial p () of degree d 1; he esimaor predics p ( ) perfecly from he values p (), T. For d =2, if we add a consan phase and frequency o x (), he esimae ˆx ( ) should auomaically adjus iself so ha he error does no change. An equivalen saemen of his invariance condiion is ha x ( ) ˆx ( ) M d (x). (10) We may also express (9) as a () j = j, j =0,...,d 1. (11) In marix form, Ga = g, where a = a ( 1 ) a ( n ) T, (12) n G =.., (13) d 1 1 n d 1 g = 1 d 1 T. (14) A random variable ˆx ( ) of form (8) ha saisfies he invariance condiion is called a linear invarian esimaor (LIE) of order d of x ( ) based on x (), T. Since n d, alieoforderd exiss, namely, he value a of he inerpolaing polynomial deermined by x ( 1 ),...,x( d ).For d =2, his LIE is he linear exrapolaor of x ( 1 ) and x ( 2 ) o, which was reaed by Boulanger and Douglas [5]. Some of he GACVs of Table I appear in heir formulas. Supposing ha 1 < 2 <, one can ofen opimize 2 1 o give an MSE ha is accepably close o he minimum MSE ha one can ge from LIEs based on he infinie pas 2 [4]. The linear and quadraic exrapolaors discussed by Vernoe e al. [6] are LIEs of order 2 and 3, respecively. The ACVs abulaed in [6] are relaed o he GACVs used here. Of all he order-d LIEs of x ( ), we wan he one wih he smalles MSE, called he bes LIE (BLIE) of order d. These of LIEs, call i {LIE}, is deermined by he inhomogeneous equaions (11) for a (). The difference of any wo LIEs is a random variable P b () x () whose coefficiens saisfy he corresponding homogenous equaions, ha is, i belongs o he linear subspace M d (x, T ). Thus, {LIE} is an affine se, a shifed version of M d (x, T ) ha does no pass hrough he origin. To find he closes poin of {LIE} o x ( ) in he mean-square sense, we drop a perpendicular from x ( ) o {LIE}. Thus,ˆx ( ) has o be a member of {LIE} and also has o saisfy he orhogonaliy condiion which means ha x ( ) ˆx ( ) M d (x, T ), (15) E[x ( ) ˆx ( )] Y =0whenever Y M d (x, T ). (16) Figure 1 should make he geomery clear. I can be proved from basic facs abou orhogonal projecions ha he BLIE ˆx ( ) exiss and is unique. Fig. 1. Md { LIE} ( x, T) x( * ) ˆx( * ) 0 Geomeric inerpreaion of he condiions for a BLIE Le x (T )= x ( 1 ) x ( n ) T T, b = b1 b n. (17) Then (16) can be rewrien as E x ( ) a T x (T ) h i x (T ) T b =0whenever Gb =0. (18) Boh facors on he lef side of (18) belong o M d (x); by Theorem 1, he expecaion can be evaluaed as if (4) were rue. Doing so gives he condiion r T a T R b =0whenever Gb =0, (19) where r is a column vecor and R asymmericn n marix formed from he GACV of x (): r = s x ( 1 ) s x ( n ) T, (20) R =[s x ( i j ):i, j =1,...,n]. (21) In urn, condiion (19) says ha he row vecor r T a T R is orhogonal o all vecors b ha are orhogonal o he rows of G. Therefore, r T a T R belongs o he row space of G, ha is, r Ra = G T θ for some d-vecor θ = T. θ 0 θ d 1 We now have he sysem of equaions Ga = g, Ra + G T θ = r, (22) which consiue n + d equaions in he n + d unknowns a, θ. They generalize he Yule-Walker equaions Ra = r (23) for saionary x (); in ha case, R is a genuine covariance marix, and ˆx ( ) is he orhogonal projecion of x ( ) on he unresriced subspace generaed by x (), T.Ford 1, R is symmeric if s x () is even, bu usually has boh posiive and negaive eigenvalues. Afer solving (22) for a and θ, we can calculae he mean square error of ˆx ( ) as follows: MSE = E [x ( ) ˆx ( )] 2 =E x ( ) a T x (T ) h i x ( ) x (T ) T a = s x (0) r T a a T (r Ra) 379

4 by Theorem 1. Then, by (22), MSE = s x (0) r T a g T θ. (24) We menion hree mehods for solving (22). Firs, if n = d, hen G is nonsingular, and we can solve (22) for a andhenfor θ. Second, if R is nonsingular 3, hen we can wrie a soluion in he following form (R 1 mehod): Λ = GR 1 G T, θ = Λ 1 GR 1 r g, a = R 1 r G T θ. Third, we can se up he sysem (22) as he single marix equaion R G T a r =, (25) G 0 θ g and ell Malab 4 o solve i in one operaion ( brue force mehod). Alhough Malab ofen says ha he big marix is badly scaled or nearly singular, he soluion seems o work anyway in he cases he auhor has ried. Before carrying ou his soluion, we should reduce he rend coefficien o zero as explained earlier, eiher subracing he rend from x () if c d is known, or increasing d by1if c d is unknown. Then ˆx ( ) is unbiased for x ( ) because x ( ) ˆx ( ) M d+1 (x), all of whose members have zero expecaion. The penaly for increasing d is a greaer MSE for he BLIE, because he se of LIEs shrinks as he order increases. A. Examples 1. Model: whie FM, h 0 =1, d =1, average frequency c 1 =0, T = {0, 1,..., 10}, =5.Wegeheexpeced resul: ˆx (5) = x (0), MSE = 1 2 h 0 = The same example, bu wih an unknown frequency (phase slope) added. We calculae he BLIE of order 2 o ge a predicor ha is invarian o he added frequency. The resul is he linear exrapolaor ˆx (5) = 3 2 x (0) 1 2x ( 10), wih MSE = This is a simple demonsraion of he penaly paid for a lack of knowledge of he rend. Bu, as T reaches farher and farher ino he pas, he esimaor recovers he unknown frequency, and he MSE ends o 1 2 h Model: whie FM (h 0 =2) + random walk FM (h 2 = ), d =2. Figure 2 shows normalized rms predicion error, MSE/τ, ofx ( + τ) for hree predicors: a) linear exrapolaion from x (),x( τ), whose rms error is 2τ imes Allan deviaion for τ; b) opimal predicion based on he enire discree pas x ( 10n), n =0, 1,..., calculaed in closed form by solving for he saionary covariance marix of akalmanfiler [7]; c) opimal predicion based on x ( 10n), n =0,...,5, calculaed by he mehod given here. 4. Model: flicker FM, h 1 =1, d =2.For =8,he squares in figure 3 show he BLIE regression coefficiens for he 33-poin predicion se T = { 32, 31,..., 0}. Because 3 The auhor does no know any condiions for R o be nonsingular. 4 Copyrigh by The MahWorks, Inc. rms predicion error / τ σ y (τ) pas = pas = predicion inerval, τ Fig. 2. Normalized rms error of predicors of a sum of whie FM and random walk FM. he coefficiens are small off he se { 32, 31, 1, 0}, we also ry he laer se for predicion (filled circles). Figure 4 shows MSE/τ for boh T ses and for = τ = 1, 2, 4,...,256. The lower horizonal line is for opimal predicion from he coninuous pas, 0. The upper horizonal line is for simple linear exrapolaion from T = { τ,0} as before. We observe only a small error penaly for using he 4- poin se insead of he 33-poin se. As τ becomes large, he linear exrapolaor becomes beer han he oher predicors, which see only 32 unis ino he pas of his long-memory process rms err = rms err = predicion inerval = Fig. 3. Regression coefficiens for predicing flicker FM IV. TREND ESTIMATION Le x () have saionary dh incremens (d 0), wih an unknown rend coefficien c d.then d x () =c d d! + x 0 (), (26) where x 0 () has rend coefficien zero. Again here will be invariance and orhogonaliy condiions o deermine he 380

5 rms predicion error /τ σ y (τ) (,0] { 32,...,0} { 32, 31, 1,0} polynomial deermined by x ( 1 ),...,x( d+1 ).Moreover,he difference of any wo LIEs is in he subspace M d+1 (x, T ), he se of P b () x () such ha b () saisfies he momen condiion of order d +1. As wih he predicion problem, a LIE ĉ d is he BLIE if i saisfies he orhogonaliy condiion By (28), (29), and (30), c d ĉ d M d+1 (x, T ). (33) c d ĉ d = a () x 0 (), (34) Fig. 4. predicion inerval,τ Normalized rms errors of predicors of flicker FM opimal esimaor of c d having he form ĉ d = a () x (). (27) The invariance condiion is deermined by he requiremen ha he esimaion error c d ĉ d be unchanged if a polynomial of degree d is added o x (). Thus he error is invarian o he rend c d d /d! iself as well as o polynomials p () of degree d 1 added o x (). Afer replacing (26) by x () = c d d /d!+x 0 ()+p(), we obain " # c d ĉ d = c d 1 1 a () d d! a ()[x 0 ()+p()]. (28) If c d ĉ d is invarian o c d and p (), hen a () j =0, j =0,...,d 1, (29) a () d = d!, (30) and he esimaor gives he righ answer c d if x () is a polynomial of degree d. A random variable of form (27) is said o be a LIE of c d if (29) and (30) hold. We wrie his in he marix form Ga = g again, bu wih differen definiions of G and g: n G =.., (31) d 1 d n g = 0 0 d! T. (32) Assume ha T has a leas d +1 poins. Then a LIE of c d exiss, namely, he coefficien of d /d! in he inerpolaing which belongs o M d (x 0 ) by (29). Also, M d+1 (x, T ) = M d+1 (x 0,T) M d (x 0 ), because every member of M d+1 (x) has coefficiens ha kill he rend. Therefore, (33) can be wrien as E a T x 0 (T ) h i x 0 (T ) T b =0whenever Gb =0, (35) where x 0 (T ) is a column vecor like x (T ). Because boh facors in he lef side of (35) belong o M d (x 0 ),wemayuse Theorem 1 for x 0 () o evaluae his expression, giving he condiion a T Rb =0whenever Gb =0, (36) where R is defined by (21) wih s x () replaced by s x0 (). By he same argumen as before, Ra = G T θ for some (d +1)-vecor θ = T.Wearriveahesysem θ 0 θ d of equaions Ga = g, Ra + G T θ =0, (37) which is like (22), excep ha now here are n+d+1 equaions and unknowns, he definiions of G and g are differen, and r has become 0. The same soluion mehods are available. By (33), ĉ d is unbiased for c d because all he members of M d+1 (x) have mean zero. By (34), Theorem 1, and (37), we can calculae he MSE of ĉ d by MSE = E (c d ĉ d ) 2 =E a T x 0 (T ) h i x 0 (T ) T a = a T Ra = g T θ = d!θ d. (38) A. Examples 1. Model: x () =whie FM + c 1 (unknown frequency), h 0 = 1, d = 1, T = {0, 1,...,10}. Resul: ĉ 1 = 1 10 [x (10) x (0)], MSE= Model: x () = x 0 () c 2 2 where x 0 () is whie FM, flicker FM, or random walk FM, T = {0, 1,..., 10}. The regression coefficiens for ĉ 2 are shown in Fig. 5. Even hough whie FM is in SI(1), we have o rea i as a member of SI(2) o exrac a quadraic rend, independenly of any linear rend ha may also be presen. For whie FM, ĉ 2 is also he slope of he leas-squares linear fi o he frequency daa. 381

6 Whie FM Flicker FM Rand. walk FM Fig. 5. Regression coefficiens for esimaing quadraic drif rae in he presence of whie FM, flicker FM, or random walk FM. V. ADDITIONAL RESULTS A. Predicion from equally spaced daa For fixed d, iakeso n 3 operaions o solve he Yule- Walker equaions (23) (when d =0) or heir generalizaion (22) by general linear equaion solving mehods, where n is he number of elemens in T.IfT is an equally spaced se of imes, however, hen R is a Toepliz marix. For saionary x (), he Levinson-Durbin algorihm [15], which is a loop on n, calculaes he regression coefficiens and he MSE in O n 2 operaions. The auhor has been able o exend his algorihm o he case d 1 while keeping he O n 2 propery for fixed d. In cases ha have been ried, he resuls of he wo general algorihms (R 1 andbrueforce)andheexendedlevinson algorihm agree wihin roundoff error. [3] P. Tavella and M. Goa, Uncerainy and predicion of clock errors in space and ground applicaions, 14h European Frequency and Time Forum, pp 77 81, Torino, [4] L.-G. Bernier, Use of he Allan deviaion and linear predicion for he deerminaion of he uncerainy on ime calibraions agains prediced imescales, IEEE Trans. Insrum. Meas., vol 52, no 2, pp , April [5] J.-S. Boulanger and R. J. Douglas, Traceabiliy of frequency conrol sysems, 1997 IEEE Inernaional Frequency Conrol Symposium, pp , Orlando. [6] F. Vernoe, J. Delpore, M. Brune, and T. Tournier, Uncerainies of drif coefficiens and exrapolaion errors: applicaion o clock error predicion, Merologia, vol 38, pp , [7] J.A.Davis,C.A.Greenhall,andR.Boudjemaa, Thedevelopmenof a Kalman filer clock predicor, 19h European Frequency and Time Forum, Besançon, [8] M. A. Weiss and C. Hackman, Confidence on he hree-poin esimaor of frequency drif, 24h Annual Precise Time and Time Inerval (PTTI) Applicaions and Planning Meeing, pp , [9] V. A. Logachev and G. P. Pashev, Esimaion of linear frequency drif coefficien of frequency sandards, 1996 IEEE Inernaional Frequency Conrol Symposium, pp [10] G. Wei, Esimaions of frequency and is drif rae, IEEE Trans. Insrum. Meas., vol 46, no 1, pp 79 82, Feb [11] C. A. Greenhall, A frequency-drif esimaor and is removal from modified Allan variance, 1997 IEEE Inernaional Frequency Conrol Symposium, pp [12] C. A. Greenhall, A srucure funcion represenaion heorem wih applicaions o frequency sabiliy esimaion, IEEE Trans. Insrum. Meas., vol IM-32, no 2, pp , June [13] C. A. Greenhall, The generalized auocovariance: a ool for clock noise saisics, The Tracking and Mission Operaions Progress Repor , pp 1 32, Je Propulsion Laboraory, May 1999, hp://mo.jpl.nasa.gov/mo/progress_repor/42-137/137j.pdf [14] F. Vernoe, Applicaion of he momen condiion o noise simulaion and o sabiliy analysis, IEEE Trans. UFFC, vol 49, no 4, pp , April [15] P. J. Brockwell and R. A. Davis, Time Series: Theory and Mehods, Second Ediion, 5.2, Springer, B. Trend from symmeric daa Suppose ha T is symmeric abou some poin, which we assume o be zero; hen T = T. I can be shown ha he opimal rend coefficien esimaor ĉ d has coefficiens ha are even (a ( ) =a ()) ifd is even, and odd (a ( ) = a ()) if d is odd. In eiher case one can se up equaions like (37) for a (), 0, and a smaller auxiliary vecor θ. Thus, he dimension n + d +1 of he sysem (37) can be reduced by approximaely a facor of wo. This work was carried ou a he Je Propulsion Laboraory, California Insiue of Technology, under a conrac wih he Naional Aeronauics and Space Adminisraion. REFERENCES [1] A. M. Yaglom, Correlaion heory of processes wih random saionary nh incremens, American Mahemaical Sociey Translaions, series 2, vol 8, pp , [2] J. A. Barnes and D. W. Allan, An approach o he predicion of coordinaed universal ime, Frequency, pp 1 8, Nov Dec