Frequency Estimation, Multiple Stationary Nonsinusoidal Resonances With Trend 1

Transcription

1 Freqency Estimation, Mltiple Stationary Nonsinsoidal Resonances With Trend 1 G. Larry Bretthorst Department of Chemistry, Washington University, St. Lois MO 6313 Abstract. In this paper, we address the problem of freqency estimation when mltiple stationary nonsinsoidal resonances oscillate abot a trend in nonniformly sampled data when the nmber and shape of the resonances are nknown. To solve this problem we postlate a model that relates the resonances to the data and then apply Bayesian probability theory to derive the posterior probability for the nmber of resonances. The calclation is implemented sing simlated annealing in a Markov chain Monte Carlo simlation to draw samples from this posterior distribtion. From these samples, sing Monte Carlo integration, we compte the posterior probability for the resonance freqencies given the model indicators as well as a nmber of other posterior distribtions of interest. For a single sinsoidal resonance, the Bayesian sfficient statistic is nmerically eqal to the Lomb-Scargle periodogram. For a nonsinsoidal resonance this statistic is a straightforward generalization of both the discrete Forier transform and the Lomb-Scargle periodogram. Finally, we illstrate the calclations sing data taken from two different astrophysical sorces. INTRODUCTION Freqency estimation sing data obtained from astrophysical sorces presents some niqe challenges becase of the natre of the resonances and the data. The data are almost always nonniformly sampled. The nmber of resonances is sally nknown. The shape of each resonance may differ. Each resonance may have both amplitde and phase modlation. Finally, the resonances may oscillate abot an nknown trend. These difficlties make the se of the discrete Forier transform and the Lomb-Scargle periodogram, [1,, 3], problematic at best and misleading at worst. To solve any problem sing Bayesian probability theory, one mst have a model that relates the hypotheses of interest to the available data. In this first preliminary analysis, the model is of mltiple nonsinsoidal stationary resonances oscillating abot a trend, so, for now, we are going to ignore amplitde and phase modlation. Note that the presence of trend may make the signal nonstationary, bt the resonances we are considering are not changing shape as a fnction of time, i.e., the resonances are stationary. One way nonsinsoidal resonances manifest themselves in the Forier transform power spectrm and the Lomb-Scargle periodogram is as peaks at integer mltiples of a fndamental resonance freqency. These peaks, called harmonics, expand the shape of the resonance, called a light crve in astrophysics, in a Forier series. In this paper we will compte the 1 in Bayesian Inference and Maximm Entropy Methods in Science and Engineering, C. W. Williams ed., pp 3-, American Institte of Physics, 3.

2 posterior probability for the nmber of stationary nonsinsoidal resonances independent of the light crve of the varios resonances given the data and the prior information. This calclation yields a sfficient statistic that redces to a discrete Forier transform power spectrm and a Lomb-Scargle periodogram in the appropriate limits. Conseqently, the calclation generalizes the discrete Forier transform to accont for the nonsinsoidal natre of the resonances. THE MODEL As noted in the introdction, to solve any problem sing Bayesian probability theory one mst relate the hypotheses of interest to the available data. The mltiple resonance model which expands the nknown shape of the resonances in a harmonically related series may be written as d i = n T T l L l (t i ) + l= m j= n j A jk cos(πk f j t i ) + B jk sin(πk f j t i ) + error (1) k=1 where d i is the data vale acqired at time t i. The first sm represents the trend, where n T is the nmber of expansion fnctions, L l (t i ), needed to represent the trend down to the noise. The amplitde of the lth expansion fnctions is T l. These expansion fnctions cold be polynomials, a Forier expansion or any other convenient set of fnctions. In the program that implements this calclation we sed orthogonal polynomials becase it made the program comptationally more stable, bt the se of orthogonal polynomials is not reqired. In the doble sm, the nmber of resonances has been designated as m and it shold be nderstood that when m =, the sm over the sinsoids does not appear. The light crve of the jth resonances has been harmonically expanded in a Forier series having n j terms. The cosine amplitde of the kth harmonic of the jth resonance has been labeled A jk, and similarly the corresponding sine amplitde is B jk. Finally, the fndamental freqency of each resonance is f j. Conseqently, k f j is the freqency of the kth harmonic of the jth resonance. This model eqation has a nmber of interesting and important special cases. For example, when none of the expansions are present, this model redces to the single stationary freqency model and will reprodce all of the reslts given in [4, 5]. As another example, when only the trend expansion is present, this model redces to a trend pls a sinsoid and sing it will effectively detrend a data set. In a similar vein, when a single freqency is harmonically expanded this model is looking for nonsinsoidal resonances and will reslt in a sfficient statistic that is a generalization of the Lomb- Scargle periodogram to this important class of problems. So, while this model does not treat the nonstationary resonance case, it nonetheless contains a wealth of important special cases and we will have more to say abot them when we apply the calclations to astrophysical data. Writing the model in the form of Eq. (1) is convenient for explaining where each of the terms comes from, bt it is a inconvenient comptationally and for applying Bayesian probability theory. In Eq. (1) there are a nmber of different expansion fnctions and

3 sinsoids with amplitdes mltiplying them. We are going to designate these amplitdes as A l {T,...,T nt,a 11,...,A mnm,b 11,...,B mnm } and their corresponding sinsoid or expansion fnctions as G l ( f 1,..., f m,t i ). The expansion fnctions are not freqency dependent, while all of the sinsoids are; so the notation, G l ( f 1,..., f m,t i ), is more general than needed the sense that some of these fnctions are not freqency dependent and some are. Nonetheless, we are designating these fnctions as if they were freqency dependent jst as a reminder that some of them are. With this notation, the model can be written as d i = A l G l ( f 1,..., f m,t i ) + error () l= where the total nmber of fnctions,, is given by 1 + n T + m n j. (3) j=1 The 1 does not appear if n T =. The order of the one-to-one mapping of the sinsoids and expansion fnctions from Eq. (1) to Eq. () does not matter in the sense that the sfficient statistics we derive will be order independent. The important point is that Eq. () is easier to work with and performing the appropriate Bayesian calclations will be relatively straightforward. We wish to solve two closely related problems, first estimate the nmber of resonances, and second, knowing the nmber of resonances estimate the freqencies. Estimating the nmber of resonances is the more fndamental problem and, as it will trn ot, if we can solve this problem, estimating the freqencies may be done as a derivative calclation. Conseqently, in what follows we will derive the posterior probability for the nmber of resonances given the data and the prior information. This posterior probability is denoted as P(m DI), where m is the nmber of resonances, D stands for all of the data, and I represents all of the prior information, for example the fnctional form of the model, the maximm nmber of resonances, etc. The posterior probability for the nmber of resonances is a marginal posterior probability where all of the effects we do not care abot have been removed sing the sm and prodct rles of probability theory. The effects we do not care abot inclde the amplitdes, the nmber of expansion fnctions and the nmber of harmonics needed to represent the light crve of each resonance down to the noise. To compte this posterior probability, one applies Bayes theorem to obtain P(m DI) = P(m I)P(D mi). (4) P(D I) The nmerator consists of two terms, the prior probability for the nmber of resonances given only the prior information, P(m I), and the direct probability for the data given the nmber of resonances and the prior information, P(D mi). The denominator, P(D I), is a normalization constant given by P(D I) = max max P(mD I) = P(m I)P(D mi) (5) m= m=

4 where max is an pper bond on the nmber of resonances. The prior probability for the nmber of resonances has been sfficiently simplified that it cold be assigned, however the direct probability for the data has not. The direct probability for the data is a marginal probability density fnction, where the marginalization is over the nisance parameters. As noted, these inclde the amplitdes, the nmber of trend expansion fnctions, the nmber of harmonics and, when P(m DI) is being compted, the fndamental freqency of each resonance. If all of the amplitdes, the nmber of harmonics and the freqencies are designated by the nadorned symbols A, n and f respectively, then the direct probability for data may be compted as P(D mi) = M T M H n T = n 1 =1 M H n m =1 da da d f 1 d f m dσ P(DAn f n T σ mi) (6) where P(DAn f n T σ mi) is the joint probability for the data and the nisance parameters. We are sing M H to indicate the maximm nmber of harmonics. Also, we have added one parameter, σ, to represent what is known abot the general scale of the errors. Applying the prodct rle to the right-hand side of this eqation one obtains P(D mi) = M T M H n T = n 1 =1 M H n m =1 da da P(An f n T σ mi)p(d An f n T σmi) d f 1 d f m dσ (7) where P(An f n T σ mi) is the joint prior probability for the parameters given the nmber of resonances, and P(D An f n T σmi) is the direct probability for the data given the parameters. As an aside, note that we can derive a nmber of other interesting probabilities sing the samples from a Markov chain Monte Carlo simlation. To see how this is possible, note that the prodct, prior direct probability, is proportional to the joint posterior probability for all of the parameters given the model. So after obtaining samples from the posterior probability for the nmber of resonances, the samples with a given nmber of resonances can be sed to generate estimates of the varios parameters inclding the freqencies and nmber of harmonics, etc. Contining with the calclation, the direct probability for the data, the likelihood, cannot be frther simplified and, shortly, we will have no choice bt to assign a nmerical vale to represent it. However, the joint prior probability for the nisance parameters has not been sfficiently simplified for assignment. We contine simplifying this probability by repeated application of the prodct rle: P(An f n T σ mi) P(σ I)P(n T I) [ ]( m P(A l I) l= k= P( f k mi) [ MH ]) P(n k mi) n k =1 where we have assmed logical independence of the varios parameters, that is to say knowing the vale of an amplitde tells s nothing abot the nmber of harmonics or freqencies, etc. With this factorization the posterior probability for the nmber of (8)

5 resonances may be written as P(m DI) M T P(m I) j= M H n 1 =1 P(σ I)P(n T I) P(D An f n T mi). M H n m =1 [ P(A l I) l= da da ]( m P( f k mi) k= d f 1 d f m dσ [ MH ]) P(n k mi) n k =1 (9) We have now reached the point in this calclation where we have no choice bt to assign nmerical vales to represent these probabilities. For the standard deviation of the noise prior probability, we will assign a Jeffreys prior probability. In model selection calclations assigning an improper prior probability can case problems becase it introdces a singlarity into the calclations. However, here it does not case a problem becase the probability for every model contains exactly the same singlarity and so it cancels ot when these probabilities are normalized. For the discrete variables we are going assign a exponential prior probability of the form: P(n LM) = C { exp{ n} if n {,1,...,M} otherwise (1) where n represents the hypotheses, for example the nmber of harmonics or the trend expansion order. The nmber C is a normalization constant and M is the maximm of the discrete variable. The normalization constant is easily compted for a discrete exponential becase 1 1 x = x + x 1 + x + + x M + xm+1 1 x where x = 1/e. Rearranging this somewhat, one obtains (11) 1 x M+1 1 x = x + x 1 + x + + x M. (1) Identifying the right-hand side of this eqation as 1/C, the normalization constant, C, is given by 1 C = 1 xm+1 = C = 1 x. (13) 1 x 1 xm+1 We will assign a niform prior probability for the freqencies. However, if yo examine the model yo will note an interesting thing: reordering the resonances still gives yo an m resonance model. This seemingly trivial fact means that the posterior probability for the freqencies has m! identical maxima. For example in a three freqency model, the posterior probability has the property that P( f 1 f f 3 DI) = P( f 3 f f 1 DI) = P( f f 3 f 1 DI), etc. These identical maxima do not correspond to different physical phenomena, they simply correspond to different identifications of what is meant by freqency f 1, f, etc. In this calclation we will order the freqencies so that f 1 is the

6 lowest freqency, f is the second lowest freqency, etc. With this convention we have restricted orselves to a sbspace whose volme is m! smaller than the fll space and m! P( f j mi) = (H L) m If L f 1 f f m H (14) m j=1 otherwise where H and L are given high and low freqencies. For the amplitdes, P(A j I), we will assign a Gassian prior probability of the form: ( ) πσ 1 P(A j I) = exp{ β β } G j j A j G j j σ where this is an nbonded zero mean Gassian. The sqared length of the model fnction, G j j defined in Eq. (1) below, is a nmber that ensres we are sing the same prior information for all of the amplitdes. For sinsoids G j j N/ and is almost freqency independent. Finally, the hyperparameter β is a gess at how strongly we believe the amplitde is zero relative to what the data are telling s. If β.1, then this prior will change the maximm likelihood estimate of the amplitdes in abot the third decimal place. In this calclation we will take β as a given and fix its vale at β =.1. Althogh in a Markov chain Monte Carlo simlation there is no reason why β cold not be treated like any other nknown nisance hypotheses and removed sing marginalization. If we now sbstitte the priors into the posterior probability for the nmber of resonances, Eq. (9), and assign the likelihood sing a Gassian prior probability for the errors, one obtains m! P(m DI) exp{ m} (H L) m M T n T = M H n 1 =1 M H n m =1 d f 1 d f m da 1 da dσ σ C nt C n1 C nm exp{ n T n 1 n m } ( ) πσ 1 ( πσ ) { N β exp Q } G j j σ j= where we have added a sbscript to the normalization constant, C x, to indicate which prior probabilities these constants are associated with. Also constants that cancel on normalization have been dropped. The fnction Q is defined as Q j= = Nd β G j j A j + l= N i=1 A l T l + [ d i j= l= k= A l G l ( f 1,..., f m,t i ) ] (15) (16) A j A k g jk (17)

7 where we again remind the reader, that when the nmber of model fnctions is zero, i.e., there is no signal, none of the sms in the above expression appear. The mean-sqare data vale, d, is defined as d 1 N N i=1 d i (18) where N is the total data vales. The projection of the data onto the jth model fnction, T j, is defined by and g jk is defined as T j N i=1 where δ jk is the Kronecker delta fnction [9] and G jk N i=1 d i G j ( f 1,..., f m,t i ) (19) g jk G jk (1 + β δ jk ) () G j ( f 1,..., f m,t i )G k ( f 1,..., f m,t i ). (1) The amplitde integrals are Gassian qadratre integrals and evalating sch integrals is straightforward, one obtains M m! T M H M H dσ P(m DI) exp{ m} (H L) m d f 1 d f m n T = n 1 =1 n m =1 σ C nt C n1 C nm g jk 1 ( ) exp{ n T n 1 n m } β 1 G j j ( πσ ) N exp { Nd h σ where g jk is the determinant of the g jk matrix. The sfficient statistic, h, is defined as h = } j= () T j A j (3) j= and is the total sqared projection of the data onto the model (see [4] for more on this type of sfficient statistic). Note that in the above the sfficient statistic is a fnction of the nmber of resonances, the freqencies, the trend expansion order and the nmber harmonics for each resonance. We cold have indicated this in the notation, bt it wold have clttered things p more than it already is. Conseqently, when dealing with this qantity keep in mind that it is a fnction of both the continos and discrete parameters. Finally, the expected amplitdes, A j, are the amplitdes that maximize the joint posterior probability for all the parameters and are given by A l g jl = T j. (4) l=

8 The integral over the standard deviation of the noise prior probability can be transformed into a gamma fnction integral and we omit the details of evalating this integral, one obtains m! P(m DI) exp{ m} (H L) m M T n T = M H n 1 =1 exp{ n T n 1 n m } g jk 1 M H n m =1 j= d f 1 d f m C nt C n1 C nm ( β G j j ) 1 [ Nd h (5) where we have dropped some additional constants that cancel on normalization. The posterior probability for the freqency is of the form of Stdent s t-distribtion [1] and it is in this form that we will apply it in the next section. The calclation has proceeded as a model selection calclation and like most model selection calclations it has bried within it a parameter estimation calclation. In particlar the qantity on the last line of Eq. (5) is proportional to the posterior probability for the resonance freqencies given the model indicators, P( f 1... f m m,n T,n 1,...,n m,d,i) g jk 1 j= ( β G j j ) 1 [ ] N ] N Nd h (6) where some additional qantities that are constants for a given model have been dropped. In the following section we will apply both Eq. (5) and Eq. (6) to two examples sing astrophysical data sets. EXAMPLES USING ASTROPHYSICAL DATA In this section we are going to apply the calclations presented in the previos section to two astrophysical data sets. Initially we will apply the calclations to a data set where the calclation will redce to single stationary freqency estimation, then we will apply it to a more complicated case to illstrate the behavior of the calclation when a nonsinsoidal resonance is present. The first data set we will analyze is data from the X-Ray Binary LS I This data is shown in Fig. 1(C) along with the error bars given by Gregory [11]. Gregory analyzed this data sing a Bayesian calclation for the freqency independent of the shape of the resonance. He finds a single resonance with a 68% credible region from 1599 to 166 days. To analyze this data sing the calclations given in the previos section we are going to initially assme that Gregory is correct and analyze this data sing a single resonance model, then we will se the fll calclation to determine the nmber of resonances and whether the resonance is sinsoidal. Assming there is only a single stationary sinsoidal resonance, we will apply Eq. (6) with a constant offset, one harmonic, and a single freqency. Conseqently, the model redces to a single stationary sinsoidal resonance oscillating abot a constant offset. Becase Eq. (6) has only a

9 1 (A).6 (B) 5.5 Log P(Period DI) -5 P(Period DI) Period in Days Fig. 1. Panel (A) is the natral logarithm of the posterior probability for a period. Note that from the peak the natral logarithm of the posterior probability for the period drops by abot 13 so the posterior probability is really qite sharp as illstrated in the flly normalized posterior probability for the period, Panel (B). Panel (C) is a plot of the data and the model evalated for the parameters that maximized the posterior probability for the period. Flx Period in Days 4 (C) Time in Days FIGURE 1. Bayesian Analysis of X-Ray Binary LS I single nknown parameter, the period (the reciprocal of the freqency), we can scan the period over any interval we choose. In this first demonstration we will scan this period from 1 p to 3 days and plot the natral logarithm of Eq. (6). This plot is displayed in Fig. 1(A). The spectrm of white noise is niform when plotted as a fnction of freqency. When the natral logarithm of the posterior probability is compted for white noise, it is also niform. However, when plotted as a fnction of period, the magnitde of the oscillations remains niform bt the width of the oscillations stretch for long periods. This stretching is an artifact de to plotting as a fnction of period. For long periods we are seeing the region arond zero freqency. So the rapid oscillations at short period and broad oscillations at high periods are the noise. Bt what abot the region arond the maximm, near 165 days? The maximm is abot 8 and from this peak the logarithm of the posterior probability drops to roghly -5; a drop of abot 13, a seemingly small nmber. Bt when this posterior probability is normalized, Fig. 1(B), 13 e-foldings (abot 44,413) is more than adeqate to sppress all of the sprios oscillations and yields a very symmetric

10 1 (A) 1 (B).8.8 P(m DI).6.4 P(n1 m=1,d,i) No Of Periods No Of Harmonics Given One Period FIGURE. Panel (A) is the posterior probability for the nmber of periods, Eq. (5). In the nmerical simlation, all of the simlations settled into a model with one period. Panel (B) is the posterior probability for the nmber of harmonics given the nmber of periods is one. This posterior probability was derived from the otpt form the Markov chain Monte Carlo simlations. After the annealing phase of the Monte Carlo simlation, all of the models contained exactly one period and this period had exactly one harmonic. posterior probability. If one ses the mean and standard deviation to estimate the period, then the period is estimated to be 165 ± 5. In good agreement with the estimate given by Gregory. In Fig. 1(C) we have overlayed the data with the model generated from the parameters that maximized the posterior probability for the period, where the amplitdes sed in this plot were compted from Eq. (4). When this model is overlayed onto the data the period virtally jmps ot at one giving good visal confirmation that the period is really there. However, becase we have sed Eq. (6) we have essentially assmed the signal is present. The only way to be certain that the period is present is to apply Eq. (5) to this data set where we allow the possibility that the nmber of resonances, m, may be zero. This calclation is illstrated in Fig., where panel (A) is the posterior probability for the nmber of periods and panel (B) is the posterior probability for the nmber of harmonics given a one period model. The nmerical calclations are implemented sing Markov chain Monte Carlo with simlated annealing. In the annealing phase of the calclation, an annealing parameter is introdced into Eq. (5). If the annealing parameter is designated as α, then the annealing parameter raises everything in Eq. (5), except the prior probability for the nmber of resonances, exp{ m}, to the α power. As the simlations begin, α =, and everything in Eq. (5) is wiped ot except the prior probability for the nmber of resonances. So the simlations start by exploring the prior probability for the nmber of resonances. The program that implements the Markov chain Monte Carlo analysis rns mltiple independent simlations. In this case 1 simlations were processed. When the simlations are initialized, the parameters (the nmber of periods, nmber of harmonics, the periods, etc.) are all set by drawing samples from their respective prior probabilities. When the annealing parameter is zero, the Markov chain is exploring the prior probability for the nmber of resonances. As the annealing parameter increases slowly from zero to one, the other terms in Eq. (5) become more and more important.

11 The priors move the simlations towards simpler models, while the data tends to move the simlations towards more complex models. The Markov chain balances these two forces to select the nmber of resonances and harmonics. Moves between models tend to occr when the annealing parameter is small. As this parameter is increased, moves across models tend to stop becase sch moves decrease the posterior probability: either the fit to the data is mch worse, or the fit to the data is essentially nchanged bt prior probabilities decrease, so the model is rejected. The simlations sally settle into a distribtion poplating only a few of the terms in the sms in Eq. (5) becase only high probability models contribte significantly to these sms. Similarly, as a fnction of the periods, the simlations clster abot the largest peak in the integrand as a fnction of the periods; again becase it is only this peak location that contribtes significantly to the integrals. In the example we are discssing, all of the simlations finished the analysis having one period, one harmonic, and the periods were all clstered near 165. So this analysis jst confirms what Gregory has already indicated privately; the resonance is a single stationary sinsoid. Next we want to illstrate the behavior of this calclation as the resonance becomes nonsinsoidal. To do this we will se a data set from the MACHO project. This ongoing project has been bilding a cataloge of millions of stars with high signal-to-noise observations for over 1 years. The people working on this project, in particlar Dog Welch, were kind enogh to point me at a nmber of rather nsal variable stars. These stars are Blazhko-type RR Lyrae variables. These RR Lyrae s have natral variations on time scales of abot half a day and also weeks or months. The case of this behavior is still not nderstood. The data shown in Fig. 3 as the asterisks are magnitdes in the ble region of the spectrm. They are nonniformly sampled covering abot 3 days. If the period is abot half a day, then the rn of the data cover some 6 or so oscillations; a tremendos rn of data. Becase the precision of a stationary freqency estimate varies inversely with the total sampling time, and ths inversely with the total nmber of periods covered by the rn of the data, we can expect resonances in this data to be very precisely determined. The apparent strctre in Fig. 3 reflects the sampling scheme, not the resonances in the data. There are gaps in the data that span hndreds of days and it is these gaps that the eye is natrally drawn to. On the scale of this plot, a period of half a day that has been nonniformly sampled wold look like noise. Finally, the pls signs on this plot shold be ignored for now. They are a fnctions of the residals compted from a model we will discss shortly. The sfficient statistic generated by the Bayesian calclation is very closely related to a discrete Forier transform and when no trend is present and the data are niformly sampled, it redces approximately to the discrete Forier transform power spectrm. Similarly, when no trend is present, bt the data are nonniformly sampled, the nmerical vale of this statistic is eqal to the vale of the Lomb-Scargle periodogram, see [5]. So in a sense this sfficient statistic is a generalization of the discrete Forier transform to the case when trend and nonsinsoidal resonances are present. We wold like to illstrate the behavior of this statistic in two regimes: first when it essentially redces to a Lomb-Scargle periodogram, Fig. 4(A) and second when the resonances are nonsinsoidal, Fig. 4(B). Panel (A) is the natral logarithm of the posterior probability for the period, the reciprocal of the freqency, given a stationary sinsoid oscillating abot a constant trend. The presence of mltiple peaks in the sfficient statistic, and ths in the

12 Magnitde Time in Days FIGURE 3. MACHO RR Lyrae variable : This data, the asterisks, are stellar magnitdes in the ble region of the spectrm. The resonance freqency is on the order of half a day, so on the scale of this plot, individal oscillations are not visible. The strctre apparent in this data jst reflect the sampling schedle, i.e., how often the telescope actally stares at the star. For this data the larger gaps span hndreds of days. logarithm of the posterior probability, reflects the presence of nonsinsoidal resonances, the sampling scheme, as well as the presence of mltiple resonances. Panel (B) is the natral logarithm of the posterior probability for a period given a nonsinsoidal resonance oscillating abot a constant trend. The nonsinsoidal resonance was expanded in a fifth order harmonic series, so there are five times as many peaks in this plot as in panel (A). If the expansion had been tenth order, there wold have been ten times as many peaks. These peaks occr whenever the freqency of any sinsoid in the model matches either a fndamental or harmonic freqency in the data. When these freqencies match, the projection of the data onto that sinsoid increases and ths the sfficient statistic, and the logarithm of the posterior probability for a freqency has a maximm. Note that every freqency with a large probability density in panel (A) is also prominent in panel (B), bt with an even larger density. This increase in probability is de to the harmonics. Harmonic models are n freqency models, when all n freqencies in the model match n freqencies in the data, the total sqared projection of the model onto the data has a maximm and the posterior has a global maximm. It is only this global maximm that is relevant to nonsinsoidal freqency estimation. Both panel (A) and (B) are the logarithm of the posterior probability for a single

13 4 35 (A) 6 5 (B) Log P(Period DI) Log P(Period DI) Period in Days Period in Days FIGURE 4. Panel (A) is the natral logarithm of the posterior probability for a single stationary sinsoid oscillating abot a constant trend and is very closely related to both the discrete Forier transform and the Lomb-Scargle periodogram. The peaks in this plot reflect both the nonsinsoidal natre of the resonances as well as the presence of mltiple resonances. Panel (B) is the natral logarithm of the posterior probability for a single stationary nonsinsoidal oscillation abot a constant trend and is a generalization of the discrete Forier transform and the Lomb-Scargle periodogram. Note that in both cases these statistics have mltiple peaks, however, the presence of those peaks reflects the model sed, the sampling scheme, as well as evidence for mltiple resonances. period, so it is only the single largest peak in these two different log probabilities that are important in estimating the period. In the panel (A) the largest peak has a height of approximately 375 while the second largest peak has a height of abot 19. If we were to normalize this posterior probability the second peak wold be abot 8 orders of magnitde smaller then the peak near.5. If yo examine panel (B) yo will find the difference between the height of the two largest peaks is essentially nchanged so again it is only the peak near.5 that is relevant. In this calclation the plotted fnction, thogh closely related to the periodogram, is a probability density rather than a power spectrm. Ths the amplitde of the largest peak is inversely proportional to the freqency ncertainty. In panel (A) the highest peak climbs from something slightly greater than zero to 375, while the peak in panel (B) climbs from something slightly less that zero to roghly 675, so one wold expect that the model sed to generate panel (B) will make a freqency estimate that is roghly two times more precise than the model sed to prodce the plot in panel (A). Normally to gain a factor of in the precision of a parameter estimate, one wold have to qadrple the nmber of data vales or doble the signal-to-noise ratio. In this example inclding the prior information abot the harmonics is roghly eqivalent to dobling the effective signal-to-noise becase the harmonic expansion fits the data mch better. Conseqently, the mean-sqare residal has been redced by roghly a factor of 4, and so the estimated signal-to-noise ratio is abot a factor of two improved. One might wonder if two models having sch very different strctre will even agree in their parameter estimates to an order of magnitde. To illstrate that the period estimate doesn t vary mch as the nmber of harmonics increases, look at Fig. 5(A). The crve with the lowest peak in this plot is the posterior probability for a stationary sinsoid oscillating abot a constant, while the highest peak is the posterior probability

14 (A).6.4 (B) P(Period DI) Resonance Intensity Period in Days Days Harmonics: Harmonics: FIGURE 5. Each crves in Panel (A) is the posterior probability for the period, as the nmber of harmonics increases from the stationary sinsoidal case, the crve with the lowest peak, to the 6 harmonic case the precision of the freqency estimate becomes better and better. Panel (B) shows the light crve for increasing nmber of harmonics. Note that that as the nmber of harmonics increases, the light crve converges to a well defined shape that is fairly independent of the nmber of harmonics. for the period sing a 6 harmonic model. As jst indicated, the estimated period is abot a factor of two better when the 6 harmonic model is sed. The remaining crves are the period estimates from the, 3, 4 and 5 harmonic models. Note the precision of the period estimates becomes better and better for increasing nmbers of harmonics. One might wonder if this trend wold contine indefinitely, the answer is no; the height of the posterior probability for the period deceases after the six harmonic model. In addition to displaying the posterior probability as a fnction of the nmber of harmonics, we have also displayed the light crve for models containing one throgh six harmonics, panel (B). Here the light crve starts ot as a sinsoid, as it mst, and then rapidly converges to a nonsinsoidal shape that is fairly independent of the nmber of harmonics sed in the model. In the example sing the data obtained by Gregory, we plotted both the data and the model signal generated from the parameters that maximized the posterior probability for a stationary resonance, Fig. 1(C). This plot helped to gide the eye in identifying the presence of the signal. We wold like to do something like that here, bt it really is not possible to overlay the model signal and the data becase the rn of the data is so long there wold be more than 6 oscillations to plot. Any sch overlay wold look like a solid mass nless we zoomed into a small section of the data and then we wold not see how well the data as a whole are represented by the model. The way this problem is typically solved in astrophysics is to generate a folded light crve. In a folded light crve one sbtracts the estimated trend from the data, and then plots the data modlo the period of the primary resonance, in this case the period at days. This has some significant advantages in that it compresses all of the data onto a single period. One can then overlay this folded light crve with the light crve from the model and everything can be compared nicely in a single plot. The folded light crve of the data along with the light crve generated from the 6 harmonic model are shown in Fig. 6. Note that the light crve of the data appears rather noisy, bt, in spite of this, the light crve of the 6 harmonic model seems to fit the folded light crve

15 FIGURE 6. The folded light crve of the data has been overlayed with the light crve generated from the 6 harmonic model. Note how this model light crve follows the noisy light crve of the data. Bt the noise level of the data is mch smaller than the deviations seen in this light crve. Indeed these deviations are not noise, bt are actally evidence of additional resonances in the data, see text for details. of the data rather well. However, we mst add one cationary note, the noise level of the data is mch mch smaller than the apparent noise in the folded light crve. Folding the data in this way prespposes a single resonance. If there are mltiple resonances, then these resonances are also folded bt with the wrong period. Conseqently, these periods appear as if they were noise becase the nonniform sampling, samples them qasi randomly. We will have more to say abot the apparent noise level in this plot shortly. As an alternative method of presentation, note that the plot of the logarithm of the posterior probability for a single stationary sinsoid oscillating abot a constant trend, Fig. 4(A), is relatively simple and does contain information abot the resonances and the harmonics. If we plot the logarithm of the posterior probability for this data and for a model signal generated from the parameters that maximized the posterior probability then we wold be able to see how mch of the strctre in Fig. 4(A) is acconted for by the model signal. We noted before that this log posterior is essentially the Lomb-Scargle periodogram and in the discssion which follows we will refer to this log posterior

16 5 4 Log P(Freqency DI) Freqency (cycles per day) FIGURE 7. We have plotted the logarithm of the posterior probability for a single stationary resonance oscillating abot a constant offset for the data and the model signal compted from the maximm of the posterior probability for a model containing six harmonics. The log posterior probability for the model has been shifted pward and to the right so that one can examine both plots careflly, and jst below these two crves we have plotted their difference. probability as the Lomb-Scargle periodogram. We wish to compare the Lomb-Scargle periodogram of the data to that calclated from the time domain model signal generated from the maximm posterior probability estimate of the period. We wish to do this to see visally how mch of the strctre in the Lomb-Scargle periodogram is acconted for by the model. Althogh we have not yet applied the general formalism to this data, we will anticipate one reslt and se a six harmonic model. So we will generate a time domain model from the parameters that maximize the posterior probability for a single resonance given a six harmonic model. The Lomb-Scargle periodogram of the data and this time domain model signal are shown in Fig. 7. The periodograms in this plot have been plotted as a fnction of freqency, not period; the harmonic strctre of the resonances are more apparent as a fnction of freqency. We have shifted the Lomb-Scargle periodogram of the time domain model signal pward and to the right becase when it was plotted withot shifting, these two periodograms overlapped exactly; one simply cold not see the differences. The shifted Lomb-Scargle periodogram was generated from data prodced sing a time domain model of a single resonance with six harmonics. A six harmonic model contains exactly six stationary sinsoids. Yet its Lomb-Scargle periodogram contains

17 at least 15 larger peaks, thosands of smaller peaks, and no noise; where did these peaks all come from? The answer is that these psedo alias peaks are the reslt of the sampling schedle. If one generates a niformly sampled time domain model signal from the same parameters sed to generate the shifted crve shown in Fig. 7 and then comptes the Lomb-Scargle periodogram, one obtains exactly six peaks as one shold. We have plotted this Lomb-Scargle periodogram as the dotted line shown in Fig. 7. This Lomb-Scargle periodogram was shifted pward, and its height was scalded to make it fit conveniently on this plot. Conseqently, one shold ignore both the height and width of this periodogram, the location of the peaks is the only important item. Three of these six peaks are visible in Fig. 7, the other three are otside of the spectral window shown. The three visible peaks sit right on top of the three harmonics associated with the fndamental period near 1.8 cycles per day. Essentially all of the other strctre in the Lomb-Scargle periodogram of this data is de to the sampling schedle. In addition to the three periodograms discssed so far, Fig. 7 also contains a plot of the difference between the periodogram of the data and the model signal. This difference has been shifted downward and is shown at the bottom of the figre. These small differences are de to two additional resonances in this data. To detect these additional periods we ran the Markov chain Monte Carlo simlations that implement the fll Bayesian calclations. When we ran the simlation, we specified 6 days as the maximm period. We did this for two reasons, first when the single resonance model is sed and one examines periods longer than 6 days, nothing is fond in the Lomb-Scargle periodogram; while not proof that no long periods exists, it is nonetheless indicative that none exist. Second, assming there are three resonances, the single resonance model estimated the period to abot one part in 1 6, if we assme similar resoltion for the other resonance, then the maximm of the posterior occpies a volme of abot 1 18 /36 1 of the total volme; a tremendos volme for any algorithm to search. Conseqently, when we ran the simlations we wished to keep the maximm period as small as possible. When we ran this simlation, the posterior probability for the nmber of resonances had a maximm at three resonances. We have plotted some of the otpts from this analysis in Fig. 8. Panels (A), (C) and (E) are the posterior probability for the three freqencies. Each of these freqencies have been determined to abot six significant figres. Panels (B), (D) and (F) are the posterior probabilities for the nmber of harmonics in these resonances. Resonance 1 and 3 are stationary sinsoidal resonances. Only resonance is nonsinsoidal, reqiring 6 harmonics to expand the light crve. Resonance is the single largest resonance in the data and it was this resonance that was picked ot by the single freqency model. The shape of the light crve generated from this resonance is the same as that shown in Fig. 5 (for the 6 harmonic model) and in Fig 6. However, the shape of the light crve for all three resonances taken together is very different from the shape of the light crve for resonance alone. To illstrate this we generated a model signal from the parameters that maximized the joint posterior probability for the parameters given a 3 resonance model. We then folded this model signal in exactly the same way the data were folded and displayed it in Fig. 6. If yo examine Fig. 6, especially in the expansion on the lower right, yo will see that the light crve of the model signal overlaps the light crve of the data almost exactly. The noise like behavior in the light crve of the data, is not noise at all, rather it is the other two resonances that cannot be acconted for in

18 .6 (A) 1 (B).5 P(Period D,3,1,I).4.3. P(Harmonics D,3,1,I) Period 1 in Days No Of Harmonics, Period 1.6 (C) 1 (D).5 P(Period D,3,,I).4.3. P(Harmonics D,3,,I) Period in Days No Of Harmonics, Period.6 (E) 1 (F).5 P(Period D,3,3,I).4.3. P(Harmonics D,3,3,I) Period 3 in Days No Of Harmonics, Period 3 FIGURE 8. Panels (A), (C) and (E) are the posterior probabilities for the three freqencies. Panels (B), (D) and (F) are the posterior probabilities for the nmber of harmonics given the three freqency model. a folded light crve. As noted, these other resonances when sampled qasi randomly appear as if they were noise to the eye. In addition to the folded light crve of the model signal and other otpts discssed, we have also plotted the residals, the difference between the data and the model (the same model sed to prodce the folded light crve). These residals, pls signs, are plotted on top of the data in Fig. 3. We have displaced these residals by the constant offset so that they may be plotted on the same scale as the data. There is no apparent pattern in these residals. However, the error bars on the data are mch smaller than the flctations visible in the residals. Given the size of the error bars these residal

19 flctations are real in the sense that there are flctations in the data that are not acconted for by nonsinsoidal resonances. There are at least two possible sorces for these flctations: the star cold have flctations other than periodicities, and there cold be systematic errors in the data, for example atmospheric flctations, that cannot be controlled by the experimenters. Almost certainly both contribte to these flctations. Two of the three resonances in this data are very close together. It wold be an interesting qestion to determine if, for some reason, the fndamental freqency of a single resonance shifted. To make this determination involves the calclations presented in this paper with two additional model selection calclations: first determine if a change occrred and then determine if the light crve is the same before and after the change. Sch calclations wold indeed be very interesting bt they are beyond the scope of this paper. SUMMARY AND CONCLUSIONS Nonsinsoidal resonances manifest themselves in the discrete Forier transform and the Lomb-Scargle periodogram as peaks at integer mltiples of a fndamental freqency. These mltiple peaks essentially expand the shape of the resonance in a Forier series. This fact can be sed to model nonsinsoidal resonances and relates the resonance freqencies to the data. This model contains both continos parameters and discrete parameters and one mst devise a formalism that allows one to estimate both types of parameter. For Bayesian probability theory, making sch an estimate is a matter of straightforward calclation: no new principles are needed. However, from a comptational standpoint the difficlties of exploring mltiple discrete dimensions and mltiple continos parameters are formidable. As explained in the previos section, the maximm of the joint posterior probability for the parameters can occpy an incredibly small fraction of the total hyper-volme. Nonetheless this Bayesian calclation can be implemented sing Markov chain Monte Carlo with simlated annealing. In the simlated annealing phase of the nmerical calclation, the program comptes the posterior probability for the model, ths selecting the nmber of resonances, the nmber of harmonics and fndamental freqencies. After the annealing phase completes, samples from the simlations are sed to approximate varios posterior distribtions inclding, the posterior probability for the nmber of resonances, the resonance freqencies and the nmber of harmonics for each resonance. Ths the se of simlated annealing, Markov chain Monte Carlo and Bayesian probability theory can solve the problem of mltiple nonsinsoidal freqency estimation, while simltaneosly determining the nmber of resonances, the light crve of each resonance and the trend. ACKNOWLEDGMENTS The athor wishes to acknowledge spport from NIH grants CA-836, NS-3591, and NS Also, this paper tilizes pblic domain data obtained by the MACHO

20 Project, jointly fnded by the US Department of Energy throgh the University of California, Lawrence Livermore National Laboratory nder contract No. W-745-Eng- 48, by the National Science Fondation throgh the Center for Particle Astrophysics of the University of California nder cooperative agreement AST , and by the Mont Stromlo and Siding Spring Observatory, part of the Astralian National University. REFERENCES 1. Lomb, N. R. (1976), Astrophysics and Space Science, 39, pp Scargle, J. D. (198), Astrophysical Jornal, 63, pp Scargle, J. D. (1989), Astrophysical Jornal, 343, pp Bretthorst, G. Larry (1988), Bayesian Spectrm Analysis and Parameter Estimation, in Lectre Notes in Statistics, 48, J. Berger, S. Fienberg, J. Gani, K. Krickenberg, and B. Singer (eds), Springer- Verlag, New York, New York. 5. Bretthorst, G. Larry, Nonniform Sampling: Aliasing and Bandwidth, in Bayesian Inference and Maximm Entropy Methods in Science and Engineering, edited by Rychert, J. T., G. J. Erickson and C. R. Smith, American Institte of Physics, New York, 1999, AIP Conference Proceedings 567, pp Bayes, Rev. T. (1763), Philos. Trans. R. Soc. London 53, 37; reprinted in Biometrika 45, 93 (1958), and Facsimiles of Two Papers by Bayes, with commentary by W. Edwards Deming Hafner, New York, See also [7]. 7. Bayes, Rev. T. (1763), Philos. Trans. R. Soc. London 53, Jeffreys, H. (1939), Theory of Probability, Oxford University Press, London. 9. Kronecker, L. (191), Vorlesngen über Zahlentheorie, Tebner, Leipzig; repblished by Springer- Verlag, Stdent (198), Biometrika, 6, Gregory, P. C. (1999) Astrophysical Jornal, 5, p Gregory, P. C. and Thomas J. Loredo (199), Astrophysical Jornal, 398, p. 146.