Comments on New Approaches in Period Analysis of Astronomical Time Series by Pavlos Protopapas (Or: A Pavlosian Response ) Don Percival

Transcription

1 Comments on New Approaches in Period Analysis of Astronomical Time Series by Pavlos Protopapas (Or: A Pavlosian Response ) Don Percival Applied Physics Laboratory Department of Statistics University of Washington Seattle, Washington, USA

2 Background: I let s set the stage by considering some background suppose we have observations y j of brightness of a star at times x j, to be modeled as where y j = g(x j ) + j, j = 1, 2,..., n, g(x) is a periodic function with unknown period p 0 j represents observational noise problem of estimating p 0 and g(x) has received considerable attention in statistical and astronomical literature (Hall, 2008, has good review of statistical approaches) 1

3 Background: II toy parametric version of problem: assume g(x) = A cos (2π 1 p 0 x + φ) x j = j (i.e., regular sampling) with 2 < p 0 j s are realizations of independent and identically distributed (IID) random variables (RVs) with finite variance σ 2 consider example with A = 1, p 0 = 1, φ = 0.72, σ = 0.5, = 1/15.3 and n = 50 2

4 g(x) Toy Parametric Model x 3

5 Toy Parametric Model with Regular Sampling g(x j ) x j 4

6 Toy Parametric Model with Additive Noise y j x j 5

7 Toy Parametric Model with Additive Noise y j x j 6

8 Background: III one solution to toy problem: form periodogram I(f) = nx 2 n x j e i2πfj Ø Ø j=1 (direct function of frequency f; indirect function of period 1/f) let ˆf be frequency at which I(f) is maximized over (0, 1/[2 ]) so that 1/ ˆf is estimate of period p 0 7

9 I(f) Periodogram for Example of Toy Parametric Model f 8

10 Background: IV here ˆf = 0.98, which is very close to truth f 0 = 1/p 0 = 1 Q: with just n = 50 observations, did we just get lucky? Whittle (1952) and Walker (1971) show E{ ˆf} f 0 and var { ˆf} C n 3, where C = 6σ2 (Aπ ) 2, with approximations improving with increasing n contrast with problem of estimating mean of IID RVs ζ j with variance σ 2 ζ using sample mean ˆζ = P j ζ j/n, for which var {ˆζ} = σ2 ζ n note: can also estimate g(x) in obvious ways 9

11 Background: V more realistic nonparametric version of problem: assume g(x) has r bounded derivatives x 1 x 2 x n are irregularly spaced, but with spacing dictated by IID nonnegative RVs (some conditions needed) j s same as in toy problem consider same example as before, with sample points dictated by uniform distribution 10

12 g(x) Toy Parametric Model x 11

13 Toy Parametric Model with Irregular Sampling g(x j ) x j 12

14 Toy Parametric Model with Additive Noise y j x j 13

15 Toy Parametric Model with Additive Noise y j x j 14

16 Background: VI for given p, let x j = x j pbx j /pc, where bxc is largest integer strictly less than x; i.e., wrap x j s circularly to get x j s 15

17 Example of Nonparametric Model (p = 0.8) y j x j 16

18 Example of Nonparametric Model (p = 0.8) y j x j 17

19 Example of Nonparametric Model (p = 1.0) y j x j 18

20 Example of Nonparametric Model (p = 1.0) y j x j 19

21 Example of Nonparametric Model (p = 1.2) y j x j 20

22 Example of Nonparametric Model (p = 1.2) y j x j 21

23 Example of Nonparametric Model (p = 0.8) y j x j 22

24 Example of Nonparametric Model (p = 1.0) y j x j 23

25 Example of Nonparametric Model (p = 1.2) y j x j 24

26 Background: VII define Nadaraya Watson estimator: P j ĝ(x p) = y jk j (x p) Pj K j(x p), where K j(x p) = K([x x j ]/h), K is a kernel (smoother), and h is its bandwidth define squared-error criterion: nx S(p) = yj ĝ(x j p) 2 j=1 ˆp = argmin S(p) is a consistent estimator of p 0 under regularity conditions, E{ˆp} p 0 and var {ˆp} C n 3 25

27 Background: VIII same n 3 convergence rate as in toy parametric problem (!), and yields estimate of g(x) also mission accomplished... or is it? many questions/issues: computationally feasible with data explosion? nonlinear optimization? choice of bandwidth? asymptotics versus finite sample? outliers? correlation? multivariate? highly irregular?... apologizes needed for advocating a non-bayesian approach!? bottom line: room for other procedures, but Hall et al. (2000) is serious competition (ain t sexy, but snickers will be ignored!) 26

28 Two New Approaches correntropy approach is similar in spirit to Hall et al. in use of circular wrapping, but has different procedure for selecting p 0 heuristic approach whose theoretical underlinings need further exploration Gaussian processes fundamentally different from circular wrapping adapts technique originally designed for 2D spatial processes to irregularly sampled time series incorporates Bayesian ideas, so has firmer theoretical backing bottom line: both approaches are of interest! 27

29 Correntropy: I for regularly sampled time series (i.e., x j = j ), periodogram I(f) = nx 2 n x j e i2πfj Ø Ø j=1 is Fourier transform of biased estimator of autocovariance sequence: ŝ τ = 1 n n τ X j=1 x j x j+τ, 0 τ n 1, with ŝ τ = 0 for τ n, and ŝ τ = ŝ τ. regarding x j as realization of RV X j, ŝ τ is estimate of s τ = E{X j X j+τ } for second-order stationary process 28

30 Correntropy: II correntropy idea is to consider E{κ(X j, X j+τ )}, where 1 κ(x j, x k ) = p exp (x j x k ) 2! 2πσ 2 κ 2σκ 2 σ κ is user-settable parameter controlling influence of higherorder moments (25 settings entertained in IEEE SPL paper) assume E{κ(X j, X j+τ )} is independent of j (sufficient condition is strict stationarity) unbiased estimator of V τ = E{κ(X j, X j+τ )} is bv τ = 1 n τ n τ X j=1 κ(x j, x j+τ ), 0 τ n 1, with b V τ = b V τ (note: biased estimator performs poorly) 29

31 Correntropy: III correntropy spectral density (CSD) is taken to be where P (f) = V = n 1 X τ= (n 1) 1 2n 1 ( b V τ V )e i2πfτ n 1 X τ= (n 1) as with periodogram, search for peaks in P (f) as a first test, reconsider regularly sampled toy time series bv τ y j = cos (2πj ) + j 30

32 I(f) Periodogram for Example of Toy Parametric Model f 31

33 P(f) CSD for Example of Toy Parametric Model, σ κ = f 32

34 P(f) CSD for Example of Toy Parametric Model, σ κ = f 33

35 P(f) CSD for Example of Toy Parametric Model, σ κ = f 34

36 P(f) CSD for Example of Toy Parametric Model, σ κ = f 35

37 P(f) CSD for Example of Toy Parametric Model, σ κ = f 36

38 P(f) CSD for Example of Toy Parametric Model, σ κ = f 37

39 P(f) CSD for Example of Toy Parametric Model, σ κ = f 38

40 P(f) CSD for Example of Toy Parametric Model, σ κ = f 39

41 P(f) CSD for Example of Toy Parametric Model, σ κ = f 40

42 estimated frequency from P(f) Estimation of f 0 : CSD P (f) vs. Periodogram I(f) P (f) I(f) bias SD RMSE (10,000 replications) estimated frequency from I(f) 41

43 Correntropy: IV Q: what is best way to pick σ κ? Q: how are peaks in CSD related to harmonic variations? Q: evidently P (f) can be negative, so in what sense can it be regarded as a density function? for irregularly sampled series, must resort to slotting procedure (unappealing, but necessary) note: locations of peaks in CSD refined through maximization of information potential (IP) metric, so inaccuracies in CSD might be irrelevant Q: computational issues? Q: robustness? 42

44 Gaussian Processes Approach: I similar to before, let y j = g(x j ) + j be model for observations y j of brightness of a star at times x j letting y = [y 1,..., y n ] T & g = [g(x 1 ),..., g(x n )] T, consider y N (g, σ 2 I n ), while g θ N (0, K θ ), where K θ is an n n covariance matrix whose (j, k)th element is given by K θ,j,k = β exp sin 2 (2πf 0 [x j x k ])/l 2 with θ = [β, f 0, l] T K θ is covariance matrix for samples from a continuous parameter stationary process with autocovariance function γ(τ) = β exp sin 2 (2πf 0 τ)/l 2 43

45 Gaussian Processes Approach: II to estimate f 0, need to compute potentially messy likelihood: 1 2 yt K θ + σ 2 I n 1 y 1 2 log Ø ØØ K θ + σ 2 I n Ø ØØ + C, alternatively, could specify Gaussian process capturing periodic or quasi-periodic variations indirectly within a state-space framework, thus allowing use of Kalman filter (KF) can adjust basic structural model (BSM) used in econometrics, which assumes regular sampling, to work with irregular sampling (can be handled readily by state space models and KF) BSM has terms for periodic variations (seasonal component γ j ) and quasi-periodic variations (business cycle ψ j ): y j = µ j + γ j + ψ j + j, where µ j is trend, and j is an error term 44

46 Gaussian Processes Approach: III dynamics of seasonal component dictated by pair(s) of state variables patterned as follows: µ µ µ µ γj+1 cos (λs ) sin (λ = s ) γj ωj +, sin (λ s ) cos (λ s ) where γ j+1 with ω j and ω j µ ωj ω j N (0, σ 2 ωi 2 ), γ j ω j being uncorrelated with previous disturbances σ 2 ω = 0 yields harmonic process with frequency dictated by λ s, which can be treated as known or estimated σ 2 ω > 0 yields nonstationary process; i.e., amplitudes & phases can evolve 45

47 Gaussian Processes Approach: IV business cycle dictated by single pair of state variables: µ µ µ ψj+1 cos (λc ) sin (λ ψj+1 = ρ c ) ψj sin (λ c ) cos (λ c ) ψj + where µ κj N (0, σκi 2 2 ), κ j µ κj κ j, with κ j and κ j being uncorrelated with previous disturbances for ρ < 1, component is a stationary ARMA(2,1) process with associated period 2π/λ c above can (presumably!!!) be adjusted to work with irregular sampling, thus providing a Gaussian process with covariance matrix implicitly defined by ρ, λ c and σκ 2 (have interpretations analogous to those for l, f 0 and β in γ(τ) dictating K θ ) 46

48 Gaussian Processes Approach: V for problem of interest here, entertain either y j = γ j + j or y j = ψ j + j as appropriate models for capturing periodicity advantage is ability to evaluate likelihood function in a computationally easy manner (maintains Bayesian framework) Gaussian process approach does not appear to have any computational advantages over Hall et al. (2000) 47

49 Conclusions period estimation is in theory an easy statistical problem (n 3 rate of convergence), so could entertain strategy of trading efficiency for computation speed-up (direction for future research?) correntropy and Gaussian process approaches are of interest, but need to compare carefully with existing methods looking forward to new developments from Protopapas et al.! 48

50 Thanks to... Eric and Jogesh for the invitation to participate and for their considerable efforts in putting SCMA V together 49

51 References J. Durbin and S. J. Koopman (2001), Time Series Analysis by State Space Models, Oxford, England: Oxford University Press P. Hall (2008), Nonparametric Methods for Estimating Periodic Functions, with Applications in Astronomy, in COMPSTAT 2008: Proceedings in Computational Statistics, 18th Symposium, Porto, Portugal, Paula Brito, editor, Physica-Verlag, Heidelberg, 2008 pp P. Hall, J. Reimann and J. Rice (2000), Nonparametric Estimation of a Periodic Function, Biometrika, 87, pp A. C. Harvey (1989), Forecasting, Structural Time Series Models and the Kalman Filter, Cambridge, England: Cambridge University Press P. Huijse, P. A. Estévez, P. Zegers, J. C. Principe and P. Protopapas (2011), Period Estimation in Astronomical Time Series Using Slotted Correntropy, IEEE Signal Processing Letters, 18, pp S. J. Koopman and K. M. Lee (2009), Seasonality with Trend and Cycle Interactions in Unobserved Components Models, Journal of the Royal Statistical Society Series C (Applied Statistics), 58, pp