WAVELET FREQUENCY DOMAIN APPROACH FOR TIME-SERIES MODELING

Transcription

1 1. Introduction WAVELET FREQUENCY DOMAIN APPROACH FOR TIME-SERIES MODELING Ranit Kumar Pau Indian Agricutura Statistics Research Institute Library Avenue, New Dehi Autoregressive integrated moving average (ARIMA) methodoogy (Box et a., 007), which is a parametric approach, has virtuay dominated anaysis of time-series data during ast severa decades. Here roe of various expanatory variabes enter into the mode impicity through response variabe observations at past epochs. However, quite often it is not possibe to postuate appropriate parametric form for the underying phenomenon and, in such cases; Nonparametric approach is caed for. Accordingy, in recent years, an extremey powerfu methodoogy of Waveet anaysis is rapidy emerging (Vidakovic, 1999; Perciva and Waden, 000). Athough, a number of research papers have been pubished deaing with various theoretica aspects of waveets, their appication to data is sti a difficut task. Waveet anaysis can be studied in two ways: one is in time domain another is in frequency domain. In respect of the former, Sunikumar and Praneshu (004) appied waveet threshoding approach for modeing and forecasting of monthy meteoroogica subdivisions rainfa in Eastern U. P., India. For the atter approach, Amasri et a. (008) have recenty proposed a test statistic by using waveet decompositions to test the significance of trend in a time-series data. The most difficut probem of testing for inear trend is the presence of dependence among the residuas because of which, tests for trend based on the cassica ordinary east squares (OLS) regression are inappropriate. In many situations, the error autocovariance function exhibits a sow decay refecting the possibe presence of ong memory process. The waveet anaysis, however, has been extensivey used for such purposes, since it suitaby matches the structure of these processes. The autocovariance function of the waveet transformed series exhibits different behaviour, in the sense that autocovariance functions of the transformed series decay hyperboicay fast at a rate much faster than the origina process. In genera, the series that are correated in the time domain become amost uncorreated in the waveet domain. Agricutura performance of a country, generay, depends to a arge extent on the quantum and distribution of rainfa. So its accurate modeing is vita in panning and poicy making. Accordingy, severa attempts have been made in the past to deveop modes for describing rainfa. In Indian context, Raeevan et a. (004) have provided an exceent review of mutipe and power regression modes empoyed since 1988 aong with various modifications made in these modes from time to time, particuary in the identification of reevant expanatory variabes. The purpose of this ecture is to discuss and appy waveet methodoogy in frequency domain for estimation and testing of significance of trend in India s monsoon rainfa data during the period 1979 to 006.

2 . Basics of Waveets The term waveet is used to refer to a set of basic functions with a very specia structure which is the key to the main fundamenta properties of waveets and their usefuness in statistics. Waveets are fundamenta buiding bock functions, anaogous to the trigonometric sine and cosine functions. As with a sine or cosine wave, a waveet function osciates about zero. This osciating property makes the function a wave. However, the osciations for a waveet damp down to zero, hence the name waveet. If (.) be a rea vaued function defined over the rea axis, and satisfying two basic properties: (i) The integra of (.) is zero: ( u) du 0 (ii) The square of (.) integrates to unity: ( u) du 1 Then the function (.) is caed a wave..1 Discrete Fourier transform Transformation of a function to its waveet components has much in common with transforming a function to its Fourier components. An introduction to waveets begins with a discussion of the usua Fourier transformation. The French mathematician ean- Baptiste Fourier discovered that any cass of square-integrabe functions, defined on the interva [-π, π], can be decomposed into component functions constructed by standard trigonometric functions. A function f beongs to the square-integrabe space L [a, b] if b f x dx a Fourier s resuts states that any function f L [-π, π] can be expressed as an infinite sum of diated cosine and sine functions given by 1 f ( x) a0 a cos( x) b sin( x) (1) 1 where 1 a f ( x)cos x. dx = 0, 1,, 1 b f ( x)sin x. dx = 1,, The series expansion is regarded as a transform, taking a function f into a set of coefficients a and b. The Fourier series expansion is extremey usefu in that any L function can be written in terms of very simpe buiding bock functions: sines and cosines, because the set of functions {sin(.), cos(.), =1,, }, together with the constant function, form a basis for the function space L [-π, π] which is orthonorma. A sequence of functions {f } are orthonorma if the f s are pairwise orthogona and if f =1, for a. M III-65

3 . Waveet anaysis versus Fourier anaysis There is an obvious anaogy between waveet anaysis and Fourier anaysis in the sense that both the techniques aim to represent a function as a inear superposition of basis functions. In the case of waveet anaysis, the basis functions are the waveets {ψ,k } whereas in Fourier anaysis they are the exponentias, {e iwx =coswx+isinwx}. The most obvious difference is that the waveet basis are indexed by two parameters ( and k) and have an infinite set of possibe basis functions whereas Fourier basis functions are indexed by the singe parameter w and have ony a singe set of basis functions. Comparing Fourier and waveet anayses, the essentia point is that the sines and cosines of the standard Fourier anaysis have specificity ony in frequency, whereas the specia structure of a waveet basis provides specificity in ocation (via transation) and aso specificity in frequency (via diation). This means that waveet anaysis provides information not ony on what frequency components are present but aso when or where they are occurring. Another notabe feature is that the waveet transforms of a function are ocaized, i.e, are time varying and depend ony on the properties of the function in the neighborhood of each time point. This impies that if the function has singuarities (such as discontinuities or spikes ), these wi affect ony the waveet transform near the singuarities. By contrast the Fourier transforms depends on the goba properties of the function and any singuarity in the function wi affect a such transforms. Hence waveets have significant advantages over basic Fourier anaysis when the function under study function has singuarities..3 Time domain versus Frequency domain The most common representation of signas and waveforms is in the time domain. However, most signa anaysis techniques work ony in the frequency domain. The concept of the frequency domain representation of a signa is quite difficut. The frequency domain is simpy another way of representing a signa. For exampe, consider a simpe sinusoid. The time - ampitude axes on which the sinusoid is shown define the time pane. If an extra axis is added to represent frequency, then the sinusoid woud be as iustrated beow. M III-66

4 The frequency - ampitude axes define the frequency pane in a manner simiar to the way the time pane is defined by the time - ampitude axes. This frequency pane is what is represented when the spectrum of a signa is shown. The frequency pane is orthogona to the time pane, and intersects with it on a ine which is the ampitude axis. Note that the time signa can be considered to be the proection of the sinusoid onto the time pane (time - ampitude axes). The actua sinusoid can be considered to be as existing some distance aong the frequency axis away from the time pane. This distance aong the frequency axis is the frequency of the sinusoid, equa to the inverse of the period of the sinusoid. The waveform aso has a proection onto the frequency pane. These two proections mean that the sinusoid appears as a sinusoid in the time pane (time - ampitude axes), and as a ine in the frequency pane (frequency - ampitude axes) going up from the frequency of the sinusoid to a height equa to the ampitude of the sinusoid. 3. Discrete Waveet Transform (DWT) There are two main waves of waveets. The first wave resuted what is known as the continuous waveet transform (CWT), which is designed to work with time-series defined over the entire rea axis; the second is the discrete waveet transform (DWT) which deas with series defined essentiay over a range of integers. DWT of a time-series observation is used to capture high and ow frequency components. This, in turn, woud enabe modeing of time-series data through computation of inverse DWT. The basic reason why the DWT is such an effective anaysis toos are the foowing: (i) The DWT re expresses a time-series in terms of coefficients that are associated with a particuar time and a particuar dyadic scae -1. These coefficients are fuy equivaent to the origina series in that we can perfecty reconstruct a time-series from its DWT coefficients. (ii) The DWT aows us to partition the energy in a time-series into pieces that are associated with different scaes and times. Energy decomposition is very cose to the statistica technique known as the anaysis of variance (ANOVA). (iii) The DWT effectivey decorreates a wide variety of time-series that occurs quite commony in the physica appications. This property is the key to the use of DWT in the statistica methodoogy. (iv) The DWT can be computed using an agorithm that is faster than the ceebrated fast Fourier transform agorithm. M III-67

5 Computation of DWT is carried out by Pyramid agorithm discussed beow: The first stage for computing the DWT simpy consists of transforming the time-series X of ength N = into the N/ first eve waveet coefficients W 1 and the N/ first eve scaing coefficients V 1. Precisey, to obtain unit scae waveet coefficients, time-series X t : t 0,..., N 1 is circuary fitered with fiter h, = 1,,, L-1, where L is the width of the fiter and must be an even integer. For h to have width L, it must satisfy the conditions: h 0 0 and h L-1 0. Now define h = 0 for < 0 and L so that h is actuay an infinite sequence wit at most L nonzero vaues. A waveet fiter must satisfy the foowing three basic properties: L 1 0 L 1 h 0, for a nonzero integers n. Compute 0 L 1 0 L1 h 1 and h h n h h n 0 ( t)mod N 0, W h X, t = 0,1,,N-1. () 1 / ~ 1, t Now define N/ waveet transforms for unit scae corresponding to t=0,,n/1 as L 1 1/ ~ W W h X, (3) 1, t 1,t 1 0 (t1 )mod N This procedure is caed Downsamping procedure. To obtain first stage scaing 1 coefficients, define scaing fiter g 1 hl 1. Then the first eve scaing coefficients are L 1 1/ ~ V V g X (4) 1, t 1,t 1 0 (t1 )mod N V,t X was treated in the first stage. Then we circuary fiter The second stage of Pyramid agorithm consists of treating t and g and subsampe to produce two new series, namey L 1 0 1,(t 1 )mod N V,t 1 in the same way as 1 separatey with h W V (5), t L 1 0 V V, t=0,1,,n/41. (6), t 1,(t 1 )mod N Above procedure is repeated times to obtain DWT s. There are - subsequent stages to the Pyramid agorithm. For = 3,,, the th stage transforms V -1 of ength N/ -1 into W and V each of ength N/. At the th stage, the eements of V -1 are fitered separatey with waveet fiterh, and scaing fiterg. The fiter outputs are subsamped to form respectivey W and V. The eements of V are caed the scaing coefficients for eve, whie those of W contain the desired waveet coefficients for eve. At the end of th stage, the DWT coefficient W is formed by concatenating the + 1 vectors. Let P be an NN rea vaued matrix defining the DWT and satisfying the orthonormaity property P`P = I N, where I N is the NN identity matrix. Then the DWT (W) of the timeseries vector X may be computed by W = P X. Now the eements of the vector W are decomposed into +1 subvectors. The first subvectors contains a of the DWT coefficients for scae. Then W can be written as M III-68

6 W W W... W 1 V 4. Mutiresoution Anaysis (MRA) Consider the waveet synthesis of X X PW 1 P W Q V, (7) where P and Q matrices are defined by partitioning the rows of P commensurate with the partitioning of W into W 1,, W and V. Thus the N / N matrix P 1 is formed from the n = 0 up to n = N/-1 rows pf P; the N / 4 N matrix P is formed from the n = N/ up to n = 3N/4-1 rows; and so forth, unti we come to the 1 N matrices P and Q, which are the ast two rows of P. Thus P P 1 P... P Q Now define D = P` W for = 1,,, which is an N dimensiona coumn vector whose eements are associated with changes in X at scae ; i.e., W = P X represents the portion of the anaysis W = PX attributabe to scae, whie P` W is the portion of the synthesis X = P`W attributabe to scae. Let S = Q`V which has a its eements equa to the sampe mean X. Then it can be seen that X D 1 S, (8) which defines a mutiresoution anaysis (MRA) of X; i.e., the time-series X is expressed as the sum of a constant vector S and other vectors D, = 1,, each of which contains a time-series reated to variations in X at a certain scae. D is caed the th eve waveet detai. 5. Estimation of Trend by Waveets Some times it is important to decompose a time-series into different components of variations ike, ow frequencies (trend), and high-frequency (noise) components. And the mutiresoution anaysis is used for decomposing and describing the ow frequencies and high-frequency components in the data in a scae by scae basis. Consider the foowing mode for a time-series data {X t }: X t = μ + T t + Z t, t = 0,..., N 1, (9) where μ is a constant term, T t is an unknown deterministic poynomia trend function of order r, Z t is a residua term which is a ong-memory process defined by 1 B Z t = t, where, is the ong memory parameter, { t } is a Gaussian white noise process with mean zero and > 0. Here, B, is the back shift operator such that BZ t = Z t-1. Now, since W W W.. W 1. V, the vector W can be written as sum of two vectors: W = W w + W s, where W w is an N 1 vector containing the waveet coefficients and zeros at a other ocations, and W s is an N 1 vector containing the scaing coefficients and zeros at a other ocations. Since X = P`W, therefore, M III-69

7 X = P`W = P` W s + P` W w = Tˆ Zˆ, (10) where Tˆ is an estimator of the poynomia trend T at eve, whie Ẑ is the estimate of residua Z. The issue of choosing the eve of the estimate depends on the goa of appication. shoud be chosen sma for detecting the oca trends and cyces. In other appications, is set to be arge, if the aim is to detect the goba trend. The orthonormaity of the matrix P impies that the DWT is an energy preserving transform so that N X t t1 X W (11) Given the structure of the waveet coefficients, the energy in X is decomposed, on a scae by scae basis, via so that X W 1 W V (1) W represents the contribution to the energy of {X t } due to changes at scae. whereas V represents the contribution due to variations at scae. So the estimated variance of the time-series in terms of waveet and scaing coefficients can be expressed as: 1 1 X N t1 N N 1 N vˆ ( ) ˆ X S 1 N 1 1 X t X W X W V X where vˆ X ( ) is the estimated variance of the waveet coefficients at scae, and ˆ S is the estimated variance of the trend. For testing the nu hypothesis H0: Trend = 0, Amasri et a. (008) proposed a test statistic that can discriminate between this nu hypothesis and the aternative hypothesis H1: Trend 0 is defined as foows: ˆ S G (14) vˆ ( ) 1 X The test statistics (N - N/ )/(N/ - 1)G wi foow an F distributed with (N/ - 1) and (N - N/ ) degrees of freedom shown (under the normaity assumption of the scaing coefficients). The distribution of the test statistic is unknown, however, in situations when the errors are not normay distributed and when they exhibit some form of dependency. It is, therefore, important to generate empirica critica vaues in such situations in order to investigate the properties of the test statistic. This is done by means of simuation experiments. The waveet estimate has the advantage over the Fourier transform in terms of the ocaization in time and frequency, which means that the detai of the estimate is seen to vary with t. This property provides additiona information of variabiity on different scaes (different ) in the case when there is a ong memory process, because such (13) M III-70

8 processes appear to be oca trends and cyces, which are, however, disappear after some time. An important issue is how to choose the waveet fiter. A centra factor to use a particuar waveet is to match the characteristics of the series anayzed. The Haar waveet, which is a piecewise constant function, preserves the discontinuities, and therefore it is most suitabe to identify a structura break in the data. By contrast, other waveets with L > are smoother and tend to bur the discontinuities. In genera, the waveets with a wider support (L is big) are smoother but spatiay ess ocaized, whie the waveets with a narrow support (L is sma) are more spatiay ocaized but ess smooth. 6. Basis Functions Every two dimensiona vector (x,y) is a combination of the vectors (1,0) and (0,1). These two vectors are said to be the basis vectors for (x,y) because mutipying x by (1,0) yieds the vector (x,0) and mutipying y by (0,1) yieds the vector (0, y). The sum of these two wi yied (x,y). Extending this theory to functions, the sines and cosines are the basis functions of the Fourier transform. Orthogonaity requirement for sines and cosines chosen can be set by choosing appropriate combination of sine and cosine function terms whose inner product add up to zero. The particuar sets of functions that are orthogona and that construct f(x) are the orthogona basis functions for the probem (Vidakovic, 1999). A variety of different waveet famiies now exist which enabe orthonorma waveet bases to be generated for a wide cass of function spaces. Two waveet bases viz, Haar and Daubechies systems are discussed here. 6.1 The Haar System The simpest waveet basis for L (R) is the Haar basis. The Haar function is a bonafide waveet, though not used much in practice, uses a mother waveet given by 1, 0 x < ½, ψ(x) = -1, ½ x 1, 0, otherwise The Haar waveet is piecewise constant over intervas of ength one-haf and can be expressed by a picture as foows (Fig.1). M III-71

9 M III: 5: Waveets for time-series anaysis `haar' mother, psi(0,0) Fig. 1. The Haar function Haar waveets possess the property of compact support, which means that it wi vanish outside of a finite interva. But if instead of a genera function in L (R), one wants to anayze a function with much ess or more reguarity, then the expansion given by the Haar system is inappropriate may be due to bad decay of coefficients at infinity. Again the Haar waveets are not continuousy differentiabe. Due to these drawbacks, Haar waveets are ess usefu in data anaysis. Repacing the scaing function in the Haar system by a more reguar function produces a system with a much better behavior with respect to smooth functions. Daubechies (199) proposed such a famiy of smooth waveet basis. 6. Daubechies Waveet Bases By imposing an appeaing set of reguarity conditions, Daubechies (199) came up with a usefu cass of waveet fiters, a of which yied a DWT in accordance with the notion of differences of adacent averages. The definition for this cass of fiters can be expressed in terms of the squared gain function for the associated Daubechies scaing fiters g, = 0,, L-1: L / 1 D L L / 1 G f cos f sin f, 0 where L is a positive even integer. D D Using the reationship H f G f 1/, the corresponding Daubechies waveet fiters have squared gain functions satisfying L / 1 D L L / 1 H f sin f cos f. 0 D H can be considered as the squared gain function of the equivaent fiter for a fiter cascade. Apart from the above, there are other famiies of smooth waveet bases that provide compacty supported orthonorma waveets and are continuousy differentiabe, ike those proposed by Stromberg, Meyer and Batte (Ogden, 1997). M III-7

10 Rainfa (mm) M III: 5: Waveets for time-series anaysis 8. An Iustration (Ghosh et a (010), and Pau et a (011)) For estimation of trend by waveet methodoogy, the Indian monsoon rainfa during the years 1879 to 006 is considered. The monsoon rainfa is cacuated as the sum of daiy rainfas from 1 st une to 31 st September of a year. The data set is obtained from the website ( of the Indian Institute of Tropica Meteoroogy, Pune, India. The rainfa data depicts a cycica variation with possiby a decining trend. The trend in the monsoon rainfa has been estimated through ARIMA methodoogy as we as by using waveets approach. Different waveets have been used for anayzing the rainfa data in a scae by scae basis to revea the ocaized nature of the data set. 8.1 Modeing of rainfa data in the framework of autoregressive process Assuming presence of deterministic inear trend in the rainfa series, foowing mode is fitted: Y t, t = 1,,, T (18) t t where t s are uncorreated with zero mean and constant variance. Let eˆ t Yt ˆ ˆ t The fitted trend equation is obtained as: Y t = t (14.6) (0.191) where the vaues within brackets ( ) denote corresponding standard errors of estimates. The trend is not significant at 5% eve of significance. The graph of trend is dispayed in Fig M III-73 Years Fig. 3. Trend in Indian monsoon rainfa data 8. Trend anaysis through waveet approach The discrete waveet transforms and the mutiresoution anaysis is done on the basis of Haar waveet, and Daubechies 4 (D4) waveet. The DWT coefficients are shown in Figure 5 and Figure 6. The waveet coefficients are reated to differences (of various order) of (weighted) average vaues of portions of X t concentrated in time. Waveet coefficients are potted as bars, up or down. The sizes of the bars are reative to magnitudes of coefficients. The number of waveet coefficients at the owest resoution eve (eve = 1) is exacty haf the number of origina data points and the number of coefficients decreases by haf at each eve (Nason and Sachs, 1999).

11 The coefficients at the top (beow) are high-frequency ( ow frequency ) information. The waveet coefficients do not remain constant over time and refects the changes of the data at various time-epochs. The ocations of abrupt umps can be spotted by ooking for vertica (between eves) custering of reativey arge coefficients. From the waveet coefficients potted above, the origina function can be reconstructed by using Inverse discrete waveet transform (IDWT). The above mentioned pattern can aso be verified from the mutiresoution anaysis (MRA) of the time-series exhibited in Figs. 7 and 8. idwt d1 d d3 d4 d5 d6 s Fig. 5. DWT by D4 waveet at eve 6 M III-74

12 Rainfa (mm) Rainfa (mm) M III: 5: Waveets for time-series anaysis idwt d1 d d3 d4 d5 d6 s Fig. 6. DWT by Haar waveet at eve 6 The estimate of trend of the rainfa data computed by Haar and D4 waveets for the eves 6 are given beow (Figure 9-10). As the eve increases the decining goba trend present in the data set is depicted ceary Years Fig. 9. Estimate of trend by Haar waveet at eve Years Fig. 10. Estimate of trend by Daubechies (D4) waveet at eve 6 M III-75

13 M III: 5: Waveets for time-series anaysis S D D5 Fig. 7. MRA by D4 waveet at eve 6 M III-76

14 Fig. 8. MRA by Haar waveet at eve 6 M III-77

15 The discrete waveet transform (DWT) and mutiresoution anaysis (MRA) of India s monsoon rainfa time-series data revea differentia behaviours at different time epochs at different scaes. Two waveets namey; Daubechies (D4) and Haar waveets are used for estimation of trend in the rainfa data. It is found that the monsoon rainfa in India is showing a decining trend over the years, which can have very serious repercussions from Goba Warming point of view. This important feature, however, coud not be captured by ARIMA methodoogy. References Amasri, A., Locking, H. and Shukur, G. (008). Testing for cimate warming in Sweden during , using waveets anaysis.. App. Stat., 35, Box, G. E. P., enkins, G. M. and Reinse, G. C. (007). Time-Series Anaysis: Forecasting and Contro. 3 rd edition. Pearson education, India. Daubechies, I. (199). Ten Lectures on Waveets. SIAM, Phiadephia. Ghosh, H., Pau, R. K. and Praneshu, (010). Waveet Frequency Domain Approach for Statistica Modeing of Rainfa Time-Series Data. ourna of Statistica Theory and Practice, 4 (4) Kukarni,. R. (000). Waveet anaysis of the association between the southern osciation and Indian summer monsoon. Int.. Cimato., 0, Nason, G. P. and von Sachs, R. (1999). Waveet anaysis in time series anaysis. Phiosophica Transactions of Roya Society of London, A 357, Ogden, T. (1997). Essentia Waveets for Statistica Appications and Data Anaysis. Birkhauser, Boston Pau, R. K., Praneshu, and Ghosh, H. (011). Waveet methodoogy for estimation of trend in Indian monsoon rainfa time-series data. Indian ourna of Agricutura Science, 81 (3), Perciva, D. B. and Waden, A. T. (000). Waveet methods for time series anaysis. Cambridge Univ. Press, U.K. Raeevan, M., Pai, D. S., Dikshit, S. K. and Kekar, R. R. (004): IMD s new operationa modes for ong range forecast of southwest monsoon rainfa over India and their verification for 003. Curr. Sci., 86, Sunikumar, G. and Praneshu (004). Modeing and forecasting meteoroogica subdivisions rainfa data using waveet threshoding approach. Ca. Stat. Assn. Bu., 54, Vidakovic, B. (1999). Statistica Modeing by Waveets. ohn Wiey, New York M III-78