Weakly Short Memory Stochastic Processes: Signal Processing Perspectives

Transcription

1 Weakly Short emory Stochastic Processes: Signal Processing Persectives by Garimella Ramamurthy Reort No: IIIT/TR/9/85 Centre for Security, Theory and Algorithms International Institute of Information Technology Hyderabad - 5 3, INDIA December 9

2 WEAKLY SHORT EORY STOCHASTIC PROCESSES: SIGNAL PROCESSING PERSPECTIVES Garimella Rama urthy, Associate Professor, International Institute of Information Technology, Gachibowli, Hyderabad-53, AP, INDIA ABSTRACT Traditionally wide sensor stationary (WSS) stochastic rocesses are classified as long memory and short memory rocesses based on the correlation coefficient series In this research aer, we roose a fine grain classification of WSS rocesses into weakly short memory stochastic rocesses using the associated set of indices Some roerties of the indices are roved The results are naturally extraolated to structured numerical and ower series Introduction: Probability theory originated in modeling games of chance Also, modeling dynamic games ( reeated exeriments ) led to the concet of a stochastic rocess characterized by the associated finite dimensional distributions By imosing some constraints on the finite dimensional distributions, strict and wide sense stationary stochastic rocesses were roosed These rocesses rovided tractable models of various natural and artificial henomena [PaP] In the case of wide sense stationary stochastic rocesses, the autocorrelation function summarizes the second order deendence of the underlying sequence of random variables Also, the autocorrelation coefficient sequence ( for different lags ), characterizes a wide sense stationary stochastic rocesses Let this sequence be denoted by { ρ ( s): s < } for non-negative lag values A wide sense stationary stochastic rocess was classified into the following two categories : (i) Short emory Stochastic Process and (ii) Long emory Stochastic Process based on the absolute summability of infinte series associated with the autocorrelation coefficient sequence The author felt that this classification was very coarse Thus, an effort to rovide a fine grain classification of WSS stochastic rocesses resulted in this research aer The classification is based on associating the autocorrelation coefficient sequence with an infinite set of indices This research aer is organized as follows In Section, discrete time weakly short memory stochastic rocesses are studied Section 3 considers the continuous time, weakly short memory stochastic rocesses In Section 4, results are generalized to structured

3 infinite series In Section 5, signal rocessing ersectives are included In Section 6, future research directions are included Finally, the conclusions are documented in Section 7 Weakly Short emory Stochastic Processes: Discrete Time Case: Consider a Wide Sense Stationary (WSS) discrete time stochastic rocess, X(t) Let the associated autocorrelation coefficient sequence be { ρ (s) } with s being the lag [PaP] The following definition is well known in the theory of such stochastic rocesses Definition: A discrete time, wide sense stationary stochastic rocess is called a short memory stochastic rocess if s < The following lemma follows easily Lemma : If X(t) is a short memory stochastic rocess with the associated autocorrelation coefficient sequence ρ (s) Then ρ ( s) is strictly equal to one for atmost finitely many lags s Proof: Suose not Thus, say that the autocorrelation coefficient is equal to one for infinitely many values of the lag Thus, consider the set B B { s: ρ ( } Suose the cardinality of B ( ie B ) is infinite ( countable ) Then, it is evident that + Z B Υ C, where C { s: ρ ( < } Thus, we necessarily have that + s s B + Hence X(t) is not a short memory rocess Thus we arrive at a contradiction Thus, we necessarily have that ρ (s) is equal to one for atmost finitely many values of the lag s QED Remark : Let L be the value of lag after which ρ (s) is strictly less than one Thus for the short memory stochastic rocess under consideration, we have the following inference: If s L, then ρ (s) or ρ (s) <

4 But for all s > L, ρ (s) < In view of this fact, let us redefine the sets B and C : B { s : s L} and C { s : s ( L + ) } () Now, let us consider the infinite dimensional vector of autocorrelation coefficient values for different lags ie ie ρ [ ρ ( ), ρ(), ρ() ] By definition, X(t) is a short memory stochastic rocess if the L norm of ρ is finite This interretation led us to consider the L norm of ρ for < < [Roy] Lemma : Let s For any discrete time WSS stochastic rocess, the following inequalities hold true ie 3 ie ' s constitute a monotone decreasing sequence Proof: By definition s ) + s ), s s B where the sets B and C are same as those defined in Remark Thus, we have + N + ρ( s) and + N + + ρ ( s) By the definition of sets B and C, we have N + N, with the strict inequality holding if for atleast one value of ρ ( < Also, it is clear that + s ) > ρ ( Hence, for all integer valuesof, we have that + Thus, QED 3 s B, Remark : The inequalities are strict for many interesting Wide Sense Stationary ( WSS ) discrete time stochastic rocesses Secifically, suose there exists an s B, for which ρ ( s) < Then > > 3 > ie

5 Lemma 3: For most interesting discrete time, Wide Sense Stationary (WSS), short memory stochastic rocesses, ( In the roof, we secify the required conditions ) < Proof: Suose there exists an s B, for which ρ ( s) < Also, suose the cardinality of set C is larger than or equal to one, then we have that Lt + < Hence, on the alication of the ratio test, we have that < QED Note: Using the modified definition of sets B, C and the integer L ( as in equation () ), we have that ρ( s L on exchanging the order of summation Thus we necessarily have that s L Remark 3: The set of quantities / indices ' s characterize a discrete time wide sense stationary stochastic rocess Equivalently, we define the following indices θ ' s : Let I s) Then s B s L s L θ I Suose, we consider a short memory stochastic rocess Then Lemma imlies that the indices s are all finite But in the case of an arbitrary discrete time stochastic rocess, it can haen that t and j < for all j t In the case of traditional classification, all such rocesses are broadly classified as long memory stochastic rocesses Thus this classification is very broad Using the following definition, we make a finer classification of long memory stochastic rocesses Definition: A discrete time wide sense stationary stochastic rocess is weakly short memory rocess of t th kind if

6 < for all j j t In the case of such stochastic rocesses, under reasonable conditions, we have that t < Remark 4: Using reasoning similar to the one utilized reviously, it is easy to see that t Lt Thus, using an aroriate truncation condition, any discrete time, short memory stochastic rocess can be characterized by finitely many indices s ie say {,,, l } Using, a similar aroach, long memory stochastic rocesses also can be characterized by finitely many indices Remark 5: Consider the vector of autocorrelation coefficients ie ρ [ ρ ( ), ρ(), ρ() ] Let this vector be associated with a discrete time short memory s stochastic rocess ie K < Consider the related vector ρ [ ), ρ(), ρ(),] and normalize all its comonents by the finite real number K Thus, we arrive at the following vector of robabilities ρ ρ() ) ),,, K K K K This robability mass function is naturally associated with the autocorrelation coefficient function Its moments will have interesting interretation 3 Weakly Short emory Stochastic Processes: Continuous Time Case: Now let us consider continuous time stochastic rocesses Secifically, we have the following definition Definition : A continuous time wide sense stationary stochastic rocess is called a short memory stochastic rocess if ρ( s) ds and <

7 As in the case of discrete time stochastic rocesses, we are led to the consideration of the following quantities [Roy]: ds Lemma 4: Using the above definition, we have that 3 for any continuous time, wide sense stationary stochastic rocess + R { s : } + { s : s) < } Proof: Let B Υ C We have that ds s ) ds + s) ds B m(b) + C C s ) ds ( m(b) denotes the Lebesgue measure of the set B ) Thus, if the stochastic rocess is a short term memory rocess, then m(b) < Goal : To rove that + for all ds m( B) + Now let us consider s) ds C C By the definition of set C, s C imlies that ρ ( < Hence, we necessarily have that + ρ ( s ) < s) Thus, we have + for all ie QED 3 Remark 6: The inequalities are strict if m( C ) > Under reasonable conditions, for a short memory continuous time, stochastic rocess, we have that ( as in Lemma 3 ) < ds

8 Remark 7: As in the case of discrete time, wide sense stationary stochastic rocesses, we have the following definition Definition: A continuous time wide sense stationary stochastic rocess is weakly short memory rocess of t th kind if t and j < for all j t For such a rocess, under reasonable conditions, we have that t < Thus, even in the case of continuous time, wide sense stationary stochastic rocesses, under reasonable truncation condition, finitely many indices ie s characterize the rocess Probability Density Function Associated with the Autocorrelation Function of a Continuous Time Short emory Process: Let ds Q Thus, on normalization, we necessarily have that ds Q Thus, g( s) is a robability density function associated with Q the autocorrelation coefficient function of a continuous time, short memory stochastic rocess It has interesting interretation in terms of the lag The moments of such a function will be of utility in characterizing the autocorrelation coefficient function The above results naturally motivate us to take a closer look of absolutely convergent numerical series This is done in the following section 4 Absolutely Convergent Series : New Direction: Consider an absolutely convergent infinite series ie s K( < (4) Hence the following necessary condition on K(s) is satisfied ie Lt K( s Thus, there exists an integer J such that K ( s) < for all s ( J +)

9 Now consider J K( K( + s s s J+ K( Thus, as in the case of autocorrelation coefficient series, we associate the following countable collection of infinite series with an arbitrary absolutely convergent series ( given in (4) ) s J + for N K( < As in the case of section, it is easy to show that N N N 3 Thus, a finite / infinite collection of indices ie N i ' s can be naturally associated with an absolutely convergent series ( as in (4)) Remark 8: As in the case of section 3, consider a bounded function of continuous variable t ie f(t) Define the following indices f ( t) dt As in section 3, it can be shown that 3 Details are avoided for brevity This result, thus deals with the relationshi between the norms of a bounded continuous function L - 5 Signal Processing Persectives: Consider the infinite sequence of correlation coefficients as a discrete time signal As in digital signal rocessing [OS], let the discrete time Fourier transform of the signal { ρ ( s): s < } be j ω ρ e ( ) s ρ ( s) e jω s j ω It is well known that ρ ( e ) is a eriodic function with eriod From the above definition, we have that j ω ρ ( e ) ρ ( s ω ) s π We know that if ρ ( < imlies that ρ ( s) < Thus, one s s can study the nature of Discrete Time Fourier Transform (DTFT) of the autocorrelation coefficient sequence leading to frequency domain ersectives

10 Consider a Linear Time Invariant ( LTI ) system with the imulse resonse sequence { h ( k) : < k < } The linear system is stable if and only if [OS] k h ( k) < The Discrete Time Fourier Transform of the imulse resonse sequence is also called the frequency resonse of the discrete time LTI system It is well known that the frequency resonse of a stable LTI system will always converge If a sequence is absolutely summable, it will also have finite energy ie n x( n) < This follows from the fact that [ x( )] ( n) n x It is not true, however, that a sequence with finite energy is absolutely summable For a general discrete time sequence x(n) j ω with the associated Discrete Time Fourier Transform X ( e ), we have the following result, called the Parseval s Theorem: n x( n) x * ( n) π + π π X ( e j ω ) X * ( e j ω ) dω In view of the results in Section, we introduce the following concet associated with a discrete time LTI system Let h (n) be the imulse resonse of such a system We call the system th order stable if n h ( n) < for < Thus, we can have an unstable system in the traditional sense, but is stable in the above modified sense Results related to such systems need to be investigated in detail 6 Future Research Directions: The investigation related to short memory rocesses naturally leads to the following research directions: In the theory of L saces and L norms of finite/infinite dimensional vectors, Holder, inkowski and Jensen inequalities

11 are well studied Those inequalities are invoked in the resent investigation [Roy] Consider a bounded infinite dimensional vector ( ie all the elements are bounded in absolute value by a finite constant) ie [ b, b,] Let the associated infinite series be absolutely convergent ie < Define the following ower series j associated with such a sequence ie b j j x ( ) b j x for x J j Thus, the set J constitutes the region of convergence Inside the region of convergence, the ower series is uniformly convergent ( by Weierstrass s -test ) We associate the following class of ower series with ( x ) ie j ( x) b j x for < j Using simle calculation the region of convergence of such ower series can easily be detemined Let ϕ () be a structured function ( such as a convex / monotone function ) Consider the case where < In other words, we have an absolutely convergent series and we would like to investigate the convergence of the associated series ϕ ( ) In such an investigation, we exect the Jensen inequality ( associated with infinite dimensional vectors ) to be of utility ore generally the convergence of associated ower series will be investigated j b j j b j ulti-dimensional Versions: Based on the results in sections and 3, one is naturally led to consider multi-variate wide sense stationary stochastic rocesses or random fields Thus, in its most generic form, we will investigate the convergence/ divergence of the following structured numerical as well as ower series ( for < ): s s sl ρ ( s, s,, s ) given that ρ ( s, s,, ) l s l s s sl ρ ( s s s, s,, sl ) x x x sl l

12 The results of this investigation are naturally extraolated to multi-variate ower series whose coefficients are bounded in magnitude Consider the stochastic rocess Y(t) such that q [ X ( t) ] for < Y ( t) q Given that X(t) is a short / long memory WSS stochastic rocess, we study the nature ( long / short memory ) of the stochastic rocess, Y(t) Secifically, suose we consider [ X ( ) ] Y ( t) t and X(t) is a Gaussian rocess Let { η Y (s) } be the autocorrelation coefficient sequence of the rocess Y(t) and { ρ X (s)} be the autocorrelation coefficient sequence of the rocess X(t) It can be easily shown that η (s) [ ( s)] Y 7 Conclusions: In this research aer, we roose a fine grain classification of long memory Wide Sense Stationary (WSS) stochastic rocesses by associating a set of indices Some roerties of the indices are roved The results are extraolated to structured absolutely convergent series Briefly signal rocessing ersectives are summarized Finally some future research directions are roosed ρ X ACKNOWLEDGEENTS The author would like to thank Prof TSubba Rao of the University of anchester During his talk on Time Series Analysis at the CR RAO Advanced Institute of athematics, Statistics and Comuter Science (AISCS) the author generated the ideas documented here REFERENCES: [Roy] HLRoyden, Real Analysis, Prentice Hall of India, New Delhi-7 [PaP] APaoulis and SUPillai, Probability, Random Variables and Stochastic Processes, Tata cgraw Hill, New Delhi [OS] AVOenheim and RWSchafer, Digital Signal Processing, Prentice Hall of India, New Delhi-