Whittaker-Kotel nikov-shannon approximation of sub-gaussian random processes

Transcription

1 Whittaker-Kotel nikov-shannon approximation of sub-gaussian random processes Andriy Olenko Department of Mathematics and Statistics, La Trobe University, Melbourne, Australia The presentation is based on the paper Yu. Kozachenko, A.Olenko. Whittaker-Kotel nikov-shannon approximation of ϕ-sub-gaussian random processes. submitted University of Granada, July, 2015 A.Olenko Shannon approximation of sub-gaussian random processes 1 / 35

2 Talk Outline 1 Information and Communication Theory 2 Deterministic sampling 3 Stochastic sampling 4 Reconstruction errors in WKS stochastic sampling 5 ϕ-sub-gaussian random processes 6 Approximation in L p ([0, T ]) 7 Uniform approximation 8 Aliasing A.Olenko Shannon approximation of sub-gaussian random processes 2 / 35

3 A.Olenko Shannon approximation of sub-gaussian random processes 3 / 35

4 The classical Whittaker-Shannon-Kotel nikov sampling theorem states that any function f PWπ 2 (Paley-Wiener class of all complex-valued L 2 (R) functions whose Fourier spectrum is bandlimited to [ π, π]) can be reconstructed uniformly on each bounded set from its values at the integers: f (x) = sinc(x n)f (n), where n= sin(πt) t 0, sinc(t) := πt 1 t = 0. A.Olenko Shannon approximation of sub-gaussian random processes 4 / 35

5 C. Shannon V. Kotel nikov Slepian: Probably no single work in this century has more profoundly altered man s understanding of communication than C.E.Shannon s article, A mathematical theory of communication, first published in The ideas in Shannon s paper were soon picked up by communication engineers and mathematicians around the world. They were elaborated upon, extended, and complemented with new related ideas. The subject thrived and grew to become a well-rounded and exciting chapter in the annals of science. A.Olenko Shannon approximation of sub-gaussian random processes 5 / 35

6 Information and Communication Theory Work by H. Nyquist in the late 1920s on bandwidth-limited pulse shaping was complemented in the 1930s and 1940s by innovations in communication theory (using probabilistic descriptions of signal and noise waveforms) from S. Rice, C. Shannon, V. Kotel nikov, and others. Data communication systems using bandwidth-efficient pulse designs, including signal equalization and detection techniques deriving from communication theory, were intensively developed throughout the second half of the twentieth century. Among the major advances were data-driven equalization; the Viterbi Algorithm (A.J.Viterbi, early 1970s) for efficient implementation of maximum likelihood sequence estimation for recovery of the transmitted data stream; trellis-coded modulation innovated by G. Ungerboeck in the early 1990s, etc. A.Olenko Shannon approximation of sub-gaussian random processes 6 / 35

7 The field of information theory is anchored by Claude Shannon in 1948 and Vladimir Kotel nikov in Information theory provides a mathematical bound on the capacity of a band-limited, noisy channel, and proof that a data encoding exists that can reach that bound. Actually creating these powerful codes, without imposing inordinate encoding delays, has been the practical challenge for both information theorists (who have their own Information Theory Society within the IEEE) and communication engineers. Many useful codes have been implemented and are important in communication today, including Reed-Solomon codes implemented in the compact disk system in the 1980s, turbo codes, etc. In recent years several authors have shown that there is a strong relationship between the well known classical sampling theorem and other fundamental theorems of real and complex analysis such as Poisson s summation formula and Cauchy s integral formula. A.Olenko Shannon approximation of sub-gaussian random processes 7 / 35

8 Prof. G. C. Goodwin s data A.Olenko Shannon approximation of sub-gaussian random processes 8 / 35

9 Deterministic sampling Let X be a normed space with the norm X endowed. Assume that the structure of X admits the sampling approximation procedure f (x) = n Z d f (t n )S(x, t n ), f X (1) where x = (x 1,, x d ) R d, and {t n } n Z d R d and S are the sampling set and the sampling function respectively. In direct numerical implementations we consider the truncated variant of (1): Y J (f ; x) = f (t n )S(x, t n ), J Z d. n J By application reasons I has to be finite (the finite sampling reconstruction sum perfectly restores f in some cases). A.Olenko Shannon approximation of sub-gaussian random processes 9 / 35

10 We are looking for optimal/minimal J such that f (x) Y J (f ; x) X ε f X, where ε > 0 is the already known relative approximation error level requesting lim J Y J(f ; x) = f (x) in certain (pointwise, uniform etc.) convenient manner. This interpolation procedure we will denote as f (x) ε Y J (f ; x). The Kotel nikov - Shannon (sampling) reconstruction formula is a natural choice for (1), with t n = n π w, n Z and S(x, t n) = sinc(w(x t n )). Then (1) becomes f (x) = n Z d f ( ) d π π n 1 w 1,, n d w d sinc(w j x j n j ). j=1 A.Olenko Shannon approximation of sub-gaussian random processes 10 / 35

11 We assume that only finite size samples for f are reading from the Z d around fixed value of the argument x so, that w j x j π n j N j, j = 1, d. Theorem Let PW 2 π,d. Then f ( ) Y N,d (f, ) f 2 8 d 1 d N i π 2d i=1 n i =1 The inequality is sharp and the extremal function is fn,d 1 (x) := π 2d d n: I > N (n; 1 2 ) i=1 1 (2n i 1) 2. sin(πx i ) ( ni 1 2) (ni x i ). A.Olenko Shannon approximation of sub-gaussian random processes 11 / 35

12 Example of stochastic sampling A.Olenko Shannon approximation of sub-gaussian random processes 12 / 35

13 Stochastic sampling In the history of development of similar stochastic results we have two main approaches. The first one began with the truncation error upper bound given by Belyaev in 1959, who established the result T N (ξ; x) 16π2 (2 + x ) 2 E ξ(x) 2 (π w) 2 N 2, w < π, valid for band-limited to [ w, w] weakly stationary random processes ξ. In the same article he connects wide sense band-limited stationary random processes and its exponential boundedness. Exploiting the exponentially bounded band-limited connection realized by the Fourier transform pair in the integral representations of both random and deterministic signal functions Piranashvili gave pioneering results is his classical article. He generalized Belyaev s results and established a truncation error upper bound of magnitude O(N 2 ) for all bounded x-subsets from R. A.Olenko Shannon approximation of sub-gaussian random processes 13 / 35

14 Prof. Yu. Belyaev Prof. Z. Piranashvili A.Olenko Shannon approximation of sub-gaussian random processes 14 / 35

15 This kind of truncation error upper bounds were obtained with different criteria upon the considered signal functions, but basically by the exponentially bounded ones. All bounds are obtained in similar way, only different assumptions upon the basic signal function (Lip-, Paley-Wiener-, Bernstein - spaces; guard-bound, oversampling, polynomially bounded correlation etc.) result in different upper bounds. The second large truncation error upper bound result class is obtained only for narrow time interval around origin. Majorizing the finite sum b sinc(x n) q, q > 0 n=a and exploiting convexity of the sin-function one obtains sharp truncation error upper bounds. A.Olenko Shannon approximation of sub-gaussian random processes 15 / 35

16 In the series of papers ( ) Olenko and Pogany obtained uniform, minimal, time shifted truncation error upper bounds in interpolating the weak Cramér class processes. It has to be mentioned as well that extremal functions were given in these papers. However, this approach is not applicable for other classes of stochastic processes or other measures of deviation. Also, from a practical point of view, measures of the closeness of trajectories are often more appropriate than estimates of mean-square errors. A.Olenko Shannon approximation of sub-gaussian random processes 16 / 35

17 Prof. T. Pogány Prof. Yu. Kozachenko A.Olenko Shannon approximation of sub-gaussian random processes 17 / 35

18 Reconstruction errors in WKS stochastic sampling Let X(t), t R, be a stationary random process with EX (t) = 0 whose spectrum is bandlimited to [ Λ, Λ), that is B(τ) := EX(t + τ)x(t) = Λ Λ e iτλ df (λ), where F ( ) is the spectral function of X(t). The process X(t) can be represented as X(t) = Λ Λ e itλ dφ(λ), where Φ( ) is a random measure on R such that E [Φ( 1 )Φ( 2 )] = F ( 1 2 ) for any measurable sets 1, 2 R. A.Olenko Shannon approximation of sub-gaussian random processes 18 / 35

19 Then, for all ω > Λ there holds sin ( ω ( t kπ )) ( ) ω kπ X(t) = ω ( t kπ ) X, (2) ω ω k= and the series (2) converges uniformly in mean square. Let us consider the truncation version of (2) given by the formula X n (t) := n k= n sin ( ω ( t kπ )) ( ) ω kπ ω ( t kπ ) X. ω ω A.Olenko Shannon approximation of sub-gaussian random processes 19 / 35

20 Theorem Let z (0, 1), t > 0, s > 0. Then 1. for n ωt π it holds that z E X(t) X n (t) 2 n 2 C n (t), where ( 4ωt C n (t) := B(0) π 2 (1 z) + 4 ( ) z n )) π(1 z) ( 2 1 Λ ; ω 2. for n ω π max(t, s) it holds that z ( ) E (Y n (t) Y n (s)) 2 t s 2 b n (t, s), n where Y n (t) := X(t) X n (t), A.Olenko Shannon approximation of sub-gaussian random processes 20 / 35

21 b n (t, s) := B(0) W n (t, s) := Q n (t, s) := ( ( 4ω π 2 (1 z) 2ω π(1 z) 2 2 W n (t, s) + Q n(t, s) ( )), 1 Λ ω ωs ω2 (s + t)s ( z n 1 + ) π 2 n 2, (1 z) 2ω(s + t) nπ 2 ). A.Olenko Shannon approximation of sub-gaussian random processes 21 / 35

22 ϕ-sub-gaussian random processes Tail distributions of sub-gaussian random variables behave similar to the Gaussian ones so that sample path properties of sub-gaussian processes rely on their mean square regularity. One of the main classical tools to study the boundedness of sub-gaussian processes was metric entropy integral estimates by Dudley (1967). These results were extended by Fernique (1975) and Ledoux and Talagrand (1991) using the generic chaining (majorizing measures) method. The space of ϕ-sub-gaussian random variables was introduced by Kozachenko and Ostrovskyi (1985) to generalize the class of sub-gaussian random variables defined by Kahane(1960). A.Olenko Shannon approximation of sub-gaussian random processes 22 / 35

23 Definition A continuous even convex function ϕ(x), x R, is called an Orlicz N-function, if it is monotonically increasing for x > 0, ϕ(0) = 0, ϕ(x)/x 0, when x 0, and ϕ(x)/x, when x. Definition Let ϕ(x), x R, be an Orlicz N-function. The function ϕ (x) := sup y R (xy ϕ(y)), x R, is called the Young-Fenchel transform of ϕ( ). The function ϕ ( ) is also an Orlicz N-function. A.Olenko Shannon approximation of sub-gaussian random processes 23 / 35

24 Definition An Orlicz N-function ϕ( ) satisfies Condition Q if lim ϕ(x)/x 2 = C > 0, x 0 where the constant C can be equal to +. Example The following functions are N-functions that satisfy Condition Q: where C > 0. ϕ(x) = C x α, 1 < α 2; ϕ(x) = exp{cx 2 } 1; ϕ(x) = { Cx 2, if x 1, C x α, if x > 1, α > 2, A.Olenko Shannon approximation of sub-gaussian random processes 24 / 35

25 Let {Ω, B, P} be a standard probability space and L p (Ω) denote a space of random variables having finite p-th absolute moments. Definition Let ϕ( ) be an Orlicz N-function satisfying the Condition Q. A zero mean random variable ξ belongs to the space Sub ϕ (Ω) (the space of ϕ-sub-gaussian random variables), if there exists a constant a ξ 0 such that the inequality E exp (λξ) exp (ϕ(a ξ λ)) holds for all λ R. The space Sub ϕ (Ω) is a Banach space with respect to the norm τ ϕ (ξ) := sup λ 0 ϕ ( 1) (ln E exp {λξ}), λ where ϕ ( 1) ( ) denotes the inverse function of ϕ( ). If ϕ(x) = x 2 /2 then Sub ϕ (Ω) is a space of subgaussian variables. A.Olenko Shannon approximation of sub-gaussian random processes 25 / 35

26 Definition Let T be a parametric space. A random process X(t), t T, belongs to the space Sub ϕ (Ω) if X(t) Sub ϕ (Ω) for all t T. Gaussian centered random process X(t), t T, belongs to the space ( Sub ϕ (Ω), where ϕ(x) = x 2 /2 and τ ϕ (X(t)) = E X(t) 2) 1/2. Definition A family Ξ of random variables ξ Sub ϕ (Ω) is called strictly Sub ϕ (Ω) if there exists a constant C Ξ > 0 such that for any finite set I, ξ i Ξ, i I, and for arbitrary λ i R, i I : τ ϕ ( i I ) ( ) 2 λ i ξ i C Ξ E λ i ξ i C Ξ is called a determinative constant. The strictly Sub ϕ (Ω) family will be denoted by SSub ϕ (Ω). i I 1/2. A.Olenko Shannon approximation of sub-gaussian random processes 26 / 35

27 Definition ϕ-sub-gaussian random process X(t), t T, is called SSub ϕ (Ω) if the family of random variables {X(t), t T} is strictly Sub ϕ (Ω). The determinative constant of this family is called a determinative constant of the process and denoted by C X. A Gaussian centered random process X(t), t T, is a SSub ϕ (Ω) process, where ϕ(x) = x 2 /2 and the determinative constant C X = 1. Lemma Let ϕ( ) be an Orlicz N-function. Then it can be represented as ϕ(u) = u 0 f (v) dv, where f ( ) is a monotonically non-decreasing, right-continuous function, such that f (0) = 0 and f (x) +, when x +. A.Olenko Shannon approximation of sub-gaussian random processes 27 / 35

28 Approximation in L p ([0, T ]) Theorem Let ω > Λ > 0, n ωt π, z (0, 1). Let X(t), t R, be a stationary z SSub ϕ (Ω) process which spectrum is bandlimited to [ Λ, Λ), and S n,p := ( CX n ) p T 0 Cn p/2 (t) dt, where C X is a determinative constant of the process X(t). Then, T 0 X(t) X n(t) p dt exists with probability 1 and the following inequality holds true for ε > S n,p (f ( p (S n,p /ε) 1/p)) p : { T } { ( P X(t) X n (t) p dt > ε 2 exp ϕ (ε/s n,p ) 1/p)}. 0 A.Olenko Shannon approximation of sub-gaussian random processes 28 / 35

29 Definition X n (t) approximates X(t) in L p ([0, T ]) with accuracy ε > 0 and reliability 1 δ, 0 < δ < 1, if { T } P X(t) X n (t) p dt > ε δ. 0 A.Olenko Shannon approximation of sub-gaussian random processes 29 / 35

30 Theorem Let X(t), t R, be a stationary SSub ϕ (Ω) process with a bounded spectrum. Then X n (t) approximates X(t) in L p ([0, T ]) with accuracy ε and reliability 1 δ if the following inequalities hold true ε > S n,p ( f ( p (S n,p /ε) 1/p)) p, { ( exp ϕ (ε/s n,p ) 1/p)} δ/2. A.Olenko Shannon approximation of sub-gaussian random processes 30 / 35

31 Example For example, for p = 2, T = B(0) = ω = 1, and Λ = 3/4 the number of terms n as a function of ε and δ is shown below A.Olenko Shannon approximation of sub-gaussian random processes 31 / 35

32 Example For fixed ε = δ = 0.1 the behaviour of the number of terms n as a function of the parameter p [1, 2] is illustrated below. A.Olenko Shannon approximation of sub-gaussian random processes 32 / 35

33 Uniform approximation Theorem Let X(t), t [0, T ], be a separable SSub ϕ (Ω) random process whose spectrum is bandlimited to [ Λ, Λ). Then, for any θ (0, 1), ε > 0, and such values of n that z := ω2 T 2 < 1 : n 2 π 2 P { sup X(t) X n (t) ε t [0,T ] } { ( )} ( ) ε(1 θ) exp ϕ etcx bn + 1, C n 2nθC n where C X is the determinative constant of the process X(t), b n := b n (T, T ) is given by (3) evaluated at z = z, C n := C XB(0) n ( 4ωT π 2 (1 z ) + 4 ( z n) 1 2 )) π(1 z ) ( 2 1 Λ. ω A.Olenko Shannon approximation of sub-gaussian random processes 33 / 35

34 Aliasing The aliasing problem s formulation is very similar to previous problems but these phenomenons are completely different. In aliasing problems the difference between f and its nontruncated cardinal series is considered and the errors appear due to difference in actual and assumed band-regions. A.Olenko Shannon approximation of sub-gaussian random processes 34 / 35

35 References Yu.K. Belyaev, Analytical random processes, Theory Probab. Appl. 4(4) (1959) Z.A. Piranashvili, On the problem of interpolation of random processes, Theory Probab. Appl. 12(4) (1967) A. Olenko and T. Pogány. Time shifted aliasing error upper bounds for truncated sampling cardinal series. J. Math. Anal. Appl. 324(1), , A. Olenko and T. Pogány. Average Sampling Restoration of Harmonizable Processes. Communications in Statistics: Theory and Methods. Vol. 40. (2011), Yu. Kozachenko, E. Ostrovskyi, Banach spaces of random variables of Sub-gaussian type, Theor. Probab. Math. Statist. 32 (1985) Yu. Kozachenko, A. Olenko, O. Polosmak, On convergence of general wavelet decompositions of nonstationary stochastic processes, Electron. J. Probab. 18(69) (2013) A.Olenko Shannon approximation of sub-gaussian random processes 35 / 35