Analytic and Asymptotic Analysis of Bayesian Cramér-Rao Bound for Dynamical Phase Offset Estimation

Transcription

1 Analytic and Asymptotic Analysis of ayesian Cramér-Rao ound for Dynamical Phase Offset Estimation Stéphanie ay, Cédric Herzet, Jean-Marc rossier, Jean-Pierre arbot, enoit Geller To cite this version: Stéphanie ay, Cédric Herzet, Jean-Marc rossier, Jean-Pierre arbot, enoit Geller. Analytic and Asymptotic Analysis of ayesian Cramér-Rao ound for Dynamical Phase Offset Estimation. IEEE Transactions on Signal Processing, Institute of Electrical and Electronics Engineers, 008, 56, pp <0.09/TSP >. <hal-0584> HAL Id: hal Submitted on 4 Dec 05 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Analytic and Asymptotic Analysis of ayesian Cramér-Rao ound for Dynamical Phase Offset Estimation S. ay, C. Herzet,, J.M. rossier 3, J.P. arbot, and. Geller ENS Cachan / Laboratoire SATIE, France UC erkeley / Dept. of EECS, erkeley, CA INPG / Laboratoire LIS, Saint Martin d Hères, France Abstract In this paper, we present a closed-form expression of a ayesian Cramér-Rao lower bound for the estimation of a dynamical phase offset in a non-data-aided PS transmitting context. This kind of bound is derived considering two different scenarios: a first expression is obtained in an off-line context and then, a second expression in an on-line context logically follows. The SNR-asymptotic expressions of this bound drive us to introduce a new asymptotic bound, namely the Asymptotic ayesian Cramér-Rao ound. This bound is close to the classical ayesian bound but is easier to evaluate. I. INTRODUCTION During the last decade, synchronization has become one of the most challenging task of digital receivers. On the one hand, since the advent of very powerful error correcting codes see e.g., 3, the synchronizers have to properly operate at Signal-to-Noise Ratios SNRs lower than ever before. On the other hand, the high data-rate requirements of future communication standards imply rapidly varying synchronization parameters at the receiver. In order to cope with such stressful environments, Maximum-Likelihood ML synchronization performance are often a desirable feature. Unfortunately, the ML estimator is often unfeasible in practice. Hence, sub-optimal synchronizers have been proposed, see e.g., 4. In order to assess the performance of such synchronizers, lower bounds on mean-squared estimation error are needed. In particular, the family of Cramér-Rao ounds CRs 5, 6 has been shown to give tight estimation lower bounds in a number of practical scenarios, see e.g., 7 for CR contributions associated to synchronization problems. Rife et al. 7 and Cowley 8 have derived CR closedform expressions for constant phase-offset estimation in the socalled Data-Aided DA and Non-Data-Aided NDA scenarios, respectively. In 9, the authors proposed a semi-analytical method enabling to efficiently evaluate CRs in Code-Aided CA scenarios. A Modified CR MCR, easier to evaluate than the standard CR, has been introduced in, 3. The MCR is usually looser than the CR but it is equivalent to the CR at high SNR 4. More recently, the problem of deriving CRs suited to time-varying parameters has been This work is partially supported by the European Community contract NO , NEWCOM. addressed throughout the ayesian context. In 5, the authors propose a general framework for deriving analytical expression of on-line or also called filtering CRs. In 6, the authors developed a numerical graph-based algorithm to evaluate the CR in time-varying scenarios. In this contribution, we address the open problem of deriving an analytical expression of the off-line or also called smoothing ayesian CR CR for time-varying phase estimation in NDA scenarios. Explicit expressions of the CR and its modified version are provided in an off-line scenario. This bound helps us to evaluate and also to predict the estimator performance without any particular assumption or simulation restriction. Moreover, an on-line interpretation follows. The asymptotic cases at low and high SNR are presented. In particular it is shown that the Modified CR MCR is equal to the derived asymptote at high SNR. A new asymptote-based bound is then introduced simplifying the general expressions of the CR. This paper is organized as follows. In Section II we set the system model. After recalling in Section III the different kinds of Cramér-Rao bounds, we derive in Section IV the off-line CR. In Section V the asymptotic cases are considered and we present the asymptote-based lower bound. Next, the closed-form expression of the CR in the on-line context is given Section VI. Finally, the different results are illustrated and interpreted in Section VII. The notational convention adopted is as follows: italic indicates a scalar quantity, as in a; boldface indicates a vector quantity, as in a and capital boldface indicates a matrix quantity as in A. The m, n th entry of matrix A is denoted A m,n. The matrix transpose operator is indicated by a superscript T as in A T.. denotes the modulus of a scalar quantity or the determinant of a matrix quantity. a n m represents the vector a m a n T, where m and n are positive integers m < n. R {a} and I {a} are respectively the real and the imaginary part of a. E xy. denotes the expectation over x and y. k is equal to the integer part of k. δ k,l is the ronecker symbol. and ψ represent the first and second-order partial derivatives T operator i.e., = and ψ = ψ T. The error function is defined as erf x = x π 0 e t dt. The actual value of an unknown parameter is indicated by the

3 superscript as in. A. The Standard Cramér-Rao ound and its Modified Version II. SYSTEM MODEL We consider the transmission of a PS modulated sequence a = a a T over an Additive White Gaussian Noise AWGN channel affected by some carrier phase offsets stacked in a vector = T. Assuming that the received signal has been ideally filtered and sampled at the optimum sampling instant, the discrete-time baseband signal y = y y T is given by y k = a k e j k + n k with k =..., where a k is the k th unknown transmitted PS symbol a k = ± and n k is a zero-mean circular Gaussian noise with known variance. We suppose that the system operates in a NDA synchronization mode, i.e., the transmitted symbols are independent and identically distributed i.i.d. with pa k = a = where a = ±. We assume that the energy per symbol is normalized, so that the noise variance is related to the E s /N 0 -ratio as E s. We consider the case of a Wiener phase-offset evolution, i.e., k = k + w k, N 0 = where w k is an i.i.d. zero-mean Gaussian noise with known variance w. This model is commonly used 7, 8 to describe the behavior of practical oscillators for which the frequency is randomly perturbated. As the phase is the integral of the frequency, the phase variance increases with time index k. This phenomenon is thus non stationary and one readily obtains that var k = var k + w. In order to recover the transmitted symbols, practical receivers have first to estimate the carrier phase offsets corrupting the received observations and perturbing the whole data recovery. Phase estimation can actually be considered following two main scenarios: i Off-line synchronization: the receiver waits until the whole observation frame, i.e., y = y y T, has been received. Then, it processes all the observations to compute the estimates of the carrier phase offsets = T. ii On-line synchronization: the receiver estimates k upon the arrival of the k th observation, i.e., y k. The phase estimate is then computed based on the current and previous observations only, i.e., y k = y y y k T. In the sequel, the Cramér-Rao bound evaluation will be considered within the context of both the off-line and the online scenarios. III. CRAMÉR-RAO OUNDS: SOME USEFUL EXPRESSIONS A natural question which arises when designing estimators is the ultimate accuracy that can be achieved in the estimation operation. The family of CRs partially answers this question by providing a lower bound on the Mean Square Error MSE achievable by any unbiased estimator. Different kinds of CRs can be considered. In the following, we briefly recall general expressions of these bounds. Let ˆy denote an unbiased estimator of. Then the estimator ˆy satisfies the following inequality E y = ˆy ˆy T SCR, 3 where SCR is the Standard CR SCR and is the actual parameter value. The SCR is equal to the inverse of the Fisher Information Matrix FIM F defined by 6 F = E y = log py =. 4 The practical evaluation of the FIM for modulated signals is generally quite tedious because of the symbols a which are nuisance parameters. In order to circumvent this problem a Modified CR has been proposed and comes from the inversion of the following matrix: G = E a E y a,= log py a, =. 5 Unlike the SCR, it is usually possible to derive a closed form expression of the MCR. In particular, in, the analytical expression associated to a constant phase offset is derived: MCR = Es N 0, 6 where is the number of observations. Note that although easier to evaluate than the SCR, the MCR is also generally looser. In the particular case of high-snr transmissions, it has however been shown 4 that the MCR corresponds to the SCR. We can also notice that equation 6 coincides with the CR provided in 7. This bound is computed in the case of an unmodulated scenario or, equivalently, of a DA scenario.. The ayesian Cramér-Rao ound and its Modified Version The standard and the modified CRs are not suited to time-varying parameter estimation. In particular, they do not take into account the statistical dependence which may exist between phase offsets at different instants. This dependence is naturally considered within the ayesian framework: on the one hand, prior distribution p implicitly models the time dependence between stochastic phase offsets; on the other hand, ayesian framework is intrinsically based on the knowledge of a vector prior distribution p. Within this context, a ayesian CR CR has been proposed in 9 such that E y, ˆy ˆy T CR. 7 Unlike the SCR and the MCR, the CR does not depend on a particular value. The CR is the inverse of the ayesian Information Matrix IM, which from 9 can be written as = E F + E log p, 8 where F is the FIM defined in 4. Each term of the IM 8 plays a different role: the first term can be interpreted as the average information about brought by the observations y whereas the second term can be regarded as the information

4 3 available from the prior knowledge of, i.e., p. This term allows us to take into account the time dependence between phase offsets at different instants. Similarly to the MCR, a modified version of the CR has been considered in 0. This Modified CR MCR is the inverse of the following information matrix C = E G + E log p, 9 where G is the modified Fisher information matrix defined in 5. The MCR is usually easier to compute than the CR but it is also looser 0. IV. THE OFF-LINE CR AND ITS MODIFIED VERSION FOR DYNAMICAL PHASE ESTIMATION In this section, we derive an analytical expression of the CR associated to an off-line carrier-phase-offset estimation. We proceed as follows. We first obtain analytical expressions of the two terms contributing to the IM 8. Then, the expression of the diagonal elements of the inverse of the IM are derived. In addition, we also give an analytical expression of the MCR. A. Computation of E F E F corresponds to the first term in the right-hand side of equation 8. This term evaluation requires the computation of the Fisher information matrix F, which in turn requires the evaluation of the Hessian of the log-likelihood function log py. Using the observation model defined in section II, one has that the log-likelihood function can be expanded as log py = log a py a, pa. 0 Using the whiteness of the noise and the independence of the transmitted symbols, one then obtains that log p y = log p y k k. k= It is important to note that each term of the sum is a matrix with only one non-zero element at most, namely, log p y k k = k,k k log p y k k. As a direct consequence, the Hessian log p y is a diagonal matrix with the k th diagonal element given by equation. Moreover, because of the circularity of the observation noise, the expectation of equation with respect to p y k k does not depend on k. One then obtains E F = J D I, 3 where I is the identity matrix and J D is defined as follows J D E y, log p y k k k. 4. Computation of E logp Due to the Wiener structure of the phase model, log p can be expanded as log p = log p + log p k k. k= 5 Let us detail the expressions of each term contributing to equation 5. The first term is a matrix with only one nonzero element, namely, the entry, which is equal to log p, = log p. 6 The other terms in 5 are matrices with only four non-zero elements, namely, the entries k, k, k, k, k, k and k, k. Due to the Gaussian nature of the noise, one finds log p k k k,k = log p k k log p k k = k,k k,k = w, 7 log p k k k,k = w. 8 Taking 6, 7 and 8 into account, one finally obtains E log p = E log p w w w.... w w w w w w w 9 In the sequel, for the sake of conciseness, we will set E log p = 0. This corresponds to the case of a noninformative prior about see. The reasoning which is made in the following could however be extended to the general case, at the expense of some more difficult derivations and would not modify our results interpretation. C. Analytical Expression of the Off-Line CR In this subsection, an analytical expression of the diagonal elements of the inverse of the IM is derived. From 3 and 9 one readily obtains that the IM has a particular mathematical structure, i.e., = b A + A A A +, 0 where A and b are defined as A wj D and b / w. In particular, is a symmetric sparse matrix having

5 4 an almost periodical structure. ased on these observations, we show in Appendix I that the k th diagonal element of can be expressed as k,k = ρ b + r r 3 + ρ b + r r 3 b A rk r k + r k r k, where r w + J D + 4 J D w r w + J D w ρ J D + 4 w J D + 4 w ρ J + + D + 4 w J D + 4 J D w w J D w J D,, 3, 4, 5 Note that together with -5 provides an analytical expression of the off-line CR associated to the estimation of k. This function of index k depends only on three parameters: the number of observations, the phase noise variance w and the observation noise variance n or equivalently J D, see equation 4. The evaluation of J D is discussed in Section V. D. The Off-line MCR We now consider the MCR, see equation 9. The second term in the right-hand side of 9 has already been computed in section IV-. The first term E G requires the evaluation of G which corresponds to the modified Fisher information matrix defined in 5. Using the observation model, one has that log py a, is a diagonal matrix where log py a, k,k = k log py k a k, k. 6 Due to the Gaussian distribution of the noise, one further finds that k log py k a k, k = R a ky k e j k. 7 Consequently, one obtains that G k,k = E a E y a, R a ky k e j k 8 = E a a k 9 =. 30, N 0 which cor- = observation. As One can observe that G k,k = responds to the MCR 6 for E s straightforwardly E G = / n I, the modified IM is obtained exactly like the IM of subsection IV-C with J M playing the role of J n D in equation 0. Note, however, that the MCR is usually looser than the CR. V. EVALUATION OF J D AND ASYMPTOTE-ASED LOWER OUND In this section we first point out that, in the general case, the evaluation of J D implies to resort to numerical integration. We then derive the high-snr and low-snr asymptotes of the CR, and emphasize that their evaluation is easy. Finally, we show that these asymptotes are themselves lower bounds on the MSE. In particular, we illustrate that the combination of the low and high SNR asymptotes leads to an alternative tight lower bound, whose evaluation is straightforward. A. Evaluation of J D In this part, we calculate an expression of J D defined in 4. First, using the Gaussian nature of the noise and the equiprobability of the data symbols, one finds that log py k k = log π n e + y k n cosh R y k e j k. 3 Taking the second derivative of 3, one easily obtains that log py k k k = Rx k tanh Rx k I x k tanh Rx k, 3 where x k y k e j k. In the general case, the expectation of 3 with respect to py does not have any simple analytical solution. Hence, in practice, we have to resort to either numerical integration methods or some approximations. In the following, we present both the high-snr and the low- SNR approximations of the CR.. High-SNR CR asymptote In this part, we investigate the CR behavior at high SNR. From the definition of the IM 8, only the first term, i.e., E F, depends on the SNR. Moreover, we have from 3 that E F is fully characterized by J D 4. Hence, in the remainder of this section, we focus on the behavior of J D. At high SNR i.e., 0, the tanh-function in 3 can be approximated as tanh Rx k sign Rx k. 33 Hence, replacing tanh in 3 by its approximation 33 and using the definition of J D 4, one has see Appendix III J D E yk, k Rx k sign Rx k 34 = e n + n π erf. 35 n

6 5 So, taking the limit of 35 for tending to zero, one has that J Dh lim J 0 D =. 36 We see therefore from 30 and 36, that the high-snr asymptote of the CR i.e., J Dh is equal to the MCR. This corroborates the result derived by Moeneclaey 4 in the non-ayesian case for a scalar parameter. C. Low-SNR CR asymptote We now consider the low-snr asymptote of the CR in the NDA PS context. Following the same reasoning as before, we equivalently focus on the behavior of J D for tending to infinity. From 3, using the fact that tanhz z around z = 0, we obtain log p y k k k Rx k + Ix k. 37 Plugging 37 into 4, one readily obtains that the CR low-snr asymptote is J Dl lim J D = In, the authors obtain a result similar to 38 in a non- ayesian scenario for a scalar parameter. D. Asymptote-based Lower ound In this subsection, we show that the combination of the low and high-snr CR asymptotes, presented in the previous subsections, still leads to a lower bound on the MSE. In particular, we define the Asymptotic CR ACR as, ACR minj Dl, J Dh I + E log p 39 which can be evaluated with equations 9, 36 and 38. We now show that MCR ACR CR. 40 The proof of the first inequality directly comes from the definition of the ACR and of the MCR. The second inequality is straightforward to show when J Dh J Dl because in such a case, we have, ACR = J Dh I + E log p = MCR. 4 Since we know that CR MCR see 0, 40 is proven for J Dh J Dl. The proof for J Dl < J Dh is more lengthy and is detailed in Appendix IV. The ACR is thus an easy-to-evaluate lower bound on the MSE. This ayesian bound which does not require any Monte-Carlo simulation is compared to the CR in Section VII. This kind of reasoning only depends on the observation model: the ACR is relevant in the case of a PS transmission. However one can adapt this asymptote-based lower bound on any phase model system. VI. THE ON-LINE CR : SEQUENTIAL CRAMÉR-RAO OUND Up to this section, we have focused on the off-line scenario. We now show how the previous results can be used in the case of an on-line synchronization mode. In this mode, the receiver updates the observation vector y k in order to estimate k : only the past and the current observations are available. In order to evaluate the performance of this kind of estimator, a Posterior on-line Cramér-Rao ound was derived in 5. Tichavsky et al. provide a method for updating the IM from the time index k to the time index k +. This method was already applied in 3 on the same model described by equations and : the lower bound for the last state parameter k is given by the following recursive sequence C k+ = w + C k J D w and C = J D C k J D On the other hand, using our previous derivations, we can provide an alternative expression of the on-line CR. Indeed, the on-line CR associated to observation vector y is clearly equal to entry, of the inverse of the IM, i.e.,. Using expression, we therefore have, C = =, ρ b + r r 3 + ρ b + r r 3 b A r r + r r. 43 The behavior of the off-line and of the on-line 43 CRs will be studied and compared in the next section. In addition, it is shown in Appendix II that this bound decreases with time index to the following limit: lim, = w + w w JD. 44 Asymptotically with the observation size, the estimation performance is limited both by the phase noise and the observation noise due to J D. Therefore, unlike the non- ayesian CR 8, the CR does not necessarily tend to zero when the number of observations goes to infinity. VII. DISCUSSION In this section we bring to the fore the behavior of the previous bounds, namely the off-line and the on-line CRs and ACR, by considering different scenarios. We first consider a transmission disturbed by an AWGN with variance n = 4 and phase noise with variance w = 0.6 rad. Figure superimposes versus time index, the on-line CR see equation 43, its asymptote described by equation 44, and the off-line CRs for different block-observation lengths see equation. We then obtain the lower bound for each phase offset k according to the considered scenario. In the off-line context, one can see that the best phase estimate is achieved at mid-block, whereas the estimates are likely to be poorer at the block boarder the proof of this property is detailed in Appendix I-D. This stems from the fact

7 6 that in the center position of the phase vector one has more adjacent past or future and strongly correlated variables than at the boarder of the phase vector. Stated more precisely, from equation, the correlation coefficient between k and k+l E +k w where l is a positive integer is equal to E +k+l w, and thus even if a boarder phase is estimated with the same number of observations than a mid-size position phase, the correlation coefficient at the boarder are globally poorer than at the center and the information cannot be exploited as well. Concerning the on-line bound, at the beginning when the number of observations increases, the estimator takes into account more and more information and the estimation is improved; the bound thus decreases and converges to its asymptote: the estimation performance is then limited by the phase noise and the observation noise independently of the number of observations taken into account. MSE d =3 =5 Off Line and On Line CRs On Line CR On Line Asymptote Off Line CR =0 =5 = Observation lock Length Fig.. CRs versus the number of observations for n = 4, w = 0.6 rad J D is evaluated over 0 5 Monte-Carlo trials. We now analyze the bound behavior versus the SNR over a block of = 0 PS transmitted symbols. We illustrate some results for two distinct phase-noise variances: w = 0.0 rad and w = 0.6 rad. Figure superimposes for the two previous phase-noise variances, the CRs see equation evaluated over 0 7 Monte-Carlo trials and the ACRs see equation 39 with k = + versus the SNR. One can recover on Figure for w = 0.6 rad and SNR = 6 d, the minimum of the off-line CR displayed on Figure for = 0. We distinguish three SNR ranges. At high SNR, we notice that the ACRs in dashed lines and the corresponding CRs logically merge. Moreover, the ACR is close to the CR in the whole range of SNR for the two phase noise variances considered. At high SNR, both CRs converge to the deterministic MCR evaluated for = observation, 3 i.e., MCR = /J Dh. In this range of SNR, the information provided by each observation y k is preponderant over the a priori knowledge of. Consequently in this case the observations are reliable enough to only take into account the present observation y k in order to estimate k and this MSE d =0 ayesian Cramér Rao ounds Versus SNR = w = 0.0 rad w = 0.6 rad CR ACR MCR = MCR =0 CR Low SNR Asymptote SNR d Fig.. Comparison between ayesian CRs and Asymptote-based CRs for two different phase noise variances w = 0.0 rad and w = 0.6 rad. is why the bounds converge to the well-known MCR for any variance w. As the a priori distribution of has no influence, the ayesian problem tends to a deterministic phase estimation problem where we estimate independent phases k with independent observations. In mid-range SNR, when the phase variance w decreases, the CR gets closer to the deterministic MCR evaluated for = 0 observations. This is because in the limiting case for which the phase variance tends to zero, the time-varying phase estimation problem can be simplified to a deterministic constant-phase estimation problem. The MCR is then a lower bound of the CRs for the same number of observations = 0. At low SNR, the CR and the ACR coincide. In this SNR-range, the observation noise masks the a priori dynamic phase evolution. The ayesian bound tends to the deterministic asymptote J Dl = 4 E s N 0 given by. One can note that the phase evolution is not stationary i.e., var k = var k + w, and consequently that uncertainty over the phase offset grows with time. Moreover as the phase model is not restricted to a finite horizon, the MSE is not upper bounded. This is why there is no observed saturation as one could expect with a traditional ayesian bound. VIII. CONCLUSION In this contribution, we have derived an analytical expression of a ayesian Cramér-Rao ound for the estimation of a realistic dynamical phase. We provide a closed-form expression in the case of a PS transmission disturbed by AWGN in a NDA context. The reasoning can be extended to any M-ary Phase Shift eying transmissions. Furthermore, we can readily predict the bound behavior from its low and high-snr asymptotes. We combine these asymptotes and we give a new ayesian bound which is easy to evaluate at the price of being slightly lower than the CR. These bounds are useful when analyzing the performance of actual phase-

8 7 tracking estimators in stressing environments for both the offline and on-line scenarios. APPENDIX I EXPRESSION OF DIAGONAL ELEMENTS OF THE CR In this appendix, we detail the calculus to obtain the diagonal elements of the inverse of the IM 0. We first use the classical matrix-inversion formula k,k = C k,k, 45 where C k,k is the cofactor of the element k,k and is the determinant of matrix. In the sequel, we focus on the derivation of particular expressions of C k,k and. We first present some preliminary calculus sections I-A to I-C aiming at simplifying the expansion of and C k,k. Then, in section I-D, we derive a closed-form expression of the diagonal elements of. A. Preliminary calculus : Determinant d k In this subsection, we derive an analytical expression of the determinant d k of a k k matrix D k defined as D k = b A A A A 46 Expanding d k along the first and the second column, one obtains the recursive equation d k = Abd k b d k with d 0 = and d = ba. 47 {d k } k=0 is thus a linear recurrent sequence and its characteristic polynomial p x = x Abx + b has real roots r and r. Then, using the initial terms, an analytical expression of d k is given by where r = b r = b d k = ρ r k + ρ r k for k N, 48. Determinant A + A A 4 + A 4, ρ = A 4, A A 4, ρ = A 4 A A 4. In this subsection, we derive an expression of by using the preliminary result derived Appendix I-A. In a first step, expanding this determinant along the first column, we obtain the sum of two cofactors only. Then, expanding in turn these cofactors along the last column and using 47, we have or with 48, = A + b = A + b d, 49 ρ r + ρ r. 50 C. Cofactor C k,k In order to calculate the cofactor C k,k, one has to delete the k th row and the k th column of the matrix and one obtains a two-block diagonal matrix. The upper respectively lower block is noted U respectively L. The cofactor which is obtained from the determinant of the previous matrix is thus the product of two determinants: C k,k = detu detl 5 = bd k + d k bd k + d k,, 5 where d k is calculated in Appendix I-A. D. Expression of the diagonal elements of An analytical expression of the diagonal elements of the inverse of 0 is now derived. Then, the behavior of these elements along their index k and the matrix size is studied. Rewriting 45 and using 5, one has k,k = bd k + d k bd k + d k. 53 Then, using 48, can be expanded as follows k,k k,k = ρ b + r r 3 + ρ b + r r 3 b A rk r k + r k r k, which is equivalent to. For a fixed block length from, note that k,k depends on the index k through the expression g k where g x = r x r x + r x r x. 54 After analyzing this function, by classically studying the sign of dg dx, one readily finds that the symmetry of g x with respect to + implies that k,k = with l = + k, l,l the minimal diagonal element is the mid-coefficient +,, + the maximal diagonal elements are and,,. APPENDIX II DERIVATION OF EXPRESSION 44 Using 53, can be written as follows:,, = A + b A + d b, 55 A + d where d is defined in Appendix I-A. Since is, d only a function of d, we consider the fixed points of a sequence {u n } n=0 defined as u n d n bd n = Au n u n, 56

9 8 with u = A. Clearly, this sequence is strictly increasing and converges to u = A A 4. Combining this result with 55, we have that, = A + b A + b A + u, 57 is a strictly decreasing sequence with the following limit: lim, = A + A b A + A b A + Using the definitions of A and b in subsection IV-C, one obtains lim, = w + w w JD. 59 APPENDIX III HIGH-SNR ASYMPTOTE OF J D Defining F H k E yk k Rx n k sign Rx k, 34 becomes J D E k F H. 60 Since the noise affecting the observation is circular, x k = a k + ñ k where ñ k = n k e j k is a circular Gaussian noise with variance, therefore, we find F H k = E ak,ñ I k k n ak + ñ I ak k sign + ñ I k, where ñ I k R{ñ k}. Using first the definition of the expectation and then the definition of the error function erf F H k = a k {,} n = + + ak + ñ I k sign a k + ñ I k pñ I k dñ I k pñ I kdñ I k + 4 = e + n π erf n +, ñ I kpñ I kdñ I k. 6 Since 6 does not depend on k, we finally obtain J D E k F H k = e n + n π erf. 6 n APPENDIX IV PROOF THAT CR > ACR FOR J Dl < J Dh To prove 40, we show that J D J Dl =E yk, k 4 4 n J D J Dl < 0, 63 Using the definition of J D and J Dl, i.e., 4 and 38, we have I x k tanh Rx k 64 + Rx k n tanh Rx k Rx k. For the sake of conciseness, we use the following notations: X Rx k and Y Ix k. Taking then into account that the expectation in 64 is independent of k, we have 4 J D J Dl = E X,Y 4 + X Y tanh X tanh n X X. 65 Since the noise affecting the observations is circular, X and Y are independent. Moreover Y is a zero-mean random Gaussian variable with variance n/. Hence, J D J Dl = E X fx, 66 where fx X tanh n X n X + tanh n X n and where X is a random variable with the following probability density function pdf px = X e n p a = + + X+ e n p a =. n π n π Then we rewrite 66 as J D J Dl = EX a=+ f X + E X a= f X. 67 Noting that fx = f X, we deduce from a change of variable X = X that the two terms in the right-hand side of 67 are equal. This equation thus becomes J D J Dl = E X a=+ f X. 68 We now prove that E X a=+ f X is negative. We use Stein s lemma 4: suppose Z is a normally distributed random variable with expectation µ and variance v. Furthermore suppose g is a function for which the two expectations EZ µgz and Eg Z both exist. Then ve Z g Z = E Z Z µgz. 69 Applying this lemma to the random variable X with expectation µ = and variance / and to gx = tanh X, n we obtain E X a=+ tanh X 70 = E X a=+ X tanh X. Moreover, we directly have 4 E X a=+ 4 X = Then, plugging equations 70 and 7 into equation 68 J D J Dl = E X a=+ tanh X Considering that X is a Gaussian variable with expectation µ =, we now show that E X a=+ tanh X n X < n 0.

10 9 Suppose Z is a normally distributed random variable with expectation µ = and variance v. Denoting its pdf by pz, we have for any λ > 0 E Z tanh λz λz = = 0 tanh λz λz pzdz 73 tanh λz λz pz p Z dz. 74 As pz p Z > 0 for any Z > 0, and as tanhλz < λz for any Z > 0 and λ > 0, we actually have E Z tanh λz λz < 0, so that E Z tanh λz < λ, λ > As X is a normally distributed random variable with expectation µ =, applying this result with λ =, we obtain n E X a=+ tanh X 4 4 < 0, 76 and finally from equations 7 and 76 we have proved equation 63 J D J Dl < 0, n > J. Dauwels and S. orl, A numerical method to compute Cramér-Raotype bounds for challenging estimation problems, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, Toulouse, Fr, May 006, pp J. A. McNeill, Jitter in ring oscillators, Ph.D. dissertation, oston University, A. Demir, A. Mehrotra, and J. Roychowdhury, Phase noise in oscillators: a unifying theory and numerical methods for characterization, IEEE Trans. Circuits Syst. I, vol. 47, pp , May H. L. V. Trees, Detection, Estimation and Modulation Theory. New York: Wiley, 968, vol.. 0. Z. obrovsky, E. Mayer-Wolf, and M. Zakai, Some classes of global Cramér-Rao bounds, Ann. Statistics, vol. 5, pp , 987. S. M. ay, Fundamentals of statistical signal processing: estimation theory. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 993. H. Steendam and M. Moeneclaey, Low-SNR limit of the Cramér- Rao bound for estimating the time delay of a PS, QAM, or PAM waveform, IEEE Commun. Lett., vol. 5, pp. 3 33, Jan P. Amblard, J. rossier, and E. Moisan, Phase tracking: what do we gain from optimality? Particle filtering versus phase-locked loops, Elsevier Signal Processing, vol. 83, pp. 5 67, Oct C. M. Stein, Estimation of the mean of a multivariate normal distribution, The Annals of Statistics, vol. 9, no. 6, pp. 35 5, Nov. 98. REFERENCES R. Gallager, Low-Density Parity-Check Codes, IEEE Trans. Inform. Theory, vol. 8, pp. 8, Jan. 96. C. errou and A. Glavieux, Near optimum error correcting coding and decoding: turbo-codes, IEEE Trans. Commun., vol. 44, pp. 6 7, Oct R. Pyndiah, Near-optimum decoding of product codes: block turbo codes, IEEE Trans. Commun., vol. 46, no. 8, pp , Aug H. Meyr, M. Moeneclaey, and S. Fechtel, Digital Communication Receivers: Synchronization, Channel Estimation, and Signal Processing. New York, NY, USA: John Wiley & Sons, Inc., R. A. Fisher, On the mathematical foundations of theoretical statistics, Phil. Trans. Royal Soc., vol., pp , 9. 6 H. Cramér, Mathematical Methods of Statistics. New York: Princeton University, Press, D. C. Rife and R. R. oorstyn, Single tone parameter estimation from discrete-time observations, IEEE Trans. Inform. Theory, vol. 0, pp , W. G. Cowley, Phase and frequency estimation for PS packets: bounds and algorithms, IEEE Trans. Commun., vol. 44, pp. 6 8, Jan N. Noels, H. Steendam, and M. Moeneclaey, The Cramér-Rao bound for phase estimation from coded linearly modulated signals, IEEE Commun. Lett., vol. 7, pp , May Y. Jiang, F. Sun, and J. aras, On the true Cramér-Rao lower bound for the DA joint estimation of carrier phase and timing offsets, in IEEE International Conference on Communications, New Orleans, USA, June 000, pp M. F. Rice,. Cowley and M. Rice, Cramér-Rao Lower ounds for QAM Phase and Frequency Estimation, IEEE Trans. Commun., vol. 49, pp , Sept. 00. A. D Andrea, U. Mengali, and R. Reggiannini, The modified Cramér- Rao bound and its application to synchronization problems, IEEE Trans. Commun., vol. 4, pp , Fev./Mar./Apr F. Gini, R. Reggiannini, and U. Mengali, The modified Cramér-Rao bound in vector parameter estimation, IEEE Trans. Commun., vol. 46, pp. 5 60, Jan M. Moeneclaey, On the true and the modified Cramér-Rao bounds for the estimation of a scalar parameter in the presence of nuisance parameters, IEEE Trans. Commun., vol. 46, pp , Nov P. Tichavsky, C. H. Muravchik, and A. Nehorai, Posterior Cramér- Rao bounds for discrete-time nonlinear filtering, IEEE Trans. Signal Processing, vol. 46, pp , May 998.