MEASURE TRANSFORMED QUASI LIKELIHOOD RATIO TEST

Transcription

1 MEASURE TRANSFORMED QUASI LIKELIHOOD RATIO TEST Koby Todros Ben-Gurion University of the Negev Alfred O. Hero University of Michigan ABSTRACT In this paper, a generalization of the Gaussian quasi likelihood ratio test GQLRT for simple hypotheses is developed. The proposed generalization, called measure-transformed GQLRT MT-GQLRT, applies GQLRT after transformation of the probability measure of the data. By judicious choice of the transform we show that, unlike the GQLRT, the proposed test can gain sensitivity to higher-order statistical information resilience to outliers leading to significant mitigation of the model mismatch effect on the decision performance. Under some mild regularity conditions we show that the MT- GQLRT is consistent its corresponding test statistic is asymptotically normal. A data driven procedure for optimal selection of the measure transformation parameters is developed that maximizes an empirical estimate of the asymptotic power given a fixed empirical asymptotic size. The MT-GQLRT is applied to signal classification in a simulation example that illustrates its sensitivity to higher-order statistical information resilience to outliers. Index Terms Higher-order statistics, hypothesis testing, probability measure transform, robust classification, signal classification.. INTRODUCTION Classical simple binary hypothesis testing deals with the decision problem between two hypotheses based on a sequence of multivariate samples from an underlying probability distribution that belongs to a pair set of probability measures with corresponding singleton parameter spaces. When the probability distributions under each hypothesis are correctly specified the likelihood ratio test LRT, which is the most powerful test for a given size 2, can be implemented that utilizes complete statistical information. In many practical scenarios the probability distributions are unknown, therefore, one must resort to other suboptimal tests that require partial statistical information. A popular test of this kind is the Gaussian quasi LRT GQLRT 3-8 that assumes Gaussian distributions under each hypothesis. The GQLRT operates by selecting the Gaussian probability model that best fits the data. When the observations are i.i.d. this selection is carried out by comparing the empirical Kullback-Leibler divergences 9 between the underlying probability distribution the assumed normal probability measures. The GQLRT has gained popularity due to its implementation simplicity, performance analysis traceability, its insightful geometrical interpretations. Despite the model mismatch, introduced by the normality assumption, the GQLRT has the appealing property of consistency when the mean vectors covariance matrices are correctly specified identifiable over the considered hypotheses 6. However, in some circumstances, such as for certain types of non-gaussian data, large deviation from normality can inflict poor decision performance. This can occur when the first second-order statistical moments are weakly identifiable over the considered hypotheses, or in the case of heavy-tailed data when the non-robust sample mean covariance provide poor estimates in the presence of outliers. In this paper, a generalization of the GQLRT is proposed that applies GQLRT after transformation of the probability distribution of the data. Under the proposed generalization new tests can be obtained that can gain sensitivity to higher-order statistical information, resilience to outliers, yet have the computational implementation advantages of the GQLRT. This generalization, called measure-transformed GQLRT MT-GQLRT, is inspired by the measure transformation approach that was recently applied to canonical correlation analysis 0,, independent component analysis ICA 2, multiple signal classification MUSIC 3, 4 parameter estimation 5. The proposed transform is structured by a non-negative function, called the MT-function, maps the probability distribution into a set of new probability measures on the observation space. By modifying the MT-function, classes of measure transformations can be obtained that have different useful properties. Under the proposed transform we define the measure-transformed MT mean vector covariance matrix, derive their strongly consistent estimates, study their sensitivity to higher-order statistical information resilience to outliers. Similarly to the GQLRT, the proposed MT-GQLRT compares the empirical Kullback-Leibler divergences between the transformed probability distribution of the data two normal probability measures that are characterized by the MT-mean vector MTcovariance matrix under each hypothesis. Under some mild regularity conditions we show that the proposed test is consistent its corresponding test statistic is asymptotically normal. Furthermore, given two training sequences from the probability distribution under each hypothesis, a data-driven procedure for optimal selection of the MT-function within some parametric class of functions is developed that maximizes an empirical estimate of the asymptotic power given a fixed empirical asymptotic size. We illustrate the MT-GQLRT for the problem of signal classification in the presence of heavy-tailed spherically contoured noise 6 that produces outliers. By specifying the MT-function within the family of zero-centered Gaussian functions parameterized by a scale parameter, we show that the MT-GQLRT outperforms the nonrobust GQLRT attains classification performance that are significantly closer to those obtained by the LRT that, unlike the MT- GQLRT, requires complete knowledge of the likelihood function under each hypothesis. The paper is organized as follows. In Section 2, a transformation on the probability distribution of the data is developed. In Section 3, we use this transformation to construct the MT-GQLRT. The proposed test is applied to a signal classification problem in Section 4. In Section 5, the main points of this contribution are summarized. Proofs for the propositions corollaries stated throughout the paper will be provided in the full length journal version.

2 2. PROBABILITY MEASURE TRANSFORM In this section, we develop a transform on the probability measure of a rom vector. Under the proposed transform, we define the measure-transformed mean vector covariance matrix, derive their strongly consistent estimates, establish their sensitivity to higher-order statistical information resilience to outliers. These quantities will be used in the following section to construct the proposed measure-transformed GQLRT. 2.. Probability measure transformation We define the measure space, S, P, where C p is the observation space of a rom vector, S is a σ-algebra over P is an unknown probability measure on S parameterized by a vector parameter θ that belongs to a pair set Θ {θ 0, θ }. Definition. Given a non-negative function u : C p R + satisfying 0 < E u ; P <, where E u ; P u x dp x x, a transform on P is defined via the relation: Q u A T u P A = ϕ u x; θ dp x, 2 where A S ϕ u x; θ u x/e u ; P. The function u is called the MT-function. Proposition Properties of the transform. Let Q u be defined by relation 2. Then Q u is a probability measure on S. 2 Q u is absolutely continuous w.r.t. P, with Radon-Nikodym derivative 7: A dq u x/dp x = ϕ u x; θ. 3 The probability measure Q u is said to be generated by the MT-function u. By modifying u, such that the condition is satisfied, virtually any probability measure on S can be obtained The MT-mean MT-covariance According to 3 the mean vector covariance matrix of under Q u are given by: µ u E ϕ u ; θ ; P 4 Σ u E H ϕ u ; θ ; P µ u µuh, 5 respectively. Equations 4 5 imply that µ u Σ u are weighted mean covariance of under P, with the weighting function ϕ u ; defined below 2. Hence, they can be estimated using only samples from the distribution P. By modifying the MT-function u, such that the condition is satisfied, the MTmean MT-covariance under Q u are modified. In particular, by choosing u to be any non-zero constant valued function we have Q u = P, for which the stard mean vector µ covariance matrix Σ are obtained. Given a sequence of N i.i.d. samples from P the estimators of µ u Σ u are defined as: ˆµ u N n ˆϕ u n 6 ˆΣ u n H n ˆϕ u n ˆµ u x ˆµuH x, 7 respectively, where ˆϕ u n u N n/ u n. According to Proposition 2 in 3, if E 2 2 u ; P < then ˆµ u w.p. u µu ˆΣ convergence with probability w.p Robustness to outliers w.p. Σu, where w.p. denotes Robustness of the empirical MT-covariance 7 to outliers was studied in 3 using its influence function 9 which describes the effect on the estimator of an infinitesimal contamination at some point y C p. An estimator is said to be B-robust if its influence function is bounded 9. In 3 we have shown that if the MT-function uy the product uy y 2 2 are bounded over C p then the influence function of the empirical MT-covariance is bounded. Similarly, it can be shown that under the same conditions the influence function of the empirical MT-mean 6 is bounded Sensitivity to higher-order statistical information Notice that for any non-constant analytic MT-function u, which has a convergent Taylor series expansion, the MT-mean 4 the MT-covariance 5 involve higher-order statistical moments of P. In particular, by choosing u x; t exp Re { t H x }, t C p, the resulting exponential MT-mean MT-covariance are the gradient Hessian of the cumulant generating function up to some scaling factors that have been used for parameter estimation, ICA channel identification in Moreover, by choosing u x; t, τ exp x t 2 /τ 2, τ R ++, we obtain the Gaussian MT-mean MT-covariance that have been used for non-linear correlation analysis, ICA, robust MUSIC parameter estimation in MEASURE-TRANSFORMED GAUSSIAN QUASI LIKELIHOOD RATIO TEST In this section we use the measure transformation 2 to construct a test between the null alternative simple hypotheses H 0 : θ = θ 0 H : θ = θ based on a sequence of samples n, n =,..., N from P. Regularity conditions for asymptotic normality of the corrsponding test statistic are derived. When these conditions are satisfied we show that the proposed test is consistent derive its asymptotic size power. Optimal selection of the MT-function u out of some parametric class of functions is also discussed. 3.. The MT-GQLRT Let Φ u denote a complex circular Gaussian probability distribution 28 that is characterized by the MT-mean µ u MT-covariance Σ. u The proposed MT-GQLRT compares the empirical Kullback- Leibler divergence KLD between Q u Φ u 0 KLD between Q u Φ u k {0, } is defined as 9: D KL Q u Φ u k E to the empirical. The KLD between Qu Φ u k, log qu ; θ φ u ; ; Qu, 8 where q u x; θ φ u x; are the density functions of Q u Φ u k, respectively. According to 3, D KL Q Φ u u k can

3 be estimated using only samples from P. Hence, similarly to 6 7, an empirical estimate of 8 given a sequence of samples n, n =,..., N from P is defined as: ˆD KL Q u Φ u k ˆϕ u n log qu n; θ φ u n;, 9 where ˆϕ u is defined below 7. Hence, the proposed test statistic is given by T u ˆD KL Q u Φ u ˆD 0 KL Q u Φ u 0 = ˆϕ u n ψ u n; θ 0, θ where = D ˆΣu LD Σ u 0 D ˆΣu LD Σ u + + ˆµ u µu 0 ˆµ u µ u ψ u ; θ 0, θ log φu n; θ φ u n; θ 0, 2 Σ u x;θ 0 2 Σ u x;θ D LD A B tr AB log det AB p is the logdeterminant divergence 29 between positive definite matrices A, B C p p a C a H Ca denotes the weighted Euclidian norm of a vector a C p with positive-definite weighting matrix C C p p. The decision rule based on the test statistic 0 is given by T u H H 0 t, where t R denotes a threshold value. Notice that for any non-zero constant MT-function, u, Q u = P the stard GQLRT is obtained which only involves first second-order moments Asymptotic performance analysis Here, we study the asymptotic performance of the proposed test. For simplicity, we assume a sequence of i.i.d. samples n, n =,..., N from P. Proposition 2 Asymptotic normality. Assume that the following conditions are satisfied: µ u 0 µu or Σu 0 Σu. 2 Σ u Σu are non-singular. 3 E u 2 ; P 0 E 4 2 u2 ; P are finite for θ = θ = θ. Then, D T u N η u θ, λ u θ θ Θ, where D denotes convergence in distribution 8, the mean η u θ E ϕ u ; θ ψ u ; θ 0, θ ; P, the variance N Var ϕ u ; θ ψ u ; θ 0, θ ; P. λ u θ Corollary Asymptotic size power. Assume that the conditions stated in Proposition 2 are satisfied. The asymptotic size power of the test are given by: α u Q t ηu λ u β u Q t ηu λ u,, 2 respectively, where Q denotes the tail probability of the stard normal distribution. Corollary 2 Consistency. Assume that the conditions in Proposition 2 are satisfied. Then, for any fixed asymptotic size the asymptotic power of the test satisfies β u as N. In the following Proposition, strongly consistent estimates of the asymptotic size power 2 are constructed based two i.i.d. sequences from P 0 P. These quantities will be used in the sequel for optimal selection of the MT-function. Proposition 3 Empirical asymptotic size power. Let k n, n =,..., N k, k = 0, denote sequences of i.i.d. samples from P 0 P, respectively. Define the empirical asymptotic size power: ˆα u Q t ˆηu ˆλu respectively, where ˆη u Nk ˆϕ2 u ˆβ u Q t ˆηu ˆλu, 3 N k ˆϕu k n ψ u k n ; θ 0, θ 2. k n ; θ 0, θ ˆη u N u ˆλ N k k N n ψu 2 Assume that E u 2 ; P E 4 u 2 ; P are fi- w.p. nite for any θ Θ. Then, ˆα u N 0 αu ˆβ w.p. u βu. N 3.3. Optimal selection of the MT-function We propose to specify the MT-function within some parametric family {u ; ω, ω Ω C r } that satisfies the conditions stated in Definition Proposition 2. An optimal choice of the MTfunction parameter ω would be this that maximizes the empirical asymptotic power 3 at a fixed empirical asymptotic size ˆα u = α, i.e., we maximize the following objective function: ˆβ α u ω = Q ˆηu ω ˆηu ω + ˆλu ωq α ˆλu. ω 4. EAMPLE 4 We consider a signal classification problem that is stated as the problem of testing the following simple hypotheses: H 0 : n = S n θ 0 + W n, n =,..., N, 5 H : n = S n θ + W n, n =,..., N, where n C p is an observation vector, S n θ S nθ is a latent rom vector, S n C is a first-order stationary rom signal that is symmetrically distributed about the origin, θ C p is a deterministic unit norm vector W n C p is a first-order stationary additive noise that is statistically independent of S n. We assume that the noise component is spherically contoured with stochastic representation 6: W n = ν nz n, 6 where ν n R ++ is a first-order stationary process Z n C p is a proper-complex wide-sense stationary Gaussian process with zeromean scaled unit covariance σ 2 ZI. The processes ν n Z n are assumed to be statistically independent.

4 In order to gain robustness against outliers, as well as sensitivity to higher-order moments, we specify the MT-function in the zerocentred Gaussian family of functions parametrized by a width parameter ω, i.e., u x; ω = exp x 2 /ω 2, ω R Notice that the MT-function 7 satisfies the B-robustness conditions stated in Subsection 2.3. Using 4, it can be shown that the MT-mean MT-covariance under Q u are: µ u ω = 0 8 Σ u ω = r S ω θθ H + r W ω I, 9 respectively, where r S ω r W ω are some strictly positive functions of ω. Hence, by substituting 7-9 into 0 followed by normalization by the observation-independent factor c ω, the MT-GQLRT simplifies to r S ω r W ωr S ω+r W ω T u T u/cω = θ H Ĉu ω θ θ H H 0 Ĉu ω θ 0 t, H 0 u where Ĉu ω ˆΣ ω+ ˆµ u ω ˆµ uh ω t t/cω. Under the considered settings, it can be shown that the conditions stated in Proposition 2 are satisfied. The resulting asymptotic power 2 at a given asymptotic size α u = α takes the form: ω = Q Gω 2N + Q α, β α u θ H where G ω EgS, 2 ν,ωh 2S, 2 ν,ω;p S,ν, ν νσ E 2 S 2 hs, ν,ω;p Z, S,ν g S, ν, ω ω 2 S 2 2 ω 2 +ν + 3ν 2 ω 2 S 2 + ν 4, h S, ν, ω 2 ω 2 +ν 2 ω 2 p+2 ν 2 +ω exp. S 2 Furthermore, its empirical estimate 2 ν 2 +ω 2 4 is given by ˆβ u α ω = Q ηu ω ηu ω + λu ωq α λu, ω where η u ω ˆη u λ u N ω N k = cω ω = N k c 2 k ˆϕ 2 ω N u ˆϕ u n k ; ω 2 ξ k n ; θ 0, θ, ω ˆλ u n k ; ω ξ 2 n k ; θ 0, θ 2, η u k = 0,, ξ ;, θ θ H 2 θ H 0 2. In the following simulation examples, the vectors θ 0 θ were set to p, e iπ sinϑ k,..., e iπp sinϑ k T, k = 0,, were ϑ 0 = 0, ϑ = π/3 p = 4. We consider a BPSK signal with variance σ 2 S an ɛ-contaminated Gaussian noise model 6 under which the texture component ν in 6 is a binary rom variable satisfying ν = w.p. ɛ ν = δ w.p. ɛ. The parameters ɛ δ that control the heaviness of the noise tails were set to 0.2 0, respectively. We define the signal-to-noise-ratio SNR as SNR 0 log 0 σ 2 S/ σ 2 Z ɛ + ɛδ 2. In all simulation examples the sample size was set to N = 0 3. The empirical asymptotic power 2 was obtained using two i.i.d. training sequences from P 0 P containing N 0 = N = 0 5 samples. In the first example, we compared the asymptotic power 20 to its empirical estimate 2 at test size equal to 0.05 as a function of ω for SNR = 0 db. Observing Fig., one sees that due to the consistency of 2 the compared quantities are very close. In the second example, we compared the empirical, asymptotic 20 empirical asymptotic 2 power of the MT-GQLRT to the empirical powers of the GQLRT the LRT for test size equal to The optimal Gaussian MT-function parameter ω opt was obtained by maximizing 2 over Ω =, 00. The empirical power curves were obtained using 0 4 Monte-Carlo simulations. The SNR is used to index the performances as depicted in Fig. 2. One sees that the MT-GQLRT outperforms the non-robust GQLRT for most examined SNR values performs similarly to the LRT that, unlike the MT-GQLRT, requires complete knowledge of the likelihood function under each hypothesis. Power Asymptotic Power Empirical asymptotic Power ω Fig.. Asymptotic power 20 its empirical estimate 2 at test size α = 0.05 versus the width parameter ω of the MT-function 7. Power Empirical power: MT GQLRT Asymptotic power: MT GQLRT Empirical asymptotic power: MT GQLRT Empirical power: GQLRT Empirical power: LRT SNR db Fig. 2. The empirical, asymptotic 20 empirical asymptotic 2 powers of the MT-GQLRT as compared to the empirical powers of the GQLRT LRT at test size α = CONCLUSION In this paper a new test, called MT-GQLRT, was developed that applies Gaussian LRT after transformation of the probability distribution of the data. By specifying the MT-function in the Gaussian family, the proposed test was applied to signal classification in non- Gaussian noise. Exploration of other MT-functions may result in additional tests in this class that have different useful properties.

5 6. REFERENCES L. L. Scharf, Statistical signal processing, Addison-Wesley, J. Neyman E. S. Pearson, On the problem of the most efficient tests of statistical hypotheses, Philosophical Transactions of the Royal Society of London A: Mathematical, Physical Engineering Sciences, vol. 23, pp , W. K. Härdle, V. Spokoiny, V. Panov, W. Wang, Basics of Modern Mathematical Statistics, Springer, M. Taniguchi Y. Kakizawa, Asymptotic theory of statistical inference for time series, Springer, J. Bai, Likelihood ratio tests for multiple structural changes, Journal of Econometrics, vol. 9, no. 2, pp , K. Sakiyama M. Taniguchi, Discriminant analysis for locally stationary processes, Journal of Multivariate Analysis, vol. 90, no. 2, pp , I. A. Canay, EL inference for partially identified models: large deviations optimality bootstrap validity, Journal of Econometrics, vol. 56, no. 2, pp , M. C. Beaulieu, J. M. Dufour L. Khalaf, Finite sample multivariate tests of asset pricing models with coskewness, Computational Statistics & Data Analysis, vol. 53, no. 6, pp , S. Kullback R. A. Leibler, On information sufficiency, The Annals of Mathematical Statistics, vol. 22, pp , K. Todros A. O. Hero, On measure-transformed canonical correlation analysis, IEEE Transactions on Signal Processing, vol. 60, no. 9, pp , 202. K. Todros A. O. Hero, Measure transformed canonical correlation analysis with application to financial data, Proceedings of SAM 202, pp , K. Todros A. O. Hero, Measure independent component analysis, available online as ariv: K. Todros A. O. Hero, Robust Multiple Signal Classification via Probability Measure Transformation, IEEE Transactions on Signal Processing, vol. 63, no. 5, K. Todros A. O. Hero, Robust Measure transformed MUSIC for DOA estimation, Proceeding of ICASSP 204, pp , K. Todros A. O. Hero, Measure-transformed quasi maximum likelihood estimation with application to source localization, Proceedings of ICASSP 205, pp E. Ollila, D. E. Tyler, V. Koivunen H. V. Poor, Complex elliptically symmetric distributions: survey, new results applications, IEEE Transactions on Signal Processing, vol. 60, no., pp , G. B. Foll, Real Analysis. John Wiley Sons, K. B. Athreya S. N. Lahiri, Measure theory probability theory, Springer- Verlag, F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw W. A. Stahel, Robust statistics: the approach based on influence functions. John Wiley & Sons, A. Slapak A. Yeredor, Charrelation Charm: generic statistics incorporating higher-order information, IEEE Transactions on Signal Processing, vol. 60, no. 0, pp , A. Slapak A. Yeredor, Charrelation-based estimation of the parameters of non-gaussian autoregressive processes, Proceedings of SSP 202, pp A. Yeredor, blind source separation via the second characteristic function, Signal Processing, vol. 80, pp , A. Slapak A. Yeredor, Weighting for more : enhancing characteristicfunction based ICA with asymptotically optimal weighting, Signal Processing, vol. 9, no. 8, pp , A. Yeredor, Blind channel estimation using first second derivatives of the characteristic function, IEEE Signal Processing Letters, vol. 9, pp , A. Yeredor, Substituting the cumulants in the super-exponential blind equalization algorithm, Proceedings of the ICASSP E. Edinger A. Yeredor, Blind MIMO identification using the second characteristic function, IEEE Transactions on Signal Processing, vol. 53, pp , A. Yeredor, MUSIC using off-origin Hessians of the second characteristic function, Proceedings of SAM 2006, pp P. Schreier L. L. Scharf, Statistical signal processing of complex-valued data: the theory of improper noncircular signals. p. 39, Cambridge University Press, I. S. Dhillon J. A. Tropp, Matrix nearness problems with Bregman divergences, SIAM Journal on Matrix Analysis Applications, vol. 29, no. 4, pp , 2007.