Finite sample size optimality of GLR tests. George V. Moustakides University of Patras, Greece

Transcription

1 Finite sample size optimality of GLR tests George V. Moustakides University of Patras, Greece

2 Outline Hypothesis testing and GLR Randomized tests Classical hypothesis testing with randomized tests Alternative implementation of randomized tests A combined hypothesis testing/isolation problem Optimality of GLR Randomized estimators A combined hypothesis testing/estimation problem Optimum GLR-type tests 2

3 Hypothesis testing and GLR Binary hypothesis testing (fixed sample size N): Collection of observations X=[x 1,...,x N ] : X H 1 : X ~ f 0 (X); ~ f 1 (X); Decide between and H 1 Likelihood Ratio Rest (LR) f 1 (X) f 0 (X) H 1 λ Optimum according to different criteria: Neyman-Pearson, Bayesian, Min-Max, Equalized error probabilities. 3

4 Composite binary hypothesis testing Observations X=[x 1,...,x N ] : X f 01 (X), π 01 f 02 (X), π 02. f 0K0 (X), π 0K0 H 1 : X f 11 (X), π 11 f 12 (X), π 12. f 1K1 (X), π 1K1 One subcase responsible for the data 4

5 If we knew before hand the two possible subcases, for example: f 0i and f 1j then f 1j (X) f 0i (X) f 1 (X) f 0 (X) = π 11f 11 (X)+ + π 1K1 f 1K1 (X) π 01 f 01 (X)+ + π 0K0 f 0K0 (X) H 1 H 1 When this knowledge is not possible: If in addition to detection, we like to isolate the subcase responsible for the data X GLR f (X) H1 1^j f (X) 0^i λ ^i =arg max f 0i (X) 1 i K 0 ^j =arg max f 1j (X) 1 j K 1 λ ML isolation 5

6 Composite binary hypothesis testing (cond.) Observations X=[x 1,...,x N ] : X f 0 (X θ 0 ), π 0 (θ 0 ) H 1 : X f 1 (X θ 1 ), π 1 (θ 1 ) Only detection f 1 (X) f 0 (X) = R H π1 (θ 1 )f 1 (X θ 1 ) dθ 1 R 1 π0 (θ 0 )f 0 (X θ 0 ) dθ 0 If we like, in addition to detection to estimate the parameter vector θ i GLR H f 1 (Xj^µ 1 ) 1 f 0 (Xj^µ 0 ) λ ^µ 0 =argmaxf 0 (Xjµ 0 ) µ 0 ^µ 1 =argmaxf 1 (Xjµ 1 ) µ 1 ML estimate 6

7 Optimality of GLR Wald (1943) Le Cam (1953) Lehmann (1959) Hoeffding (1965) Neyman (1965) M Asymptotic optimality under regularity conditions. As the number of samples increases without limit, performance of GLR approaches the optimum test (with true parameters considered known). 7

8 Randomized tests Binary hypothesis testing (fixed sample size N): Collection of observations X=[x 1,...,x N ] : X ~ f 0 (X); H 1 : X ~ f 1 (X); Decide between and H 1 Deterministic tests X R N wo complementary sets in R N : A 0,A 1 with A 1 =A 0 When X A i decide for H i. c Randomized tests wo complementary probabilities: δ 0 (X),δ 1 (X) with δ 0 (X)+δ 1 (X)=1. In a random game select H i with probability δ i (X). 8

9 Randomized more general than Deterministic Neyman-Pearson: ± i (X) =1 Ai (X) When X A 1 then we decide with probability 1 for H 1 and with probability 0 for (the equivalent of a deterministic decision). max P[d =1jH 1 ]; subject to ± 0 ;± 1 P[d =1j ] ± 1 (X)f 1 (X) dx - λ ± 1 (X)f 0 (X) dx = ± 1 (X)[f 1 (X) f 0 (X)] dx maxff 1 (X) f 0 (X); 0g dx 9

10 ± 1 (X) = 8 < : 1 f 1 (X) f 0 (X) > 0 0 f 1 (X) f 0 (X) < 0 f 1 (X) f 0 (X) =0: ± 0 (X) =1 ± 1 (X) = 8 < : 0 f 1 (X) f 0 (X) > 0 1 f 1 (X) f 0 (X) < 0 1 f 1 (X) f 0 (X) =0: Likelihood Ratio Rest (LR) f 1 (X) f 0 (X) H 1 λ 10

11 Composite binary hypothesis testing Observations X=[x 1,...,x N ] : X f 01 (X), π 01 f 02 (X), π 02. f 0K0 (X), π 0K0 H 1 : X f 11 (X), π 11 f 12 (X), π 12. f 1K1 (X), π 1K1 δ 01 (X) δ 02 (X). ± 0K0 (X) δ 11 (X) δ 12 (X). ± 1K1 (X) 11

12 δ 01 (X) δ 02 (X) ± 0K0 (X) δ 11 (X) δ 12 (X) ± 1K1 (X) X i=0;1 XK i j=1 ± ij (X) =1 In a random game, with probability δ ij (X): decide for Hypothesis H i and subcase j. We SIMULANEOUSLY Detect and Isolate 12

13 Alternative two-step implementation ± 01 (X);± 02 (X);:::;± 0K0 (X) + {z } ± 0 (X) ± 11 (X);± 12 (X);:::;± 1K1 (X) {z + } ± 1 (X) q ij (X) = ± ij(x) ± i (X) ± 0 (X)+± 1 (X) =1 q 01 (X)+ + q 0K0 (X) =1 q 11 (X)+ + q 1K1 (X) =1 ± 0 (x);± 1 (X); [q 01 (X);:::;q 0K0 (X)]; [q 11 (X);:::;q 1K1 (X)] With probabilities δ 0 (X),δ 1 (X) decide between,h 1. Given we selected H i, isolate among the subcases of H i with probabilities q i1 (X), q i2 (X),..., q iki (X): 13

14 Optimality of GLR : X» f 01 (X);¼ 01» f 02 (X);¼ 02.» f 0K0 (X);¼ 0K0 H 1 : X» f 11 (X);¼ 11» f 12 (X);¼ 12.» f 1K1 (X);¼ 1K1 A Combined Detection / Isolation problem Following a Neyman-Pearson like approach, we propose max P[Correct-detection/isolationjH 1 ] subject to the false alarm constraint P[Miss-detection/isolationj ] 14

15 HEOREM: he test that solves the combined detection/isolation problem is the following max ¼ 1j f 1j (X) 1 j K 1 max ¼ 0j f 0j (X) 1 j K 0 H 1 ( Randomization probability γ ) hreshold λ and randomization probability γ are selected to satisfy the false alarm constraint with equality. 15

16 PROOF P[Correct-detection/isolationjH i ] = XK i j=1 P[Correct-detection/isolationjH ij ]¼ ij P[Correct-detection/isolationjH ij ] = ± i (X)q ij (X)f ij (X) dx: Constrained optimization problem: Lagrange multiplier P[Correct-detection/isolationjH 1 ] P[Miss-detection/isolationj ] = P[Correct-detection/isolationjH 1 ] + P[Correct-detection/isolationj ] 16

17 P[Correct-detection/isolationjH 1 ] + P[Correct-detection/isolationj ] = ± 1 (X) ( K1 X j=1 + q 1j (X)¼ 1j f 1j (X) ± 0 (X) ( K0 X j=1 ) dx q 0j (X)¼ 0j f 0j (X) ± 1 (X) max f¼ 1j f 1j (X)gdX 1 j K 1 + ± 0 (X) max f¼ 0j f 0j (X)gdX 1 j K 0 ) dx 17

18 = h ± 1 (X) max 1 j K 1 f¼ 1j f 1j (X)g ½ max + ± 0 (X) max 1 j K 0 f¼ 0j f 0j (X)g i dx max 1 j K 1 f¼ 1j f 1j (X)g; max 1 j K 0 f¼ 0j f 0j (X)g max ¼ 1j f 1j (X) 1 j K 1 max ¼ 0j f 0j (X) 1 j K 0 H 1 ¾ dx If priors not known, assume equiprobable subcases: max K 1 1 f 1j (X) 1 j K 1 max K 0 1 f 0j(X) 1 j K 0 H 1 ) max f 1j (X) 1 j K 1 max f 0j (X) 1 j K 0 H 1 K1 K 2 Classical GLR 18

19 Randomized estimators Observations X=[x 1,...,x N ] X» f(xjµ); ¼(µ) We would like to obtain an estimate of θ Deterministic estimator: ^µ = G(X) ^µ Randomized estimator: ±^µ(x) ) ±(^µjx); ±(^µjx) d^µ =1; a PDF! How do we obtain a randomized estimate? We generate a r.v. distributed according to ±(^µjx) Deterministic estimator ^µ o ±(^µjx) =Dirac n^µ G(X) 19

20 Optimum Bayes estimation If true parameter is θ and estimate ^µ, cost is: C(^µ; µ) Select randomized estimator to minimize average cost E[C(^µ; µ)] = C(^µ; µ)±(^µjx)f(xjµ)¼(µ) dx d^µdµ ½ ¾ = ±(^µjx) C(^µ; µ)f(xjµ)¼(µ) dµ d^µ dx {z } D(^µ; X) min D(U; X) U min D(U; X) dx U o ±(^µjx) =Diracn^µ arg min D(U; X) U 20

21 Combined detection/estimation Observations X=[x 1,...,x N ] : X» f(xjµ =0) H 1 : X» f(xjµ); ¼(µ) Decide between,h 1 Whenever we decide in favor of H 1 we also like to provide an estimate of θ ^µ Detection/estimation procedure: wo-step strategy ± 0 (X);± 1 (X);q(^µjX) 21

22 Following a Neyman-Pearson like approach, we propose min E[C(^µ; µ)jh 1 ] subject to: P[Miss-detectionj ] P[Miss-detectionj ]= ± 1 (X)f(Xj0)dX Cost computation Correct Hypothesis: C(^µ; µ)± 1 (X)q(^µjX)¼(µ)f(Xjµ)dµ d^µdx Wrong Hypothesis: C(0;µ)± 0 (X)¼(µ)f(Xjµ)dµ dx 22

23 C(^µ; µ)± 1 (X)q(^µjX)¼(µ)f(Xjµ)dµ d^µdx + C(0;µ)± 0 (X)¼(µ)f(Xjµ)dµ dx + ± 1 (X)f(Xj0)dX D(U; X) ½z } { ¾ ± 1 (X)min C(U; µ)¼(µ)f(xjµ)dµ dx U ½ ¾ + ± 0 (X) C(0;µ)¼(µ)f(Xjµ)dµ dx {z } + ± 1 (X)f(Xj0)dX D(0;X) 23

24 = n ± 1 (X)[ f(xj0) + min D(U; X)] U min n f(xj0) + min U We decide in favor of H 1 when o + ± 0 (X)D(0;X) dx o D(U; X); D(0;X) dx he final test is: D(0;X) > f(xj0) + min U D(U; X) D(0;X) min D(U; X) U f(xj0) C(U; X) = C(U; µ)f(xjµ)¼(µ)dµ H 1 24

25 Examples MAP estimator: C(^µ; µ) = ½ 0 for k^µ µk 1 1 otherwise Optimum test: max µ ¼(µ)f(Xjµ) f(xj0) H 1 If prior π(θ) is uniform we recover the classical GLR max f(xjµ) µ f(xj0) H 1 25

26 MMSE estimator: C(^µ; µ) =k^µ µk 2 Optimum test: k^µ MMSE k 2 R ¼(µ)f(Xjµ) dµ f(xj0) H 1 ^µ MMSE = R µ¼(µ)f(xjµ) dµ R ¼(µ)f(Xjµ) dµ = E[µjX] 26

27 MEDIAN estimator: C(^µ; µ) =j^µ µj Optimum test: ^µmedian >< ^µ MEDIAN =arg ^µ : >: 0 8 µ¼(µ)f(xjµ) dµ f(xj0) ^µ H 1 ¼(µ)f(Xjµ) dµ ¼(µ)f(Xjµ) dµ =argf^µ : P[µ ^µjx] =0:5g 9 >= =0:5 >; 27