Detection and Estimation Theory

Transcription

1 ESE 524 Detection and Estimation Theory Joseph A. O Sullivan Samuel C. Sachs Professor Electronic Systems and Signals Research Laboratory Electrical and Systems Engineering Washington University 2 Urbauer all (Lynda answers) jao@wustl.edu J. A. O'S. ESE 524, Lecture 2, /5/9

2 Dt Detection ti Theory Which model best fits the measured data? Decision or detection or hypothesis testing Principled approach Define the concept of a model Define the concept of a decision Define an objective function to be minimized Given these, derive an optimal decision Quantify the performance (as a function of parameters) J. A. O'S. ESE 524, Lecture 2, /5/9 2

3 Example: Throwing Darts Suppose that when people throw darts at a blackboard, there is a distribution ib i of the locations of the darts. The first individual is very good and his darts land in the vicinity of the target, with a circularly symmetric distribution. The second individual has a systematic error that he does not seem to be able to correct. is darts land in a circularly symmetric distribution offset from the truth. The third individual has no systematic error, but his darts, while still landing according to a circularly symmetric distribution around the truth, are more widespread than the first individuals. id Problem : Individual or 2 Problem 2: Individual or 3 Other problems J. A. O'S. ESE 524, Lecture 2, /5/9 3

4 Example: Throwing Darts These problems are still not -4 well posed. Need additional -6 assumptions such as: -8 A: The distributions are two dimensional Gaussian distributions around the truth 6 with known covariance matrix. 5 4 A2: All throws are independent 3 of all other throws. A3: The covariance matrix is diagonal with equal variance in each direction J. A. O'S. ESE 524, Lecture 2, /5/9 4

5 Matlab Code for Plots function errf=dartslecture2 errf=; x=randn(2,); -8 y=2*ones(2,)+randn(2,); - z=3*randn(2,); figure Can decide among hypotheses plot(x(,:),x(2,:),'bx') 6 if more throws (data) are hold on observed 5 4 plot(y(,:),y(2,:),'ro') 3 plot(z(,:),z(2,:),'gv') 2 axis equal; figure plot(x(,:),x(2,:),'bx') - hold on -2 plot(2+y(,:),2+y(2,:),'ro'), ) axis equal; errf=; J. A. O'S. ESE 524, Lecture 2, /5/

6 Probability Density Functions: M l bm Matlab Mesh h Pl Plots J. A. O'S. ESE 524, Lecture 2, /5/9 6

7 MtlbCd Matlab Code frpdfc for Computation ptti function errf=dartslecture2pdf errf=; x=-5:.:5; [x,y]=meshgrid(x); p=exp(-(x.^2+y.^2)/2)/(2*pi); (x ^2)/2)/(2*pi); p2=exp(-((x-2).^2+(y-2).^2)/2)/(2*pi); p3=exp(-(x.^2+y.^2)/8)/(8*pi); figure mesh(x,y,p) figure mesh(x,y,p2) figure mesh(x,y,p3) figure mesh(x,y,max(max(p,p2),p3)) max(max(p p2) p3)) errf=; J. A. O'S. ESE 524, Lecture 2, /5/9 7

8 Example: Throwing (One Dim) )Darts Suppose that one of two possible known signals is transmitted during the interval [,T]: s(t) or s(t) The signal is measured in white Gaussian noise: rt () = a Est () + wt (), t T { } a +, The receiver first computes the integral of r(t) times s(t) (unit energy) over the interval to get r = a E + n n N (, N / 2) Problem: a=+ or a=- J. A. O'S. ESE 524, Lecture 2, /5/9 8

9 Example: Throwing 2DD Darts In QAM (quadrature amplitude modulation), one of four signals is sent over [,T]: s( t) cos ( ωct), cos ( ωct), sin ( ωct), cos( ωct) T T T T Assume measurements are in white Gaussian noise Assuming the cosine and sine are orthogonal over the interval, the measurements are integrated against each to get two measurements. Problem: which of four signals was sent? = + r I E T T,,, 2 2 r Q () cos( ω ) () cos( ω ) T T n (, T T I N N / 2) 2 2 () sin( ω ) () sin( ω ) n N (, N / 2) rt () Est () wt (), t T r = r t t dt = E s t t dt+ n I c c I r = r t t dt = E s t t dt+ n Q c c Q T T Q J. A. O'S. ESE 524, Lecture 2, /5/9 9 n I n Q

10 Finite Dimensional Binary Detection Theory Problem: Given a measurement of a random vector r, decide which h of two models it is drawn from. Source System Decision or r or Source produces or probabilities P and P =-P. Probabilities may not be known. Random measurement vector results. Transition probabilities (system) are often assumed known. Decision rule is a partition of measurement space into two sets Z and Z corresponding to decisions and. J. A. O'S. ESE 524, Lecture 2, /5/9

11 Ot Outcomes, Bayes Criterion Critri There are four possible outcomes: Decide and is true: P (-P D ) Decide and is true: P P D Decide and is true: P (-P F ) Decide and d is true: P P F Probability of False Alarm: P F Decide given is true Probability bili of Detection: P D Decide given is true Probability of Miss: P M =-P D Decide given is true Error probabilities equal integrals of conditional (or transition) probability density functions over decision i regions J. A. O'S. ESE 524, Lecture 2, /5/9

12 Bayes Criterion Assumptions: Source prior probabilities P and P are known Transition densities are known There are costs of decision outcomes and these are known: C ij cost of deciding i and j is true C -C > is the relative cost of a miss C -C > is the relative cost of a false alarm Bayes decision criterion: Pick the decision rule to minimize the average risk (cost) R = C P ( PF ) + C P ( PD ) + C PP F + C PP D = ( C C) P ( PD) + ( C C ) PP F + CP + CP = CM P ( PD ) + CF PP F Simplified notation + C P + C P J. A. O'S. ESE 524, Lecture 2, /5/9 2

13 Results Theorem: Likelihood ratio test is optimal Corrolary: The log-likelihood ratio test is optimal For proofs, see notes and text Example: deterministic signal in Gaussian noise J. A. O'S. ESE 524, Lecture 2, /5/9 3

14 Decision i Rules Criterion Bayes: mimize expected risk or cost Transition Priors Costs Densities P &P C ij Yes Yes Yes Minimum Probability of Error Yes Yes No Minimax Yes No Yes Neyman-Pearson Yes No No R = C P P + C P P M M F F P = pr ( R ) dr M Z P = pr ( R ) dr F Z The entire space is divided into two regions: Decide or The integrals of the probability density functions determine the probabilities of error Integrate over the other or wrong region J. A. O'S. ESE 524, Lecture 4, /25/7 4

15 Bayes Criterion Define the two decision regions. Minimize Bayes risk over the decision regions. Every point in the space must be included in one integral or the other Put a point in a region if it contributes less to that integral than to the other Introduce new notation for the result of the comparison R = C P P + C P P M M F F = C MP p ( ) d + CFP r R R pr ( R ) dr C Z Pp ( R ) C Pp ( R ) M r F r Z J. A. O'S. ESE 524, Lecture 2, /5/9 5

16 Bayes Criterion Compare CMP pr ( R ) and C P p F r ( R ) Introduce new notation for the result of the comparison If the left side is bigger, decide If the right side is bigger, decide For continuous pdfs, do not worry about equality R = C P P + C P P M M F F = C P p ( R ) dr+ C P p ( R ) dr C M r F r Z Z Pp ( R ) > C Pp ( R ) < M r F r < J. A. O'S. ESE 524, Lecture 2, /5/9 6

17 Bayes Criterion: Derivation of Likelihood Ratio Test Compare CMP pr ( R ) and C P p F r ( R ) There are several equivalent ways to compare these values including the likelihood ratio and the log-likelihood ratio > C Λ P p ( R ) > C P p ( R ) < M r F r R ( ) r ( R ) F p > C P p ( R ) < C P η r M l R ( ) p ( ) r R CF P ln > ln p ( ) < r R MP C γ J. A. O'S. ESE 524, Lecture 2, /5/9 7

18 Bayes Criterion: Computation of Error r Probabilities biliti The likelihood ratio is a nonnegative-valued function of the observed data. Evaluated at a random realization of the data, the likelihood lih ratio is a random variable. Comparing the likelihood ratio to a threshold is the optimal test. The probabilities of error can be computed in terms of the probability density functions for either the likelihood lih ratio or the log-likelihood lik lih ratio. J. A. O'S. ESE 524, Lecture 2, /5/9 8

19 Optimal Decision i Rules Likelihood Ratio Test Threshold for Bayes Rule: CF P ( C C) P η = = CM P ( C C ) P Computations of the error probabilities p p ( ) ( r R > ) r R > Λ ( R) = η l ( R) = ln pr ( R pr ( R ) ) < < η P = p ( Λ ) d Λ M F p( ( ) η P = p Λ dλ γ P = p ( L ) dl M l P = p ( L ) dl F γ l ln η J. A. O'S. ESE 524, Lecture 4, /25/7 9

20 Minimum i Probability bilit of Errorr Minimum Probability of Error P = P P + P P CM = CF = P η = P e M F Loglikelihood Ratio Test J. A. O'S. ESE 524, Lecture 2, /5/9 2

21 Minimum i Probability bilit of Errorr Alternative View of Decision Rule: Compute Posterior Probabilities on ypotheses Select Most Likely ypothesis p ( ) r R > P p ( R ) < P r pr ( R ) P > pr ( R ) P g( R) < g( R) ( R P ) > r ( R ) p P r p P p( R) < p( R) p ( R ) = p ( R ) P + p ( R ) P r r < > P ( R) P ( R) for any g( R) > J. A. O'S. ESE 524, Lecture 4, /25/7 2

22 Gaussian Example J. A. O'S. ESE 524, Lecture 2, /5/9 22

23 Minimax i Decision i Rule Find the decision rule that minimizes the maximum Bayes Risk over all possible priors. Criterion Bayes: mimize expected risk or cost min maxr R Z P Transition Priors Costs Densities P & P C ij Yes Yes Yes Minimum Probability of Error Yes Yes No Minimax Yes No Yes Neyman-Pearson Yes No No J. A. O'S. ESE 524, Lecture 4, /25/7 23

24 Minimax i Decision i Rule: Analysis For any fixed decision rule the risk is linear in P The maximum over P is achieved at an end point To make that end point as low as possible, the risk should be constant with respect to P To minimize that constant value, the risk should achieve the minimum risk at some P *. At that value of the prior, the best decision rule is a likelihood ratio test R = C P P + C P P M M F F P = pr ( R ) dr M Z P = pr ( R ) dr F Z J. A. O'S. ESE 524, Lecture 4, /25/7 24

25 Minimax i Decision i Rule: Analysis Bayes risk is concave (it is always below its tangent) Minimax is achieved ed at an end point or at an interior point on the Bayes risk curve where the tangent is zero J. A. O'S. ESE 524, Lecture 4, /25/7 25

26 .9.8 Minimax Decision Rule: 5 X.7 : x 5 e, X.6 Example X Matlab Code pf=:.:; pd=pf.^.2; eta=.2*(pf.^(-.8)); % eta=dp_d/dp_f figure plot(pf,pd);xlabel('p_f');ylabel('p_d') pd);xlabel('p cm=;cf=; pstar=./(+cm*eta/cf); riskoptimal=cm*(-pd).*pstar+cf*pf.*(-pstar); figure plot(pstar,riskoptimal,'b'), hold on p=:.:; r=cm*(-pd())*p+cf*pf()*(-p); plot(p,r,'r'), hold on r2=cm*(-pd(2))*p+cf*pf(2)*(-p); p+cf pf(2) (-p); plot(p,r2,'g'), hold on r3=cm*(-pd(3))*p+cf*pf(3)*(-p); plot(p,r3,'c') xlabel('p_');ylabel('risk') : x e, X l = 4X ln5 P = 5e dx = e F γ ' γ ' 5X 5 γ ' X P = e dx = e M P = P = P D.2.2 M( ) = F F F M C P C P F γ ' P D Risk P F P J. A. O'S. ESE 524, Lecture 4, /25/7 26

27 Neyman-Pearson Decision i Rule M ( α) F = P + η P F Minimize P M subject to P F α Variational approach Upper bound usually achieved Likelihood ratio test; threshold? = p ( ) d + η p ( ) d α r R R r R R Z Z Criterion Bayes: mimize expected risk or cost Transition Densities Priors P & P Costs C ij Yes Yes Yes Minimum Probability of Error Yes Yes No Minimax Yes No Yes Neyman-Pearson Yes No No J. A. O'S. ESE 524, Lecture 4, /25/7 27

28 Neyman-Pearson Decision i Rule M ( α) F = P + η P F Minimize P M subject to P F α Variational approach Upper bound usually achieved Likelihood ratio test; threshold? = p ( ) d + η p ( ) d α r R R r R R Z Z Plot the ROC: P D versus P F for the family of likelihood ratio tests Draw a vertical line where P F = α Find the corresponding P D At that point, the threshold equals the derivative of the ROC η = = dp dp dp dp D F D F dη dη J. A. O'S. ESE 524, Lecture 4, /25/7 28

29 Summary Several decision rules Likelihood (and loglikelihood) ratio test is optimal Receiver operating characteristic (ROC) plots probability of detection versus probability of false alarm with the threshold as a parameter all possible optimal performance Neyman-Pearson is a point on the ROC (P F = α) Minimax is a point on the ROC (P F C F =P M C M ) Probability of error is a point on the ROC (slope η = (-P )/P ) J. A. O'S. ESE 524, Lecture 4, /25/7 29