Estimation Tasks. Short Course on Image Quality. Matthew A. Kupinski. Introduction

Transcription

1 Estimation Tasks Short Course on Image Quality Matthew A. Kupinski Introduction Section 13.3 in B&M Keep in mind the similarities between estimation and classification Image-quality is a statistical concept

2 What we Will Cover Bias, variance, MSE, EMSE Bayesian estimation Maximum-likelihood estimation Fisher information Linear estimators Basic Concept g = Hf + n Classification g Di Compare to Hi Estimation = (g) Compare to

3 Some PDFs Random quantities g = Hf + n = (g) pr() pr(f ) pr(g f ) pr(g ) = df pr(g f )pr(f ) pr( ) Example Estimate tumor volume g = Hf + n = (g) pr() Distribution of volumes pr(f ) Object distribution given a volume pr(g f ) Noise pr(g ) = df pr(g f )pr(f ) Data distribution given the true value of the parameter pr( ) Distribution of estimate given truth

4 Example g = Hf + n T (g) = Di Detect a tumor pr(hi ) Probability of truth states pr(f Hi ) Object distribution given truth pr(g f ) Noise pr(g Hi ) = df pr(g f )pr(f Hi ) Data distribution given the truth state pr(di Hj ) Sensitivity or specificity Bias Given a fixed value for the parameter we wish to estimate () = (g) = g # dg pr(g ) (g) Compare and (13.274)

5 Bias b() = (13.276) If b() = 0 for all then (g) is unbiased Variance Given a fixed value for the parameter we wish to estimate Var() = σ 2 2 = (g) (13.279) g Would like to have a small variance

6 Mean-Square Error Given a fixed value for the parameter we wish to estimate 2 MSE() = (13.280) g MSE() = b()2 + Var() Would like to have a small MSE Ensemble Forms # # = () = dpr() dg pr(g ) (g) (13.277) = Ensemble bias b = b() = (13.277) Ensemble variance Var = Var() Ensemble MSE (EMSE) $ EMSE = 2 # g (13.281)

7 Vector Forms Bias b() = Variance K () = MSE (13.283) # # $ (13.285) g MSE() = tr K () + tr b()b() (13.287) EMSE EMSE = tr K + tr b()b() (13.288) Bayesian Estimation Determination of though minimization of Bayes risk Knowledge of pr() is assumed Cost function must be specified C(, ) Choose that minimizes the average cost C( (g), ) g,

8 MMSE Let us choose C(, ) = EMSE = #2 $ (13.309) g, Estimator is given by the posterior mean MMSE = d pr( g) = g # pr(g )pr() = d pr(g) (13.311b) (13.312) MAP Estimation C(, ) = C( ) = 1 rect 2 (13.322) All wrong answers beyond are equally costly

9 MAP Estimation MAP = arg max pr(g )pr() (13.325) = arg max pr( g) (13.327) = arg max {ln [pr(g )] + ln [pr()]} (13.328) MAP estimation is a type of Bayesian estimation MAP and MMSE are equivalent under certain constraints (eqn ) MAP Estimation pr() pr(g ) pr( g)

10 Maximum Likelihood ML estimation does not require us to know or assert pr() ML estimation is purely data driven, i.e., focus is on g As we will see, ML estimator have many desirable properties Maximum Likelihood ML = arg max pr(g ) (13.348) = arg max ln [pr(g )] (13.349) Limit of MAP when the prior pr() is sufficiently broad that it can be ignored

11 Fisher Information The score is given by s= ln [pr(g )] sg = 0 (13.358) (13.359) The Fisher information matrix F () = ss g (13.360) The Fisher information matrix measures the curvature of the average log-likelihood Cramér-Rao Bound Var n # 1 $ F nn (13.371) Fisher information provides us with a lower bound on all unbiased estimators of n An unbiased estimator that meets this lower bound is known as an efficient estimator

12 Properties of ML Estimators Invariance Θ = τ () Θ ML = τ ( ML ) Efficiency If an efficient estimator exists ML is it Asymptotic properties ML estimators are asymptotically unbiased, efficient, and normal Bayesian and ML Limitations We need pr(g ) pr() or both Consider the case of varying backgrounds pr(g ) = df pr(g f )pr(f ) How can we possibly hope to know this?

13 Before we Continue... Random variable x σx2 = (x x)2 x Random vector x K x = (x x)(x x) x [K x ]i,j = (xi xi )(xj xj )#xi,xj Two random vectors x y Kx,y = (x x)(y y) x,y [Kx,y ]i,j = (xi xi )(yj y j ) x i,yj Linear Estimators Consider a linear estimator of the form = W g + c What is the matrix W that minimizes the EMSE? EMSE = #$ # g, = W g # W g #$ The Wiener estimator is given by WE = + W g g W = K,g K 1 g (13.391) g, (13.385)

14 Wiener Estimator W = K,g K 1 g (13.391) K g is the covariance of the data averaged over all sources of randomness K g, is the cross-covariance of and g = gg Wiener Estimator The WE is the linear observer that minimizes the EMSE In the case of jointly Gaussian data, the WE minimizes the EMSE of all estimators

15 Similarities with Detection Bayesian estimation Minimize Bayes risk Requires complete knowledge of risks, priors, and data densities Similarities with Detection MMSE = pr(g )pr() d pr(g) # D2 > (C12 C22 )P r(h2 ) Λ(g) < (C21 C11 )P r(h1 ) D1 (13.312) (13.58)

16 Similarities with Detection MAP Estimation Applies equal cost to all wrong decisions and 0 cost to correct decisions Requires complete knowledge of prior and data densities Similarities with Detection MAP = arg max pr(g )pr() (13.325) D2 > P r(h2 ) Λ(g) < P r(h1 ) D1 (13.61)

17 Similarities with Detection ML Estimation Assumes a flat (or wide) prior distribution Requires complete knowledge data densities Similarities with Detection ML = arg max pr(g ) (13.348) D2 > 1 Λ(g) < D1 (13.71)

18 Similarities with Detection Ideal linear observers Limited to linear manipulations of the data Requires only first- and second-order statistics Similarities with Detection WE = + W g g W = K,g K 1 g D2 > t g K 1 g < c D1 (13.391)

19 To Reiterate Estimation and classification are very similar Estimation is a statistical concept and one should consider all sources of variation when dealing with estimation tasks Final Thoughts Possible estimation task figures of merit EMSE Bayes risk Fisher information matrix Computational methods will be discussed in a later talk