PIMS-MITACS 2001 1 Statistical Estimation of the Parameters of a PDE Colin Fox, Geoff Nicholls (University of Auckland) Nomenclature for image recovery Statistical model for inverse problems Traditional approaches deconvolution example Recovering electrical conductivity via inference
PIMS-MITACS 2001 2 Image Recovery nomenclature for Inverse Problems x-rays in x-rays out X-ray tomography Image: spatially varying quantity of interest optical reflectance of a scene optical or radio brightness of sky sound speed in tissue / ocean / earth electrical conductivity of tissue / mud Recovery: estimate image from indirect data Forward Problem image data physical model (PDE) direct computation well posed unique Inverse Problem data image implicit indirect ill posed never unique
PIMS-MITACS 2001 3 Conductivity Imaging Measurement Set Electrodes at x 1,x 2,,x E Assert currents at electrodes j =(j (x 1 ),j(x 2 ),,j(x E )) T Measure voltages v =(φ (x 1 ), φ (x 2 ),, φ (x E )) T. Unknown σ (x) related to measurements via Neumann BVP σ (x) φ (x) =0 x Ω σ (x) φ(x) n (x) = j (x) x Ω Set of measurements is current-voltage pairs {j n,v n } N n=1 Inverse problem is to find σ from these measurements (non linear)
PIMS-MITACS 2001 4 Statistical model of Inverse Problem true image σ forward map K ideal data data loss P noise free data n + + noise actual data v=pk σ+n σ^ estimate -1 K image recovery algorithm prior knowledge - constraints - regularization - preferences If n N(0,s 2 ), v N(PKσ,s 2 ) Given measurements v, the likelihood for σ is L v (σ) Pr (v σ) exp( v φ(σ) 2 /2s 2 ) Posterior distribution for σ conditional on v Pr (v σ)pr(σ) Pr (σ v) = Pr (v) Pr (σ)is the prior distribution (Bayes rule)
PIMS-MITACS 2001 5 Solutions = Summary Statistics All information contained in posterior distribution Pr (σ v) Traditional Solutions - modes ˆσ MLE = arg max L v (σ) arg max Pr (v σ) ˆσ MAP = arg max Pr (σ v) arg max Pr (v σ)pr(σ) e.g. simple Gaussian prior: Pr (σ) exp ³ 2 σ 2 /2λ ˆσ MAP = arg min v φ(σ) 2 + α σ 2 α = s 2 /λ 2 Tikhonov regularization, Kalman filtering, Backus-Gilbert α 0 Moore-Penrose inverse, α =0least-squares Inferential Solutions Answers are expectations over the posterior Z E [f (σ)] = Pr (σ v) f (σ) dσ Pr( σv ) σ
PIMS-MITACS 2001 6 Traditional Solutions Fourier Deconvolution Noisey blurred image Exact inverse MAP solution
PIMS-MITACS 2001 7 Bayesian Formulation for Conductivity Imaging current potential voltage current conductivity in Ω in Ω electrode electrode r.v. R Φ V J Σ value ρ φ v j σ Posterior Pr {Σ = σ, Φ n = φ n,r n = ρ n {J n,v n } = {j n,v n }} =Pr{{J n,v n } = {j n,v n } Σ = σ, Φ n = φ n,r n = ρ n } Pr {Σ = σ, Φ n = φ n,r n = ρ n } R = Σ Φ, φ = Γ σ (ρ Ω ) and ρ = σ φ Pr {Σ = σ, Φ n = φ n,r n = ρ n } =Pr{Σ = σ,r n = ρ n } Stipulate Pr {Σ = σ} only usually a MRF L (σ, φ n, ρ n ) =Pr{{J n,v n } = {j n,v n } Σ = σ, Φ n = φ n,r n = ρ n } =Pr{{V n } = {v n } Φ n = φ n (σ, ρ n )} Pr {{J n } = {j n } R n = ρ n } Errors i.i.d. L (σ, φ n, ρ n )=Π N n=1pr {V n = v n Φ n = Γ σ (ρ n Ω )} Pr {J n = j n R n = ρ n }. Noise is normal (say) Pr {J = j R = ρ} N ³ (ρ (x 1 ), ρ (x 2 ),, ρ (x k )) T, s 2 ρ
PIMS-MITACS 2001 8 Samples from the Prior
PIMS-MITACS 2001 9 Markov chain Monte Carlo Monte Carlo integration If {X t,t= 1, 2,...,n} are sampled from Pr (σ v) E [f (σ)] 1 nx f (X t ) n Markov chain Generate {X t } t=0 as a Markov chain of random variables X t Σ Ω, with a t-step distribution Pr(X t = σ X 0 = σ (0) ) that tends to Pr(σ v),ast. t=1 Metopolis-Hastings algorithm (1) given state σ t at time t generate candidate state σ 0 from a proposal distribution q (. σ t ) (2) Accept candidate with probability µ Pr(Y v)q (X Y ) α (X Y )=min 1, Pr(X v)q (Y X) (3) If accepted, X t+1 = σ 0 otherwise X t+1 = σ t (4) Repeat q (. σ t ) can be any distribution that ensures the chain is: irreducible aperiodic
PIMS-MITACS 2001 10 Three-Move Metropolis Hastings Choose one of 3 moves with probability ζ p,p= 1, 2, 3 Transition probabilities {Pr (p) (X t+1 = σ t+1 X t = σ t )} 3 p=1 (reversible w.r.t. Pr(σ v)). Overall transition probability is Pr(X t+1 = σ t+1 X t = σ t ) 3X = ζ p Pr (p) (X t+1 = σ t+1 X t = σ t ). p=1 If at least one of the moves is irreducible on Σ Ω, then the equilibrium distribution is Pr(σ v). Apixeln is a near-neighbour of pixel m if their lattice distance is less than 8. An update-edge is a pair of near-neighbouring pixels of unequal conductivity. ( N (σ), Nm(σ) ) Move 1Flipapixel. Select a pixel m at random and assign σ m a new conductivity σ 0 m chosen uniformly at random from the other C 1 conductivity values. Move 2Flip a pixel near a conductivity boundary. Pick an update-edge at random from N (σ). Pick one of the two pixels in that edge at random, pixel m say.proceedasinmove1. Move 3Swap conductivities at a pair of pixels. Pick an updateedge at random from N (σ). Setσ 0 m = σ n and σ 0 n = σ m.
PIMS-MITACS 2001 11 Experiment 1 (discrete variables three conductivity levels) A B1 B2 C D
PIMS-MITACS 2001 12 Experiment 2 (continuous variables three conductivity types) A B C D E
PIMS-MITACS 2001 13 Experiment 3 (shielding) A E B F C G D H
PIMS-MITACS 2001 14 Summary If you can simulate the forward map then you can sample and calculate expectations over the posterior, i.e., solve the inverse problem Statistical inference provides a unifying framework for inverse problems