Phase Retrieval with Random Illumination Albert Fannjiang, UC Davis Collaborator: Wenjing Liao NCTU, July 212 1
Phase retrieval Problem Reconstruct the object f from the Fourier magnitude Φf. Why do we care? X-ray crystallography, single-molecule imaging, astronomy etc. 1985 Nobel Prize in Chemistry for Hauptman and Karle: partial solution of phasing problem. 2
Phasing problem formulation Discrete finite objects Let n =(n 1,n 2 ) Z 2 and z =(z 1,z 2 ) C 2. multi-index : z n = z n 1 1 zn 2 2 Let the object be represented by f(n), n N =(N, N). Binary objects: white = 1, black =. 3
f L = Lena F L (w) = F L (w) e iθ L(w) f B = Barbara F B (w) = F B (w) e iθ B(w) F 1 (w) = F B (w) e iθ L(w) F 2 (w) = F L (w) e iθ B(w) f 1 = Φ F 1 f 2 = Φ F 2 4
F 1 (w) = F B (w) e iθ L(w) F 2 (w) = F L (w) e iθ B(w) f 1 = Φ F 1 f 2 = Φ F 2 5
Phase retrieval Discrete finite objects Let n =(n 1,n 2 ) Z 2 and z =(z 1,z 2 ) C 2. multi-index : z n = z n 1 1 zn 2 2 Let the object be represented by f(n), n N =(N, N). Fourier transform describes wave propagation F (e i2πw 1,e i2πw 2)= n f(n)e i2πn w Analytic continuation = z-transform F (z) = n f(n)z n. Discrete phase retrieval problem: Determine f(n) from Fourier magnitude data F (w), w =(e i2πw 1,e i2πw 2) [, 1] 2 6
Fourier magnitude data: F (w) 2 = N n= N m f(m + n)f (m)e i2πn w = N n= N C f (n)e i2πn w where C f (n) = m f(m + n)f (m) is the autocorrelation function of f. Fourier magnitude data contain complete information about autocorrelation function. Sampling Theorem: supp(c f ) [ N, N] 2 = [, 1] 2 is reduced to the Nyquist grid M = (k 1,k 2 ):k j =, 1 2N +1, 2 2N +1,, 2N 2N +1 7
Oversampling ratio σ = Fourier magnitude data number unknown image pixel number Standard ratio: Compressed sensing: 8
Trivial ambiguities Autocorrelation: Invariant under: (i) global phase, C f (n) = m+n N f(m + n)f (m) f(n) e iθ f(n), for some θ [, 2π], (ii) spatial translation f(n) f(n m), n m = n + m mod(n), some m Z 2 (iii) conjugate inversion (twin image) f(n) f (N n). 9
Nontrivial ambiguity THEOREM (Hayes 82, Pitts-Greenleaf 3) Let the z-transform F (z) of a finite complex-valued sequence {f(n)} be given by F (z) =αz m p k=1 F k (z), m N 2, α C where F k,k =1,...,p are nontrivial irreducible polynomials. Let G(z) be the z-transform of another finite sequence g(n). Suppose F (w) = G(w), w [, 1] 2. Then G(z) must have the form G(z) = α e iθ z p F k (z) Fk (1/z ), p N 2, θ R k I k I c where I is a subset of {1, 2,...,p}. Nontrivial ambiguity: Partial conjugate inversion on factors. 1
Random illumination Random Illumination P.F. Almoro et al. / Optics and Lasers in En cs and Lasers in Engineering 49 (211) 252 257 Coded aperture imaging expanding the laser beam and impinging on a plate of translucent perpex, which acts as an opal diffuser, with unnoticeable grain and nearly Lambertian scattering of the light. In the fluorescence experiments the sample is coated with a thin layer (~5 µm) of solution of fluorescein diacetate (FDA) that reemits incoherent light in the green wavelengths of the optical spectrum. The process requires a high resolution image of the speckle that acts as the encodingdecoding mask. We take these reference images prior to each experiment by focusing at a transparent region in the sample plane using a lens with high NA (.4). Figure 2 displays the Diffuser generated pattern: Garcia-Zalevsky-Fixler 5 reference and its speckle autocorrelation. The size of the autocorrelation peak is the expected d: (a) plane wave orimage uniform illumination; (b) diffused illumination and (c mental setups for the after multiple-plane phase retrieval method: (a) plane wave resolution the superresolution process when a low NA lens is used. (a) (b) Fig. 2. (a) Encoding speckle pattern. (b) Autocorrelation of the encoding pattern. Once the reference speckle pattern is captured, 11 the sample is set in place and the lens is replaced by a lens with a low NA in the horizontal direction. Then the sample position is laterally scanned and the image set is captured. Note that instead of displacing the projected
X-ray holograhy with URA (Marchesini et al. 8). Coherent X-ray imaging will be a key technique for developing nanoscience and nanotechnology. One shot imaging: Bright and ultrashort X-ray pulse vaporizes the sample right after the pulse passes the sample. Random illumination amounts to replacing the original object f(n) by f(n) = f(n)λ(n) (illuminated object) 12
Random illumination f(n) =f(n)λ(n) (illuminated object) λ(n), representing the illumination field, is a known sequence of samples of random variables. Let λ(n) be continuous random variables with respect to the Lebesque measure on S 1 (the unit circle), R or C. Case of S 1 can be facilitated by a random phase modulator with λ(n) =e iφ(n) where φ(n) are continuous random variables on [, 2π]. Case of R: random amplitude modulator. Case of C: both phase and amplitude modulations. 13
Irreducibility THEOREM. Suppose that the support of the object {f(n)} has rank 2. Then the the z-transform of the illuminated object f(n)λ(n) is irreducible with probability one. False for rank 1 objects: fundamental thm of algebra σ = Fourier magnitude data number unknown image pixel number = 14
Absolute uniqueness Positivity THEOREM If f(n) is real and nonnegative for every n then, with probability one, f is determined absolutely uniquely by the Fourier magnitude measurement on the lattice L. Sector constraint THEOREM Suppose the phases of the object belong to [a, b] [, 2π]. Then the solution to the Fourier phasing problem has auniquesolutionwithprobabilityexponentiallyclosetounity (depending on the sparsity and the phase range b a.) 15
Complex objects w/o constraint THEOREM. Suppose that {λ 1 (n)} are i.i.d. continuous random variables with respect to the Lebesgue measure on S 1, R or C and in addition either one of the following conditions is true. (i) {λ 2 (n)} are i.i.d. continuous random variables with respect to the Lebesgue measure on S 1, R or C and {λ 2 (n)} are independent of {λ 1 (n)}. (ii) {λ 2 (n)} are deterministic. Then with probability one f(n) is uniquely determined, up to a constant phase factor, by the Fourier magnitude measurements with two illuminations λ 1 and λ 2. 16
Alternated projections Gerchberg-Saxton; Error Reduction (Fienup) f k+1 /f k Λ g k Φ G k P o T f k Λ 1 g k Φ 1 G k Oversampling method (Miao et al.) σ = image pixel number + zero-padding pixel number image pixel number. 17
Zero-padding cameraman 18
Multiple illuminations P 1 = Λ 1 1 Φ 1 T 1 ΦΛ 1 P 2 = Λ 1 2 Φ 1 T 2 ΦΛ 2. f k+1 = P o P 2 P 1 f k. 19
Error Reduction (Gerchberg-Saxton) Bregman 65: convex constraints = convergence to afeasible solution. Fourier magnitude data are a non-convex constraint! Nonconvexity or nonuniqueness? 2
Convergence Theorem 4. Let f C(N ) be an array with f() = and rank 2. Let λ(n) be i.i.d. continuous random variables on S 1. Let the Fourier magnitude be sampled on L. Let h be a fixed point of P o Pf θ such that Pf θ h satisfies the zero-padding condition. (a) If f is real-valued, h = ±f with probability one, (b) If f satisfies the sector condition of Theorem 2, then h = e iν f, for some ν, and satisfies the same sector constraint with probability at least 1 N (β α) S/2 (2π) S/2. 21
Hybrid-Input-Output (HIO) f k+1 /f k Λ g k Φ G k P o f k Λ 1 g k Φ 1 T G k (f k+1 (n)) = (f k(n)) (f k+1 (n)) = (f k (n)) β (f k(n)), Real-valued objects 22
Error metrics Relative error e( ˆf) = f ˆf/f min f ν [,2π) eiν ˆf/f Relative residual r( ˆf) = Y ΦΛP o{ ˆf} Y 23
(a) (b) 269 x 269 2 x 2 24
at each.5.45.45.4.4.35.35 at each.5.3.25.2.15.3.25.2.15.1.1.5.5 5 1 15 (a) 2 25 3 35 4 45 5 (b) 1 2 3 (c).45.45.4.4.35.35.3.25.2.15 8 9 1.3.25.2.15.1.1.5.5 2 4 6 8 1 12 14 16 18 2 (f) 5 1 15 (g) 2 25 3 35 (h) at each at each.5.5.45.45.4.4.35.35 7 at each.5 at each.3.25.2.15.3.25.2.15.1.1.5.5 5 1 15 2 25 3 35 4 45 5 55 6 (i) 5 6 (d).5 (e) 4 (j) 1 2 3 4 5 6 (k) (l) (e)-(h) Low resolution 4 x 4 block illumination with OR=2 Figure 1. (a)recovered cameraman by 5 ER s with single uniform illumina(i)-(l) High resolution illumination with OR=1 tion when ρ = 4. f 5 f4999 / f4999.2%, ε f (f5 ) 5.47%. (b)normalized error ε (fk ) at each. (c)recovered cameraman by 1 HIO +5 ER with one 25 uniform illumination when ρ = 4. f15 f149 / f149.3%, ε f (f15 ).93%. (d) ε (fk ) at each. (e)recovered cameraman by 2 ER iter-
.5 at each.5 at each.45.45.4.4.35.35.3.25.2.3.25.2.15.15.1.1.5.5 5 1 15 2 25 3 35 4 45 5 1 2 3 4 5 6 7 8 9 1 (a) (b) (c) (d).5 at each 1 at each.45.9.4.8.35.7.3.25.2.6.5.4.15.3.1.2.5.1 5 1 15 2 25 3 35 4 45 5 5 1 15 2 25 3 35 (e) (f) (g) (h).5 at each.5 at each.45.45.4.4.35.35.3.25.2.3.25.2.15.15.1.1.5.5 1 2 3 4 5 6 7 8 9 1 11 12 5 1 15 2 25 3 35 (i) (j) (k) (l) (e) - (h) Low resolution 4 x 4 block illumination with OR=2 Figure 2. (a)recovered phantom by 5 ER s with single uniform illumination when (i) ρ - = (l) 4. High f 5 resolution f 4999 / f 4999 illumination.6%, ε f (f 5 with ) OR=1 14.7%. (b)normalized error ε(f k ) at each. (c)recovered phantom by 1 HIO +5 ER with single uniform illumination when ρ = 4. f 15 26 f 149 / f 149.7%, ε f (f 15 ) 3.95%. (d) ε(f k ) at each. (e)recovered phantom by 5 ER s with single low resolution(block size: 4 4) random phase illumination when ρ = 2.
.5 relative residual at each.5 relative residual at each.45.45.4.4.35.35 relative residual.3.25.2 relative residual.3.25.2.15.15.1.1.5.5 5 1 15 2 25 5 1 15 2 25 (a) (b) (c) (d).5 relative residual at each.5 relative residual at each.45.45.4.4.35.35 relative residual.3.25.2 relative residual.3.25.2.15.15.1.1.5.5 5 1 15 2 25 5 1 15 2 25 (e) (f) (g) (h).5 relative residual at each.5 relative residual at each.45.45.4.4.35.35 relative residual.3.25.2 relative residual.3.25.2.15.15.1.1.5.5 2 4 6 8 1 12 14 16 18 2 22 24 5 1 15 2 25 (i) (j) (k) (l) High resolution illumination with 5% Gaussian, Poisson and illuminator errors 27
.5 at each 1 at each.45.9.4.8.35.7.3.25.2.6.5.4.15.3.1.2.5.1 5 1 15 2 25 3 5 1 15 2 25 3 35 4 45 (a) (b) (c) (d).5 at each 1 at each.45.9.4.8.35.7.3.25.2.6.5.4.15.3.1.2.5.1 5 1 15 2 25 5 1 15 2 25 3 35 4 45 5 (e) (f) (g) (h).5 at each 1 at each.45.9.4.8.35.7.3.25.2.6.5.4.15.3.1.2.5.1 5 1 15 2 25 5 1 15 2 25 3 35 4 45 5 (i) (j) (k) (l) Low resolution illumination with 5% Gaussian, Poisson and illuminator errors 28
Compressed measurement average relative error in 5 trials.5.45.4.35.3.25.2.15 relative error versus oversampling rate high resolution & noise free low resolution & noise free high resolution & 5% gaussian nosie low resolution & 5% gaussian nosie high resolution & 5% poisson nosie low resolution & 5% poisson noise high resolution & 5% illuminator noise low resolution & 5% illuminator nosie average relative error in 5 trials 1.2 1.8.6.4 relative error versus oversampling rate high resolution & noise free low resolution & noise free high resolution & 5% gaussian nosie low resolution & 5% gaussian nosie high resolution & 5% poisson nosie low resolution & 5% poisson noise high resolution & 5% illuminator noise low resolution & 5% illuminator nosie average relative error in 5 trials 1.5 1.5 relative error versus oversampling rate high resolution & noise free low resolution & noise free high resolution & 5% gaussian nosie low resolution & 5% gaussian nosie high resolution & 5% poisson nosie low resolution & 5% poisson noise high resolution & 5% illuminator noise low resolution & 5% illuminator nosie.1.2.5 1 1.5 2 2.5 3 3.5 4 oversampling rate 1 2 3 4 5 6 7 8 oversampling rate 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 oversampling rate (a) (b) (c) Figure 3 Figure 4. (a)average (a) relative real-valued error by adaptive HIO + adaptive ER in 5 trials versus oversampling rate for phantom with one random phase illumination. (b)average relative error by adaptive HIO (b) + adaptive positive ER in real 5 trials & versus imaginary oversampling parts rate for phantom with random phases between (c) no andconstraint π/2 with one random phase illumination. (c)average relative error by 2 HIO +3 ER in 5 trials versus oversampling rate for phantom with random phases between and 2π with one uniform illumination and one random phase illumination. 29
Noise stability average relative error in 5 trials 1.9.8.7.6.5.4.3 relative error versus noise level high resolution & gaussian noise low resolution & gaussian noise high resolution & poisson noise low resolution & poisson noise high resolution & illuminator noise low resolution & illuminator noise average relative error in 5 trials 1.9.8.7.6.5.4.3 relative error versus noise level high resolution & gaussian noise low resolution & gaussian noise high resolution & poisson noise low resolution & poisson noise high resolution & illuminator noise low resolution & illuminator noise average relative error in 5 trials 1.9.8.7.6.5.4.3 relative error versus noise level high resolution & gaussian noise low resolution & gaussian noise high resolution & poisson noise low resolution & poisson noise high resolution & illuminator noise low resolution & illuminator noise.2.2.2.1.1.1 5 1 15 2 25 3 noise percentage 5 1 15 2 25 3 noise percentage 5 1 15 2 25 3 noise percentage (a) (b) (c) Figure 6 Figure 7. (a)average relative error by adaptive HIO + adaptive ER in 5 trials versus noise level for phantom with (a) real-valued one randomobjects phase illumination when ρ = 4. (b)average relative error by adaptive HIO + adaptive ER in 5 trials versus noise level for phantom with random phases between(b) and positive π/2 withreal one random & imaginary phase illumination partswhen ρ = 4. (c)average relative error by (c) 2 no HIO constraint +3 ER in 5 trials versus noise level for phantom with random phases between and 2π with one uniform illumination and one random phase illumination when ρ = 3. 3
Conclusions Random illumination as enabling tool for phase retrieval. Absolute uniqueness Fast convergence OR =1 (real) or 2 (complex) Proof of convergence: HIO? References: A. Fannjiang Absolute uniqueness in phase retrieval with random illumination Inverse Problems 212 ( arxiv:111.597) A. Fannjiang and W. Liao Phase retrieval with random phase illumination arxiv:126.11 31