Orientation Assignment in Cryo-EM

Transcription

1 Orientation Assignment in Cryo-EM Amit Singer Princeton University Department of Mathematics and Program in Applied and Computational Mathematics July 24, 2014 Amit Singer (Princeton University) July / 30

2 Single Particle Reconstruction using cryo-em Schematic drawing of the imaging process: The cryo-em problem: Amit Singer (Princeton University) July / 30

3 Orientation Estimation: Fourier projection-slice theorem (x ij,y ij) R ic ij c ij = (x ij,y ij,0) T Projection I i Projection I i Î i 3D Fourier space Molecule φ R i SO(3) (x ji,y ji) R ic ij = R jc ji Electron source Projection I j Î j 3D Fourier space Cryo-EM inverse problem: Find φ (and R 1,...,R n ) given I 1,...,I n. n = 3: Vainshtein and Goncharov 1986, van Heel 1987 n > 3: S, Shkolnisky SIAM Imaging 2011 n > 3: Bandeira, Charikar, Chen, S, in preparation Amit Singer (Princeton University) July / 30

4 Assumptions for today s talk Trivial point-group symmetry Homogeneity Amit Singer (Princeton University) July / 30

5 Experiments with simulated noisy projections Each projection is 129x129 pixels. SNR = Var(Signal) Var(Noise), (a) Clean (b) SNR=2 0 (c) SNR=2 1 (d) SNR=2 2 (e) SNR=2 3 (f) SNR=2 4 (g) SNR=2 5 (h) SNR=2 6 (i) SNR=2 7 (j) SNR=2 8 Amit Singer (Princeton University) July / 30

6 Fraction of correctly identified common lines and the SNR Define common line as being correctly identified if both radial lines deviate by no more than 10 from true directions. Fraction p of correctly identified common lines increases by PCA log 2 (SNR) p Amit Singer (Princeton University) July / 30

7 Least Squares Approach (x ij,y ij) R ic ij c ij = (x ij,y ij,0) T Projection I i Î i 3D Fourier space (x ji,y ji) R ic ij = R jc ji Projection I j Î j 3D Fourier space Least squares: min R 1,R 2,...,R n SO(3) R i c ij R j c ji 2 i j Search space is exponentially large and non-convex. Amit Singer (Princeton University) July / 30

8 Quadratic Optimization Under Orthogonality Constraints Quadratic cost: i j R ic ij R j c ji 2 Quadratic constraints: R T i R i = I 3 3 (det(r i ) = +1 constraint is ignored) We approximate the solution using SDP and rounding (S, Shkolnisky 2011) Related to: MAX-CUT (Goemans and Williamson 1995) Generalized Orthogonal Procrustes Problem (Nemirovski 2007) PhaseLift (Candes, Strohmer, Voroninski 2013) Non-commutative Grothendick Problem (Naor, Regev, Vidick 2013) Robust version Least Unsquared Deviations (Wang, S, Wen 2013) min R 1,R 2,...,R n SO(3) R i c ij R j c ji i j Amit Singer (Princeton University) July / 30

9 Detour: MAX-CUT MAX-CUT: Given a weighted undirected graph with n vertices and non-negative weights w ij 0, split the vertices into two sets such that the total weight of edges across the cut is maximized. 1 w 14 w 12 2 w 24 w 25 4 w 45 3 w 34 w 35 5 CUT({1,2,3}, {4,5}) = w 14 +w 24 +w 25 +w 34 +w 35 Amit Singer (Princeton University) July / 30

10 The MAX-CUT Approximation of Goemans & Williamson OPT = 1 max y 1,y 2,...,y n { 1,+1} 4 w ij (1 y i y j ) Combinatorial problem that is known to be NP-complete SDP relaxation of Goemans-Williamson (1995): The n n Gram matrix G = yy T (G ij = y i y j ) is PSD (G 0), G ii = 1, rank(g) = 1. 1 max G R n n 4 i j w ij 1 4 Tr(GW) i,j s.t. G 0, G ii = 1, i = 1,2,...,n. Randomize a random vector z with i.i.d standard Gaussian entries z i N(0,1), and compute Yz, where G = YY T is the Cholesky decomposition of G. Round y i = sign[(yz) i ]. Theorem (G-W): E[CUT GW ] > 0.87 OPT. Amit Singer (Princeton University) July / 30

11 SDP Relaxation for the Common-Lines Problem Least squares is equivalent to maximizing the sum of inner products: R i c ij R j c ji 2 R i c ij,r j c ji min R 1,R 2,...,R n SO(3) i j max R 1,R 2,...,R n SO(3) y ji Tr(c ji cij T RT i R j ) i j max R 1,R 2,...,R n SO(3) i j max Tr(CG) R 1,R 2,...,R n SO(3) C is the 2n 2n matrix ( the common lines matrix ) with [ ] [ ] [ C ij = c ji c ij T xji [ ] xji x = xij y ij = ij x ji y ij 0 0, C y ji x ij y ji y ii = ij 0 0 G is the 2n 2n Gram matrix G = R T R with Gij = R i T R j : R T 1 I 2 2 R T R R T 1 2 R T R 1 n [ ] R G = R 1 R 2 R n = 2. T R 2 1 I 2 2 RT 2 Rn R n T R 1 R n T R 2 I 2 2 R T n ] Amit Singer (Princeton University) July / 30

12 SDP Relaxation and Rounding SDP Relaxation: max Tr(CG) R 1,R 2,...,R n SO(3) max G R 2n 2nTr(CG) s.t. G 0, G ii = I 2 2, i = 1,2,...,n. Missing is the non-convex constraint rank(g) = 3. Randomize a 2n 3 orthogonal matrix Q using (careful) QR factorization of a 2n 3 matrix with i.i.d standard Gaussian entries Compute Cholesky decomposition G = YY T Round using SVD: (YQ) i = U i Σ i Vi T = R i T = U i V T Use the cross product to find Ri T. Loss of handedness. Amit Singer (Princeton University) July / 30 i.

13 Spectral Relaxation for Uniformly Distributed Rotations [ R 1 i R 2 i ] = Define 3 vectors of length 2n [ x 1 i xi 2 yi 1 yi 2 zi 1 zi 2 ], i = 1,...,n. x = [ x 1 1 x 2 1 x 1 2 x 2 2 x 1 n x 2 n y = [ y 1 1 y 2 1 y 1 2 y 2 2 y 1 n y 2 n z = [ z 1 1 z 2 1 z 1 2 z 2 2 z 1 n z 2 n Rewrite the least squares objective function as max R 1,...,R n SO(3) i j R i c ij,r j c ji = max R 1,...,R n SO(3) xt Cx +y T Cy +z T Cz By symmetry, if rotations are uniformly distributed over SO(3), then the top eigenvalue of C has multiplicity 3 and corresponding eigenvectors are x,y,z from which we recover R 1,R 2,...,R n! Amit Singer (Princeton University) July / 30 ] T ] T ] T

14 Spectrum of C Numerical simulation with n = 1000 rotations sampled from the Haar measure; no noise. Bar plot of positive (left) and negative (right) eigenvalues of C: λ λ Eigenvalues: λ l n ( 1)l+1 l(l+1), l = 1,2,3,... (1 2, 1 6, 1 12,...) Multiplicities: 2l + 1. Two basic questions: 1 Why this spectrum? Answer: Representation Theory of SO(3) (Hadani, S, 2011) 2 Is it stable to noise? Answer: Yes, due to random matrix theory. Amit Singer (Princeton University) July / 30

15 Common Lines Kernel Common line equation: R i c ij = R j c ji = ± R3 i R 3 j R 3 i R 3 j c ij = ±R 1 i Ri 3 Rj 3 Ri 3 Rj 3, c ji = ±R 1 j Basic properties of the cross product: C ij is only a function of the ratio U ij = R 1 i R j ( ) ( ) xij x C ij = ji x ij y ji xij ( = xji y y ij x ji y ij y ji ji = P [I3 U 3 ij ][(U 1 ij ) 3 I 3 ] T I 3 U 3 ij 2 P T, where P = y ij Ri 3 Rj 3 Ri 3 Rj 3 ) = (Pcij )(Pc ji ) T ( Common lines kernel: C ij = k(r i,r j ) = k(r 1 i R j ) : R 2 R 2. ). Amit Singer (Princeton University) July / 30

16 Common Lines Operator Common lines kernel: C ij = k(r i,r j ) = k(r 1 i R j ) : R 2 R 2. Suppose f : SO(3) R 2 then 1 n (Cf)(R i) = 1 n n C ij f(r j ) j=1 SO(3) k(r i,r j )f(r j )dµ(r j ) = (Cf)(R i ). Common lines integral operator C is a convolution over SO(3): (Cf)(R i ) = k(r 1 i R j )f(r j )dµ(r j ) = λf(r i ) SO(3) Representation theory of SO(3) explains eigenvalues and eigenfunctions. Eigenfunctions are tangent vector fields to the 2-sphere (sections of the tangent bundle). They are the projected gradient of the scalar spherical harmonics. Amit Singer (Princeton University) July / 30

17 Probabilistic Model and Wigner s Semi-Circle Law Simplistic Model: every common line is detected correctly with probability p, independently of all other common-lines, and with probability 1 p the common lines are falsely detected and are uniformly distributed over the unit circle. Let C clean be the matrix C when all common-lines are detected correctly (p = 1). The expected value of the noisy matrix C is E[C] = pc clean, as the contribution of the falsely detected common lines to the expected value vanishes. Decompose C as C = pc clean +W, where W is a 2n 2n zero-mean random matrix. The eigenvalues of W are distributed according to Wigner s semi-circle law whose support, up to O(p) and finite sample fluctuations, is [ 2n, 2n]. Amit Singer (Princeton University) July / 30

18 Threshold probability Sufficient condition for top three eigenvalues to be pushed away from the semi-circle and no other eigenvalue crossings: (rank-1 and finite rank deformed Wigner matrices, Füredi and Komlós 1981, Féral and Péché 2007,...) p (C clean ) > 1 2 λ 1(W) Spectral gap (C clean ) and spectral norm λ 1 (W) are given by (C clean ) ( )n and Threshold probability λ 1 (W) 2n. p c = n. Amit Singer (Princeton University) July / 30

19 Numerical Spectra of C, n = λ (a) SNR= λ (b) SNR= λ (c) SNR= λ (d) SNR= λ (e) SNR= λ (f) SNR= λ (g) SNR= λ (h) SNR= λ (i) SNR=2 8 Amit Singer (Princeton University) July / 30

20 Estimation Error True rotations: R 1,...,R n. Estimated rotations: ˆR1,..., ˆR n. Registration: Ô = argmin O SO(3) n R i OˆR i 2 F i=1 Mean squared error: MSE = 1 n n R i ÔˆR i 2 F i=1 Amit Singer (Princeton University) July / 30

21 MSE for n = 1000 SNR p MSE Model fails at low SNR. Why? Wigner model is too simplistic cannot have n 2 independent random variables from just n images. C ij = K(I i,i j ), random kernel matrix (Koltchinskii and Giné 2000, El-Karoui 2010). Kernel is discontinuous (Cheng, S, 2013) Amit Singer (Princeton University) July / 30

22 From Common Lines to Maximum Likelihood The images contain more information than that expressed by the common lines. Common line based algorithms can succeed only at high SNR. (Quasi) Maximum Likelihood: We would like to try all possible rotations R 1,...,R n and choose the combination for which the agreement on the common lines as observed in the images is maximal. Computationally intractable: exponentially large search space. Amit Singer (Princeton University) July / 30

23 Maximum Likelihood Solution using SDP Main idea: Lift SO(3) to Sym(S 2 ) Suppose ω 1,ω 2,...,ω L S 2 are evenly distributed points over the sphere (e.g., a spherical t-design). g SO(3) π S L ( Assignment, Procrustes ) (in practice no explicit construction is needed). Amit Singer (Princeton University) July / 30

24 Convex Relaxation of Permutations arising from Rotations The convex hull of permutation matrices are the doubly stochastic matrices (Birkhoff-von Neumann polytope): Π R L L, Π 0, 1 T Π = 1 T, Π1 = 1 Rotation by an element of SO(3) should map nearby-points to nearby-points. More precisely, SO(3) preserves inner products: Ω ij = ω i,ω j ǫ = ω π(i),ω π(j) = Ω π(i),π(j) ΠΩΠ T ǫ = Ω = ΠΩ ǫ = ΩΠ Axis of rotation (Euler s rotation theorem): Tr(Π) ǫ 2 Notice: We are discretizing S 2, not SO(3) (substantial gain in computational complexity) Amit Singer (Princeton University) July / 30

25 Convex Relaxation of Cycle Consistency: SDP Let G be a block-matrix of size n n with G ij R L L. We want G ij = Π i Π T j. G is PSD, G ii = I L L, and rank(g) = L (the rank constraint is dropped). Convex relaxation of search space (putting it all together): G R nl nl G 0 G ii = I L L (cycle consistency) G ij 0 G ij 1 = 1 1 T G ij = 1 T (doubly stochastic) G ij Ω = ǫ ΩG ij Tr(G ij ) ǫ 2 (SO(3)) Related work: Charikar, Makarychev, Makarychev, Near optimal algorithms for Unique Games (STOC 2006) Amit Singer (Princeton University) July / 30

26 Maximum Likelihood: Linear Objective Function The common line depends on R i R T j. Log-Likelihood function is of the form f ij (R i Rj T ) i j Nonlinear in R i R T j, but linear in G. Proof by picture. Amit Singer (Princeton University) July / 30

27 Linear Objective: Proof by picture Amit Singer (Princeton University) July / 30

28 Tightness of the SDP p σ p σ Figure : Fraction of trials (among 100) on which rank recovery was observed. Left: Generalized Orthogonal Procrustes. Right: multi-reference alignment (registration). The solution of the SDP has the desired rank up to a certain level of noise (w.h.p). In other words, even though the search-space is exponentially large, we typically find the MLE in polynomial time. This is a viable alternative to heuristic methods such as EM and alternating minimization. The SDP gives a certificate whenever it finds the MLE. Amit Singer (Princeton University) July / 30

29 Are we there yet? The SDP approach can be used in a variety of problems where the objective function is a sum of pairwise interactions Need better theoretical understanding for the phase transition behavior and conditions for tightness of the SDP. Need better computational tools for solving large scale SDPs. (we use ADMM). Amit Singer (Princeton University) July / 30

30 References A. Singer, Y. Shkolnisky, Three-Dimensional Structure Determination from Common Lines in Cryo-EM by Eigenvectors and Semidefinite Programming, SIAM Journal on Imaging Sciences, 4 (2), pp (2011). R. Hadani, A. Singer, Representation theoretic patterns in three dimensional Cryo-Electron Microscopy I The intrinsic reconstitution algorithm, Annals of Mathematics, 174 (2), pp (2011). L. Wang, A. Singer, Z. Wen, Orientation Determination of Cryo-EM images using Least Unsquared Deviations, SIAM Journal on Imaging Sciences, 6(4), pp (2013). A. S. Bandeira, M. Charikar, A. Singer, A. Zhu, Multireference Alignment using Semidefinite Programming, 5th Innovations in Theoretical Computer Science (ITCS 2014). A. S. Bandeira, Y. Khoo, A. Singer, Open problem: Tightness of maximum likelihood semidenite relaxations, COLT Also available at Amit Singer (Princeton University) July / 30