3D Structure Determination using Cryo-Electron Microscopy Computational Challenges Amit Singer Princeton University Department of Mathematics and Program in Applied and Computational Mathematics July 28, 2015 Mathematics in Data Science Amit Singer (Princeton University) July 2015 1 / 28
Single Particle Reconstruction using cryo-em Schematic drawing of the imaging process: The cryo-em problem: Amit Singer (Princeton University) July 2015 2 / 28
New detector technology: Exciting times for cryo-em BIOCHEMISTRY The Resolution Revolution Werner Kühlbrandt P recise knowledge of the structure of macromolecules in the cell is essential for understanding how they function. Structures of large macromolecules can now be obtained at near-atomic resolution by averaging thousands of electron microscope images recorded before radiation damage accumulates. This is what Amunts et al. have done in their research article on page 1485 of this issue ( 1), reporting the structure of the large subunit of the mitochondrial ribosome at 3.2 Å resolution by electron cryo-microscopy (cryo-em). Together with other recent high-resolution cryo-em structures ( 2 4) (see the figure), this achievement heralds the beginning of a new era in molecular biology, where structures at near-atomic resolution are no longer the prerogative of x-ray crystallography or nuclear magnetic resonance (NMR) spectroscopy. Ribosomes are ancient, massive protein- RNA complexes that translate the linear genetic code into three-dimensional proteins. Mitochondria semi-autonomous organelles www.sciencemag.org SCIENCE VOL 343 28 MARCH 2014 1443 A B C Advances in detector technology and image processing are yielding high-resolution electron cryo-microscopy structures of biomolecules. Near-atomic resolution with cryo-em. (A) The large subunit of the yeast mitochondrial ribosome at 3.2 Å reported by Amunts et al. In the detailed view below, the base pairs of an RNA double helix and a magnesium ion (blue) are clearly resolved. (B) TRPV1 ion channel at 3.4 Å ( 2), with a detailed view of residues lining the ion pore on the four-fold axis of the tetrameric channel. (C) F 420-reducing [NiFe] hydrogenase at 3.36 Å ( 3). The detail shows an α helix in the FrhA subunit with resolved side chains. The maps are not drawn to scale. Amit Singer (Princeton University) July 2015 3 / 28
Big Movie Data, Publicly Available http://www.ebi.ac.uk/pdbe/emdb/empiar/ Amit Singer (Princeton University) July 2015 4 / 28
Image Formation Model and Inverse Problem Projection I i Molecule φ R i = Ri 1 Ri 2 Ri 3 T T T SO(3) Electron source Projection images I i (x,y) = φ(xr1 i +yri 2 +zri 3 )dz + noise. φ : R 3 R is the electric potential of the molecule. Cryo-EM problem: Find φ and R 1,...,R n given I 1,...,I n. Amit Singer (Princeton University) July 2015 5 / 28
Toy Example Amit Singer (Princeton University) July 2015 6 / 28
E. coli 50S ribosomal subunit 27,000 particle images provided by Dr. Fred Sigworth, Yale Medical School 3D reconstruction by S, Lanhui Wang, and Jane Zhao Amit Singer (Princeton University) July 2015 7 / 28
Main Algorithmic Challenges 1 Orientation assignment 2 Heterogeneity (resolving structural variability) 3 2D Class averaging (de-noising) 4 Particle picking 5 Symmetry detection 6 Motion correction Amit Singer (Princeton University) July 2015 8 / 28
The heterogeneity problem What if the molecule has more than one possible structure? (Image source: H. Liao and J. Frank, Classification by bootstrapping in single particle methods, ISBI, 2010.) Katsevich, Katsevich, S (SIAM Journal on Imaging Sciences, 2015) Covariance matrix estimation of the 3-D structures from their 2-D projections (high-dimensional statistics, random matrices, low-rank matrix completion) Amit Singer (Princeton University) July 2015 9 / 28
Experimental Data: 70S Ribosome 10000 image dataset (130-by-130), courtesy Joachim Frank (Columbia University) Class 1 Class 2 Morphing video by S, Joakim Andén, and Eugene Katsevich Amit Singer (Princeton University) July 2015 10 / 28
Class averaging for image denoising Rotation invariant representation (steerable PCA, bispectrum) Vector diffusion maps (S, Wu 2012), generalization of Laplacian Eigenmaps (Belkin, Niyogi 2003) and Diffusion Maps (Coifman, Lafon 2006) Graph Connection Laplacian Experimental images (70S) courtesy of Dr. Joachim Frank (Columbia) Class averages by vector diffusion maps (averaging with 20 nearest neighbors) (Zhao, S 2014) Amit Singer (Princeton University) July 2015 11 / 28
Orientation Estimation Standard procedure is iterative refinement. Alternating minimization or expectation-maximization, starting from an initial guess φ 0 for the 3-D structure I i = P(R i φ)+ǫ i, i = 1,...,n. R i φ(r) = φ(r 1 i r) is the left group action P is integration in the z-direction and grid sampling. Converges to a local optimum, not necessarily the global one. Model bias is a well-known pitfall Is reference free orientation assignment and reconstruction possible? Amit Singer (Princeton University) July 2015 12 / 28
3D Puzzle Optimization problem: min g 1,g 2,...,g n G n i,j=1 f ij (g i g 1 j ) G = SO(3) is the group of rotations in space. Parameter space G G G is exponentially large. Amit Singer (Princeton University) July 2015 13 / 28
Non-Unique Games over Compact Groups min g 1,g 2,...,g n G n i,j=1 f ij (g i g 1 j ) For G = Z 2 this encodes Max-Cut. 1 3 w 12 w 34 2 w 14 w 24 w 35 w 25 4 5 w 45 Max-2-Lin(Z L ) formulation of Unique Games (Khot et al 2005): Find x 1,...,x n Z L that satisfy as many difference eqs as possible x i x j = b ij This corresponds to G = Z L and mod L, (i,j) E f ij (x i x j ) = 1 {xi x j =b ij } Our games are non-unique in general, and the group is not necessarily finite. Amit Singer (Princeton University) July 2015 14 / 28
Orientation Estimation: Fourier projection-slice theorem (x ij,y ij ) R i c ij c ij = (x ij,y ij,0) T Projection I i Î i (x ji,y ji ) 3D Fourier space R i c ij = R j c ji Projection I j Î j 3D Fourier space Amit Singer (Princeton University) July 2015 15 / 28
Angular Reconstitution (Vainshtein and Goncharov 1986, Van Heel 1987) Amit Singer (Princeton University) July 2015 16 / 28
Least Squares Approach (x ij,y ij) R ic ij c ij = (x ij,y ij,0) T Projection I i Î i 3D Fourier space (x ji,y ji) R ic ij = R jc ji Projection I j Î j 3D Fourier space min R 1,R 2,...,R n SO(3) R i c ij R j c ji 2 i j Search space is exponentially large and non-convex. Spectral and semidefinite programming relaxations (non-commutative Grothendieck, Max-Cut) S, Shkolnisky (SIAM Journal on Imaging Sciences, 2011) Amit Singer (Princeton University) July 2015 17 / 28
MLE The images contain more information than that expressed by optimal pairwise matching of common lines. Algorithms based on pairwise matching can succeed only at high SNR. (Quasi) Maximum Likelihood: We would like to try all possible rotations R 1,...,R n and choose the combination for which the agreement on the common lines (implied by the rotations) as observed in the images is maximal. Computationally intractable: exponentially large search space, complicated cost function. Amit Singer (Princeton University) July 2015 18 / 28
Quasi MLE Common line equation: R i c ij = R j c ji = R ie 3 R j e 3 R i e 3 R j e 3 with e 3 = (0,0,1) T. Quasi MLE min c ij = R 1 i c ji = R 1 j n R 1,...,R n SO(3) i,j=1 R i e 3 R j e 3 R i e 3 R j e 3 = e 3 R 1 i R j e 3 e 3 R 1 i R j e 3 R i e 3 R j e 3 R i e 3 R j e 3 = R 1 j R i e 3 e 3 R 1 i R i e 3 e 3 Îi(,c ij ) Îj(,c ji ) 2 min g 1,...,g n G n i,j=1 f ij (g i g 1 j ) Amit Singer (Princeton University) July 2015 19 / 28
Fourier transform over G Recall for G = SO(2) f(α) = ˆf(k) = 1 2π k= 2π 0 ˆf(k)e ıkα f(α)e ıkα dα In general, for a compact group G ] f(g) = d k Tr[ˆf(k)ρ k (g) ˆf(k) = k=0 G Here ρ k are the unitary irreducible representations of G d k is the dimension of the representation ρ k (e.g., d k = 1 for SO(2), d k = 2k +1 for SO(3)) dg is the Haar measure on G f(g)ρ k (g) dg Amit Singer (Princeton University) July 2015 20 / 28
Linearization of the cost function Introduce matrix variables Fourier expansion of f ij Linear cost function f(g 1,...,g n ) = f ij (g) = n i,j=1 X (k) ij k=0 = ρ k (g i g 1 j ) ] d k Tr[ˆf ij (k)ρ k (g) f ij (g i g 1 j ) = n i,j=1k=0 ] d k Tr [ˆf ij (k)x (k) ij Amit Singer (Princeton University) July 2015 21 / 28
Constraints on the variables X (k) ij = ρ k (g i g 1 j ) 1 X (k) 0 2 X (k) ii = I dk, for i = 1,...,n 3 rank(x (k) ) = d k = ρ k (g i g 1 j ) = ρ k (g i )ρ k (g 1 j ) = ρ k (g i )ρ k (g j ) ρ k (g 1 ) X (k) ρ k (g 2 ) [ = ρk (g 1 ) ρ k (g 2 ) ρ k (g n ) ]. ρ k (g n ) X (k) ij We drop the non-convex rank constraint. The relaxation is too loose, as we can have X (k) ij = 0 (for i j). Even with the rank constraint, nothing ensures that X (k) ij and X (k ) ij correspond to the same group element g i g 1 j. Amit Singer (Princeton University) July 2015 22 / 28
Additional constraints on X (k) ij = ρ k (g i g 1 j ) The delta function for G = SO(2) δ(α) = k= Shifting the delta function to α i α j δ(α (α i α j )) = k= e ıkα e ıkα e ık(α i α j ) = k= The delta function is non-negative and integrates to 1: e ıkα X (k) ij 0, α [0,2π) k= 1 2π 2π 0 k= e ıkα X (k) ij (0) dα = X ij = 1 e ıkα X (k) ij Amit Singer (Princeton University) July 2015 23 / 28
Finite truncation via Fejér kernel In practice, we cannot use infinite number of representations to compose the delta function. Simple truncation leads to the Dirichlet kernel which changes sign D m (α) = m k= m e ıkα This is also the source for the Gibbs phenomenon and the non-uniform convergence of the Fourier series. The Fejér kernel is non-negative and leads to uniform convergence F m (α) = 1 m m 1 k=0 D k (α) = m k= m ( 1 k ) e ıkα m The Fejér kernel is the first order Cesàro mean of the Dirichlet kernel. Amit Singer (Princeton University) July 2015 24 / 28
Finite truncation via Fejér-Riesz factorization Non-negativity constraints over SO(2) m ( 1 k ) e ıkα X (k) ij 0, α [0,2π) m k= m Fejér-Riesz: P is a non-negative trigonometric polynomial over the circle, i.e. P(e ıα ) 0 α [0,2π) iff P(e ıα ) = Q(e ıα ) 2 for some polynomial Q. { Leads to semidefinite constraints on X (k) ij } k for each i,j. Similar non-negativity constraints hold for general G using the delta function over G δ(g) = d k Tr[ρ k (g)] k=0 For example, Fejér proved that for SO(3) the second order Cesàro mean of the Dirichlet kernel is non-negative. Amit Singer (Princeton University) July 2015 25 / 28
Tightness of the semidefinite program We solve an SDP for the matrices X (0),...,X (m). Numerically, the solution of the SDP has the desired ranks up to a certain level of noise (w.h.p). In other words, even though the search-space is exponentially large, we typically find the MLE in polynomial time. This is a viable alternative to heuristic methods such as EM and alternating minimization. The SDP gives a certificate whenever it finds the MLE. Amit Singer (Princeton University) July 2015 26 / 28
Final Remarks Loss of handedness ambiguity in cryo-em: If g 1,...,g n SO(3) is the solution, then so is Jg 1 J 1,...,Jg n J 1 for J = diag( 1, 1,1). [ ] Define X (k) ij = 1 2 ρ k (g i g 1 j )+ρ k (Jg i g 1 j J 1 ) Splits the representation: 2k +1 = d k = k +(k +1), reduced computation Point group symmetry (cyclic, dihedral, etc.): reduces the dimension of the representation (invariant polynomials) Translations and rotations simultaneously: SE(3) is a non-compact group, but we can map it to SO(4). Amit Singer (Princeton University) July 2015 27 / 28
References A. Singer, Y. Shkolnisky, Three-Dimensional Structure Determination from Common Lines in Cryo-EM by Eigenvectors and Semidefinite Programming, SIAM Journal on Imaging Sciences, 4 (2), pp. 543 572 (2011). A. Singer, H.-T. Wu, Vector Diffusion Maps and the Connection Laplacian, Communications on Pure and Applied Mathematics, 65 (8), pp. 1067 1144 (2012). Z. Zhao, A. Singer, Rotationally Invariant Image Representation for Viewing Direction Classification in Cryo-EM, Journal of Structural Biology, 186 (1), pp. 153 166 (2014). A. S. Bandeira, M. Charikar, A. Singer, A. Zhu, Multireference Alignment using Semidefinite Programming, 5th Innovations in Theoretical Computer Science (ITCS 2014). A. S. Bandeira, Y. Chen, A. Singer Non-Unique Games over Compact Groups and Orientation Estimation in Cryo-EM, http://arxiv.org/abs/1505.03840. G. Katsevich, A. Katsevich, A. Singer, Covariance Matrix Estimation for the Cryo-EM Heterogeneity Problem, SIAM Journal on Imaging Sciences, 8 (1), pp. 126 185 (2015). J. Andén, E. Katsevich, A. Singer, Covariance estimation using conjugate gradient for 3D classification in Cryo-EM, 12th IEEE International Symposium on Biomedical Imaging (ISBI 2015). Amit Singer (Princeton University) July 2015 28 / 28