Mathematical analysis of ultrafast ultrasound imaging Giovanni S Alberti Department of Mathematics, University of Genoa Joint work with H. Ammari (ETH), F. Romero (ETH) and T. Wintz (Sony). IPMS 18, May 1-5 Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 1 / 33
From ultrasonography to ultrafast ultrasonography Filippo, 17 Errico et. al, Nature 57, 99 5, 15 Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 / 33
Conventional ultrasonography Conventional ultrasound imaging: focused ultrasonic waves high spatial resolution long acquisition time very low contrast: soft biological tissues are almost acoustically homogeneous, due to the high water concentration fine details (such as blood vessels) are completely invisible Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 3 / 33
Ultrafast ultrasound imaging Use of plane waves instead of focused waves High frame rate: up to, frames per second Lots of data to post-process Demené et al., IEEE Trans Med Imaging, 15. Errico et. al, Nature, 15 Single frame of Ultrafast ultrasound brain of a thinned skull rat Power doppler image obtained via a SVD filter applied to 5 frames. Superresolution: 75, frames with blinking microbubbles Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 / 33
Summary 1 Mathematical modeling of Ultrafast Ultrasonography and blood flow imaging Super-resolution in Ultrafast Ultrasound Localization Microscopy Giovanni S. Alberti, Habib Ammari, Francisco Romero, and Timothée Wintz, Mathematical analysis of ultrafast ultrasound imaging, SIAM J. Appl. Math. 77 (17), no. 1, 1 5. Giovanni S. Alberti, Habib Ammari, Francisco Romero, and Timothée Wintz, Dynamic spike super-resolution and applications to ultrafast ultrasound imaging, arxiv preprint arxiv:183.351 (18). Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 5 / 33
Blood flow imaging The main issue is the removal of the clutter signal (the scattering coming from the tissue) Ultrafast ultrasonography allows us to overcome this issue, thanks to the very high frame rate. Idea: blood moves, tissue does not (in general). Temporal filters (Bercoff et al., 11): high-pass filtering the data to remove clutter signals. Drawback: not applicable when the clutter and blood velocities are close. Idea: tissue movement is spatially coherent, while blood flow is not. Spatiotemporal method based on the SVD of the data (Demene et al., 15): exploits the different spatial coherence of the clutter and blood scatterers. Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 7 / 33
The static direct problem y x 1.8.6. Receptor array z. -. -. Imaging plane -.6 -.8 The imaging system -1-1.5-1 -.5.5 1 1.5 Time (s) 1-6 The pulse f(t) = e πiν t χ (ν t) Incident field in the direction k θ = (sin θ, cos θ): u i (x, y, z, t) = A z (y)f ( t c 1 k θ (x, z) ) c : background speed of sound. c(x): speed of sound. Perturbation: n(x) = 1 c (x) 1 c In the Born approximation, the scattered field takes the form: u s (u, t) = ˆR (π) 1 u x f ( t x k θ + u x c ) n (x ) dx, u = (u, ) Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 8 / 33
The static inverse problem: beamforming Scattered field u x x u k θ x x u s (u, t), u = (u, ), t > Travel time from the receptor array Γ to a point x and back to a receptor in u : τ θ x(u) = c 1 (k θ x + x u ) k θ Beamforming: averaging the signals z s θ (x, z) := ˆ x+f z x F z Inserting the expression for u s obtained before we obtain u s ( u, τ θ x (u) ) du ˆ ˆ x+f z s θ (x) = n (x ) (π) 1 x R x F z x u f ( τx θ (u) τx θ (u)) du dx }{{} =g θ (x,x ) Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 9 / 33
The static inverse problem: the point spread function The static image s θ may be rewritten as ˆ s θ (x) = g θ (x, x ) n (x ) dx, x R where g θ is the point spread function of the system: ˆ x+f z g θ (x, x (π) 1 ) = x F z x u f ( τx θ (u) τx θ (u)) du The PSF may be approximated with a convolution g θ (x, x ) g θ (x x ), s θ = g θ n, where (f = ν c 1 and χ = πiχ + χ ) g θ (x) iν F χ (f z) e πifz e πifθx sinc(πf F x) Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 1 / 33
The point spread function In the particular case θ = : g (x) iν F χ (f z) e πifz sinc(πf F x) The real part of the PSF g 1 13 8 6 - - -6-8 The real part of the PSF g (The size of the square is mm mm, and the horizontal and vertical axes are the x and z axes.) 1 13 8 6 - - -6-8 Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 11 / 33
Angle compounding In order to improve the decay in the x direction, (Montaldo et al., 9) introduced angle compounding: s ac Θ (x) = 1 ˆ Θ s θ (x) dθ, gθ ac (x) = 1 ˆ Θ g θ (x) dθ Θ Θ Θ Θ A simple derivation shows that the PSF is g ac Θ (x) = g (x) sinc(πν c 1 Θx) Θ = : we recover g θ for θ =. Θ > : this PSF enjoys faster decay in the variable x. 1 13 8 6 - - -6-8 1 13 8 6 - - -6-8 (a) g θ, θ = (b) g ac Θ, Θ =.5 Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 1 / 33
The dynamic forward problem The dynamic imaging setup consists in the repetition of the static imaging method over time to acquire a collection of images of a medium in motion. There are two time scales: the fast one related to the propagation of the wave is considered instantaneous with respect to the slow one, related to the sequence of the images. We now neglect the time of the propagation of a single wave to obtain static imaging. The time t considered here is related to the slow time scale. At fixed time t, we obtain a static image of the medium n = n(x, t): s(x, t) = (g ac Θ n(, t)) (x). Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 13 / 33
The dynamic inverse problem: Source separation Repeating the process for t [, T ] we obtain the movie s(x, t), which represents the main data we now need to process. Main aim: locating the (possibly very small) blood vessels. Main issue: s(x, t) is highly corrupted by clutter signal, namely, the signal scattered from tissues. Decompose The measurements are n(x, t) = n c (x, t) + n b (x, t) s(x, t) = s c (x, t) + s b (x, t) Inverse problem: determine the spatial support of n b. Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 1 / 33
The SVD separation algorithm (Demené et al., 15) (x i ) i=1,...,mx, (t j ) j=1,...,mt : sampling locations and times. Casorati matrix S C mx mt (m t m x ): The singular value decomposition of S S(i, j) = s(x i, t j ). m t S(i, j) = σ k u k (i) v k (j) k=1 singular vectors: (u1,..., u mx ) and (v 1,..., v mt ) are ONB of C mx and C m t singular values: σ1 σ... σ mt the dynamic data S are expressed as a sum of spatial components uk moving with time profiles v k, with weights σ k. Since the tissue movement has higher spatial coherence than the blood flow, the first factors are expected to contain the clutter signal, and the remainder to provide information about the blood location The blood location may be recovered by looking at the power Doppler m t k=k+1 σ k u k (i) Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 15 / 33
A general multiple scatterer random model Consider N point particles, with positions a k : i.i.d. stochastic processes a k (t), k = 1,..., N. The medium and the measurements are given by n (x, t) = C N N k=1 δ ak (t) (x), s(x, t) = C N N k=1 C > : scattering intensity N : natural normalization factor (central limit theorem) 1 Casorati matrix S N C mx mt : S N (i, j) = s(x i, t j ). g (x a k (t)) Multivariate central limit theorem: S N converges in distribution to a Gaussian complex matrix S C mx mt The distribution of S is entirely determined by g and the law of a k Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 16 / 33
Justification of the SVD method (1D) Using the multiple scatterer random model introduced above, we construct two Casorati matrices S b, S c as limits of particles with the following statistics. Clutter: large support, constant velocities a k (t) = u k + v c t where u k is uniformly distributed in (, L c ). Blood: small support, varying velocities: a k (t) = u k + v b t + σb t where u k is uniformly distributed in (, L b ) (L b L c ) and B t is a Brownian motion. S b and S c may be constructed using the Gaussian limit approximation Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 17 / 33
Justification of the SVD method (1D) #1-1 1.5 Clutter Blood Noise #1-1.5 Clutter (:5 cm.s!1 ) Clutter (1 cm.s!1 ) Clutter ( cm.s!1 ) 1 1.5 1.5.5 6 8 1 #1 1 (a) The clutter model (v c = 1 m s 1 ), the blood model (σ = 1 6 m s 1, v b = 1 m s 1 ) and a white noise model with same variance as the blood. 6 8 1 #1 1 (b) The clutter model with different velocities. Figure: The distribution of the singular values of the Casorati matrix S in different cases. Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 18 / 33
z Numerical simulations We put one blood vessel in a moving tissue: domain: 5 mm 5 mm Θ = 7 The density of particles for both blood and clutter is, per mm C c = 5C b A single frame of the measurements s(x, t ) is #1-3.5 1 1.5 #1 16.5 3-3.5 -.5 5 - -1 1 x #1-3 -6 Need further processing to locate the blood vessel! Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 19 / 33
Numerical simulations: v b > v c - - 1.5 1.5 1.5 1.5 1 3 1 3 - - 3 1 3 1 1 15 1 1 18 1 16 (a) Maximum blood velocity: cm s 1 ; mean clutter velocity: 1 cm s 1. 1 Clutter Blood Both Clutter Blood Both Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 / 33
Numerical simulations: v b = v c 1.8 1 3 1 -.6.. 3-1 1 15 Clutter Blood Both 1.8 1 3 1 -.6.. 3-1 1 15 (b) Maximum blood velocity: 1 cm s 1 ; mean clutter velocity: 1 cm s 1. Clutter Blood Both Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 1 / 33
Numerical simulations: v b < v c - -.5..3..1.5..3..1 1 3 1 3 - - 3 1 3 1 1 15 1 1 15 (c) Maximum blood velocity:.5 cm s 1 ; mean clutter velocity: 1 cm s 1. 1 Clutter Blood Both Clutter Blood Both Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 / 33
Super-resolution with ultrafast ultrasound Errico et. al, Nature 57, 99 5, 15 Use of randomly activated microbubbles as in Super-resolved Fluorescence Microscopy (Nobel Prize in Chemistry 1) Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 / 33
Superresolution: static vs dynamic Current method a b: SVD b c: identify center of PSF, if well-separated Track bubbles to obtain velocities Drawbacks: slow discard a lot of data Errico et. al, Nature 57, 99 5, 15 New method dynamical superresolution in time and space based on l 1 minimization obtain locations and velocities in one step Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 5 / 33
The static super-resolution problem x 1,..., x N [, 1] d : locations of N particles w 1,..., w N C: weights The corresponding unknown Radon measure is t µ = N w i δ xi M([, 1] d ) i=1 For simplicity, we consider only low frequency Fourier measurements ˆ (Fµ) l = e πil x dµ(x), l { f c,..., f c } d, [,1] d where f c N is the highest measurable frequency. IP: reconstruct µ (i.e. the locations x i with infinite resolution) from (Fµ) l The unknown µ may be recovered via the convex problem min ν TV ν M([,1] d ) subject to Fν = Fµ, provided that the particles are well separated (Candes, Fernandez-Granda, CPAM 13) or d = 1 and w i (De Castro, Gamboa, JMAA, 1) Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 6 / 33
The dynamical super-resolution problem t t 1 t t1 t Time steps t k = kτ where τ > and k = K,..., K Dynamic spikes with positions x i (t k ) at time k: µ tk = N w i δ xi(t k ) i=1 For each t k, consider the static measurements Fµ tk, and so the full data are (Fµ tk ) l, l { f c,..., f c } d, k { K,..., K}. Aim: recover positions and velocities of the particles x i Static approach: at each time step t k, perform static reconstruction, and then use tracking to obtain the velocities We wish to design a fully dynamical reconstruction algorithm Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 7 / 33
Lifting the particles to the phase space Linear approximation of (local) movement of particles: µ tk = N w i δ xi+t k v i, where x i is the position at time t = and v i is the velocity. Lifting the unknown measure to N ω = w i δ (xi,v i) i=1 i=1 allows to separate the particles in the phase space: v 1 x 1 x v x 1 x Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 8 / 33
The measurements The measurements become (Gω) l,k = ˆ e πi(x+kτv) l dω(x, v) = ˆ e πi(x,v) (l,kτl) dω(x, v). In other words, we measure a d-dimensional Fourier transform restricted to { (l, kτl) : l Z d, l f c, k = K,..., K }, Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 9 / 33
The dynamical reconstruction Unknown measure: Measurements: N ω = w i δ (xi,v i) i=1 ˆ (Gω) l,k = e πi(x,v) (l,kτl) dω(x, v). t t 1 t t1 t We seek to reconstruct positions and velocities simultaneously by minimizing min λ TV subject to Gλ = Gω λ Theoretical results (see also (Dossal, Duval, Poon, SINUM 17)): exact reconstruction in the noiseless case, under suitable assumptions on (x i, v i) through a distance dyn in the phase space stable reconsctruction in presence of noise Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 3 / 33
Dynamical vs static reconstructions The results of 18, simulations in 1D with randomly generated particles are: 1. Correct recontruction rate.8.6.......6.8 1. 1. dyn dynamic static static3 1/f c Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 31 / 33
An example We simulate blinking microbubbles in two close vessels with different velocities (a) frame (b) frame (c) frame 8 (d) B-mode image Super resolved reconstruction of the simulated particles The colors represent the velocity in the vertical direction 1..8.6.......6.8 1. 1 5-5 -1 Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 3 / 33
Conclusions I Mathematical modeling of ultrafast ultrasonography I Justification of the SVD algorithm by means of random particles I Design of a fully dynamical reconstruction method for super-resolution images of blood vessels Future directions I I I I Higher order approximation of the particles movement Efficient numerical implementation Simulations with real data Giovanni S Alberti (University of Genoa) Ultrafast ultrasound imaging IPMS 18, May 1-5 33 / 33