Tutorial: Sparse Signal Recovery

Transcription

1 Tutorial: Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan

2 (Sparse) Signal recovery problem signal or population length N k important Φ x = y measurements or tests: length m features Under-determined linear system: Φx = y Given Φ and y, recover information about x Solving LS: arg min Φx y 2 returns a non-sparse x

3 Two main examples: group testing and compressed sensing Φ x = y Group testing Φ binary = pooling design x binary, 1 = defective, 0 = acceptable OUTPUT: defective set success = find defective set exactly Arithmetic: OR

4 Example: Combinatorial group testing Rat dies only 1 week after drinking poisoned wine

5 Being good (computer) scientists, they do the following:

6

7 Unique encoding of each bottle

8 If bottle 5 were poison...

9 ...after 1 week

10 Two main examples: group testing and compressed sensing Φ x = y Compressed sensing Φ = measurement matrix x signal, sparse/compressible OUTPUT: x good approximation to x success = x x p versus x x k q Arithmetic: R or C Head xk k Tail x xk N

11 Biological applications are a combination of these examples Sparse (binary?) matrices Quantitative (non-boolean) measurements Noisy or missing measurements Various signal models

12 Theoretical design problems: jointly design matrices and algorithms Design/Characterize Φ with m < N rows and recovery algorithm s.t. output x is close to good approximation of x: x x p C x x k q.

13 Algorithmic resources Computational time: return x in time proportional to N or to m? (impacts number of items returned) Computational memory: use working space proportional to N or to m? Small-space random constructions: can we generate rows/cols of Φ from collections of random vars with limited independence? Explicit constructions: how do we construct Φ? Randomly or deterministically? Time to calculate each entry? Matrix density: number of non-zero entries in Φ?

14 Signal models Adversarial or for all recover all x with guarantee x x 2 C k x x k 1. When x is compressible, or x x k 1 k x x k 2, obtain better result x x 2 C x x k 2. Probabilistic or for each : recover all x that satisfy a statistical constraint fixed signal x, recover with high probability over construction of Φ uniform distribution over k-sparse signals i.e., random signal model

15 Empirical algorithmic performance Polynomial time algorithms: typically fewer measurements required for desired accuracy, by constant factor 1 Convex optimization: constrained minimization (e.g., Bregman iteration) arg min x 1 s.t. y Φx 2 δ. Unconstrained minimization (e.g., l 1 -regularized LS) minimize L(x; γ, y) = 1 2 y Φx γ x 1. 2 Greedy algorithms: iterative thresholding, subspace iteration z = Φ y then threshold z, keep largest T entries Sublinear time algorithms: for sparse, binary matrices only (currently) Strive to work in space and time proportional to m, specially designed matrices/distributions and algorithms

16 Sparse matrices: Expander graphs S N(S) Adjacency matrix Φ of a d regular (1, ɛ) expander graph Graph G = (X, Y, E), X = n, Y = m For any S X, S k, the neighbor set Probabilistic construction: N(S) (1 ɛ)d S d = O(log(n/k)/ɛ), m = O(k log(n/k)/ɛ 2 ) Deterministic construction: d = O(2 O(log3 (log(n)/ɛ)) ), m = k/ɛ 2 O(log3 (log(n)/ɛ))

17 Bipartite graph Adjacency matrix Measurement matrix (larger example)

18 RIP(p) A measurement matrix Φ satisfies RIP(p, k, δ) property if for any k-sparse vector x, Examples: (1 δ) x p Φx p (1 + δ) x p. RIP(2): Gaussian random matrix, random rows of discrete Fourier matrix RIP(1): adjacency matrix of expander

19 RIP(p) expander Theorem (k, ɛ) expansion implies (1 2ɛ)d x 1 Φx 1 d x 1 for any k-sparse x. d = degree of expander. Theorem RIP(1) + binary sparse matrix implies (k, ɛ) expander for ɛ = 1 1/(1 + δ) 2. 2 Joint work with Berinde, Indyk, Karloff, Strauss

20 RIP(2) and LP decoding Let Φ be an RIP(2) matrix and collect measurements Φx = y. Then LP recovers a good approximation to the top k entries of x: arg min x 1 s.t. Φx = y LP returns x with x 2 C k x x k 1. And, noise-resilient version too. [Candés, Tao, + Romberg; Donoho]

21 Expansion = LP decoding Theorem Φ adjacency matrix of (2k, ɛ) expander. Consider two vectors x, x such that Φx = Φx and x 1 x 1. Then x x α(ɛ) x x k 1 where x k is the optimal k-term representation for x and α(ɛ) = (2ɛ)/(1 2ɛ). Guarantees that Linear Program recovers good sparse approximation Noise-resilient version too

22 RIP(1) RIP(2) Any binary sparse matrix which satisfies RIP(2) must have Ω(k 2 ) rows [Chandar 07] Gaussian random matrix m = O(k log(n/k)) (scaled) satisfies RIP(2) but not RIP(1) x T = ( ) y T = ( 1/k 1/k 0 0 ) x 1 = y 1 but Gx 1 k Gy 1 For biological applications, it may be useful to use sparse, binary RIP(1)+RIP(2) matrices with many rows.

23 RIP(1) = LP decoding l 1 uncertainty principle Lemma Let y satisfy Φy = 0. Let S the set of k largest coordinates of y. Then y S 1 α(ɛ) y 1. LP guarantee Theorem Consider any two vectors u, v such that for y = u v we have Ay = 0, v 1 u 1. S set of k largest entries of u. Then y α(ɛ) u S c 1.

24 Greedy, iterative algorithms for sparse matrices Polynomial time algorithms: several variants, similar to IHT [Berinde, Indyk, Ruszic] Sublinear time algorithms: several variants, depend on norms, signal models, [Gilbert, Li, Porat, Strauss; Porat, Strauss; Ngo, Porat, Rudra; Price, Woodruff]

25 Motivation Pooling in HTS QUAPO Results High Throughput Screening (HTS) High Throughput Screening (HTS) HTS is an essential step in drug discovery (and HTS elsewhere in biology) is an essential step in drug discovery (and elsewhere in biology). Large chemical libraries screened on a biological targetscreened for activity. Large chemical libraries on a biologicalbasic target for type activity yes-no biological assays active compounds. to find Basic {0,to1}find type biological assays active compounds Usually a small number of Usually acompounds found.of compounds found small number One-at-a-time screening: automation One-at-a-time screening powerand miniaturization comes from automation and miniaturization. Noisy assays with false positives and negative Noisy errorsassays with false positive and negative errors. Conclus

26 Current HTS Design Current HTS uses one-at-a-time testing scheme (with repeated Current trials). HTS uses a one-at-a-time testing scheme.

27 Sparse Recovery Problem Mathematical set-up QUAPO Binary Binary measurement matrix: : adjacency Adjacency matrix of unbalanced expander graph. matrix of unbalanced expander graph a Appropriate linear biochemical model. Appropriate Decoding : linear via Linear biochemical Programming. model Decoding a Berinde et. al. (2008) Combining geometry and via linear programming combinatorics: A unified approach to sparse signal recovery

28 Pooled HTS Design Potential Advantages Pooled testing of compounds Uses fewer Propose tests the use of pooled testing of compounds. Work moved from testing (costly) Uses tofewer computational tests. analysis (cheap) Work is moved from testing Handles (costly) errorstoinanalysis testing(cheap). better due to built-in replication Handles errors in testing due to in-built redundancy. Additional quantitative information

29 Motivation Pooling in HTS QUAPO Results Conclusions HTS and signal recovery Analogy to HTS To use compressed sensing approach we need a linear model.

30 Experimental set-up Deterministic binary matrix = Shifted Transversal Design (STD) STD based Time-resolved screen FRET screen to identify inhibitors of RGS4C Singleplex: 316 compounds screened twice Multiplex: Time-resolved 316 FRET compounds screen to identify screened inhibitors 4 per of RGS4C well using 316 Singleplex: 316 compounds screened twice pooled wells Motivation poolhits QUAPO poolmc Summary Multiplex: 316 compounds screened 4 per well using 316 pooled wells (4X)

31 Motivation poolhits QUAPO poolmc Summary Experimental results QUAPO screen results Multiplex results match Singleplex 1 very closely Decoding Multiplex results in the match presence Singleplex of large 1 very errors closely. Decoding in the presence of large errors. 34 / 47

32 Theoretical Practical challenges in HTS applications Accuracy of output not speed of algorithm Joint matrix and algorithm design, maximize recovered information within budget Incorporate prior information Incorporate noise Incorporate measurement device configuration