Lecture 3: Compressive Classification

Transcription

1 Lecture 3: Compressive Classification Richard Baraniuk Rice University dsp.rice.edu/cs

2 Compressive Sampling

3 Signal Sparsity wideband signal samples large Gabor (TF) coefficients Fourier matrix

4 Compressive Sampling Random measurements measurements signal sparse in basis

5 Universality Compressive Sampling

6 Why Does It Work? (1) Random projection not full rank, but stably embeds sparse/compressible signal models (CS) point clouds (JL) into lower dimensional space with high probability Stable embedding: preserves structure distances between points, angles between vectors, provided M is large enough: Compressive Sampling K-sparse model K-dim planes

7 Why Does It Work? (2) Random projection not full rank, but stably embeds sparse/compressible signal models (CS) point clouds (JL) into lower dimensional space with high probability Stable embedding: preserves structure distances between points, angles between vectors, provided M is large enough: Johnson-Lindenstrauss Q points

8 Information Scalability In many CS applications, full signal recovery may not be required If we can reconstruct a signal from compressive measurements, then we should be able to perform other kinds of statistical signal processing: detection classification estimation

9 Compressive Detection/Classification

10 Classification of Signals in Noise Observe one of P known signals in noise Probability density function of the noise ex: zero mean, white Gaussian noise (AWGN) Probability density function of signal >>> mean shifted to

11 Multiclass Likelihood Ratio Test (LRT) Observe one of P known signals in noise Classify according to: AWGN: nearest-neighbor classification Sufficient statistic:

12 Compressive LRT Compressive observations: by the JL Lemma these distances are preserved [Waagen et al 05, Davenport et al 06, Haupt et al 06]

13 Compressive LRT Compressive observations: If are normalized, these angles are preserved

14 Performance of Compressive LRT ROC curve for Neyman-Pearson detector (2 classes): false alarm probability From JL lemma, for random orthoprojector Thus Penalty for compressive measurements:

15 Performance of Compressive LRT better SNR fewer measurements

16 Summary If CS system is a random orthoprojector, then detection/classification problems in the signal space map into analogous problems in the measurement space SNR penalty for compressive measurements Note: Signal sparsity unexploited!

17 Matched Filtering

18 Matched Filter In many applications, signals are transformed with an unknown parameter; ex: translation Elegant solution: matched filter Compute for all convolution of measurement with template reversed in time Simultaneously: estimates parameter classifies signal

19 Compressive Matched Filter Challenge: Extend matched filter concept to compressive measurements GLRT: GLRT approach extends to any case where each class can be parameterized with K parameters If mapping from parameters to signal is well-behaved, then each class forms a manifold in

20 Signal Manifolds

21 What is a Manifold? Manifolds are a bit like pornography: hard to define, but you know one when you see one. S. Weinberger [Lee] Locally Euclidean topological space Roughly speaking: a collection of mappings of open sets of R K glued together ( coordinate charts ) can be an abstract space, not a subset of Euclidean space e.g., SO3, Grassmannian Typically for signal processing: nonlinear K-dimensional surface in signal space R N

22 Examples Circle in R N parameter: angle Chirp in R N parameters: start frequency end frequency Image appearance manifold parameters: position of object, camera, lighting, etc.

23 Object Rotation Manifold each image is a point in R N K=1

24 Up/Down Left/Right Manifold K=2 [Tenenbaum, de Silva, Langford]

25 Manifold Learning from Training Data Translating disk parameters: left/right, up/down shift (K=2) Generate training data by sampling from the manifold Learn the structure of the manifold ISOMAP HLLE Laplacian Eigenmaps R 4096

26 Manifold Classification

27 Manifold Classification Now suppose data is drawn from one of P possible manifolds: AWGN: nearest manifold classification M 1 M 3 M 2

28 Compressive Manifold Classification Compressive observations: Good news: structure of smooth manifolds is preserved by random projection provided distances, geodesic distances, angles, [Wakin et al, 06, Haupt et al 07]

29 Aside: Random Projections of Smooth Manifolds

30 Theorem: Stable Manifold Embedding Let F R N be a compact K-dimensional manifold with condition number 1/τ (curvature, self-avoiding) volume V Let Φ be a random MxN orthoprojector with Then with probability at least 1-ρ, the following statement holds: For every pair x,y F [Wakin et al 06, Haupt et al 07]

31 Stable Manifold Embedding Theorem tells us that random projections preserve smooth manifold dimensionality ambient distances geodesic distances local angles topology local neighborhoods Volume Also there exists extension to some kinds of non-smooth manifolds

32 Manifold Learning from Compressive Measurements ISOMAP HLLE Laplacian Eigenmaps R 4096 R M M=15 M=20 M=15

33 Multiple Manifold Embedding Corollary: Let M 1,,M P R N be compact K-dimensional manifolds with condition number 1/τ (curvature, self-avoiding) volume V min dist(m j,m k ) > τ (can be relaxed) Let Φ be a random MxN orthoprojector with Then with probability at least 1-ρ, the following statement holds: For every pair x,y U M j

34 Compressive Manifold Classification

35 Compressive Manifold Classification Compressive observations: Good news: structure of smooth manifolds is preserved by random projection provided distances, geodesic distances, angles, [Wakin et al, 06, Haupt et al 07]

36 Smashed Filter Compressive manifold classification with GLRT nearest-manifold classifier based on manifolds M 1 M 3 M 2 Φ M 1 Φ M 2 Φ M 3 [Davenport et al 06, Healy and Rohode 07]

37 Smashed Filter Experiments 3 image classes: tank, school bus, SUV N = 65,536 pixels Imaged using single-pixel CS camera with unknown shift unknown rotation

38 Smashed Filter Unknown Position Object shifted at random (K=2 manifold) Noise added to measurements Goal: identify most likely position for each image class identify most likely class using nearest-neighbor test avg. shift estimate error more noise classification rate (%) more noise number of measurements M number of measurements M

39 Smashed Filter Unknown Rotation Object rotated each 10 o Goals: identify most likely rotation for each image class identify most likely class using nearest-neighbor test Perfect classification with as few as 6 measurements Good estimates of rotation with under 10 measurements avg. rot. est. error number of measurements M

40 Summary Compressive measurements are information scalable reconstruction > estimation > classification > detection Random projections preserve structure of smooth manifolds (analogous to sparse signals) Smashed filter: dimension-reduced GLRT for parametrically transformed signals exploits compressive measurements and manifold structure broadly applicable: targets do not have to have sparse representation in any basis effective for detection/classification

41 Open Issues Compressive classification does not exploit sparse signal structure to improve performance Non-smooth manifolds and local minima in GLRT one approach: multiscale random projections Experiments with real data

42 Some References R. G. Baraniuk, M. Davenport, R. DeVore, M. Wakin, A simple proof of the restricted isometry property for random matrices, to appear in Constructive Approximation, R. G. Baraniuk and M. Wakin, Random projections of smooth manifolds, 2006; see also ICASSP M. Davenport, M. Duarte, M. Wakin, J. Laska, D. Takhar, K. Kelly, R. G. Baraniuk, The smashed filter for compressive classification and target recognition, Proc. of Computational Imaging V at SPIE Electronic Imaging, San Jose, California, January M. Wakin, D. Donoho, H. Choi, R. G. Baraniuk. The multiscale structure of non-differentiable image manifolds, Proc. Wavelets XI at SPIE Optics and Photonics, J. Haupt, R. Castro, R. Nowak, G. Fudge, A. Yeh, Compressive sampling for signal classification, Proc. Asilomar Conference on Signals, Systems, and Computers, D. Healy, G. Rohode, Fast global image registration using random projections, for more, see dsp.rice.edu/cs