Analog-to-Information Conversion

Transcription

1 Analog-to-Information Conversion Sergiy A. Vorobyov Dept. Signal Processing and Acoustics, Aalto University February 2013 Winter School on Compressed Sensing, Ruka 1/55

2 Outline 1 Compressed Sampling (CS) motivations and basics 2 AIC Approaches and Xampling 3 Example of frequency sparce signal recontruction 4 Analog domain CS models 5 The segmented CS method 6 Analytical results 7 Conclusion 2/55

3 CS motivation Nyquist rate sampling. Signal recovery from fewer samples? CS: sensing and compressing simultaneously Sparsity / compressibility 1 1 Picture from Compressive sensing, by Richard Baraniuk 3/55

4 The sensing problem Interested in undersampled situations! 2 Possibility of accurate recovery? Design of sensing waveforms? Ill-posed recovery problem y = Φx R K. 2 Picture from Tutorial on compressive sensing, by Richard Baraniuk, Justin Romberg and Michael Wakin 4/55

5 The sensing problem But our signal of interest is sparse. 5/55

6 The sensing problem We can think of Φ being K S. But we do not know the location of the nonzero coefficients. 6/55

7 The sensing problem Design Φ so its K 2S submatrices are full rank. Two S-sparse signals x 1 and x 2 will be mapped to different measurements. 7/55

8 Restricted Isometry Property (RIP) S-restricted isometry constant δ S, the smallest number satisfying K N (1 δ S) c 2 l 2 Φ T c 2 l 2 K N (1 + δ S) c 2 l 2 T : index set with cardinality S. To be able to recover S-sparse signals, δ 2S < 1. Forming columns of Φ by taking i.i.d. samples of a zero mean normal distribution with variance 1/N The RIP holds if K CS log (N/S) 8/55

9 The discrete CS problem Sampling procedure: y = Φf + w = ΦΨ T x + w. f: TheN 1 discrete-time sparse signal. Ψ: TheN N sparsity basis. x: The sparse representation of f in Ψ. Φ: The K N measurement matrix. w: TheK 1 noise vector. y: TheK 1 vector of compressed samples. Gaussian random measurement matrices Φ are universal. 9/55

10 Recovery The noise less case with Ψ = I Sampling y k =< f, φ k >, k = {1, 2,...,K} Select f with the sparsest representation x which leads to the same y k l 0 -norm minimization min x l0 subject to < φ k, Ψ T x >= y k k. The problem is NP-complete 10 / 55

11 Recovery... Recovery by l 2 -norm minimization min x l2 subject to < φ k, Ψ T x >= y k k. 11 / 55

12 Least square recovery... It does not result in a sparse solution. 12 / 55

13 Recovery... Recovery by l 1 -norm minimization min x l1 subject to < φ k, Ψ T x >= y k k. A convex optimization problem. Can be solved by a linear program of O(N 3 ). 13 / 55

14 Recovery... The noisy case min x l1 subject to Φ x y l2 γ. Φ = ΦΨ T. γ : Bound on the noise vector s energy. The solution x to the above problem obeys x x l2 C 1,S γ + C 2,S x x S l1 / S Other recovery algorithms Convex optimization: Dantzig selector,... Iterative greedy algorithms: Matching pursuit (MP), OMP, / 55

15 Recovery... Empirical risk minimization based recovery { ˆf K = arg min ˆf F(B) ˆr(ˆf)+ c(ˆf)log2 ɛk }. (1) F(B) {f : f 2 NB 2 },andɛ =1/ ( 50(B + σ) 2). c(ˆf): A nonnegative number assigned to a candidate signal ˆf. ˆr(ˆf) 1 ( K 2: K j=1 y j φ jˆf) The empirical risk. 15 / 55

16 Recovery... ˆfK in (1) satisfies { } ˆf K f 2 E C 1 min N ˆf F(B) { ˆf f 2 For a sparse signal f F s (B, S) wehave { } ˆf K f 2 sup E C 1 C 2 f F s (B,S) N F s (B, S) {f : f 2 NB 2, f l0 S}. N } + c(ˆf)log2+4. ɛk ( ) K 1. (2) S log N 16 / 55

17 Applications... Compressive radar. Spectrum sensing in cognitive radios. Sparse channel estimation for communication systems. Analog to information converters to replace ADCs. 17 / 55

18 AIC In most of the CS literature The signal is sampled with Nyquist rate Then, the measurement matrix is applied We can avoid high speed sampling Xampling extension of Shannon-Nyquist theorem to any subspace model if signal lives in a certain subspace, it can be sampled and recovered using simple filtering operation and a digital correction Non-uniform sampler (NUM) Random Modulation Pre-Integration (RMPI) A system of parallel mixers and integrators (BMIs) Sampling with a random sequence of ±1 Easier to change polarity at a high rate rather than digitize The RMPI is less complex. 18 / 55

19 Xampling Motivation There is more structure in some signals than just living in a subspace Multi-band spectrum Multi-band spectrum: unknown carrier frequencies. 19 / 55

20 Xampling Motivation Multi-path (small number of pulses from different targets), TOAs are not known Estimate distances to the targets and Dopplers (velocities) Multi-path in radar or multi-path fading in communications. 20 / 55

21 Xampling Motivation Have to sample at the Nyquist rate of the pulse However, pulse is known, the unknowns are the TOAs, i.e., just few unknowns Model for such signals is Union of Subspaces (US) Normally we look at the sum of subspaces Known: the signal lives in a low dimensional subspace Unknown: which of the subspaces from the union of many possible subspaces 21 / 55

22 US Subspace versus union of subspaces. 3 In multi-band case: we could sample at low rate, but the carriers are unknown - so, normally we use Nyquist rate sampling 3 Picture from Xampling web-site, Eldar and Michaeli 22 / 55

23 Xampling Idea If pulses are sparse and we sample with low rate, the probability to sample zero is high Solution: alias the date before sampling - pre-processing in analog domain In the simplest case, use low-pass filter, but optimally multiply by sinusoids and then low pass Then each sample will contain information about the signal Pulse location from the samples of mixed data can be done in digital domain - recovery algorithms 23 / 55

24 Xampler Block Scheme Xampler block scheme. 4 In analog domain: sample so that the data contains combinations from all of the possible subspaces: alias with sinusoids or just one periodic signal, which already contains many sinusoids In digital domain: 1) Detection to identify the subspaces involved - Use classical array processing methods or compressed sensing methods 2) Reconstruct, which is a solved problem as soon as you know the subspace 4 Picture from Xampling web-site, Eldar and Michaeli 24 / 55

25 Model Data model x n = K s k e jω kn k=1 s k and ω k (1 k K) are unknown amplitude and frequency of the kth sinusoid, respectively In matrix-vector form x = As x =[x 0, x 1,..., x N 1 ] T C N 1 is a linear combination of K sinusoids, K N s =[s 1, s 2,..., s K ] T C K 1 A =[a(ω 1 ), a(ω 2 ),..., a(ω K )] C N K 25 / 55

26 Vector of measurements Measurements y = Φx + w Φ R M N is the measurement matrix w C M 1 is the measurement noise with circularly symmetric complex normal distribution N C (0,σ 2 I ) Elements of Φ are drawn independently from the Gaussian distribution N (0, 1/M) Idea: 1) minimize the estimation error, i.e., ˆx = arg min x y Φx 2 2 2) match the estimated signal to the sparsity model Iterate 26 / 55

27 Nested Least Squares Algorithm 1 Initialize: ˆx 0 =0, i =1 repeat x e ˆx i 1 + λφ T (y Φˆx i 1 ) ˆΩ root-music(x e, K) Â [a(ˆω 1 ) a(ˆω 2 )... a(ˆω K )] ˆB ΦÂ 5 ŝ i (ˆB H ˆB) 1 ˆB H y ˆx i Âŝ i i i +1 until halting criterion true Alternative is Spectral Iterative Hard Thresholding (SIHT) by Duarte and Baraniuk 5 From ICASSP 11 paper, Shaghaghi and Vorobyov 27 / 55

28 Simulation results snr λ=1.0 λ=0.9 λ=0.8 λ=0.7 λ=0.6 λ=0.5 λ=0.4 SIHT via root MUSIC NCRB itr The AIC with parallel branches of mixers and integrators. 6 K = 20, N = 1024, ( M = 300, ) the noise standard deviation ) = 2, E{ x ˆx 2 NMSE =10log 2 } E{ x 2 2 },andncrb =10log( CRB E{ x 2 2 } 6 From ICASSP 11 paper, Shaghaghi and Vorobyov 28 / 55

29 Analog signal model Analog signal representation f (t) = N x n ψ n (t) =x T Ψ(t). n=1 Ψ(t) (ψ 1 (t),...,ψ N (t)) T : The sparsity basis. Basis functions {ψ n (t)} N n=1 defined over period t [0, T ]. Sample collection y k = T 0 f (t)φ k (t) dt, k =1,...,K. Φ(t) (φ 1 (t),...,φ K (t)) T : The measurement operator. φ k (t): Random ±1 chip sequence with period T c = T /N c, N c N. 29 / 55

30 Representing as a discrete model) Discrete CS counterpart y = Φ x + w. Φ =Φ(t)Ψ(t): K N matrix with its (k, n)-th entry given as [Φ ] k,n = T 0 ψ n (t)φ k (t) dt. The entries of the K N c measurement matrix Φ: [Φ] k,n = ntc (n 1)T c φ k (t) dt The entris of the N N c sparsity basis Ψ: [Ψ] m,n = ntc (n 1)T c ψ m (t) dt 30 / 55

31 RMPI structure for AIC The RMPI structure f (t) T 0 φ 1 (t) y 1 T 0 y 2 φ 2 (t) T 0 y K φ K (t) The AIC with parallel branches of mixers and integrators. 31 / 55

32 Segmented CS 7 Measurement matrix Φ with K rows Spliting the integration period into M sub-periods M sub-samples instead of one sample A K M matrix of sub-samples Y = y 1,1 y 1,2... y 1,M y 2,1 y 2,2... y 2,M.... y K,1 y K,2... y K,M. Adding up the rows result in K samples Adding M sub-samples of main diagonal: K +1-stsample Sub-samples of second diagonal K +2-ndsample... Wind up with K e = K + K a samples 7 Taheri and Vorobyov, Segmented CS for AIC, IEEE TSP, Feb / 55

33 Segmented CS... Sub-sample selection diagram (K a = K) y K+1 y K+2 y 1,1 y 1,2 y 1,3 y 1,M y 2,1 y 2,2 y 2,3 y 2,M y 3,1 y 3,2 y 3,3 y 3,M y 2K 1 y 2K y K 1,1 y K 1,2 y K 1,3 y K 1,M y K,1 y K,2 y K,3 y K,M The previously described sub-sample selection method. 33 / 55

34 The extended measurement matrix Sampling procedure can be explained in terms of the extended measurement matrix Φ e of size K e N φ 1,1 φ 1,2... φ 1,M ( ).... Φ Φ e = = φ K,1 φ K,2... φ K,M Φ 1 φ 1,1 φ 2,2... φ M,M..... φ Ka,1 φ π2 (K a),m... φ πm (K a),m φ k,j, j =1,...,M are some vectors whose concatenation results in φ k = ( ) φ k,1,...,φ k,m. π s (k) =((s + k 2) mod K)+1, s, k =1,...,K are permutations applied to different columns of Φ. Is this a valid CS matrix? 34 / 55

35 Sketch of the proof Lemma For T {1, 2,...,N} of cardinality S, (Φ e ) T overwhelming probability if satisfies RIP with min{k, K a + M 1} (K + K a ) /2 We partition the rows of (Φ e ) T Then, we use existing results. into two independent subsets. 35 / 55

36 Sketch of the proof... Φ 1 uses the first K a + M 1rowsofΦ or all of it If min{k, K a + M 1} (K + K a ) /2 (Φ e ) T can be partitioned into two sets of size K + K a 2 and K + K a 2 These sub-sets have independent entries. If both subsets satisfy RIP, so does (Φ e ) T with probability 1 4(12/δ S ) S e C 0 K +Ka 2 36 / 55

37 Sketch of the proof... Union bounding on all different subsets (Φ e ) T We get the main result which is Pr{Φ e satisfies RIP} 1 4e C 4 (K+K a)/2 for S C 3 (K + K a )/2 / log(n/s). 37 / 55

38 Analytical results for empirical risk minimization Theorem Let ɛ be chosen as ɛ = 1 (60 (B + σ) 2). Then the signal reconstruction ˆf Ke ˆfKe = arg min ˆf F(B) given by { ˆr(ˆf)+ c(ˆf)log2 ɛk e } (3) satisfies the following inequality { } ˆf Ke f 2 E C 1e N min ˆf F(B) { ˆf f 2 N } + c(ˆf)log2+4 ɛk e 38 / 55

39 Analytical results... Theorem For a sparse signal f F s (B, S) and corresponding reconstructed signal ˆfKe obtained according to (3), there exists a constant C 2e = C 2e (B,σ) > 0, such that sup E f F s(b,s) { } ˆf Ke f 2 C 1e C 2e N ( ) 1 Ke. (4) S log N 39 / 55

40 Example Two sets of samples are collected (i) With a K N measurement matrix with all i.i.d. (Bernoulli) elements. (ii) With a K e N extended measurement matrix. Parameters are chosen as: K a = K, M =8 K e =2K. Assuming the same ɛ in (2) and (4) C 2e = C 2. Performance comparison comes down to C 1e and 2C C 1e 2C / 55

41 Simulation results Three sampling schemes are compared Original K N measurement matrix Extended K e N measurement matrix Enlarged K e N matrix with i.i.d. entries Bernoulli and Gaussian measurement matrices are considered. Different sparsity bases are considered. Empirical risk minimization and l 1 -norm minimization are used. Results in terms of MSE vs. K a /K MSE is averaged over 1000 simulation runs. 41 / 55

42 Scenario 1 Time sparse signal with l 1 -norm minimization N = 128 basis functions ψ n (t) = { N T, t [(n 1)T /N, nt /N] 0, otherwise, n =1,...,N. Sparsity level: 3 Original measurements: K =16 Additional measurements: K a =0to5K Number of Segments: M =8 42 / 55

43 Scenario 1 (Gaussian measurement matrix) 10 1 SNR = 5 db SNR = 15 db SNR = 25 db Original Gaussian Matrix Extended Matrix Enlarged Matrix 10 2 MSE K a /K 43 / 55

44 Scenario 1 (Bernoulli measurement matrix) 10 1 SNR = 5 db SNR = 15 db SNR = 25 db Original Bernoulli Matrix Extended Matrix Enlarged Matrix 10 2 MSE K a /K 44 / 55

45 Scenario 2 Same time sparse signal The number of original samples: K =24 Empirical risk minimization based recovery 45 / 55

46 Scenario 2 (Gaussian measurement matrix) 10 1 SNR = 5 db SNR = 15 db SNR = 25 db Original Gaussian Matrix Extended Matrix Enlarged Matrix 10 2 MSE K a /K 46 / 55

47 Scenario 2 (Bernoulli measurement matrix) 10 1 SNR = 5 db SNR = 15 db SNR = 25 db Original Bernoulli Matrix Extended Matrix Enlarged Matrix 10 2 MSE K a /K 47 / 55

48 Scenario 3 OFDM signal with l 1 -norm minimization QPSK modulation N = 128 basis functions ( ψ n (t) =cos (n 1) 2π ) T t t [0, T ], n =1,...,N. Sparsity level: 3 The number of chips per duration (N c = 256) Original measurements: K =16 Additional measurements: K a =0to5K Number of Segments: M =8 48 / 55

49 Scenario 3 (Gaussian measurement matrix) 10 1 SNR = 5 db SNR = 15 db SNR = 25 db Original Gaussian Matrix Extended Matrix Enlarged Matrix 10 2 MSE K a /K 49 / 55

50 Scenario 3 (Bernoulli measurement matrix) 10 1 SNR = 5 db SNR = 15 db SNR = 25 db Original Bernoulli Matrix Extended Matrix Enlarged Matrix 10 2 MSE K a /K 50 / 55

51 Scenario 4 A case where the number of original BMIs is insufficient. Time sparse signals as in scenario 1. N = 128, S =5,K =9,andM =8. Performance merit Percentage of successful support recovery The average MSE for cases of successful support recovery 51 / 55

52 Scenario 4... Table: Percentage that the positions of the nonzero signal values are correctly identified. Noiseless SNR=25 db K a K e BMIs % MSE N/A Segmented % MSE N/A K e BMIs Segmented % MSE N/A % MSE N/A / 55

53 Conclusion Three bacis approaches to AIC Xampling also known as modulated wideband converter (MWC): may be energy consuming - needs to multiply the signal by a number of sinusoids in analog domain NUS (Wakin, Romberg, Candes, et. al.): didn t talk about it, but non-uniform sampling otherwise has a very long history RMPI also known as random demodulator (RD): some implementation issues are more involved than for MWC Extentions of RD: Polyphase RD (Laska, Slavinsky, Baraniuk) 53 / 55

54 Conclusion The segmented CS method Reusing the samples already obtained Results in improvement in recovered signal quality without added complexity to the sampler Extentions/improvenets: Partial Segmented CS, Random Circulant Orthogonal Matrix based Analog Compressed Sensing 54 / 55

55 55 / 55