Novel Quantization Strategies for Linear Prediction with Guarantees

Transcription

1 Novel Novel for Linear Simon Du 1/10 Yichong Xu Yuan Li Aarti Zhang Singh Pulkit

2 Background Motivation: Brain Computer Interface (BCI). Predict whether an individual is trying to move his hand towards left or right based on the brain signals. Energy constraint Number of bits transmitted per second limit. Both training and testing are done under energy constraints. Reduce data transmission through in-sensor computing. 2/10

3 Distributed Sensing Constraints: m i=1 R i R. Questions: How to assign bits? How to design encoders and decoders? 3/10

4 Binary Classification Goal: minimize classification error. min Ĉ Pr (Ĉ (X ) class (X ) ). Classifier: Diagonal Linear Discriminant Analysis (DLDA): Pr (X class (X ) = 0) N ( µ, Σ) and Pr (X class (X ) = 1) N (µ, Σ), Σ diagonal. The optimal Bayes classification rule: w T X 0 Ĉ (X ) = 0, w T X > 0 Ĉ (X ) = 1 where w = Σ 1 µ. Main Result Both theoretically and experimentally showed we can achieve near-optimal classification accuracy limited bits. 4/10

5 Scheme X i q(x i ) 3 0 Uniform quantization: Codebook C = {, b + i, b + 2 i,, i + b, b} where i = 2 /(2 R i 1). q(x i ) = arg min c { X i c, c C}. Only need to decide R i and. 3 5/10

6 Two-Round Training Algorithm Inputs: initial guesses of µ init and Σ init. Round one: Choose Ri init s and bi init s based on µ init and Σ init. Sample n 1 data points. Compute sample mean µ and sample covariance Σ. Round two: Choose R i s and s based on µ and Σ. Sample n 2 data points. Compute sample mean ˆµ and sample covariance Σ. Outputs: ˆµ, Σ and ŵ = Σ 1ˆµ. 6/10

7 Algorithm Input: X, a testing sample to be classified. Choose R i s and s based on ˆµ and Σ. Add a dithering noise γ Unif [ i /2, i /2] before quantization: To avoid correlation of quantization error between sensors. X i X i + γ q(x i ) Classify X to class 0 if ŵ q(x ) 0, to class 1 otherwise where q(x ) is the quantized data vector. 7/10

8 Theoretical Analysis Theorem Assumptions: For i = 1,, m, µ i µ init i, Σ ii Σ init ( ( µ init i log In the first stage Ri init = Ω n 1 = Ω ( log ( )) m δ, In the second stage R i = Ω ( ( log 1 ɛ n 2 = Ω ( 1 log 2 ( ) ( m ɛ 2 ɛ log m )) δ, With probability at least 1 δ, ii. µ i + Σinit ii ( µi Σ ii ) Pr (Ĉ (X ) class (x) opt + ɛ. )) µ i and + Σ ii µ i ))) and where opt denotes the classification error of the best possible classifier. 8/10

9 Experiments on EEG (electroencephalogram) data Task: predict whether an individual is trying to move his hand toward left or right based on EEG data. m = 59 sensors, 160 training data points and 40 testing data points. With just an average of 3 bits per sensor, full (infinite number of bits) quantization accuracy can be achieved Accuracy / Unquantized Active Bits

10 Conclusions Summary: Proposed a two-round feedback-driven learning algorithm and a dithering noise based prediction algorithm for DLDA. Theoretically and experimentally showed that given enough training samples and bits the algorithms can achieve near optimal classification accuracy. Future works: schemes for other (nonlinear) classifiers. schemes for regression problems. 10/10