RECOGNITION ALGORITHM FOR DEVELOPING A BRAIN- COMPUTER INTERFACE USING FUNCTIONAL NEAR INFRARED SPECTROSCOPY

Transcription

1 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) RECOGNITION ALGORITHM FOR DEVELOPING A BRAIN- COMPUTER INTERFACE USING FUNCTIONAL NEAR INFRARED SPECTROSCOPY Cuong 1 and Nguyen Thanh Hai 2, Vo Van Toi 2 1 University of Technical Education HCMC, Vietnam 2 International University, VNU, HCMC, Vietnam ABSTRACT Researches of human Brain Computer Interface (BCI) for the objective of diagnosis and rehabilitation have been recently increased. Cerebral oxygenation and blood flow on particular regions of human brain can be measured using a non-invasive technique fnirs (functional Near Infrared Spectroscopy). In this paper, a study of recognition algorithm will be described for recognition whether one taps his/her left hand or right hand. Data with noises and artifacts collected from a multi-channel system will be pre-processed using a Savitzky- Golay filter for getting more smoothly data. Characteristics of the filtered signals during left and right hand tapping process will be figured out using a polynomial regression algorithm. Coefficients of the polynomial, which correspond to Oxygen-Hemoglobin (Oxy- Hb) concentration, will be applied for the recognition of hand tapping. Then Artificial Neural Networks (ANN) will be employed to validate the obtained coefficient data for hand tapping recognition. Experimental results have been done many trials on a subject to illustrate the effectiveness of the proposed method. KEYWORDS: polynomial regression algorithm, artificial neural networks, hand tapping recognition, functional Near Infrared Spectroscopy. 1. INTRODUCTION In recently decades, a lot of achievements have been obtained in imaging and cognitive neuroscience of human brain. Brain s activities can be shown by a number of different kinds of non-invasive technologies, such as: functional Near- Infrared Spectroscopy (fnirs), Magnetic Resonance Imaging (MRI) and ElectroEncephaloGraphy (EEG). Many researchers have been attracted by these technologies with many approaches to explore problems due to human brain. FNIRS has become the convenient technology for experimental brain purposes. This non-invasive technique uses the method of emitting near infrared light into the human brain to measure cerebral hemodynamics as well as to detect localized blood volume and oxygenation changes. The change of oxy- Hb along task period depending on location of channel on the cortex has been studied: sustained activation in the motor cortex, transient activation during the initial segments in the somatosensory cortex, and accumulating activation in the frontal lobe [1]. Oxy- Hb concentration at the aforementioned sites in the brain can also be used as a predictive factor allows prediction of subjects investment behavior with a considerable degree of precision [2]. Wavelet decomposition can be used as feature extractions and neural networks can be used for cognition brain tasks as a classification module [3]. Also, wavelet can be used to remove artifacts [4]. Base on the slope of straight line, hand side tapping can be distinguished []. Oxy- Hb and DeOxy- Hb can also be used directly with SVM algorithm for the recognition of hand tapping [6]. In this paper, we proposed the recognition algorithm for developing a brain computer interface using fnirs. First of all, 26

2 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) Savitzky- Golay filter is used to reduce noises as well as artifacts. After that, features of Oxy- Hb are found by a polynomial regression based on its coefficients. Finally, Artificial Neural Network is used to determine whether left hand or right hand is tapped. This process is shown in Figure 1 below. hemispheres using the fnirs technique as shown in Figure 3. This system operates at three different wavelengths of 78 nm, 8 nm and 83 nm. Moreover, the distance between pairs of emitter and detector probes was set at 3 cm and all probes were attached with holders arranged on different sides of human brain hemispheres depending on users. A subject (male, 2 years old, 62 kg weights, right-handed) was participated into this study. After reading and understanding the experiment protocol and the fnirs technique, he will start doing hand tapping. The subject was required to perform hand tapping motions, both left and right sides as motor activities. In these hand tapping motions, a protocol includes 2s (Rest) 2s (Task) 2s (Rest), it means that the subject relaxed in 2s, tapped his hand up/down about 1 times in 2s, and then rested 2s, as shown in Figure 4. Figure 1. Recognition algorithm block diagram 2. MATERIALS AND METHODS 2.1 Data Acquisition We used a multichannel fnirs instrument, FOIRE- 3 in order to acquire oxygen hemoglobin. This machine is made by Shimadzu Corporation in Japan and located at Room 14 of Biomedical Engineering Department, International University, Vietnam as shown in Figure 2. Figure 4. Setting of experiment protocol Data were collected from 2 channels. 1 channels are of the left brain side and that of the opposite side will be obtained for hand tapping recognition. Figure a. Probes location (red- emitter, blue- detector) and channels on the motor control area of the left hemisphere Figure b. Probes location and channels (yellow) on the motor control area of the right hemisphere Figure 2. fnirs FOIRE- 3 system Figure 3. Subject head with installed probes Oxy- Hb alters in motor control area of human brain were captured from a set of the holder with 2 channels for both 261 However, we just chose 4 channels of each side which had reliable information to analyze and to estimate features. More clearly, the left brain channels are 2,, 6, 9 and the 12, 1, 16, 19 channels are of the right brain side predicted on the motor

3 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) control area of the hemispheres as in Figure a- b. 2.2 Data Pre-processing Brain data of a subject acquired from the channels have noises and artifacts. In order to obtain brain data with less annoyance, a filter as the Savitzky Golay filter was applied in this paper. Savitzky- Golay filters [7] are also known as polynomial smoothing. The idea of polynomial smoothing is replacing samples of signal by the values that lie on the smoothing curve. In general, we can evaluate a polynomial with order of d to smooth an arbitrary odd length N of data vector x with the condition N d+1. It is replaced by a vector having M points on two sides of x, N= 2M +1: T x = x,, x 1, x, x, ] (1) [ M 1 x M N data samples in x are then fitted by the polynomial with order of d as follow d xˆ m = c + c1m + + cdm, M m M (2) The values are estimated x ˆ m = b T mx, m= - M,,, M (3) In which T T 1 T B = SG = GS = SF S = b, b, b (4) T F = S S G = SF 1 [ ] M, M () With this type of filter: length 11, order of 3, the original and smoothed signals are shown in Figure 6. Sudden rising in original signals causing by noises are smoothed. 262 Figure 6. The signal of channel 2 before and after filtered 2.3 Polynomial regression Polynomial regression [8] (PR) algorithm presents the relationship between amplitude and time of a signal. In this paper, the PR algorithm was applied to analyze brain data corresponding to hand tapping tasks. From the processed data, one can distinguish the difference between the left and right hand tapping times corresponding to the difference of the oxy- Hb concentration. Assumed that we have two-dimensional data, (x 1, y 1 ),, (x n, y n ), where each of x and y has no information about the other. Our problem is fitting a polynomial curve generated by a typical data. Thus, the relationship between x and y can be found out. Based on the coefficients of the regression curve with the order of, one can estimate the hand tapping task. The polynomial regression equation between independent variable x and y fitted can be expressed as follows: y h ˆ h ˆ x h ˆ x 2 h ˆ m ˆ = m x (6) In which, hˆ, hˆ, hˆ,, hˆ 1 2 m are estimated values of h, h 1, h 2,, h m and there are m regressors and n observations, ( xi1, xi2, 2, xim, yi ), i = 1, 2 m 2,, n corresponding to ( xi, xi, 2, xi, yi ). In this equation, the powers of x play the role of different independent variables. The polynomial regression (PR) model can

4 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) be written as a system of linear equations y = Xh + ε (7) Where ε = [ ε ε ] T error. 1 2 ε n is a vector of The ordinary least square ĥ of h is given by the arguments that minimize the residual sum of squares and the distributive law is employed. In the case of the inverse of X ' X exists, it is given by hˆ ' 1 ' = ( X X ) X y (8) From these coefficients, one can determine problems of left hand (LH) tapping or right hand (RH) tapping with given brain data measured using fnirs technology. Figure 7 represents the regressed signal of the channel-2 corresponding to Eq. (9). Similarly, the regression signals of channels, 6, 9, 12, 1, 16 and 19 can be shown. 4 3 =.112x.183x x Amplitude y c 2.247x x.146 Smoothed vs Regressed signal (9) Smoothed signal Regressed signal Time Figure 7. The regression signal of filtered channel Artificial Neural Network Artificial neural networks are the very powerful tools for the problems of classification and pattern recognition. We can use the estimated coefficients from PR algorithm as the features for recognition with a multilayer feed forward network. This network used here consists of an input layer, one hidden layer, and the output layer. Figure 8 shows this architecture. 263 Figure 8. Architecture of classification network Input samples are the features from channel coefficients mentioned above. The number of hidden nodes is carefully chosen. Practically, it can be chosen as an average of number of input nodes and output nodes. With the hidden layer, we used the double sigmoid function while sigmoid function was used for the output layer. Standard back propagation is used for training the above 3- layer network. It is a gradient descent algorithm, in which the network weights are moved along the negative of the gradient of the performance function. With this argument, the training is based on the minimization of the error function: E = N ( o n d n ) n= 1 2, (1) Where N is number of samples, o is network output, d is desired output. With left hand tapping, the output is desired to get the value of [1; ] and [; 1] is the desirable value of right tapping. 3. RESULTS AND DISCUSSION In this work, the PR algorithm with the order- polynomial has six coefficients and its equation is as follows y = h m x m= m (11)

5 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) where x is from -7 seconds with resolution of.7. Amplitude Channel 2 - Channel Channel 6 - Channel Time Channel - Channel Channel 9 - Channel Figure 9. Regressed polynomial of 8 channels of the left hand tapping. Blue curves are the signals on the left brain side and red curves are of the right brain side. This equation was applied to consider for the regressed signals of channels 2,, 6, 9 (left hemisphere) and channels 12, 1, 16, 19 (right hemisphere) and compared right hand tapping with left hand tapping as shown in Figure 9 and 1. Amplitude Channel 2 - Channel Channel 6 - Channel Time Channel - Channel x 1-3 Channel 9 - Channel Figure 1. Regressed polynomial of 8 channels of the right hand tapping. Blue curves are the signals on the left brain side and red curves are of the right brain side. Results in Figure 9 and 1 are not enough to distinguish cases of right tapping or left tapping. Training data, which are coefficients of the regressed polynomials, were applied to identify hand tapping tasks. The coefficients arrangement can be shown in Table 1. Table 1. The regressed coefficients arrangement of hand tapping tasks for obtaining input of recognition networks Left hemisphere coefficients Right hemisphere coefficients Ch-2 Ch- Ch-6 Ch-9 Ch-1 2 Ch-1 Ch-1 6 Ch-1 9 h 21 h 26 h 1 h 61 h 9 h 6 h 66 1 h 96 h 121 h 126 h 11 h 16 h 161 h 166 h 191 h 196 In each times of hand tapping, the regressed coefficients will be obtained corresponding to oxy- Hb changes from two hemispheres for training using the above neural network (NN). Moreover, six coefficients of each channel in Table 1 give us a vector. Therefore, one input of the NN for training is a vector of 48 elements. By choosing the number of hidden nodes is of 1, the performance of the proposed network can be shown in Figure 11. Assume that v r is the vector of the right hand tapping and v l is the vector of the left hand tapping. In one run of experiment, the subject performed 2 times tapping, 1- left tapping, 1- right tapping as shown in Figure 12. So, a set of the left tapping coefficients S l including 1 vectors from v l1 to v l1 and that of the right tapping coefficients S r including 1 vectors from v r1 to v r1. With 8 sample vectors, we used the methodology of 4 runs of 2- fold cross recognition. For identifying left hand tapping, one used 9 vectors of the S l set combined with the 1-vectors of S r set and the remaining vector of the S l set is used to be a sample vector for identification. In the case of identifying right hand tapping, S l, 9 vectors of the S r set and the remaining one is used to identify. 264

6 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) Figure 11. Performance of the network of 48 input nodes, 1 hidden nodes and 2 output nodes. With error of.1, it stop training at epoch 164. Figure 12. One run of experiment consists of 1 times of left tapping, 1 times of right tapping Experimental results are represented in Table 2. Table 2. Results of 4 runs Run Hand tapping side Accuracy (%) 1 Right 8 Left 7 2 Right 9 Left 8 3 Right 9 Left 7 4 Right 8 Left 7 Average Right 8 Left 72. The accuracy of hand tapping tasks is more than or equal to 7%. In this work, the accuracy in the case of right tapping is always greater than that of left hand tapping. We got the average accuracy of 8% in the case of right hand tapping, and just 72.% in left tapping. But with these values, we can say that the proposed algorithm used to recognize right hand or left hand tapping is reliable. 4. CONCLUSION 26 In this paper, original brain signals of hand tapping tasks were filtered by the Savitzky Golay filter to produce the smooth signals. Moreover, the smoothed signals of the left and right hand tapping tasks corresponding to oxy-hb changes in human brain were analyzed using the PR algorithm when the LH or the RH moves up/down. For the more accurately recognition of the hand tapping times, the ANN was employed to validate data. Experimental results with hand tapping times showed that one could distinguish when the subject was doing the left or right hand tapping tasks based on different coefficients of the curves. Our future work will be a problem of improving the accuracy of the proposed algorithm and doing it on many subjects.. ACKNOWLEDGMENTS The authors would like to acknowledge the support of the Shimadzu through the Shimadzu-IU Research Collaborative Program. We would also like to thank the students of Biomedical Engineering Department at International University for their supports during data collection. 6. REFERENCES [1] C. H. Rodolphe J. Gentili, Hasan Ayaz, Patricia A. Shewokis and José L. Contreras-Vidal, "Hemodynamic Correlates of Visuomotor Motor Adaptation by Functional Near Infrared Spectroscopy," presented at the 32nd Annual International Conference of the IEEE EMBS, 21, pp [2] K. S. T. SHIMOKAWA, T. MISAWA, K. MIYAGAWA, "Predictability of investment behavior from brain information measured by functional near-infrared spectroscopy: a bayesian neural network model," Neuroscinece research, 29, pp [3] Truong Quang Dang Khoa, M. N., "Functional Near Infrared Spectroscope for Cognition Brain Tasks by Wavelets Analysis and Neural Networks,"

7 The 212 International Conference on Green Technology and Sustainable Development (GTSD212) International journal of Biological and Life science, 28, pp [4] S. M. Behnam Molavi, Guy A. Dumont, Fellow, "Wavelet Based Motion Artifact Removal for Functional Near Infrared Spectroscopy," presented at the 32nd Annual International Conference of the IEEE EMBS, 21, pp. -8. [] C. Q. Ngo, T. H. N,T. V. Vo, "Linear Regression Algorithm for Hand Tapping Recognition Using Functional Near Infrared Spectroscopy," presented at the Fourth International Conference on the Development of Biomedical Engineering, VietNam, 212, pp [6] H. Z. Ranganatha Sitaram, Cuntai Guan,Manoj Thulasidas,Yoko Hoshi,Akihiro Ishikawa,Koji Shimizu,Niels Birbaumer,, "Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain computer interface," NeuroImage 34, 27, pp [7] S. J. Orfanidis, "Signal Processing Applications," in Introduction to signal processing, ed: Pearson Education Inc, 21, pp [8] G. C. R. Douglas C. Montgomery, "Applied Statistics and Probability for Engineers" in Applied Statistics and Probability for Engineers, ed: John Sons Inc, 23, pp Contact : Ngo Quoc Cuong Cellphone : (84) Employee : Lecturer ngoquoccuong17@gmail.com; cuongnq@hcmute.edu.vn 266