1. Introduction
Brain-computer interface (BCI) is a modern technology that helps people to control external machines or devices without using any muscles or peripheral nerves [
1]. It has potential applications in neuroscience and neuro-engineering. The recent application of BCI offers neurorehabilitation to assist stroke patients in restoring their impaired motor functions [
2]. The usage of prosthetics, robots, and other electronic devices in neurorehabilitation tasks is fully controlled by motor imagination [
3]. The non-invasive electroencephalography (EEG) is a comfortable and relatively easy method for BCI implementation. The BCI user's brain activity is typically measured by EEG [
4]. In general, the analysis of EEG signal has been the subject of several studies, because of its ability to yield an objective mode of recording brain stimulation which is widely used in brain-computer interface researches with application in medical diagnosis and rehabilitation engineering.
Motor imagery (MI) is a common mental task that is widely used in BCI implementation [
5]. In MI based BCI, a subject is required to perform an imagination in the brain corresponding to specific task. It provides high degree of freedom and it helps motor disabled people to communicate with the device by performing sequence of MI tasks. The recorded EEG signals related to MI are classified to translate into corresponding control command for different imaginary tasks like movement of hand, foot etc. [
6]. In terms of neurophysiology, motor imagery accompanies attenuation or enhancement of rhythmical synchrony over the sensorimotor cortex. MI has been used to encourage neuroplasticity in a patient's brain after a stroke [
3]. This paper focuses on EEG based classification of two motor imagery tasks.
Feature extraction over EEG signals for MI based BCI system is crucial to the classification performance. It is an important step in the process of electroencephalogram (EEG) signal classification. Any feature represents a distinguishing property, a recognizable measurement, and a functional component obtained from a section of a pattern. Extracted features are meant to minimize the loss of important information embedded in the signal. In addition, they also simplify the amount of resources needed to describe a huge set of data accurately. It is necessary to minimize the complexity of implementation, to reduce the computational cost, and to cancel the potential need to compress the information. Recently, a variety of methods have been widely used to extract the features from EEG signals, among these methods are fast Fourier transform (FFT) [
7], time frequency distributions (TFD) [
8], eigenvector methods (EM) [
9], wavelet transform (WT) [
10] and auto regressive method (ARM) [
11].
The multivariate EEG signal is collected using a set of sensors spatially distributed over the scalp. The spatial filtering approach is very much effective to extract features from EEG. Spatial features were extracted using Common Spatial Pattern (CSP) filters in the cortical source space. The CSP algorithm has been widely used for feature extraction in EEG-based BCI systems for motor imagery (MI) [
12]. As the EEG signals have noise and over-the-fitting issues, various regularized CSP algorithms are introduced to cater to these issues [
13]. It is a feature extraction method that uses spatial filters to maximize the discriminability of two classes.
A number of methods including filter bank CSP (FBCSP) [
14,
15], subband CSP [
16], sparse filter-bank CSP [
17], and discriminative filter bank CSP [
18] have been proposed to extract the features from the narrowband EEG signals for MI classification. The sparse representation of CSP feature is implemented in [
19] for two classes of MI discrimination. These works promote the implementation of subband CSP to obtain the discriminative features, thereby yielding a reliable classification of MI tasks. Different subband decomposition approaches are already implemented including empirical mode decomposition (EMD) [
20]. Although EMD is fully data adaptive approach, it requires a high computational cost. To resolve the problem of high computational cost, discrete wavelet transform (DWT) is widely used for signal decomposition into a finite set of subbands [
21]. The multivariate wavelet transform (mWT) is introduced in [
22] to decompose multichannel EEG signal followed by feature extraction. The wavelet transformation decomposes the signal with the nature of dyadic filterbank. It is difficult to obtain a subband with desired cut-off frequencies. The use of Fourier transform based bandpass filter resolves such problem.
The features obtained from narrowband EEG signals become more discriminative for classification [
23]. The performance of MI-BCI significantly depends on the selection of effective frequency bands of the EEG signal from which the features are extracted [
24]. To determine the changes of motor imagery task in narrowband signals, it is required to include the subband signals in the trials to extract the spatial features. Therefore, the subband approach with CSP method is implemented in this work to extract the effective features from the narrowband EEG. In addition to the narrowband signals, the wideband signal can contain some apparent features to enhance MI classification. Without considering this issue, the narrowband signals are used for feature extraction in the subband CSP-based methods.
The existing methods extract features from individual subband and the features obtained from all the subbands are combined. In that case, the co-variation of different subbands is not considered during feature extraction. It is noted that different frequency components are not independent to represent any motor imagery task and hence it is required to extract features by considering the co-variation of different narrowband signals. In this paper, multivariate EEG is decomposed into a finite set of subbands using Fourier transform based bandpass filter. The obtained narrowband signals are arranged to extend the size of trial. The CSP features are extracted from the extended trial. It includes the narrowband signals as individual row in the trial. Thus, all the narrowband signals as well as the fullband signal are considered together to derive the features used in effective MI classification system. The discrimination of MI tasks is performed by using artificial neural network.
The rest of the paper is organized as follows. Section II provides the description of data used in this study. The methodology is described in Section III. The experimental results are presented in Section IV and discussion is illustrated in section V. Finally, Section VI draws a conclusion about this study.
2. Data Description
The Publicly available two benchmark datasets are used to evaluate the performance of the proposed method. The datasets are described in the following subsections.
Dataset I: BCI Competition III dataset 4a is well known dataset to evaluate the performance of motor imagery classification. The data were recorded from five healthy subjects of age group 24 – 25 years, denoted as ‘aa’, ‘al’, ‘av’, ‘aw’, and ‘ay’ [
25]. The subjects are properly instructed in priory about the experimental condition. The EEG data are recorded while they sat on a comfortable chair and their eye movements were avoided. The visual stimulus was presented for 3.5s and during that time duration, the participant was asked to perform three motor imagery tasks, i.e., right hand, left hand and right foot movement. Two motor imagery tasks of right hand and foot were taken into consideration for classification. Total 280 trials of EEG with 118 channels were recorded for each subject while they performed the motor imagery tasks according to the instruction. The trials were divided into training and testing groups in the dataset. The labeled training data are used in this study to evaluate the performance of the proposed method. The recorded signals are filtered using a band-pass filter between 0.05-200 Hz and sampled at 1000 Hz and quantized by 16 bit resolution. The EEG signal is downsampled at 100 Hz for further processing. The detail about the experimental setup is provided in the study [
25]. In this study, the EEG trial of 2 s length (0.5 – 2.5 s) is extracted to obtain the meaningful feature to be used classification. It is consider that the first 0.5s (0–0.5 s) and last 0.5 s (3.5 – 4.0 s) are the duration for pre and post imagination.
Each trial of EEG recorded with 118 channels includes some irrelevant signal because all the channels are not required to discriminate two motor imagery tasks. The selection of relevant and minimum number of channels will be effective in terms of computational cost. The relevant zone of motor activity is motor cortex region including primary, supplementary and premotor cortex area [
26]. It is required to select the electrodes placed at these areas. Several studies are performed to use a selected number of channels to design the MI-BCI. The 18 channels from the area of sensorimotor cortex are used in, while 30 channels are selected in [
27,
28] to classify two motor imagery tasks. Considering the previous studies [
27,
28], 30 channels in sensory-motor cortex area are selected for MI tasks classification as indicated in
Figure 1. Throughout this paper, the multichannel EEG refers to the signals recorded from the selected 30 electrodes. These are “C5, C3, C1, C2, C4, C6, CP5, CP3, CP1, CP2, CP4, CP6, P5, P3, P1, P2, P4, and P6”.
Dataset II: Publicly available EEG data obtained from BCI Competition IV dataset I [
29] is used to evaluate the performance of the proposed motor imagery classification method. The EEG data was collected from seven healthy subjects (labeled as ‘a’, ‘b’, ‘c’, ‘d’ ‘e’, ‘f’, and ‘g’). The motor imagery was performed without feedback at the time of recording. Each subject performed two motor imagery tasks among three of left hand movement, right hand movement, and foot movement. Every different MI task is treated as a different class. Visual cues were displayed for the duration of 4s during and each subject was required to perform the predefined motor imagery task. The number of total trials is 200 and each trail was randomized so that all subjects imagined two tasks equally. Trails were interleaved with 2s of a blank screen and 2s with a fixation cross shown in the center of the screen with superimposed on the cues. All EEG signal was recorded from 59 electrodes according to the international 10-20 system using BrainAmp MR plus amplifiers and an Ag/AgCl electrode cap. Signals were band-pass filtered between 0.05 and 200Hz and then digitized at 1000Hz with 16-bit resolution. The data used in this study is down-sampled at 100Hz.
It is well known that the center part of the brain is mostly active during the motor imagery. We have selected 23 channels out of 59 to be used in this study. The selected channels are: “FC5”, “FC3”, “FC1”, “FC
Z”, “FC2”, “FC4”, “FC6”, “C
Z”, “C3”, “C4”, “C1”, “C2”, “C5”, “C6”, “T7”, “T8”, “CCP3”, “CCP4z”, “CP5”, “CP1”, “CP
Z”, “CP2”, “CP6” according to the 10-20 system [
30]. It is also known that the time segment selection of motor imagery is a critical issue for EEG classification [
24]. The EEG of 3.5s (0.5s to 4s) segment is used here. A zero phase Butterworth bandpass (8 – 30Hz) filter is applied to all the trial before any further processing.
4. Experimental Results
Publicly available two datasets denoted by Dataset I and Dataset II are used to evaluate the performance of the proposed method. The BCI competition III (IVa) is referred here as Dataset I and BCI competition IV (I) is referred as Dataset II containing five and seven subjects respectively. Different experiments are conducted with these two datasets to illustrate the efficiency of the proposed approach for MI classification. Binary classification is considered in this study to discriminate two motor imagery tasks. After pre-processing, each channel of any EEG trial is decomposed into four subbands signals which are the four rhythmic components alpha, low beta, high beta and gamma. The components reflect different motor activities and hence have a vital role in MI classification. The obtained subbands i.e. rhythmic components are arranged in spatial dimension to implement the proposed trial extension method. The four components obtained from a channel are appended as individual row after the channel itself. Then the dimension of any trial becomes 5C×N, where C is the number of channels and N is the number of temporal samples.
The CSP is applied to the regenerated (extended) EEG trials to extract the spatial features. Thus computed features are applied to train the classifier leading to evaluate the performance of the proposed method by using artificial neural network (ANN). The
k-fold (
k=5) cross-validation is used to measure the algorithm’s performance in term of classification accuracy. For each subject, the dataset is divided randomly into
k equal groups. The (
k-1) groups are assigned for training and the rest one is for testing. The process is repetitive for
k times. The accuracy is calculated by averaging the results of
k repetitions. The performance is evaluated by classification accuracy
Acc=100×(
TC/TN) where
TN and
TC is the number of trails in test dataset and number of trials correctly recognized out of
TN respectively. The performance of the proposed method including trial extension (iTE) with FfNN is evaluated with Dataset I. In addition to iTE the performance evaluation is also carried out excluding trial extension (eTE) approach. Each of the EEG channel is bandpass filtered using the frequency range 7Hz – 35Hz in eTE. It is implemented by applying CSP on the original trial (without extension) to obtain the spatial features leading to apply FNN for MI classification. The two pairs of CSP features and five neurons are assigned to the hidden layer of FNN are used for both the methods. The performances of iTE and eTE are illustrated in
Figure 4.
It is observed that the average classification performance is enhanced by 2.52% with iTE compared to eTE method for Dataset I. The performances of iTE for individual subjects are relatively higher than that of eTE. The standard deviation of the MI classification accuracy with iTE is relatively lower than that with eTE. The similar results are presented in
Figure 5 with Dataset II. It is observed that there is an improvement of MI classification performance using iTE compared to eTE, whereas the standard deviation with iTE is lower than with eTE.
The selection of the number of pairs of CSP filters is a vital factor for MI classification. On the other hand the use of the number of neurons in hidden layer is also a crucial factor to fixup the optimal performance of FNN. Both the factors are determined experimentally to maximize the classification accuracy. The scalp EEG is very much subject-sensitive tool to measure the neural response against motor activities. Hence, the number of pairs of CSP features and the number of neurons in hidden layer are selected through grid search approach for a specific subject. The features’ pairs and the number of neurons are gradually increased to observe the performance of FNN. The maximum accuracy is spotted in the grid as illustrated in
Figure 6. The numbers of neurons as well as the features’ pairs corresponding to the maximum accuracy are selected as usable factors to train the FNN for the specific subject. The similar method is repeated for each subject of both datasets to obtain the maximal performance of the proposed algorithm. Thus the parameters are tuned to maximize the performance of MI classification with iTE and it is termed as iTE-tP (iTE with tuned parameters). In case of subject ‘aa’ of Dataset I, the maximum classification accuracy is obtained with 3 pairs of CSP features and 6 neurons used in hidden layers of FNN as illustrated in
Figure 6.
The performance of iTE-tP is compared with iTE as well as eTE methods for Dataset I as illustrated in
Table 1.
It is observed that the proposed method iTE-tP outperforms the other two approaches. The average MI classification accuracy over five subjects with iTE-tP is 93.88% which is higher than both iTE (90.12%) and eTE (87.64%). The performance of the proposed iTE-tP method is enhanced by 3.76% and 5.24% compared to iTE and eTE methods respectively. There is a significant improvement of MI classification performance with respect to iTE and eTE. The statistical significant analysis is also performed to illustrate the superiority of the proposed method iTE-tP. Statistical test is performed to measure the significance of the proposed method. Tukey-Kramer based post-hoc test [
38] suggests that the proposed iTE-tP method achieved significantly higher accuracy across 5 subjects for motor imagery classification than the other methods (iTE-tP vs iTE: p<0.05, iTE-tP vs eTE: p<0.002)
The performances of iTE-tP, iTE as well as eTE based methods with Dataset II are illustrated in
Table 2.
The proposed tuned parameter based method (iTE-tP) exhibits maximum classification accuracy among the mentioned three methods. Its accuracy is enhanced by about 1.12% and 4.58% with respect to iTE and eTE methods respectively. The 100% MI classification accuracy is achieved using iTE-tP method for the subject ‘e’. Although the maximum classification accuracy for subject ‘b’ is achieved by iTE the average MI classification accuracy (91.55%) is accomplished by the proposed iTE-tP. Tukey-Kramer based post-hoc test [
38] suggests that the proposed iTE-tP method achieved significantly higher accuracy across 5 subjects for motor imagery classification than the other methods (iTE-tP vs iTE: p>0.06, iTE-tP vs eTE: p<0.002).
5. Discussion
Publicly available two datasets (Dataset I and Dataset II) are used to evaluate the performance of the method introduced in this study. The Dataset I i.e. BCI Competition III (IVa) is used in several recently reported methods to evaluate the MI classification performance [
39,
40,
41,
42]. The comparative performances in terms of classification accuracy of the proposed method with Dataset I are illustrated in
Table 5. The average classification accuracy of the proposed approach iTE-tP over all subjects is 93.88%. The performance of this method is compared with the methods implemented using regularized Riemannian features (RRF) [
43] and sparse group representation model (SGRM) of the CSP features [
19]. The average classification accuracies of RRF and SGRM with dataset III (4a) are 87.21% and 77.70%, respectively. It is noted that the Riemannian manifold-based feature is used in regularized Riemannian features (RRF) [
43] method rather than CSP. The attractor metagene-based feature selection is used in [
19] with proper parameter optimization of SVM (AM-SVM) to implement the MI classification for BCI application with an average accuracy of 85.00%. There is an improvement in classification accuracy (92.20%) using neighborhood component analysis based feature selection (NCFS) [
24]. It is observed that SSCSP method [
40] uses sparse CSP to obtain an accuracy of 73.36%. The spatial regularization of CSP is implemented in SRCSP [
38] with a classification accuracy of 76.37% using BCI Competition III (4a) dataset. The transfer kernel common spatial pattern (TKCSP) is introduced by Dai et al. [
42]. The proposed method outperforms TKCSP by 13.44% in terms of classification accuracy. The accuracy with unsupervised discriminative feature selection (UDFS) based method [
34] is 89.86%. The average performance of iTE-tP across all subjects in Dataset I is 93.88% which is at least 1.68% higher than that of all the recently reported works illustrated in
Table 3.
Friedman’s ANOVA is performed to study the significance level. The test is performed to detect the differences in the performances of the various methods. According to the result of Friedman’s ANOVA, the methods have a significant main effect on classification accuracy (p<0.05). To test the statistical significance of the methods mentioned in
Table 3, the Tukey–Kramer-based post-hoc test is performed. Based on the results of this statistical test, the proposed method iTE-tP achieves more significant improvement of performance for MI-BCI over the subjects than other methods (iTE-tP vs. RRF: p<0.03; iTE-tP vs. SGRM: p<0.02; iTE-tP vs. SSCSP: p<0.01; iTE-tP vs. SRCSP: p<0.03; iTE-tP vs. TKCSP: p<0.03; iTE-tP vs. AM-SVM: p<0.04; iTE-tP vs. UDFS: p<0.04; iTE-tP vs. NCFS: p>0.06; iTE-tP vs. eTE: p<0.002). The EEG trial extension using narrowband signals of individual channels has a significant role in the improvement of classification accuracy. The narrowband features as well as the selection of the number of CSP features improves the classifier performance. It is also observed that the proposed iTE-tP approach outperforms the recently reported algorithms.
The MI classification performance in term of accuracy the proposed method is compared with a number of recently reported methods using Dataset II. There are seven subjects (namely a, b, c, d, e, f, g) in the dataset, whereas, only four (a, b, f, g) of them are used to evaluate the performances in several algorithms [
44,
45,
46,
47,
48]. The classification accuracies of these four subjects are illustrated in
Table 4. The maximum average classification accuracy (89.53%) over the mentioned four subjects is achieved using iTE-tP. It is higher than any other methods reported in
Table 4. The average accuracy of the method VaS-SVM
lkis 87.72% which is 1.81% lower than that of iTE-tP. The performance of proposed method iTE-tP is compared to noise assisted multivariate EMD (NA-MEMD) [
44], correlation-based channel selection with regularized CSP features (CCS-RCSP) [
45], channel selection using correlation coefficient with features extraction by filter-bank CSP (CC-FBCSP) [
46] and channel selection time domain parameters and correlation coefficient (TDP-CC) [
47]. The non-stationary property is considered to calculate CSP (NS-CSP) in [
49] and bi-spectrum based channel selection implemented in [
50]. The performance of iTE-tP is also compared with [
49] and [
50]. The MI classification accuracy of iTE-tP is higher than that of all algorithms as mentioned in
Table 4.
The 83.30% of classification accuracy is achieved for four subjects by using NA-MEDM based method in [
44]. Its accuracy is 6.23% lower than that of the proposed iTE-tP. With NA-MEMD the IMFs effective for MI classification are selected heuristically, whereas, all the subbands within the specified frequency band are used in the proposed method and hence the performance is improved.
The channel selection based MI classification methods recently draw attention of the related research community. The performances of proposed methods are compared with recently developed three algorithms described in [
45,
46,
47] as illustrated in
Table 4. The effective EEG channels are selected using correlation based method, correlation coefficient, time domain parameter with correlation coefficient in CCS-RCSP [
45], CC-FBCSP [
46] and TDP-CC [
47] respectively. The CSP based feature has been used in all of the three methods. The average classification accuracy of the proposed iTE-tP method is at least 5.13% higher than any one of the channel selection based approaches [
45,
46,
47]. The proposed methods emphasizes on the extraction of potential features from trials which are regenerated using narrowband signals of each channel. The components representing MI task are localized in frequency scale and hence the performance of MI classification is increased. The channel selection based methods [
45,
46,
47] have only focused on the selection of effective channels rather than the extraction of distinctive features.
Statistical test is performed to measure the significance of the proposed method. Tukey-Kramer based post-hoc test [
38] suggests that the proposed iTE-tP achieved significantly higher accuracy across four subjects for motor imagery classification than the other methods (iTE-tP vs VaS-SVM
lk: p>0.052, iTE-tP vs NA-MEMD: p<0.05; iTE-tP vs, CCS-RCSP: p<0.04, iTE-tP vs CC-FBCSP: p<0.04, iTE-tP vs TDP-CC: p<0.05, iTE-tP vs NS-CSP: p<0.01, iTE-tP vs BCS-CSP: p<0.04, iTE-tP vs eTE: p<0.001).
The performance of the proposed method iTE-tP is evaluated using all of seven subjects of Dataset II as shown in
Table 5. The frequency-optimized spatial region based CSP features are introduced in (LRFCSP) [
51]. The effective features are obtained from the selected spatial regions. Feng et al. proposed a novel correlation-based time window selection (CTWS) algorithm to solve the problem of fixed time window of the MI-based BCIs system [
52]. The frequency-optimized features perform for MI classification. The few local regions are selected to derive the optimum set of features to improve the classification performance. The subspace optimization based feature has been used to attain optimal performance. The average MI classification accuracies over seven subjects of recently developed algorithms CTWS [
52], LRFCSP [
51] and VaS-SVM
lk [
48] are compared with iTE-tP as illustrated in Table III. It is observed that the average classification accuracies of iTE-tP is higher than that of LRFCSP [
51], CTWS [
52] as well as VaS-SVM
lk [
48] over the seven subjects. The highest average accuracy (91.55%) is achieved by iTE-tP over all the method mentioned in
Table 5.
Table 5.
Comparison of MI classification accuracy (%) of the proposed work with recently developed methods using all subjects of Dataset II.
Table 5.
Comparison of MI classification accuracy (%) of the proposed work with recently developed methods using all subjects of Dataset II.
Subject |
LRFCSP [51] |
CTWS [52] |
VaS-SVMlk [48] |
eTE |
Proposed iTE-tP
|
a |
87.40 |
83.00 |
92.50 |
83.45 |
94.14 |
b |
70.00 |
67.00 |
77.00 |
60.88 |
78.08 |
c |
67.40 |
85.50 |
82.70 |
76.44 |
84.89 |
d |
92.90 |
93.00 |
96.40 |
90.43 |
97.78 |
e |
93.40 |
99.00 |
97.20 |
92.68 |
100.00 |
f |
88.80 |
85.50 |
88.80 |
79.54 |
91.04 |
g |
93.20 |
81.00 |
92.60 |
81.88 |
94.86 |
Mean |
84.70 |
84.86 |
89.60 |
80.76 |
91.55 |
SD |
5.78 |
10.4 |
7.41 |
10.48 |
7.11 |
The maximum accuracy only for subject ‘c’ (85.50%) is attained by the method CTWS, whereas, the proposed method iTE-tP exhibits maximum accuracy for other six subjects. The maximum average accuracy is achieved by iTE-tP for seven subjects (shown in
Table 4). The average classification accuracy (over seven subjects) of iTE-tP is 6.85%, 6.69% and 1.95% higher than that of LRFCSP [
51] and CTWS [
52] and VaS-SVM
lk [
48] respectively. The results imply the superiority of the proposed method iTE-tP over the seven subjects of Dataset II. The statistical test is performed to check the significance of the proposed method. Tukey-Kramer based post-hoc test [
38] suggests that the proposed iTE-tP achieved significantly higher accuracy across the seven subjects for motor imagery classification than the other methods (iTE-tP vs LRFCSP: p<0.03; iTE-tP vs CTWS: p<0.03; iTE-tP vs VaS-SVM
lk: p>0.05; iTE-tP vs eTE: p<0.002). The important reason of performance improvement of the proposed method is the integration of the rhythmic components of EEG signals in trial regeneration. The rhythmic components exhibit better representation of neural activities represented by scalp EEG signals. Hence, the features extracted from the proposed regenerated trials are more discriminative for MI classification.