2.1. Datasets

2.2. Preprocessing

• Loading the Dataset: The first step involves loading the datasets. This is done by accessing the data files containing the EEG recordings, metadata, and necessary labels. For both the Nakanishi2015 and Lee2019_SSVEP datasets, we use appropriate functions to load the data into our working environment.
• Fetching Data: Once the datasets are loaded, we fetch the raw EEG data, associated labels, and metadata. This includes extracting the dataset’s EEG signal values, trial information, and class labels.
• Creating MNE Info Structure: Using the MNE-Python library, we create an info structure that contains metadata about the EEG data, such as the names of the channels, the sampling rate, and the type of data (e.g., EEG, stim). This info structure is essential for further processing and analysis using MNE functions.
• Reshaping Data for MNE RawArray: The EEG data is reshaped to fit the format required by the MNE RawArray object. This involves organizing the data into a 2D array where each row represents a channel, and each column represents a time point.
• Setting Montage: We set the montage, which defines the spatial arrangement of the EEG electrodes on the scalp. The standard 10-20 system ensures accurate spatial representation of the EEG data.
• Setting Common Average Reference: A common average reference (CAR) is applied to the EEG data. This involves re-referencing the data by subtracting the average of all channels from each channel. CAR helps reduce the influence of common noise and improve signal quality.
• Applying Bandpass Filter: A bandpass filter is applied to the EEG data to retain only the frequencies of interest. For SSVEP data, this typically involves filtering the data to keep frequencies within the range of the SSVEP stimuli (4-16 Hz). This step helps in removing noise and irrelevant frequency components.
• Normalizing the EEG Data: Normalization is performed to standardize the EEG data, ensuring all signals have a mean of zero and a standard deviation of one. This step reduces variability due to different measurement scales and improves the performance of subsequent analysis methods. While it is acknowledged that this process may lead to the loss of information regarding possible differences in EEG amplitude distribution on the scalp, the benefits of normalization, such as improved comparability and enhanced performance of ML algorithms, outweigh this limitation. Furthermore, advanced techniques could be explored in future work to preserve some amplitude distribution information while still achieving the desired standardization.
• Creating Event Array: An event array is created to mark specific events in the EEG data, such as the onset of a stimulus. This array is essential for segmenting the continuous EEG data into epochs corresponding to individual trials.
• Creating Epochs: The continuous EEG data is segmented into epochs, time-locked segments around the events of interest (the presentation of SSVEP stimuli). Each epoch corresponds to a single trial and contains the EEG data from a specified time window around the event.
• Creating Reference Signals: Reference signals for the SSVEP stimuli are created based on the known flicker frequencies. These reference signals are used for subsequent analysis and classification of the SSVEP responses.
• Reshaping Data for CNN: The EEG data epochs are reshaped into a format suitable for input to a Convolutional Neural Network (CNN). This typically involves organizing the data into 3D arrays where each dimension represents time points, channels, and epochs.
• Mapping Frequency Labels to Numerical Indices: The frequency labels associated with the SSVEP stimuli are mapped to numerical indices to facilitate their use in Deep Learning (DL) algorithms. Each unique frequency is assigned a distinct numerical index.
• Converting Labels to One-Hot Encoding: The numerical labels are converted to one-hot encoding, a binary representation of categorical data. This step is necessary for training the CNN, allowing the network to output probabilities for each class.
• Splitting Data into Training and Testing Sets: Finally, the preprocessed data is split into training (80%) and testing (20%) sets. This ensures that the model can be trained on one subset of the data and evaluated on another unseen subset, enabling an unbiased assessment of its performance.

2.3. System Model

• Data Acquisition and Preprocessing:

  –

  We start by loading the SSVEP EEG data from our chosen datasets, Nakanishi2015 and Lee2019_SSVEP.

  –

  Data preprocessing is done as described in the previous section.

• Convolutional Neural Network (CNN):

  –

  The first layer is a Conv2D layer with 32 filters and a (3x3) kernel size, followed by a ReLU activation function.

  –

  This is followed by a MaxPooling2D layer with a (2x2) pool size to reduce the spatial dimensions.

  –

  Another Conv2D layer with 64 filters and a (3x3) kernel size, again with ReLU activation, is applied to extract more complex features.

• Temporal Convolutional Network (TCN): The reshaped data is passed through a TCN layer with 64 filters and a (3x3) kernel size, focusing on capturing the temporal dependencies in the EEG signals.
• Dense Layer: Finally, a Dense layer with a number of units equal to the number of classes (12 for Nakanishi2015 and 4 for Lee2019_SSVEP) and a softmax activation function is used for classification.
• Evaluation: The model’s robustness is tested across three scenarios: classification of clean data, classification of adversarially attacked data, and classification of attacked data using ANNT.

Figure 1. CNN-TCN Model Architecture and Evaluation Scenarios. This figure outlines the overall model architecture, showing the flow from loading EEG data, preprocessing, and CNN-TCN model layers to evaluating clean and attacked data scenarios, with and without ANNT [25].

Figure 1. CNN-TCN Model Architecture and Evaluation Scenarios. This figure outlines the overall model architecture, showing the flow from loading EEG data, preprocessing, and CNN-TCN model layers to evaluating clean and attacked data scenarios, with and without ANNT [25].

2.4. System Flow Process

• Data Extraction: EEG data is extracted from Brain 1, capturing the neural responses to visual stimuli flickering at specific frequencies.
• Preprocessing: The raw EEG data undergoes preprocessing steps, including normalization and filtering, to retain the relevant frequency components associated with SSVEP.
• Label Vector Creation: Labels corresponding to the SSVEP stimuli are created and encoded for model training.
• Feature Extraction:

  –

  The preprocessed EEG data is split into training and evaluation sets.

  –

  The CNN layers extract spatial features from the clean and attacked data.

  –

  The TCN layers capture temporal dependencies in the EEG signals.

• Adversarial Training:

  –

  Adversarial examples are generated using the FGSM to simulate potential attacks, a method chosen for its effectiveness in challenging the model’s resilience, thereby evaluating its robustness against potential threats.

  –

  The model is trained on clean and adversarially perturbed data using ANNT, enhancing its ability to recognize and resist adversarial patterns.

• Classification and Model Output:

  –

  The Dense layer classifies the EEG signals into the respective classes based on the extracted features.

  –

  The model’s performance is evaluated using accuracy and AUC under clean and attacked scenarios.

• Comparison and Analysis:

  –

  The model’s performance is compared across different scenarios, with and without ANNT, to assess its robustness and security.

  –

  The results are analyzed to identify the optimal conditions under which ANNT enhances the security and robustness of SSVEP-based B2B-C systems.

Figure 2. Comprehensive Workflow of Data Processing and Model Evaluation: systematic progression from EEG data acquisition to model assessment. The workflow includes preprocessing (purple), the generation of adversarial examples and their evaluation (red), training with clean data (yellow), and the evaluation of model performance with ANNT (green). Each color highlights a specific phase in the process, detailing steps such as adversarial perturbation, feature extraction, classification, and the robustness conferred by ANNT. Dashed lines represent the flow of evaluation and training labels, while straight lines indicate the progression of data processing steps. The workflow compares clean vs attacked performance metrics with and without ANNT (blue) [25].

Figure 2. Comprehensive Workflow of Data Processing and Model Evaluation: systematic progression from EEG data acquisition to model assessment. The workflow includes preprocessing (purple), the generation of adversarial examples and their evaluation (red), training with clean data (yellow), and the evaluation of model performance with ANNT (green). Each color highlights a specific phase in the process, detailing steps such as adversarial perturbation, feature extraction, classification, and the robustness conferred by ANNT. Dashed lines represent the flow of evaluation and training labels, while straight lines indicate the progression of data processing steps. The workflow compares clean vs attacked performance metrics with and without ANNT (blue) [25].

4.1. Key Findings

• Effectiveness of ANNT: Our study reaffirms the effectiveness of ANNT in enhancing the robustness and security of B2B-C systems. Consistent with our previous findings, ANNT significantly improves accuracy and AUC, particularly under adversarial conditions. This indicates that ANNT is reliable for fortifying B2B-C systems against potential attacks.
• Impact of Dataset Characteristics:

  –

  Subject and Channel Count: The Lee2019_SSVEP dataset, with its higher subject and channel count, demonstrates superior baseline performance and significant improvement with ANNT. This suggests that datasets with more diverse and extensive data can benefit greatly from adversarial training.

  –

  Number of Classes: The Nakanishi2015 dataset highlights the challenges of complex classification tasks with its larger number of classes. The substantial accuracy improvement with ANNT for this dataset underscores the method’s effectiveness in handling multi-class scenarios.

  –

  Sampling Rate and Data Quality: The higher sampling rate of the Lee2019_SSVEP dataset contributes to better baseline performance. However, ANNT’s ability to enhance performance in the lower-resolution Nakanishi2015 dataset indicates its robustness across varying data quality levels.

  –

  Trial Length and Number of Trials: The significant number of trials in the Lee2019_SSVEP dataset supports better learning and generalization. ANNT’s effectiveness in the Nakanishi2015 dataset with fewer trials suggests its potential in limited-data scenarios.

4.2. Comparison with Previous Work

4.3. Broader Impact and Implications

• Robustness and Security: The consistent improvements in robustness and security with ANNT across different datasets highlight its potential as a standard approach for enhancing B2B-C systems. This is crucial for security applications, such as medical and military communications.
• Dataset Design Considerations: Our findings suggest that when designing datasets for B2B-C systems, factors such as the number of classes, channels, sampling rate, and trial length should be carefully considered. These factors influence baseline performance and how adversarial training can improve system robustness.
• Future Research Directions:

  –

  Exploring Other Adversarial Techniques: While ANNT has proven effective, exploring other adversarial training techniques could further enhance robustness and security.

  –

  Larger and More Diverse Datasets: Future research should focus on larger and more diverse datasets to validate these findings and understand the limitations of ANNT.

  –

  Real-World Applications: Implementing ANNT in real-world B2B-C systems will be crucial to understanding its practical implications and any challenges that may arise.

  –

  Normalization Limitations: The normalization process, while reducing variability and improving analysis performance, may lead to the loss of information regarding differences in EEG amplitude distribution on the scalp. Future research could explore advanced techniques to preserve this amplitude distribution information while achieving the desired standardization.

  –

  Exploring Additional Attacks: Future work should consider and apply other types of attacks, such as jamming and eavesdropping, to further test the robustness and security of the B2B-C systems. Investigating countermeasures against these attacks will provide a more comprehensive understanding of system vulnerabilities and defenses.