1. Introduction

2. Method

2.2. Network Architectures

Deep Context Feature Fusion Network (DCFF-Net) is proposed for the classification of hyperspectral feature diagrams. This network consists of three essential modules: Special Information Embedding (SIE, Figure 2(b)), Deep Special Feature Extraction (DSFE, Figure 2(c)), and Context Information Fusion (CIF, Figure 2(a)). To streamline the model parameters, the input image undergoes a 7×7 convolution followed by batch normalization, ReLU activation, and max-pooling. The resulting computations are then fed into the SIE and DSFE modules. The DSFE module extracts high-level feature representations of the spectral feature diagrams through a sequence of convolutional and batch normalization operations, followed by max-pooling. Then, the CIF module integrates the features from the SIE and DSFE modules. The output from this fusion process undergoes further processing through the SIE module. After passing through four layers of DSFE and CIF modules, max-pooling and global average pooling are applied to enrich the characteristic features. Finally, following the fully connected layer, the classification outcome is obtained through the softmax activation function.

The CIF module serves to seamlessly amalgamate context information, effectively integrating characteristics from various levels and yielding the ultimate feature representation for different pixels. Within the spectral feature extraction module, the features extracted by DSFE are bifurcated, with a portion integrated into the SIE module for contextual feature integration, while the remainder proceeds to the subsequent DSFE module. Figure 1 provides a simplified representation of the network architecture. Following each convolutional operation in the model, batch normalization and ReLU activation are applied. CIF comprises five distinct SIE modules and four DSFE and SIE fusion layers.

Figure 2. DCFF-NET Network Architectures.

Figure 2. DCFF-NET Network Architectures.

2.2.1. Spectral Information Embedding

With the advent of residual networks, the potential for deep learning network layers has substantially increased, leading to enhanced network depth and improved classification outcomes [39]. The Spectral Information Embedding (SIE) is configured as a residual block that includes a Squeeze and Excitation (SE) self-attention mechanism [55]. These residual blocks accelerate network training and mitigate challenges such as vanishing and exploding gradients. In the SIE module, the input x first undergoes a 3x3 convolution, batch normalization, and ReLU activation. Then, the SE Attention operation integrates all channels to enhance those contributing significantly to the classification task. Further convolution and batch normalization are then performed. When the stride in the input parameter is 1, the results are added back to the input layer. If the stride is greater than 1, a 1x1 convolution is applied to restore the size of the convolutional layer before adding it to the input layer to achieve the final feature encoding, which is then activated using the ReLU function. The specific structure is illustrated in Figure 2 (b).

The residual block is the basic module in the residual network model structure. The function can be expressed as follows:

$$F\left(x\right)=H\left(x\right)+x$$

(4)

where $\mathrm{H}\left(\mathrm{x}\right)$ is the residual function, which can be regarded as the part that performs a nonlinear transformation on the input $\mathrm{x}$. The addition operation in the formula means adding the input $\mathrm{x}$ and the output of the residual function $\mathrm{H}\left(\mathrm{x}\right)$ to $\mathrm{F}\left(\mathrm{x}\right)$ which obtains the output of the residual block.

Convolutional layer operations form the core of deep learning. To enhance model performance, researchers have proposed several processing methods, such as batch normalization (BN) [53], activation functions [54], maximum pooling, and average pooling. These methods improve the model's overall performance in different ways. The BN and ReLU activation functions are widely used in deep learning models. Batch normalization reduces the distribution variance across different data batches, mitigates gradient vanishing or explosion issues, and improves the training effect, generalization capability, and robustness of the deep neural network model. The ReLU activation function, known for its simplicity, speed, and effectiveness, significantly enhances model accuracy and convergence speed, demonstrating strong performance in numerous deep learning models. The formulas for these functions are defined as follows:

$$X_n=\frac{x-\overline{x}}{\sqrt{s+\epsilon }}$$

(5)

where $\overline{x}$ is mean; $\mathrm{s}$ is the total variance; $\mathsf{\epsilon }$ is a small positive number to avoid division by zero. The Relu activation function is defined as follows:

$$f\left(x\right)=max\left(0,x\right)$$

(6)

where $\mathrm{x}$ is the element value in the input tensor. For input values greater than or equal to 0, the original value is output; for input values less than 0, the output value is 0.

2.2.2. Deep Spectral Feature Extract

After each convolutional computation, batch normalization, and ReLU activation are applied, and a maximum pooling operation is employed to extract image features. This module employs a two-layer 3×3 convolutional calculation, incorporating the SE-Attention module to reconstruct the characteristics of different channels. It connects the feature map after the initial convolutional layer with that of the SE-Attention and subsequently with the feature map after the final convolutional layer to produce the ultimate feature map output of the DSFE module.

The max pooling layer preserves the most significant features and reduces the spatial dimension, diminishing the model's sensitivity to spatial location, decreasing the number of parameters, and improving computational efficiency. Max pooling methods reduce the feature map size through downsampling operations, allowing the model to learn overall features and local invariance, enhancing the model's robustness and generalization capability. For example, if the input two-dimensional feature map size is (H, W), the pooling operation employs a window of size $\mathrm{k}$×$\mathrm{k}$ and a step size of $\mathrm{s}$. Hence, the size of the feature map output by the pooling is ($⌊\frac{H-k}{s}⌋+1$, $⌊\frac{W-k}{s}⌋+1$), and the maximum pooling formula is defined as follows:

$$Y_{i,j}=\underset{p=0}{\overset{k-1}{\max }}\underset{q=0}{\overset{k-1}{\max }}{X}_{i\times s+p,j\times s+q},i=0，\dots ，\lfloor \frac{H-k}{s}\rfloor ,j=0，\dots ，\lfloor \frac{W-k}{s}\rfloor$$

(7)

where $\mathrm{i}$ and $\mathrm{j}$ are the row and column index of the output feature map, respectively; $\mathrm{p}$ and $\mathrm{q}$ are the row and column index in the pooling window, respectively. The function represents taking the maximum value in the window.

2.2.3. Cross Entropy Loss Function and Activation Function

Cross Entropy Loss is a commonly used metric in classification tasks to evaluate the difference between predicted outcomes and actual labels. It draws from the concept of cross-entropy in information theory, assessing the performance of a model by calculating the difference between the predicted probability distribution and the true label distribution. During the training process, model parameters are refined by minimizing the cross-entropy loss function, enabling more accurate predictions of the real labels. Optimization algorithms such as Gradient Descent are frequently employed to update model parameters based on the gradient of the loss function, reducing the loss function. The mathematical expression for sparse categorical cross-entropy loss function is presented in Equation (8):

$$L\left(y,\widehat{y}\right)=-\sum _{i=1}^k{y}_i\log \left(\widehat{y_i}\right)$$

(8)

where $\mathrm{y}$ is the probability distribution of the real label; $\widehat{y}$ is the predicted probability distribution of the model; $\mathrm{k}$is the number of categories.

The softmax activation function is employed following the fully connected layer in DCFF-NET. This function is typically utilized in the output layer of multi-classification challenges, converting the model's initial outputs into a probability distribution. Each category's probability value ranges between 0 and 1, with the total probability across all categories summing to 1. The formula for the softmax function is provided below:

$$\sigma \left(xi\right)=\frac{e^{xi}}{{\sum }_{j=1}^k{e}^{xj}},i=\mathrm{1,2},\cdots ,k$$

(9)

where $xi$ is the value of the i-th element in the input vector; $\mathrm{k}$ is the number of categories; $\mathsf{\sigma }\left(xi\right)$ is the predicted probability of the i-th category.

The output from the softmax activation function delivers a probability value for each category, with the category exhibiting the highest probability designated as the final recognized category. The formula for this determination is specified below:

$$Class\left(X\text{'}\right)=argmax{P}_i,i=\mathrm{1,2},\cdots ,k$$

(10)

3. Results and Analysis

3.1. Experimental Datasets and Implementation

The Indian Pines (IP) Aerial hyperspectral dataset, collected by the AVIRIS sensor on June 12, 1992, encompasses an area of 2.9 km × 2.9 km in northwest Indiana. The dataset features a pixel size of 145 × 145 (21025 pixels), a spatial resolution of 20 m, a spectral range from 0.4 to 2.5 µm, and includes 16 land cover types, predominantly in agricultural regions. Originally comprising 220 bands, the dataset was refined to 200 bands after excluding 20 bands that captured atmospheric water absorption and exhibited low SNR. The retained data from these 200 bands were utilized in this experiment. Table 1 and Figure 3 (a) (b) illustrate the actual marking information of ground objects in this dataset. The sample distribution across different types is highly variable, with quantities ranging from 20 to 2455.

Aerial hyperspectral remote sensing images of Pavia University (PU) were obtained by the German airborne Reflective Optical Spectral Imager (ROSIS) on July 8, 2002, in the campus area of the University of Pavia, Italy. The spatial resolution of the data is 1.3 m, the image size is 610 x 340 pixels (207,400 pixels), and the spectral range of the image spans from 0.43 to 0.86 µm, encompassing 115 spectral channels. The dataset includes nine urban land cover types and 42,776 labeled pixels. The image on the right displays a false-color composite image of the Pavia University data and the actual overlay type of the surface. Due to noise, 12 noise bands were eliminated, leaving 103 bands to verify the performance of the proposed method. The PU dataset contains nine categories of ground objects, as shown in Figure 3 (c) (d).

The Salinas dataset (SA) consists of hyperspectral remote sensing images of the Salinas Valley region in southern California, United States, acquired by the AVIRIS sensor. The image measures 512 x 217 pixels, with a spatial resolution of 3.7 m, a spectral resolution between 9.7 and 12 nm, and a spectral range from 400 to 2500 nm. It includes 16 types of ground objects and 54,129 labeled samples. The dataset initially had 224 bands, but 20 bands affected by atmospheric water absorption and low signal-to-noise ratios were removed. The remaining 204 bands, retained after processing, are utilized in this experiment, as indicated in Figure 3 (e) (f).

Figure 3. Figure 3. Hyperspectral dataset: Indian pines (a)(b), Pavia University (c)(d), Salinas(e)(f)

Figure 3. Figure 3. Hyperspectral dataset: Indian pines (a)(b), Pavia University (c)(d), Salinas(e)(f)

All experiments were conducted in identical environments using two computers. The configuration for these computers was as follows: the hardware platform included an Intel Core i7-9700K (8 cores/8 threading)/i7-1100F (8 cores/16 threading) processor with 12M/16M L3-cache/, 64GB/128G DDR4 memory at 3200MHz serial speed, NVIDIA GeForce RTX 2070 GPU with 8GB DDR5/RTX 3060 GPU with 12GB DDR5 video memory, and a 1TB HDD with 7200 RPM. The software platform comprised the Windows 10 Professional operating system, Keras 2.5.0 based on TensorFlow-gpu 2.5.0, and Python 3.7.7. Models were trained using a batch size of 12, with IP models employing an Adam optimizer with a learning rate of 2×10⁻³, a decay of 1×10⁻⁴, and PU&SA models using a learning rate of 1×10⁻⁴ and a decay of 1×10⁻⁵.

3.3. Results Analysis Based on Feature Map

Traditional Naive Bayes (NB), K-Nearest Neighbors (KNN), Random Forest (RF), and Multi-Layer Perceptron (MLP), 1D-CNN, VGG16, and ResNet50 deep learning methods were selected for comparative analysis with DCFF-NET. NB, KNN, and RF are all open-source machine learning libraries in Python. The data partitioning for random seeds was set to fixed values to achieve data recovery. NB, KNN, and RF use automatic searches (Grid Search) to select the optimal parameters with a 50% off verification method. MLP, 1D-CNN, VGG16, ResNet50, and DCFF-Net methods were implemented using TensorFlow2.5.0, and a grid search algorithm was also employed to obtain optimal parameters. When using a grid search method, important parameter selection was performed first, reducing the parameter space range and gradually narrowing the search range of the parameters based on the calculation results to improve search efficiency. This study uses different proportions of training samples for training and prediction. As the proportion of the training sample changes, the training process and performance of the model are also affected. Therefore, under different training sample proportions, the best parameter combination is sought to ensure the best performance and generalization ability of the model. The best parameters and optimal models were selected to predict and evaluate the three hyperspectral datasets. The results are depicted in Table 2.

When various classification methods use different training sample ratios, the accuracy of image prediction results increases significantly as the proportion of training samples increases. The DCFF-Net classification method performs well on the three datasets. The three different datasets exhibit higher performance than the lower training ratios in the case of 10%, 20%, and 30% training ratios. Except for the maximum value of AA, all other values achieve the maximum value and demonstrate the best classification performance overall. With 30% training samples, the OA, KA, and AA of the IP datasets are 86.68%, 85.05%, and 85.08%; the OA, KA, and AA of the PU dataset are 94.73%, 92.99%, and 92.60%; and the OA, KA, and AA of the SA dataset are 95.14%, 94.59%, and 97.48%.

3.3.1. Results of Indian Pines

Figure 4 shows the classification result chart obtained using 30% of the training samples from the Indian Pines dataset. The UAs of each category in this dataset are listed in Table 3. It can be observed from the table that out of the 16 land types, seven types achieve the best classification effects compared to other classification methods when using the DCFF-Net classification method. The OA, KA, and AA are 86.68%, 85.04%, and 85.08%, respectively, higher than other classification methods. The classification effect of VGG16 is less than that of DCFF-Net, with OA, KA, and AA at 85.18%, 83.16%, and 83.21%, respectively. Secondly, ResNet50 also performs well. These three deep learning methods outperform other classification methods. The optimal UA values for the 16 land types are scattered across various classification methods and are not concentrated in a specific classification method. In some categories, such as Corn-no Till, Corn, Oats, and others, DCFF-Net achieves a better classification effect. The NB classification methods show significant variation in UA across different categories. For Alfalfa, Grass-Pasture-Mowed, and Oats, the UA is 0.00% due to too few samples, while the highest, Hay-Windrowed, reaches an AA of 99.40%. However, the average accuracy across the 16 categories is only 53.85%, far lower than several other methods.

Figure 4. Predicted classification map of 30% Samples for Training. (a) Three bands false color composite. (b) Ground truth data. (c) NB. (d) KNN. (e) RF. (f) MLP. (g) 1DCNN. (h) VGG16. (i) Resnet50. (j) DCFF-NET.

Figure 4. Predicted classification map of 30% Samples for Training. (a) Three bands false color composite. (b) Ground truth data. (c) NB. (d) KNN. (e) RF. (f) MLP. (g) 1DCNN. (h) VGG16. (i) Resnet50. (j) DCFF-NET.

3.3.2. Results of Pavia University

Figure 5 illustrates the classification result diagram obtained using 30% training samples on the Pavia University dataset. The OA, KA, and AA of the DCFF-Net model are 94.73%, 92.99%, and 92.60%, respectively, and these three indicators have achieved the highest classification accuracy. Trees and Bitumen's UA are the highest compared to other methods. The 1D-CNN classification result's OA is second only to DCFF-Net, reaching 93.97%. The categories Gravel, Painted Metal Sheets, Bare Soil, and Shadows are higher than other methods. For Gravel and Bitumen, no classification methods have exceeded 90.00%. In Figure 1, Gravel, Bitumen, and Self-Blocking Bricks have similar ground features, leading to generally low UA in these categories. Among all comparison methods, the KNN classification method has the lowest accuracy.

Figure 5. Predicted classification map of 30% Samples for Training. (a) Three bands false color composite. (b) Ground truth data. (c) NB. (d) KNN. (e) RF. (f) MLP. (g) 1DCNN. (h) VGG16. (i) Resnet50. (j) DCFF-NET.

Figure 5. Predicted classification map of 30% Samples for Training. (a) Three bands false color composite. (b) Ground truth data. (c) NB. (d) KNN. (e) RF. (f) MLP. (g) 1DCNN. (h) VGG16. (i) Resnet50. (j) DCFF-NET.

3.3.3. Results of Salinas

Figure 6 presents the classification result obtained using 30% training samples on the Salinas dataset. The number of training samples is 16,231. Table 5 shows the user accuracy (UA) of various types in SA. DCFF-Net classification OA, KA, and AA are 95.14%, 94.59%, and 97.48%, respectively, better than other comparison methods. VGG16 and ResNet50 have also achieved better classification effects, second only to DCFF-Net. Among the 16 types of land, the DCFF-Net classification results are the highest in the UA of six places. Except for the low UA in Celery, the classification accuracy in other categories is not significantly different from other classification methods. The Celery UA is extremely low. Figure 6 indicates that the characteristic graphic features of Celery and Broccoli_green_weeds_1 and Broccoli_green_weeds_2 are highly similar, making them difficult to distinguish effectively and significantly low accuracy. In the current classification methods, Grapes_untrained and Vineyard_untrained are difficult to distinguish. As shown in Table 5, the three methods—DCFF-Net, VGG16, and ResNet50—are significantly higher than other classification methods. DCFF-Net significantly increases the characteristics of similar objects, obtaining higher classification accuracy. Grapes_untrained and Vineyard_untrained achieve 98.96% and 91.71%, respectively. In other methods, the UAs of Grapes_untrained and Vineyard_untrained are lower than 90.00%, with NB, KNN, RF, MLP, 1D-CNN, and other methods achieving less than 80.00%.

Figure 6. Predicted classification map of 30% Samples for Training. (a) Three bands false color composite. (b) Ground truth data. (c) NB. (d) KNN. (e) RF. (f) MLP. (g) 1DCNN. (h) VGG16. (i) Resnet50. (j) DCFF-NET.

Figure 6. Predicted classification map of 30% Samples for Training. (a) Three bands false color composite. (b) Ground truth data. (c) NB. (d) KNN. (e) RF. (f) MLP. (g) 1DCNN. (h) VGG16. (i) Resnet50. (j) DCFF-NET.

The eight classification methods mentioned in the text are NB, KNN, RF, MLP, 1D-CNN, VGG16, ResNet50, and DCFF-Net. The first five methods use one-dimensional serial vector data as the model input data, while the latter three are based on two-dimensional images. The latter three methods extract image features through 2D-CNN, classifying the data into different categories. In the IP data concentration, including Corn-Min Till, Corn, Oats, and Buildings-Grass-Trees-Drives, and on the SA dataset, including Grapes_untrained and Vineyard_untrained, the accuracy of the latter three methods is significantly higher than the first five methods. This indicates that 2D-CNN can achieve better classification effects in some easily mixed categories. However, for some categories, the first five methods perform better than the latter three methods, such as Soybean-Min Till, Woods, and Stone-Steel-Towers in the IP dataset. More suitable classification methods can be selected based on these observations in practical applications.

3.4. Results Analysis Based on Pixel-Patched

An additional evaluation was conducted using the pixel-patched method to examine the classification effectiveness of DCFF-NET and assess the performance of the proposed model. The methods NB, KNN, RF, MLP, and 1D-CNN use one-dimensional vector data and cannot directly use pixels as input data. Therefore, a comparative analysis was conducted using HybridSN, 3D-CNN, and A2S2KNet models. The pixel block size was uniformly extracted at 24 × 24 × S as the input data, where S is the number of image channels, and 10% of the categories were selected as training samples. The models underwent a fine-tuning process to facilitate the utilization of VGG16 and ResNet50 models for training and testing with the provided input data. After model training was completed, the optimal model was selected for testing, using 90% of the data as test data, performing the test 10 times. The average value and standard deviation were selected as the test results, as shown in Table 6, Table 7 and Table 8. The OA and KA on the IP dataset were 98.15% and 97.89%, respectively, second only to A2S2K but higher than all other methods. The AA value of 97.73% was the highest of all methods. The OA, KA, and AA of 3D-CNN were the lowest, with values of 93.18%, 92.26%, and 94.52%, respectively. The DCFF-Net model achieved the best classification performance regarding OA, KA, and AA on the PU and SA datasets. The results on the two datasets were as follows: 99.86%, 99.82%, 99.79% and 99.98%, 99.98%, 99.94%, respectively. In addition, all comparison methods also achieved very high classification accuracy.

Figure 7. Classification map of IP(A) PU(B)&SA(C) based on patched-based input. (a) False color composite. (b) Ground truth. (c) VGG16. (d) Resnet50. (e) 3-DCNN. (f) HybridSN. (g) A2S2K. (h) DCFF-NET.

Figure 7. Classification map of IP(A) PU(B)&SA(C) based on patched-based input. (a) False color composite. (b) Ground truth. (c) VGG16. (d) Resnet50. (e) 3-DCNN. (f) HybridSN. (g) A2S2K. (h) DCFF-NET.

The pixel-patched block contains surrounding pixel information, specifically spectral-spatial information, which provides more discriminative cues for the target pixel. Using pixel-patched block data as the input for the model can effectively improve classification accuracy, as mentioned earlier, but it can also lead to potential label information leakage. In addition, the larger the pixel block, the more spatial information it contains; however, it will significantly increase computational complexity.

4. Discussion

4.1. Effect of Different Filling Methods

This study discusses the impact of feature map-filling characteristics on the classification accuracy of high spectral images. The spectral feature diagram employs a variety of filling methods. In Figure 8, the initial graphic (NotFill) remains unfilled, displaying a default output of white with a pixel value of 255. The second method entails internal filling in yellow, while the exterior remains unfilled (InnerFill), with the default external color being white. The third method involves internal yellow and external green filling (BothFill). The spectral characteristic graph type is determined in blue for all three methods. Through the same classification method and parameters of the three characteristic diagrams for training and verification, test on the three datasets, the IP dataset epochs is set to 75, the other two dataset epochs set to 100, 10%, 20 %, 30% of the three different proportional training samples are trained. The verification accuracy curve is illustrated in Figure 9, and the accuracy results are listed in Table 9. The results indicate that the method involving internal and external filling yields the highest classification accuracy of the feature diagram. During the training process, the convergence rates of the three characteristic diagrams are different, and the curve volatility is inconsistent. The third characteristic diagram exhibits fewer fluctuations in training accuracy, better stability, and the highest final verification accuracy. Therefore, this filling method is selected during the spectral feature diagram conversion process. The first graphic displays general fluctuations in training accuracy and a significantly slower convergence rate, whereas the second method's performance lies between the first and third methods.

Figure 8. Different filling methods.

Figure 8. Different filling methods.

Figure 9. Three different filling methods training accuracies curve diagrams.

Figure 9. Three different filling methods training accuracies curve diagrams.

4.2. Effect of the Different Percentages of Training Samples for DCFF-NET

Different proportions of sample tag data affect classification accuracy performance significantly. Different proportions of labeled data from three datasets were selected to analyze the accuracy performance of each classification method, utilizing OA as the evaluation indicator. Proportions of 5%, 7%, 10%, 15%, 20%, 25%, and 30% from the Indian Pines dataset were chosen. In addition, 0.5%, 1%, 3%, 5%, 7%, 10%, 20%, and 30% from Pavia University and Salinas datasets were examined. Figure 10 shows that classification accuracy improves as the training sample proportion increases. For the Indian Pines dataset, the DCFF-Net classification accuracy surpasses other methods when the training sample size exceeds 7%. When the training set size is below 7%, VGG16 achieves superior overall classification accuracy. For the Pavia University and Salinas datasets, DCFF-Net outperforms other methods when the sample ratio exceeds 3%. In contrast, 1D-CNN shows better overall classification accuracy when the training sample is below 3%.

Figure 10. Effect of the different numbers of training samples for different methods.

Figure 10. Effect of the different numbers of training samples for different methods.

4.3. Ablation Analysis

The DCFF-NET model primarily utilizes the SIE and DSFE modules, with the SIE module as the core component, and integrates the DSFE module to enhance classification performance. The effectiveness of these modules was verified through individual and combined tests. When evaluating the performance of the SIE module, only the SIE module in Figure 1(a) CIF module was retained without integrating the DSFE module. In contrast, the DSFE module assessment replaced the SIE module in Figure 1(a) CIF module while retaining only the primary model for experimentation. This section replicated all experiments ten times independently with random seeds under consistent conditions. Similar to previous sections, experiments on the IP, PU, and SA datasets used three different sample proportions: 30% (Exp 1), 20% (Exp 2), and 10% (Exp 3). The final values were averaged and are displayed in Table11.

Table 11. Comparison of ablation experiment results by different modules on three datasets. The best performing results are shown in bold.

Table 11. Comparison of ablation experiment results by different modules on three datasets. The best performing results are shown in bold.

Dtaset	Modules		Exp1			Exp2			Exp3
	SIE	DSFE	OA	KA	AA	OA	KA	AA	OA	KA	AA
Indian Pines	√	--	84.72±0.77	82.56±0.89	81.13±1.58	81.94±1.28	79.38±1.47	77.49±2.66	75.00±1.90	71.46±2.19	66.66±4.21
	--	√	84.92±0.63	82.82±0.71	83.05±1.98	81.77±0.89	79.22±1.02	77.10±2.64	74.04±1.92	70.34±2.19	66.44±2.10
	√	√	86.06±0.37	84.12±0.41	84.41±1.07	83.46±0.62	81.01±0.71	79.11±2.87	77.56±0.95	74.22±1.09	69.9±2.87
Pavia University	√	--	94.18±0.12	92.26±0.16	91.88±0.25	93.36±0.28	91.17±0.37	90.89±0.38	92.28±0.13	89.73±0.18	89.53±0.37
	--	√	94.44±0.15	92.63±0.20	92.71±0.22	93.71±0.22	91.66±0.29	91.70±0.27	91.61±2.94	88.92±3.73	89.60±1.63
	√	√	94.58±0.10	92.67±0.14	92.35±0.17	93.85±0.18	91.83±0.24	91.51±0.31	92.65±0.29	90.23±0.39	89.91±0.39
Salinas	√	--	95.07±0.13	94.51±0.14	97.32±0.07	94.25±0.10	93.59±0.11	96.77±0.10	92.89±0.18	92.08±0.20	95.76±0.17
	--	√	94.92±0.22	94.34±0.25	97.37±0.15	94.06±0.12	93.39±0.13	96.82±0.13	92.71±0.19	91.88±0.21	95.93±0.28
	√	√	95.19±0.15	94.64±0.17	97.51±0.09	94.44±0.17	93.81±0.19	97.02±0.16	92.94±0.29	92.13±0.32	95.98±0.18

Under three experimental conditions, the fusion of the SIE and DSFE modules enhanced classification results compared to individual modules. On the IP dataset, the fused classification results for Experiment 1 were as follows: OA 86.06%, KA 84.12%, and AA 84.11%. The fused model showed improvements of 1.14% in OA, 1.30% in KA, and 1.35% in AA over the DSFE module. Compared to the SIE module, improvements were 1.34% in OA, 1.56% in KA, and 3.28% in AA. The fused model also exhibited lower variance, indicating a more stable performance. Significant improvements in classification accuracy were observed under other experimental conditions as well. The improvement in classification results from the fused model on the PU and SA datasets was less pronounced than that on the IP dataset. For Experiment 1, the OA and KA of the PU dataset improved by only 0.14% and 0.04%, respectively, compared to the DSFE, with a slight decrease in AA. Compared to the SIE, the OA, KA, and AA improved by 0.40%, 0.41%, and 0.47%, respectively. For the SA dataset, the improvements were 0.27% in OA, 0.30% in KA, and 0.14% in AA compared to the DSFE, and 0.12%, 0.07%, and 0.19% compared to the SIE.

5. Conclusion

References

1. Plaza, A., Benediktsson, J. A., & Boardman, J. W. Recent advances in techniques for hyperspectral image processing. Remote sensing of environment 2009, 113, S110–S122. [Google Scholar] [CrossRef]
2. Ghamisi, P., Couceiro, M. S., & Benediktsson, J. A. Deep learning-based classification of hyperspectral data. IEEE Trans. Geosci. Remote Sens. 2017, 55, 2383–2395. [Google Scholar]
3. Liu, B., Zhang, L., Zhang, L., & Du, Q. Hyperspectral image classification using deep pixel-pair features. IEEE Trans. Geosci. Remote Sens. 2015, 53, 3658–3668. [Google Scholar]
4. Belhumeur, P. N. , Hespanha, J. P., & Kriegman, D. J. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1997, 19(7), 711-720.
5. Tan, S. , Chen, X. , & Zhang, D. Kernel discriminant analysis for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2011, 33(11), 2106–2112. [Google Scholar]
6. Li, Y. , Chen, Y., & Jiang, J. A Novel KNN Classifier Based on Multi-Feature Fusion for Hyperspectral Image Classification. Remote Sensing 2020, 12, 745. [Google Scholar]
7. Li, X. , & Liang, Y. An Improved KNN Algorithm for Intrusion Detection Based on Feature Selection and Data Augmentation. IEEE Access. 2021, 9, 17132–17144. [Google Scholar]
8. Domingos, P. , & Pazzani, M. On the optimality of the simple Bayesian classifier under zero-one loss. Machine learning. 1997, 29, 103-130.
9. Zhang, H. , & Wang, X. A survey on naive Bayes classifiers. Neurocomputing. 2020, 399, 14–23. [Google Scholar]
10. J. Li, J. M. J. Li, J. M. Bioucas-Dias, and A. Plaza, “Spectral–spatial hyperspectral image segmentation using subspace multinomial logistic regression and Markov random fields,” IEEE Trans. Geosci. Remote Sens. 2012, 50, 809–823.
11. Mo, Y. , Li, M., & Liu, Y. Multinomial logistic regression with pairwise constraints for multi-label classification. IEEE Access. 2020, 8, 74005-74016.
12. Zhang, C. , & Li, J. Multinomial logistic regression with manifold regularization for image classification. Pattern Recognition Letters. 2021, 139, 55–63. [Google Scholar]
13. Osuna, E. , Freund, R., & Girosi, F. Support vector machines: Training and applications. Neural Networks. 1997, 10, 631-642.
14. F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Trans. Geosci. Remote Sens. 2004, 40, 1778–1790.
15. Wang, L. , Li, X., & Zhang, L. Support vector machines for image classification: A comprehensive review. Neurocomputing. 2016, 191, 1-9.
16. Plaza, A. , Benediktsson, J. A., & Boardman, J. W. Recent advances in techniques for hyperspectral image processing. Remote sensing of environment. 2009, 113, S110-S122.
17. Jia K, Li Q Z, Tian Y C, et al. A review of classification methods of remote sensing imagery[J]. Spectroscopy and Spectral Analysis. 2011, 31, 2618-2623.
18. Liu, S. , Li, T., & Sun, Z. Discrimination of tea varieties using hyperspectral data based on wavelet transform and partial least squares-discriminant analysis. Food Chemistry. 2020, 325, 126914.
19. He, L., Huang, W., & Zhang, X. Hyperspectral image classification with principal component analysis and support vector machine. Neurocomputing 2015, 149, 962–971. [Google Scholar]
20. Zhang, J., Wang, X., & Gao, S. Fractal feature extraction of hyperspectral images using wavelet transform. In Proceedings of the 2nd International Conference on Image, Vision and Computing (ICIVC). 2018; pp. 901–905.
21. Cheng, H. , Zhang, Y. , & Yang, Z. Spectral feature extraction based on key point detection and clustering algorithm. IEEE Access. 2019, 7, 43100–43107. [Google Scholar]
22. Huang, H. , Chen, X. , & Guo, L. A novel method for spectral angle classification based on the support vector machine. PLOS ONE. 2020, 15, e0237772. [Google Scholar]
23. Wu, D. , Zhang, L., & Shen, X. A spectral curve matching algorithm based on dynamic programming and frequency-domain filtering. IEEE Trans. Geosci. Remote Sens. 2021, 59, 4293-4307.
24. Wang, Y. , & Wang, C. Spectral curve shape index: A new spectral feature for hyperspectral image classification. Journal of Applied Remote Sensing. 2021, 15, 016524.
25. Huang, J. , Zhang, J. , & Li, Y. Discrimination of tea varieties using near infrared spectroscopy and chemometrics. Journal of Food Engineering. 2015, 144, 75–80. [Google Scholar]
26. Liu, S. , Li, T., & Sun, Z. Discrimination of tea varieties using hyperspectral data based on wavelet transform and partial least squares-discriminant analysis. Food Chemistry. 2020, 325, 126914.
27. Tao, Y. , Liu, F., & Pan, M. Rapid identification of intact paddy rice varieties using near-infrared spectroscopy and chemometric analysis. Journal of Cereal Science. 2015, 62, 59-64.
28. Schmidhuber, J. Deep learning in neural networks: An overview. Neural Networks. 2015, 61, 85–117. [Google Scholar] [PubMed]
29. LeCun, Y., Bengio, Y., & Hinton, G. Deep learning. nature. nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
30. LeCun, Y. , Bottou, L., Bengio, Y., & Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE. 1998, 86, 2278-2324.
31. Hinton, G. E. Learning multiple layers of representation. Trends in Cognitive Sciences. 2007, 11, 428–434. [Google Scholar] [CrossRef]
32. Hinton, G. E. , Osindero, S. , & Teh, Y. W. A fast learning algorithm for deep belief nets. Neural computation. 2006, 18, 1527–1554. [Google Scholar]
33. Krizhevsky, A., Sutskever, I., & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Advances in neural information processing systems. 2012, 1097–1105.
34. Graves, A. , Mohamed, A. R., & Hinton, G. E. Speech recognition with deep recurrent neural networks. In IEEE international conference on acoustics, speech and signal processing. 2013, 6645-6649.
35. Mikolov, T. , Sutskever, I. , Chen, K., Corrado, G. S., & Dean, J. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 2013, 3111–3119. [Google Scholar]
36. Simonyan, K. , & Zisserman, A. Very deep convolutional networks for large-scale image recognition. International Journal of Computer Vision. 2015, 113, 136-158.
37. Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. Return of the devil in the details: Delving deep into convolutional nets. In Proceedings of the British Machine Vision Conference (BMVC). 2014; 1–12.
38. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D, & Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2015; 1–9.
39. He, K., Zhang, X., Ren, S., & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, 770–778.
40. He, K., Zhang, X., Ren, S., & Sun, J. Identity mappings in deep residual networks. In European Conference on Computer Vision (ECCV). 2016, 630–645. [Google Scholar]
41. Xie, S., Girshick, R., Dollár, P., Tu, Z., & He, K. Aggregated residual transformations for deep neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, 1492–1500.
42. Zhang, H. , Wu, C., Zhang, J., Zhu, Y., Zhang, Z., Lin, H.,... & Sun, Y. ResNeSt: Split-Attention Networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 163-172.
43. Wei, H. , Yangyu, H., Li, W., Fan, Z., & Hengchao, L. Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors, 2015, 1-12.
44. J. Mao, W. J. Mao, W. Xu, Y. Yang, J. Wang, Z. Huang, and A. Yuille, Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN). 2014; arXiv:1412.6632. [Google Scholar]
45. S. K. Roy, G. S. K. Roy, G. Krishna, S. R. Dubey, and B. B. Chaudhuri, Hybridsn: Exploring 3-d–2-d cnn feature hierarchy for hyperspectral image classification, IEEE Geoscience and Remote Sensing Letters. 2019, 17, 277–281.
46. A. B. Hamida, A. A. B. Hamida, A. Benoit, P. Lambert, and C. B. Amar, 3-d deep learning approach for remote sensing image classification, IEEE Trans. Geosci. Remote Sens. 2018, 56,4420 4434.
47. Z. Zhong, J. Z. Zhong, J. Li, Z. Luo, and M. Chapman, Spectralspatial residual network for hyperspectral image classification: A 3-d deep learning framework, IEEE Trans. Geosci. Remote Sens. 2018, 56, 847-858.
48. Qing,Y.; Huang,Q.; Feng, L.; Qi, Y.; Liu, W. Multiscale Feature Fusion Network Incorporating 3D Self-Attention for Hyperspectral Image Classification. Remote Sens. 2022, 14, 742.
49. Qing, Y.; Liu, W. Hyperspectral image classification based on multi-scale residual network with attention mechanism. Remote Sens. 2021, 13, 335. [Google Scholar] [CrossRef]
50. Y. Chen, H. Y. Chen, H. Jiang, C. Li, X. Jia, and P. Ghamisi, Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks, IEEE Trans. Geosci. Remote Sens.2016, 54, 6232–6251.
51. Wu, J.; Sun, X.; Qu, L.; Tian, X.; Yang, G. Learning Spatial–Spectral-Dimensional-Transformation-Based Features for Hyperspectral Image Classification. Appl. Sci. 2023, 13, 8451. [Google Scholar] [CrossRef]
52. Wei, Y. , Feng, J., Liang, X., Cheng, M.-M., Zhao, Y., & Yan, S. Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
53. S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
54. He, K., Zhang, X., Ren, S., & Sun, J. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2015; 1026–1034. [Google Scholar]
55. Jie Hu, Li Shen, Gang Sun. Squeeze-and-Excitation Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, 7132-7141.
56. Bhattacharyya, C.; Kim, S. Black Ice Classification with Hyperspectral Imaging and Deep Learning. Appl. Sci. 2023, 13, 11977. [Google Scholar] [CrossRef]