4.3.1. Data Calibration and Pre-Processing

• Normalization;

• Derivative Method;

• Multiple Scattering Correction;

• Standard Normal Variate Transformation;

• Wavelet Transform.

• Basic formula for Continuous Wavelet Transform;

  $$W\left(a,b\right)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x\left(t\right)·\psi \left({\displaystyle \frac{t-b}{a}}\right)dt,$$

  (9)

  where W(a,b) is the wavelet coefficient, x(t) is the original signal, ψ(t) is the wavelet basis function, a is the scale parameter, and b is the shift parameter.
• Basic formula for Discrete Wavelet Transform.

  $$W\left(j,k\right)={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}x\left(t\right)·{\psi }_{j,k}\left(t\right)dt,$$

  (10)

  where W(j,k) is the discrete wavelet coefficient, and ψj,k(t) is the wavelet basis function.

4.3.2. Data Calibration and Pre-Processing

• Principal Component Analysis;

• Standardize the matrix X with a mean of x̅ and a standard deviation of Sn;

  $$X_{ij}={\displaystyle \frac{x_{ij}-\overline{x}}{S_n}},$$

  (12)

  Whereby, the original matrix is standardized to obtain matrix X1=(X1,…,Xj).
• Calculate the correlation matrix along with its eigenvalues and eigenvectors;
• Compute the variance contribution rates and determine the principal components Typically, select the top k principal components whose cumulative contribution rate exceeds 90% for data analysis.

• Independent Component Analysis;

• The purpose is to subtract the mean of each feature (variable) of the data to ensure that the data has a mean of zero on each dimension.;

  $$X_{centered}=X-E\left[X\right],$$

  (14)

  where E[X] is the mean of each column (feature) in the original data matrix X.
• The aim is to transform the observed data into a new space with a covariance matrix that is the identity matrix, reducing data redundancy and speeding up the convergence of ICA. Whitening involves transforming the centered data into a new coordinate system such that the transformed signals are uncorrelated in each direction and have unit variance. Whitening can be achieved by computing the covariance matrix C of the data and then performing eigendecomposition to accomplish this;

  $$C={\displaystyle \frac{1}{n-1}}{X}_{centered}{X}_{centered}^T,$$

  (15)

  $$C=ED{E}^T,$$

  (16)

  where E is the matrix of eigenvectors, and D is the diagonal matrix of eigenvalues. The whitening transformation is as follows:

  $$X_{white}=D^{-1/2}{E}^T{X}_{centered},$$

  (17)
• Select a measure of non-Gaussianity (such as negative entropy) and find the projection weights that maximize this measure. The ICA algorithm typically uses a nonlinear function g(∙) to approximate the maximization of negative entropy, with the iterative update rule as follows;

  $$w^{+}=E\left[X_{white}g\left(w^T{X}_{white}\right)\right]-E\left[g^{\prime }\left(w^T\right)X_{white}\right]w,$$

  (18)
  Normalize the new weight vector w⁺：

  $$w={\displaystyle \frac{w^{+}}{\parallel w^{+}\parallel }},$$

  (19)
• In multidimensional scenarios, it is necessary to ensure that the estimated components are orthogonal to each other. The Gram-Schmidt process is commonly employed for this purpose. For the i-th component, the update rule takes into account the previous i-1 components;

  $$w_i=w_i-{\sum }_{j=1}^{i-1}{w}^T{w}_j{w}_j,$$

  (20)
  After normalizing w_i, check for convergence. If the difference between w_i and $w_i^{+}$ is less than a certain threshold, it is considered that an independent component has been found.
• Once all the weight vectors W=[ w₁, w₂,…,w_n] are found., independent components can be estimated as follows.

  $$S=W^T{X}_{white},$$

  (21)

• Main Steps of MNF;

• Utilize a high-pass filter template to filter the entire image or image data blocks with the same characteristics, obtaining the noise covariance matrix (C_N). Diagonalize it into a matrix (D_N);

  $$D_N=U^T{C}_{N}U,$$

  (22)

  where D_N is the diagonal matrix of eigenvalues of C_N arranged in descending order; U is an orthogonal matrix composed of eigenvectors. Further transformation formulas can be derived as follows:

  $$I=P^T{C}_{N}P,$$

  (23)

  $$P=U{D}_N^{-1/2},$$

  (24)

  where I is the identity matrix, P is the transformation matrix. When P is applied to image data X, through the transformation Y = PX, the original image is projected into a new space, generating transformed data where the noise has unit variance and is uncorrelated between bands.
• Conduct standard principal component transformation on the noise data, with the formula.

  $$C_{D-adj}=P^T{C}_{D}P,$$

  (25)

  where C_D is the covariance matrix of image X. C_D-adj is the matrix after transformation by P, further diagonalized into matrix D_D-adj.

  $$D_{D-adj}=V^T{C}_{D-adj}V,$$

  (26)

  where D_D-adj is a diagonal matrix of eigenvalues of C_D-adj arranged in descending order， V is an orthogonal matrix composed of eigenvectors.

• Genetic Algorithm.

• Generate a set of solutions with random feature combinations;
• Evaluate the quality of each solution using an evaluation function such as information gain, variance, etc;
• Choose excellent solutions based on fitness evaluation results as parents for the next generation of the population;
• Exchange and recombine chromosomes in parents, such as single-point crossover, multi-point crossover.;
• Introduce randomness by mutating some solutions to increase population diversity, like bit flips, bit swaps;
• Determine which solutions to retain based on a replacement strategy like elitism, randomness;
• Set stopping conditions for the algorithm, such as maximum number of iterations, fitness function reaching a threshold value.

4.4.1. Machine Learning Methods

• Support Vector Machine;

• Assuming the training dataset is linearly separable, meaning there exists a hyperplane that can completely separate samples from different classes. The hyperplane can be represented by the following equation;

  $$w·x+b=0,$$

  (27)

  where w is the normal vector, x is the sample feature, and b is the bias term.
• The goal of SVM is to find an optimal hyperplane that maximizes the margin between the sample points and the hyperplane. The margin refers to the distance from the sample points to the hyperplane, and maximizing this margin helps improve the generalization ability of SVM [85]. The margin can be expressed as;

  $${\gamma }_i=y_i\left({\displaystyle \frac{w}{\parallel w\parallel }}{x}_i+{\displaystyle \frac{b}{\parallel w\parallel }}\right),$$

  (28)

  where y_i is the label of the sample (1 or -1), ||w|| is the norm of the normal vector w.
• During the process of maximizing the margin, only a subset of sample points have an impact on the position of the hyperplane, and these points are called support vectors. Support vectors are the sample points closest to the hyperplane, determining the position and orientation of the hyperplane;
• Transforming the goal of maximizing the geometric margin into a convex optimization problem can be solved using the Lagrange multiplier method. By maximizing the margin, the optimization problem can be reformulated in its dual form, where the objective is to minimize a function involving Lagrange multipliers;
• In practical applications, many problems cannot be perfectly separated by a linear hyperplane. To address this issue, SVM introduces the concept of kernel functions, mapping nonlinear problems from low-dimensional spaces to high-dimensional spaces to make them linearly separable. Common kernel functions include linear kernel, polynomial kernel, Gaussian kernel, etc [86];
• In real-world scenarios, data often contain noise and outliers. To enhance the robustness of the model, SVM introduces the concepts of soft margin and penalty terms. Soft margin allows some sample points to fall within the margin by introducing slack variables to control this. The penalty term balances maximizing the margin and penalizing misclassifications through regularization [87];
• For multi-class problems, one can use the one-vs-one or one-vs-rest methods to con-struct multiple binary classifiers, then combine them for multiclass classification.

• Random Forest;

• Sample training set samples from the sample set using sampling with replacement;
• Randomly select k variables without replacement from all predictor variables, where k is much less than m. The selected set of variables can be represented as {f1, f2, ..., fk};
• Build a decision tree based on the training samples and the generated subset of variables, using a certain criterion to measure the importance of features to determine the feature for the first split node in the tree.
• Repeat step (3) n times to generate n decision trees.

• K-Nearest Neighbors Algorithm;

• Collect and prepare a dataset for training and testing. Each sample should include a set of features and corresponding labels (for classification) or target values (for regression);
• Determine the value of K, which represents the number of nearest neighbor samples to consider. The choice of K value affects the algorithm's performance and is typically selected through methods like cross-validation;
• For a given test sample, calculate the distance between it and each sample in the training set. Common distance metrics include Euclidean distance, Manhattan distance, etc;
• Based on the calculated distances, select the K nearest training samples to the test sample as its nearest neighbors;
• For classification problems, determine the category of the test sample using a majority voting approach. Predict based on the most frequently occurring category among the labels of the K nearest neighbors. For regression problems, average the target values of the K nearest neighbors to obtain the predicted value for the test sample;
• Based on the results of the majority voting or averaging operation, consider it as the final prediction result for the test sample.

• Linear Discriminant Analysis.

• Compute the within-class scatter matrix and between-class scatter matrix: The within-class scatter matrix (S_W) represents the dispersion of data points within the same class, while the between-class scatter matrix (S_B) represents the dispersion between different classes;

  $$S_W={\sum }_{i=1}^c{\sum }_{x\in D_i}\left(x-m_i\right){\left(x-m_i\right)}^T,$$

  (30)

  where c is the number of classes, Di is the dataset of the ith class, and mi is the mean vector of the ith class.

  $$S_B={\sum }_{i=1}^c{N}_i\left(m_i-m\right){\left(m_i-m\right)}^T,$$

  (31)

  where Ni is the number of samples in the ith class, and m is the global mean vector of all samples.
• Calculate the eigenvalues and eigenvectors of matrix ${\mathrm{S}}_{\mathrm{W}}^{-1}{\mathrm{S}}_{\mathrm{B}}$: The eigenvalues of matrix ${\mathrm{S}}_{\mathrm{W}}^{-1}{\mathrm{S}}_{\mathrm{B}}$ represent the ratio of between-class scatter to within-class scatter in the direction defined by the corresponding eigenvectors after projecting the data. These eigenvectors define the projection directions from the original high-dimensional space to the new low-dimensional space. The purpose of this step is to find a matrix that, when multiplied by the data points, maximizes between-class scatter while minimizing within-class scatter;
• Select eigenvectors with the largest eigenvalues: Choose the eigenvectors corresponding to the largest eigenvalues. These eigenvectors determine the optimal new coordinate axes;
• Project the data onto the new coordinate axes: Project the original data onto the new coordinate system using the selected eigenvectors to achieve dimensionality reduction.

4.4.2. Machine Learning Methods

• Convolutional Neural Networks;

• Deep Neural Networks;

• Data preprocessing: Transforming raw data into a format acceptable by the model. Common preprocessing methods include normalization, standardization, encoding, etc;
• Building network structure: Deep neural networks consist of multiple layers, including input layers, hidden layers, and output layers. The input layer receives input data, hidden layers can contain multiple levels based on the model's complexity, and the output layer is responsible for outputting the model's prediction results;
• Defining the loss function: The loss function measures the gap between the model's prediction results and the ground truth. Common loss functions include Mean Squared Error (MSE), Cross Entropy, etc;
• Backpropagation: Through the backpropagation algorithm, the model's parameters are updated layer by layer based on the gradient information of the loss function to minimize the loss function;
• Model training: Input training data into the model, continuously update parameters through the backpropagation algorithm until the model converges or reaches a predefined stopping condition;
• Model evaluation: Evaluate the model's performance using an independent test dataset, including metrics such as accuracy, precision, recall, etc.

• Residual Networks.