Prerpints.org logo
Preprint
Review

Deep Learning for Optical Sensor Applications: An Insight Review

This version is not peer-reviewed.

Submitted:

09 June 2023

Posted:

12 June 2023

You are already at the latest version

A peer-reviewed article of this preprint also exists.

Abstract
Over the past decade, Deep Learning (DL) had been applied in a large number of optical sensors’ applications. DL algorithms can improve the accuracy and reduce the noise level in optical sensors. Optical sensors are considered as promise technology for the modern intelligent sensing platforms. These sensors are widely used to process monitoring, quality prediction, pollution, defense, security and many other applications. However, they suffer major challenges such as the large generated data and low processing speed for that data and moreover the much cost of these sensor. These challenges can be mitigated by integrating deep learning system with the optical sensor technologies. This paper presents recent studies that integrate DL algorithms with optical sensors applications. This paper also highlights several of DL algorithms directions that promise a considerable impact on use for optical sensor applications. Moreover, this study provides new directions for the future related research development.
Keywords: 
Subject: 
Computer Science and Mathematics  -   Artificial Intelligence and Machine Learning

1. Introduction

The most modern digital development includes the combination of sensors (hardware) and artificial intelligence (AI) (software) to perform intelligent tasks which is now a key component in machine learning (ML). Machine learning, one of AI branches, has a large impact on optical sensor. This new model takes a data driven approach, without focusing on underlying physics of the design. It also brings new advancements to conventional design tools and opens up numerous opportunities. Sensing has a significant impact on a broad range of scientific and engineering problems [1,2,3]. There are many types of sensors, one of them is optical sensors. Optical sensors have many worth features include light-weight, low-cost, small size, flexible deployment and operating at high pressures [4], high/low temperatures [5,6] and high electromagnetic fields [7]; without reduction of their performance. Due to these advantages, optical sensors have attracted many applications such as intrusion detection [8], monitoring of railways and general transport [9], pipelines [10] and bridge structure [11]. They also are used in detection and localization of seismic events [12], human events recognition [13], healthy tasks [14], building structure [15], landslide detection [16].
Multiple sensors usage will generate huge raw data which causes a serious challenges in processing and managing that amount of data. Furthermore, conventional processing techniques in traditional sensing devices are not suitable for labeling, processing, and analyzing that data [17]. Moreover, the collected data requires long time to be processed. The cost is another problem, where some applications require deploy many and many number of sensors. Deep Learning (DL), one of the ML branches, is incorporated with the optical sensors to solve the mentioned challenges [18,19,20,21]. Deep Neural Network (DNN) is a modern and promising technology that can be used with various optical sensor applications. The main advantage of DNN is the capability to extract dynamically the features from the collected raw data with a high accuracy that often outperforms the capability of humans [22] and [23].
In the state-of-art research of this field there are some previous survey papers reviewed the using of deep learning in specific applications for optical sensor. As an example, the authors in [24] presented an extensive review of the modern advances in the estimation of flows of multiphase fluid. The distributed fiber optical sensors and their working mechanism were addressed. The article provided some recent works which were used for characterizing multiphase fluid flows in the production optimization of the oil and gas industry. It also included traditional methods, such as the estimation of the sound speed and Joule-Thomson coefficient, in addition to data-driven ML techniques such as CNN, Ensemble Kalman Filter (EnKF) and Support Vector Machine (SVM) algorithms. Some related papers that used CNN and ANN models to perform flow regime classification and multiphase estimation have been mentioned in [25] and [10]. LSTM algorithm was adopted by another mentioned related papers to estimate fluid flow rate [26,27,28,29]. Another survey example is [30] in which the authors presented the latest advancements in pattern recognition models utilized in distributed optical fiber vibration sensing systems (DOVS). The main issues that were presented are the features extraction, the structure of the model, and the models performance. Some applications were introduced including railway safety monitoring, perimeter security and pipeline monitoring. The authors also provided the pattern recognition prospects for DOVS in addition to some related references which realized the pattern recognition of vibration signals using ML and DL. In [15], the authors reviewed the current state of the optical sensor and the DL application for structural health monitoring of civil infrastructures. The review considered the past five years and it is found that optical fibre sensors were applied to the measurement of concrete properties, leakage monitoring and corrosion, and fatigue responses.
The objective of this work is to review DL for optical sensors applications. This review comprehensively covers all published optical sensor types that have been utilized in conjunction with deep learning techniques. DL can benefit optical sensors in many directions such as process hug data, preprocess noisy data, automate feature extraction, predict the results with high accurately and reliability, and reduce the number of optical sensors that are required for deployment of any project. To our best knowledge, this is the first study that discusses the applications of deep learning for optical sensors.
This paper is organized as follows. In the second Section, we will introduce the operating principles of optical sensors. Third Section will present a brief discussion on deep learning. The survey of DL application on optical sensors is presented in the fourth Section. Discussion and future perspectives are given in the fifth Section.

2. Optical Sensors Technologies

Since the 1900s, plasmonic sensors have been vastly used in many areas to serve numerous applications [31,32,33,34,35]. Since then, they were utilized in abundant fields including biology and medical diagnosis [36,37,38,39,40], chemistry [41], food safety [42,43], and environmental monitoring and evaluation [44,45,46,47,48]. In addition, plasmonic sensors are being used in negative refractive index materials [49,50,51], optical meta surface [52,53,54], and integrated circuits [55,56,57]. Consequently, the effects formulated by the surface plasmon resonance (SPR) or localized surface plasmon resonance (LSPR) have proven astounding sensitivity needed in those applications [58]. To design plasmonic sensors, appropriate selection of the operating wavelength in addition to the type and thickness of the metal film to be used are required to have optimum sensitivity [59]. If the sensors will be used in the visible range and the near-infrared range of the electromagnet spectrum, the most common metals used are gold, copper, silver, and aluminum since they have the sharpest resonance compared to other metals [60]. Among those metals, gold is mostly preferred since it is the most chemically stable when exposed to the atmosphere. However, gold does not show SPR when the wavelength used is less than 0.5 μ m [61] and [62].
The SPR functions when photons from the incident light directed onto a metal surface layer excites the conductive electrons on its surface at a specific angle to undergo collective oscillations and then be propagated parallel to the surface [63]. This occurs since the surface of the SPR has a highly sensitive nature due to the SPR-generated evanescent field which occurs at the interface between the metal and the dielectric while undergoing total internal reflection. The point of interface is considered the strongest part at which the evanescent field happens because of the resonance coupling that happens amongst the incident rays and the surface plasmon wave [64]. As the evanescent field infuses into the dielectric media and the metal film, it decreases exponentially. The greatest decay of the field makes the SPR sensors significantly sensitive to the thickness of the material and the refractive index alterations of the dielectric film affixed to the metal-based surface [65,66,67,68,69,70]. The binding occurring at the surface of the metal and the thickness of the dielectric film affect the signal measured the plasmonic sensor [70]. This happens since the resonance of the surface plasmon wave shifts with any changes in the thickness of the material which is observed when the SPR curve shifts [64]. There is a linear relationship between the SPR signal and the dielectric film thickness and the refractive index which facilitates SPR spectroscopy of the interaction happening to be quantitatively analyzed. Hence, studying the SPR signal as a function of time explains the binding kinetics and interactions occurring at the plasmonic sensor to be measured in real time [70].
The measurement of the reflective index changes along with the binding of the sample for recognition of the immobilized molecules on the SPR sensor as shown in Figure 1. Hence, the structures’ size, shape, and composition along with the dielectric properties of the neighboring environment utterly determines the intensity and position of the SPR which are the key components in creating an optical sensor [71] and [72]. Hence, any minor adjustment to the reflective index of the sensing medium would alter the SPR occurrence which is used for detecting the analyte or the chemical [73] and [74]. The numerous variables involved in the analysis of the SPR certifies high sensitivity which makes it highly essential to be utilized in various applications [69]. Another sensor is nanoparticle-based plasmonic biosensors which have high sensitivity and low LOD so that they have been employed in pathogen detection, which allowed the detection of various diseases due to the wide spectrum of antibody binding. They are especially crucial for POCT because, when employed as biosensors, they are non-intrusive, quick, and accurate. The sensitivity of plasmonic biosensors can be further increased by adding metamaterials, which enables the biosensor to be reliable and reproducible. Recently, some researchers have been focusing on improving biosensors so that they might be created as a lab-on-a-chip diagnostic tool, making them ubiquitous. Additionally, it is being used by other researchers to look for airborne illnesses in the environment. Metamaterial-based plasmonic biosensors would enable highly accurate and rapid detection of pathogens that could improve human well-being and shield humanity from any pandemic in the future, regardless of their physical or airborne modes of transmission [75].

3. Deep Neural Networks Overview

Deep learning is a subset of ML. The main advantage of DL lies in its capability to automatically learn representative features from input data, which is called ”feature learning”. Three factors are behind deep learning’s success: availability of large datasets, advances in computing power and improvements in algorithms.
There are three types of learning in DNN, the first one is supervised algorithm which works with labeled data. Here, the model is trained to reduce the cost function which reflects the difference between the model’s predictions and the actual values. CNN [76] and LSTM [77,78] are examples of this type. The second type is semi-supervised algorithm where a small part of the samples has the annotations necessary to train the model. This algorithm construct a self-learning strategy where it generates its own annotations [79]. Examples of this type are Generative Adversarial Networks (GAN) [80] and Deep Reinforcement Learning (DRL)[81]. Third type is unsupervised algorithm where the model finds out the structure or relationships of the input data without labels or annotations. Restricted Boltzmann Machines (RBM) [82] and Auto Encoders (AE) [83] are examples of this type and perform dimensionality reduction functions or clustering.
In optical sensors applications, most commonly adopted learning architectures are CNN, AEs and Multiple Layer Perceptron (MLP) [84].

3.1. Convolutional Neural Network (CNN

In CNN, there are three types of layers: convolutional layer which works as a feature extractor from the input image, Pooling layer that reduces the dimensionality of features maps [85] and Fully-connected layers which located nearby the output layer. A softmax classifier is usually used as the final output layer, see Figure 2 . These layers learn the associations between features maps and gets output predictions that minimize the cost function. By employing a shared kernel in the convolution operation, deep learning models are able to learn space-invariant features. Compared to a fully-connected neural network, convolutional neural networks (CNNs) are good at capturing local dependencies. LSTMs are employed to make time-series predictions as they resolve the issue of vanishing gradient which arises in conventional RNNs [77].

3.2. Autoencoders

An autoencoder is neural network designed with the objective of learning a representation that closely mirrors the input data when presented as output [83,86]. As shown in Figure 3, the autoencoder consists of two components, the encoder and the decoder. The input layer and the output layer have the same number of neurons while all layers are fully connected. However, the network incorporates a bottleneck to encourage the learning of only the essential features. To create a bottleneck effect in the autoencoder, the number of nodes in the connecting layer, located between the encoder and decoder parts, is reduced compared to the number of nodes in the input layer. Similar to other neural networks, the training process of the autoencoder involves learning the weights and biases of the network by minimizing a loss function. The encoder component is responsible for learning a compact representation of the input data, while the decoder component reconstructs the original input from this learned representation. The process of learning and reconstruction in the autoencoder finds utility in a range of applications, including anomaly detection. By leveraging the learned representation and the reconstruction capability, the autoencoder can effectively identify anomalies or deviations from the normal patterns in the input data. This enables the autoencoder to serve as a valuable tool for detecting and flagging unusual or anomalous instances in various domains.

3.3. Multi layer perceptron (MLP)

Multi layer perceptron (MLP) is a complement of feed forward neural network. It comprises three layer types: input layer, output layer and hidden layer, as shown in Figure 4. The input layer is responsible for receiving the input signal that needs to be processed.. The output layer is responsible for performing the desired task, such as prediction or classification. The true computational engine of the MLP lies within an arbitrary number of hidden layers, positioned between the input and output layers. These hidden layers carry out the complex computations and transformations that enable the MLP to learn and extract meaningful patterns from the input data.. In an MLP, data follows a similar flow to a feed-forward network, progressing from the input layer to the output layer in a forward direction. The neurons within the MLP are trained using the backpropagation learning algorithm. MLPs are specifically designed to approximate any continuous function and are capable of solving problems that are not linearly separable. Some of the key applications of MLP include pattern classification, recognition, prediction, and approximation.

4. DL APPLICATIONS FOR OPTICAL SENSORS

DL development is highly iterative empirical process and it can be implemented in three steps including the choices of the initial weights and hyperparameter values, coding, and experimenting. These steps are connected through interactive process. For optical sensors applications, the first step is to figure out the existing problem with those applications such as huge data, noisy data, missing data and delay in data processing. Then try to come up with a new idea worth trying to integrate DL into optical sensors’ applications to solve these problems. The next step is to code the proposed solution using related and modern frameworks or toolkits such as TensorFlow, Keras, PyTorch, CNTK etc. After that, for training and testing the model, raw data are gathered and pre-processed and fed into the proposed DL model. Based on results, the developer should refine iteratively the proposed model and try to apply the appropriate change in the model to obtain a better accuracy. The overall view of these steps are depicted in Figure 5. Optical sensors applications based on DL techniques are surveyed here to provide interested researchers and readers with knowledge for developing high-performance optical sensors. This work introduces and discusses the related recent applications with focusing mainly on some factors and issues such as motivation, strategy, and effectiveness. The contents of the survey are expanded according to the used DL model to which each work belongs.

4.1. CNN-Based Applications

CNN is a class of DL model, most commonly used to analyze images [87]. It is attracting interest across a variety of domains including optical sensor applications. In this section, some of recent works that apply this model for optical sensors will be presented in brief.
In [88], a CNN model was developed to realize an optical fiber curvature sensor. A large number of specklegrams have been detected from facet of multimode fiber (MMF) automatically in the experiments. The detected specklegrams was preprocessed and fed to the model for training, validation and testing. The dataset was collected as light beam by designing automatic detection experimental setup as shown in Figure 6. The light beam was detected by a CCD camera which has a resolution of 1280 × 960 and pixel size of 3.75 × 3.75 μ m 2 . As shown in Figure 7, the architecture of VGG-Nets was adopted to build CCN. The mean squared error (MSE) was used as the loss function. The prediction accuracy of the proposed CNN was 94.7 % of specklegrams with the error of curvature prediction within ± 0.3 m − 1 . However, the learning-based scheme that was reported has the capability to only predict a solitary parameter and does not fully utilize the potential of deep learning.
In [9], the authors proposed semi-supervised deep learning for a track detection. An experimental setup was created using a portion of a highspeed railway track, and a distributed optical fiber acoustic system (DAS) was installed. In the proposed model, an image recognition with a specific dataset pre-processing and greedy algorithm for selecting hyperparatmeters had been used.
The considered events that were supposed to recognize in this model are shown in Table 1.
In addition, the hyper-parameters were selected based on a greedy algorithm. The obtained dataset after augemntation process is shown in Table 2.
Four structural hyper-parameters were used in this work as shown in Table 3. The obtained accuracy of the proposed model was 97.91 % . However, it is important to highlight that traditional methods perform better spatial accuracy. Some other related works can be referred in [89,90,91,92,93,94,95,96].
In [97], a distributed fiber optical sensor using a hybrid Michelson-Sagnac interferometer has been proposed. The motivation of the proposed model was to solve the problems of inability of the conventional hybrid structure to locate in the near and the flawed frequency response. The proposed model utilized basic mathematical operations and 3 × 3 optical coupler to obtain two phase signals with time difference that can be used for both location and pattern recognition. The received phase signals were converted into two-dimensional images. These images are used as dataset and were fed into CNN to obtain the required pattern recognition. The dataset contained 5488 images with 6 categories, and the size of each image was 5000 × 5000 in jpg format. The description of the dataset is shown in Table 4. The Structure diagram of CNN is shown in Figure 8. The accuracy of the proposed model was 97.83 % . However, the sensing structure employed is relatively simple and does not consider factors such as the influence of backward scattered light.
In [98], DL model was proposed to extract time-frequency sequence correlation from signals and spectrograms to improve the robustness of the recognition system. The authors designed a targeted Time Attention Model (TAM) to extract features in the time frequency domain. The architecture of the TAM model comprises two stages, namely the convolution stage for extracting features, and the time attention stage for reconstruction. The process of a data streaming, domain transformation and features extraction to output is shown in Figure 9. The knocking event is taken as an example. The convolution stage is used to extract characteristic features. Here, the convolutional filter established a local connection in the convolution and shared the weights between receiving domains. The pooling layers emphasized the shift-invariance feature. A usual CNN model is used as the backbone. As shown in Figure 9, in the left stage, information was extracted from the spectrogram and transformed into a feature map ( 1 × 128 × 200 ) , where 1 represents the number input channels (gray image has one channel), 128 and 200 represents the height and the width of the input respectively. The authors collected and labeled a large scale of dataset of vibration scenes included of 48 , 000 pieces of data of 8 vibration types. The experimental results indicated that this approach significantly improved the accuracy at a too low additional computational cost when compared with the related experiments [99] and [100]. The time attention stage was designed for features reconstruction in which TAM was used to serve two purposes. The first purpose is to extract the sequence correlation by cyclic element. The second purpose is assign the weight matrices for the attention mechanism. F1 and F2 were unique in their emphasize on investigating the "where" and "what" features of time. A F-OTDR system was constructed to classify and recognize vibration signals. The F-OTDR system contains a sensing system in addition to a producing system. This study was verified using a vibration dataset including eight different scenarios which were collected by a F-OTDR system. The achieved classification was accuracy of 96.02 % . However, this method not only complicate the data processing procedure but also has the potential to result in the loss of information during the data processing phase.
In [101], a real-time action recognition model has been proposed for long-distance oil–gas PSEW systems using a scattered distributed optical fiber sensor. They used two methods to calculate two complementary features, a peak and an energy features, that describe signals. Based on the calculated features, deep learning network (DLN) was built for a new action recognition. This DLN can effectively describe the situation of long-distance oil–gas PSEW systems. The collected datasets were 494 GB with existing of several types of noise at a China National Petroleum Corporation pipeline. The collected signal involved four types of events include background noise, mechanical excavation, manual excavation, and vehicle driving. As shown in Figure 10, the architecture of the proposed model consists of two parts. The first part deals with a peak and the second part deals with an energy. Each part consists of many layers including ConvD1, batch normalization, maxpool, dropout, Bi-LSTM and Fully Connected layer. Any damage events can be located and identified with accuracies of 99.26% (at 500 Hz) and 97.20% (at 100 Hz). Nonetheless, all the aforementioned methods consider an acquisition sample as a singular vibration event. However, for dynamic time series identification tasks, the ratio of valid data within a sample to the overall data is not constant. This means that the position of the label in relation to the valid portion of the input sequence remains uncertain. Another related researches can be seen in [102,103,104].
In [105], the authors presented application of signal processing and ML algorithms to detect events using signals generated based on DAS along a pipeline. ML approach and DL approach were implemented and combined for event detection as shown in Figure 11. A novel method to efficiently generate training dataset was developed. Excavator and none excavator events had been considered.
The sensor signals have been converted into gray image that was used to recognize the events depending on the proposed DL model. The proposed model was evaluated in a real-time deployment within three months in a suburban location which its architecture is shown in Figure 12.
The results showed that DL is the more promising approach due to its advantages over ML as shown in Table 5. However, the proposed model only differentiated between two events, namely ’excavator’ and ’no excavator,’ while there are multiple distinct events. Additionally, the system was tested in a real-time deployment for a duration of three months in a suburban area. However, for further validation and verification, it is crucial to conduct tests in different areas and over an extended period of time.
In [106], an improved WaveNet was applied to recognize man-made threat events using Distributed optical fiber Vibration Sensing (DVS). The improved WaveNet is called SE-WaveNet (squeeze and excitation WaveNet). WaveNet is a one-dimension CNN (1-DCNN) model. As a deep 1-DCNN, it can quickly achieve training and testing, while also boasting a large receptive field that enables it to retain complete information from 1-D time series data. The SE structure functions deployed to the residual block of WaveNet in order to recognize 2-D signals. The SE structure functions as an attention mechanism, which allowing the model to pay focus on channel features to obtain more information. It can also suppress the unimportant channel features. The structure of the proposed model is shown in Figure 13. The input of SE-WaveNet is an n × m matrix which was synthesized from n points spatial signals beside m groups of time signals. The used dataset is shown in Table 6. Results showed that the SE-WaveNet accuracy can reach approximately 97.73%. However, it is important to note that the model employed in this study was only tested on a limited number of events, and further testing is necessary to evaluate its performance in more complex events, particularly in engineering applications. Additionally, additional research is needed to validate the effectiveness of SE-WaveNet in practical real-world settings.
In [14], CNN and Extreme Learning Machine (ELM) were applied to discriminate between ballistocardiogram (BCG) and non-BCG signals. CNNs was used to extract relevant features. ELM, [107], is a feedforward neural network that takes as input the features extracted from CNN and provides the category matrix as output. Figure 14 and Table 7 show the architecture of the proposed CNN-ELM and the proposed CNN respectively.
BCG signals were obtained with a microbend fiber optical sensor based on IoT which was taken from ten patients diagnosed with obstructive sleep apnea and submit drug-induced sleep endoscopy. To balance the BCG (ballistocardiogram) and non-BCG signal samples, three techniques were employed: undersampling, oversampling, and generative adversarial networks (GANs). The performance of the system was evaluated using 10-fold cross-validation. Using GANs to balance the data, the CNN-ELM approach produced the best results. The average accuracy was 94%, precision was 90%, recall was 98%, and F-score was 94% as shown in Table 8. Inspired by [108] the architecture of the used model is presented in Figure 15 to balance BCG and non-BCG chunks. Another related works are presented in [109,110].
In [11], the efficiency and accuracy enhancements of bridge structure damage detection has been addressed by monitoring the deflection of the bridge using the fiber optic gyroscope. DL algorithm is then applied to detect any structural damage. They proposed a supervised learning model using CNN to perform structural damage detection. It contains eleven hidden layers that can be trained to automatically identify and classify any bridge damage. Adam optimization method was considered and the hyperparameters that were used are listed in Table 9. The obtained accuracy of the proposed model was 96.9% and better than random forest (RF) which was (81.6%), SVM which was (79.9%), k-nearest neighbor (KNN) which was (77.7%), and decision trees (DT). In the same direction, there is a work has been done in [111] and [112].
The authors in [113] proposed an intrusion pattern recognition model based on the combination Gramian Angular Field (GAF) and CNN, which possessed both high recognition speed and accuracy rate in recognition. They used GAF algorithm for mapping 1-D vibration sensing signals into 2-D images with more distinguishing features. The GAF algorithm retained and highlighted the distinguishing differences of intrusion signals. This was useful for CNN to detect intrusion events with more subtle characteristic variations differences. CNN-based framework was used for processing vibration sensing signals input images. According to the experimental results, the average accuracy rate for recognizing three natural intrusion events (light rain, wind blowing, heavy rain) and three human intrusion events (impacting, knocking, slapping) on the fence was found to be 97.67%. With a response time of 0.58 seconds, the system satisfied the real-time monitoring requirements. By considering both accuracy and speed, this model achieved automated recognition of intrusion events. However, the application of complex pre-processing and denoising techniques to the original signal presents a challenge for intrusion recognition systems when it comes to effectively addressing emergency response scenarios. Another work in the same direction was presented in [114].
Bending recognition model using the analysis of MMF specklegrams with diameter being 105 and 200 μ m was proposed and tested in [115]. The proposed model utilized a DL-based image recognition algorithm. The specklegrams detected from the facet of the MMF while subject to various bendings were utilized as input data. Figure 16 shows the used experimental setup to collect and detect fiber specklegrams.
The architecture of the model was based on VGG-Nets as shown in Figure 17.
The obtained accuracy of the proposed model for two multimode fibers is shown in Table 10.
For more related researches, it can be referred to [115,116,117,118,119,120,121,122,123].
The authors in [124] used CNN to demonstrate the capability for the identification of specific species of pollen from the backscattered light. Thirty-core optical fiber were used to collect the backscattered light. The input to CNN was camera images, these input data have been divided into two sets: distance prediction and particle identification. In the first type, the total number of collected images was 1500, 90% of them were used as a training set and 10% were as a validation set of the CNN. In the second type, the 2200 images were collected, and 90% of them were used as training set and 10% were a validation set. The training procedure of the proposed model is depicted in Figure 18. The second version of ResNet-18 ( [125,126]) was used to propose the required model with the batch normalization [127] with mini-batch size of 32 and momentum of 0.95. The output was single regression (single output). The neural network, trained to identify pollen grain types, achieved a real-time detection accuracy of approximately 97%. The developed system can be used in environments where transmission imaging is not possible or suitable.
In [13], A DL-based distributed optical fiber sensing system was proposed for event recognition. A temporal-spatial data matrix from F-OTDR system was used as input data to CNN. The proposed method has some good characteristics such as a gray-scale image transformation and a bandpass filtering which were needed as pre-processing before classification instead of the usual complex data processing, small size and high training speed, and classification accuracy. The developed system was applied to recognize five distinct events: background, jumping,walking, digging with a shovel, and striking with a shovel. The collected data was split into two types as shown in Table 11. The combined dataset for the five events consisted of 5644 instances..
Some common CNNs are examined and the results are shown in Table 12.
The considered training parameters for all CNNs were the same. The total training steps were 50,000, learning rate was 0.01, and the adopted optimizer was root mean square prop (RMSProp) [128]. This work concluded that VGGNet and GoogLeNet obtained better classification accuracy (grater than 95%) and GoogLeNet was selected to be the basic CNN structure due its model size. Further improvement of the model, inception-v3 of GoogLeNet was used. Table 13 shows the classification accuracy achieved for the five events. The authors have optimized the network by tunning the size of some layers of the model. The Table 14 shows the comparison between the optimized model and Inception-v3. However, it is important to note that this study trained the network using relatively small datasets consisting of only 4000 samples. Moreover, traditional data augmentation strategies employed in image processing, such as image rotation, cannot be directly applied to feature maps generated from fiber optic sensing data.
In [8], the authors designed a deep neural network for identifying and classifying external intrusion signals from a 33km optical fiber sensing system in the real environment. In that article, the time-domain data was putted directly into a DL model to deeply learn the destructive intrusion events characteristics and establish a reference model. This model included two CNN layers, one linear layer, one LSTM layer, and one fully connected layer as shown in Figure 19. It was called Convolutional, Long Short-Term Memory, Fully Connected Deep Neural Networks (CLDNN). The model effectively learned the signal characteristics captured by the DAS and was able to process the time-domain signal directly from the distributed optical-fiber vibration monitoring systems. It was found to be simpler and more effective than feature vector extraction through the frequency domain. The experimental results demonstrated an average intrusion event recognition rate exceeding 97% for the proposed model. Figure 20 shows DAS system using the F-OTDR and the process of pattern recognition using the CLDNN. However, the proposed model was not evaluated as a prospective solution for addressing the issue of sample contamination caused by external environmental factors, which can lead to a decline in the accuracy of recognition. Another related work can be seen in [129].
A novel method was developed in [12] to generate efficiently a training dataset using GAN [130]. End-to-end neural networks was used for processing the data that was collected by using DAS system. The proposed model’s architecture utilized the VGG16 network [23]. The purpose of the proposed model was for detection and localization of seismic events. One extra convolutional layer was added in order to match the image size then a fully connected layer was added at the end of model. Batch normalization for regularization and ReLU activation function were used. The model was tested with experimental collected data within 5km long DAS sensor and the obtained classification accuracy was 94%. Nevertheless, achieving a reliable automatic classification using the DAS system remains computationally and resource-intensive, primarily due to the demanding task of constructing a comprehensive training database, which involves collecting labeled signals for different phenomena to be classified. Furthermore, overly complex approaches may render real-time applications impractical, introducing potential processing-delay issues. Other works in the same direction were presented in [131] and [132].
In [129], the authors presented a DL model to recognize six activities, which are walking, digging with a shovel, digging with a harrow, digging with a pickaxe, facility noise and strong wind. The DAS system based on F-OTDR was presented along with novel threat detection, signal conditioning, and threat classification techniques. The CNN architecture used for classification was trained with real sensor data and consisted of five layers, as illustrated in Figure as shown in Figure 21. In that algorithm, an RGB image with dimensions 257×125×3 had been constructed. This image was constructed for each detection point on the optical fiber, which helped determine the classification of the event through the network. The results indicated that the accuracy of threat classification exceeded 93%. However, increasing the depth of the network structure in the proposed model will unavoidably result in a significant slowdown in training speed and can potentially lead to overfitting.
In their study published in [133], the authors proposed an approach to detect defects on large-sized PCBs and measure their copper thickness before the mass production process using a hybrid optical sensor HOS based on CNN. The method involves combining microscopic fringe projection profilometry (MFPP) with the lateral shearing digital holographic microscopy (LSDHM) for imaging and defect detection, utilizing an optical microscopic sensor with minimal components. This allowed for more precise and accurate identification of different types of defects on the PCBs. The proposed approach has the potential to significantly improve the quality control process in PCB manufacturing, leading to more efficient and effective production. The researchers’ findings demonstrate a remarkable success rate, with an accuracy of 99

4.2. Mltilayer Perceptron (MLP)-based Applications

In [134], MLP was proposed to achieve a specific event measurement in the existence of various noises, without shielding the sensor against undesired perturbations. The proposed model was used for temperature sensing based on sapphire crystal optical fiber (SOF). MMF interference spectra inclusive is used as input of temperature changes and noise. The trained DNN learns the relationship between the temperature and the transmission spectra as shown the Figure 22. The proposed model consisted of four hidden layers. An Adam optimizer with a learning rate of 10 − 3 was utilized alongside ReLU activation function for each output.However, due to the restrictions of the demodulation terminal, the demodulation speed is slow and as a result it has a limited scope of application.
In [135] MLP and CNN have been used to demonstrate deep learning for improving the analysis of specklegram analysis for sensing air temperature and the measurements of water immersion length. A comparison was made between the deep neural network and a traditional correlation technique. The input of MPL was a 60 × 60 input image which was fed into 3600 nodes as input layer. On the other hand, the output layer comprises a single node that represents a value of either temperature or immersion length. ReLU activation function was used after each hidden layer. The total number of the trainable parameters was 9,752,929 parameters. On the other hand, VGG-16 architecture was used for CNN model with 2014 input images. In the CNN model, the total of trainable parameters was 29,787,329. The architecture of both models is shown in Figure 23. Both models have obtained better accuracy which was in term of average errors.

4.3. Autoencoder (AE)-based Applications

In [136], A novel deep AE model was proposed to detect water-level anomalies and report abnormal situation. Time-series data have been collected from various sensors to train the proposed model. The proposed model consisted of some steps including preprocessing data, training the model and evaluating the model using normal and abnormal data as shown in the Figure 24. Combinations of hyperparameters were tuned to get the best results from each experiment configuration. Different models architectures have been used (model through momdel). Those models architectures differed from each other by the number of units at each layer, where there are 5 layers. The studies concluded that the model with 600×200×100×200×600 achieved the best result with F1-score of 99.9% and AUC of 1.00 when a window size of 36,000 was used.
In [137], DL model based on distributed optical fiber sensor (DOFS) was proposed to collect temperature data along the optic fiber and figure out the anomaly detected temperature at the early phase. The proposed model had the potential to be used for monitoring abnormal temperatures in crude oil tanks. The structure network that was used is shown in Figure 25 and Table 15, Table 16, Table 17 and Table 18.
The temperature collected by DOFS was used as normal temperature (NT) and used as a training set. The threshold value for anomaly detection was set using NT and a small amount of artificially added ambient temperature (AAT). The test set was comprised of self-heating temperature (ST) along with AAT and NT collected from the experimental apparatus. The proposed model achieved accuracy level of 0.98.
Table 19 provides a summary of the DL techniques used with optical sensor applications in this article. CNN was used in the applications from 1 through 16, MLP used in application number 17 and AE used in the remaining applications. It shows the applications and their findings (accuracy). However, the common limitations of all the previous applications are how to collect the data and how to pre-processing these data. Int addition, there are few DL models that are appropriate for using with the optical sensors. Moreover, the used method for identifying anomalies was very simple, focusing solely on the impact of ambient temperature changes on the detection of sulfurized rust self-heating anomalies. This method did not consider diverse weather conditions such as strong winds, rainfall, and temperature variations resulting from seasonal changes or diurnal fluctuations.

5. Conclusions and Future Perspectives

This study summarized the application of DL in the field of optical sensor. The necessity and significance of DL for optical sensor applications were demonstrated first by presenting the merit of DL. Next, some past and present related works have been summarized and discussed to provide the researchers with a wide view of the recent development in this field. This study was based on the type of the used DL model. It is concluded that the common models that were used are CNN, MLP and AE models due to the fact that these models are suitable with most optical sensor applications. It also is noted that the main challenges in combination of DL with the optical sensor devices are the type of data and how to be collected and pre-processed before feeding into the DL model. Treatment of an image classification problem using MLP requires converting a 2-D image into a 1-D vector before training the model. There are two issues with this approach: firstly, the number of parameters increases significantly with larger image sizes, and secondly, the MLP ignores the spatial arrangement of pixels (i.e., spatial features) in an image. For these reasons it is better to use CNN to deal with image classification models. Because the data of optical sensors can be modelled as 2D arrays (images) CNN has become dominant models in various optical sensor applications.
Finally, there are some promising applications that can be considered when DL combination with some optical sensor applications include detecting some viruses and bacteria, environmental pollution, smart city and optical communication systems. In addition, upon our search, it becomes apparent that there is a noticeable gap in the literature regarding the application of some modern networks such as Graphical Neural Networks (GNN) and Spiking Neural Networks (SNN) to optical sensors. Therefore, this article brings attention to this gap and highlights the potential for future research in exploring the utilization of GNN and SNN in optical sensor applications.

Author Contributions

N.H and K. A. M. writing—original draft, data curation, visualization, analyzing the collected articles. M.A.S. conceptualization, supervision, project administration, review. M. E. H. helped in writing and editing. All the authors contributed to the general discussion and revision of the manuscript.

Data Availability Statement

The data in this study is available from the corresponding author upon request.

Acknowledgments

This work was made using the facilities of department of Physics, AUC, Cairo, Egypt, and it was possible by fellowship awarded from Scholar Rescue Fund-IIE. The statements made herein are solely the responsibility of the authors.

Conflicts of Interest

The authors declare no competing interests.

Abbreviations

The following abbreviations are used in this manuscript:
DL Deep Learning
ML Machine learning
DNN Deep Neural Network
ANN Artificial Neural Network
LSTM Long Short Term Memory
AE Auto Encoders
MLP Mltilayer Perceptron
SVM Support Vector Machine
EnKF Ensemble Kalman Filter
DOVS distributed optical fiber vibration sensing
MMF multimode fiber

References

  1. Ignatov, A.I.; Merzlikin, A.M. Two optical sensing elements for H2O and NO2 gas sensing based on the single plasmonic–photonic crystal slab. Advanced Optical Technologies 2020, 9, 203–208. [Google Scholar] [CrossRef]
  2. Nechepurenko, I.; Andrianov, E.; Zyablovsky, A.; Dorofeenko, A.; Pukhov, A.; Lozovik, Y.E. Absorption sensor based on graphene plasmon quantum amplifier. Physical Review B 2018, 98, 075411. [Google Scholar] [CrossRef]
  3. Tomyshev, K.; Manuilovich, E.; Tazhetdinova, D.; Dolzhenko, E.; Butov, O.V. High-precision data analysis for TFBG-assisted refractometer. Sensors and Actuators A: Physical 2020, 308, 112016. [Google Scholar] [CrossRef]
  4. Kumari, C.U.; Samiappan, D.; Kumar, R.; Sudhakar, T. Fiber optic sensors in ocean observation: A comprehensive review. Optik 2019, 179, 351–360. [Google Scholar] [CrossRef]
  5. Roriz, P.; Frazão, O.; Lobo-Ribeiro, A.B.; Santos, J.L.; Simões, J.A. Review of fiber-optic pressure sensors for biomedical and biomechanical applications. Journal of biomedical optics 2013, 18, 050903. [Google Scholar] [CrossRef] [PubMed]
  6. Gupta, S.; Mizunami, T.; Yamao, T.; Shimomura, T. Fiber Bragg grating cryogenic temperature sensors. Applied optics 1996, 35, 5202–5205. [Google Scholar] [CrossRef]
  7. Taffoni, F.; Formica, D.; Saccomandi, P.; Pino, G.D.; Schena, E. Optical fiber-based MR-compatible sensors for medical applications: An overview. Sensors 2013, 13, 14105–14120. [Google Scholar] [CrossRef]
  8. Bai, Y.; Xing, J.; Xie, F.; Liu, S.; Li, J. Detection and identification of external intrusion signals from 33 km optical fiber sensing system based on deep learning. Optical Fiber Technology 2019, 53, 102060. [Google Scholar] [CrossRef]
  9. Wang, S.; Liu, F.; Liu, B. Semi-Supervised Deep Learning in High-Speed Railway Track Detection Based on Distributed Fiber Acoustic Sensing. Sensors 2022, 22, 413. [Google Scholar] [CrossRef]
  10. Vahabi, N.; Selviah, D.R. Convolutional neural networks to classify oil, water and gas wells fluid using acoustic signals. In Proceedings of the 2019 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT). IEEE; 2019; pp. 1–6. [Google Scholar]
  11. Li, S.; Zuo, X.; Li, Z.; Wang, H. Applying deep learning to continuous bridge deflection detected by fiber optic gyroscope for damage detection. Sensors 2020, 20, 911. [Google Scholar] [CrossRef]
  12. Shiloh, L.; Eyal, A.; Giryes, R. Deep learning approach for processing fiber-optic DAS seismic data. In Proceedings of the Optical Fiber Sensors. Optical Society of America; 2018. [Google Scholar] [CrossRef]
  13. Shi, Y.; Wang, Y.; Zhao, L.; Fan, Z. An event recognition method for Φ-OTDR sensing system based on deep learning. Sensors 2019, 19, 3421. [Google Scholar] [CrossRef]
  14. Tahir, S.; Sadek, I.; Abdulrazak, B. A CNN-ELM-Based Method for Ballistocardiogram Classification in a Clinical Environment. In Proceedings of the 2021 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). IEEE; 2021; pp. 1–6. [Google Scholar]
  15. Jayawickrema, U.; Herath, H.; Hettiarachchi, N.; Sooriyaarachchi, H.; Epaarachchi, J. FIBRE-OPTIC SENSOR AND DEEP LEARNING-BASED STRUCTURAL HEALTH MONITORING SYSTEMS FOR CIVIL STRUCTURES: A REVIEW. Measurement, 1115. [Google Scholar]
  16. Schenato, L.; Palmieri, L.; Camporese, M.; Bersan, S.; Cola, S.; Pasuto, A.; Galtarossa, A.; Salandin, P.; Simonini, P. Distributed optical fibre sensing for early detection of shallow landslides triggering. Scientific Reports 2017, 7, 1–7. [Google Scholar] [CrossRef]
  17. Kornienko, V.V.; Nechepurenko, I.A.; Tananaev, P.N.; Chubchev, E.D.; Baburin, A.S.; Echeistov, V.V.; Zverev, A.V.; Novoselov, I.I.; Kruglov, I.A.; Rodionov, I.A.; et al. Machine learning for optical gas sensing: a leaky-mode humidity sensor as example. IEEE Sensors Journal 2020, 20, 6954–6963. [Google Scholar] [CrossRef]
  18. Hinton, G.E.; Osindero, S.; Teh, Y.W. A fast learning algorithm for deep belief nets. Neural computation 2006, 18, 1527–1554. [Google Scholar] [CrossRef] [PubMed]
  19. Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Proceedings of the thirteenth international conference on artificial intelligence and statistics.; pp. 2010249–256.
  20. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
  21. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp.; pp. 1–9.
  22. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 2012, 25. [Google Scholar] [CrossRef]
  23. Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, arXiv:1409.1556 2014.
  24. Arief, H.A.; Wiktorski, T.; Thomas, P.J. A survey on distributed fibre optic sensor data modelling techniques and machine learning algorithms for multiphase fluid flow estimation. Sensors 2021, 21, 2801. [Google Scholar] [CrossRef] [PubMed]
  25. Silkina, T. Application of distributed acoustic sensing to flow regime classification. Master’s thesis, Institutt for petroleumsteknologi og anvendt geofysikk, 2014.
  26. Al-Naser, M.; Elshafei, M.; Al-Sarkhi, A. Artificial neural network application for multiphase flow patterns detection: A new approach. Journal of Petroleum Science and Engineering 2016, 145, 548–564. [Google Scholar] [CrossRef]
  27. Andrianov, N. A machine learning approach for virtual flow metering and forecasting. IFAC-PapersOnLine 2018, 51, 191–196. [Google Scholar] [CrossRef]
  28. Vahabi, N.; Willman, E.; Baghsiahi, H.; Selviah, D.R. Fluid flow velocity measurement in active Wells using fiber optic distributed acoustic sensors. IEEE Sensors Journal 2020, 20, 11499–11507. [Google Scholar] [CrossRef]
  29. Loh, K.; Omrani, P.S.; van der Linden, R. Deep learning and data assimilation for real-time production prediction in natural gas wells. arXiv preprint arXiv:1802.05141, arXiv:1802.05141 2018.
  30. Li, J.; Wang, Y.; Wang, P.; Bai, Q.; Gao, Y.; Zhang, H.; Jin, B. Pattern recognition for distributed optical fiber vibration sensing: A review. IEEE Sensors Journal 2021. [Google Scholar] [CrossRef]
  31. Wood, R.W. XLII. On a remarkable case of uneven distribution of light in a diffraction grating spectrum. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 1902, 4, 396–402. [Google Scholar] [CrossRef]
  32. Mie, G. Beiträge zur Optik trüber Medien, speziell kolloidaler Metallösungen. Annalen der physik 1908, 330, 377–445. [Google Scholar] [CrossRef]
  33. Fano, U. The theory of anomalous diffraction gratings and of quasi-stationary waves on metallic surfaces (Sommerfeld’s waves). JOSA 1941, 31, 213–222. [Google Scholar] [CrossRef]
  34. Ritchie, R.H. Plasma losses by fast electrons in thin films. Physical review 1957, 106, 874. [Google Scholar] [CrossRef]
  35. Hessel, A.; Oliner, A. A new theory of Wood’s anomalies on optical gratings. Applied optics 1965, 4, 1275–1297. [Google Scholar] [CrossRef]
  36. Hamza, M.E.; Othman, M.A.; Swillam, M.A. Plasmonic Biosensors. Biology 2022, 11, 621. [Google Scholar] [CrossRef] [PubMed]
  37. Hirsch, L.R.; Stafford, R.J.; Bankson, J.; Sershen, S.R.; Rivera, B.; Price, R.; Hazle, J.D.; Halas, N.J.; West, J.L. Nanoshell-mediated near-infrared thermal therapy of tumors under magnetic resonance guidance. Proceedings of the National Academy of Sciences 2003, 100, 13549–13554. [Google Scholar] [CrossRef] [PubMed]
  38. Rifat, A.A.; Ahmed, R.; Mahdiraji, G.A.; Adikan, F.M. Highly sensitive D-shaped photonic crystal fiber-based plasmonic biosensor in visible to near-IR. IEEE Sensors Journal 2017, 17, 2776–2783. [Google Scholar] [CrossRef]
  39. Sánchez-Purrà, M.; Carré-Camps, M.; de Puig, H.; Bosch, I.; Gehrke, L.; Hamad-Schifferli, K. Surface-enhanced Raman spectroscopy-based sandwich immunoassays for multiplexed detection of Zika and Dengue viral biomarkers. ACS infectious diseases 2017, 3, 767–776. [Google Scholar] [CrossRef] [PubMed]
  40. Mauriz, E.; Dey, P.; Lechuga, L.M. Advances in nanoplasmonic biosensors for clinical applications. Analyst 2019, 144, 7105–7129. [Google Scholar] [CrossRef] [PubMed]
  41. Masson, J.F.; Breault-Turcot, J.; Faid, R.; Poirier-Richard, H.P.; Yockell-Lelièvre, H.; Lussier, F.; Spatz, J.P. Plasmonic nanopipette biosensor. Analytical chemistry 2014, 86, 8998–9005. [Google Scholar] [CrossRef]
  42. Saylan, Y.; Akgönüllü, S.; Denizli, A. Plasmonic sensors for monitoring biological and chemical threat agents. Biosensors 2020, 10, 142. [Google Scholar] [CrossRef]
  43. Balbinot, S.; Srivastav, A.M.; Vidic, J.; Abdulhalim, I.; Manzano, M. Plasmonic biosensors for food control. Trends in Food Science & Technology 2021, 111, 128–140. [Google Scholar]
  44. Mauriz, E.; Calle, A.; Lechuga, L.M.; Quintana, J.; Montoya, A.; Manclus, J. Real-time detection of chlorpyrifos at part per trillion levels in ground, surface and drinking water samples by a portable surface plasmon resonance immunosensor. Analytica Chimica Acta 2006, 561, 40–47. [Google Scholar] [CrossRef]
  45. Wang, D.; Pillai, S.C.; Ho, S.H.; Zeng, J.; Li, Y.; Dionysiou, D.D. Plasmonic-based nanomaterials for environmental remediation. Applied Catalysis B: Environmental 2018, 237, 721–741. [Google Scholar] [CrossRef]
  46. Wei, H.; Abtahi, S.M.H.; Vikesland, P.J. Plasmonic colorimetric and SERS sensors for environmental analysis. Environmental Science: Nano 2015, 2, 120–135. [Google Scholar] [CrossRef]
  47. Erdem, Ö.; Saylan, Y.; Cihangir, N.; Denizli, A. Molecularly imprinted nanoparticles based plasmonic sensors for real-time Enterococcus faecalis detection. Biosensors and Bioelectronics 2019, 126, 608–614. [Google Scholar] [CrossRef]
  48. Kołataj, K.; Krajczewski, J.; Kudelski, A. Plasmonic nanoparticles for environmental analysis. Environmental Chemistry Letters 2020, 18, 529–542. [Google Scholar] [CrossRef]
  49. Farhadi, S.; Miri, M.; Farmani, A. Plasmon-induced transparency sensor for detection of minuscule refractive index changes in ultra-low index materials. Scientific Reports 2021, 11, 1–10. [Google Scholar] [CrossRef]
  50. Nishijima, Y.; Hashimoto, Y.; Balčytis, A.; Seniutinas, G.; Juodkazis, S. Alloy materials for plasmonic refractive index sensing. Sensors and Materials 2017, 29, 1233–1239. [Google Scholar]
  51. Xu, Y.; Bai, P.; Zhou, X.; Akimov, Y.; Png, C.E.; Ang, L.K.; Knoll, W.; Wu, L. Optical refractive index sensors with plasmonic and photonic structures: promising and inconvenient truth. Advanced Optical Materials 2019, 7, 1801433. [Google Scholar] [CrossRef]
  52. Nugroho, F.A.A.; Albinsson, D.; Antosiewicz, T.J.; Langhammer, C. Plasmonic metasurface for spatially resolved optical sensing in three dimensions. ACS nano 2020, 14, 2345–2353. [Google Scholar] [CrossRef] [PubMed]
  53. Zhang, J.; Ou, J.; MacDonald, K.; Zheludev, N. Optical response of plasmonic relief meta-surfaces. Journal of Optics 2012, 14, 114002. [Google Scholar] [CrossRef]
  54. Hess, O.; Gric, T. Phenomena of optical metamaterials; Elsevier, 2018.
  55. Harter, T.; Muehlbrandt, S.; Ummethala, S.; Schmid, A.; Nellen, S.; Hahn, L.; Freude, W.; Koos, C. Silicon–plasmonic integrated circuits for terahertz signal generation and coherent detection. Nature Photonics 2018, 12, 625–633. [Google Scholar] [CrossRef]
  56. Tuniz, A.; Bickerton, O.; Diaz, F.J.; Käsebier, T.; Kley, E.B.; Kroker, S.; Palomba, S.; de Sterke, C.M. Modular nonlinear hybrid plasmonic circuit. Nature communications 2020, 11, 1–8. [Google Scholar] [CrossRef] [PubMed]
  57. Sorger, V.J.; Oulton, R.F.; Ma, R.M.; Zhang, X. Toward integrated plasmonic circuits. MRS bulletin 2012, 37, 728–738. [Google Scholar] [CrossRef]
  58. Duan, Q.; Liu, Y.; Chang, S.; Chen, H.; Chen, J.h. Surface plasmonic sensors: sensing mechanism and recent applications. Sensors 2021, 21, 5262. [Google Scholar] [CrossRef]
  59. Andam, N.; Refki, S.; Hayashi, S.; Sekkat, Z. Plasmonic mode coupling and thin film sensing in metal–insulator–metal structures. Scientific reports 2021, 11, 1–12. [Google Scholar] [CrossRef]
  60. Mohammadi, A.; Sandoghdar, V.; Agio, M. Gold, copper, silver and aluminum nanoantennas to enhance spontaneous emission. Journal of Computational and Theoretical Nanoscience 2009, 6, 2024–2030. [Google Scholar] [CrossRef]
  61. Shafkat, A. Analysis of a gold coated plasmonic sensor based on a duplex core photonic crystal fiber. Sensing and Bio-Sensing Research 2020, 28, 100324. [Google Scholar] [CrossRef]
  62. Hemsley, S.J. Physical properties of gold electrodeposits and their effect on thickness measurement. Gold Bulletin 1996, 29, 19–25. [Google Scholar] [CrossRef]
  63. Li, G. Nano-inspired biosensors for protein assay with clinical applications; Elsevier, 2018.
  64. Ekgasit, S.; Tangcharoenbumrungsuk, A.; Yu, F.; Baba, A.; Knoll, W. Resonance shifts in SPR curves of nonabsorbing, weakly absorbing, and strongly absorbing dielectrics. Sensors and Actuators B: Chemical 2005, 105, 532–541. [Google Scholar] [CrossRef]
  65. Willets, K.A.; Van Duyne, R.P. Localized surface plasmon resonance spectroscopy and sensing. Annual review of physical chemistry 2007, 58, 267–297. [Google Scholar] [CrossRef]
  66. Barnes, W.L.; Dereux, A.; Ebbesen, T.W. Surface plasmon subwavelength optics. nature 2003, 424, 824–830. [Google Scholar] [CrossRef]
  67. Wang, J.; Gao, M.; He, Y.; Yang, Z. Ultrasensitive and ultrafast nonlinear optical characterization of surface plasmons. APL Materials 2022, 10, 030701. [Google Scholar] [CrossRef]
  68. Philip, A.; Kumar, A.R. The performance enhancement of surface plasmon resonance optical sensors using nanomaterials: A review. Coordination Chemistry Reviews 2022, 458, 214424. [Google Scholar] [CrossRef]
  69. Rodrigues, M.S.; Borges, J.; Lopes, C.; Pereira, R.M.; Vasilevskiy, M.I.; Vaz, F. Gas sensors based on localized surface plasmon resonances: Synthesis of oxide films with embedded metal nanoparticles, theory and simulation, and sensitivity enhancement strategies. Applied Sciences 2021, 11, 5388. [Google Scholar] [CrossRef]
  70. Ekgasit, S.; Thammacharoen, C.; Yu, F.; Knoll, W. Influence of the metal film thickness on the sensitivity of surface plasmon resonance biosensors. Applied spectroscopy 2005, 59, 661–667. [Google Scholar] [CrossRef]
  71. Ashley, J.; Shahbazi, M.A.; Kant, K.; Chidambara, V.A.; Wolff, A.; Bang, D.D.; Sun, Y. Molecularly imprinted polymers for sample preparation and biosensing in food analysis: Progress and perspectives. Biosensors and Bioelectronics 2017, 91, 606–615. [Google Scholar] [CrossRef]
  72. Drescher, D.G.; Drescher, M.J.; Ramakrishnan, N.A. Surface plasmon resonance (SPR) analysis of binding interactions of proteins in inner-ear sensory epithelia. In Auditory and Vestibular Research; Springer, 2009; pp. 323–343.
  73. Chlebus, R.; Chylek, J.; Ciprian, D.; Hlubina, P. Surface plasmon resonance based measurement of the dielectric function of a thin metal film. Sensors 2018, 18, 3693. [Google Scholar] [CrossRef]
  74. Kravets, V.G.; Wu, F.; Yu, T.; Grigorenko, A.N. Metal-dielectric-graphene hybrid heterostructures with enhanced surface plasmon resonance sensitivity based on amplitude and phase measurements. Plasmonics, 2022. [Google Scholar]
  75. Hamza, M.E.; Othman, M.A.; Swillam, M.A. Plasmonic Biosensors: Review. Biology 2022, 11. [Google Scholar] [CrossRef]
  76. LeCun, Y.; Bengio, Y.; et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks 1995, 3361, 1995. [Google Scholar]
  77. Hochreiter, S.; Schmidhuber, J. Long short-term memory Neural computation 9, 1997.
  78. Sherstinsky, A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Physica D: Nonlinear Phenomena 2020, 404, 132306. [Google Scholar] [CrossRef]
  79. Lundervold, A.S.; Lundervold, A. An overview of deep learning in medical imaging focusing on MRI. Zeitschrift für Medizinische Physik 2019, 29, 102–127. [Google Scholar] [CrossRef] [PubMed]
  80. Aggarwal, A.; Mittal, M.; Battineni, G. Generative adversarial network: An overview of theory and applications. International Journal of Information Management Data Insights 2021, 1, 100004. [Google Scholar] [CrossRef]
  81. Arulkumaran, K.; Deisenroth, M.P.; Brundage, M.; Bharath, A.A. Deep reinforcement learning: A brief survey. IEEE Signal Processing Magazine 2017, 34, 26–38. [Google Scholar] [CrossRef]
  82. Zhang, N.; Ding, S.; Zhang, J.; Xue, Y. An overview on restricted Boltzmann machines. Neurocomputing 2018, 275, 1186–1199. [Google Scholar] [CrossRef]
  83. Chen, S.; Guo, W. Auto-Encoders in Deep Learning—A Review with New Perspectives. Mathematics 2023, 11, 1777. [Google Scholar] [CrossRef]
  84. Taud, H.; Mas, J. Multilayer perceptron (MLP). Geomatic approaches for modeling land change scenarios, 2018; 451–455. [Google Scholar]
  85. Scherer, D.; Müller, A.; Behnke, S. Evaluation of pooling operations in convolutional architectures for object recognition. In Proceedings of the International conference on artificial neural networks. Springer; 2010; pp. 92–101. [Google Scholar]
  86. Jie LAI, Xiaodan WANG, Q. X.Y.S.W.Q. Review on autoencoder and its application. Journal on Communications 2021, 42, 218. [Google Scholar] [CrossRef]
  87. Valueva, M.V.; Nagornov, N.; Lyakhov, P.A.; Valuev, G.V.; Chervyakov, N.I. Application of the residue number system to reduce hardware costs of the convolutional neural network implementation. Mathematics and Computers in Simulation 2020, 177, 232–243. [Google Scholar] [CrossRef]
  88. Li, G.; Liu, Y.; Qin, Q.; Zou, X.; Wang, M.; Yan, F. Deep learning based optical curvature sensor through specklegram detection of multimode fiber. Optics & Laser Technology 2022, 149, 107873. [Google Scholar] [CrossRef]
  89. Yao, Z.; He, D.; Chen, Y.; Liu, B.; Miao, J.; Deng, J.; Shan, S. Inspection of exterior substance on high-speed train bottom based on improved deep learning method. Measurement 2020, 163, 108013. [Google Scholar] [CrossRef]
  90. Wei, X.; Yang, Z.; Liu, Y.; Wei, D.; Jia, L.; Li, Y. Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study. Engineering Applications of Artificial Intelligence 2019, 80, 66–81. [Google Scholar] [CrossRef]
  91. Zheng, Z.; Qi, H.; Zhuang, L.; Zhang, Z. Automated rail surface crack analytics using deep data-driven models and transfer learning. Sustainable Cities and Society 2021, 70, 102898. [Google Scholar] [CrossRef]
  92. Wei, X.; Wei, D.; Suo, D.; Jia, L.; Li, Y. Multi-target defect identification for railway track line based on image processing and improved YOLOv3 model. IEEE Access 2020, 8, 61973–61988. [Google Scholar] [CrossRef]
  93. Wang, S.; Liu, F.; Liu, B. Research on application of deep convolutional network in high-speed railway track inspection based on distributed fiber acoustic sensing. Optics Communications 2021, 492, 126981. [Google Scholar] [CrossRef]
  94. Fan, C.; Ai, F.; Liu, Y.; Xu, Z.; Wu, G.; Zhang, W.; Liu, C.; Yan, Z.; Liu, D.; Sun, Q. Rail Crack Detection by Analyzing the Acoustic Transmission Process Based on Fiber Distributed Acoustic Sensor. In Proceedings of the Optical Fiber Communication Conference. Optical Society of America; 2019; pp. 2–17. [Google Scholar]
  95. Chen, J.; Li, A.; Bao, C.; Dai, Y.; Liu, M.; Lin, Z.; Niu, F.; Zhou, T. A deep learning forecasting method for frost heave deformation of high-speed railway subgrade. Cold Regions Science and Technology 2021, 185, 103265. [Google Scholar] [CrossRef]
  96. Li, Z.; Zhang, J.; Wang, M.; Chai, J.; Wu, Y.; Peng, F. An anti-noise ϕ-OTDR based distributed acoustic sensing system for high-speed railway intrusion detection. Laser Physics 2020, 30, 085103. [Google Scholar] [CrossRef]
  97. Ma, Y.; Song, Y.; Xiao, Q.; Song, Q.; Jia, B. MI-SI Based Distributed Optical Fiber Sensor for No-blind Zone Location and Pattern Recognition. Journal of Lightwave Technology 2022. [Google Scholar] [CrossRef]
  98. Pan, Y.; Wen, T.; Ye, W. Time attention analysis method for vibration pattern recognition of distributed optic fiber sensor. Optik 2022, 251, 168127. [Google Scholar] [CrossRef]
  99. Xu, C.; Guan, J.; Bao, M.; Lu, J.; Ye, W. Pattern recognition based on time-frequency analysis and convolutional neural networks for vibrational events in φ-OTDR. Optical Engineering 2018, 57, 016103. [Google Scholar] [CrossRef]
  100. Zhu, P.; Xu, C.; Ye, W.; Bao, M. Self-learning filtering method based on classification error in distributed fiber optic system. IEEE Sensors Journal 2019, 19, 8929–8933. [Google Scholar] [CrossRef]
  101. Yang, Y.; Li, Y.; Zhang, T.; Zhou, Y.; Zhang, H. Early safety warnings for long-distance pipelines: A distributed optical fiber sensor machine learning approach. In Proceedings of the Proceedings of the AAAI Conference on Artificial Intelligence, 2021, Vol. 35, pp. 14991–14999.
  102. Wu, H.; Qian, Y.; Zhang, W.; Tang, C. Feature extraction and identification in distributed optical-fiber vibration sensing system for oil pipeline safety monitoring. Photonic Sensors 2017, 7, 305–310. [Google Scholar] [CrossRef]
  103. Liu, K.; Tian, M.; Liu, T.; Jiang, J.; Ding, Z.; Chen, Q.; Ma, C.; He, C.; Hu, H.; Zhang, X. A high-efficiency multiple events discrimination method in optical fiber perimeter security system. Journal of Lightwave Technology 2015, 33, 4885–4890. [Google Scholar]
  104. Zhu, C.; Pu, Y.; Yang, K.; Yang, Q.; Philip Chen, C.L. Distributed Optical Fiber Intrusion Detection by Image Encoding and SwinT in Multi-Interference Environment of Long-Distance Pipeline. IEEE Transactions on Instrumentation and Measurement. [CrossRef]
  105. Bublin, M. Event Detection for Distributed Acoustic Sensing: Combining Knowledge-Based, Classical Machine Learning, and Deep Learning Approaches. Sensors 2021, 21, 7527. [Google Scholar] [CrossRef]
  106. Sun, M.; Yu, M.; Lv, P.; Li, A.; Wang, H.; Zhang, X.; Fan, T.; Zhang, T. Man-made threat event recognition based on distributed optical fiber vibration sensing and SE-WaveNet. IEEE Transactions on Instrumentation and Measurement 2021, 70, 1–11. [Google Scholar] [CrossRef]
  107. Ding, S.; Zhao, H.; Zhang, Y.; Xu, X.; Nie, R. Extreme learning machine: algorithm, theory and applications. Artificial Intelligence Review 2015, 44, 103–115. [Google Scholar] [CrossRef]
  108. Hatamian, F.N.; Ravikumar, N.; Vesal, S.; Kemeth, F.P.; Struck, M.; Maier, A. The effect of data augmentation on classification of atrial fibrillation in short single-lead ECG signals using deep neural networks. In Proceedings of the ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2020; pp. 1264–1268. [Google Scholar]
  109. Yoo, H.; Han, S.; Chung, K. A frequency pattern mining model based on deep neural network for real-time classification of heart conditions. In Proceedings of the Healthcare. Multidisciplinary Digital Publishing Institute, Vol. 8; 2020; p. 234. [Google Scholar]
  110. Wang, Q.; Lyu, W.; Cheng, Z.; Yu, C. Noninvasive Measurement of Vital Signs with the Optical Fiber Sensor Based on Deep Learning. Journal of Lightwave Technology. [CrossRef]
  111. Fernandez-Navamuel, A.; Magalhães, F.; Zamora-Sánchez, D.; Omella, Á.J.; Garcia-Sanchez, D.; Pardo, D. Deep learning enhanced principal component analysis for structural health monitoring. Structural Health Monitoring, 1475. [Google Scholar]
  112. Wang, H.; Guo, J.K.; Mo, H.; Zhou, X.; Han, Y. Fiber Optic Sensing Technology and Vision Sensing Technology for Structural Health Monitoring. Sensors 2023, 23. [Google Scholar] [CrossRef] [PubMed]
  113. Lyu, C.; Huo, Z.; Cheng, X.; Jiang, J.; Alimasi, A.; Liu, H. Distributed optical fiber sensing intrusion pattern recognition based on GAF and CNN. Journal of Lightwave Technology 2020, 38, 4174–4182. [Google Scholar] [CrossRef]
  114. Wu, J.; Zhuo, R.; Wan, S.; Xiong, X.; Xu, X.; Liu, B.; Liu, J.; Shi, J.; Sun, J.; He, X.; et al. Intrusion location technology of Sagnac distributed fiber optical sensing system based on deep learning. IEEE Sensors Journal 2021, 21, 13327–13334. [Google Scholar] [CrossRef]
  115. Liu, Y.; Li, G.; Qin, Q.; Tan, Z.; Wang, M.; Yan, F. Bending recognition based on the analysis of fiber specklegrams using deep learning. Optics & Laser Technology 2020, 131, 106424. [Google Scholar]
  116. Sun, K.; Ding, Z.; Zhang, Z. Fiber directional position sensor based on multimode interference imaging and machine learning. Applied Optics 2020, 59, 5745–5751. [Google Scholar] [CrossRef]
  117. Ding, Z.; Zhang, Z. 2D tactile sensor based on multimode interference and deep learning. Optics & Laser Technology 2021, 136, 106760. [Google Scholar]
  118. Wei, M.; Tang, G.; Liu, J.; Zhu, L.; Liu, J.; Huang, C.; Zhang, J.; Shen, L.; Yu, S. Neural network based perturbation-location fiber specklegram sensing system towards applications with limited number of training samples. Journal of Lightwave Technology 2021, 39, 6315–6326. [Google Scholar] [CrossRef]
  119. Gupta, R.K.; Bruce, G.D.; Powis, S.J.; Dholakia, K. Deep learning enabled laser speckle wavemeter with a high dynamic range. Laser & Photonics Reviews 2020, 14, 2000120. [Google Scholar]
  120. Cuevas, A.R.; Fontana, M.; Rodriguez-Cobo, L.; Lomer, M.; López-Higuera, J.M. Machine learning for turning optical fiber specklegram sensor into a spatially-resolved sensing system. Proof of concept. Journal of Lightwave Technology 2018, 36, 3733–3738. [Google Scholar] [CrossRef]
  121. Razmyar, S.; Mostafavi, M.T. Deep learning for estimating deflection direction of a multimode fiber from specklegram. Journal of Lightwave Technology 2020, 39, 1850–1857. [Google Scholar] [CrossRef]
  122. Bender, D.; Çakır, U.; Yüce, E. Deep Learning-Based Fiber Bending Recognition for Sensor Applications. IEEE Sensors Journal 2023, 23, 6956–6962. [Google Scholar] [CrossRef]
  123. Lu, J.; Gao, H.; Liu, Y.; Hu, H. Deep learning based force recognition using the specklegrams from multimode fiber. Instrumentation Science & Technology 2023, 0, 1–11. [Google Scholar] [CrossRef]
  124. Grant-Jacob, J.A.; Jain, S.; Xie, Y.; Mackay, B.S.; McDonnell, M.D.; Praeger, M.; Loxham, M.; Richardson, D.J.; Eason, R.W.; Mills, B. Fibre-optic based particle sensing via deep learning. Journal of Physics: Photonics 2019, 1, 044004. [Google Scholar] [CrossRef]
  125. He, K.; Zhang, X.; Ren, S.; Sun, J. Identity mappings in deep residual networks. In Proceedings of the European conference on computer vision. Springer; 2016; pp. 630–645. [Google Scholar]
  126. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp.; pp. 770–778.
  127. Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International conference on machine learning. PMLR; 2015; pp. 448–456. [Google Scholar]
  128. Yazan, E.; Talu, M.F. Comparison of the stochastic gradient descent based optimization techniques. In Proceedings of the 2017 International Artificial Intelligence and Data Processing Symposium (IDAP). IEEE; 2017; pp. 1–5. [Google Scholar]
  129. Aktas, M.; Akgun, T.; Demircin, M.U.; Buyukaydin, D. Deep learning based multi-threat classification for phase-OTDR fiber optic distributed acoustic sensing applications. In Proceedings of the Fiber Optic Sensors and Applications XIV. International Society for Optics and Photonics, Vol. 10208; 2017; p. 102080. [Google Scholar]
  130. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. Advances in neural information processing systems 2014, 27. [Google Scholar]
  131. Hernandez, P.D.; Ramirez, J.A.; Soto, M.A. Deep-Learning-Based Earthquake Detection for Fiber-Optic Distributed Acoustic Sensing. Journal of Lightwave Technology 2021. [Google Scholar] [CrossRef]
  132. Xie, Y.; Wang, M.; Zhong, Y.; Deng, L.; Zhang, J. Label-Free Anomaly Detection Using Distributed Optical Fiber Acoustic Sensing. Sensors 2023, 23. [Google Scholar] [CrossRef]
  133. Kaya, G.U. Development of hybrid optical sensor based on deep learning to detect and classify the micro-size defects in printed circuit board. Measurement 2023, 206, 112247. [Google Scholar] [CrossRef]
  134. Nguyen, L.V.; Nguyen, C.C.; Carneiro, G.; Ebendorff-Heidepriem, H.; Warren-Smith, S.C. Sensing in the presence of strong noise by deep learning of dynamic multimode fiber interference. Photonics Research 2021, 9, B109–B118. [Google Scholar] [CrossRef]
  135. Smith, D.L.; Nguyen, L.V.; Ottaway, D.J.; Cabral, T.D.; Fujiwara, E.; Cordeiro, C.M.; Warren-Smith, S.C. Machine learning for sensing with a multimode exposed core fiber specklegram sensor. Optics Express 2022, 30, 10443–10455. [Google Scholar] [CrossRef] [PubMed]
  136. Nicholaus, I.T.; Park, J.R.; Jung, K.; Lee, J.S.; Kang, D.K. Anomaly Detection of Water Level Using Deep Autoencoder. Sensors 2021, 21, 6679. [Google Scholar] [CrossRef] [PubMed]
  137. Zhu, Z.C.; Chu, C.W.; Bian, H.T.; Jiang, J.C. An integration method using distributed optical fiber sensor and Auto-Encoder based deep learning for detecting sulfurized rust self-heating of crude oil tanks. Journal of Loss Prevention in the Process Industries 2022, 74, 104623. [Google Scholar] [CrossRef]
Preprints 76226 g001
Figure 1. (a) A diagram of the mechanism of plasmonic optical sensors, and (b) Stages of SPR sensor from detecting analytes to detaching to be reused [36].
Figure 1. (a) A diagram of the mechanism of plasmonic optical sensors, and (b) Stages of SPR sensor from detecting analytes to detaching to be reused [36].
Preprints 76226 g001
Preprints 76226 g002
Figure 2. CNN Architecture.
Figure 2. CNN Architecture.
Preprints 76226 g002
Preprints 76226 g003
Figure 3. Diagram of Autoencoder.
Figure 3. Diagram of Autoencoder.
Preprints 76226 g003
Preprints 76226 g004
Figure 4. Schematic representation of a MLP with two hidden layers.
Figure 4. Schematic representation of a MLP with two hidden layers.
Preprints 76226 g004
Preprints 76226 g005
Figure 5. General view of DL with optical sensor applications.
Figure 5. General view of DL with optical sensor applications.
Preprints 76226 g005
Preprints 76226 g006
Figure 6. Experimental setup for the detection of fiber specklegrams under different curvatures.
Figure 6. Experimental setup for the detection of fiber specklegrams under different curvatures.
Preprints 76226 g006
Preprints 76226 g007
Figure 7. The structure of the CNN used for training and testing in [88].
Figure 7. The structure of the CNN used for training and testing in [88].
Preprints 76226 g007
Preprints 76226 g008
Figure 8. Structure diagram of CNN used in [97].
Figure 8. Structure diagram of CNN used in [97].
Preprints 76226 g008
Preprints 76226 g009
Figure 9. The structure of vibration sensing system working with F-OTDR.
Figure 9. The structure of vibration sensing system working with F-OTDR.
Preprints 76226 g009
Preprints 76226 g010
Figure 10. The architecture of the proposed model used in [101].
Figure 10. The architecture of the proposed model used in [101].
Preprints 76226 g010
Preprints 76226 g011
Figure 11. Two methods to detect events by DAS: classic ML approach (left) and DNN roach (right). Note the role of human knowledge [105].
Figure 11. Two methods to detect events by DAS: classic ML approach (left) and DNN roach (right). Note the role of human knowledge [105].
Preprints 76226 g011
Preprints 76226 g012
Figure 12. CNN to detect excavator. (a) Input image (b) Convolutional Layer (c) Max pooling layer (d) Fully connected layer [105].
Figure 12. CNN to detect excavator. (a) Input image (b) Convolutional Layer (c) Max pooling layer (d) Fully connected layer [105].
Preprints 76226 g012
Preprints 76226 g013
Figure 13. The structure of the model proposed in [106].
Figure 13. The structure of the model proposed in [106].
Preprints 76226 g013
Preprints 76226 g014
Figure 14. The CNN architecture proposed in [14].
Figure 14. The CNN architecture proposed in [14].
Preprints 76226 g014
Preprints 76226 g015
Figure 15. The architecture of the model of BCG and non-BCG chunks
Figure 15. The architecture of the model of BCG and non-BCG chunks
Preprints 76226 g015
Preprints 76226 g016
Figure 16. The experiment of schematic setup. (a) fiber specklegrams detection. (b) A graphical depiction of the moving distance x and bending radius R of the translation stage.
Figure 16. The experiment of schematic setup. (a) fiber specklegrams detection. (b) A graphical depiction of the moving distance x and bending radius R of the translation stage.
Preprints 76226 g016
Preprints 76226 g017
Figure 17. The model structure of CNN proposed in [115].
Figure 17. The model structure of CNN proposed in [115].
Preprints 76226 g017
Preprints 76226 g018
Figure 18. The training procedure of the model proposed in [124].
Figure 18. The training procedure of the model proposed in [124].
Preprints 76226 g018
Preprints 76226 g019
Figure 19. The CLDNN architecture.
Figure 19. The CLDNN architecture.
Preprints 76226 g019
Preprints 76226 g020
Figure 20. The architecture of the intelligent alarm system proposed in [8].
Figure 20. The architecture of the intelligent alarm system proposed in [8].
Preprints 76226 g020
Preprints 76226 g021
Figure 21. The structure of a F-OTDR and a CNN used for threat classification in [129].
Figure 21. The structure of a F-OTDR and a CNN used for threat classification in [129].
Preprints 76226 g021
Preprints 76226 g022
Figure 22. The multilayer perceptron architecture proposed in [134].
Figure 22. The multilayer perceptron architecture proposed in [134].
Preprints 76226 g022
Preprints 76226 g023
Figure 23. (a) Schematic of DNN. (b) Schematic representation of CNN used in [135].
Figure 23. (a) Schematic of DNN. (b) Schematic representation of CNN used in [135].
Preprints 76226 g023
Preprints 76226 g024
Figure 24. AEs used for the detection of abnormal temperature changes.
Figure 24. AEs used for the detection of abnormal temperature changes.
Preprints 76226 g024
Preprints 76226 g025
Figure 25. A pipeline for anomaly detection using DNN model based on autoencoders in [137].
Figure 25. A pipeline for anomaly detection using DNN model based on autoencoders in [137].
Preprints 76226 g025
Table 1. Events distribution.
Table 1. Events distribution.
Event Location
Crevice 120 m, 790 m, 830 m, 1010 m, 1270 m
Beam Gap 400 m, 500 m, 600 m, 700 m, 800 m, 900 m, 1000 m, 1100 m, 1200 m, 1300 m, 1400 m
Cracking 100m, 480m, 730m, 1030m, 1420m
Bulge 420 m, 560 m, 730 m, 1030 m, 1420 m
Switches 200 m, 350 m, 450 m, 1350 m
Highway Below 300 m, 750 m, 1250 m
Table 2. dataset obtained after augemntation process and Training dataset.
Table 2. dataset obtained after augemntation process and Training dataset.
size Channels Event Dataset Training data
5 × 6 3 Crevice 6853 2708
5 × 6 3 Beam gap 6853 2788
5 × 6 3 Cracking 6853 2735
5 × 6 3 Bulge 6853 2558
5 × 6 3 Switches 6853 2776
5 × 6 3 Highway below 6853 2736
5 × 6 3 No event 6853 2716
Table 3. Structural hyper-parameters.
Table 3. Structural hyper-parameters.
Hyper-Parameters Options
Structure of deep learning network VGG-16, ResNet, Inception V3, AlexNet, Mobilenet V3, LeNet
Data balance method SMOTE-TL, S-ENN, Border-1, MWMOTE, Safe-level
SSL model Fix-match, Tri-training, UDA
Time interval (t) [0, 54 ]
Table 4. DATASET DETAILS.
Table 4. DATASET DETAILS.
Type Amount
Cutting 948
Impacting 737
Knocking 1235
Rocking 550
Trampling 849
Wind 1169
Total 5488
Table 5. Performance comparison of CNN and MLP algorithms .
Table 5. Performance comparison of CNN and MLP algorithms .
ML Algorithm Accuracy 99% Exec. Time (µs)
MLP + feature extraction 99.88% 554.63
CNN 99.91% 34.33
Table 6. The number of each event in Dataset.
Table 6. The number of each event in Dataset.
Event type Climbing Violent demolition net Digging Electric drill damage Total number
Training set 4319 3616 3713 3825 17564
Validation set 539 542 562 549 2192
Testing set 539 542 562 549 2192
Total number 5397 5428 5625 5498 21948
Table 7. The proposed CNN Architecture.
Table 7. The proposed CNN Architecture.
Event type Climbing Violent demolition net Digging Electric drill damage Total number
Layer No of filters Activation function Kernel size Strides Output size
Input (50,1)
Conv1D 50 ReLU 5 1 (46,40)
Maxpooling1D Max 2 2 (23,50)
Conv1D 50 ReLU 4 1 (20,50)
Maxpooling1D Max 2 2 (10,50)
Conv1D 50 ReLU 4 1 (7,50)
Maxpooling1D Max 2 2 (3,50)
Flatten (None, 150)
FC1 ReLU (None, 50)
FC2 Softmax (None, 2)
Table 8. Results of the proposed CNN-ELM.
Table 8. Results of the proposed CNN-ELM.
Data balancing method Undersampling Accuracy 0.89
Precision 0.9
Recall 0.87
F-score 0.88
Oversampling Accuracy 0.88
Precision 0.93
Recall 0.81
F-score 0.87
GAN Accuracy 0.94
Precision 0.9
Recall 0.98
F-score 0.94
Table 9. hyperparameters
Table 9. hyperparameters
Bath Size Epoch Patience in early stopping Adam
α β 1 β 2 ϵ
128 5000 500 0.001 09 0.009 1.0 × 10 − 8
Table 10. Results of the average accuracy.
Table 10. Results of the average accuracy.
Fiber Number of training data Number of testing data Average accuracy
105 6300 2100 92.8%
200 6300 2100 96.6%
Table 11. The number of each data type .
Table 11. The number of each data type .
Event Type I II III IV V
Training Set 307 1122 1101 1237 748
Validation Set 77 280 275 310 187
Total Number 384 1402 1376 1547 935
Table 12. The common CNNs performance.
Table 12. The common CNNs performance.
Model Name Training Model Size (MB) Speed (step/s) Classification Accuracy (%) Top 2 (%)
LeNet 39.3 90.9 60 86.5
AlexNet 554.7 19.6 94.25 99.08
VggNet 1638.4 2.53 95.25 100
GoogLeNet 292.2 4.1 97.08 99.25
ResNet 282.4 7.35 91.9 97.75
Table 13. the classification accuracy achieved for the five events.
Table 13. the classification accuracy achieved for the five events.
Type of Accuracy I II III IV V
Accuracy (%) 98.02 98.67 100 92.1 95.5
Top 2 accuracy (%) 100 100 100 99 100
Table 14. The comparison between the optimized model and Inception-v3.
Table 14. The comparison between the optimized model and Inception-v3.
Network Accuracy % Top 3 Accuracy % Training speed Model size (MB)
The optimized network 96.67 99.75 35.61 20
Inception-v3 97.08 99.25 4.35 292.2
Table 15. Structures of MLP-AE.
Table 15. Structures of MLP-AE.
Composition Type Input size Output size
Encoder Linear 140 32
Linear 32 4
Decoder Linear 4 32
Linear 32 140
Table 16. Structures of MLP-VAE.
Table 16. Structures of MLP-VAE.
Composition Type Input size Output size
Encoder Linear 140 64
Linear Reparameterization 64 4
Decoder Linear 4 64
Linear 64 140
Table 17. Structures of LSTM-AE.
Table 17. Structures of LSTM-AE.
Composition Type Input Output
Encoder LSTM Repeat 1 32
Decoder LSTM 32 32
Dense 32 1
Table 18. Structures of CNN-AE.
Table 18. Structures of CNN-AE.
Composition Type Input channels Output channels Kernel size/ Stride/ Padding
Encoder
Conv. 1 16 5/2/1
ReLU
Maxpool 2/2/0
Conv. 16 8 5/2/1
ReLU
Maxpool 2/2/0
Conv. 8 2 3/1/1
ReLU
Maxpool 2/2/0
Decoder Conv-Transpose 2 16 5/4/0
ReLU
Conv-Transpose 16 8 5/4/0
ReLU
Conv-Transpose 8 1 3/2/1
Table 19. A Summary of surveyed DL for optical sensor applications
Table 19. A Summary of surveyed DL for optical sensor applications
No Applications Finding References
1 • Realizing an optical fiber curvature sensor. • A large number of specklegrams have been detected from facet of multimode fiber (MMF) automatically in the experiments. 94.7% [88]
2 Detection of a highspeed railway track 97.91% [9]
3 Solving the problems of inability of the conventional hybrid structure to locate in the near field and the flawed frequency response. 97.83%. [97]
4 Extracting correlation of time-frequency sequence from signals and spectrograms to improve the robustness of the recognition system. 96.02%  [98]
5 Describing the situation of long-distance oil–gas PSEW systems. 99.26% and 97.20% [101]
6 Event detection. 99.91%  [105]
7 Discriminate between ballistocardiogram (BCG) and non-BCG signals. 94%, 90%, 98%, and 94% [14]
8 Recognize man-made threat event 97.73% [106]
9 Bridge structure damage detection. 96.9% [11]
10 An intrusion pattern recognition model 97.67% [113]
11 Bending recognition model using the analysis of MMF specklegrams with diameter being and 200 μ m . 92.8% and 96.6% [115]
12 Demonstrate the capability for the identification of specific species of pollen from the backscattered light. 97% [124]
13 Event recognition. 99.75% [13]
14 Identification and classification of external intrusion signals in the real environment using a 33km optical fiber sensing system. 97% [8]
15 Detection and localization of seismic events. 94% [12]
16 Identification of six activities, walking, digging with a shovel, digging with a harrow, digging with a pickaxe, strong wind and facility noise. 93% [129]
17 Temperature sensing based on sapphire crystal optical fiber (SOF). 99% [134]
18 Sensing water immersion length measurements and air temperature. N/A [135]
19 Detect water-level anomalies and report abnormal situation. 99.9% [136]
20 Collect the temperature data along the optic fiber set and figure out the anomaly detected temperature at the early phase. 98% [137]
21 detection of the defects on large-sized PCBs and measure their copper thickness before the mass production process [133]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Alerts
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2025 MDPI (Basel, Switzerland) unless otherwise stated