In the world, and especially in the geographical area of Europe, the problem of munitions, chemical weapons, and shipwrecks with fuel in tanks dumped in water reservoirs is becoming an increasingly urgent problem to solve. Particularly after World War II, munitions and chemical weapons remaining after the war, which various countries wanted to get rid of in the cheapest possible way, were sunk in masse. It is estimated that in the Baltic Sea there may be at least 40 thousand tons of the war remnants mentioned above [1]. Although two main areas for dumping weapons were selected, the Gotland Deep and the Bornholm Deep, a significant part of the weapons was dumped in unidentified places. Currently, due to progressing corrosion, threatening unsealing of weapons and release of poisonous substances into the environment, and the intensive installation works carried out in maritime areas, they force the development of effective and efficient methods to identify dangerous objects.

The task of searching for dangerous objects in the aquatic environment, in particular unexploded ordnance (UXO), requires scanning the surveyed area with various types of sensors (magnetometers, sonars, video cameras, although in this article we focus only on magnetometers). In the next step, specialists analyze the collected data and mark places containing potentially dangerous objects. These sites are subject to further inspection or extraction work carried out by sapper divers. Their work is very expensive because of the limited number of such experts, as well as the limited time possible to spend under the water.

The scanning process is often conducted by a set of magnetometers (usually 2 or more) mounted on the wing pulled behind the ship or mounted on a formation of the AUVs scanning the interest area in parallel. An example of an AUV is a self-constructed vehicle shown in Figure 1.

In order to optimize the above process, one of the possibilities is to apply machine learning methods for the automatic identification of dangerous objects in particular UXO [2,3]. Considering that the scanned areas are usually calculated in square kilometers, automatization of data analysis should speed up the process. Even more, it is often possible to overcome the performance of human experts by more accurate classification of underwater objects by the use of machine learning models [4]. However, there is one potential weakness in the application of machine learning methods, which is the requirement for a large amount of good-quality training data. This issue is becoming even more important for the marine environment, where the collection of such data is associated with very high costs, labor-intensive and time-consuming. Additionally, the popular approaches for machine learning applications for UXO detection are based on electromagnetic induction (EMI) which is not applicable in underwater exploration, where the electromagnetic field is strongly attenuated. In that scenario, only magnetometers based on the magneto-static effect are useful, although they are much harder to correctly identify UXO from nonUXO objects [5].

In order to overcome the problem of lack of magnetometer-based data, we propose to build a full-feature digital twin of the UXO objects and their surroundings. Initially, a similar approach was already taken in [6] but the authors ignored remanent magnetization, which is in particular important within the water environment. Moreover, in [6] the authors have compared a commonly used dipole model of UXO objects which does not provide sufficiently good accuracy. Therefore, in our work, we propose a UXO model that is based on a digital twin modeled with remanent magnetization using the finite element method. This approach match more accurately simulates the shape of real ammunition are real signals. The quality of the model is calibrated using real measurements. Obtaining a validated model allows us to conduct a series of simulations to achieve full training datasets that contain simulations of a passage of a group of probes equipped with magnetometers over potential UXO-based scans. This approach faces several issues:

In this article we provide a detailed explanation of the process of developing such a digital twin and how we overcome the issues presented above. It must be noted that in the physical experiments conducted to validate the numerical model, we used a surrogate UXO object which was a pipe with a length of 343mm and 75mm of the diameter corresponding to the typical ammunition. The term typical ammunition is defined as ammunition considered dangerous while conducting work at the bottom of water reservoirs. The pipe was used for safety reasons and allowed us to carry out the experiments without the supervision of sappers. Similarly, in numerical experiments, the UXO object was modeled with a pipe, allowing to confirm and validate the model. When the numerical model is calibrated, the resultant dataset generated with the digital twin would enable us to verify whether it is possible to classify UXO / nonUXO objects based on the disturbance of the earth’s static magnetic field. Therefore, in the virtual environment, when generating the training set, we model not only the pipe under consideration, but also other pipes of different lengths and diameters that are too big or too small for UXO and other objects like ferromagnetic sheets, rings, etc.

The novelty of this article is:

The structure of this article is as follows: In Section 2 we provide the state-of-the-art in UXO identification and classification including digital twin models. Then in Section 3 we describe the mathematical model of the virtual environment, also introducing simplification of the model. In Section 4 an experiments comparing the digital twin and the physical model are discussed. The results of the experiments are provided in Section 5. The next section describes the details of how we created the training set. The last section concludes the manuscripts and summarizes the results obtained.

The subject of this article is a popular real-life problem that affects almost all countries. There are several topics that are important in terms of UXO classification. The first is the development of better sensors. For example, in [7] the authors indicate that the source of the noise in the transient electromagnetic sensors is mainly caused by the receiving coil where the noise is dominated by the internal thermal noise of the damping resistor. Reducing the bandwidth of the system and increasing the size of the coil effectively reduces internal noise. Another group of sensors constitutes magnetometers where two are very popular, the fluxgate magnetometers [8] which have the advantage of delivering a three-dimensional vector of the magnetic field, and optically pumped magnetometers (OPM) which gives more information at greater depth/height [9], although they return only the module of the signal. The OPMs are also under development and some recent developments include high-power off-resonant optical pumping; Mz configuration, where pumping light and magnetic field of interest are oriented parallel to each other; use of small alkali metal vapor cells of identical properties in integrated array structures. Another example of OPM optimization can be found in [10] where after sensor optimization the authors obtained a radio-optical cesium magnetometer sensitivity of 82.5 fT/$\sqrt{\mathrm{Hz}}$, a resolution within 1 pT to an external magnetic field change, and a noise fluctuation of 0.2 pT for an integration time of 1 s. In addition to the sensors, an important stage is the recorded signal processing. There are several approaches to UXO identification. A nice overview can be found in a report [11] in which the authors provide general approaches. Among them, they identify two main approaches. The first method is based on fitting measured magnetic data neither TDEM, nor FDEM to a parametric model, and then using the recovered model parameters to classify the anomaly as UXO or non-UXO. Here, usually, a magnetic dipole model is used as a reference. The second approach is based on matching the measured data to the signature of known UXO and other objects. The authors also briefly discuss an approach based on SVM machine learning classification based on the parameters obtained after matching the signals to the model. The properties of the dipole model were extensively evaluated by many researchers. For example, in [6] the authors study the properties of the physical dipole, which are then compared favorably with alternative models, including the limited cases of prolate spheroids and other shapes. Additionally, in the work, the authors critically review the explicit modeling of the demagnetization properties of magnetic materials. A more advanced study on UXO numerical models was conducted in [12] where the authors compared the prolate spheroid model with two more realistic UXO geometries using a finite element method. The results obtained showed that the calculated dipole moment response for complex models that resemble actual UXO is up to 50% higher than the dipole moments for the prolate spheroid model. They also found that altering the shape of a model from a prolate spheroid to a complex shape has a greater effect on dipole moment than maintaining the same shape and altering the volume. Finally, they found that complex models more closely match actual field data than prolate spheroid models. An application of the recorded signals to the fitted model can be found in many works. For example in [13] the authors have used a Normalized Surface Magnetic Source (NSMS) model and a variant of the simple dipole model to the data recorded using electromagnetic induction (EMI). For discrimination, the authors used two sets of parameters: intrinsic parameters associated with the size, shape, and material composition of the target; and extrinsic parameters related to the orientation and location of the anomaly. They found that the discrimination performance significantly depends on the mathematical models: single dipole, multidipole, and NSMS. In particular, when the noise is low and the UXO is isolated, the basic methods work well, but with noise and multiple targets placed close to each other, the complete method is more attractive. Similar work can also be found in [14] and [15]. In the last, the authors tested the spheroid model of UXO objects and analyzed how the object behaves in terms of the height of the recorded magnetic field and the caliber of ammunition. The authors have also studied how shock demagnetization behaves. The problem of inverse can also be considered in terms of various approaches. For example, in [16] the authors considered two inversion approaches: cooperative or constrained inversion; and (2) joint inversion. Cooperative inversion is the process of using inversion parameters from one dataset to constrain the inversion of other data. In a true joint inversion, the target model parameters common to the forward models for each type of data are identified, and the procedure seeks to recover the model parameters from all the survey data simultaneously. Besides the inversion approaches, a key element is the direct inverse algorithm. For example, this topic was covered in [17] where the eigenvector decomposition of the magnetic gradient tensor was used to locate dipole-like magnetic sources, allowing automatic detection of dipole-like magnetic sources without estimating the magnetic moment direction. A similar approach has also been discussed in [18], where the authors proposed a new algorithm with a magnetic gradient tensor and singular value decomposition (SVD) to estimate the target position and characterization quickly and accurately. Another group of methods focuses on the use of electromagnetic data and the application of machine learning models [5]. In that article, the authors point out that "(...) magnetic data can only provide limited information about intrinsic target properties (i.e., size and shape) and are rarely used to classify detected targets as UXO and non-UXO." Therefore, most machine learning applications focus on EMI data. One of the early approaches to utilize machine learning methods, in particular the Probabilistic Neural Networks can be found in [19]. In [4] the authors discuss the concept of using linear genetic programming for UXO/nonUXO classification. A broader work that discusses many machine learning approaches can be found in [20] where the use of SVM and Probabilistic Neural Networks is discussed. Additionally, the authors discuss feature extraction techniques; in particular, they used a combination of a size and time-decay vector. As a result of their work, they developed the UXOLab software. The next example of utilizing a machine learning model is presented in [21] where the authors combine supervised learning such as SVM and neural networks with unsupervised learning such as Gaussian Mixture Modelling. As a feature space, they use features extracted from the EMI decay curves of the physics-based intrinsic, effective dipole moment, called the total Normalized Surface Magnetic Source (NSMS). They found that such a combination provides a reduction in the amount of required training data and allows for a convenient probabilistic interpretation of the classification.

A similar approach to ours can be found in [22] where the authors use total field magnetic responses obtained using the finite element method to train various classifiers. In particular, they used Random Forest, Support Vector Machine, and Neural Networks, additionally using several types of labels, where the based performance was obtained when the classes were derived from a multiclass self-organizing feature map (SOFM). Although in their work the authors assumed the lack of remanent magnetization which may be of particular importance in terms of the underwater classification. Another example of using machine learning methods in UXO classification was discussed in [3] where the authors used data obtained from ground penetrating radar. The results obtained allowed them to achieve an accuracy that ranged between 89% and 92%. Above, we noted several times the problem of remanent magnetization, where it is often assumed that, as a result of the hitting shock, a demagnetization occurs. But what is raised by many authors is that remanent magnetization can not be ignored and is very important. In particular, as indicated in [23] the remanent magnetization may remain. Moreover, the conducted experiments indicated that the opposite effect can occur, that the missile initially started with a very small remanence acquired a magnetic remanence in the direction of the inducing field at the time of impact, and the magnetic remanence is stable for time scales of up to one thousand years. Finally, it must be noted that in underwater research, many dangerous objects were sunk, therefore shock demagnetization cannot be assumed and needs to be included in the model.

As mentioned in the introduction, one of the techniques used to detect dangerous objects is the study of the distortion of the earth’s magnetic field. This is a passive technique and requires the use of magnetic field sensors. As known, the earth is the source of the magnetostatic field, the distribution of the field on a global scale changes intensively (Figure 2), but on a local scale it can be assumed that the earth’s magnetic field is homogeneous, and the field lines are parallel to each other (Figure 2, Figure 2).

If the field force lines are not parallel, it means that there are objects (natural or anthropogenic) disturbing this distribution. To distort the distribution of the magnetic field (according to the formula 5), there must be a change in the magnetic permeability of the medium (change in magnetic permeability $\mu$) or an additional magnetic field must occur (magnetization M). It follows that the technique of studying changes in the earth’s magnetic field allows the detection of ferromagnetic objects. Theoretically, also diamagnetic or paramagnetic, but due to small differences in the value of relative magnetic permeability, the detection of paramagnetic and diamagnetic in the aspect analyzed in this article has no practical significance. An example of distortion of the magnetic field force lines by a ferromagnetic object is shown in Figure 3.

In order to create a digital twin of the UXO object two elements are needed these are a proper mathematical model of the system and an environment which will be used to apply this model.

The experiments consisted of 3 stages, which included: the construction of a mathematical-physical model, which has already been described in Section 3. Next, physical experiments were carried out involving the study of the UXO substitute with the use of magnetometers. The obtained results of the physical experiments were used to verify the quality of the mathematical model. In the next step, the mathematical model was optimized in order to reduce the computational complexity, without losing its performance. Below we present a description of each of these stages.

The first group of experiments was designed to achieve a full match between the digital twin and the physical experiments. First, we collected the data from the physical objects. For that purpose, the UXO object was scanned in different orientations and for different directions of scanning the object using AUV. In particular, we scanned in the following conditions:

For the performance calculation, the $R^2$ was used as the base reference performance metric because it combines the mean square error together with the level of noise. The obtained results for various performance metrics are shown in Table 4.

In the table, in almost all cases the obtained performances of $R^2$ are on the level of 0.99 or 0.95. For easier analysis of the results, the last column of the matrix indicates the amplitude of the signal, that is the difference between the maximum of the recorded physical signal and the minimum value of that signal. These results also indicate that the error rates nor RMSE or MAE are of order or two orders of magnitude smaller than the amplitude of the signal. The results indicate an almost perfect match. The quality of the match between the numerical model and the physical one is shown in Figure 8.

The only exception, that is $R^2=0.8724$ was obtained when UXO orientation was N-S and the scanning route was in E-W direction. This is also plotted in Figure 8f where it can be observed that the shape is almost right except for the magnitude of the signal. This can be explained by a small change in the orientation of the UXO object or E-W orientation that changes the active length of the object as shown in Figure 9.

Therefore, it can be concluded, that the digital twin very well matches the real physical signal.

The same calculations were also repeated for the cylinder substitute which allows for a significant reduction of the calculation time. In that case, we obtained similar results as presented in Table 5. Here, a small, insignificant increase in all error rates can be observed. Therefore, for the final training set creation, the cylinder substitute was used.

The results presented above allowed us to conclude that the digital twin can be used for creating a training set for the machine learning purpose. Here, appeared some additional benefits of the tandem of GMesh and GetDP. These two tools are controlled by a script, therefore it was very easy to automatize the process of generalization of the digital twin to produce many objects of different orientations. In particular, the great advantage of this solution is full control based on scripts over all stages of building the digital twin. In a single step of data generation, the following parameters were introduced to the model in the Gmsh/GetDP environment:

Figure 10 shows the scheme of generation of the training set. The process consists of two loops. In the main loop, a random generator provides orientations and sizes of modeled objects (in the range determined by the input parameters of the process). On this basis and the material parameters given at the input of the process, the Gmsh/GetDP model calculates the distribution of magnetic induction represented by a cloud of points. At this point, the inner subloop begins. The subgenerator randomizes the parameters of the probe route and, on this basis, the interpolator implemented in MATLAB simulates the magnetic field measurements during the passage of a group of probes. The scope of route randomization is also described by a set of input parameters of the generation process. After each flow simulation, another case is added to the training set.

For the final training set creation, most of the machine learning models need positive and negative cases (the so-called supervised learning). The final training set was obtained using a set of objects which are considered positive cases - UXO objects, and negative cases - non-UXO objects. The positive cases included substitute cylinders of predefined properties - the relation between the diameter and length of the cylinder, in particular, the length = 4 up to 6 diameters (that is the typical relationship between the length of the munition and its caliber). When the pipe has other properties it is considered a non-UXO object. In particular, as non-UXO objects, we considered munition that has too small caliber, less them 70mm, because it doesn’t cause significant danger for the works conducted on the bottom of the water reservoirs, and too large caliber> 200 mm and too long objects which were considered as simple long pipes - not a munition.

Additionally, as non-UXO objects also sheets of metal were considered. They had a thickness of 10mm to 30mm and different lengths and widths. The length and width were selected to have a similar mass to the UXO objects. The constructed model also allows the addition of two sources of magnetization of the objects. First, corresponds to the remanence magnetization resulting from the manufacturing or storage of the objects, and it can have any direction of the magnetization. The second source of magnetization depends on the time the object stayed at the bottom of the water reservoir. It results from the fact that many munitions were sunk right after the second world war, therefore they were magnetized by the Earth’s magnetic field. This allowed us to run around 1600 simulations of the digital twins of various objects and their parameters (some simulations failed due to meshing problems). Then for each digital twin, 30 random scanning routes were generated using the Matlab subroutine. In total, it resulted in around 48000 training samples. In the conducted simulations it was assumed that a single scanning route consisted of a formation of 9 virtual magnetometers parallel to each other as shown in Figure 11. These 9 parallel paths allow us to train and validate the prediction model in several scenarios. For example, by switching on and off particular routes we can simulate scenarios when a particular magnetometer is scanning on the side of the UXO object instead of the center, or to identify the effect of the distance between virtual magnetometers on the prediction quality.

In the manuscript, a process of creating a digital twin of the environment containing UXO and nonUXO objects was presented. The virtual environment was then optimized in terms of computational complexity and used to virtually scan the objects with virtual magnetometers. Such an environment has several advantages, it allows for easy collection of the training data for building machine learning models. It is safe, and do not require the supervision of the sapper, and allows for creating a massive training set in a range of several days. In our case, the calculations for creating the training set consisting of 46,000 samples took 3 days on a single two-processor machine (2xAMD EPIC processor and 512GB RAM), whereas collecting the data for the physical experiments to collect 8 scans took a single day.

In the manuscript, we have also shown how to simplify the model to speed up the calculations without sacrificing the quality of the obtained results. The proposed solution significantly reduces the number of nodes of the mash which resulted in reducing the calculation time by 1/3. In general, we have shown that the approach of building virtual environments for generating data for training machine learning models is not only possible for vision problems but can be also applied to other environments, although it requires careful tuning of the model to achieve results comparable to the one recorded with physical sensors. An example of such an environment is detecting the UXO objects in an underwater environment using magnetometer sensors. In the manuscript, we omit the problem of creating a machine learning model as it is the subject of another article and is out of the scope of this article.