Preprint
Communication

Mineral Identification based on Out-Of-Distribution Detection

Altmetrics

Downloads

77

Views

28

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

06 May 2024

Posted:

07 May 2024

You are already at the latest version

Alerts
Abstract
Deep learning has increasingly been employed to identify minerals. However, deep learning can only be used to identify minerals in the distribution of the training set, while any mineral outside the spectrum of the training set is inevitably categorized erroneously within a predetermined class from the training set. To solve this problem, the study introduces the approach that amalgamates One-Class Support Vector Machines (OCSVM) with the ResNet architecture for the out-of-distribution mineral detection. Initially, ResNet undergoes training using a training set comprising well-defined minerals. Subsequently, the earlier layers of the trained ResNet are employed to extract the discriminative features of the mineral under consideration. These extracted mineral features then become the input for OCSVM. When OCSVM discerns the mineral in the training set's distribution, it triggers the subsequent layers within the trained ResNet, facilitating the accurate classification of the mineral into one of the predefined categories encompassing the known minerals. In the event OCSVM identifies the mineral out of the training set's distribution, it is unequivocally categorized as an unclassified or 'unknown' mineral. Empirical results substantiate the method's capability to identify out-of-distribution minerals while concurrently maintaining a commendably high accuracy rate for the classification of the 36 in-distribution minerals.
Keywords: 
Subject: Environmental and Earth Sciences  -   Geophysics and Geology

1. Introduction

Rocks serve as the foundational constituents of the Earth and record the evolutionary narrative of our planet. They hold a pivotal role within the multidisciplinary realm of Earth sciences. As rocks are composed of a variety of minerals, the accurate identification of minerals is of paramount importance [1,2]. Traditional mineral identification techniques primarily rely on the visual observation of physical properties like shape, color, and texture, but their precision is contingent upon the expertise of the observer [1,2]. Alternatively, although methods such as chemical analysis, X-ray diffraction analysis, differential thermal analysis, and polarizing microscope analysis offer enhanced accuracy in mineral identification, the defects of these methods are expensive, a long time to execute, and especially sample damage [3,4,5,6,7]. In contrast to these resource-intensive approaches, the acquisition of mineral images is an expedient, efficient, and cost-effective avenue for analysis. Consequently, an increasing body of research has begun to pivot towards mineral identification through image-based techniques.
In particular, numerous studies have harnessed the potential of deep learning to identify minerals from images, yielding commendable results [3,4,5,6,7]. However, it is crucial to underscore the intrinsic limitation of traditional deep learning methodologies, as they can exclusively identify minerals within the purview of the training dataset's distribution. Any mineral falling beyond the confines of this distribution is erroneously categorized within one of the predefined classes from the training datasets—an evident and undesirable misclassification. This limitation is exacerbated by the extensive diversity with more than 6000 known mineral categories worldwide [1], rendering it impractical to encompass all of them within the training datasets. Minerals that fall outside the scope of the training datasets necessitate distinct methods for isolation and identification, such as manual or instrumental techniques.
Current strategies addressing the inherent limitation of conventional deep learning models, specifically their ability to exclusively recognize in-distribution (ID) categories from the training set, encompass an array of techniques, including out-of-distribution (OOD) detection [8], uncertainty estimation [9], semi-supervised learning [10], and generative models [11]. Notably, OOD detection methodologies have emerged as particularly reliable, affording accurate predictions for samples existing outside the training set distribution and necessitating solely in-distribution data for training [12,13,14,15,16].
Exemplifying the efficacy of OOD detection, Jiang et al. [17] adeptly employed this technique to discern between known and unknown instances of plant diseases, while Saadati et al. [18] similarly conducted OOD detection to bolster the robustness of insect classification models. Furthermore, the utility of OOD detection extends beyond these domains, showcasing notable promise in the arenas of medical image diagnosis [19], network security [20], and quality control [21]. In light of these compelling precedents, it becomes evident that the isolation and identification of out-of-distribution minerals require specific attention. The main contributions of the paper are as follows:
(1)
OOD detection is adopted for the identification of minerals residing outside the training set's distribution, providing an opportunity for further identification of these instances.
(2)
A machine learning model that combines One-Class Support Vector Machines (OCSVM) with ResNet is designed for mineral identification.
(3)
Comprehensive experiments show the high performance of the proposed model.

2. Datasets

In this study, we collect a comprehensive dataset of 183,688 mineral images, encompassing 36 distinct categories of common minerals, as detailed in Table 1. These images were meticulously curated, drawing from the diligent efforts of Zeng et al. [6] and Wu et al. [3], and sourced from the reputable repository of mineral data, Mindat.org [22]. Notably, the dataset is divided into training, validation, and testing subsets, each allocated in a ratio of 8:1:1, respectively. Some of the 36 catagories of the mineral images are shown in Figure 1. In addition to the in-distribution dataset, a separate collection of 18,368 mineral images is amassed. These images correspond to 15 categories of minerals, as cataloged in Table 2, and have been acquired from the same authoritative source, Mindat.org. This auxiliary dataset, representative of out-of-distribution minerals, has been assembled to assess the model's proficiency in recognizing and distinguishing mineral types beyond the purview of the training set. Some of the images of the out-of-distribution minerals are shown in Figure 2.

3. Methodology

The methodology employed for mineral identification is illustrated in Figure 3. To discern minerals that fall outside the established set of 36 known minerals, One-Class Support Vector Machines (OCSVM) is leveraged for Out-of-Distribution (OOD) detection. Similar to the techniques outlined in previous works [23,24,25], the process initiates with feature extraction from the mineral image, with the intent of refining and augmenting the efficacy of OCSVM [23,24,25]. Crucially, a Deep Neural Network (DNN) is integrated into our model for the extraction of mineral-specific features. The DNN is meticulously trained on the training set, which comprises the 36 recognized mineral categories as shown in Table 1. Subsequently, OCSVM is deployed, with the mineral features derived from the initial layers of the DNN serving as input. This pivotal step serves to ascertain whether the mineral in question pertains to the in-distribution category of the 36 known minerals or falls into the realm of out-of-distribution. Upon OCSVM's determination that the mineral is classified as out-of-distribution, the model promptly halts and apprises the user that the input image represents an unknown mineral. In contrast, when OCSVM identifies the mineral as in-distribution, the model seamlessly proceeds to deploy the remaining layers of the DNN to apprise the user of the specific known mineral category to which the input image belongs.

3.1. Mineral Feature Extraction

The mineral feature extraction process capitalizes on the remarkable image classification capabilities of ResNet, a convolutional neural network architecture with a proven track record [26]. Assuming the feature extracted by ResNet is written as f R W × H × C (W is the width, H is the height, and C is the number of channels of the feature extracted). To enhance the performance of OCSVM in the context of OOD detection, a pivotal dimensionality reduction step is introduced. This process is elucidated by Formula (1), which involves the concatenation of individual channel values, q k to create a more concise representation of f . Each q k corresponds to the value derived from the kth channel within the mineral feature map, as elucidated in Formula (2). This dimensionality reduction facilitates the OOD detection process and bolsters the overall performance of the model.
q = q 1 , q 2 , , q c ,
q k = 1 W × H i = 1 W j = 1 H f i j k , k 1 , C

3.2. OOD Detection by OCSVM

To ascertain whether an input image pertains to the in-distribution category of the 36 known minerals, the mineral-specific features x extracted from the DNN, are provided as input to the OCSVM. These features undergo a crucial transformation, being mapped to a higher-dimensional space, as outlined in Formula (3).
f x = s g n ( i = 1 n α i K x i , x ρ )
The classification outcome for the input image hinges on the result of Formula (3): if this result surpasses zero, the image is identified as an in-distribution mineral; conversely, if the result is less than or equal to zero, the image is categorized as an out-of-distribution mineral. In Formula (3), s g n designates the sign function, xi corresponds to the features derived from the ith known mineral training data. K x i , x represents the Radial Basis Function (RBF), as expounded in Formula (4), responsible for the transformation of the known mineral training data into a higher-dimensional space with the objective of maximizing the separation between these training data points and the origin within that space. The parameters α i and ρ are determined through the training process using the known mineral training datasets.
K x i , x j = e x p x i x j 2 2 σ 2 , σ R
In Formula (4), the parameter denoted as σ represents the bandwidth, a pivotal factor governing the behavior of the Radial Basis Function (RBF). The significance of σ within this context is notably profound, as its magnitude inherently influences the classification process. Specifically, a larger value of σ tilts the balance toward categorizing a greater number of in-distribution samples as out-of-distribution, while conversely, a smaller σ biases the model toward classifying a greater proportion of out-of-distribution samples as in-distribution. In alignment with prior research and in accordance with established convention, the present study maintains σ at the value 1/|x|. It is essential to underscore that |x| in this context designates the feature dimension.

4. Experimental Results and Analysis

The model's implementation is facilitated through the utilization of the Python programming language, executed on a Linux environment, while drawing upon the robust framework provided by Keras, Tensorflow, and Sklearn. In pursuit of optimal efficiency during the DNN training process, a GPU (Graphics Processing Unit) is judiciously employed. The precise specifications of the experimental configuration are comprehensively detailed in Table 3 for reference.

4.1. Evaluation Metrics

The evaluation of the model's performance hinges on two key metrics: OOD Detection Accuracy and Mineral Identification Accuracy. These metrics serve as crucial indicators of the model's proficiency in its respective tasks. OOD Detection Accuracy, a binary classification metric, assesses the model's effectiveness in distinguishing whether a mineral is in-distribution or out-of-distribution. This metric includes three essential components: ID Accuracy, OOD Accuracy, and Overall Accuracy, which are calculated as that in Formula (5), (6) and (7). ID Accuracy gauges the ratio of correctly identified in-distribution minerals to the total known mineral test datasets. Conversely, OOD Accuracy quantifies the ratio of correctly identified out-of-distribution minerals to the overall count within the unknown mineral datasets. Notably, the Overall Accuracy mirrors the average of ID Accuracy and OOD Accuracy, given that the known and unknown mineral test data are maintained at equal proportions in this study. Mineral Identification Accuracy, a metric applicable to multi-class classification, evaluates the model's capacity to correctly identify minerals within their respective categories. This metric, akin to OOD Detection Accuracy, contains the trio of ID Accuracy, OOD Accuracy, and Overall Accuracy, but focuses on the performance of the model in identifying the concrete categories of in-distribution and out-of-distribution minerals. These rigorous and multifaceted metrics offer a comprehensive assessment of the model's performance in distinguishing between mineral categories and detecting minerals that deviate from the established training datasets.
ID   Accuracy = c o r r e c t l y   i d e n t i f i e d   i n d i s t r i b u t i o n   m i n e r a l s t o t a l   k n o w n   m i n e r a l   t e s t   d a t a s e t
OOD   Accuracy = c o r r e c t l y   i d e n t i f i e d   o u t o f d i s t r i b u t i o n   m i n e r a l s t o t a l   u n k n o w n   m i n e r a l   d a t a s e t
Overall   Accuracy = c o r r e c t l y   i d e n t i f i e d   m i n e r a l s t o t a l   m i n e r a l   d a t a s e t

4.2. Mineral Features Selection

As expounded in Section 3, the mineral features are meticulously extracted by the well-trained ResNet prior to OCSVM detection. In the case of ResNet50, a total of 49 mineral features can be derived from this process. To ascertain the optimal mineral features for OCSVM OOD detection, each of the 49 sets of features is independently subjected to OCSVM analysis, yielding 49 distinct accuracy values. The culmination of this analysis is graphically presented in Figure 4, showcasing the Overall Accuracy associated with each mineral feature extracted by the 49 layers of ResNet.
Upon careful examination of Figure 4, it becomes evident that the mineral features extracted by the second layer of ResNet50 emerge as the most promising, attaining a remarkable Overall Accuracy of 82.1%. Consequently, the features derived from the second layer of ResNet50 are judiciously chosen as the prime candidates for OCSVM-based OOD detection, given their demonstrably robust performance.

4.2. Performance

Table 4 presents a comprehensive overview of the OOD Detection Accuracy and Mineral Identification Accuracy, offering profound insights into the model's performance. Notably, this analysis reveals that the model excels in its ability to correctly identify 82.1% of the test minerals as either known or unknown categories, with 96.4% accuracy achieved in discerning in-distribution test minerals as known categories. Moreover, 67.8% of the out-of-distribution test minerals are adeptly classified as unknown categories, substantiating the model's competence in addressing the challenge of minerals that deviate from the training set. As highlighted in the introduction section, contemporary mineral image identification methods are often constrained to categorize minerals within the bounds of the training set's distribution, leading to erroneous identifications of out-of-distribution minerals. In this context, the model distinguishes itself by achieving a 67.8% accuracy in classifying out-of-distribution minerals as unknown categories. This OOD Accuracy is lower than that of other applications listed in references [17,18,19,20,21] because minerals of the same category may have different colors and textures, while different categories of minerals may have the same colors and textures [6]. This makes mineral identification more challenging, resulting in similarly lower ID Accuracy than other applications. The model attains a commendable 74.1% accuracy in identifying in-distribution minerals through the utilization of the state-of-the-art convolutional neural network, ResNet. The performance of each of the 36 known mineral categories is presented in Figure 5, affording a granular understanding of the model's accuracy across distinct mineral types.
Additionally, a comparative analysis with other related studies, detailed in Table 5, underscores the model's superiority. Compared with the study of Zeng et al. [6], which employed the same dataset of 36 known minerals, the model exhibits marginally lower ID Accuracy but substantially higher OOD Accuracy. Notably, the model surpasses other related studies in OOD Accuracy, highlighting its proficiency in mineral identification tasks beyond the training set's confines.

5. Conclusions

A novel model designed to excel in the task of identifying out-of-distribution minerals, harnessing the combined capabilities of OCSVM and the ResNet50 network is introduced. OCSVM plays a pivotal role in classifying mineral features extracted through ResNet50, endowing the model with the capacity to detect both out-of-distribution and in-distribution minerals. In comparison to traditional methods reliant on labor-intensive and time-consuming experimental mineral species determination, the approach emerges as a more practical, expedient, and cost-effective alternative. Additionally, when contrasted with other conventional deep learning methodologies, the model exhibits the unique capability to differentiate out-of-distribution minerals, addressing a critical limitation in the field of mineral identification. Further expanding the in-distribution datasets would enhance the model's performance and its broader applicability in the field of mineral identification.

Author Contributions

Conceptualization, X.J. and Y.Y.; methodology, Y.Y.; software, Y.Y. and K.L.; validation, M.Y., M.H. and Z.Z.; formal analysis, X.J. and Y.Y; investigation, Y.Y.; resources, Y.Y. and K.L.; data curation, Y.Y.; writing—original draft preparation, X.J. and K.L.; writing—review and editing, Z.Z., S.Z. and Y.W.; visualization, K.L.; supervision, X.J. and Z.Z.; project administration, X.J.; funding acquisition, X.J. M.Y. and M.H.. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Program of National Mineral Rock and Fossil Specimens Resource Center from MOST, grant number NCSTI-RMF20240109.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author, X.J.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Nesteruk, S.; Agafonova, J.; Pavlov, I. et al. MineralImage5k: A benchmark for zero-shot raw mineral visual recognition and description. Computers and Geosciences, 2023, 178: 105414.
  2. Lou, W.; Zhang, D.; Bayless, R. C. Review of mineral recognition and its future. Applied Geochemistry, 2020, 122, 104727. [Google Scholar] [CrossRef]
  3. Wu, B.; Ji, X.; He, M.; Yang, M.; Zhang, Z.; Chen, Y.; Wang, Y.; Zheng, X. Mineral Identification Based on Multi-Label Image Classification. Minerals, 2022, 12(11), 1338. [CrossRef]
  4. Singh, T.; Jhariya, D.C.; Sahu, M.; Dewangan, P.; Dhekne P.Y. Classifying Minerals using Deep Learning Algorithms. Earth and Environmental Science, 2022, 1032(1), 012046. [CrossRef]
  5. Jia, L.; Yang, M.; Meng, F.; He M.; Liu, H. Mineral Photos Recognition Based on Feature Fusion and Online Hard Sample Mining. Minerals, 2021, 11(12), 1354. [CrossRef]
  6. Zeng, X.; Xiao, Y.; Ji, X.; Wang, G. Mineral Identification Based on Deep Learning That Combines Image and Mohs Hardness. Minerals, 2021, 11(5), 506. [CrossRef]
  7. Liu, C.; Li, M.; Zhang, Y.; Han, Y; Zhu, Y.; An enhanced rock mineral recognition method integrating a deep learning model and clustering algorithm. Minerals, 2019, 9(9),516. [CrossRef]
  8. Yang, J.; Zhou, K.; Li, Y.; Liu, Z. Generalized Out-of-Distribution Detection: A Survey. arXiv, 2022, 2110, 11334. [Google Scholar] [CrossRef]
  9. Loquercio, A.; Segu, M.; Scaramuzza, D. A General Framework for Uncertainty Estimation in Deep Learning. IEEE Robotics and Automation Letters, 2020, 5(2), 3153-3160. [CrossRef]
  10. Van Engelen, J.E., Hoos, H.H. A survey on semi-supervised learning. Machine Learning, 2020, 109(2), 373-440. [CrossRef]
  11. Cai, M., Li, Y.; Out-of-Distribution Detection via Frequency-Regularized Generative Models. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, 5521-5530. [CrossRef]
  12. Hsu, Y. C.; Shen, Y.; Jin, H.; Kira, Z. Generalized odin: Detecting out-of-distribution image without learning from out-of-distribution data. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, 10951-10960. [CrossRef]
  13. Liang, S.; Li, Y.; Srikant, R. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. arXiv, 2017, 1706, 02690. [Google Scholar] [CrossRef]
  14. DeVries, T.; Taylor, G.W. Learning Confidence for Out-of-Distribution Detection in Neural Networks. arXiv, 2018, 1802, 04865. [Google Scholar] [CrossRef]
  15. Bendale, A.; Boult, T. E. Towards open set deep networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, 1563-1572. [CrossRef]
  16. Hendrycks, D.; Gimpel, K. A Baseline for Detecting Misclassified and Out-of-Distribution Examples. arXiv, 2016, 1610, 02136. [Google Scholar] [CrossRef]
  17. Jiang, K.; You, J.; Dorj, U.; Kim, H. Detection of unknown strawberry diseases based on OpenMatch and two-head network for continual learning. Frontiers in Plant Science, 2022, 13, 989086. [Google Scholar] [CrossRef]
  18. Saadati, M.; Chiranjeevi, S.; Balu, A.; Jubery, T.Z.; Asheesh, K.S.; Soumik, S.: Arti, S.; Baskar G. Out-of-distribution algorithms for robust insect classification. 2nd AAAI Workshop on AI for Agriculture and Food Systems, 2023. [CrossRef]
  19. Zhang, O.; Delbrouck, JB.; Rubin, D.L. Out of Distribution Detection for Medical Images. In: Sudre, C.H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis. UNSURE PIPPI 2021 2021. Lecture Notes in Computer Science, vol 12959. Springer, Cham. [CrossRef]
  20. Mattei, E.; Dalton, C.; Draganov, A. Feature Learning for Enhanced Security in the Internet of Things. 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP). IEEE, 2019, 1-5. [CrossRef]
  21. Lindgren, E.; Zach, C. Autoencoder-Based Anomaly Detection in Industrial X-ray Images. Quantitative Nondestructive Evaluation. American Society of Mechanical Engineers, 2021, 85529, V001T07A001. [CrossRef]
  22. A Mineral Database. Available online: https://www.mindat.org/. (accessed on 5 May 2024).
  23. Erfani, S. M.; Rajasegarar, S.; Karunasekera, S.; Leckie, C. High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning. Pattern Recognition, 2016, 58, 121–134. [Google Scholar] [CrossRef]
  24. Widodo, A.; Yang, B. S.; Han, T. Combination of independent component analysis and support vector machines for intelligent faults diagnosis of induction motors. Expert systems with applications, 2007, 32(2), 299-312. [CrossRef]
  25. Shen, K.; Ong, C.; Li, X., et al. Feature selection via sensitivity analysis of SVM probabilistic outputs. Machine Learning 2008, 70, 1–20. [CrossRef]
  26. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, 770-778. [CrossRef]
Figure 1. Examples of in-distribution/known minerals.
Figure 1. Examples of in-distribution/known minerals.
Preprints 105709 g001
Figure 2. Examples of out-of-distribution/unknown minerals.
Figure 2. Examples of out-of-distribution/unknown minerals.
Preprints 105709 g002
Figure 3. The architecture of the model proposed in the paper.
Figure 3. The architecture of the model proposed in the paper.
Preprints 105709 g003
Figure 4. Overall Accuracy associated with each mineral feature extracted by Layer 1 to 49 of ResNet50.
Figure 4. Overall Accuracy associated with each mineral feature extracted by Layer 1 to 49 of ResNet50.
Preprints 105709 g004
Figure 5. Accuracy of the 36 known category minerals.
Figure 5. Accuracy of the 36 known category minerals.
Preprints 105709 g005
Table 1. Mineral names and number of samples in in-distribution/known category datasets.
Table 1. Mineral names and number of samples in in-distribution/known category datasets.
#No. mineral quantities #No. mineral quantities
1 agate 3,225 19 hematite 5,728
2 albite 1,775 20 magnetite 2,445
3 almandine 2,018 21 malachite 6,796
4 anglesite 1,797 22 marcasite 1,608
5 azurite 7,924 23 opal 3,197
6 beryl 8,957 24 orpiment 720
7 cassiterite 3,205 25 pyrite 8,769
8 chalcopyrite 3,253 26 quartz 34,883
9 cinnabar 1,605 27 rhodochrosite 4,276
10 copper 5,288 28 ruby 820
11 demantoid 755 29 sapphire 996
12 diopside 1,586 30 schorl 2,099
13 elbaite 5,439 31 sphalerite 6,354
14 epidote 3,720 32 stibnite 2,475
15 fluorite 26,336 33 sulphur 1,890
16 galena 6,188 34 topaz 3,577
17 gold 4,545 35 torbernite 1,100
18 halite 756 36 wulfenite 7,583
Total 183,688
Table 2. Mineral names and number of samples in the out-of-distribution/unknown category datasets.
Table 2. Mineral names and number of samples in the out-of-distribution/unknown category datasets.
#No. mineral quantities #No. mineral quantities
1 adularia 759 9 moissanite 10
2 aegirine 918 10 niccolite 256
3 amber 1,478 11 nitratine 10
4 aragonite 4,020 12 ozocerite 26
5 biotite 1,478 13 selenium 108
6 boracite 241 14 turquoise 991
7 goethite 4,176 15 whewellite 106
8 gypsum 4,950
Total 18,368
Table 3. Experimental configuration.
Table 3. Experimental configuration.
Configuration Settings
Programming Language Python 3.6.9
Keras Keras 2.6.0
Tensorflow Tensorflow 2.1.0
Sklearn Scikit-learn 0.24.2
GPU Tesla P100-PCIE
GPU Toolkit Version CUDA 10.0
Table 4. Accuracy of our mineral identification model combing OCSVM and ResNet50.
Table 4. Accuracy of our mineral identification model combing OCSVM and ResNet50.
Accuracy (%) ID OOD Overall
OOD Detection 96.4 67.8 82.1
Mineral Identification 74.1 67.8 71.0
Table 5. Comparisons of Mineral Identification Accuracy with other studies.
Table 5. Comparisons of Mineral Identification Accuracy with other studies.
Model Study Number of Known Mineral categories Accuracy (%)
ID OOD
Inception-v3 [7] 12 73.1 0
ResNet-50 [5] 14 88.0 0
EfficientNet-b4 [6] 36 78.3 0
ResNet50 this study 36 76.6 0
OCSVM+ResNet50 this study 36 74.1 67.8
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated