Prerpints.org logo
Preprint
Article

Diagnostic Accuracy of Saliva-Based Testing as a Vibrio cholerae Surveillance Tool

Altmetrics

Downloads

134

Views

63

Comments

0

Submitted:

05 March 2024

Posted:

08 March 2024

You are already at the latest version

Alerts
Abstract
Saliva, as a diagnostic medium, offers a promising alternative to blood by virtue of its non-invasive collection, which enhances patient compliance, especially in paediatric and geriatric populations. In this study, we assessed the utility of saliva as a non-invasive medium for measuring V. cholerae-specific serum antibodies in naturally infected individuals. We tested paired serum and saliva samples obtained from a total of 63 cholera patients enrolled in a cohort study. Vibriocidal antibodies assay (IgM/IgG) as markers for accurate determination was used to determine cholera specific antibody levels. Using receiver operating characteristics (ROC) curve, we found that the best cut-off that maximizes (sensitivity + specificity) is 10 titers. At this saliva titer, the sensitivity is 76.9% (95%CI: 60.9%, 87.7%) and specificity is 80.0% (95%CI: 56.6%, 92.5%). Using Spearman’s correlation coefficient, we also found evidence of a positive correlation between Vibrio Cholerae saliva and serum antibodies (rho=0.66, p<0.001). In conclusion, saliva-based diagnostic cholera tests has high diagnostic accuracy, and would be advantageous, cheaper, and quicker for early diagnosis of severe cholera outcomes.
Keywords: 
Subject: Biology and Life Sciences  -   Immunology and Microbiology

1. Introduction

The global burden of cholera, primarily caused by the bacterium Vibrio cholerae (V. cholerae), continues to be a pressing public health challenge, particularly in regions with limited access to improved water and sanitation. The recent outbreak, which started in October 2023, has rapidly spread to over 50 districts. Approximately 20,843 cholera cases and close to 703 deaths have been reported per the national daily cholera update given by the Zambia National Public Health Institute. The high mortality and morbidity associated with classical cholera has had a tremendous tragic impact on the personal as well as social life of people living in the affected areas. This devasting impact of cholera on health systems, economy and social life require huge resources to be mobilised in a short space of time if the outbreak is to be quickly contained [1]. Traditionally, serum has been the most widely used biological sample for measuring immunoglobulin isotypes following exposure to natural infection and vaccination. This can be invasive, resource-intensive, and challenging to perform in field settings when compared with saliva [2]. There is the need for non-invasive, efficient, and accurate techniques to measure V. cholerae-specific antibodies, which could revolutionise surveillance, diagnosis, and vaccine efficacy studies.
Saliva, as a diagnostic medium, offers a promising alternative to blood by virtue of its non-invasive collection, which enhances patient compliance, especially in paediatric and geriatric populations. Previous studies have hinted at the potential of salivary antibodies as proxies for systemic immunity [3,4], yet there remains a gap in systematically validating this approach specifically for V. cholerae infections. This gap is especially long overdue as Jertborn and colleagues did show that may be useful for monitoring gut mucosal response to naturally acquired cholera [5]. Saliva addresses the logistical and ethical constraints associated with blood sampling; it also aligns with the increasing demand for point-of-care diagnostics that can be deployed widely, including in resource-limited settings.
The aim of this study is to assess the utility of saliva as a non-invasive medium for measuring V. cholerae-specific serum antibodies in naturally infected individuals. Our approach utilizes vibriocidal antibodies (IgM/IgG) as markers for accurate determination. By assessing the correlation between saliva titers and serum titers, we present compelling evidence showcasing the potential of measuring V. cholerae antibody levels in saliva samples as a reliable indicator of corresponding serum levels. Ultimately, our study sought to lay the groundwork for the broader application of salivary diagnostics in infectious disease surveillance, contributing to more effective and accessible public health interventions.

2. Materials and Methods

Study Design and Participants

This was a cohort study of individuals that contracted cholera and had been admitted to the cholera treatment centres in Eastern Province of Zambia. Participants were enrolled from cholera treatment centres in Chipata, Chipangali and Vubwi (near Malawi boarder). Participants were enrolled if they were aged between 18 to 65 years. A total of 63 cholera patients were identified and enrolled into the study. Both demographic and clinical data was obtained from all consenting individuals.

Biological Sample Collection

(a)
Blood
Approximately 20 mls of blood was collected in EDTA blood collection tubes by trained study personnel using the veinpuncuture procedure. Thereafter, samples were transported at 2 - 8°C to the lab for plasma separation. Plasma samples were stored at −20°C at the Centre for Infectious Diseases Research in Zambia (CIDRZ) Central laboratory (CIDRZ-CLab), Lusaka, Zambia until vibriocidal antibody assays were performed at the CIDRZ research laboratory. Blood samples were collected one at different stages of disease but within 0 and 20 days after confirmation of cholera on culture.
(b)
Saliva
The saliva was collected prior to collecting blood samples. Study participants were given mineral water to mouth rinse three times prior to saliva collection. After the mouth rinse, participants were asked to pull and spit saliva in a sterile saliva collection tube until ~5mls of saliva was collected. All saliva samples were kept at 2- 8 °C and transported to CIDRZ-CLab for long term storage at –20 °C until testing was done.
(c)
Vibriocidal immunological assay
The vibriocidal titers were determined using previously described vibriocidal methods with some modifications [6]. Zambian V. cholerae strains (Inaba -EDVRU/ZM/09-10 and Ogawa EDVRU/ZM/2016) were used as the target vibrio strains for these assays in Zambia and results were comparable to results using strains Inaba (T19479) and Ogawa (X25049). Briefly, serum and saliva were heat-inactivated at 56 °C for 30 minutes. Appropriate dilutions were made as shown in Table 1 below. V. cholerae strains, diluted samples and exogenous guinea pig complement was incubated at 37 °C for 1 hour, shaking (50 rev/min). Vibriocidal titers were defined as the reciprocal of the highest serum dilution resulting in a 50% reduction in optical density (595 nm) compared to controls without serum. A standard O-antigen specific monoclonal antibody (mAb) and a high titer standard serum were used to normalize the results in case of inter-assay variations [7].

Statistical Analysis

Categorical variables were summarized using frequency and percentages while median and interquartile interval were used for continuous variables. Geometric mean titres (GMTs) of serum and saliva vibriocidal antibody titers were calculated as the anti-logarithm of the mean natural log-transformed vibriocidal antibody titers. The corresponding 95% confidence intervals were also calculated. Spearman’s rank correlation coefficient was used to assess the correlation between blood and saliva vibriocidal antibody titers. Using finite mixture model of log-transformed serum antibody titers, we classified participants as responders and nonresponders at a cut-off determined as mean log-titer plus three standard deviation of the latent class 1 distribution. To determine the diagnostic accuracy (sensitivity and specificity) of saliva, we used receiver operating characteristics (ROC) curve. All statistical analyses were performed using Stata 18 MP (StataCorp, College Station, TX, USA).

3. Results

Background Characteristics of Participants

Of the 63 cholera patients enrolled into the study, 4 had missing data and 59 were included in the final analysis. Among the 59 participants, 30 (51%) were males, 21 (35%) were aged between 26-45 years, 16 (27%) were domestic workers, and 14 (23.7%) had stream as source of drinking water (Table 2). Also, 52 (88%) were HIV negative, 7 (12%) were from Chipangali, 21 (36%) from Chipata and the bulk of them from Vubwi i.e., 31 (52%).

Diagnostic Accuracy of Saliva Relative to Serum

It seems from the receiver operating characteristics (ROC) curve that saliva titer is a good measure of antibody response as measured by serum. The best cut-off that maximizes (sensitivity + specificity) is 10 titers. At this saliva titer, the sensitivity is 76.9% (95%CI: 60.9%, 87.7%) and specificity is 80.0% (95%CI: 56.6%, 92.5%) (Table 3, Figure 1, ROC Curve). There was also evidence of a positive correlation between Vibrio Cholerae saliva and serum antibodies (rho=0.66, p<0.001) (Figure 1, Correlation Curve).

Kinetics of Cholera Vibriocidal Antibodies in Saliva

V. Cholerae saliva antibodies increased gradually from point of infection with cholera with the highest being recorded between 15 and 19 days after infection. After day 19, there was a notable decrease with the lowest levels of vibriocidal antibodies being recorded between 30-34 days after infection. Afterwards, there was a slight elevation of V. Cholerae saliva antibodies between days 35-40. Though this increase was lower when compared to levels recorded between 10-14 days (Figure 2).
Similarly, there was a gradual increase in V. Cholerae serum antibodies from point of infection with cholera with the highest also being recorded between 15 and 19 days after infection. After day 19, there was a slight drop with the lowest levels of vibriocidal antibodies being recorded between 30-34 days after infection. Afterwards, there was a slight elevation of V. Cholerae serum antibodies between days 35-40. Though this increase was lower when compared to levels recorded between 10-14 days. However, when compared with saliva, V. Cholerae serum antibodies were generally higher after 5 days of infection as shown in Figure 2.
There was no evidence that V. Cholerae serum and saliva antibody geometric mean titres among people living with HIV were higher compared to people without HIV. (Figure 3).

4. Discussion

We report for the first time that during natural infection with cholera, there is robust mucosal immune response that can easily be detected using saliva as a proxy for serum. This response was not seen when saliva from vaccinated individuals was tested in a similar manner (data not shown). The analysis of paired saliva and serum samples from 59 cholera patients has revealed that at a saliva titer cut-off of 10, the diagnostic sensitivity and specificity of saliva-based testing are 76.9% and 80.0%, respectively. Furthermore, a significant positive correlation (rho=0.66, p<0.001) between saliva and serum antibody levels underscores the potential of saliva as a non-invasive diagnostic tool. These findings highlight the effectiveness of saliva-based testing in identifying cholera infections, which could significantly enhance diagnostic approaches, particularly in settings where blood collection poses logistical or compliance challenges.
The diagnostic accuracy of saliva-based testing for cholera, as evidenced by our findings, aligns with the growing body of research advocating for non-invasive diagnostic methods. While several studies have explored saliva as a diagnostic medium for various infectious diseases such as COVID-19 [4], arbovirus [8,9,10,11], and rotavirus infection [3], its application for cholera has been less documented. The sensitivity and specificity obtained in our study are comparable to those reported for other non-invasive testing methods [4,12] suggesting that saliva testing can offer a viable alternative to serum-based assays. The recent surge in cholera cases among children under-five, along with the significant economic strain caused by the disease [1], underscores the urgent need to adopt rapid detection and containment technologies like those used during the COVID-19 [12] pandemic to effectively combat cholera.
Notably, the positive correlation between saliva and serum antibody levels further validates the reliability of saliva as a diagnostic medium. Our study’s findings contrast with some earlier studies that questioned the diagnostic utility of saliva, primarily due to concerns about lower antibody concentrations compared to serum [5]. However, our results, supported by robust statistical analysis, affirm the diagnostic value of saliva, particularly when utilizing optimized assays and appropriate cut-off values. We demonstrated that measurements of cholera antibody levels in saliva specimens can be used to accurately determine serum levels as has been demonstrated by another study that compared SARS-CoV-2 (COVID 19) serum antibodies with saliva and similarly reported a strong correlation [12].
Zambia has recently experienced its largest cholera outbreak despite ongoing vaccination efforts, with the outbreak declared in Lusaka in October 2023 escalating to a public health emergency that has spread across all ten provinces. As of 14th February 2024, the outbreak has affected 62 districts, resulting in 18,804 cases and 658 deaths, making a case fatality rate of 3.8%. This situation calls for innovative detection methods, such as the development of a saliva-based rapid diagnostic kit, to enhance surveillance and combat the disease amidst challenges like climate change and the WHO’s 2030 target to end cholera. Our study emphasizes the advantages of using saliva over serum for diagnostic purposes, noting saliva’s accessibility, non-invasive collection, and suitability for large-scale testing without professional assistance and is appropriate for measuring immunological biomarkers for both children and adults [13,14,15]. It contrasts with the limitations of serum collection, which is invasive, riskier, and more resource-intensive [16,17,18,19]. Our research focuses on measuring vibriocidal antibodies [20,21], considered the ’gold standard’ for assessing mucosal immune response to V. cholerae [22], in both saliva and serum to validate saliva’s efficacy as an alternative diagnostic medium. This approach is driven by the premise that vibriocidal antibodies present in blood would also be detectable in saliva, potentially offering a simpler, cost-effective method for cholera surveillance and response.
This study had several strengths. Firstly, it is the first to show vibriocidal activity in saliva, secondly, that saliva vibriocidal activity remain elevated in convalescent individuals, third that the level of vibriocidal antibodies are comparable with those present in serum with strong correlation and fourth that there is robust mucosal immune response during natural infection with cholera. Nevertheless, the study has limitations that warrant consideration. While the sample size is sufficient to demonstrate significant findings, it could be expanded in future study to validate our results across broader populations. Additionally, the study focused on naturally-infected individuals, which might limit the generalizability of the findings to asymptomatic carriers or those with mild infections. Although we have shown that there is robust immune response after natural infection with cholera, we don’t know if this is also the case during vaccine induced immunity.
Our study has important implications for practice and future research. The evidence presented supports the integration of saliva-based testing into cholera diagnostic protocols, offering a non-invasive, patient-friendly alternative to serum assays. This could be particularly beneficial in resource-limited settings and among populations where blood collection is challenging. Future research should aim to validate these findings in larger, more diverse cohorts, including asymptomatic and mildly symptomatic individuals. Further exploration into the optimization of saliva collection and assay techniques could enhance the diagnostic accuracy and feasibility of saliva-based tests. Additionally, investigating the potential for saliva diagnostics in other infectious diseases could broaden the scope of non-invasive testing methodologies.

5. Conclusions

Our study supports the diagnostic accuracy of saliva-based testing for cholera, demonstrating its potential as a non-invasive, efficient, and patient-friendly alternative to traditional serum assays. The significant correlation between saliva and serum antibody levels underscores the reliability of saliva as a diagnostic medium. These findings pave the way for future studies and the potential adoption of saliva-based diagnostics in clinical practice, which could transform the landscape of infectious disease diagnosis and would be ideal for surveillance, particularly in settings where non-invasive methods are most needed.

Author Contributions

Conceptualization, C.C.C., M.M., and H.N., methodology, C.C.C., M.M., H.N., and S.B.; software, B.P. and S.B.; validation, C.C.C., and H.N.; formal analysis, C.C.C., B.P. and S.B.; investigation, C.C.C., A.F.C., B.B.., M.M., H.N., and D.S.; resources, C.C.C. and A.F.C.; data curation, M.M. and H.N.; writing—original draft preparation, C.C.C., S.B.; writing—review and editing, D.S., A.F.C., B.B., F.L., C.C.C., B.P., M.M., H.N., and S.B.; visualization, C.C.C., S.B. and H.N.; supervision, B.B., S.B., D.S., and A.F.C.; funding acquisition, C.C.C. and A.F.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by Bacterial Vaccines Network (BACTIVAC) through a Pump Priming grant awarded to Dr Caroline Cleopatra Chisenga with grant number: BVNCP5-08. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Institutional Review Board Statement

The study was approved by the by the University of Zambia Biomedical Research Ethics Committee (UNZABREC-REF No. 001-02-23). The National Health Research Authority also approved this study. Study information and procedures were also provided to the participants. Participants who consented to take part in the study and publication of findings were screened for eligibility by the study team.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study including written informed consent to publish this paper.

Data Availability Statement

The data set cannot be shared publicly because it contains human research participant data; however, it can be made available to any interested researchers upon request. The Centre for Infectious Disease Research in Zambia (CIDRZ) Ethics and Compliance Committee is responsible for approving such requests. To request data access, one must write to the Secretary to the Committee/ Head of Research Operations through this email address: info@cidrz.org, mentioning the intended use for the data, contact information, a research project title, and a description of the analysis being proposed as well as the format it is expected. The requested data should only be used for the purposes related to the original research or study. The CIDRZ Ethics and Compliance Committee will normally review all data requests within 48–72 hours (Monday-Friday) and provide notification if access has been granted or additional project information is needed.

Acknowledgments

We want to acknowledge the Health Workers in Chipata and Vubwi, Environmental Health Specialists who helped in the collection of data and the participants who volunteered to be in the study. Lastly, we would like to acknowledge CIDRZ who facilitated all travel costs, Enteric Disease and Vaccine Research Unit for the hard work. Special thanks to Mr. Chilambwe Mulenga.

Conflicts of Interest

The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Kirigia JM, Sambo LG, Yokouide A, Soumbey-Alley E, Muthuri LK, Kirigia DG. Economic burden of cholera in the WHO African region. BMC Int Health Hum Rights. 2009;9. [CrossRef]
  2. Singh S, Ramesh V, Oza N, Balamurali PD, Prashad KV, Balakrishnan P. Evaluation of serum and salivary lipid profile: A correlative study. Journal of Oral and Maxillofacial Pathology. 2014;18: 4–8. [CrossRef]
  3. Friedman MG, Entin N, Zedaka R, Dagan R. Subclasses of IgA antibodies in serum and saliva samples of newborns and infants immunized against rotavirus. Clinical and Experimental Immunology, Volume 103, Issue 2, February 1996, Pages 206–211. [CrossRef]
  4. Vohra P, Belkhode V, Nimonkar S, Potdar S, Bhanot R, Izna, et al. Evaluation and diagnostic usefulness of saliva for detection of HIV antibodies: A cross-sectional study. J Family Med Prim Care. 2020;9: 2437. [CrossRef]
  5. Jertborn M, Svennerholm A-M, Holmgren J. Saliva, Breast Milk, and Serum Antibody Responses as Indirect Measures of Intestinal Immunity after Oral Cholera Vaccination or Natural Disease. J Clin Microbiol. 1986. [CrossRef]
  6. Charles RC, Hilaire IJ, Mayo-Smith LM, Teng JE, Jerome JG, Franke MF, et al. Immunogenicity of a killed bivalent (O1 and O139) whole cell oral cholera vaccine, Shanchol, in Haiti. PLoS Negl Trop Dis. 2014;8: e2828. [CrossRef]
  7. Azman AS, Lessler J, Luquero FJ, Bhuiyan TR, Khan AI, Chowdhury F, et al. Estimating cholera incidence with cross-sectional serology. Sci Transl Med. 2019;11: 1–11. [CrossRef]
  8. Musso D, Teissier A, Rouault E, Teururai S, De Pina JJ, Nhan TX. Detection of chikungunya virus in saliva and urine. Virol J. 2016;13: 1–4. [CrossRef]
  9. Cuzzubbo AJ, Vaughn DW, Nisalak A, Suntayakorn S, Aaskov J, Devine PL. Detection of Specific Antibodies in Saliva during Dengue Infection. 1998;36: 3737–3739. [CrossRef]
  10. Gardner J, Rudd PA, Prow NA, Belarbi E, Roques P, Larcher T, et al. Infectious Chikungunya Virus in the Saliva of Mice, Monkeys and Humans. PLoS One. 2015;10: e0139481. [CrossRef]
  11. Vázquez S, Cabezas S, Pérez AB, Pupo M, Ruiz D, Calzada N, et al. Kinetics of antibodies in sera, saliva, and urine samples from adult patients with primary or secondary dengue 3 virus infections. International Journal of Infectious Diseases. 2007;11: 256–262. [CrossRef]
  12. Schuit E, Venekamp RP, Veldhuijzen IK, van den Bijllaardt W, Pas SD, Stohr JJJM, et al. Head-to-head comparison of the accuracy of saliva and nasal rapid antigen SARS-CoV-2 self-testing: cross-sectional study. BMC Med. 2022;20. [CrossRef]
  13. Moe CL, Sair A, Lindesmith L, Estes MK, Jaykus LA. Diagnosis of norwalk virus infection by indirect enzyme immunoassay detection of salivary antibodies to recombinant norwalk virus antigen. Clin Diagn Lab Immunol. 2004;11: 1028–1034. [CrossRef]
  14. Morris-Cunnington MC, Edmunds WJ, Miller E, Brown DWG. A population-based seroprevalence study of hepatitis A virus using oral fluid in England and Wales. Am J Epidemiol. 2004;159: 786–794. [CrossRef]
  15. Francavilla VC, Vitale F, Ciaccio M, Bongiovanni T, Marotta C, Caldarella R, et al. Use of Saliva in Alternative to Serum Sampling to Monitor Biomarkers Modifications in Professional Soccer Players. 2018;9: 1–9. [CrossRef]
  16. Emmons, W. Accuracy of oral specimen testing for human immunodeficiency virus. American Journal of Medicine. Elsevier Inc.; 1997. pp. 15–20. [CrossRef]
  17. De Melker HE, Nagelkerde NJD, Conyn-Van Spaendonck MAE. Non-participation in a population-based seroprevalence study of vaccine-preventable diseases. Epidemiol Infect. 2000;124: 255–262. [CrossRef]
  18. Roderick P, Wheeler J, Cowdex J, Sockett P, Skinner R, Mortimer P, et al. A pilot study of infectious intestinal disease in England. Epidemiol Infect. 1995;114: 277–288. [CrossRef]
  19. McMurtry CM, Riddell RP, Taddio A, Racine N, Asmundson GJG, Noel M, et al. Far from “just a poke”: Common painful needle procedures and the development of needle fear. Clinical Journal of Pain. 2015;31: S3–S11. [CrossRef]
  20. Yang JS, An SJ, Jang MS, Song M, Han SH. IgM specific to lipopolysaccharide of Vibrio cholerae is a surrogate antibody isotype responsible for serum vibriocidal activity. PLoS One. 2019;14: e0213507. [CrossRef]
  21. Qadri F, Wennerås C, Albert MJ, Hossain J, Mannoor K, Begum YA, et al. Comparison of Immune Responses in Patients Infected with Vibrio cholerae O139 and O1 Downloaded from. Infect Immun. 1997. INFECTION AND IMMUNITY, 1997 Sep;65(9):3571-6. [CrossRef]
  22. Best JM, Reef S. The Immunological Basis for Immunization Series. The Immunological Basis for Immunization Series - Module 11: Rubella. 2008.
Preprints 100658 g001
Figure 1. Correlation between log-transformed saliva and serum V. Cholerae antibody response (Correlation Curve); Receiver characteristics curve (ROC Curve).
Figure 1. Correlation between log-transformed saliva and serum V. Cholerae antibody response (Correlation Curve); Receiver characteristics curve (ROC Curve).
Preprints 100658 g001
Preprints 100658 g002
Figure 2. V. Cholerae serum and saliva antibody geometric mean titres in patients at different infection days (A).
Figure 2. V. Cholerae serum and saliva antibody geometric mean titres in patients at different infection days (A).
Preprints 100658 g002
Preprints 100658 g003
Figure 3. Serum and Saliva antibody response by HIV status. NOTE: GMT and associated 95% CI.
Figure 3. Serum and Saliva antibody response by HIV status. NOTE: GMT and associated 95% CI.
Preprints 100658 g003
Table 1. Assay sample dilutions.
Table 1. Assay sample dilutions.
Sample type Dilution Sample volume (µl) Diluents (µl) Final volume (µl)
Serum 1:10 15 135 150
Saliva 1:2 75 75 150
Table 2. Baseline Characteristics of Cholera Patient Enrolled in the Study.
Table 2. Baseline Characteristics of Cholera Patient Enrolled in the Study.
Preprints 100658 i001
Table 3. Diagnostic accuracy of saliva-based testing compared with serum.
Table 3. Diagnostic accuracy of saliva-based testing compared with serum.
Serum Saliva Total
Negative Positive
Negative (Nonresponder) 16 4 20
Positive (Responder) 9 30 39
Total 25 34 59
Sensitivity: 76.9% (95%CI: 60.9%, 87.7%)
Specificity: 80.0% (95%CI: 56.6%, 92.5%)
Positive Predictive Value: 88.2% (95%CI: 72.5%, 96.7%)
Negative Predictive Value: 64.0% (95%CI: 42.5%, 82.0%)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated