Preprint
Article

Impact on Accessibility to Care and Outcomes of a Clinician-to-Clinician Electronic Consultation Program in Heart Failure Patients with Previous Hospital Admissions: Implications for Heart Failure Care

Altmetrics

Downloads

123

Views

48

Comments

0

Submitted:

12 May 2023

Posted:

15 May 2023

You are already at the latest version

Alerts
Abstract
(1) Objectives. Patients with heart failure (HF) who experience hospitalizations for worsening HF (HFH) are at high risk of subsequent events. We aimed to evaluate the impact of an outpatient care management program that includes a clinician-to-clinician e-consultation using an integrated electronic medical record in a healthcare area with a widely dispersed population on delay time in care, hospital admissions, and mortality in a high-risk group of patients with HF and previous episodes of HFH. (2) Methods. We selected 6,444 HF patients who visited the cardiology service at least once between 2010 and 2021. Of these, 4,851 were attended in e-consult, and 2,008 in one-time in-person consultations. In 2,230 HF patients, there was documentation of a previous episode of HFH. Using an interrupted time series regression model, we analysed the impact of incorporating e-consult into the health care model in the group of patients with HFH and evaluated the elapsed time to cardiology care, heart failure (HF), cardiovascular (CV), and all-cause hospital admissions and mortality, calculating the incidence relative risk (iRR). (3) Results. In the group of patients with previous HF hospitalizations, the introduction of e-consult substantially decreased waiting times to cardiology care. The time elapsed to care after e-consult implementation was significantly reduced compared with the previous in-person period (8.6 [8.7] vs 55.4 [79.9] days, p<0.001). In that group of patients, after e-consult implantation, hospital admissions for HF were reduced (iRR [CI95%]: 0,837 [0,840-0,833]), 0,900 [0,862-0,949] for CV and 0,699 [0,678-0,726] for all-cause hospitalizations. There was also lower mortality (iRR [CI95%]: 0.715 [0.657-0.798] due to HF, 0,737 [0.764-0.706] for CV and 0,687 [0.652-0,718] for all-cause). The improved outcomes after e-consultation implementation were significantly higher in the group of patients with previous HFH and were independent of the patient’s clinical characteristics managed during the in-person or e-consultation periods. (4) Conclusions. In HF patients with previous HFH, an outpatient care program that includes an e-consult significantly reduced waiting times to cardiology care and was safe, with a lower rate of hospital admissions and mortality in the first year.
Keywords: 
Subject: Medicine and Pharmacology  -   Cardiac and Cardiovascular Systems

1. Introduction

Despite the progress made in heart failure (HF) management, the incidence of worsening HF episodes requiring ambulatory treatment intensification, emergency department visits and hospitalization remains high. Early clinical assessment and treatments are necessary to prevent a progressive clinical deterioration that leads to disease progression, functional capacity deterioration, and hospitalization [1]. Most of these episodes are initially evaluated by primary care physicians (PCPs) followed by a referral for specialist care (cardiologist or internal medicine) in most cases. A healthcare organization is needed that allows for early management of such worsening episodes, including specialist care, to prevent disease progression and improve clinical outcomes[2].
Various digital health technologies have been proposed to improve patient accessibility to care and early recognition and management of clinical manifestations associated with congestion [3]. However, most experiences describe the results of digital health programs for communication between patients and health professionals, including tele-visits/virtual visits, remote monitoring and management, patient engagement with care activities, and consumer/patient access to clinical data, with very limited data on experiences aimed at improving communication between health providers [4,5,6].
Clinician-to-clinician electronic consultation programs (e-Consults) are an emerging health care innovation developed to address excess wait times for specialist care by enabling primary care physicians (PCPs) to obtain a specialist consultant’s expert opinion in a timely manner. E-consultation is defined by three characteristics: a) asynchronous communication between two healthcare professionals; b) performance of both the consultation and the response in a secure electronic system and their documentation in the patients’ official medical records; and c) the management of a specific clinical problem in the entire medical act. While other alternatives for health professional interchange of patient’s clinical information have been described, we believe that the models using institutional clinical health records are the most efficient and secure models for performing this type of ambulatory care [7,8].
A clinician-to-clinician e-consultation program may not only improve the accessibility to care but may also impact patient outcomes, particularly in HF patients with a previous episode of hospitalization (HFH), group of patients associated with a worse outcomes [2].
We aimed to evaluate the impacts on delay time in care, hospital admissions, and mortality resulting from the use of an outpatient care management program that includes a clinician-to-clinician e-consultation using the integrated electronic medical record in a healthcare area with a widely dispersed population compared to previous face-to-face visits for all the PCP referrals in a high-risk group of patients with HF and previous episodes of HF hospitalization.

2. Materials and Methods

2.1. Patient Population

The Cardiology Department (CD) and 301 PCP in the Santiago de Compostela healthcare area provide coverage to a population of 446,603 individuals. For the current analysis of our database, we included 6,444 patients with previous diagnosis of HF referred to the CD from 2010 to 2021. We compared the patients in two groups: those who had never been hospitalized due to worsening HF (n=4,214), and those with a history of previous HFH (n=2,230).

2.2. Ethics Approval

This study was approved by the local ethics committee on 23rd March 2022, with reference number 2021/496.

2.3. Consultation Models

We analysed two distinct time periods. From 2008 to 2012, the CD used an outpatient model based on a single in-person consultation, during which the cardiologist had to resolve the reason for patient’s visit and order any necessary complementary test. Since 2013 to the present day, an e-consult has been added as the initial step, allowing us to triage referrals and determine whether an in-person consultation is necessary [8]. Following the in-person consultation in both models, some patients required cardiology follow-up visits.
The e-consult takes place via our integrated electronic health record, which contains all patient information from primary care and hospitals across the Spanish region of Galicia. The e-consult must include all clinically relevant information, and a cardiologist reviews it a few days later along with any additional tests performed in primary care (e.g., electrocardiograms, chest x-rays, blood tests) and relevant information about the patient’s disease history (e.g., prior hospitalizations for heart failure and their timing). Based on all this information, the cardiologist determines the most appropriate type of consultation for each patient. This cardiologist may resolve the consultation without requiring an in-person visit by recording the answer to the e-consultation in the same electronic health record or may schedule an in-person single-act consultation.
The characteristics of our ambulatory outpatient care program, which includes an e-consultation as first step, have been previously described [8].

2.4. Variables

The available information for all the patients included sex, age at the time of first e-consultation, date of the e-consultation, diagnoses related to cardiovascular risk factors and previous history of CVD, date of cardiology consultation, number of follow-up consultations, previous HFH, and whether a face-to-face consultation was conducted after the e-consultation. We consider patients with HF who had the previous diagnosis in the electronic medical record.
Additionally, we assessed emergency department visits, hospital admissions, the main diagnosis for each visit, and deaths during the first year after the first consultation or e-consultation in the CD [8,11].
In addition to the descriptive analysis discussed above, we conducted an analysis of temporal trends in waiting times for CD consultations (defined as the elapse time to cardiology consultation -in-person or e-consultation- from the PCP referral), hospital admissions, and mortality during the first year after consultation.

2.5. Statistics

Qualitative variables were expressed as percentages (%), and quantitative continuous variables as means (standard deviation -SD-). To verify differences between groups, we used the chi-square test for qualitative variables and the t-test for quantitative variables. We considered statistical significance at p<0.05.
To investigate the impact of the e-consultation program on delay time in care, hospital admissions, and mortality in both groups of analyses, we performed an interrupted time series (ITS) regression approach [16]. Predictors entered in the model included time elapsed from the beginning of the study (months), type of consultation (0, in-person consultation; 1, e-consultation), and interaction time x type of consultation. We considered and controlled for overdisposition and estimated the incidence relative risk (iRR) with a 95% confidence interval for each outcome in three periods. The outcomes included in these analyses were HF-related, CV and all-cause hospitalizations and deaths ocurring up to 1 year after the first cardiologist evaluation.
We performed a multivariate logistic regression for each of these outcomes in both groups The variables included in the model were those that could influence the prognosis, such as personal characteristics (age, gender), comorbidities (arterial hypertension, diabetes mellitus, ischemic heart disease, atrial fibrillation, cerebrovascular disease, peripheral arterial disease), and features related to disease management (previous hospitalizations status, waiting time until the first cardiologist evaluation, type of management model, and number of visits to the emergency department during the first year).
For data analysis, we used the statistic package SPSS, version 25.0 (SPSS Inc., United States). ITS analyses were conducted using R version 3·5·1 and open-source BayesX software.

3. Results

3.1. Sample Overview and Outcomes

We included 6,444 patients, of which 4,214 had no history of HFH and 2,230 had a history of HFH, with a mean time of the previous admission of 37.6 months (range between 1 and 266 months) Patients with HFH had a higher prevalence of men (p<0.001) but had a similar age (p=0.267) compared to patients without HFH. Patients with HFH had a higher prevalence of diabetes (p<0.001), ischaemic heart disease (p=0.036), and peripheral arterial disease (p=0.008). Of the PCP referrals, 17.7% were resolved without in-person consultation, with no significant difference between patients with or without HFH (Table 1).
After e-consult implementation, the time to cardiology care was significantly reduced compared to the previous in-person period (8.6 [8.7] vs 55.4 [79.9] days, p<0.001) with no significant difference between both groups. More than 50%of referrals were solved in less than 8 days, with a faster response in the group of patients with previous HFH (57.7% vs 46.4%, p<0.001) (Table 1).
Compared to patients without previous HFH, patients with HFH required more complementary tests (p=0.053) and had a higher need for emergency department assistance (p<0.001) at 1 year after consultation. These patients also had a higher incidence of all-cause hospitalizations (p<0.001), CV hospitalizations (p<0.001) and HF-related hospitalizations (p<0.001). Furthermore, all-cause, CV and HF-related mortality was higher in the group with previous HFH (p<0.001, for all), as shown in Table 1.

3.2. Outcomes after E-Consultation Implementation

3.2.1. Delay from PCP Referral to Cardiology Consultation

During the in-person consultation period, there was a gradual reduction in delays, which fell sharply upon the implementation of e-consults in both groups (with and without episodes of HFH). The implementation of e-consults resulted in a significant reduction in the delay time to cardiology care in patients with previous HFH (iRR: 0.548 [0.546-0.551]) and iRR: 0.522 [0.520-0.525], respectively), as shown in Figure 1.

3.2.2. Hospital Admissions at 1-Year After Consultation

Throughout the first year after consultation with the CD, 1,418 patients presented a total of 2,083 hospital admissions for all-causes (22,0%), 1,565 hospital admissions for CV causes (14.9%), and 890 hospital admissions for worsening HF (9.3%). The multivariate analyses showed that men had a higher risk of the three outcomes of hospitalization. Patients with previous hospitalizations and a greater number of Emergency Department attendances had a higher risk of three the outcomes of hospitalization. Moreover, mortality was higher in patients with a delay time after referral greater than 8 days (Table 2).
The multivariate analyses showed a higher risk of hospitalizations in men, patients with HFH, and those who required more emergency department assistance. Furthermore, we observed a higher risk in patients with longer delay times after referral and in-person models, but the differences were only statistically significant in HF-related hospitalizations, as shown in Table 3.
The analysis of interrupted temporal trends indicated that the rate of all-cause hospital admissions after e-consult implementation showed a downward trend during the e-consultation period, with a slight increase in the first years and a subsequent downtrend in both groups.
In patients with previous HFH, the implementation of e-consults represented a reduction in the all-cause hospitalizations (iRR: 0,699 [0,678-0,726]), CV hospitalizations (iRR: 0,900 [0,862-0,949]) and HF-related hospitalizations (iRR: 0,837 [0,840-0,833]), as shown in Figure 2. In this group, we observed a higher risk in the three outcomes of hospitalization in patients with a greater number of emergency department attendances and in patients attending in the in-person model (Table 3).
In patients without previous HFH, the e-consult implementation showed a decrease in all-cause hospitalizations (iRR: 0.676 [0.669-0.683]) and HF-hospitalizations (iRR: 0,938 [0,900-0,986]). The CV-hospitalizations were stable along the period (iRR: 1,001 [0,990-1,013]), as shown in Figure 2. In this group, the multivariate analyses showed a higher risk of the three outcomes of hospitalization in men and diabetic patients and also patients with a greater number of emergency department attendances (Table 4).

3.2.3. Mortality at 1-Year After Consultation

Throughout the first year after consultation with the cardiology department, 604 patients (9.4%) died; Table 1 summarized the proportion of all-cause, CV and HF-related deaths for both groups and the main cause of death. The multivariate analyses showed that age had a higher risk of the three outcomes of mortality. Patients with previous hospitalizations and a greater number of Emergency Department attendances had a higher risk of three the outcomes of mortality. Moreover, mortality was higher in patients with a delay time after referral greater than 8 days (Table 2).
In patients with previous HFH, the interrupted time series analysis showed that the rate of mortality after e-consult implementation showed a reduction in the all-cause mortality (iRR: 0,687 [0.652-0,718]) and also CV-deaths (iRR: 0,737 [0.764-0.706]) and HF-deaths (iRR: 0.715 [0.657-0.798]), as shown in Figure 3.
In this group of patients, the multivariate analyses showed a higher risk of mortality in older patients, with a greater number of emergency department attendances, and patients with delay time higher than 8 days and attended in the in-person model (Table 3).
In patients without previous HFH, our interrupted time series analysis showed that the implementation of e-consults resulted in a reduction in mortality rates for all-cause deaths (iRR: 0.714 [0.699-0.732]), CV-deaths (iRR: 0.771 [0.748-0.799]) and HF-related deaths (iRR: 0.911 [0.866-0.949]), as shown in Figure 3. Notably in this patient population, only age and delay time in the assistance were associated with a higher risk of mortality, as illustrated in Table 4.
The benefits of e-consultation implementation on hospitalization and mortality were even more significant in patients with previous HFH and were independent of the patient’s clinical characteristics managed during the in-person or e-consultation periods.

4. Discussion

Our study suggests that implementing clinician-to-clinician e-consults as the first step in outpatient management for HF patients with a previous episode of HFH led to an improved 1-year outcomes and reduced time elapsed to cardiology care for all PCP referrals compared to the previous model of face-to-face visits for all referrals. These improvements were independent of patient’s clinical characteristics managed during the in-person and e-consultation periods.
Furthermore, our results indicate that HF patients referred by PCPs for cardiology consultation with a previous episode of HFH had worse clinical profiles and 1-year cardiovascular outcomes compared to a group of HF patients without HFH. The improved outcomes after e-consultation implementation were significantly higher in patients with previous HFH.
To our knowledge, our study is the first to describe the long-term results of managing care for HF patients, both with and without previous HFH, through the application of e-consults for all PCP referrals to a cardiology department.
The study emphasizes the ESC HF guidelines recommendation for an early clinical comprehensive assessment of patients at 7 to 14 days after hospital discharge [1,12] and suggests that this recommendation should be extended to outpatient HF patients, particularly those with a previous HFH [13]. These findings may have implications for the clinical management of HF patients, suggesting that a care plan that takes into consideration the need for a reduced elapsed time to cardiology care is necessary for all PCP referrals, where digital health technologies, such as an electronic inter-clinician consultation program, may be useful in overcoming barriers to speed up patient care. In this regard, we describe that a reduction in the elapsed time to cardiology care after e-consultation was an independent predictor of better 1-year outcomes with the highest predictive value in patients with a previous hospitalization. The risk of hospital admissions and mortality was significantly higher in patients with a delay to e-consultation of more than 7 days.
HF is a chronic and heterogeneous clinical syndrome that is difficult to diagnose and manage, leading to disease progression. Chronic care models using multidisciplinary healthcare professionals, such as care coordinators, self-care/wellness educators, group therapy/education, dietitians, clinical pharmacists and social workers, have been effective in other chronic disease models [14]. Several telemedicine experiences and remote monitoring of HF patients involving transmission of patient-obtained weight and vital signs or more physiological markers such as thoracic impedance or intracardiac and arterial pulmonary pressures, have demonstrated a positive impact on maintaining patient clinical stability and reducing the risk of repeated HF hospitalizations [15]. In this regard, several studies from various countries have suggested a positive impact of patient-to-health-care provider telehealth programmes [16]. However, adoption is limited by the fact that most programs often require the patient’s ability to use a computer, a tablet, or a mobile phone, together with other medical equipment, and the absence of specific budgets for these activities in the vast majority of health systems [7].
These telemedicine programs may be of special interest in HF patients after an episode of hospitalization since those patients have a worse prognosis; telemedicine may provide a sustainable, cost-effective, and patient-centered approach to helping reduce rehospitalization in patients with HF [5,6,12,13,15,17].
Previous reports have assessed the impact of the use of telemedicine for the interaction between health care professionals and HF patients, a care model that has seen significant development during the COVID-19 pandemic. The AMULET clinical trial showed that an interventional telemedicine program, conducted remotely by nurses, was associated with a significant reduction in the risk of the composite outcome of HF-related hospitalizations and cardiovascular mortality for 12 months [16]. Similar results published by Salzano et al and Sammour et al with a program using several modalities of telemedicine (telephonic consultations, on-line chats, and video consultations). Compared to cohorts managed with in-person consultations, telemedicine was associated with better outcomes, particularly a lower risk of HF-related hospitalizations [18,19].
Xu et al recently published a study on the effectiveness of telemedicine visits in reducing 30-days readmissions among HF patients who had been hospitalized during the COVID-19 pandemic. Patients who received either an in-person or telemedicine follow-up within 14 days of discharge had a lower risk of 30-day re-admission compared with those who did not receive an early follow-up visit. After covariate adjustment, patients who received either telemedicine (OR: 0.55 [95%CI, 0.44–0.72]) or in-person (OR:0.52 [95%CI, 0.45–0.60]) visits were similarly less likely to be readmitted within 30 days compared with patients with no follow-up [20]. There were no significant differences in patients’ age, sex, race, and/or socioeconomic status between those who were followed up in person and via telemedicine.
Our study extends these observations to HF patients with a previous episode of hospitalization who require cardiology consultation. The early provision of patient risk stratification based on the clinical information provided by the PCP through an integrated medical record may explain the better outcomes over a long observation period [21,22,23]. Patients with HF and with greater critical needs were identified much sooner with e-consultation, and they may be treated significantly sooner compared to the previous period of in-person consultation for all the referrals [24]. Our findings underscore the need for targeted interventions to improve access to care for HF patients who exhibit initial symptoms and signs of clinical deterioration and who require an early personalized clinical evaluation and management. The development of integrated health records can make clinician-to-clinician communication more fluent and contain all the necessary information to optimize the resolution of the HF patient clinical problem [23].
Several publications have described our experience using this care model in our health care area, demonstrating that the use of an universal electronic consultation between PCPs and cardiologists as the first step to attend to referrals is associated with a reduction in the elapsed time to care and better health care outcomes [8,11] particularly in high risk patients such as those with HF and previous hospitalization for a worsening HF event [9,10,25].
Our analysis has some limitations. Firstly, we did not have specific information on the reasons for primary care physician referrals, and we did not consider phenotypical classification of HF based on left ventricular ejection fraction. Additionally, we did not have data on the medications patients were taking or how they were modified over time, which could have influenced our results. Secondly, our study was retrospective, and although we are aware of all deaths during the follow-up period, it was not always possible to determine the exact cause of death, which may have affected our findings. Thirdly, we did not have information on visits made by some patients to private health-care providers, which could have also influenced our results. However, given the low implantation of private healthcare in our health area and the positive results of our satisfaction survey among patients and PCPs, the potential for bias in this situation is likely reduced [8].
Despite these limitations, we believe that our study is clinically relevant and informative for healthcare management purposes. Our large cohort of patients with HF and comprehensive demographic, clinical and prognostic information integrated into the electronic medical record of our health system strengthens the relevance of our findings for the ambulatory care of patients with HF, particularly those with a history of HFH. Our study design using a temporal-trend series analysis is robust for assessing the effect of an intervention in non-randomized studies. Compared with a classical transversal analysis, this approach allows better control of changes related to an intervention, such as the implementation of an e-consult program in the care of HF patients. Additionally, our analysis provides a more intuitive evaluation of the dynamic responses to changes following e-consult implementation, as compared to the previous in-person consultation model. This allows us to assess the effects temporal trends such as whether changes are immediate or delayed, persistent or reversible, and whether medication effects can be avoided, as previously discussed.

5. Conclusions

Our findings represent the first description of patients with HF and a previous HFH, who have undergone a clinician-to-clinician e-consultation as the first step in the outpatient management model, followed by an in-person visit when necessary. This approach was associated with increased demand for care, reduced time elapsed to cardiology care for all PCP referrals and improved 1-year outcomes compared to the previous face-to-face visit model. Importantly, these improvements were independent of the patient’s clinical characteristics during both in-person and e-consultation periods and were significantly greater in patients with a history of HFH.
Throughout the follow-up period, reduced elapsed time to care was independently associated with better 1-year outcomes. We believe that our experience can contribute to the development of a more efficient ambulatory care pathway for HF patients, particularly those at high risk such as those with a history of hospitalization for worsening events.

Author Contributions

DGV: Conceptualization, validation, investigation, data curation, writing—original draft preparation, writing—review and editing, visualization, project administration. PMR: Conceptualization, validation, investigation, data curation, writing—original draft preparation, writing—review and editing, visualization, supervision. MPR: Methodology, validation, formal analysis, investigation, writing—original draft preparation, writing—review and editing. MRM: Methodology, validation, investigation, data curation, writing—original draft preparation, writing—review and editing. DRA: Conceptualization, methodology, validation, formal analysis, investigation, writing—original draft preparation, writing—review and editing. MSF: Validation, investigation, data curation, writing—original draft preparation, writing—review and editing. SCS: Conceptualization, methodology, validation, formal analysis, investigation, writing—original draft preparation, writing—review and editing, visualization, supervision, project administration. JRGJ: Conceptualization, validation, formal analysis, investigation, data curation, writing—original draft preparation, writing—review and editing, visualization, supervision, project administration.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of Santiago de Compostela y Lugo on 23rd March 2022, with reference number 2021/496

Informed Consent Statement

Not apply.

Data Availability Statement

Data availability: the data underlying this article are available in RUNA (https://runa.sergas.gal/) and can be accessed at https://dx.doi.org/.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. McDonagh, T.A.; Metra, M.; Adamo, M.; Gardner, R.S.; Baumbach, A.; Böhm, M.; Burri, H.; Butler, J.; Celutkiene, J.; Chioncel, O.; et al. 2021 ESC Guidelines for the Diagnosis and Treatment of Acute and Chronic Heart Failure. Eur Heart J 2021, 42, 3599–3726. [Google Scholar] [CrossRef] [PubMed]
  2. Comin-Colet, J.; Calero-Molina, E.; Corbella, X.; Muñiz, J.; San Saturnino, M.; Ibarrola, C.; Comín-Colet, J.; Alcober, L.; Calero-Molina, E.; Cobo, M.; et al. Recomendaciones Para Desarrollar Modelos Asistenciales de Atención al Paciente Con Insuficiencia Cardiaca (Proyecto MAIC): Estudio Delphi. J Healthc Qual Res 2022, 37, 100–109. [Google Scholar] [CrossRef] [PubMed]
  3. Yun, S.; Enjuanes, C.; Calero-Molina, E.; Hidalgo, E.; José, N.; Calvo, E.; Verdú-Rotellar, J.M.; Garcimartín, P.; Chivite, D.; Formiga, F.; et al. Effectiveness of Telemedicine in Patients with Heart Failure According to Frailty Phenotypes: Insights from the ICOR Randomised Controlled Trial. Eur J Intern Med 2022, 96, 49–59. [Google Scholar] [CrossRef] [PubMed]
  4. Yun, S.; Enjuanes, C.; Calero-Molina, E.; Hidalgo, E.; José-Bazán, N.; Ruiz, M.; Verdú-Rotellar, J.M.; Garcimartín, P.; Jiménez-Marrero, S.; Garay, A.; et al. Usefulness of Telemedicine-Based Heart Failure Monitoring According to ‘EHealth Literacy’ Domains: Insights from the ICOR Randomized Controlled Trial. Eur J Intern Med 2022, 101, 56–67. [Google Scholar] [CrossRef]
  5. González-Franco, Á.; Cerqueiro González, J.M.; Arévalo-Lorido, J.C.; Álvarez-Rocha, P.; Carrascosa-García, S.; Armengou, A.; Guzmán-García, M.; Trullàs, J.C.; Montero-Pérez-Barquero, M.; Manzano, L. Morbidity and Mortality in Elderly Patients with Heart Failure Managed with a Comprehensive Care Model vs. Usual Care: The UMIPIC Program. Revista Clínica Española (English Edition) 2022, 222, 123–130. [Google Scholar] [CrossRef]
  6. Jackevicius, C.A.; de Leon, N.K.; Lu, L.; Chang, D.S.; Warner, A.L.; Mody, F.V. Impact of a Multidisciplinary Heart Failure Post-Hospitalization Program on Heart Failure Readmission Rates. Annals of Pharmacotherapy 2015, 49, 1189–1196. [Google Scholar] [CrossRef]
  7. Tuckson, R. V; Edmunds, M.; Hodgkins, M.L. Telehealth. New England Journal of Medicine 2017, 377, 1585–1592. [Google Scholar] [CrossRef]
  8. Rey-Aldana, D.; Cinza-Sanjurjo, S.; Portela-Romero, M.; López-Barreiro, J.L.; Garcia-Castelo, A.; Pazos-Mareque, J.M.; Mazón-Ramos, P.; González-Juanatey, J.R. Programa de Consulta Electrónica Universal (e-Consulta) de Un Servicio de Cardiología. Resultados a Largo Plazo. Rev Esp Cardiol 2022, 75, 159–165. [Google Scholar] [CrossRef]
  9. Cinza Sanjurjo, S.; Mazón Ramos, P.; Iglesias Álvarez, D.; Rey Aldana, D.; Portela Romero, M.; González-Juanatey, J.R. Consulta Electrónica (e-Consulta) Para Pacientes Con Insuficiencia Cardiaca. Resultados a Largo Plazo. Rev Esp Cardiol 2022, 75, 93–95. [Google Scholar] [CrossRef]
  10. Mazón-Ramos, P.; Cinza-Sanjurjo, S.; Garcia-Vega, D.; Portela-Romero, M.; Sanmartin-Pena, J.C.; Rey-Aldana, D.; Martínez Monzonís, A.; Espasandín-Domínguez, J.; Gude-Sampedro, F.; González-Juanatey, J.R. A Clinician-to-Clinician Universal Electronic Consultation Programme at the Cardiology Department of a Galician Healthcare Area Improves Healthcare Accessibility and Outcomes in Elderly Patients. European Heart Journal - Digital Health 2023. [Google Scholar] [CrossRef]
  11. Rey-Aldana, D.; Mazón-Ramos, P.; Portela-Romero, M.; Cinza-Sanjurjo, S.; Alvarez-Alvarez, B.; Agra-Bermejo, R.; Rigueiro-Veloso, P.; Espasandín-Domínguez, J.; Gude-Sampedro, F.; González-Juanatey, J.R. Longer-Term Results of a Universal Electronic Consultation Program at the Cardiology Department of a Galician Healthcare Area. Circ Cardiovasc Qual Outcomes 2022, 15, 16–24. [Google Scholar] [CrossRef] [PubMed]
  12. Alcoberro, L.; Moliner, P.; Vime, J.; Jiménez-Marrero, S.; Garay, A.; Yun, S.; Pons-Riverola, A.; Ramos-Polo, R.; Ras-Jiménez, M.; Tajes, M.; et al. Breaking the 30-Day Barrier: Long-Term Effectiveness of a Nurse-Led 7-Step Transitional Intervention Program in Heart Failure. PLoS One 2023, 18, e0279815. [Google Scholar] [CrossRef] [PubMed]
  13. Van Spall, H.G.C.; Lee, S.F.; Xie, F.; Oz, U.E.; Perez, R.; Mitoff, P.R.; Maingi, M.; Tjandrawidjaja, M.C.; Heffernan, M.; Zia, M.I.; et al. Effect of Patient-Centered Transitional Care Services on Clinical Outcomes in Patients Hospitalized for Heart Failure. JAMA 2019, 321, 753. [Google Scholar] [CrossRef] [PubMed]
  14. Méndez-Bailón, M.; Comín-Colet, J. Towards a Comprehensive and Multidisciplinary Care in Heart Failure. Revista Clínica Española (English Edition) 2022, 222, 174–175. [Google Scholar] [CrossRef] [PubMed]
  15. Baecker, A.; Meyers, M.; Koyama, S.; Taitano, M.; Watson, H.; Machado, M.; Nguyen, H.Q. Evaluation of a Transitional Care Program After Hospitalization for Heart Failure in an Integrated Health Care System. JAMA Netw Open 2020, 3, e2027410. [Google Scholar] [CrossRef]
  16. Krzesiński, P.; Jankowska, E.A.; Siebert, J.; Galas, A.; Piotrowicz, K.; Stańczyk, A.; Siwołowski, P.; Gutknecht, P.; Chrom, P.; Murawski, P.; et al. Effects of an Outpatient Intervention Comprising Nurse-led Non-invasive Assessments, Telemedicine Support and Remote Cardiologists’ Decisions in Patients with Heart Failure (AMULET Study): A Randomised Controlled Trial. Eur J Heart Fail 2022, 24, 565–577. [Google Scholar] [CrossRef]
  17. Lawson, C.A.; Lam, C.; Jaarsma, T.; Kadam, U.; Stromberg, A.; Ali, M.; Tay, W.T.; Clayton, L.; Khunti, K.; Squire, I. Developing a Core Outcome Set for Patient-Reported Symptom Monitoring to Reduce Hospital Admissions for Patients with Heart Failure. European Journal of Cardiovascular Nursing 2022, 21, 830–839. [Google Scholar] [CrossRef]
  18. Salzano, A.; D’Assante, R.; Stagnaro, F.M.; Valente, V.; Crisci, G.; Giardino, F.; Arcopinto, M.; Bossone, E.; Marra, A.M.; Cittadini, A. Heart Failure Management during the COVID-19 Outbreak in Italy: A Telemedicine Experience from a Heart Failure University Tertiary Referral Centre. Eur J Heart Fail 2020, 22, 1048–1050. [Google Scholar] [CrossRef]
  19. Sammour, Y.; Spertus, J.A.; Austin, B.A.; Magalski, A.; Gupta, S.K.; Shatla, I.; Dean, E.; Kennedy, K.F.; Jones, P.G.; Nassif, M.E.; et al. Outpatient Management of Heart Failure During the COVID-19 Pandemic After Adoption of a Telehealth Model. JACC Heart Fail 2021, 9, 916–924. [Google Scholar] [CrossRef]
  20. Xu, H.; Granger, B.B.; Drake, C.D.; Peterson, E.D.; Dupre, M.E. Effectiveness of Telemedicine Visits in Reducing 30-Day Readmissions Among Patients With Heart Failure During the COVID-19 Pandemic. J Am Heart Assoc 2022, 11. [Google Scholar] [CrossRef]
  21. Yun, S.; Enjuanes, C.; Calero, E.; Hidalgo, E.; Cobo, M.; Llàcer, P.; García-Pinilla, J.M.; González-Franco, Á.; Núñez, J.; Morales-Rull, J.L.; et al. Study Design of Heart Failure Events Reduction with Remote Monitoring and EHealth Support (HERMeS). ESC Heart Fail 2020, 7, 4448–4457. [Google Scholar] [CrossRef] [PubMed]
  22. Jiménez-Marrero, S.; Yun, S.; Cainzos-Achirica, M.; Enjuanes, C.; Garay, A.; Farre, N.; Verdú, J.M.; Linas, A.; Ruiz, P.; Hidalgo, E.; et al. Impact of Telemedicine on the Clinical Outcomes and Healthcare Costs of Patients with Chronic Heart Failure and Mid-Range or Preserved Ejection Fraction Managed in a Multidisciplinary Chronic Heart Failure Programme: A Sub-Analysis of the ICOR Randomized Trial. J Telemed Telecare 2020, 26, 64–72. [Google Scholar] [CrossRef] [PubMed]
  23. Silva-Cardoso, J.; Juanatey, J.R.G.; Comin-Colet, J.; Sousa, J.M.; Cavalheiro, A.; Moreira, E. The Future of Telemedicine in the Management of Heart Failure Patients. Card Fail Rev 2021, 7. [Google Scholar] [CrossRef] [PubMed]
  24. Driscoll, A.; Cartledge, S. The Effect of Telemedicine in Heart Failure Based on Frailty Phenotypes. Eur J Intern Med 2022, 96, 44–45. [Google Scholar] [CrossRef]
  25. Mazón-Ramos, P.; Cinza-Sanjurjo, S.; Garcia-Vega, D.; Portela-Romero, M.; Rodríguez-Mañero, M.; Rey-Aldana, D.; Morandeira, A.P.; Lage-Fernández, R.; Gude-Sampedro, F.; González-Juanatey, J.R. The Impact of Inter-clinician Electronic Consultation in Patients Diagnosed with Atrial Fibrillation in Primary Care. Eur J Clin Invest 2023, 53. [Google Scholar] [CrossRef]
Figure 1. Analysis of the interrupted temporal trends of elapsed time to care in both groups.
Figure 1. Analysis of the interrupted temporal trends of elapsed time to care in both groups.
Preprints 73550 g001
Figure 2. Analysis of the interrupted temporal trends of hospital admissions for all cause (A), cardiovascular disease (B) and heart failure (C) in the first year after consultation in both gender.
Figure 2. Analysis of the interrupted temporal trends of hospital admissions for all cause (A), cardiovascular disease (B) and heart failure (C) in the first year after consultation in both gender.
Preprints 73550 g002
Figure 3. Analysis of the interrupted temporal trends of mortality for all cause (A), cardiovascular disease (B) and heart failure (C) in the first year after consultation in both gender.
Figure 3. Analysis of the interrupted temporal trends of mortality for all cause (A), cardiovascular disease (B) and heart failure (C) in the first year after consultation in both gender.
Preprints 73550 g003
Table 1. Clinical and healthcare characteristics and prognostic events in the sample by care models.
Table 1. Clinical and healthcare characteristics and prognostic events in the sample by care models.
Total Without previous hospitalizations With previous hospitalizations p
6,444 4,214 2,230
Agea (years) 77.7 (9.7) 77.6 (9.7) 77.9 (9.8) 0.267
Women (%) 49.6% 51.5% 46.0% <0.001
Comorbidities
Arterial hypertension (%) 78.8% 79.7% 77.0% 0.013
Diabetes mellitus (%) 34.5% 31.1% 40.9% <0.001
Ischemic heart disease (%) 20.7% 19.9% 22.2% 0.036
Atrial fibrillation (%) 44.9% 46.3% 42.2% 0.002
Cerebrovascular disease (%) 8.5% 8.3% 8.9% 0.380
Peripheral arterial disease (%) 7.3% 6.7% 8.5% 0.008
Consultation model. Periods <0.001
In-person consultation 35.6% 40.3% 26.9%
e-Consult 64.4% 59.7% 73.1%
E-consultation resolution
e-consult solves (%) 17.7% 17.0% 18.8% 0.171
1 single-act consultation (%) 39.7% 39.8% 39.6%
Follow-up visits (%) 42.6% 43.2% 41.6%
Delay to in-person consultation
Time to answera (days) 25.0 (53.5) 27.2 (53.9) 21.0 (52.5) <0.001
<8 days (%) 50.3% 46.4% 57.7% <0.001
8-14 days (%) 16.0% 16.0% 15.9%
15-30 days (%) 16.1% 17.5% 13.5%
>30 days (%) 17.6% 20.0% 13.0%
Healthcare activity
Cardiovascular testa,b 1.56 (2.29) 1.52 (2.29) 1.63 (2.29) 0.053
Emergency department consultationsa,b (1-year) 3.7 (5.8) 3.92 (6.21) 3.21 (5.06) <0.001
Emergency department consultationsb (%) 69.7% 68.1% 72.9% <0.001
Hospitalizationsb
All-cause hospitalizationsb (%) 22.0% 15.2% 34.9% <0.001
CV hospitalizationsb (%) 14.9% 8.3% 27.3% <0.001
HF-related hospitalizationsb (%) 9.3% 2.4% 22.4% <0.001
Deathsb
All-cause deathsb (%) 9.4% 6.3% 15.2% <0.001
CV deathsb (%) 4.8% 2.8% 8.6% <0.001
HF-related deathsb (%) 1.3% 0.7% 2.5% <0.001
Death causesc
Ischemic cardiopathy (%) 10.2% 9.9% 10.6% <0.001
HF (%) 14.2% 12.2% 16.5%
Cancer (%) 11.5% 14.6% 7.9%
Valvulopathy (%) 4.7% 4.7% 4.6%
Ischemic stroke (%) 3.2% 3.1% 3.2%
COPD (%) 4.1% 4.7% 3.4%
Chronic kidney disease (%) 1.6% 2.3% 0.7%
Respiratory infection (%) 2.2% 2.1% 2.4%
Atrial Fibrillacion (%) 3.3% 3.1% 3.6%
Haemorrhagic stroke (%) 0.9% 0.8% 1.1%
aMean ± (standard deviation). b1st year after the e-consultation. cPercentages over the total number of deaths. dStatistics: chi-square test, statistical significance at p<0.05.
CV: cardiovascular; HF: heart failure; COPD: chronic obstructive pulmonary disease
Table 2. Multivariate analyses in patients with heart failure of the relationship between clinical and healthcare variables with the outcomes analysed.
Table 2. Multivariate analyses in patients with heart failure of the relationship between clinical and healthcare variables with the outcomes analysed.
HF-r hospitalization CV hospitalization All-cause hospitalization HF-r mortality CV mortality All-cause mortality
OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI)
Age 1.01 (1.00-1.02) 0.999 (0.992-1.007) 1.00 (0.99-1.01) 1.12 (1.09-1.16) 1.08 (1.06-1.09) 1.07 (1.06-1.09)
Gender (men) 1.26 (1.05-1.52) 1.17 (1.01-1.35) 1.15 (1.01-1.30) 0.99 (0.64-1.56) 1.14 (0.89-1.46) 1.58 (1.31-1.89)
Comorbidities
Arterial hypertension 0.77 (0.62-0.96) 0.83 (0.70-0.98) 0.86 (0.74-0.99) 0.65 (0.40-1.08) 0.81 (0.61-1.07) 0.66 (0.54-0.88)
Diabetes mellitus 0.89 (0.74-1.08) 1.08 (0.93-1.25) 1.09 (0.96-1.25) 1.25 (0.78-2.01) 0.99 (0.77-1.29) 0.99 (0.82-1.19)
Ischemic cardiopathy 1.11 (0.89-1.38) 1.40 (1.18-1.66) 1.18 (1.02-1.37) 0.70 (0.39-1.28) 1.14 (0.85-1.52) 1.13 (0.91-1.40)
Atrial fibrillation 1.24 (1.03-1.48) 1.02 (0.88-1.18) 0.97 (0.86-1.10) 0.83 (0.53-1.30) 0.78 (0.61-0.99) 0.81 (0.68-0.97)
Cerebrovascular disease 0.87 (0.63-1.21) 1.23 (0.96-1.16) 1.19 (0.96-1.48) 1.48 (0.77-2.87) 1.03 (0.68-1.55) 0.86 (0.63-1.19)
Peripheral arterial disease 0.98 (0.70-1.37) 0.95 (0.72-1.24) 1.05 (0.83-1.32) 1.48 (0.69-3.20) 1.67 (1.12-2.50) 1.49 (1.09-2.03)
HF-related previous hospitalizations
Without hospitalizations 1.00 1.00 1.00 1.00 1.00 1.00
With hospitalizations 11.97 (9.54-15.01) 4.05 (3.50-4.70) 2.69 (2.38-3.05) 3.32 (2.11-5.23) 3.11 (2.44-3.95) 2.60 (2.18-3.11)
Cardiology assitance
Emergency assistance 1.05 (1.03-1.07) 1.04 (1.03-1.05) 1.05 (1.03-1.06) 0.96 (0.89-1.03) 0.89 (0.85-0.93) 0.92 (0.89-0.95)
Delay time
0-7 days (ref) 1.00 1.00 1.00 1.00 1.00 1.00
8-14 days 1.23 (1.01-1.49) 1.18 (1.01-1.37) 1.13 (0.99-1.30) 2.76 (1.60-4.77) 2.04 (1.57-2.66) 1.81 (1.49-2.20)
15-30 days 0.93 (0.68-1.26) 1.02 (0.80-1.30) 1.12 (0.91-1.38) 2.19 (0.89-5.39) 1.62 (1.05-2.51) 1.60 (1.18-2.18)
>30 days 1.04 (0.72-1.49) 1.09 (0.82-1.44) 1.13 (0.87-1.46) 1.83 (0.51-6.50) 1.07 (0.62-1.82) 1.22 (0.84-1.79)
Assitance model
Econsult (ref) 1.00 1.00 1.00 1.00 1.00 1.00
In-person consultation 1.24 (0.96-1.61) 1.20 (0.97-1.48) 2.53 (2.11-3.11) 0.61 (0.34-1.11) 0.36 (0.26-0.50) 0.44 (0.34-0.56)
In bold: statistically significant factors (p<0.05)
Abbrevations: OR: odds ratio; CI: confidence interval; CV, cardiovascular; HF, heart failure; HF-r, heart failure-related
Table 3. Multivariate analyses in patients with previous worsening heart failure hospitalizations of the relationship between clinical and healthcare variables with the outcomes analysed.
Table 3. Multivariate analyses in patients with previous worsening heart failure hospitalizations of the relationship between clinical and healthcare variables with the outcomes analysed.
HF-r hospitalization CV hospitalization All-cause hospitalization HF-r mortality CV mortality All-cause mortality
OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI)
Age 1.01 (1.00-1.02) 1.00 (0.99-1.01) 1.00 (0.99-1.01) 1.16 (1.11-1.22) 1.07 (1.05-1.09) 1.07 (1.05-1.09)
Gender (men) 1.09 (0.89-1.35) 1.03 (0.85-1.26) 1.02 (0.85-1.23) 1.09 (0.62-1.91) 1.04 (0.76-1.43) 1.48 (1.15-1.92)
Comorbidities
Arterial hypertension 0.84 (0.66-1.08) 0.85 (0.68-1.07) 0.79 (0.64-0.99) 0.73 (0.38-1.39) 0.88 (0.61-1.28) 0.69 (0.52-0.93)
Diabetes mellitus 0.75 (0.60-0.93) 0.84 (0.68-1.02) 0.89 (0.73-1.07) 1.29 (0.71-2.32) 1.02 (0.73-1.42) 0.88 (0.67-1.14)
Ischemic cardiopathy 1.08 (0.84-1.38) 1.30 (1.03-1.63) 1.20 (0.97-1.49) 0.64 (0.30-1.36) 1.12 (0.77-1.63) 1.18 (0.88-1.58)
Atrial fibrillation 1.26 (1.02-1.55) 1.09 (0.90-1.33) 1.02 (0.85-1.22) 0.77 (0.43-1.36) 0.65 (0.47-0.91) 0.81 (0.63-1.05)
Cerebrovascular disease 0.99 (0.70-1.42) 1.07 (0.77-1.49) 1.23 (0.90-1.67) 1.09 (0.45-2.68) 0.76 (0.43-1.34) 0.79 (0.51-1.23)
Peripheral arterial disease 1.01 (0.69-1.46) 0.91 (0.64-1.29) 0.99 (0.72-1.36) 1.40 (0.52-3.76) 1.65 (0.99-2.75) 1.63 (1.08-2.46)
Cardiology assitance
Emergency assistance (1 year) 1.06 (1.04-1.08) 1.05 (1.03-1.08) 2.33 (1.77-3.07) 0.97 (0.89-1.05) 0.92 (0.88-0.96) 0.93 (0.89-0.95)
Delay time
0-7 days (ref) 1.00 1.00 1.00 1.00 1.00 1.00
8-14 days 1.36 (1.09-1.69) 1.32 (1.08-1.62) 1.23 (1.01-1.48) 2.53 (1.31-4.87) 1.71 (1.22-2.41) 1.74 (1.34-2.27)
15-30 days 0.92 (0.64-1.31) 0.85 (0.61-1.19) 0.92 (0.68-1.25) 2.82 (0.94-8.45) 2.38 (1.37-4.14) 1.87 (1.21-2.89)
>30 days 0.81 (0.52-1.27) 0.84 (0.56-1.26) 0.94 (0.64-1.39) 1.62 (0.34-7.75) 0.71 (0.34-1.49) 0.91 (0.52-1.57)
Assitance model
Econsult (ref) 1.00 1.00 1.00 1.00 1.00 1.00
In-person consultation 1.52 (1.12-2.06) 1.53 (1.16-2.03) 2.33 (1.77-3.07) 2.59 (1.32-5.05) 4.04 (2.75-5.93) 4.19 (3.05-5.75)
In bold: statistically significant factors (p<0.05)
Abbrevations: OR: odds ratio; CI: confidence interval; CV, cardiovascular; HF, heart failure; HF-r, heart failure-related
Table 4. Multivariate analyses in patients without previous worsening heart failure hospitalizations of the relationship between clinical and healthcare variables with the outcomes analysed.
Table 4. Multivariate analyses in patients without previous worsening heart failure hospitalizations of the relationship between clinical and healthcare variables with the outcomes analysed.
HF-r hospitalization CV hospitalization All-cause hospitalization HF-r mortality CV mortality All-cause mortality
OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI) OR (95%CI)
Age 1.00 (0.98-1.02) 0.99 (0.98-1.01) 1.00 (0.99-1.01) 1.07 (1.02-1.12) 1.08 (1.05-1.11) 1.08 (1.06-1.10)
Gender (men) 2.16 (1.41-3.31) 1.38 (1.10-1.73) 1.29 (1.08-1.54) 0.90 (0.43-1.90) 1.40 (0.95-2.04) 1.81 (1.39-2.36)
Comorbidities
Arterial hypertension 0.55 (0.35-0.86) 0.79 (0.61-1.04) 0.96 (0.77-1.19) 0.54 (0.24-1.22) 0.82 (0.53-1.28) 0.68 (0.50-0.91)
Diabetes mellitus 1.82 (1.21-2.75) 1.57 (1.25-1.97) 1.38 (1.15-1.65) 1.28 (0.58-2.83) 1.02 (0.66-1.52) 1.22 (0.92-1.61)
Ischemic cardiopathy 1.26 (0.80-1.98) 1.59 (1.24-2.05) 1.22 (0.99-1.51) 0.85 (0.32-2.28) 1.22 (0.77-1.93) 1.12 (0.81-1.54)
Atrial fibrillation 1.01 (0.67-1.51) 0.89 (0.71-1.12) 0.93 (0.78-1.11) 1.01 (0.48-2.09) 1.01 (0.69-1.47) 0.83 (0.64-1.08)
Cerebrovascular disease 0.40 (0.15-1.10) 1.45 (1.02-2.06) 1.17 (0.87-1.58) 2.29 (0.85-6.22) 1.57 (0.87-2.83) 0.97 (0.60-1.56)
Peripheral arterial disease 0.81 (0.37-1.79) 1.03 (0.69-1.56) 1.17 (0.84-1.61) 1.74 (0.50-5.99) 1.74 (0.89-3.37) 1.38 (0.85-2.25)
Cardiology assitance
Emergency assistance (1 year) 1.04 (1.01-1.06) 1.02 (1.00-1.04) 1.04 (1.03-1.06) 0.99 (0.85-1.14) 0.86 (0.78-0.95) 0.94 (0.89-0.98)
Delay time
0-7 days (ref) 1.00 1.00 1.00 1.00 1.00 1.00
8-14 days 0.85 (0.54-1.36) 0.99 (0.77-1.27) 0.99 (0.81-1.21) 2.38 (0.82-6.89) 2.51 (1.54-4.09) 1.58 (1.15-2.16)
15-30 days 0.91 (0.49-1.68) 1.20 (0.85-1.69) 1.26 (0.95-1.67) 1.16 (0.21-6.52) 0.92 (0.41-2.08) 1.36 (0.85-2.18)
>30 days 1.55 (0.85-2.86) 1.31 (0.89-1.92) 1.29 (0.90-1.82) 1.74 (0.16-19.47) 2.18 (0.85-5.57) 1.87 (1.02-3.43)
Assitance model
Econsult (ref) 1.00 1.00 1.00 1.00 1.00 1.00
In-person consultation 0.84 (0.49-1.46) 1.03 (0.75-1.42) 3.15 (2.37-4.20) 0.34 (0.07-1.77) 1.48 (0.78-2.81) 0.85 (0.55-1.32)
In bold: statistically significant factors (p<0.05)
Abbrevations: OR: odds ratio; CI: confidence interval; CV, cardiovascular; HF, heart failure; HF-r, heart failure-related
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated