Prerpints.org logo
Preprint
Article

The Impact of Virtual Reality as a Rehabilitation Method using TRAVEE system on Functional Outcomes, and Disability in Stroke Patients: A Pilot Study

Altmetrics

Downloads

75

Views

57

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

25 September 2024

Posted:

26 September 2024

You are already at the latest version

Alerts
Abstract
Background: Stroke is the third leading cause of disability, primarily due to motor, sensory, and cognitive impairments. Virtual reality (VR) has shown promising results in post-stroke rehabilita-tion, particularly for upper limb recovery. The World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0), aligned with the International Classification of Functioning, Disa-bility, and Health (ICF) principles, is a valid tool for measuring disability regardless of its cause. This study aimed to investigate the feasibility of VR TRAVEE system in upper limb rehabilitation for stroke patients. Methods: 14 stroke patients with residual hemiparesis were enrolled in the study. They underwent a 10-day program combining conventional therapy (CnvT) with VR rehabilitation. At baseline (T0), upper limb was assessed using the Modified Ashworth Scale (MAS), Active Range of Motion (AROM), and the Numeric Rating Scale (NRS) for pain. These assessments were repeated after the 10-day rehabilitation program (T1). Additionally, disability was measured using WHODAS 2.0 at T0 and again 30 days after completing the program. Results: Significant im-provements were observed in AROM, MAS scores for the shoulder, elbow, wrist, and metacar-pophalangeal joints as well as in the reduction of shoulder pain (p˂0.001). WHODAS scores de-creased across all six domains, with a statistically significant improvement in the cognition domain (p=0.011). Conclusions: Combining CnvT with VR as a rehabilitation approach enhances motor function in the upper limb. This method has the potential to reduce disability scores and promote neuroplasticity.
Keywords: 
Subject: Medicine and Pharmacology  -   Neuroscience and Neurology

1. Introduction

The second cause of death worldwide and the third leading cause of Disability- Adjusted Life Years (DALY), is represented by stroke, a common health issue, that often leads to disability [1]. Recently, stroke is associated with a substantially incremental burden increased from 1990 to 2019, with low- and lower-middle-income countries contributing considerably to this increase. There was a 70% increase in incidence and 102% in prevalence, a 43% increase in death rate, and a 143% increase in DALYs [2]. Worldwide, the incidence of stroke is increasing in young and middle-aged people (<55 years) [3]. Even though most patients survive a stroke, the long-term deficits caused by motor, sensory and cognitive impairments, that lead to diminished functionality, activity limitation, and participation restriction have the biggest impact on the patient and their family [4,5].
Motor impairment, which affects about 80% of stroke survivors, is one of the most well-known effects of stroke. This impairment is due to lack of central nervous system control over the muscles and movements in one side of the body's face, hand, and leg muscles, which limits mobility and affects the patient's functional abilities [6,7]. To help the patient regain a level of functioning as close as possible to that it was prior to the event, a large portion of the medical rehabilitation team's effort is directed toward motor rehabilitation. Changes in motor ability can be achieved through three different pathways: restitution, substitution, or compensation [8]. Levin et al. differentiated between motor compensation and motor rehabilitation based on the International Classification of Functioning, Disability, and Health (ICF). They concluded that motor rehabilitation included the successful completion of tasks performed, the restoration of lost function in neural tissue, and the ability to perform movement, all in the same way as before the event. In these areas, motor compensation can take several forms, such as the brain tissue learning a new function that was absent before the damage, performing the movement in a novel fashion, and utilizing alternative methods to complete tasks [9]. Following a brain injury, the neurons in the affected areas must reorganize to facilitate the recovery of motor functions. Behavior is a result of this restructured brain activity, and the process of learning new skills promotes adaptive neural restructuring, which in turn enhances brain plasticity. Repetition alone, however, does not determine reorganization. This occurs only when learning is a part of the experience. Thus, the goal of motor rehabilitation should be to promote plasticity in patients by giving them experiences that test their motor abilities. Furthermore, it has been demonstrated that motor learning concepts like intensity, repetition, task orientation, and feedback influence how well a person functions following a stroke [10,11,12].
Virtual reality (VR) enables the construction of computer-generated environments and provides individualized experiences with respect to three senses: touch, sound, and vision [13]. Studies on VR use for post-stroke motor therapy are showing encouraging outcomes, especially for upper limb rehabilitation. Reaching, gripping, carrying, and releasing things in a virtual environment all follow similar kinematics to real-world movements, which suggests that practicing arm movements in VR can be a practical substitute [14]. VR has also been successfully used to improve distal motor control but usually combined with conventional therapy (CnvT) or robotic-assisted therapy [15,16]. It has been proposed that proprioceptive feedback can take advantage of the multimodal characteristics of goal-oriented movement monitoring and self-awareness feedback. However, without particularly addressing hand and finger dexterity, clinical research on these systems has up to this point mostly concentrated on shoulder and elbow training [17].
TRAVEE system has been in existence since 2019 and has recently undergone a substantial update. The TRAVEE system offers patients the opportunity to perform upper limb exercises through gaming, using VR. The system utilizes a Meta Quest 2 VR headset, which provides the capability to automatically track limb movements, from the wrist to the hands and fingers. The system also offers visual and auditory adjustments to diversify the virtual world context in which the patient performs the game-based exercises. In addition to the set of game-based exercises that can be performed in the virtual world, the TRAVEE system offers therapists a control panel where they can configure specific properties for each exercise. This panel has a web-based graphical interface that is accessed via a web client running on a laptop. Additionally, to view the content the patient sees in the virtual world, a casting window displaying the VR headset's image is launched alongside the control panel interface [18].
Following a stroke, rehabilitation can be challenging and often unclear. Numerous randomized clinical studies combine diverse rehabilitation techniques, such as CnvT, robot-assisted arm training, and VR therapy, based on the principles mentioned above. Motor rehabilitation is difficult to assess; the range of motion (ROM) can be improved, but this might not result in any overall functional gains [19]. Individualized rehabilitation programs that address all deficits remained after a stroke, with a particular focus on motor deficits, should be designed in a SMART (Specific, Measurable, Achievable, Relevant, and Time-bound) manner based on Goal Attainment Scaling (GAS) [20]. Additionally, assessing health and disability remains challenging. To address this, the World Health Organization (WHO) developed the WHO Disability Assessment Schedule 2.0 (WHODAS 2.0) to evaluate disability and health independent of the underlying medical cause [21,22].
The aim of this study is to test a VR system designed for stroke survivors’ rehabilitation that are based on the principles of motor learning and neural plasticity, with the goal of promoting neuromotor rehabilitation. The benefits of VR therapy were measured using the WHODAS 2.0 and ROM. A secondary goal was to evaluate the acceptance of this intervention among both patients and therapists.

2. Materials and Methods

2.1. Study Design, Protocol and Patients

The present study was conducted in accordance with the principles of the Declaration of Helsinki and received approval from the Institutional Ethics Committee of the Romanian National Institute of Rehabilitation, Physical Medicine, and Balneoclimatology, with the registration number code NRCRT6/18.09.2023 and the date of approval of 18.09.2023. All patients completed a written informed consent.
This is a single-center cohort study involving patients admitted for stroke rehabilitation program between October 5, 2023, and July 1, 2024, who completed a 10-day CnvT combined with VR rehabilitation program. To be included in this study patients had to meet the following inclusion criteria: 1) patients with ischemic or hemorrhagic stroke (confirmed by a CT/MRI brain scan): 2) moderate to severe deficit of the paretic upper limb; 3) patients with no aphasia to moderate aphasia; 4) no cognitive deficits as assessed by the Mini-Mental State Examination (MMSE ≥25); 5) cooperative behavior; 6) age ˂ 80 years. The exclusion criteria were: 1) other neurological, condition that results in paresis of the upper limb; 2) severe aphasia; 3) inability to understand instructions; 4) cognitive impairment; 5) other psychiatric impairments that limit the ability to participate or give informed consent; 6) patients who did not respond to WHODAS 2.0 questionnaire reevaluation after 30 days.
From the patients' medical history, data regarding the type of stroke (ischemic or hemorrhagic), the time passed since the onset of stroke, comorbidities (the presence or absence of high blood pressure (HBP), type 2 diabetes mellitus (T2DM), dyslipidemia, and atrial fibrillation (AF)) were collected. Dyslipidemia was considered if they were undergoing treatment with lipid-lowering therapy, or if they had a total cholesterol level over 200 mg/dl or low-density lipoprotein cholesterol (LDL-C) greater than 130 mg/dl. HBP was defined as having a systolic blood pressure (SBP) or diastolic blood pressure (DBP) ≥140/90 mmHg. Patients were considered to have T2DM if they were receiving hypoglycemic medication or if their glycated hemoglobin (HbA1c) was ≥6.5%. Patients were considered with AF if they had an episode on electrocardiogram of AF. Additionally, information on the patients’ alcohol and smoking habits (whether they were smokers or non-smokers) and their dominant hand was collected through direct questioning. Alcohol consumption was measured using an online calculator recommended on the internet [23]. Less than 7 units of alcohol per week for women and 14 units per week for men were considered a normal dosage and were categorized as non-alcohol usage [24].
The patients were evaluated at two different time points: at the admission in the rehabilitation unit (T0), before rehabilitation program was applied, and after completing 10 days of CnvT combined with VR rehabilitation (T1), and again for disability assessment with WHODAS 2.0 questionnaire at 30 days after finishing the 10 days rehabilitation program.
At T0, patients were assessed using the Modified Ashworth Scale (MAS) for shoulder abductors, elbow flexors, pronators, and finger flexors. Additionally, active range of motion (ROM) was evaluated for shoulder abductors, shoulder flexors, internal and external shoulder rotation, elbow flexors, pronation, supination, wrist extension and flexion, and flexion and extension of the metacarpophalangeal joint. Shoulder pain was measured using the Numeric Rating Scale (NRS). The WHODAS 2.0 questionnaire was also administered to assess disability and functioning.
At T1, after completing the 10-day rehabilitation program, these assessments were repeated, including MAS, ROM evaluations, and NRS for shoulder pain.
30 days post-discharge, patients were contacted via telephone and reassessed with the WHODAS 2.0 questionnaire to evaluate their long-term progress

2.2. CnvT and VR Rehabilitation Program

Specific to each rehabilitation department center, CnvT represents a variety of therapies and rehabilitation procedures. For the patients included strength exercises, aerobic exercises, both passive and active range of motion (ROM) exercises, stretching, balance training, occupational therapy sessions, and speech therapy when needed. In addition to physical exercises, other physical agents were used, such as electrotherapy with electrical stimulation, pain management electrotherapy, ultrasound therapy, massage, and the application of splints or orthoses for spastic limb posture. The conventional therapy consisted of 1.5 hours per day, divided into two sessions, 5 days/week, over a 10-day rehabilitation program. All patients followed the same program.
VR rehabilitation was carried out using the TRAVEE system, which is designed for neuromotor rehabilitation of the upper limb after stroke. The TRAVEE system incorporates multiple advanced technologies (Functional Electrical Stimulation, electromyography, visual and auditory feedback) that cover the rehabilitation needs of most patients. However, in this study, we utilized a Meta Quest 2 VR headset, which provides the capability to automatically track limb movements, from the wrist to the hands and fingers. This way, patients can see their hands in the virtual world and synchronize their hand movements with specific tasks they need to accomplish, without the need to hold a hardware device like a controller. During the exercises, the TRAVEE system provided auditory feedback and visual augmentation of the patient’s movements to enhance motivation during rehabilitation sessions [18,25]. In Figure 1, on the right, a patient using the TRAVEE system can be seen. The patient is wearing a Meta Quest 2 VR device and is performing a hand movement to complete a task in the virtual world.
There were 10 games that patients could play: Each game was designed to train different impairments of the upper limb, and at the same time offer an engaging, immersive and motivating experience. The games are summarized in the following Table 1.
Each game was designed to train different impairments of the upper limb. For instance, if the goal was to train more abduction and flexion of the shoulder, the selected games were Fire game and Rocket game. To train internal and external rotation of the shoulder, patients were assigned to the cooking game. For training the flexors and extensors of the elbow, patients were assigned to the Fruit games—specifically, Fruit to Basket game for extension and supination, and Fruit to Mouth game for flexion and pronation. If the aim was to train supination and pronation, the patient played the Road game. Lastly, for training flexion and extension of the wrist, patients engaged in the Piano game and Diorama game.

2.3. Muscle Tone Assessment

The muscle tone of the upper extremity in the shoulder, elbow, and wrist was assessed using the MAS. MAS is a numerical scale that grades spasticity on a scale of 0 to 4. A score of 0 indicates no increase in muscle tone, while a score of 4 indicates rigid flexion or extension. The scale includes an additional grade, 1+, which represents a slight increase in muscle tone, manifesting as a catch, followed by minimal resistance throughout less than half of the range ROM.[26]. To ensure accuracy and reliability, two doctors trained in MAS evaluation independently assessed each patient. If there were any disagreements, they reached a consensus on the final score.

2.4. Range of Motion Joint Assessment

Range of motion (ROM) was assessed in the shoulder, elbow, wrist, and metacarpophalangeal joint. ROM was evaluated actively (AROM). ROM is the range of motion, expressed in degrees, that can be obtained at one or more joints and covers the start and finish of a movement in a certain plane. It shows the angle that the joint travels from its anatomical location to its maximum range of motion. ROM was measured using a goniometer for each joint and delivered in degrees. AROM represents the arc of motion produced by the individual's voluntary, unassisted muscle contraction. [27].

2.5. Health and Disability Evaluation

The health and disability status of all patients was assessed using WHODAS 2.0 questionnaire, which was administered through interviews conducted at admission, and 30 days following the rehabilitation program. The 36-item version of the WHODAS 2.0 was administered to patients who continued to work or engaged in school activities. The 32-item version was used for those were retired or unemployed. The scoring of WHODAS 2.0 was performed using the complex scoring method available from the WHO website

2.6. Statistical Analysis

The data was organized using Excel Office 365, Version 16.0, for Windows 10 and Gnu PSPP 1.4.1 software, which facilitated a thorough statistical assessment of the parameters. Patient characteristics were described as mean and standard deviation (SD) for normally distributed data or median and interquartile range (IQR), for non-normally distributed data. Categorical variables were described using frequencies and percentages The distribution of the data was assessed using the Kolmogorov-Smirnov and Shapiro-Wilk tests. Variables with p-values greater than 0.05 in both tests were considered normally distributed, allowing for parametric analysis. For variables with p-values less than 0.05, non-parametric methods were applied. For normally distributed variables, paired sample t-tests were conducted to evaluate the significance of changes between baseline (T0) and post-intervention (T1) measurements. For non-normally distributed variables, the Wilcoxon Signed-Rank Test was used to assess changes between T0 and T1. Correlation analyses were performed using the Pearson correlation coefficient for normally distributed data and the Spearman correlation coefficient for non-normally distributed data. The scoring of WHODAS 2.0 was performed using the complex scoring method available from the WHO website [28]. p<0.05 was considered statistically significant.

3. Results

A total of 169 consecutive inpatients were screened for eligibility criteria. Among them, 59 met the inclusion criteria and were deemed eligible. Of these, 41 patients participated only in trial sessions using the TRAVEE VR system. Subsequently, 18 patients with post-stroke hemiparesis were enrolled to complete a 2-week CnvT combined with a VR rehabilitation program. Of these, one patient discontinued the program due to a Clostridium difficile infection, which required transfer to a gastroenterology unit, and 3 patients did not respond to phone call for the WHODAS 2.0 assessment at 30 days. Ultimately, 14 patients were included in the study and analyzed as shown in Figure 1. The study included 12 males and 2 females, with a mean age of 55.29±8.6 years. The participants had an average education duration of 13.36±2.72 years. Regarding employment, 71.42% were retirees, while 28.57% continued with paid work. Most participants (78.57%, 11 patients) were from urban areas, and 21.42% (3 patients) were from rural areas.
In terms of lifestyle, 50% of the participants were smokers, and 3 patients (21.42%) consumed alcohol. All patients were right-handed. Stroke characteristics showed that 11 patients (78.57%) had ischemic strokes, and 3 patients (21.42%) had hemorrhagic strokes. Among those with ischemic strokes, 2 patients received thrombolysis treatment.
Preprints 119349 g002
Figure 2. STROBE flow chart of the study. STROBE: STrengthening the Reporting of OBservational studies in Epidemiology.
Figure 2. STROBE flow chart of the study. STROBE: STrengthening the Reporting of OBservational studies in Epidemiology.
Preprints 119349 g002
Most patients (85.71%, 12 patients) had left-sided involvement, while 14.29% (2 patients) had right-sided involvement. The median time since stroke onset was 26.25 months, with an interquartile range of 5.25 to 31.5 months. The average MMSE score was 27.57±2.03. Additionally, 8 patients received botulinum toxin injections, with a mean time of 1.81 months since the last injection as seen in Table 2.
Regarding comorbidities, 13 patients had HBP (92.86%), 3 patients had T2DM (21.43%), 12 patients had dyslipidemia (85.71%), and 2 patients had AF (14.29%), as seen in Table 3.
The analysis of the Modified Ashworth Scale (MAS) for various muscle groups between T0 and T1 showed statistically significant reductions in spasticity for most of the measured endpoints, as seen in Table 4.
The analysis revealed statistically significant improvements in Active Range of Motion (AROM) and reductions in Numeric Rating Scale (NRS) shoulder pain between T0 and T1, as seen in Table 5.
WHODAS 2.0 scores across all six domains are presented in Table 6, both at T0 (before the rehabilitation program) and 30 days after the completion of the combined individual rehabilitation program.

4. Discussion

This observational study aimed to assess the impact of using immersive VR, respectively TRAVEE system in association with CnvT as a rehabilitation technique, on the upper limb motor impairment and functional outcomes in stroke patients with residual hemiparesis. Upper limb motor functions were evaluated using the MAS and AROM before and after 10 days of the combined rehabilitation program. Functional outcomes were assessed using the WHODAS 2.0 questionnaire administered before treatment and 30 days after completion. Existing data demonstrate that the combination of VR with CnvT improve both motor and cognitive functions, data that are similar to our results, were the cognition domain (D1) decrease significantly after treatment [29]. Furthermore, the effectiveness of immersive VR is supported by its significant advantages in upper limb rehabilitation, as evidenced by improvements in motor impairment measured using the Fugl-Meyer Arm Scale and functional abilities assessed with the Box and Blocks Test and the Manual Function Test, when compared to no therapy [30]. Moreover, studies show that, after a four-week follow-up, there is a significant difference in WHODAS 2.0 scores in stroke patients who underwent 270 minutes of CnvT combined with VR as rehabilitation program [31]. This highlights the potential of VR to significantly improve motor functioning and overall recovery in stroke patients.
The mean age of the cohort was 55.29±8.6 years, with a predominance of male patients 85.71%. Being middle-aged, the patients did not have the conditions commonly associated with older adults, such as hearing loss, cataracts, or other visual acuity issues like macular degeneration, which could otherwise be significant barriers to using new technologies. Additionally, it is likely that these patients had previous contact with new technologies and are more familiar with using them, rather than the older population who may be less technologically adept [32]. The average education level was 13.36±2.72 years, which may indicate a higher capacity for understanding more complex instructions. Most of the patients (70.4%) were married, indicating that they had support, were not living alone, and could receive help when needed. Most patients (85.71%) had the left side of their body affected by the stroke, but all were right-handed, indicating that the dominant hemisphere was not affected in most cases. Although 14.27% of patients had strokes affecting the dominant hemisphere, they did not exhibit significant aphasia, even though most studies indicate that strokes in the dominant hemisphere are often accompanied by aphasia [33]. The cohort studies have diverse comorbidities, such as HBP, T2DM, dyslipidemia, and atrial fibrillation, which can influence the rehabilitation program [34]. For instance, oxidative stress, which is commonly associated with these conditions, may impact the efficacy of rehabilitation. Physical activities and various supplements typically used in comprehensive rehabilitation programs can contribute to the reduction of plasma fasting glucose, serum oxidized low-density lipoprotein (LDL), and oxidative stress. This suggests that a rehabilitation program incorporating physical activity may help lower fasting glucose levels, reduce dyslipidemia, and decrease hypertension [35,36,37].
In our study, the MAS scores decreased significantly after 10 days of combined VR with CnvT in shoulder adductors, elbow flexors, pronators, wrist flexors, and finger flexors. These muscles were assessed because they are the most commonly affected by spasticity following a stroke, often corresponding to postural patterns Type III and IV of spasticity as classified by Hefter [38]. Patients without spasticity have much better motor recovery and functional outcomes than patients with spasticity after rehabilitation programs, particularly in self-care, mobility, and sphincter control [39]. The median time since stroke onset is 26.25 (5.25 to 31.5) months, and studies show that spasticity increases within the first 12 months after the onset of stroke. Studies have reported a prevalence of 17% for spasticity one year after a first-ever stroke, with only 4% experiencing disabling spasticity [40]. Other studies have reported a prevalence of 42.4% of spasticity in patients admitted for stroke rehabilitation [39]. 8 out of 14 patients were injected with botulinum toxin for spasticity, with the median time since the last injection being 1.81 months. The injections were administered only to spastic muscles if the spasticity affected the patient’s functionality, indicating that spasticity impacted the functionality of more than half of the patients. As a result, the reduction in spasticity demonstrated in our study could contribute to improvements in AROM and reductions in WHODAS 2.0 scores, thereby being associated with a decrease in disability. The VR TRAVEE system, when paired with occupational therapy, has the potential to improve upper limb function. However, more research is needed to evaluate whether VR can completely replace traditional occupational therapy. Despite technological developments, the presence of a therapist is still required, particularly for patients with high spasticity and poor motor control, to guide and support them in achieving complete ROM during exercises. [41].
Our study focused on measuring AROM, as we are particularly interested in assessing what patients can achieve independently. Given that the median time since stroke onset is 26.25 (5.25 to 31.5), we fist aim to improve the functionality of our patients. Study shows that in the acute phase (the first three months), patients who received PROM exercises showed significant improvements in motor function, as measured by muscle strength using the Oxford Scale, compared to those who received CnvT without specific PROM [42]. For recovery after stroke, it is better to use both PROM and AROM exercises. AROM exercises are more effective in improving motor function and muscle strength when patients can independently move their joints, while PROM exercises are beneficial for patients with severe motor impairments who cannot perform active movements [43]. In our study, the values of upper limb AROM indicate that patients had sufficiently good motor control to perform the VR exercises. This suggests that the improvements observed are likely attributable not only to the effectiveness of the rehabilitation program but also to the patients' ability to actively engage with the VR system. Taking into consideration the MMSE mean value of 27.50 (ranging from 25 to 30), this indicates that the patients did not have significant cognitive deficits and possessed the cognitive capacity to understand the exercises and instructions. The significant gains across various joints demonstrate that the patients could successfully participate in and benefit from the VR exercises, further reinforcing the role of VR as a viable tool for enhancing motor recovery in individuals with stroke.
One of the most common complications of stroke is shoulder pain, which often creates activity limitations, especially during dressing and ambulation [44]. In our study, shoulder pain, as measured by the Numerical Rating Scale (NRS), was significantly reduced after combined CnvT and VR therapy, from 2.28 (1.85) to 2.28 (1.85), with a p-value of ˂0.001. Some VR technologies that incorporate relaxation programs can effectively reduce pain during rehabilitation. Different virtual environments can have varying effects on pain perception [45]. VR can enhance patient engagement in rehabilitation programs by providing an immersive and distracting experience [46]. Lowering pain levels can help maintain motivation and adherence to rehabilitation programs. Physical therapy, including transcutaneous electrical stimulation (TENS), helps reduce pain, and botulinum toxin (BoNT) injections can also effectively alleviate pain [47,48]. In our study, all patients with pain received TENS as part of CnVT, and 8 patients benefited from BoNT injections, with an average time since the last injection of 1.81 months.
WHODAS 2.0 is a reliable tool for measuring functioning, limitation of activity, and restriction of participation in stroke survivors across all six domains. It demonstrates strong internal consistency, with Cronbach's alpha values ranging from 0.83 to 0.99. Additionally, the WHODAS 2.0 questionnaire correlates strongly with the Functional Independence Measure (FIM), a widely used tool for assessing disability in stroke patients [49]. In our study, the disability scores decreased across all six domains, with a significant reduction particularly observed in the Cognition (D1) domain (p = 0.011). The reduction in disability scores across all six domains — Mobility (D2), Self-Care (D3), Getting along with people (D4), Life Activities (D5), and Participation in Society (D6) — suggests that the combination of CnvT with VR enhances functionality in stroke survivors, having a positive impact on social interactions and interpersonal relationships. It also shows beneficial effects on daily living tasks and occupational roles, even though these changes are not statistically significant. Additionally, the improvement observed in the Cognition (D1) domain further indicates that VR training may effectively promote neuroplasticity, contributing to overall recovery [50,51].
A rehabilitation program that works best would include the following: 1) the patient actively participating in the training process; 2) training that challenges the person's skills in addition to requiring a lot of repetitions; 3) motivation and reward; 4) training and practice over an extended period of time; 5) careful planning of the training in relation to other activities; and 6) modification of other habits like sleep and diet [52] . VR TRAVEE system incorporates some of these elements, such as active participation, challenging training with many repetitions, even without the therapist present. However, CnvT remains a foundational pillar of rehabilitation and long-term recovery for stroke survivors.
The limitations of the present study are as follow: firstly, the lack of a control group, which makes it challenging to attribute the observed improvements only to the VR intervention, secondly, the limited sample size, 14 patients, that may restrict how broadly the results may be applied. Another limitation is the short (30 days) follow-up period, which lacks the sustainability of the improvements made with VR system. However, considering that the mean time since stroke onset was 26.25 months (5.25 to 31.5), this may not be a critical limitation. The inclusion criteria, which excluded patients with severe aphasia (required MMSE≥25) may selected a group of patients that is more likely to recover after rehabilitation therapy. Furthermore, the specificity of the TRAVEE VR system used, with games designed for different motor functions may limit the generalizability of the results to other VR systems. Limited functional assessment tools were used in the study, specifically WHODAS 2.0. However, it is important to note that WHODAS 2.0 is a validated and comprehensive tool for assessing disability across multiple domains, which could be seen as a strength. Lastly, an important strength to mention is that being a pilot study, this research serves as an important preliminary investigation.

5. Conclusions

Using immersive VR, respectively TRAVEE system in association with CnvT as a rehabilitation technique, in stroke patients with residual hemiparesis can significantly improve upper limb motor function, as measured by active range of motion. Additionally, disability scores assessed by WHODAS 2.0 showed reductions across all six domains, with the cognition domain being the only one to decrease significantly, suggesting the potential of VR in enhancing neuroplasticity. Further studies with larger sample sizes and extended follow-up periods are necessary to establish standardized protocols for VR use in stroke rehabilitation.

Author Contributions

Conceptualization, D.C., and C-G. P; methodology, M.B.; software, O.C.; validation, A.M., D.C. and M.B.; formal analysis, D.P.; investigation, M.S.; resources, D.P.; data curation, A.M.A.; writing—original draft preparation, C-G.P.; writing—review and editing, C.L.; visualization, M.I.; supervision, A.M.; project administration, D.C.; funding acquisition, D.P. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Ethics Committee of Romanian National Institute of Rehabilitation, Physical Medicine, and Balneoclimatology (protocol code NRCRT6, 18.09.2023).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Also, written informed consent has been obtained from the patients to publish this paper.

Data Availability Statement

The data supporting this study’s findings are available upon request from the authors. Please note that the data are also being utilized in an ongoing doctoral project evaluating novel rehabilitation assessment techniques.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Feigin, V.L.; Brainin, M.; Norrving, B.; Martins, S.; Sacco, R.L.; Hacke, W.; Fisher, M.; Pandian, J.; Lindsay, P. World Stroke Organization (WSO): Global Stroke Fact Sheet 2022. International Journal of Stroke 2022, 17, 18–29. [CrossRef]
  2. GBD 2019 Stroke Collaborators Global, Regional, and National Burden of Stroke and Its Risk Factors, 1990–2019: A Systematic Analysis for the Global Burden of Disease Study 2019. The Lancet. Neurology 2021, 20, 795. [CrossRef]
  3. Feigin, V.L.; Norrving, B.; Mensah, G.A. Global Burden of Stroke. Circulation research 2017, 120, 439–448.
  4. World Health Organization International Classification of Functioning, Disability, and Health: Children & Youth Version: ICF-CY.; World Health Organization, 2007; ISBN 92-4-154732-4.
  5. Miller, E.L.; Murray, L.; Richards, L.; Zorowitz, R.D.; Bakas, T.; Clark, P.; Billinger, S.A. Comprehensive Overview of Nursing and Interdisciplinary Rehabilitation Care of the Stroke Patient: A Scientific Statement from the American Heart Association. Stroke 2010, 41, 2402–2448. [CrossRef]
  6. Warburton, E.; Alawneh, J.A.; Clatworthy, P.L.; Morris, R.S. Stroke Management. BMJ Clinical Evidence 2010, 2010.
  7. Wade, D.T. Measurement in Neurological Rehabilitation. Current Opinion in Neurology 1992, 5, 682–686.
  8. Langhorne, P.; Coupar, F.; Pollock, A. Motor Recovery after Stroke: A Systematic Review. The Lancet Neurology 2009, 8, 741–754. [CrossRef]
  9. Dobkin, B.; Carmichael, T. Principles of Recovery after Stroke. Recovery after stroke 2005, 47–66.
  10. Levin, M.F.; Kleim, J.A.; Wolf, S.L. What Do Motor “Recovery” and “Compensation” Mean in Patients Following Stroke? Neurorehabilitation and neural repair 2009, 23, 313–319. [CrossRef]
  11. Dancause, N.; Nudo, R.J. Shaping Plasticity to Enhance Recovery after Injury. Progress in brain research 2011, 192, 273–295. [CrossRef]
  12. Kwakkel, G.; Kollen, B.; Lindeman, E. Understanding the Pattern of Functional Recovery after Stroke: Facts and Theories. Restorative neurology and neuroscience 2004, 22, 281–299.
  13. Levin, M.F.; Weiss, P.L.; Keshner, E.A. Emergence of Virtual Reality as a Tool for Upper Limb Rehabilitation: Incorporation of Motor Control and Motor Learning Principles. Physical therapy 2015, 95, 415–425. [CrossRef]
  14. Viau, A.; Feldman, A.G.; McFadyen, B.J.; Levin, M.F. Reaching in Reality and Virtual Reality: A Comparison of Movement Kinematics in Healthy Subjects and in Adults with Hemiparesis. Journal of neuroengineering and rehabilitation 2004, 1, 1–7. [CrossRef]
  15. Thornton, M.; Marshall, S.; McComas, J.; Finestone, H.; McCormick, A.; Sveistrup, H. Benefits of Activity and Virtual Reality Based Balance Exercise Programmes for Adults with Traumatic Brain Injury: Perceptions of Participants and Their Caregivers. Brain injury 2005, 19, 989–1000. [CrossRef]
  16. Ferche, O.; Moldoveanu, A.; Cinteza, D.; Toader, C.; Moldoveanu, F.; Voinea, A.; Taslitchi, C. From Neuromotor Command to Feedback: A Survey of Techniques for Rehabilitation through Altered Perception.; IEEE, 2015; pp. 1–4.
  17. Cameirao, M.S.; Badia, S.B. i; Duarte, E.; Frisoli, A.; Verschure, P.F. The Combined Impact of Virtual Reality Neurorehabilitation and Its Interfaces on Upper Extremity Functional Recovery in Patients with Chronic Stroke. Stroke 2012, 43, 2720–2728. [CrossRef]
  18. Moldoveanu, A.; Ferche, O.-M.; Moldoveanu, F.; Lupu, R.G.; Cinteză, D.; Constantin Irimia, D.; Toader, C. The TRAVEE System for a Multimodal Neuromotor Rehabilitation. IEEE Access 2019, 7, 8151–8171. [CrossRef]
  19. Gibson, L.M.; Brazzelli, M.; Thomas, B.M.; Sandercock, P.A. A Systematic Review of Clinical Trials of Pharmacological Interventions for Acute Ischaemic Stroke (1955-2008) That Were Completed, but Not Published in Full. Trials 2010, 11, 1–12. [CrossRef]
  20. ZAMPOLINI, M.; SELB, M.; BOLDRINI, P.; BRANCO, C.A.; GOLYK, V.; HU, X.; KIEKENS, C.; NEGRINI, S.; NULLE, A.; ORAL, A.; et al. The Individual Rehabilitation Project as the Core of Person-Centered Rehabilitation: The Physical and Rehabilitation Medicine Section and Board of the European Union of Medical Specialists Framework for Rehabilitation in Europe. Eur J Phys Rehabil Med 2022, 58, 503–510. [CrossRef]
  21. Ustun, T.B.; Kostanjesek, N.; Chatterji, S.; Rehm, J.; World Health Organization Measuring Health and Disability : Manual for WHO Disability Assessment Schedule (WHODAS 2.0) / Edited by T.B. Üstün, N. Kostanjsek, S. Chatterji, J.Rehm. 2010.
  22. Potcovaru, C.-G.; Salmen, T.; Bîgu, D.; Săndulescu, M.I.; Filip, P.V.; Diaconu, L.S.; Pop, C.; Ciobanu, I.; Cinteză, D.; Berteanu, M. Assessing the Effectiveness of Rehabilitation Interventions through the World Health Organization Disability Assessment Schedule 2.0 on Disability—A Systematic Review. Journal of Clinical Medicine 2024, 13, 1252. [CrossRef]
  23. Unit Calculator Available online: https://alcoholchange.org.uk/alcohol-facts/interactive-tools/unit-calculator (accessed on 15 August 2024).
  24. White, I.R.; Altmann, D.R.; Nanchahal, K. Alcohol Consumption and Mortality: Modelling Risks for Men and Women at Different Ages. Bmj 2002, 325, 191. [CrossRef]
  25. Caraiman, S.; Stan, A.; Botezatu, N.; Herghelegiu, P.; Lupu, R.G.; Moldoveanu, A. Architectural Design of a Real-Time Augmented Feedback System for Neuromotor Rehabilitation.; IEEE, 2015; pp. 850–855.
  26. Harb, A.; Kishner, S. Modified Ashworth Scale. In StatPearls [Internet]; StatPearls Publishing, 2023.
  27. Norkin, C.C.; White, D.J. Measurement of Joint Motion: A Guide to Goniometry; FA Davis, 2016; ISBN 0-8036-5847-8.
  28. WHO Disability Assessment Schedule (WHODAS 2.0) Available online: https://www.who.int/standards/classifications/international-classification-of-functioning-disability-and-health/who-disability-assessment-schedule (accessed on 30 June 2024).
  29. Faria, A.L.; Cameirão, M.S.; Couras, J.F.; Aguiar, J.R.O.; Costa, G.M.; Bermúdez i Badia, S. Combined Cognitive-Motor Rehabilitation in Virtual Reality Improves Motor Outcomes in Chronic Stroke – A Pilot Study. Front. Psychol. 2018, 9. [CrossRef]
  30. Henderson, A.; Korner-Bitensky, N.; Levin, M. Virtual Reality in Stroke Rehabilitation: A Systematic Review of Its Effectiveness for Upper Limb Motor Recovery. Topics in stroke rehabilitation 2007, 14, 52–61. [CrossRef]
  31. Dąbrowská, M.; Pastucha, D.; Janura, M.; Tomášková, H.; Honzíková, L.; Baníková, Š.; Filip, M.; Fiedorová, I. Effect of Virtual Reality Therapy on Quality of Life and Self-Sufficiency in Post-Stroke Patients. Medicina 2023, 59, 1669. [CrossRef]
  32. Kuerbis, A.; Mulliken, A.; Muench, F.; Moore, A.; Gardner, D. Older Adults and Mobile Technology: Factors That Enhance and Inhibit Utilization in the Context of Behavioral Health. Publications and Research 2017. [CrossRef]
  33. Hodgson, J.C.; Benattayallah, A.; Hodgson, T.L. The Role of the Dominant versus the Non-Dominant Hemisphere: An fMRI Study of Aphasia Recovery Following Stroke. Aphasiology 2014, 28, 1426–1447. [CrossRef]
  34. Salmen, T.; Serbanoiu, L.-I.; Bica, I.-C.; Serafinceanu, C.; Muzurović, E.; Janez, A.; Busnatu, S.; Banach, M.; Rizvi, A.A.; Rizzo, M. A Critical View over the Newest Antidiabetic Molecules in Light of Efficacy—A Systematic Review and Meta-Analysis. International Journal of Molecular Sciences 2023, 24, 9760. [CrossRef]
  35. Oprea, E.; Berteanu, M.; Cintezã, D.; Manolescu, B.N. The Effect of the ALAnerv Nutritional Supplement on Some Oxidative Stress Markers in Postacute Stroke Patients Undergoing Rehabilitation. Applied Physiology, Nutrition, and Metabolism 2013, 38, 613–620. [CrossRef]
  36. Weintraub, M.S.; Rosen, Y.; Otto, R.; Eisenberg, S.; Breslow, J.L. Physical Exercise Conditioning in the Absence of Weight Loss Reduces Fasting and Postprandial Triglyceride-Rich Lipoprotein Levels. Circulation 1989, 79, 1007–1014. [CrossRef]
  37. Carraro, E.; Schilirò, T.; Biorci, F.; Romanazzi, V.; Degan, R.; Buonocore, D.; Verri, M.; Dossena, M.; Bonetta, S.; Gilli, G. Physical Activity, Lifestyle Factors and Oxidative Stress in Middle Age Healthy Subjects. International journal of environmental research and public health 2018, 15, 1152.
  38. Doussoulin, A.; Rivas, C.; Bacco, J.; Sepúlveda, P.; Carvallo, G.; Gajardo, C.; Soto, A.; Rivas, R. Prevalence of Spasticity and Postural Patterns in the Upper Extremity Post Stroke. Journal of Stroke and Cerebrovascular Diseases 2020, 29, 105253. [CrossRef]
  39. Seok Ryu, J.; Woo Lee, J.; Il Lee, S.; Ho Chun, M. Factors Predictive of Spasticity and Their Effects on Motor Recovery and Functional Outcomes in Stroke Patients. Topics in stroke rehabilitation 2010, 17, 380–388. [CrossRef]
  40. Lundström, E.; Terént, A.; Borg, J. Prevalence of Disabling Spasticity 1 Year after First-ever Stroke. European journal of neurology 2008, 15, 533–539. [CrossRef]
  41. Chheang, V.; Lokesh, R.; Chaudhari, A.; Wang, Q.; Baron, L.; Kiafar, B.; Doshi, S.; Thostenson, E.; Cashaback, J.; Barmaki, R.L. Immersive Virtual Reality and Robotics for Upper Extremity Rehabilitation. arXiv preprint arXiv:2304.11110 2023.
  42. Hosseini, Z.-S.; Peyrovi, H.; Gohari, M. The Effect of Early Passive Range of Motion Exercise on Motor Function of People with Stroke: A Randomized Controlled Trial. Journal of caring sciences 2019, 8, 39. [CrossRef]
  43. Indrawati, I.; Sudiana, K.; Sajidin, M. Active Passive and Active-Assistive Range of Motion (ROM) Exercise to Improve Muscle Strength in Post Stroke Clients: A Systematic Review.; 2019; pp. 329–337.
  44. Lindgren, I.; Jönsson, A.-C.; Norrving, B.; Lindgren, A. Shoulder Pain After Stroke: A Prospective Population-Based Study. Stroke 2007, 38, 343–348. [CrossRef]
  45. Funao, H.; Tsujikawa, M.; Momosaki, R.; Shimaoka, M. Virtual Reality Applied to Home-Visit Rehabilitation for Hemiplegic Shoulder Pain in a Stroke Patient: A Case Report. Journal of Rural Medicine 2021, 16, 174–178. [CrossRef]
  46. Shahrbanian, S.; Simmonds, M.J. Effects of Different Virtual Reality Environments on Experimental Pain Rating in Post-Stroke Individuals with and without Pain in Comparison to Pain Free Healthy Individuals. Annual Review Of Cybertherapy And Telemedicine 2008, 653, 659.
  47. Poenaru, D.; Sandulescu, M.I.; Cinteza, D. Pain Modulation in Chronic Musculoskeletal Disorders: Botulinum Toxin, a Descriptive Analysis. Biomedicines 2023, 11, 1888. [CrossRef]
  48. Trofin, D.; Salmen, B.-M.; Salmen, T.; Trofin, D.M.; Reurean-Pintilei, D. Advancing the Diagnosis of Diabetic Neuropathies: Electrodiagnostic and Skin Autofluorescence Methods. Journal of Personalized Medicine 2024, 14, 884. [CrossRef]
  49. Küçükdeveci, A.A.; Kutlay, Ş.; Yıldızlar, D.; Öztuna, D.; Elhan, A.H.; Tennant, A. The Reliability and Validity of the World Health Organization Disability Assessment Schedule (WHODAS-II) in Stroke. Disability and rehabilitation 2013, 35, 214–220. [CrossRef]
  50. Cheung, K.L.; Tunik, E.; Adamovich, S.V.; Boyd, L.A. Neuroplasticity and Virtual Reality. Virtual reality for physical and motor rehabilitation 2014, 5–24.
  51. Hao, J.; Xie, H.; Harp, K.; Chen, Z.; Siu, K.-C. Effects of Virtual Reality Intervention on Neural Plasticity in Stroke Rehabilitation: A Systematic Review. Archives of Physical Medicine and Rehabilitation 2022, 103, 523–541. [CrossRef]
  52. Nielsen, J.B.; Willerslev-Olsen, M.; Christiansen, L.; Lundbye-Jensen, J.; Lorentzen, J. Science-Based Neurorehabilitation: Recommendations for Neurorehabilitation from Basic Science. Journal of motor behavior 2015, 47, 7–17. [CrossRef]
Preprints 119349 g001
Figure 1. The use of the TRAVEE system by a patient, marked by the letter A, and by the therapist through the control panel running on the laptop marked with the letter B. In addition to the control panel interface, the therapist has access to the content viewed by the patient via a casting interface, visible on the laptop monitor in the image. The control panel allows the therapist to modify the parameters of the game-based exercise.
Figure 1. The use of the TRAVEE system by a patient, marked by the letter A, and by the therapist through the control panel running on the laptop marked with the letter B. In addition to the control panel interface, the therapist has access to the content viewed by the patient via a casting interface, visible on the laptop monitor in the image. The control panel allows the therapist to modify the parameters of the game-based exercise.
Preprints 119349 g001
Table 1. Presentation, screenshot and description of all 10 game-based exercises used in this study, namely, Balls game, Fires game, Rocket game, Cooking game, Fruit to Basket game, Fruit to Mouth game, Road game, Piano Keys game, Piano Touch game, and Diorama game.
Table 1. Presentation, screenshot and description of all 10 game-based exercises used in this study, namely, Balls game, Fires game, Rocket game, Cooking game, Fruit to Basket game, Fruit to Mouth game, Road game, Piano Keys game, Piano Touch game, and Diorama game.
Game Description Gameplay screenshot Impairments/
movements
targeted
Balls The patients are asked to touch a set of balls coming toward them in the virtual environment. Preprints 119349 i001 Extension of the shoulder
Flexion and extension of the wrist
Fruits The patients are asked to "pick" a set of fruits floating in the air and place them in a basket in front of them. Preprints 119349 i002 Extension and supination
An alternative to the game mentioned above is one where the patients are asked to "pick" the fruits and bring them to their mouths. Flexion and pronation
Road The patients are asked to touch the entire perimeter of a curve with their hands. This curve is shaped like a branch and fills in with leaves, as shown in the adjacent figure. Preprints 119349 i003 Supination and pronation
Rocket The patients are asked to raise their hands to a certain height to launch a rocket aimed at a little monster. At regular intervals, a little monster appears in front of the patients and moves toward them. The patients have a few seconds to raise their hands to the required height to launch the rocket that stops the monster. Preprints 119349 i004 Abduction and flexion of the shoulder
Fires The patients are asked to control a stream of water coming from their hands to extinguish flames at the windows of a building. There are several types of games. Preprints 119349 i005
Cooking The patients are asked to prepare burgers on a tray in front of them. Ingredients for the burgers appear on the left and right sides of the tray, and the patients must move them to a specific position in the center of the tray to layer the burger. The first and last ingredients are always bun slices. Preprints 119349 i006 Internal and external rotation of the shoulder
Piano The patients are asked to touch the keys of a keyboard to play the piano. Markers indicating which key to press come toward the patients. Preprints 119349 i007 Flexion and extension of the wrist
An alternative version of the game described above is where the patients are only asked to touch the markers coming toward them.
Diorama The patients are asked to place 14 pieces of a diorama. The pieces appear on the left or right side of the patients and must be placed in a fixed position on a model. Preprints 119349 i008 Flexion and extension of the wrist
Table 2. Characteristics of patients.
Table 2. Characteristics of patients.
Characteristic n=14
Age (years) ± mean (SD) 55.29±8.6
Females
Males		 14.27%
85.71%
Urban
Rural		 71,42%
28,57%
Education (years) ± mean (SD) 13.36±2.72
Marital status (married) 70.4%
Employment status (retirees) 78.57%
Stroke
Ischemic
Haemorrhagic
78.57%
21.42%
Left side affected
Right side affected		 85.71%
14.27%
Months since the onset of stroke, median (IQR) 26.25 (5.25 to 31.5)
Smoker 50%
Alcohol consumer 21.42
MMSE, median (IQR) 27.50 (25 to 30)
Number of patients injected with Botulinum toxin 8
Average time since last injection (months) 1.81
SC – standard deviation, IQR – interquartile range.
Table 3. Patients’ comorbidities.
Table 3. Patients’ comorbidities.
Characteristic n=14
HBP (%) 92.86%
Type 2 Diabetes Mellitus (%) 21.43%
Dyslipidemia 85.71%
Atrial Fibrillation 14.29%
HBP – High blood pressure.
Table 4. Modified Ashworth scale assessed pre-treatemnt (T0) and post-treatment (T1).
Table 4. Modified Ashworth scale assessed pre-treatemnt (T0) and post-treatment (T1).
Clinical endpoints T0
Median (IQR)		 T1
Median (IQR)		 p
MAS shoulders adductors 1.5 (1.0 to 2.0) 1.0 (1.0 to 1.5) 0.02
MAS elbow flexors 1.5 (1.0 to 2.0) 1.0 (1.0 to 1.5) 0,008
MAS pronators 1 (1 to 1.5) 1 (1 to 1.5) 0.083
MAS wrist flexors 1.5 (0.75 to 2.0) 1 (0.75 to 1.5) 0,023
MAS fingers flexors
 1.25 (0.0 to 1.62) 1 (0.0 to 1.5) 0.034
IQR-interquartile range; MAS-modified Ashworth scale.
Table 5. Range of motion and shoulder pain assessed pre-treatemnt (T0) and post-treatment (T1).
Table 5. Range of motion and shoulder pain assessed pre-treatemnt (T0) and post-treatment (T1).
Clinical endpoints T0
Media (SD)		 T1
Media (SD)		 p
AROM abduction (°) 98.57 (18.95) 102,28 (18.28) ˂0.001
AROM shoulder flexion (°) 101,07 (31.90) 104.42 (30.85) ˂0.001
AROM internal rotation (°) 53 (5.96) 54 (5.96) ˂0.001
AROM external rotation (°) 41.07 (28.07) 45.28 (26.28) ˂0.001
AROM elbow flexion (°) 135.28 (16.16) 136.14 (14.26) ˂0.001
AROM pronation (°) 80.07 (3.51) 80.28 (3.60) ˂0.001
AROM supination (°) 61.28 (28.26) 64.28 (26.01) ˂0.001
AROM wrist flexion (°) 65.28 (14.77) 65.71 (14.67) ˂0.001
AROM wrist extension (°) 32.92 (22.27) 35.21 (21.41) ˂0.001
AROM metacarpophalangeal flexion (°) 83.00 (9.68) 83.71 (9.44) ˂0.001
AROM metacarpophalangeal extension (°) 7.64 (8.68) 10.42 (8.13) ˂0.001
NRS shoulder 2.28 (1.85) 1.68 (1.27) ˂0.001
SD- standard deviation; AROM-active range of motion; NRS-Numeric Rating Scale.
Table 6. WHODAS 2.0 scores across all six domains.
Table 6. WHODAS 2.0 scores across all six domains.
Disability categories T0 Disability percentage 30 days Disability percentage P value
Cognition 23.21% 14.88% 0.011
Mobility 39.29% 37.50% 0.714
Self-care 41.07% 36.16% 0.366
Getting along with people 27.86% 21.07% 0.142
Life activities Household 65.63% 0.363 0.363
Work or school activities 54.69% 0.391 0.392
Participation in society 34.82% 31.55% 0.340
Overall disability 40.94% 36.30% 0.127
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated