Prerpints.org logo
Preprint
Article

Error-enhancement for upper limb rehabilitation in the chronic phase after stroke: a 5-day pre-post intervention study

Altmetrics

Downloads

127

Views

75

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

13 December 2023

Posted:

14 December 2023

You are already at the latest version

Alerts
Abstract
A large proportion of chronic stroke survivors still struggle with upper limb (UL) problems in daily activities. We investigated clinical outcomes and assessed the quality of movement before and after a 5-hour error-enhancement training. The pilot study had a pre-post intervention design, recruiting 22 patients in the chronic phase post-stroke with UL motor impairments. Patients received 1-hour robot treatment for five days. Patients were assessed at baseline and after training, collecting (1) clinical (UL Fugl-Meyer assessment (FMA-UE)), action research arm test (ARAT), motor assessment scale UL and tone), (2) patient-reported (hand subscale of the stroke impact scale, motor activity log (MAL), visual analogue scale of pain and tone) and (3) kinematic (KINARM, BKIN Technologies Ltd., Canada) outcome measures. Our analysis revealed significant improvements (median im-provement (Q1-Q3); p<0.05) in (1) FMA-UE (1.0 (0.8–3.0), p<0.001) and ARAT (2 (0.8–2.0), p<0.001); (2) MAL amount of use (0.1 (0.0–0.3), p<0.001) and MAL quality of use (0.1 (0.1–0.5), p<0.001); (3) KINARM-evaluated position sense ((-0.45 (-0.81–0.09), p=0.030) after training. These findings provide insight into kinematic, clinical, and self-reported improvements in UL functioning after five hours of error-enhancement UL training.
Keywords: 
Subject: Public Health and Healthcare  -   Physical Therapy, Sports Therapy and Rehabilitation

1. Introduction

Good upper limb (UL) motor function is needed for daily life activities [1], therefore, regaining UL function is often a priority for the stroke survivor [2]. However, almost half of the people after a stroke have contralesional UL deficits that restrict UL activities [3] and remain present even after six months post-stroke [3,4]. In this chronic phase after stroke, spontaneous recovery is no longer observed, motor recovery plateaus and the motor function remains lower than the pre-stroke [5,6,7,8] However, there is still potential for enhancing UL motor function through exercise-dependent plasticity using high-dose therapy [9,10]. Rehabilitation in the chronic phase thus remains important and in order to achieve these high doses, robotic UL rehabilitation seems promising.
Recently, the use of robotic UL rehabilitation has become more widespread as it has several advantages [11]. Firstly, the number of movement repetitions can be increased in a safe manner and can be automatically captured. Several studies have shown a dose-response relationship, indicating that more repetitions result in greater motor recovery benefits [11,12,13]. Secondly, current literature shows the effectiveness of robot-based treatment in addition to conventional therapy, improving motor function [14,15] and enhancing motor learning [16,17]. Thirdly, robots enable the assessment of kinematic movement correlates, providing a means to evaluate the quality of movement [18]. Assessing the movement quality is important to understand improvements in UL capacity post-therapy, as recommended by the Stroke Recovery and Rehabilitation Roundtable [18].
One way to improve motor function and movement quality is through robot-based error-enhancement. When a person performs a movement and deviates from the intended path, the robot will enlarge this error by applying external forces. As a result, the person will try to counter this error-driven disturbance, prompting them to strengthen their control. [19]. As movement error plays an important role in learning, magnifying this error will likely stimulate this learning process [20,21], resulting in a refinement of movement coordination [22]. In addition, people after a stroke often have an impaired nervous system that is less sensitive to error and hence does not react to small errors. Augmentation of errors might make them noticeable and increase the likelihood that the patient will learn from them [21]. Besides, training with error-enhancement is a form of implicit learning [23]. Implicit learning might be more feasible for patients after a stroke as it aims to minimize the involvement of cognitive resources [24]. This is what differentiates robot-based error-enhancement from other robot-based rehabilitation.
Robot-based error-enhancement has recently been investigated in reaching studies [19,22,25]. Reaching is important for activities in daily life, but is a common problem in people after stroke [26]. Reaching movements are less smooth and appear with more variability and an abnormal speed profile compared to healthy individuals [27]. This is where robot-based error-enhancement can help. In healthy participants, error-enhancement was shown to increase the accuracy of reaching movements [19,25]. In people after a stroke, a systematic review provides the first evidence of the effectiveness of this new method on UL motor impairment [28]. One study in a group of 26 chronic stroke participants reported an improvement in clinical outcomes [29], and another showed a positive effect on patient-reported outcomes [30]. In a group of 18 chronic stroke participants improvements in a range of kinematics were identified [31]. Studies with the deXtreme prototype (BioXtreme Ltd., Israel) revealed an improvement in movement error in healthy individuals [19] and movement smoothness in a stroke population [32].
While most studies included either observation-based clinical or kinematic outcomes to evaluate the effect of training on motor performance, the combination of both outcome measures was rare and only one study included a patient-reported outcome. However, the use of patient-reported outcomes is important as they can reveal deficits in a majority of patients with stroke that are not detected using observation-based assessments [33]. Other studies included in the systematic review had small sample sizes, limited training time, and lack of a control group, resulting in inconclusive results. Lastly, most studies focused on two-dimensional movements in the horizontal plane, whereas functional reaching movements are nearly always conducted three-dimensionally (3D).
Therefore, we designed a pilot study in the chronic phase post-stroke using the deXtreme robot (BioXtreme Ltd., Israel) that allows error-enhancement during 3D reaching movements. We examined the effects of this novel robotic training approach with standardized clinical measures, kinematic measures of UL function, and patient-reported outcomes. We hypothesized that after five hours of error-enhancement training whereby participants would perform on average more than 1000 reaching movements, patients would (1) improve on clinical measures [19,25,28,32,34], (2) report better arm use in daily life [30], and (3) improve movement quality as measured with kinematics [30,31].

2. Materials and Methods

2.1. Participants

Adults with chronic stroke participated in this pilot study. They were recruited from our database and the discharge records of the University Hospitals Leuven Rehabilitation Center Pellenberg. In addition, we encouraged first-line general practitioners and physiotherapists to inform potential participants.
The inclusion criteria were: (1) first-ever stroke, (2) minimum six months after stroke, (3) maximum 85 years old, and (4) an UL motor impairment, yet no severe stiffness: having less than 66 points (maximum) on the Fugl-Meyer Assessment [35] for the UL (FMA-UE) but being able to open and close the hand five times, and bend and extend the elbow two times. Exclusion criteria were: (1) sensory aphasia (item 9 of the National Institutes of Health Stroke Scale [36]: ≤ 2/3); (2) apraxia (apraxia screen of TULIA [37]: < 9/12); (3) neglect (Star Cancellation Test [38]: < 44/54); (4) having a cognitive deficit (as defined by Mini-Mental State Examination [39]: ≤ 24/ 30), or (5) the presence of shoulder pain in rest or during active shoulder movements.

2.2. Procedure

For this study, a pre-post-intervention design was used. The total protocol duration was seven consecutive weekdays, starting with a pre-intervention assessment on day one, followed by five one-hour training sessions on five consecutive weekdays, and concluding with a post-intervention assessment on day seven. The study was conducted between January 2022 and November 2022 and obtained ethical approval from the Ethics Committee Research of KU/University Hospitals Leuven, Belgium (registration number: B3222021000614, internal ref. nr: S65699).
During the study, the DeXtreme robot (BioXtreme Ltd., Israel) was used, which is an end-effector robot (Figure 1) that focuses on the facilitation of accuracy, range of movement, stability, and smoothness of UL movements. During a game, the patient has to actively make reaching movements in a 3D space, while the robot exerts error-enhancement forces on the UL to magnify the errors. The patient is seated in a chair placed on a standardized position and restrained with seatbelts to prevent trunk compensation movements. Before each training session, the robot is calibrated. Afterwards, the system is adjusted to the patient, requiring the patient to bring the arm to 90° anteflexion and fully extend the elbow while holding the gimbal. Anti-gravitation support can be offered according to the needs of the patient.
One training session lasted one hour and consisted of two blocks of twenty-minute robot training, alternated with an active break (stretching and auto-mobilization). During the robot training, two games were played: 1) the market stand, which focused on the range of motion and the accuracy of the movement (Figure 2a), and the alchemist game, which emphasized stability and smoothness of movement (Figure 2b). Algorithms provide progression in terms of accuracy, range of movement, stability, and smoothness, depending on the performance of the patient. Each training session began with a game without error-enhancement forces to establish the participant's baseline.
On average, one does 12-20 reaching movements per game and about 12 games are played per therapy session. This results in about 192 movements per therapy session, and in total (on average) 960 reaching movements over 5 days. When needed, feedback was given by the therapist. Feedback was offered verbally, e.g. “Try to fully extend the elbow.”, or tactile by guiding the patient once in the right direction. On-screen information in both games provided real-time feedback about the successfulness of the movement performed.

2.3. Outcome Measures

Demographic and health information was collected at pre-intervention, including age, gender, working status, time after and type of stroke, lateralization of symptoms, and pre-stroke hand dominance. At pre- and post-intervention, we collected a battery of reliable and valid clinical, patient-reported and kinematic measurements.

2.3.1. Clinical Measurements

ICF body function level: UL motor impairment was assessed with the upper extremity subscale of the Fugl-Meyer Assessment (FMA-UE) [35,40,41,42]. A lower score indicates a more severe impairment. The first international stroke recovery and rehabilitation roundtable on measuring sensorimotor outcome agreed that the FMA-UE is the recommended UL motor function outcome for stroke recovery trials [43]. Two Visual Analogue Scales (VAS) were used [44] to evaluate the pain and stiffness that the patient feels in the most affected UL, through visualization on a 10-cm line on paper [45]. Zero, on the left end of the line, represented no pain or stiffness. Both scores were converted to a score on 100. The VAS was not only assessed at pre- and post-intervention but also before and after every therapy session. Lastly, The Motor Assessment Scale for tone (MAS-tone) assessed muscle tonus [46,47]. Scores higher than 4 indicate persistent hypertonicity.
ICF activity level: The Action Research Arm Test (ARAT) evaluated the functional performance of the UL [42,48]. Higher values represent better performance. The first international stroke recovery and rehabilitation roundtable on measuring sensorimotor outcomes agreed that the ARAT is the recommended UL activity outcome for stroke recovery trials [43]. Besides, the 7-point motor assessment scale [46,47,49] for the UL (MAS-UE) assessed everyday motor skills. On both scales, a higher score represents better performance.

2.3.2. Patient-Reported Measurements

The hand subscale of the Stroke Impact Scale (SIS) [50] evaluated patients' perceptions of difficulties in using the affected hand to perform five activities of daily living. The total score is converted to a 100-point scale and a higher score indicates a better perceived performance. In addition, the amount of use (MAL-AOU) and quality of the movements (MAL-QOL) of the UL during daily living tasks were measured by the upper-extremity Motor Activity Log-14 items (MAL-14) [51]. The MAL-14 is a structured interview of 14 questions. A higher score indicates a higher amount and quality of use of the affected UL.

2.3.3. Kinematic Measurements of Sensorimotor Function

The KINARM robot (BKIN Technologies Ltd., Kingston, Canada) was used to test sensorimotor impairment. This bimanual end-point robot allows 2D movements in the horizontal plane. The virtual reality screen permits the control of visual feedback. Tests with the robot are performed in a seated position, with seatbelts to restrain trunk movements and a black cloth to prevent the vision of the arms. If needed, hand fixation was provided.
To test motor function, the 4-target visually guided reaching (VGR) test was performed with the affected arm. The patients were instructed to move the cursor to a red dot, as accurately and fast as possible. Ten outcome parameters were calculated, including reaction time, speed and accuracy of reaching. All parameters were combined into a single task score with higher values meaning worse motor function [52,53].
A 4-target arm position-matching (APM) test was performed to assess the proprioception (position sense) of the affected arm. The robot brought the most affected arm into a position and the patient must actively move the less affected arm into the same position but mirrored, without any form of visual feedback. Twelve outcome parameters were calculated, covering variability and magnitude of position errors, and combined into a single task score with higher values meaning worse proprioception [53,54]. Both tests show good validity and reliability in participants with stroke [52,54,55]. Dexterit-E Explorer (version 3.9.3) was used to obtain the parameters of the VRG and the APM test.
Last, to test sensory processing, the discrimination task (DT) was performed. The patient was instructed to move the most affected arm and track down a 3-, 4- or 5-angle figure, delineated by virtual walls, which were not visible to the patient. In the next step, the patient had to draw the same figure with the less affected arm without mirroring. Visual feedback was provided on the hand position. Finally, the patient had to identify the explored figure out of six options. A more detailed description of this task is described elsewhere and was found to be valid for people in the chronic phase after stroke [56]. To analyze the parameters of the DT, Dexterit-E Explorer (version 3.9.3) and Matlab (version R2022b) were used [56]. Five parameters were calculated and combined in one factor score, as proposed by Saenen et al [56].

2.4. Statistical Analysis

Normality was checked for all variables with the Shapiro–Wilk test. Mean and standard deviations (SD) were calculated for normally distributed variables, medians with first quartile (Q1) and third quartile (Q3) for non-normally distributed variables or ordinal scales. Normally distributed variables were compared pre-post through parametric (paired t-test) and not-normally distributed variables or ordinal scales through non-parametric analysis (Wilcoxon Signed Ranks Test). Data were analyzed with IBM SPSS Statistics for Windows, version 28.0 (IBM Corp, Armonk, NY, USA) with the level of two-tailed statistical significance set at p < 0.05. As this was a pilot study, an exploratory data analysis was conducted without correction for multiple testing.

3. Results

3.1. Participant Characteristics

We recruited 22 patients and Table 1 reports the patient characteristics for the demographic and general stroke-related variables. Our sample included 12 women and 10 men with a mean age of 57 years. The mean days since stroke for the total group were 1571 days (range: 184-11751 days), showing that we recruited mostly people who experienced their stroke several years ago. For 12 patients, their right UL was most affected. The vast majority of patients (N=20) were right-handed pre-stroke.
Two patients dropped out because of adverse effects, one after the second, and one after the third therapy session. The first patient reported increased pain and tension in the neck-shoulder line and headaches, and the other reported increased stiffness in the hand. Both patients were followed up, and the complaints were resolved but the two patients decided not to continue with the study. Thus, we present analyses based on results from 20 patients.

3.2. Clinical Results

Clinical results pre- and post-intervention are presented in Table 2. The median (IQR) FMA-UE value pre-treatment was 54 (50-58) out of 66 points and the median (IQR) ARAT was 50 (39–53) out of 57 points, demonstrating that our sample included people with moderate to mild UL motor impairment. Results from pre to post-intervention analyses for the clinical variables are also presented in Table 2. A significant pre to post-intervention improvement was found for UL function measured with FMA-UE (median (IQR) improvement of 1.0 (0.8–3.0) points, p<0.001), and UL activity assessed with ARAT (median (IQR) improvement of 2 (0.8–2.0) points, p<0.001). There were no significant pre- to post-intervention differences for VAS and MAS. A detailed overview of the score per participant can be found in Appendix A.

3.3. Results in Patient-Reported Outcomes

Patient-reported results pre- and post-intervention and analyses are presented in Table 3. A significant pre to post-intervention improvement was found for the self-perceived amount of UL use evaluated by MAL-AOU (median (IQR) improvement of 0.1 (0.0–0.3) points, p<0.001), and perceived quality of movement investigated by MAL-QOM (median (IQR) improvement of 0.1 (0.1–0.5) points, p<0.001). There were no significant pre- to post-intervention differences for the SIS-hand. A detailed overview of the score per participant can be found in Appendix B.

3.4. Kinematic Results

Results of the kinematic variables collected pre- and post-intervention are presented in Table 4. For the visually guided reaching, only the posture speed, the median hand speed when the hand should be at rest, change reached significance (p<0.001) with a greater speed registered at the end of the visually guided reaching protocol. For the arm position matching task, a significant change was observed in absolute error in the X direction (p=0.048) with an error reduction in the frontal plane when matching arm positions with the less affected upper limb when the robot offers these positions to the more affected UL. Also, for the arm position matching composite task score (p=0.03), a significant improvement was noted in the overall performance on this task. For the other variables and the composite task score of the discrimination test, no significant differences were found. A detailed overview of the score per participant can be found in Appendix C.

3.5. Number of Reaching Movements during 5 Hours of Error-Enhancement Training

Table 3 presents the number of reaching movements participants performed during the 5-day intervention protocol. The mean (SD) amount was 1043 (127) reaching movements with a minimum of 723 and a maximum of 1236 movements.

4. Discussion

Our study investigated the hypothesis that five one-hour sessions on five consecutive days of reaching training incorporating error-enhancement would provide clinical and kinematic improvements in chronic stroke survivors with residual UL impairments and activity limitations. Our results support this postulation, as we observed improvements in UL motor function and capacity, perceived upper limb performance, and position sense through kinematic evaluation.
Clinically, we demonstrated significant improvements in UL motor function, UL activity and self-perceived performance, as measured by the Fugl-Meyer assessment for the upper extremity, action research arm test and motor activity log amount of use and quality of movement, respectively. Although the improvements are rather small, they are noteworthy given the relatively limited duration of our intervention. While five hours of therapy is rather limited, it is important to note that time in training may not accurately reflect training intensity [57]. The number of repetitions performed during this training is a more accurate indicator of training intensity [58], and we found that our participants on average performed 1043 reaching repetitions during the five-hour training period. Moreover, as the active error-enhancement training time was only 40 minutes per hour, we argue that our sample performed a large number of reaches within the available time, making our intervention of interest for further consideration in clinical research and practice. These findings are especially relevant when considering the provision of training in the chronic phase of stroke recovery and add to the existing body of knowledge in this domain.
Our study has several notable strengths. First and foremost, our intervention focused on the provision of a high number of reaching movements, providing a concentrated and targeted approach to UL rehabilitation. Additionally, our protocol was comprehensive, including clinical, self-reported and kinematic evaluation. This allowed us to obtain a robust understanding of the efficacy of our intervention and provides a strong foundation for future research in this area. Moreover, the clinical outcomes were well-established and widely accepted as important measures of recovery and rehabilitation in stroke survivors. This consensus-based methodology [43,59] ensures that our findings are not only relevant for research purposes but also have practical implications for clinical practice. Another important finding was the absence of a negative impact of five one-hour sessions performed every day for one week, as revealed by no changes in the visual analogue scale for tone and pain. While two participants did experience increased tension and pain and ultimately chose to discontinue the study, it is important to note that one participant was already suffering from increased tension in the neck-shoulder line before baseline, and the other had a history of increased stiffness in the UL. Based on this experience, we would consider refining our inclusion criteria for future studies to ensure that individuals with pain or increased muscle tension are more carefully screened. While we did observe significant improvements in clinical outcomes at the ICF body function (FMA-UE) and activity level (ARAT), we did not observe changes in the Motor Assessment Scale (MAS), also at activity level. The pre-score on the MAS was already high (median: 14/18), which may explain the limited progress. Moreover, there is some criticism of the hierarchy of items in this scale; the ranking of the items seems inconsistent [60,61]. There was no improvement in the self-reported stroke impact hand subscale. However, we did see an improvement in the Motor Activity Log, another self-report assessment. We believe that the MAL may be a more relevant outcome measure for our intervention than SIS-Hand, which focuses specifically on hand function and may be limited in its ability to capture improvements in overall limb function.
Our study also included kinematic UL evaluation, which showed a significant improvement in arm position matching, indicating better position sense after the intervention. This improvement may be attributed to the error-enhancement component of the training, which provided increased somatosensory input during reaching movements, which may have resulted in better somatosensory awareness [62,63], reflected in a better position-sense outcome. It would have been of interest to see whether this improvement could also have been present when position sense was tested clinically but our protocol did not include standard clinical somatosensory evaluation. We did not find any significant changes in visually-guided reaching and discrimination tasks, which may be due to the task-specific nature of these evaluations. The discrimination task evaluates movement sense and sensory discrimination, while sensory discrimination was not an element included in our training protocol. Surprisingly, we observed a worsening in posture speed, which evaluates the stability of the UL before and after reaching, when the hand should be at rest. This may be linked to anticipation during the reaching training when performing the reaches. When re-evaluating reaching with a kinematic task after training, the training anticipation may have reflected in moving quicker, however, this could have increased posture speed. There are several potential explanations for the lack of improvement in kinematic reaching. Firstly, our evaluation protocol involved two-dimensional reaching while our task involved three-dimensional reaching, which may have impacted learning. Secondly, reaching kinematics is considered a parameter of the quality of movement and may reflect restitution [43,64], which is unlikely to occur in the chronic phase after stroke. Lastly, the provided intensity may not have been sufficient to induce kinematic changes. The average number of repetitions was 1043, which may still not be enough for detecting kinematic changes in motor control of reaching.
Some limitations of our study have to be acknowledged. One limitation is that the average age of our sample was younger than the usual age of people after a stroke in Belgium [65]. The literature, however, shows that age has limited influence on motor recovery after stroke on long-term outcome measures [66,67]. Another limitation is the large range in time after stroke of our participants. However, all patients were in the chronic stage after stroke and there is currently no research indicating that long-term after stroke, movement adaptability may alter. Our intervention protocol was also limited in duration, consisting of only one hour per day for five days and no follow-up measurement. However, our main focus was on the number of reaches participants would perform, and we reached our proposed target with an average of 1043 repetitions. Further studies should include a follow-up measurement to investigate the sustainability of the improvements. In addition, our protocol included a two-dimensional kinematic analysis, while our intervention trained three-dimensional reaching. A three-dimensional reaching task would be beneficial to include in future studies to better evaluate the quality of UL movements. Unfortunately, the availability of technology is a limiting factor in study development. Therefore, a three-dimensional drinking task to evaluate the quality of UL movement is recommended by the Second Stroke Recovery and Rehabilitation Roundtable [18]. Besides, this study did not include a blinded assessor, thus we cannot exclude assessor bias for the clinical outcomes. Further studies with a blinded assessor would be beneficial to strengthen the validity of our results. Finally, we did not correct for multiple testing due to the exploratory nature of the study.
In summary, the study suggests that an hourly intervention for five days which actively stimulates reaching movements through serious gaming with error-enhancement might improve UL function, capacity, and self-reported UL performance in people in the chronic phase after stroke with mild residual impairments in UL function and activity. Future work can expand on these findings by integrating the therapy concept in an overall UL treatment package for people in the chronic phase after stroke to improve the quality of movement post-stroke.

Author Contributions

Conceptualization, M.C., E.C., R.L. and G.V.; methodology, M.C., R.L. and G.V.; formal analysis, M.C.; investigation, M.C.; resources, M.C.; data curation, M.C.; writing—original draft preparation, M.C., B.E. and G.V.; writing—review and editing, M.C., E.C., B.E., I. DB., R.L. and G.V.; visualization, M.C.; supervision, R.L. and G.V.; project administration, M.C.; funding acquisition, M.C., R.L. and G.V. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee Research (EC Research) of University Hospitals Leuven (UZ Leuven) (protocol code: S65699, date of approval: 2 December 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data analyzed during the current study are available from the corresponding author, Marjan Coremans, upon reasonable request.

Acknowledgments

The authors would like to thank Ante Schuermans for her support in data processing. In addition, we would also like to thank the participants for their contribution to the study. Robin Lemmens is a senior clinical investigator of FWO Flanders.

Conflicts of Interest

The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

Appendix A

Appendix A. Clinical outcomes per participant.
Appendix A. Clinical outcomes per participant.
Preprints 93281 i004
IQ1-IQ3: interquartile range; FMA UE: Fugl-Meyer Assessment upper extremity; VAS: Visual Analogue Scale; ARAT: Action Research Arm Test; MAS EU: Motor Assessment Scale upper extremity; * p < 0.05.
Appendix B. Patient-reported outcomes per participant.
Appendix B. Patient-reported outcomes per participant.
Preprints 93281 i005
IQ1-IQ3: interquartile range; MAL, aou / qom: Motor activity log, amount of use/quality of use; SIS-hand: hand domain of the Stroke Impact Scale.

Appendix C

Appendix C1. Kinematic outcome parameters per participant: Arm Position Matching task, part 1.
Appendix C1. Kinematic outcome parameters per participant: Arm Position Matching task, part 1.
Preprints 93281 i006
SD: standard deviation; IQ1-IQ3: interquartile range; * p < 0.05.
Appendix C2. Kinematic outcome parameters per participant: Arm Position Matching task, part 2.
Appendix C2. Kinematic outcome parameters per participant: Arm Position Matching task, part 2.
Preprints 93281 i007
SD: standard deviation; IQ1-IQ3: interquartile range.
Appendix C3. Kinematic outcome parameters per participant: Arm Position Matching task, part 3.
Appendix C3. Kinematic outcome parameters per participant: Arm Position Matching task, part 3.
Preprints 93281 i008
SD: standard deviation; IQ1-IQ3: interquartile range.
Appendix C4. Kinematic outcome parameters per participant: Visually Guided Reaching task, part 1.
Appendix C4. Kinematic outcome parameters per participant: Visually Guided Reaching task, part 1.
Preprints 93281 i009
SD: standard deviation; IQ1-IQ3: interquartile range; * p < 0.05.
Appendix C5. Kinematic outcome parameters per participant: Visually Guided Reaching task, part 2.
Appendix C5. Kinematic outcome parameters per participant: Visually Guided Reaching task, part 2.
Preprints 93281 i010
SD: standard deviation; IQ1-IQ3: interquartile range.
Appendix C6. Kinematic outcome parameters per participant: Discrimination task.
Appendix C6. Kinematic outcome parameters per participant: Discrimination task.
Preprints 93281 i011
SD: standard deviation; IQ1-IQ3: interquartile range.

References

  1. Duncan, P.W.; Goldstein, L.B.; Horner, R.D.; Landsman, P.B.; Samsa, G.P.; Matchar, D.B. Similar Motor Recovery of Upper and Lower Extremities after Stroke. Stroke 1994, 25, 1181–1188. [CrossRef]
  2. Hebert, D.; Lindsay, M.P.; McIntyre, A.; Kirton, A.; Rumney, P.G.; Bagg, S.; Bayley, M.; Dowlatshahi, D.; Dukelow, S.; Garnhum, M.; et al. Canadian Stroke Best Practice Recommendations: Stroke Rehabilitation Practice Guidelines, Update 2015. International Journal of Stroke 2016, 11, 459–484. [CrossRef]
  3. Levin, M.F.; Kleim, J.A.; Wolf, S.L. What Do Motor “Recovery” and “Compensation” Mean in Patients Following Stroke? Neurorehabil Neural Repair 2009, 23, 313–319. [CrossRef]
  4. Bernhardt, J.; Hayward, K.S.; Kwakkel, G.; Ward, N.S.; Wolf, S.L.; Borschmann, K.; Krakauer, J.W.; Boyd, L.A.; Carmichael, S.T.; Corbett, D.; et al. Agreed Definitions and a Shared Vision for New Standards in Stroke Recovery Research: The Stroke Recovery and Rehabilitation Roundtable Taskforce. International Journal of Stroke 2017, 12, 444–450. [CrossRef]
  5. Kwakkel, G.; Kollen, B.; Lindeman, E. Understanding the Pattern of Functional Recovery after Stroke: Facts and Theories. Restor Neurol Neurosci 2004, 22, 281–299.
  6. Langhorne, P.; Bernhardt, J.; Kwakkel, G. Stroke Rehabilitation. The Lancet 2011, 377, 1693–1702. [CrossRef]
  7. Page, S.J.; Gater, D.R.; Bach-Y-Rita, P. Reconsidering the Motor Recovery Plateau in Stroke Rehabilitation. Arch Phys Med Rehabil 2004, 85, 1377–1381. [CrossRef]
  8. Demain, S.; Wiles, R.; Roberts, L.; McPherson, K. Recovery Plateau Following Stroke: Fact or Fiction? Disabil Rehabil 2006, 28, 815–821. [CrossRef]
  9. Ward, N.S.; Brander, F.; Kelly, K. Intensive Upper Limb Neurorehabilitation in Chronic Stroke: Outcomes from the Queen Square Programme. J Neurol Neurosurg Psychiatry 2019, 90, 498–506. [CrossRef]
  10. Ballester, B.R.; Ward, N.S.; Brander, F.; Maier, M.; Kelly, K.; Verschure, P.F.M.J. Relationship between Intensity and Recovery in Post-Stroke Rehabilitation: A Retrospective Analysis. J Neurol Neurosurg Psychiatry 2022, 93, 226–228. [CrossRef]
  11. Pollock, A.; Farmer, S.E.; Brady, M.C.; Langhorne, P.; Mead, G.E.; Mehrholz, J.; van Wijck, F. Interventions for Improving Upper Limb Function after Stroke. Cochrane Database of Systematic Reviews 2014, 2014. [CrossRef]
  12. Cooke, E. V.; Mares, K.; Clark, A.; Tallis, R.C.; Pomeroy, V.M. The Effects of Increased Dose of Exercise-Based Therapies to Enhance Motor Recovery after Stroke: A Systematic Review and Meta-Analysis. BMC Med 2010, 8. [CrossRef]
  13. Hsieh, Y.W.; Wu, C.Y.; Liao, W.W.; Lin, K.C.; Wu, K.Y.; Lee, C.Y. Effects of Treatment Intensity in Upper Limb Robot-Assisted Therapy for Chronic Stroke: A Pilot Randomized Controlled Trial. Neurorehabil Neural Repair 2011, 25, 503–511. [CrossRef]
  14. Chang, W.H.; Kim, Y.-H. Robot-Assisted Therapy in Stroke Rehabilitation. J Stroke 2013, 15, 174–181. [CrossRef]
  15. Fasoli, S.E.; Krebs, H.I.; Stein, J.; Frontera, W.R.; Hogan, N. Effects of Robotic Therapy on Motor Impairment and Recovery in Chronic Stroke. Arch Phys Med Rehabil 2003, 84, 477–482. [CrossRef]
  16. Boyd, L.A.; Vidoni, E.D.; Wessel, B.D. Motor Learning after Stroke: Is Skill Acquisition a Prerequisite for Contralesional Neuroplastic Change? Neurosci Lett 2010, 482, 21–25. [CrossRef]
  17. Wilkins, K.B.; Owen, M.; Ingo, C.; Carmona, C.; Dewald, J.P.A.; Yao, J. Neural Plasticity in Moderate to Severe Chronic Stroke Following a Device-Assisted Task-Specific Arm/Hand Intervention. Front Neurol 2017, 8, 1–11. [CrossRef]
  18. Kwakkel, G.; Van Wegen, E.; Burridge, J.; Winstein, C.; van Dokkum, L.; Alt Murphy, M.; Levin, M.; Krakauer, J. Standardized Measurement of Quality of Upper Limb Movement after Stroke: Consensus-Based Core Recommendations from the Second Stroke Recovery and Rehabilitation Roundtable. International Journal of Stroke 2019, 14, 783–791. [CrossRef]
  19. Israely, S.; Leisman, G.; Carmeli, E. Improvement in Hand Trajectory of Reaching Movements by Error-Augmentation. Adv Exp Med Biol 2018, 1070, 71–84. [CrossRef]
  20. Kleim, J.A.; Jones, T.A. Principles of Experience-Dependent Neural Plasticity: Implications for Rehabilitation after Brain Damage. J Speech Lang Hear Res 2008, 51. [CrossRef]
  21. Patton, J.L.; Huang, F.C. Sensory-Motor Interactions and Error Augmentation. Neurorehabilitation Technology, Second Edition 2016, 79–95. [CrossRef]
  22. Patton, J.L.; Mussa-Ivaldi, F.A. Robot-Assisted Adaptive Training: Custom Force Fields for Teaching Movement Patterns. IEEE Trans Biomed Eng 2004, 51, 636–646. [CrossRef]
  23. Izawa, J.; Criscimagna-Hemminger, S.E.; Shadmehr, R. Cerebellar Contributions to Reach Adaptation and Learning Sensory Consequences of Action. Journal of Neuroscience 2012, 32, 4230–4239. [CrossRef]
  24. Masters, R.; Maxwell, J. Implicit Motor Learning, Reinvestment and Movement Disruption : What You Don’t Know Won’t Hurt You. Skill Acquisition in Sport 2004, 231–252. [CrossRef]
  25. Williams, C.K.; Tremblay, L.; Carnahan, H. It Pays to Go Off-Track: Practicing with Error-Augmenting Haptic Feedback Facilitates Learning of a Curve-Tracing Task. Front Psychol 2016, 7, 2010. [CrossRef]
  26. Parker, V.M.; Wade, D.T.; Hewer, R.L. Loss of Arm Function after Stroke: Measurement, Frequency, and Recovery. Int Rehabil Med 1986, 8, 69–73. [CrossRef]
  27. Cirstea, M.C.; Levin, M.F. Compensatory Strategies for Reaching in Stroke. Brain 2000, 123 ( Pt 5), 940–953. [CrossRef]
  28. Israely, S.; Carmeli, E. Error Augmentation as a Possible Technique for Improving Upper Extremity Motor Performance after a Stroke - A Systematic Review. Top Stroke Rehabil 2016, 23, 116–125. [CrossRef]
  29. Abdollahi, F.; Case Lazarro, E.D.; Listenberger, M.; Kenyon, R. V.; Kovic, M.; Bogey, R.A.; Hedeker, D.; Jovanovic, B.D.; Patton, J.L. Error Augmentation Enhancing Arm Recovery in Individuals with Chronic Stroke: A Randomized Crossover Design. Neurorehabil Neural Repair 2014, 28, 120–128. [CrossRef]
  30. Abdollahi, F.; Corrigan, M.; Lazzaro, E.D.C.; Kenyon, R. V.; Patton, J.L. Error-Augmented Bimanual Therapy for Stroke Survivors. NeuroRehabilitation 2018, 43, 51–61. [CrossRef]
  31. Tropea, P.; Cesqui, B.; Monaco, V.; Aliboni, S.; Posteraro, F.; Micera, S. Effects of the Alternate Combination of “Error-Enhancing” and “Active Assistive” Robot-Mediated Treatments on Stroke Patients. IEEE J Transl Eng Health Med 2013, 1, 2100109–2100109. [CrossRef]
  32. Givon-Mayo, R.; Simons, E.; Reuth, A.O.; Karpin, H.; Israely, S.; Carmeli, E. A Preliminary Investigation of Error Enhancement of the Velocity Component in Stroke Patients’ Reaching Movements. Int J Ther Rehabil 2014, 21, 160–168. [CrossRef]
  33. Stewart, J.C.; Cramer, S.C. Patient-Reported Measures Provide Unique Insights into Motor Function after Stroke. Stroke 2013, 44, 1111–1116. [CrossRef]
  34. Liu, L.Y.; Li, Y.; Lamontagne, A. The Effects of Error-Augmentation versus Error-Reduction Paradigms in Robotic Therapy to Enhance Upper Extremity Performance and Recovery Post-Stroke: A Systematic Review. J Neuroeng Rehabil 2018, 15, 1–25. [CrossRef]
  35. Fugl Meyer, A.R.; Jaasko, L.; Leyman, I. The Post Stroke Hemiplegic Patient. I. A Method for Evaluation of Physical Performance. Scand J Rehabil Med 1975, 7, 13–31.
  36. Brott, T.; Adams, H.P.; Olinger, C.P.; Marle, J.R.; Barsan, W.G.; Biller, J.; Spilker, J.; Holleran, R.; Eberle, R.; Hertzberg, V.; et al. Measurements of Acute Cerebral Infarction: A Clinical Examination Scale. Stroke 1989, 20, 864–870. [CrossRef]
  37. Vanbellingen, T.; Kersten, B.; Van De Winckel, A.; Bellion, M.; Baronti, F.; Müri, R.; Bohlhalter, S. A New Bedside Test of Gestures in Stroke: The Apraxia Screen of TULIA (AST). J Neurol Neurosurg Psychiatry 2011, 82, 389–392. [CrossRef]
  38. Wilson, B.; Cockburn, J.; Halligan, P. Development of a Behavioral Test of Visuospatial Neglect. Arch Phys Med Rehabil 1987.
  39. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-Mental State”: A Practical Method for Grading the Cognitive State of Patients for the Clinician. J Psychiatr Res 1975, 12, 189–198. [CrossRef]
  40. Kim, H.; Her, J.; Ko, J.; Park, D.S.; Woo, J.H.; You, Y.; Choi, Y. Reliability, Concurrent Validity, and Responsiveness of the Fugl-Meyer Assessment (FMA) for Hemiplegic Patients. J Phys Ther Sci 2012, 24, 893–899. [CrossRef]
  41. Page, S.J.; Fulk, G.D.; Boyne, P. Clinically Important Differences for the Upper-Extremity Fugl-Meyer Scale in People with Minimal to Moderate Impairment Due to Chronic Stroke. Phys Ther 2012, 92, 791–798. [CrossRef]
  42. Lin, J.-H.; Hsu, M.-J.; Sheu, C.-F.; Wu, T.-S.; Lin, R.-T.; Chen, C.-H.; Hsieh, C.-L. Psychometric Comparisons of 4 Measures for Assessing Upper-Extremity Function in People With Stroke. 2009. [CrossRef]
  43. Kwakkel, G.; Lannin, N.A.; Borschmann, K.; English, C.; Ali, M.; Churilov, L.; Saposnik, G.; Winstein, C.; van Wegen, E.E.H.; Wolf, S.L.; et al. Standardized Measurement of Sensorimotor Recovery in Stroke Trials: Consensus-Based Core Recommendations from the Stroke Recovery and Rehabilitation Roundtable. International Journal of Stroke 2017, 12, 451–461. [CrossRef]
  44. Williamson, A.; Hoggart, B. Pain: A Review of Three Commonly Used Pain Rating Scales. J Clin Nurs 2005, 14, 798–804. [CrossRef]
  45. Klimek, L.; Bergmann, K.C.; Biedermann, T.; Bousquet, J.; Hellings, P.; Jung, K.; Merk, H.; Olze, H.; Schlenter, W.; Stock, P.; et al. Visual Analogue Scales (VAS) - Measuring Instruments for the Documentation of Symptoms and Therapy Monitoring in Case of Allergic Rhinitis in Everyday Health Care. Allergo Journal 2017, 26, 36–47. [CrossRef]
  46. Poole, J.L.; Whitney, S.L. Motor Assessment Scale for Stroke Patients: Concurrent Validity and Interrater Reliability. Arch Phys Med Rehabil 1988, 69, 195–197.
  47. Carr, J.H.; Shepherd, R.B.; Nordholm, L.; Lynne, D. Investigation of a New Motor Assessment Scale for Stroke Patients. Phys Ther 1985, 65, 175–180. [CrossRef]
  48. Van der Lee, J.H.; De Groot, V.; Beckerman, H.; Wagenaar, R.C.; Lankhorst, G.J.; Bouter, L.M. The Intra- and Interrater Reliability of the Action Research Arm Test: A Practical Test of Upper Extremity Function in Patients with Stroke. Arch Phys Med Rehabil 2001, 82, 14–19. [CrossRef]
  49. Lannin, N.A. Reliability, Validity and Factor Structure of the Upper Limb Subscale of the Motor Assessment Scale (UL-MAS) in Adults Following Stroke. 2009, 26, 109–116. [CrossRef]
  50. Vellone, E.; Savini, S.; Fida, R.; Dickson, V.V.; Melkus, G.D.E.; Carod-Artal, F.J.; Rocco, G.; Alvaro, R. Psychometric Evaluation of the Stroke Impact Scale 3.0. Journal of Cardiovascular Nursing 2015, 30, 229–241. [CrossRef]
  51. Uswatte, G.; Taub, E.; Morris, D.; Vignolo, M.; McCulloch, K. Reliability and Validity of the Upper-Extremity Motor Activity Log-14 for Measuring Real-World Arm Use. Stroke 2005, 36, 2493–2496. [CrossRef]
  52. Coderre, A.M.; Amr Abou Zeid; Dukelow, S.P.; Demmer, M.J.; Moore, K.D.; Demers, M.J.; Bretzke, H.; Herter, T.M.; Glasgow, J.I.; Norman, K.E.; et al. Assessment of Upper-Limb Sensorimotor Function of Subacute Stroke Patients Using Visually Guided Reaching. Neurorehabil Neural Repair 2010, 24, 528–541. [CrossRef]
  53. BKIN Technologies Ltd Kinarm Standard Tests. BKIN Technologies Ltd.: Kingston, ON, Canada 2018.
  54. Dukelow, S.P.; Herter, T.M.; Moore, K.D.; Demers, M.J.; Glasgow, J.I.; Bagg, S.D.; Norman, K.E.; Scott, S.H. Quantitative Assessment of Limb Position Sense Following Stroke. Neurorehabil Neural Repair 2010, 24, 178–187. [CrossRef]
  55. Otaka, E.; Otaka, Y.; Kasuga, S.; Nishimoto, A.; Yamazaki, K.; Kawakami, M.; Ushiba, J.; Liu, M. Clinical Usefulness and Validity of Robotic Measures of Reaching Movement in Hemiparetic Stroke Patients. J Neuroeng Rehabil 2015, 12, 1–10. [CrossRef]
  56. Saenen, L.; Xivry, J.-J.O. de; Verheyden, G. Development and Validation of a Novel Robot-Based Assessment of Upper Limb Sensory Processing in Chronic Stroke. Brain Sciences 2022, Vol. 12, Page 1005 2022, 12, 1005. [CrossRef]
  57. Lohse, K.R.; Lang, C.E.; Boyd, L.A. Is More Better? Using Metadata to Explore Dose-Response Relationships in Stroke Rehabilitation. Stroke 2014, 45, 2053–2058. [CrossRef]
  58. Scrivener, K.; Sherrington, C.; Schurr, K.; Treacy, D. Many Participants in Inpatient Rehabilitation Can Quantify Their Exercise Dosage Accurately: An Observational Study. J Physiother 2011, 57, 117–122. [CrossRef]
  59. Pohl, J.; Held, J.P.O.; Verheyden, G.; Alt Murphy, M.; Engelter, S.; Flöel, A.; Keller, T.; Kwakkel, G.; Nef, T.; Ward, N.; et al. Consensus-Based Core Set of Outcome Measures for Clinical Motor Rehabilitation After Stroke—A Delphi Study. Front Neurol 2020, 11, 1–9. [CrossRef]
  60. Miller, K.J.; Slade, A.L.; Pallant, J.F.; Galea, M.P. Evaluation of the Psychometric Properties of the Upper Limb Subscales of the Motor Assessment Scale Using a Rasch Analysis Model. J Rehabil Med 2010, 42, 315–322. [CrossRef]
  61. Sabari, J.S.; Ai, L.L.; Velozo, C.A.; Lehman, L.; Kieran, O.; Lai, J.S. Assessing Arm and Hand Function after Stroke: A Validity Test of the Hierarchical Scoring System Used in the Motor Assessment Scale for Stroke. Arch Phys Med Rehabil 2005, 86, 1609–1615. [CrossRef]
  62. Wei, Y.; Bajaj, P.; Scheidt, R.A.; Patton, J.L. Visual Error Augmentation for Enhancing Motor Learning and Visual Error Augmentation for Enhancing Motor Learning and Rehabilitative Relearning Rehabilitative Relearning Recommended Citation Recommended Citation “Visual Error Augmentation for Enhancing Motor Learning and Rehabilitative Relearning” (2005). Biomedical Engineering Faculty Research and Publications. 2005.
  63. Huang, F.C.; Patton, J.L.; Mussa-Ivaldi, F.A. Manual Skill Generalization Enhanced by Negative Viscosity. J Neurophysiol 2010, 104, 2008. [CrossRef]
  64. Zeiler, S.R.; Krakauer, J.W. The Interaction between Training and Plasticity in the Poststroke Brain. Curr Opin Neurol 2013, 26, 609–616. [CrossRef]
  65. Meeus, P.; Dalcq, V.; Beauport, D.; Declercq, K.; Swine, B. Neurology-Stroke-Diagnosis and Treatment Planning Appropriate Care Unit.
  66. Bagg, S.; Alicia, ; Pombo, P.; Hopman, W. Effect of Age on Functional Outcomes After Stroke Rehabilitation. 2002. [CrossRef]
  67. Mutai, H.; Furukawa, T.; Araki, K.; Misawa, K.; Hanihara, T. Factors Associated with Functional Recovery and Home Discharge in Stroke Patients Admitted to a Convalescent Rehabilitation Ward. Geriatr Gerontol Int 2012, 12, 215–222. [CrossRef]
Preprints 93281 g001
Figure 1. DeXtreme robot (BioXtreme Ltd., Israel).
Figure 1. DeXtreme robot (BioXtreme Ltd., Israel).
Preprints 93281 g001
Preprints 93281 g002
Figure 2. DeXtreme games using error-enhancement forces during 3D reaching movements: (a) Market stand game: the subject must follow the trajectory of a bee as quickly and accurately as possible. The bee moves from the starting point (white circle) to a random fruit box; (b) Alchemist game: the subject must fill a glass of water and move towards the coloured tap that lights up randomly, without spilling the water.
Figure 2. DeXtreme games using error-enhancement forces during 3D reaching movements: (a) Market stand game: the subject must follow the trajectory of a bee as quickly and accurately as possible. The bee moves from the starting point (white circle) to a random fruit box; (b) Alchemist game: the subject must fill a glass of water and move towards the coloured tap that lights up randomly, without spilling the water.
Preprints 93281 g002
Table 1. Demographic and stroke-related characteristics presented as mean and range (grey), or number in % (white).
Table 1. Demographic and stroke-related characteristics presented as mean and range (grey), or number in % (white).
Preprints 93281 i001
SD: standard deviation; M: man; F: Female.
Table 2. Clinical outcome measures pre- and post-intervention presented as median (IQ1-IQ3).
Table 2. Clinical outcome measures pre- and post-intervention presented as median (IQ1-IQ3).
Outcome parameter Median (IQ1-IQ3) Median difference (IQ1-IQ3) P-value
PRE POST
Fugle-Meyer Assessment – UEa 54.0 (50.0-57.8) 55.0 (51.3 – 59.5) 1.0 (0.8 – 3.0) <.001*
Visual Analogue Scale – tonea 15.5 (5.8 – 30.0) 6.0 (1.3 – 30.0) -5.0 (-13.0 – 0.2) 0.089
Visual Analogue Scale – paina 0.0 (0.0 – 4.5) 0 (0.0 – 1.8) 0.0 (-1.5 – 0.3) 0.178
Action Research Arm Testa 49.5 (39.3 – 53.0) 50.5 (41.3 – 55.0) 2.0 (0.8 – 2.0) <.001*
Motor Assessment Scale – tonea 4.0 (4.0 – 4.8) 4.0 (4.0 – 4.8) 0.0 (0.0 – 0.0) 1.000
Motor Assessment Scale – UEa 14.0 (12.0 – 15.5) 14.0 (11.3 – 16.3) 0.0 (0.0 – 0.0) 0.317
UE: Upper Extremity; IQ1-IQ3: interquartile range; a: Wilcoxon signed-rank test; * p < 0.05.
Table 3. Patient-reported outcome measures pre- and post-intervention presented as median (IQ1-IQ3).
Table 3. Patient-reported outcome measures pre- and post-intervention presented as median (IQ1-IQ3).
Outcome parameter Median (IQ1-IQ3) Median difference (IQ1-IQ3) P-value
PRE POST
Motor activity log – AOUa 1.9 (1.5 – 2.5) 2.1 (1.6 – 2.7) 0.1 (0.0 – 0.3) <.001*
Motor activity log – QOMa 1.9 (1.3 – 2.5) 2.1 (1.5 – 3.1) 0.1 (0.1 – 0.5) <.001*
Stroke Impact Scale – Handa 57.5 (32.5 – 75.0) 65.0 (35.0 – 75.0) 0.0 (0.0 – 6.3) 0.837
AOU: amount of use; QOM: quality of movement; IQ1-IQ3: interquartile range; a: Wilcoxon signed-rank test; * p < 0.05.
Table 4. Kinematic outcome measures pre- and post-intervention presented as mean (SD) or median (IQ1-IQ3).
Table 4. Kinematic outcome measures pre- and post-intervention presented as mean (SD) or median (IQ1-IQ3).
Preprints 93281 i002
SD: standard deviation, IQ1-IQ3: interquartile range, a: paired t-test; b: Wilcoxon signed-rank test; * p < 0.05.
Table 3. Number of reaching movements during 5 hours of error-enhancement training.
Table 3. Number of reaching movements during 5 hours of error-enhancement training.
Preprints 93281 i003
SD: standard deviation.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated