Preprint
Article
Co-Design/Co-Development and Evaluation of an Online Frailty Check Application for Older Adults: Participatory Action Research with Older Adults
Altmetrics
Downloads
184
Views
80
Comments
1
A peer-reviewed article of this preprint also exists.
This version is not peer-reviewed
Submitted:
15 May 2023
Posted:
16 May 2023
You are already at the latest version
Alerts
Abstract
Frailty, an age-related decline in homeostatic reserves, markedly proceeded during the coronavirus disease 2019 pandemic. To continuously assess frailty status, a remote system is urgently required. We aimed to co-design/co-develop an online frailty check (FC) application alongside FC supporters who were facilitators in a pre-existing onsite FC program. The online FC included a screening measurement for sarcopenia and an 11-item questionnaire covering dietary, physical, and social behaviors. Using prototype applications, 55 opinions obtained from 32 FC supporters (median 74.0 years) were categorized and reflected refinement. Regarding the self-efficacy of FC supporters, a significant increase in social positioning was seen after the mock test (P=.031). For FC supporters and participants, the average system usability scale (SUS) score was 70.2±10.3 points, which was “marginally high” for acceptability and “good” for the adjective range. Multiple regression analysis showed that the SUS score was significantly correlated with online-onsite reliability but not online communication, even after adjusting by age, sex, education level, and ICT proficiency (b=0.400, 95% CI: 0.243-1.951, P=.013). Additionally, a significant association between onsite and online FC scores was observed (R=0.670, P=.001). Our online FC application was evaluated to be a valuable tool to practically assess frailty status remotely.
Keywords:
Subject: Public Health and Healthcare - Public Health and Health Services
1. Introduction
Frailty is a complex age-related clinical condition characterized by a decline in physiological capacity across several organ systems [1,2]. In Japan, to help community-dwelling older adults monitor their own frailty status, we developed a frailty check (FC) program based on evidence from Kashiwa cohort study, which was a large-scale cohort study of older adults in Kashiwa City who were not certified as requiring long-term care [3]. This FC program consists of self-assessments for sarcopenia (Yubi-wakka test) [4], an 11-item self-reporting questionnaire, and physical measurements of skeletal muscle mass, calf circumference, and handgrip strength. Between April 2015 and February 2020, a total of 8,855 community-dwelling older adults in 47 local government prefectures participated in the FC program. Notably, this citizen-centered program is staffed by FC supporters, who are community-dwelling older volunteers. FC supporters encourage participants to improve their lifestyle and practice frailty prevention [5].
Recently, the coronavirus disease 2019 (COVID-19) pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) resulted in restrictions on going outdoors, which caused a decrease in physical activity among older adults. This decrease in physical activity can lead to sarcopenia and frailty [6,7,8,9]. Our recent study found, among Japanese community dwelling older adults, a significant decrease in the trunk muscle mass (one of the risk factors for postural instability and falls) immediately after the pandemic’s first wave (April−May 2020) [10]. Frailty caused by the COVID-19 pandemic became known as corona-frailty. Thus, to continuously monitor frailty status during the COVID-19 pandemic, a mobile application that can be used for remote check-ups is urgently required.
In this study, we aimed to co-design/co-develop and evaluate an online FC application based on a pre-existing onsite FC program with the help of FC supporters through participatory action research.
2. Materials and Methods
2.1. Older Adults: Frailty Check Supporters and Participants
In this study, 32 FC supporters were involved in the development and implementation of the online FC application. The FC supporters were community-dwelling older adults who had facilitated the onsite FC program in Bunkyo-ku and Nishitokyo-shi in Tokyo, Japan. Additionally, we recruited 20 community-dwelling older adult participants who had participated in the onsite FC program in Nishitokyo-shi, Tokyo, Japan.
2.2. Study Design
A flow diagram of the development and implementation process is shown in Figure 1. To co-design and refine the application, focus group interviews and mock tests were conducted with FC supporters. Self-efficacy questionnaires were assessed before and after the mock test. After online FC implementation, we conducted surveys on the reliability, communication, and usability of the user interface (Figure 1a).
2.2.1. Online Frailty Check Application: Device and Contents
The online FC application was conducted using a tablet device through video conferencing in a room containing a maximum of six people (Figure 1b). The application utilized the same system used by online cram schools, which enables communication between teachers and students and allows for document sharing by teachers. In the application, FC supporters acted as the teachers and projected questionnaires onto the right upper side wall of the online room. Participants would input data or answer questions using an electronic pencil while following the FC supporters’ instructions.
The program consisted of three sections: basic information (age, sex, body weight, height), Yubi-wakka test (sarcopenia test with video description [4]) and an 11-item FC questionnaire (yes/no, validated questionnaires including questions on nutrition, oral and physical function, and social activity) (Figure 1b). This structure was also used in the onsite FC program. Each section of the questionnaire is addressed in Table S1 and a video of the online FC implementation is also supplied as Video S1 (in Japanese).
2.2.2. Development and Refinement of Application; Participatory Action Research Method
To collect information from various sources, we used the participatory survey method to conduct the research. To facilitate smooth execution of the experiment and the collection of authentic and reliable data, the process was divided into two main parts. To collect primary data, we began by conducting online FC briefing sessions and conducted several user-experience-sharing meetings. The primary data was extracted from focus group interviews, discussions, and observations as outlined in Figure 1. User experience meetings for the prototype application were held over 2 days, and a total of 32 FC supporters participated. Subsequently, we made improvements based on the opinions and feedback received.
Data analysis was conducted using the following steps. First, we coded the interview data using first-order codes: (1) reliability, (2) social interaction, and (3) user-friendliness. Second, we added the frequency to indicate the importance of each code. After the coding and categorization, we reviewed and refined each opinion through discussion.
2.3. Questionnaires
For FC supporters, we assessed self-efficacy using a validated 16-item questionnaire on the general self-efficacy scale (GSES) consisting of three categories: positive behavior, non-anxiety about failure, and social positioning of ability [11]. For both the FC supporters and participants, we evaluated the user interface of the online FC application using relevant questionnaires. To assess the online-onsite reliability (3 items) and interaction quality (4 items), we utilized modified questionnaires on telehealth usability (Table S2) [12]. We also scored the system usability scale (SUS) [13] using 10 questionaries, with possible values ranging from 0 to 100, to examine the overall usability.
2.4. Analysis
The differences between FC supporters and participants were examined using Student’s t-test or the Mann−Whitney U-test after confirming normal distribution using the Shapiro–Wilk test. We examined the effect on self-efficacy before and after the mock test using a Wilcoxon rank-sum test, after confirmation of normal distribution by the Shapiro–Wilk test. Cronbach’s alpha was calculated to appraise the scale’s internal consistency. To analyze the relationship between the SUS and related independent variables, as well as between online and onsite FC results, we applied Pearson’s or Spearman’s correlation analysis. Multiple regression analysis was performed using SUS as the dependent variable and age, sex, education, ICT proficiency, reliability, and interaction quality as independent variables. Data were analyzed using IBM SPSS Statistics version 26 (IBM Japan, Tokyo, Japan). Statistical significance was set at P<.05.
3. Results
3.1. Frailty Check Supporters and Participants
The basic characteristics of the FC supporters and participants are shown in Table 1. Of the 12 FC supporters, seven were men. Compared to FC supporters, the participants were significantly older and had lower education levels, while subjective health and well-being did not differ between the two groups. The subjective proficiency of ICT tended to be low for the participants (P=.082). Among participants, two major reasons were identified for their participation in the online FC: (1) interest in the online FC program (40%) and (2) the need for continuous frailty checks (40%) owing to cessation of the onsite FC program during the COVID-19 pandemic (data not shown).
3.2. Development and Refinement of the Application
Opinions collected from the focus group interviews and the refinement process are shown in Table 2. Fifty-five detailed opinions were obtained and classified into five categories based on key words such as reliability, social interaction, and user-friendliness. The largest number of opinions (n=18) were related to reliability. Refinement was performed based on the categorized opinions. For example, to improve familiarity with the online questionnaire format, the onsite format of the 11-item self-reported questionnaire was adapted and modified to display one question at a time on a single page. In addition, a confirmation page was added so each participant could reconfirm all the responses. Furthermore, FC supporters expressed difficulties in online communication, such as identifying the speaker and capturing the other side’s reaction. To enhance and encourage communication, we implemented several measures, including spotlighting the person speaking, providing an explanation of the system, and removed the daily topic section before the check.
FC supporters expressed difficulty in conducting the Yubi-wakka test over the screen. To address this challenge, we created a video explaining the test together with the supporters and embedded it into the application to provide an accurate and consistent explanation similar to the onsite FC.
Furthermore, several opinions on user-friendliness were collected and improvements based on this feedback were implemented: 1) using a stylus, 2) enlargement of font size, and 3) introducing a button pressing simulator.
3.3. Self-efficacy of Frailty Check Supporters and Usability Evaluation
In the co-design and co-development of the online FC application, we investigated whether the engagement of FC supporters in these processes affected their self-efficacy. As shown in Table 3, a slight but not significant increase in the total GSES score was observed after the mock test (before: 12.0 vs after: 13.0). However, among the three GSES categories, we noticed a trend of improvement in the non-anxiety of failure category, as the proportion who answered “no” regarding (Q5) “I am more concerned about small failures than others all the time”, tended to increase after the mock test (before: 63.0% vs. after: 77.8%, P=.094). Additionally, in the social positioning of ability category, the proportion of “yes” responses to (Q3) “there are areas where I have better knowledge than friends,” significantly increased (P=.031) in comparison to results from before the mock test. Similarly, the proportion of “yes” responses to (Q1) “I have better ability than friends,” also increased (P=.063).
The overall system usability of the online FC application was evaluated using SUS, a widely used questionnaire with an acceptability and adjective range scale (Figure 2). We found that the average SUS scores both of FC supporters and participants were similar and in the “marginally high” section of the acceptability range. Additionally, the scores were above the average (68 points [15]) of the adjective ranges, which implies that the developed FC application is suitable for older adults.
Next, we examined the overall SUS score and the factors that affect the user interface. As shown in Table 4, we found that age, education level, subjective ICT proficiency, reliability, and interaction quality were significantly associated with SUS (P<.05). Furthermore, when we assessed the multiple (linear) regression analyses, we observed a significant correlation between the SUS score and the reliability of the online FC application after adjusting for covariates including age, sex, education duration, and ICT proficiency (Table 5). On the other hand, the interaction quality did not display a significant association with the SUS score.
We further validated the results of the online FC program by comparing them against those of the onsite FC, which were conducted within 4 months of each other. Pearson’s correlation coefficient revealed a significant association between the onsite and online results (P=.001) (Figure 3).
4. Discussion
In this study, we developed an online FC application for community-dwelling older adults delivered by video conferencing, during the COVID-19 pandemic. To enhance its affinity for older adults, we co-designed and co-developed it alongside FC supporters, who are older community-resident volunteers from a pre-existing onsite FC program. Our focus was on improving onsite-online reliability and online communication, two major determinants of the user interface.
To the best of our knowledge, this is the first online application for frailty checks designed by older adults, for older adults. To achieve our goal, we utilized a participatory action research design [15,16,17], consisting of focus group interviews and a mock test. FC supporters (users who acted as session facilitators) from two research fields were involved in the development of the online FC application, which achieved high scores on the SUS for both participants and FC supporters. Based on this, it is conceivable that a participatory design might be a useful methodology when developing digital healthcare interventions for older adults.
Video conferencing has been proposed as a medium for digital health intervention that can be used for communication, providing care and support, and enhancing health status, such as home-based tele-exercise for patients with chronic diseases [18,19]. In this study, we also explored the use of video conferencing through tablets and found it to be a potentially useful online tool for older adults, as they can communicate and collaborate to solve problems together during the frailty check. However, the efficiency and cost-effectiveness of video-conferencing technology should be examined in future research.
Intriguingly, we found that the FC supporters were able to strengthen their own social positioning of ability and relieve anxiety about failure through participating in the development of the application. This highlights the beneficial effects of co-design and co-development on self-efficacy for FC supporters. Furthermore, the empowerment of the FC supporters equips them with the confidence to operate the online FC platform, which subsequently could enhance the usability for all participants.
The results of this study showed that online-onsite reliability was significantly associated with SUS scores, even after adjustment for several variables including sex, age, educational level, and subjective ICT proficiency. This finding suggests that the similarity between the contents and operation of the online FC program and the pre-existing onsite FC program contributes to the safety, satisfaction, and usability for older participants. However, interaction quality was not significantly associated with SUS scores, indicating that several issues need to be addressed in online communication among older adults. For example, audio-video quality should be improved to help prevent frailty, particularly in terms of social isolation and loneliness. Moreover, enhancing online communication should be done with an understanding of the intention to use medical applications among older adults [20].
Finally, we validated the results of online FC assessment by comparing them with those of onsite FC. The significant similarity observed between the two could be attributed to the reliability and simplicity of the application (11 yes/no questions). With this observation, it is feasible that the online FC application could be used as a complementary tool for onsite FC among older participants, especially in emergency situations like the COVID-19 pandemic, or for sustained monitoring of frailty status. Although older adults cannot accurately assess their frailty status using video conferencing alone, this online FC application, which was implemented by older adults themselves, is a significant step toward the necessary training and system changes needed for adequate assessment of frailty status.
4.1. Limitations
Our study has several limitations. The primary focus was on developing the application and conducting a preliminary evaluation of its feasibility and acceptability. The evaluation study was not designed to rigorously test the efficacy of the online FC assessment in improving frailty status. Additionally, the use of non-systematic recruitment methods may have introduced bias toward participants who were more open to this type of intervention.
Despite these limitations, our study has several important implications: (1) individual FC supporters and participants may benefit from using the online FC application; (2) a participatory design approach was a useful methodology for developing a relevant, useful, and accessible tool for older adult users.
5. Conclusions
We co-designed and co-developed an online FC application using a participatory action research approach to create a reliable tool for older adults to assess their frailty. Our study found that online-onsite reliability is an important determinant affecting the usability and satisfaction of the online FC application. Furthermore, it is conceivable that enhanced empowerment of FS supporters may lead to high usability scores in participants, a win-win situation. Further investigations are needed to rigorously test the efficacy of the developed online FC application with larger sample sizes.
Supplementary Materials
The following supporting information can be downloaded at the website of this paper posted on Preprints.org, Table S1: Items of an 11-item questionnaire; Table S2: Reliability and interaction quality of online frailty check application; Video S1: A video of the online FC implementation (in Japanese).
Author Contributions
Conceptualization, BK.S.; Data curation, T.M., K.Y., Y.S., D.K., W.L., Y.Y., and M.T.; Formal analysis, BK.S.; Funding acquisition, K.I.; Investigation, BK.S., T.M., K.Y., Y.S., Y.Y., and T.T.; Project administration, BK.S., K.I.; Validation, BK.S., D.K., W.L., Y.Y., and M.T.; Writing-original draft, BK.S. All authors have read and agreed to the published version of the manuscript.
Funding
This study was supported by AMED (20dk0110035h0002), JST MIRAI (JPMJMI21J1), JST Moonshot (JPMJMS2237-X) , and Mitsui Sumitomo Insurance Welfare Foundation.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the University of Tokyo Life Science Ethics Committee (Approval Number: 21-190).
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data in this study is restricted because the ethical approval was not sought for public data sharing from the Ethics Committee at the University of Tokyo and the participants were not informed of possible public data sharing when they provided informed consent. However, data can be made available from a non-author of contact at Institute of Gerontology, the University of Tokyo (contact via info.frail@iog.u-tokyo.ac.jp) for researchers who meet the criteria for access to confidential data.
Acknowledgments
We are sincerely grateful to the participants and staff of this study. The authors would like to especially thank Y. Sato., N. Sekine., K. Imanaka., T. Iguchi.
Conflicts of Interest
The authors declare no conflict of interest. The authors declare no conflict of interest. The funder had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.
References
- Walston, J.; Hadley, E.C.; Ferrucci, L.; Guralnik, J.M.; Newman, A.B.; Studenski, S.A.; Ershler, W.B.; Harris, T.; Fried, L.P. Research agenda for frailty in older adults: toward a better understanding of physiology and etiology: summary from the American Geriatrics Society/National Institute on Aging Research Conference on Frailty in Older Adults. J Am Geriatr Soc. 2006, 54, 991–1001. [Google Scholar] [CrossRef] [PubMed]
- Fried, L.P.; Tangen, C.M.; Walston, J.; Newman, A.B.; Hirsch, C.; Gottdiener, J.; Seeman, T.; Tracy, R.; Kop, W.J.; Burke, G.; et al. Frailty in older adults: evidence for a phenotype. J Gerontol A Biol Sci Med Sci. 2001, 56, M146–M156. [Google Scholar] [CrossRef] [PubMed]
- Ishii, S.; Tanaka, T.; Shibasaki, K.; Ouchi, Y.; Kikutani, T.; Higashiguchi, T.; Obuchi, S.P.; Ishikawa-Takata, K.; Hirano, H.; Kawai, H.; et al. Development of a simple screening test for sarcopenia in older adults. Geriatr Gerontol Int. 2014, 14, 93–101. [Google Scholar] [CrossRef] [PubMed]
- Tanaka, T.; Takahashi, K.; Akishita, M.; Tsuji, T.; Iijima, K. Yubi-wakka (finger-ring) test: a practical self-screening method for sarcopenia, and a predictor of disability and mortality among Japanese community-dwelling older adults. Geriatr Gerontol Int. 2018, 18, 224–232. [Google Scholar] [CrossRef] [PubMed]
- Johnson, H.H.; Glascoff, M.A.; Lovelace, K.; Bibeau, D.L.; Tyler, E.T. Assessment of public health educator practice: health educator responsibilities. Health Promot Pract. 2005, 6, 89–96. [Google Scholar] [CrossRef] [PubMed]
- Morley, J.E.; Kalantar-Zadeh, K.; Anker, S.D. COVID-19: a major cause of cachexia and sarcopenia? J Cachexia Sarcopenia Muscle. 2020, 11, 863–865. [Google Scholar] [CrossRef] [PubMed]
- Kumar, N.B. Does COVID-19-related cachexia mimic cancer-related cachexia? Examining mechanisms, clinical biomarkers, and potential targets for clinical management. J Cachexia Sarcopenia Muscle. 2021, 12, 519–522. [Google Scholar] [CrossRef] [PubMed]
- Yamada, M.; Kimura, Y.; Ishiyama, D.; Otobe, Y.; Suzuki, M.; Koyama, S.; Kikuchi, T.; Kusumi, H.; Arai, H. The influence of the COVID-19 pandemic on physical activity and new incidence of frailty among initially non-frail older adults in Japan: a follow-up online survey. J Nutr Health Aging. 2021, 25, 751–756. [Google Scholar] [CrossRef]
- Shimokihara, S.; Maruta, M.; Hidaka, Y.; Akasaki, Y.; Tokuda, K.; Han, G.; Ikeda, Y.; Tabira, T. Relationship of decrease in frequency of socialization to daily life, social life, and physical function in community-dwelling adults aged 60 and over after the COVID-19 pandemic. Int J Environ Res Public Health. 2021, 18, 2573. [Google Scholar] [CrossRef]
- Son, B.K.; Imoto, T.; Inoue, T.; Nishimura, T.; Tanaka, T.; Iijima, K. Social detachment influenced muscle mass and strength during the COVID-19 pandemic in Japanese community-dwelling older women. J Frailty Aging. 2022, 11, 231–235. [Google Scholar] [CrossRef] [PubMed]
- Sakano, Y.; Tohjoh, M. The general self-efficacy scale (GSES): scale development and validation. Jpn J Behav Ther. 1986, 12, 173–182. [Google Scholar]
- Parmanto, B.; Lewis, A.N. Jr.; Graham, K.M.; Bertolet, M.H. Development of the telehealth usability questionnaire (TUQ). Int J Telerehabil. 2016, 8, 3–10. [Google Scholar] [CrossRef] [PubMed]
- Brooke, J. SUS: a ‘quick and dirty’ usability scale. Usability Eval Ind. 1996, 189, 4–7. [Google Scholar]
- Bangor, A.; Kortum, P.T.; Miller, J.T. An empirical evaluation of the system usability scale. Int J Hum-Comput Int. 2008, 24, 574–594. [Google Scholar] [CrossRef]
- Baum, F.; MacDougall, C.; Smith, D. Participatory action research. J Epidemiol Community Health. 2006, 60, 854–857. [Google Scholar] [CrossRef] [PubMed]
- Hand, C.; Rudman, D.L.; McGrath, C.; Donnelly, C.; Sands, M. Initiating participatory action research with older adults: lessons learned through reflexivity. Can J Aging. 2019, 38, 512–520. [Google Scholar] [CrossRef] [PubMed]
- Seino, S.; Tomine, Y.; Nishi, M.; Hata, T.; Fujiwara, Y.; Shinkai, S.; Kitamura, A. Effectiveness of a community-wide intervention for population-level frailty and functional health in older adults: a 2-year cluster nonrandomized controlled trial. Prev Med. 2021, 149, 106620. [Google Scholar] [CrossRef] [PubMed]
- Jiménez-Rodríguez, D.; Santillán García, A.; Montoro Robles, J.; Rodríguez Salvador, M.D.M.; Muñoz Ronda, F.J.; Arrogante, O. Increase in video consultations during the COVID-19 pandemic: healthcare professionals’ perceptions about their implementation and adequate management. Int J Environ Res Public Health. 2020, 17, 5112. [Google Scholar] [CrossRef] [PubMed]
- Pywell, J.; Vijaykumar, S.; Dodd, A.; Coventry, L. Barriers to older adults’ uptake of mobile-based mental health interventions. Digit Health. 2020, 6, 2055207620905422. [Google Scholar] [CrossRef] [PubMed]
- Askari, M.; Klaver, N.S.; van Gestel, T.J.; van de Klundert, J. Intention to use medical apps among older adults in the Netherlands: cross-sectional study. J Med Internet Res. 2020, 22, e18080. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Development and implementation of an online frailty check application. Co-design and co-development of the application were performed with frailty check (FC) supporters, followed by online FC implementation and surveys (a). The contents of the online FC and the role of the FC supporters as facilitators (b).
Figure 1.
Development and implementation of an online frailty check application. Co-design and co-development of the application were performed with frailty check (FC) supporters, followed by online FC implementation and surveys (a). The contents of the online FC and the role of the FC supporters as facilitators (b).
Figure 2.
System usability scale (SUS) scores of the online frailty check application (a and b). Error bars represent the standard errors of each SUS score. The red dotted line represents a rough standard of the adjective ratings (68 points). n=43 (FC supporter n=23, participants n=20).
Figure 2.
System usability scale (SUS) scores of the online frailty check application (a and b). Error bars represent the standard errors of each SUS score. The red dotted line represents a rough standard of the adjective ratings (68 points). n=43 (FC supporter n=23, participants n=20).
Figure 3.
Association between the onsite and online frailty check score of the participants (n=20). Pearson’s correlation analysis (0.670, p=.001).
Figure 3.
Association between the onsite and online frailty check score of the participants (n=20). Pearson’s correlation analysis (0.670, p=.001).
Table 1.
Basic characteristics of the frailty check supporters and participants.
| Frailty check supporters |
Participants | p | |
| n | 32 | 20 | |
| Men/Women | 12/20 | 7/13 | 0.855 |
| Age (y) | 74.0 (67-86) | 82.0 (70-91) | 0.001 |
| Education level (longer than 14 years, %) |
23 (71.9%) | 7 (35.0%) | 0.009 |
| Resident period (years) | 40.5±16.0 | 35.3±17.2 | 0.277 |
| Living alone (%) | 7 (21.9%) | 6 (30.0%) | 0.752 |
| Subjective health (healthy status %) |
32 (100.0%) | 19 (95.0%) | 0.202 |
| Subjective well-being (10 points) |
8.0 (3.0-10.0) |
8.0 (5.0-10.0) |
0.854 |
| Subjective proficiency of ICT (10 points) |
5.5 (1.0-9.0) |
5.0 (0.0-10.0) |
0.082 |
*Values are presented as median (min-max), non-normal distribution. *Values are presented as mean±standard deviation, normal distribution. *T-test or Mann-Whitney test for continuous value with normal distribution or non-normal distribution.
Table 2.
Opinions and refinements for the online frailty check application by frailty check supporters
Table 2.
Opinions and refinements for the online frailty check application by frailty check supporters
| Category | Opinions (No. of opinions) | Example for opinions | Refinement |
|---|---|---|---|
| Reliability | Item entry (1) | “It is difficult for participants to enter the numbers unless FC supporters ask them verbally.” | Preparation of information to be entered before the test |
| Progress status (3) | “It would be better to have a display showing that other participants are still answering.” | Addition of a page to check the progress | |
| Final confirmation page (3) | “It would be better to be able to recognize which questions have not been answered at the end.” | Addition of confirmation page | |
| Homogeneity with onsite frailty check (3) |
“I want the answer page to be displayed similarly the onsite 11 self-reporting FC questionnaires page.” | Change to the similar design as the onsite questionnaire | |
| Change of display name (1) | “It would be better to be able to change the display name on the app.” | Indication of the participants' affiliation in their native language for easy understanding | |
| Need for training & practice (4) | “I think it would be preferable for supporters to work as a pair so that they could help each other.” | Conduction of self-directed learning activities by the supporters | |
| Encouragement (1) | “I thought I should support everyone by saying, ‘I couldn't do it either, but now I can.’” | Addition to cheering sound effect function | |
| Provision of frailty check results (2) | “In the onsite frailty check, participants can take home a paper of the results, but what about that during online frailty check?” “Can each participant receive their own data?” | Addition of a printing function and distribution of printed result to users | |
| Social interaction | Identification of speaker (3) | “I want a function/signal designed that allows me to recognize who the speaker is.” | Manualization of communication function |
| Tablet camera setting (2) | “My finger hits the camera when I hold the tablet.” | Manualization of setting and facilitating by the supporter | |
| Online communication (3) | “When we're online, we can't properly capture the other side's reaction, so we talk less often.” | Building intimacy through daily conversation before the measurement | |
| User-friendliness | Button (4) | “Buttons are small and difficult to touch with fingertips. I think it would be easier to do with a stylus.” | Use of a stylus |
| Touch screen sensitivity (2) | “Buttons do not respond when my fingers are dry.” | Use of a stylus | |
| Touch screen skills (4) | “I don't know how to press the button.” | Instruction provision, Addition of button pressing simulator | |
| Font (3) | “A larger font is better.” | Use of a larger font size | |
| Initialization function (1) | “It is better to design the question form so that the users can go back to the previous question.” | Manualization of operation method and clarification of troubleshooting | |
| Tutorial function (1) | “It would be good to have a button-pressing practice function. Elderly people can't learn it in one session.” | Addition of button pressing simulator | |
| Visual design (2) | “I want the design to be relevant to frailty check and the elderly.” | Improvement of layout | |
| Social interaction & Reliability |
Timing to watch the Yubi-wakka tutorial video (2) | “Some participants think the video starts automatically, so they need to be verbally informed.” | Addition of step-by-step tutorial video |
| Calling attention (1) | “To prevent accidents, appropriate explanations such as ‘Please sit in a chair during the examination’ are needed.” | Emphasizing in the step-by-step tutorial video, with verbal reminders by the supporter | |
| Frailty check procedures (1) | “I think it would be easier for participants to 11 self-reporting FC questionnaires, if the supporters read the questions one by one.” | Checking the progress by calling participant's name | |
| Announcement for ending (1) | “It would be better to all say goodbye and then guide participants to press the exit button.” | Instruction manual and verbal reminder | |
| Reliability & User-friendly |
Answer format (1) | “It would be better for all participants to answer each question simultaneously.” | Converting one page forms to one question per page form |
| Backward function (1) | “On all pages, I want a button to go back to the previous page.” | Addition of backward function | |
| Display of the test tutorial video (5) | “Please consider making an Yubi-wakka test explanation video in advance and showing it during the check.” | Verbal guidance on how to play the tutorial video |
Table 3.
Self-efficacy of the frailty check supporters before and after the mock test.
| General Self-Efficacy Scale | Before | After | p | |
|---|---|---|---|---|
| Self-efficacy (total score, points/ 16 points) | 12.0 (5.0-16.0) |
13.0 (6.0-16.0) |
0.498 | |
| Positive behavior ( points/ 7 points) | 6.0 (1.0-7.0) |
6.0 (0.0-7.0) |
0.952 | |
| Q1 | I am confident when I do something. (yes, %) | 77.8 | 77.8 | 0.375 |
| Q2 | I am worried compared to people. (no, %) | 55.6 | 51.9 | 0.312 |
| Q3 | I decide without hesitation when I decide something. (yes, %) | 63.0 | 55.6 | 0.250 |
| Q4 | I think I am a shy person. (no, %) | 70.4 | 70.4 | 0.375 |
| Q5 | I think it's better to work proactively even in jobs where the results are uncertain. (yes, %). | 85.2 | 88.9 | 0.500 |
| Q6 | I am a person who is willing to do anything. (yes, %) | 66.7 | 77.8 | 0.125 |
| Q7 | I am a person who are not good at actively working. (no, %) | 85.2 | 85.2 | 0.500 |
| Non-anxiety about failure ( points/ 5 points) | 4.0 (2.0-5.0) |
5.0 (1.0-5.0) |
0.582 | |
| Q1 | I often feel dark remembering the mistakes and unpleasant experiences I made in the past. (no, %) |
81.5 | 85.2 | 0.375 |
| Q2 | I often feel that I have failed, after finishing work. (no, %) | 92.6 | 92.6 | 0.500 |
| Q3 | I'm often worried that it won't work when to do something. (no, %) | 77.8 | 74.1 | 0.375 |
| Q4 | I often can't get to work since I couldn't decide what to do. (no, %) | 92.6 | 88.9 | 0.312 |
| Q5 | I am concerned all the time for small failure than others（no, %） | 63.0 | 77.8 | 0.094 |
| Social positioning of ability ( points/ 4 points) | 2.0 (1.0-4.0) |
3.0 (0.0-4.0) |
0.593 | |
| Q1 | I have better ability than friends（yes, %） | 40.7 | 55.6 | 0.063 |
| Q2 | I have better memory than humans. (yes, %) | 44.4 | 51.9 | 0.234 |
| Q3 | there are areas where I have a particularly good knowledge than friends（yes, %） | 55.6 | 74.1 | 0.031* |
| Q4 | I think I have the power to contribute to the world. (yes, %) | 85.2 | 88.9 | 0.375 |
N=27. +Values are presented as median (min-max). ‡Comparison of values before and after the mock test using the Wilcoxon rank-sum test.
Table 4.
Association between the system usability scale (SUS) score and factors affecting user interface.
Table 4.
Association between the system usability scale (SUS) score and factors affecting user interface.
| SUS score | p | |
|---|---|---|
| Age | -0.319* | 0.037 |
| Sex | -0.153 | 0.328 |
| Education | 0.376* | 0.013 |
| ICT proficiency | 0.376* | 0.013 |
| Reliability | 0.312* | 0.042 |
| Interaction | 0.309* | 0.044 |
N=43, *p<0.05, the Spearman correlation analysis.
Table 5.
Multiple regression analysis of the system usability scale
| Model 1 | Model 2 | Model 3 | |||||||
| b | 95% CI | p | b | 95% CI | p | b | 95% CI | p | |
| Reliability | 0.228 | -0.343-1.591 | 0.200 | 0.326* | 0.004-1.782 | 0.049 | 0.400* | 0.243-1.951 | 0.013 |
| Interaction | 0.178 | -0.408-1.244 | 0.313 | 0.196 | -0.289-1.206 | 0.222 | 0.139 | -0.389-1.038 | 0.362 |
Model 1: Unadjusted, Model 2: Adjusted for age, sex, Model 3: Adjusted for age, sex, education level, ICT proficiency, R=0.658, R2=0.433, Durbin-Watson=2.076.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated