Preprint
Article

Investigating the Effects of Gamification Elements on Student Motivation, Engagement, and Academic Performance in Higher Education Courses

Altmetrics

Downloads

145

Views

76

Comments

0

Submitted:

06 May 2024

Posted:

08 May 2024

You are already at the latest version

Alerts
Abstract
This study investigated how different gamification elements in mobile apps influence users' basic psychological needs (autonomy, competence, relatedness) and subsequent engagement outcomes. Survey data was analyzed using PLS-SEM. Social gamification elements enhanced autonomy and competence needs, while immersive elements boosted autonomy and relatedness. Achievement/progression components increased competence and relatedness perceptions. Interestingly, personal traits did not significantly impact psychological needs. Crucially, satisfying autonomy and relatedness predicted higher user satisfaction and continued usage intentions, highlighting their importance for sustained engagement. Competence influenced satisfaction but not continuance intention. The findings demonstrate the motivational roles of distinct gamification features in fostering need-supportive experiences. Designers should incorporate varied social, immersive, and achievement components to address users' psychological requirements holistically. Prioritizing autonomy and relatedness support appears vital for achieving enjoyable, self-motivated, and persistent user engagement across gamified systems and domains. The study provides actionable insights for optimizing motivational affordances through psychologically-informed gamification implementations.
Keywords: 
Subject: Social Sciences  -   Education

1. Introduction

Innovative Gamification (IG) refers to incorporating game-like elements and mechanics into non-game contexts to engage audiences, solve problems, and promote desired behaviors or outcomes. It has been defined in various ways, such as using game thinking and mechanics (Zichermann, 2010; Hellín et al., 2023), applying game design elements in non-game contexts (Deterding et al., 2011), and deriving fun and engaging elements from games for productive activities (Chou, 2015). In educational settings, IG is described as employing game-based mechanics, aesthetics, and game thinking to engage people, motivate action, and facilitate learning (Knutas et al., 2019; Kapp, 2012). Chou (2015) distinguishes between explicit and implicit IG. Explicit IG incorporates overtly game-like applications (e.g., McDonald's Monopoly Game, Foldit), while implicit IG subtly integrates game components and strategies into the user experience (e.g., LinkedIn's progress bar). IG approaches have been utilized across various domains, such as education, health/fitness, crowdsourcing, software development, business, marketing, and entertainment (Koivisto & Hamari, 2019; Kashive & Phanshikar, 2023). Examples include Khan Academy's points and badges for learning activities, Domino's Pizza's game app for building pizzas (leading to a 30% sales increase), Nike's gamified feedback for motivating fitness goals, Samsung's gamified system for increasing consumer engagement, and Foldit players solving a long-standing AIDS protein challenge in just 10 days. These instances demonstrate the effective use of IG strategies across different contexts (Sun et al., 2018; Chou, 2015; McGonigal, 2011).
Innovative Gamification (IG) techniques aim to promote psychological outcomes such as enhanced motivation and engagement in users through their voluntary interaction with the system and its affordances, ultimately shaping their behaviors (Koivisto & Hamari, 2019). Despite mobile applications becoming an integral part of daily life, empirical research on how IG strategies influence user engagement with mobile apps remains limited, with a few exceptions (Cechetti et al., 2019; Featherstone & Habgood, 2019; Kamboj et al., 2020). Recent studies have called for a better understanding of the antecedents and implications of interaction with mobile applications (Fang et al., 2017; Ho & Chung, 2020).
The existing literature has limitations. Many studies treat gamification merely as a research context (Kamboj et al., 2020) without establishing connections to established theories that could explain the motivational processes driven by individual elements of IG techniques (Suh et al., 2018). Furthermore, most studies examine a narrow set of game elements (e.g., competition and leaderboards in Featherstone & Habgood, 2019; scoring systems, progress bars, levels, leaderboards, and feedback in Cechetti et al., 2019) and do not measure users' interactions with individual game elements, as advocated in the literature (Xi & Hamari, 2020). Finally, most research models utilize user engagement as the dependent variable.
There is a need to gain deeper insights into the processes through which innovative gamification (IG) strategies can enhance user engagement in the context of mobile apps, and how such engagement can lead to positive outcomes. Addressing these gaps, the current study draws upon the self-system model of motivational development (SSMMD; Connell & Wellborn, 1991) to explore how new gamification methods may drive user involvement with mobile apps and yield positive marketing results. Specifically, the paper proposes a model to examine how three categories of game elements incorporated in mobile gamified apps (achievement and progression-oriented, social-oriented, and immersion-oriented) contribute to fulfilling individuals' psychological needs for competence, autonomy, and relatedness, and subsequently, how these psychological needs affect user engagement with mobile applications. Additionally, it examines the effect of user engagement on individuals' intentions for continued use, word-of-mouth (WOM) intentions, and app ratings.
This study contributes to both theory and practice in several ways. Firstly, it offers important insights into the user engagement literature, addressing a gap where research on interaction with mobile applications and its outcomes is still nascent. While previous studies have primarily focused on identifying specific app aspects or features that promote user engagement, few have studied the effect of gamification. Therefore, this study advances previous research by studying gamified mobile apps and exploring how different types of game features could enhance user engagement. Secondly, earlier research has often relied on performance metrics to assess user engagement, but these measures do not fully explain user behavior. This study addresses this gap by delving into the role of distinct game elements and shedding light on their effects. Finally, this study addresses methodological shortcomings found in past studies, such as small sample sizes and invalidated measures, by applying rigorous empirical methods.
The focus of this study is on user engagement, defined as "a quality of user experience characterized by the depth of an actor's cognitive, temporal, affective, and behavioral investment when interacting with a digital system" (O'Brien et al., 2018). It aims to examine the role of innovative gamification (IG) strategies in encouraging students to engage in the learning process and identify the key factors that contribute to the success of these strategies in improving students' desire to learn. By developing and quantitatively validating a research model to explain the factors affecting students' desire to learn using new gamification tactics, this study makes a significant contribution to the existing body of knowledge. The latter part of the paper presents relevant literature, followed by the development of hypotheses to address the research question, and subsequently, the analysis and discussion of the results. Finally, practical implications and concluding remarks are provided.

2.0. Theoretical Background and Literature Review2.1. Theoretical Background-The Self-System Model of Motivational Development (SSMMD)

The self-system model of motivational development (SSMMD) (Connell, 1990; Connell & Wellborn, 1991; Skinner et al., 2008) is a theoretical framework that originated from self-determination theory (SDT; Deci, 1975; Deci & Ryan, 1985). It explains how social contextual factors impact individuals' self-system processes, consequently either promoting or undermining their engagement. The SSMMD posits that when individuals experience satisfaction with their autonomy, competence, and relatedness needs, they are more likely to exhibit intrinsic motivation, which is crucial for sustained engagement and well-being. In line with this theory, our study reveals that satisfying users' needs for autonomy and relatedness predicted higher user satisfaction and continued usage intentions, underscoring the importance of these needs for fostering enjoyable, self-motivated, and persistent user engagement. According to the SSMMD, individuals possess three fundamental psychological needs: competence, autonomy, and relatedness (also central to SDT). Competence refers to one's belief in one's ability to perform an activity successfully and achieve a specific goal (White, 1959). Autonomy involves the capacity for behavioral choice (Connell, 1990; de Charms, 1968), while relatedness entails the sense of connection with others (Baumeister & Leary, 1995). Self-system processes revolve around these psychological needs (Connell, 1990; Connell & Wellborn, 1991), with engagement occurring when these needs are met, and disenfranchisement when they are not fulfilled (Connell & Wellborn, 1991). By incorporating varied social, immersive, and achievement components, designers can create a need-supportive environment that holistically addresses users' psychological requirements, as suggested by the SSMMD. This approach can enhance users' intrinsic motivation and lead to sustained engagement across gamified systems and domains.

2.2. Hypothesis Development

2.2.1. Motivation Stimuli (Social Element, Personal Traits, Achievement/ Success Metrics, Immersion Elements) and Psychological Outcomes (Competence, Autonomy, Relatedness)

Drawing from the SSMMD and following Koivisto and Hamari's (2019) conceptualization of gamification, the research model posits that motivational features within gamified systems (e.g., achievement and progression-oriented elements, personal and social-oriented elements, and immersion-oriented elements) lead to psychological outcomes like competence, autonomy, and relatedness satisfaction, as well as user engagement and other behavioral outcomes. For instance, achievement and progression-oriented features, such as badges, points, and leaderboards, evoke feelings of competence among users (Hassan et al., 2020; Peng et al., 2012; Sailer et al., 2017; van Roy & Zaman, 2019; Wee & Choong, 2019; Xi & Hamari, 2019). Motivational cues like social components and personal attributes impact students' desires for competence, autonomy, and relatedness (Khosa & Burch, 2023; Collie, R. J, 2020). The flipped classroom method satisfies students' need for relatedness by enhancing interaction opportunities, and fostering competency (Wang et al., 2019). Providing support for autonomy, competence, and relatedness in an online learning environment improves cognitive outcomes and reduces motivation (Wang et al., 2019; Tian et al., 2022). Autonomous motivation positively influences perceptions of autonomy, competence, and relatedness, impacting social and emotional development. Students' psychological needs relate positively to autonomous motivation and negatively to controlled motivation. Teachers' encouragement of student autonomy and perceptions of academic ability significantly affect reading success. Additionally, when users create characters or avatars in gamified systems, generating feelings of social relatedness (Sailer et al., 2017).
Based on the arguments above, we propose:
H1 
: The user’s interaction with social-oriented elements (SE) in the app helps to satisfy his/her needs for (a) competence (CPT), (b) autonomy (AoK), and (c) relatedness (RTD)
H2 
: The Personal traits (PT) elements in the app help to satisfy his/her needs for (a) competence (CPT), (b) autonomy (AoK), and (c) relatedness (RTD)
H3 
: The user’s interaction with achievement and progression-oriented elements (ASM) in the app helps to satisfy his/her needs for (a) competence (CPT), (b) autonomy (AoK), and (c) relatedness (RTD)
H4 
: The user’s interaction with immersion-oriented elements (IE) in the app helps to satisfy his/her needs for (a) competence (CPT), (b) autonomy (AoK), and (c) relatedness (RTD)

2.2.2. Gamification Elements, Psychological Outcomes, and Long-Term Engagement

Competence, autonomy, and relatedness significantly influence sustained user engagement (Borah et al., 2021; Saleem et al., 2022). This connection between individuals' self-system processes and engagement has been evidenced across various contexts. For example, students' perceptions of competence, autonomy, and relatedness have been linked to cognitive outcomes (Baah et al., 2023; Kashive & Phanshikar, 2023; Samah et al., 2022). Similarly, research on work engagement (Kovjanic et al., 2013; Schreurs et al., 2014) suggests that meeting these needs leads to greater engagement and reduced intention to leave activities. Experiences that fulfill these psychological needs are associated with increased academic engagement and well-being (Martin et al., 2018). Self-determination theory posits that these needs are crucial for individuals' interpretation and meaning-making of experiences. Life narratives, aiding in identity development, express the need for competence and relatedness. In the design of massive open online courses (MOOCs), addressing these psychological needs is vital for fostering intrinsic motivation, enhancing engagement, and improving participant retention. Utilizing a design framework based on self-determination theory in a MOOC led to increased engagement and intrinsic motivation among learners. Furthermore, Suh et al. (2018) found that gamified information systems meeting users' psychological needs effectively engage users by offering hedonic value. Hence, we hypothesize that motivational stimuli, such as innovative gamification (IG) strategies, may induce psychological effects like engagement, subsequently leading to behavioral outcomes (Koivisto & Hamari, 2019). Wearable Activity Trackers (WATs) also possess tangible elements that may support behavioral outcomes, as users perceive them in relation to their health goals (Wasserman et al., 2019). Organizational ICTs with visibility and persistence affordances may influence subjective stress and additional work behaviors (Van Zoonen et al., 2022). Moreover, perceived control and task value may predict behavioral engagement and disaffection, impacting academic performance (Antônio Paz González et al., 2015). Emotional engagement with school has been found to causally relate to youth behavioral and psychological outcomes, although this association may diminish as youth age (Markowitz, 2017). Similarly, Puig et al. (2023) observed that within a brand community, engagement predicts continued intention to participate in the community.
Taking these arguments into account, we hypothesize that:
H5 
: The satisfaction of the need for competence has a positive impact on (a) long-term continued user intention and (b) user satisfaction
H6 
: The satisfaction of the need for autonomy has a positive impact on (a) long-term continued user intention and (b) user satisfaction
H7 
: The satisfaction of the need for relatedness has a positive impact on (a) long-term continued user intention and (b) user satisfaction
Figure 1. conceptual framework of the study.
Figure 1. conceptual framework of the study.
Preprints 105759 g001

3. Methodology

3.1. Data Collection and Procedures

The University Of Cape Coast was chosen as the study area due to its diverse student population and active engagement in mobile learning initiatives. The criteria used for the selection of this area of the study included the university's focus on educational innovation and technology integration making it an ideal setting to evaluate gamification techniques in mobile learning. Both graduate and undergraduate students were selected as participants to ensure a comprehensive understanding of the impact of gamification on mobile learning across different academic levels over four months, from October 2023 to January 2024. The surveys were created using previous literary works as a guide. In light of this, the researchers conducted the study at the University Of Cape Coast, one of the major universities in Ghana with emerging educational dynamics. The research questions were constructed in the English language. Experts in the field advocated sticking to using English only on the survey surveys. The target respondents needed to fulfill certain criteria, such as possessing enough understanding of Innovative Gamification Techniques in Enhancing Mobile Learning Engagements among others, to be selected as participants. The position and profile of the respondents are undergraduate levels 100,200,300,400 and graduate students.
Along with the surveys, the researchers also gave the respondents a consent form and the study's goals. The current study’s participants were guaranteed the utmost secrecy about their answers. Their confidential information will only be used for the research. The writers selected the study survey participants using the stratified sampling technique. A stratified sampling approach was adopted as it is a commonly used technique in survey research when the population consists of heterogeneous groups. It ensures that adequate data is collected from the various subgroups or strata of interest to the study, in this case, the different industry sectors. After the population has been split into subgroups or strata according to preset criteria, a sample is chosen from each stratum. The stratified sampling approach is frequently utilized in various domains, such as environmental studies, public health studies, market research, and educational research. The research used a stratified data-collecting approach since respondents were selected from a range of educational levels. Using WhatsApp and email, this was communicated with 160 respondents throughout the first round of the data-gathering procedure. At the height of the data-collecting period, 90% (N = 144) was acquired by the authors, suggesting a strong response rate to the study's questions. This investigation did not need any animal or clinical trials; hence, ethical clearance is not needed. Furthermore, the study data was gathered anonymously, and the respondents provided their answers to the questions.

3.2. Respondents' Demographic Profile

This section shows that 84 males (58%) and 60 females (42%), made up the participants. According to the respondents' ages, 28% (n=40) are less than 18 years; 28% (n=40) are between the ages of 19 and 29; 44% (n=64) are above 30 years. Of the respondents, 47% had bachelor's level 100, and 33% had bachelor's level 200. Furthermore, 10% of the population had a bachelor's level 300 and 10% level 400.

Data collection and participants

Measures

The research variables were measured using 5-point scales adapted from existing literature. Individuals' interactions with achievement and progression-oriented features, personal traits, social elements, and immersion-oriented aspects were examined following Xi and Hamari (2019). The fulfillment of competence and relatedness needs was assessed based on Xi and Hamari (2019), while the need for autonomy was measured using items from Xi and Hamari (2019) and Standage et al. (2005). User engagement was gauged using the UES-SF developed by O'Brien et al. (2018), encompassing four dimensions: visual appeal, focused attention, perceived usefulness, and reward. Intention for continued usage was assessed using the scale by Tu et al. (2019). Word-of-mouth (WOM) intention was evaluated using items from Hamari and Koivisto (2015), and app ratings were measured using items from Peng et al. (2012).

3.4. The Analytical Tool

The SMART PLS software was used to analyze the data, and a reflecting indicator structural equation modeling (SEM) estimate technique was used. PLS-SEM is unique in the field of business research because of its capacity to evaluate complex prediction relationships. Researchers like Peng and Lai (2012) and Richter et al. (2022) have acknowledged its wide use. Experts like Bodoff & Ho, 2016 and Hair et al., 2017 suggest PLS-SEM as the most appropriate predictive approach for studies aimed at explanation. This is so that it can evaluate how well one model component predicts the values of the other components. A minimum of 5000 bootstraps and a one-tailed significance threshold were used to create the model based on a 360-sample size. A two-phase method was used to evaluate the model. This included the structural model evaluation and the measurement model assessment. The reflecting repeated indicator technique served as the foundation for these models' estimations (Becker et al., 2023).
A composite reliability of (≥0.7) and rho_A (≥0.7) were used to assess the data's dependability. By confirming that the measurements accurately reflect the construct they represent, these indices make the internal consistency of the data visible. AVE (≥0.5), which assesses how closely the measures relate to the underlying idea, was used to examine convergent validity. Using an HTMT ratio (<1) and a common method bias VIF (<5.0), discriminant validity was investigated. A factor loading (>0.7; p≤0.05) indicating that the indicator significantly contributes to the underlying construct and path coefficient, as well as the corresponding effect size (f2 above 0.35 [strong], 0.15 [moderate], and 0.02 [weak]), were used to evaluate the indicator's reliability in the structural model evaluation. The coefficient of determination was also examined, and the findings showed that r2 was above 0.67

4.0. Results and Discussion

The reflecting model configuration (repeated indicator) was used to examine the Innovative Gamification Techniques in Enhancing Mobile Learning Engagements. This method made it easier to investigate in-depth the causal connections between many aspects of gamified techniques, motivational affordances, psychological impacts, and user engagement. Before performing the significance test for the structural model, the measurement model was first assessed using the two-stage models. This methodically made sure that the data was consistent and that the metrics appropriately represented the constructs.

4.1. Model of Measurement

Construct Reliability and Validity
Table 1. Construct Reliability and Validity.
Table 1. Construct Reliability and Validity.
Cronbach's alpha Composite reliability (rho_a) Composite reliability (rho_c) Average variance extracted (AVE)
ASM 0.789 0.791 0.864 0.614
AoK 0.859 0.860 0.905 0.704
CPT 0.839 0.840 0.892 0.674
IE 0.828 0.834 0.879 0.594
L-CUI 0.847 0.848 0.897 0.686
PT 0.766 0.767 0.849 0.585
RTD 0.852 0.854 0.900 0.693
SE 0.769 0.790 0.866 0.683
USF 0.836 0.838 0.890 0.670
Reliability results prove most of the constructs appropriately were reliable in terms of the quality of data collected because they nearly had 0.7 scores of rho_A. ASM and L-CUI were emphatically reliable (rho_A >0.7). Composite reliability for the constructs was satisfactory (rho_c>0.7). Convergent validity was satisfactory for the constructs (AVEs>0.5).
Discriminant Validity
Table 2. Discriminant Validity – Fornell-Larcker Criterion.
Table 2. Discriminant Validity – Fornell-Larcker Criterion.
ASM AoK CPT IE L-CUI PT RTD SE USF
ASM 0.784
AoK 0.505 0.839
CPT 0.578 0.640 0.821
IE 0.667 0.514 0.479 0.770
L-CUI 0.527 0.593 0.592 0.580 0.828
PT 0.685 0.415 0.403 0.655 0.495 0.765
RTD 0.650 0.620 0.731 0.606 0.667 0.490 0.833
SE 0.628 0.541 0.547 0.574 0.511 0.576 0.497 0.826
USF 0.636 0.703 0.681 0.605 0.727 0.516 0.720 0.495 0.818
Discriminant validity between the pairs of the constructs is satisfactory (HTMT ratio<1). The constructs are therefore distinct given their measurement by their respective indicators.
Collinearity Statistics
Based on the Outer VIF values below 3 in Table III, there is no significant multicollinearity issue among the indicators in the measurement model. Multicollinearity refers to the presence of high correlations among the predictor variables (indicators or manifest variables) associated with a particular latent variable (construct). High multicollinearity can lead to biased estimates and instability in the model. The Outer VIF (Variance Inflation Factor) is a measure used to assess this issue in the PLS-SEM (Partial Least Squares Structural Equation Modeling) approach. Generally, an Outer VIF value of 5 or higher for an indicator is considered a cause for concern, indicating potential collinearity problems. However, as evident from Table 3, all the Outer VIF values are well below this threshold, suggesting that the indicators are sufficiently distinct and do not explain a significant portion of each other's variance. This outcome is desirable as it ensures that the indicators are reliable and valid measures of their respective latent variables, and the estimates of the relationships between the latent variables and their indicators are likely to be accurate and stable. Low multicollinearity among the indicators is a prerequisite for obtaining trustworthy results from the PLS-SEM analysis, and the Outer VIF values reported in Table III meet this assumption.
Table 3. Outer VIF.
Table 3. Outer VIF.
ASM1
ASM2 VIF
ASM3 1.807
ASM4 1.895
AoK1 1.556
AoK2 1.384
AoK3 1.707
AoK4 2.100
CPT1 2.922
CPT2 2.151
CPT3 1.838
CPT4 2.129
IE1 2.174
IE2 1.820
IE3 1.934
IE4 2.138
IE5 1.862
L-CUI1 1.643
L-CUI2 1.486
L-CUI3 1.585
L-CUI4 1.962
PT1 2.577
PT3 2.376
PT4 1.286
PT5 1.494
RTD1 1.933
RTD2 1.726
RTD3 2.429
RTD4 2.397
SE2 2.015
SE3 1.657
SE4 1.439
USF1 1.654
USF2 1.712
USF3 1.779
USF4 1.853
1.907
1.901
Common Method Bias
The threat of common method bias was strongly guarded against in this empirical study. The study adopted some previously validated scales for the measurement of the constructs. Also, relatively fewer items were included in the structured questionnaire used for the primary data collection. The items were worded in a simple straightforward manner, making them easily comprehensible. Some of the items were negatively worded in the questionnaire but reverse coding was done during the data processing phase before the actual analysis of the hypotheses. Also, different rating scales were used to assess the opinions of the respondents on the opinions. Inner VIF was used as a statistical measure of the presence of common method bias in the primary data collected via the scales or otherwise. The findings are presented in Table 5.
Table 4. Inner VIF.
Table 4. Inner VIF.
ASM AoK CPT IE L-CUI PT RTD SE USF
ASM 2.467 2.467 2.467
AoK 1.848 1.848
CPT 2.449 2.449
IE 2.159 2.159 2.159
L-CUI
PT 2.245 2.245 2.245
RTD 2.347 2.347
SE 1.836 1.836 1.836
USF
The inner variance inflation factor (VIF) values assess the potential influence of common method bias on the structural model relationships. Ideally, VIF values should be below the conservative threshold of 3.3 to indicate a lack of common method bias issues. In this model, most constructs exhibit acceptable VIF values under 3.3. These relationships appear free from substantial common method bias.
Structural Model Evaluation
Table 5. Outer Loadings.
Table 5. Outer Loadings.
Matrix
ASM AoK CPT IE L-CUI PT RTD SE USF
ASM1 0.810
ASM2 0.823
ASM3 0.775
ASM4 0.724
AoK1 0.784
AoK2 0.842
AoK3 0.896
AoK4 0.830
CPT1 0.790
CPT2 0.838
CPT3 0.853
CPT4 0.801
IE1 0.789
IE2 0.824
IE3 0.782
IE4 0.737
IE5 0.716
L-CUI1 0.777
L-CUI2 0.827
L-CUI3 0.864
L-CUI4 0.842
PT1 0.741
PT3 0.745
PT4 0.793
PT5 0.779
RTD1 0.866
RTD2 0.837
RTD3 0.835
RTD4 0.791
SE2 0.759
SE3 0.864
SE4 0.852
USF1 0.805
USF2 0.808
USF3 0.823
USF4 0.837
All indicator loadings are well above the 0.7 threshold and highly significant (p < 0.001), indicating that the indicators are reliable in representing their respective latent constructs. The t-statistics are very high, providing strong evidence that the indicator loadings are statistically different from zero
Path Co-efficient
Table 6. Path Co-efficient
Table 6. Path Co-efficient
Original sample (O) Sample mean (M) Standard deviation (STDEV) T statistics (|O/STDEV|) P values
ASM -> AoK 0.173 0.173 0.113 1.525 0.127
ASM -> CPT 0.387 0.386 0.088 4.375 0.000
ASM -> RTD 0.431 0.431 0.100 4.307 0.000
AoK -> L-CUI 0.255 0.270 0.089 2.853 0.004
AoK -> USF 0.361 0.367 0.075 4.820 0.000
CPT -> L-CUI 0.123 0.122 0.119 1.033 0.302
CPT -> USF 0.187 0.193 0.085 2.218 0.027
IE -> AoK 0.247 0.253 0.1001 2.459 0.014
IE -> CPT 0.126 0.132 0.104 1.206 0.228
IE -> RTD 0.306 0.305 0.116 2.628 0.009
PT -> AoK -0.049 -0.040 0.114 0.433 0.665
PT -> CPT -0.117 -0.106 0.102 1.148 0.251
PT -> RTD -0.051 -0.049 0.096 0.536 0.592
RTD -> L-CUI 0.419 0.410 0.134 3.118 0.002
RTD -> USF 0.360 0.352 0.110 3.256 0.001
SE -> AoK 0.319 0.310 0.098 3.260 0.001
SE -> CPT 0.299 0.293 0.087 3.442 0.001
SE -> RTD 0.080 0.086 0.101 0.798 0.425
Social elements (SE) as a component of motivational stimuli made a statistically significant positive contribution to predicting the change in psychological Autonomy outcome (AoK) (Beta=-0.319; p=0.001: p<0.05), therefore Hypothesis 1b is supported. Again Social elements (SE) significantly contributed positively to the changes in Competence (CPT) (Beta=-0.299; p=0.001: p<0.05) therefore Hypothesis 1a is supported. Social elements (SE) insignificantly contributed to the changes in psychological Relatedness (RTD) (Beta=-0.080; p=0.425: p>0.05) with a standard deviation of 0.096 therefore Hypothesis 1c is not supported.
Personal traits (PT) on the other hand of gamified motivational systems made a statistically insignificant negative contribution to predicting the change in psychological Autonomy outcome (AoK) (Beta=-0.049; p=0.665: p>0.05) with a standard deviation of 0.114 therefore Hypothesis 2b is not supported. . Again Personal traits (PT) insignificantly contributed negatively to the changes in Competence (CPT) (Beta=-0.117; p=0.251: p>0.05) with a weak standard deviation of 0.102 therefore Hypothesis 2a is not supported.. Personal traits (PT) insignificantly contributed to the changes in psychological Relatedness (RTD) (Beta=-0.051; p=0.592: p<0.05) with a standard deviation of 0.096 therefore Hypothesis 2c is not supported.
Achievement/success metrics (ASM) significantly contributed positively to competence as a component of changes in psychological outcomes competence (CPT) (Beta=0.387; p=0.000: p<0.05) therefore Hypothesis 3 is supported. Achievement /success metrics (ASM) made some positive significant contributions to predicting the change in Relatedness (RTD) (Beta=0.431; p=0.00: p<0.05) with sample mean 0.431 therefore Hypothesis 3c is supported. ASM made some positive but insignificant contributions in predicting the change in Autonomy (AoK) (Beta=0.173; p=0.127: p>0.05) Hypothesis 3b is not supported.
Again, immersion elements (IE) of gamified systems made a statistically significant positive contribution to predicting the change in psychological Autonomy (AoK) (Beta=0.247; p=0.014: p<0.05) with a standard deviation of 0.1001 therefore Hypothesis 4b is supported. Again immersion elements (IE) insignificantly contributed positively to the changes in Competence (CPT) (Beta=0.126; p=0.228: p>0.05) with a weak standard deviation of 1.206 therefore Hypothesis 4a is not supported. Immersion elements (IE) significantly contributed positively to the changes in psychological Relatedness (RTD) (Beta=0.306; p=0.009: p<0.05) with a standard deviation of 2.628 therefore Hypothesis 4c is supported
The relationship between psychological outcomes (competence, Autonomy, and Relatedness) and continued user engagements and satisfaction were analyzed. The predictive relationship between Competence (CPT) and user satisfaction (USF) (Beta=0.187; p=0.027: p<0.05) was positive and significant indicating increased user satisfaction. Competence (CPT) and Continued user intention (L-CUI) (Beta=-0.123; p=0.302: p>0.05) indicate a statistically insignificant relationship
The predictive relationship between Autonomy (AoK) and user satisfaction (USF) (Beta=0.361; p=0.000: p<0.05) was positive and significant indicating increased user satisfaction. Autonomy (AoK) and Continued user intention (L-CUI) (Beta=0.255; p=0.004: p<0.05) indicate a statistically significant relationship indicating high continual and long-term user intention.
Table 7. Model fit – Fit Summary
Table 7. Model fit – Fit Summary
Saturated model Estimated model
SRMR 0.068 0.101
d_ULS 3.079 6.752
d_G 1.674 1.923
Chi-square 1266.116 1347.521
NFI 0.668 0.646
The relationship between Relatedness (RTD) as a component of user psychological outcome and user satisfaction (USF) (Beta=0.360; p=0.001: p<0.05) was positive and significant indicating increased user satisfaction. Relatedness (RTD) as a component of user psychological outcome and Continued user intention (L-CUI) (Beta=0.419; p=0.002: p<0.05) indicate a statistically significant relationship indicating high continual and long-term user intention.
The Model fit provides a summary of model fit statistics comparing a saturated model and an estimated model. It includes various fit indices for both models, such as the Standardized Root Mean Square Residual (SRMR), discrepancy function (d_ULS and d_G), Chi-square values, and the Normed Fit Index (NFI). The saturated model, which is a model with perfect fit, is compared to the estimated model, which is a model based on data. The statistics presented in the document show the differences in fit between these two models. For instance, the SRMR values for the saturated and estimated models are 0.068 and 0.101, respectively. Similarly, the Chi-square values for the two models are 1266.116 and 1347.521.
Figure 2. a: Structural Model; B: Structural Model with indicator loadings.
Figure 2. a: Structural Model; B: Structural Model with indicator loadings.
Preprints 105759 g002aPreprints 105759 g002b

Discussion

This study investigates the impact of several components of gamification systems on users' core psychological demands of autonomy (perceiving control), competence (perceiving capability), and relatedness (perceiving connection to others). Additionally, the study examined the subsequent impacts on user satisfaction levels and their intentions to sustain their involvement. Firstly, the results reveal that social elements (SE) in gamified systems substantially contributed to boosting users' psychological requirements for autonomy and competence, supporting Hypotheses 1a and 1b thereby confirming the findings of (Khosa & Burch, 2023; Collie, R. J, 2020). However, SE did not substantially alter relatedness, failing to support Hypothesis 1c, contrary to the research of (Khosa & Burch, 2023 & Mulcahy et al., 2020). This shows that although social components like leaderboards and social sharing might build emotions of choice and mastery, they may not be as efficient in creating a sense of belonging or closeness among users.
Secondly, personal qualities (PT) did not substantially influence any of the three psychological demands (autonomy, competence, and relatedness), resulting in the rejection of Hypotheses 2a, 2b, and 2c confirming the study of (Wang et al., 2019 & Tian et al., 2022). This means that individual variations in qualities like competition or desire for accomplishment may not play a big influence in how gamification affects users' psychological experiences. Also, achievement/success metrics (ASM) considerably boosted competence and relatedness, supporting Hypotheses 3a and 3c thereby confirming the studies of (Hassan et al., 2020; Peng et al., 2012; Sailer et al., 2017). However, ASM did not substantially influence autonomy, failing to support Hypothesis 3b contrary to the study by (Wee & Choong, 2019; Xi & Hamari, 2019). This implies that items like points, badges, and performance feedback may improve users' feeling of effectiveness and connectivity, but may not appreciably influence their perceived control and choice inside the system. Furthermore, immersion elements (IE) strongly contributed to autonomy and relatedness, supporting Hypotheses 4b and 4c thereby confirming the studies of (Xi & Hamari, 2019), but did not significantly affect competence (Hypothesis 4a unsupported) contrary to the study by (Wang et al., 2019). This shows that immersive components like narratives and role-play may enhance autonomy by enabling users to explore and make meaningful choices, while simultaneously building relatedness via shared experiences and identities. However, they may be less useful for increasing competence compared to more overt accomplishment measurements. Importantly, the psychological demands addressed by the gamified system translated to higher user pleasure, notably for autonomy and relatedness. Additionally, autonomy and relatedness predicted greater continuing user intention, underlining their significance in sustaining long-term involvement. Competence was associated with user happiness but not continuing usage intention. These results show the need to implement multiple gamification aspects to respond to users' varying psychological demands and motivations. Social, accomplishment, and immersion components may synergistically contribute to good user experiences, satisfaction, and persistent engagement, with autonomy and relatedness appearing as especially essential characteristics. However, individual variances in attributes may have a lower influence compared to the gamification design itself.

5.0. Conclusion and Recommendations

5.1. Conclusion

This research studied how diverse components of gamified motivational systems impact users' core psychological requirements of autonomy, competence, and relatedness, as well as downstream consequences on user satisfaction and continuing participation intentions. The results emphasize the complex significance of gamification aspects in generating psychological experiences. Social components significantly boosted autonomy and competence, possibly by creating competitive and collaborative environments that stimulate choice and skill development. Immersion components improved autonomy and relatedness, enabling users to explore options inside narratives and shared virtual worlds. Achievement measures were especially significant for competence and relatedness, offering practical feedback on performance while establishing a feeling of shared journey and success. Critically, addressing users' requirements for autonomy and relatedness translated into improved satisfaction and expected long-term participation with the system. This underlines the necessity of building users' feelings of choice and social connectivity via careful gamification design.

5.2. Recommendations

Based on these insights, add a varied assortment of gamification features targeting distinct psychological demands. Social, immersive, and achievement components may synergize to provide an engaging and need-supportive user experience. Prioritize characteristics that improve autonomy and relatedness, since they were important drivers of happiness and continuous use intentions. Emphasis should be given to aspects that enable meaningful choices, narratives, social connectedness, and shared challenges/accomplishments. While individual personality features were less effective, designers should still consider including aspects that appeal to varied motivational profiles (e.g., competitive versus cooperative rewards). Continuously measure user psychological requirement fulfillment and adapt the system appropriately. Gamification is not one-size-fits-all, thus iterative design tweaks are important. For systems targeted at long-term engagement, lay additional focus on autonomy and relatedness support, since competence alone may not maintain motivation over time. By applying a psychological needs-supportive approach to gamification design, developers may build systems that are intrinsically motivated, highly fulfilling, and encourage prolonged user engagement and pleasant experiences.

5.3. Practical Implications

The results from this research provide some useful practical implications for designers and developers of gamified systems across multiple domains, such as education, workplace training, health and wellness applications, and consumer loyalty programs. First, knowing the varied psychological effects of different gamification features is vital for designing motivationally supporting experiences. Incorporating social aspects like leaderboards and team challenges may boost users' feelings of autonomy and competence, making activities seem more volitional and mastery-oriented. Designers should intentionally employ these components to support autonomous motivation and skill development. Second, immersive gamification via narrative, role-play, and virtual settings looks especially helpful for encouraging autonomy and relatedness. In environments where building user agency and social belonging is crucial (e.g., educational games, health behavior change applications), investing in rich storylines and virtual worlds may provide considerable motivating advantages. Third, the strength of visible accomplishment measures like points, badges, and performance feedback in improving competence and relatedness underlines their worth beyond gamified apps. Progress monitoring and actual successes may build competence, while shared journeys toward objectives can unify people in a psychological sense of community. Furthermore, this research stresses autonomy and relatedness as crucial psychological demands associated with user happiness and persistent engagement. Practically, this means promoting gamification elements that give meaningful choices, chances for self-expression, and social connectedness. While competence is crucial, designers should be careful about overemphasizing pure performance at the price of autonomy and relatedness support. Finally, the data imply an iterative, user-centered strategy may be optimum for gamification design. Since psychological demand fulfillment may differ between situations and people, regularly monitoring users' experiences and adaptively modifying the system is advised to optimize the motivating effect. Through the utilization of these insights regarding the motivational functions of social, immersive, and achievement gamification elements, designers can create experiences that fully satisfy users' psychological requirements, resulting in increased contentment, self-motivation, and sustained engagement on a range of gamified platforms and applications.

5.4. Limitations and Future Research Directions

The main constraints of the present study offer avenues for future exploration. Initially, data collection relied on a single instance of self-administered questionnaires. Therefore, it would be compelling for forthcoming research to utilize longitudinal data to ascertain gamification's sustained success and to gather data directly from the app for an objective quantification of its effectiveness. Another intriguing area for future investigation would entail a thorough examination of usability within gamified mobile applications. While the current study integrated end-users perceived usability as a component of user engagement, further research could delve deeper into this aspect.

Data Availability Statement

Data used for the study will be provided upon request from the corresponding author

Conflicts of Interest

The authors affirm that they do not have any identifiable competing financial interests or personal relationships that could have been perceived to influence the findings presented in this paper.

References

  1. 1. Antônio Paz González, Paoloni, P.-V., Danilo Silvio Donolo, & Rinaudo, C. (2015). Behavioral engagement and disaffection in school activities: exploring a model of motivational facilitators and performance outcomes. Anales de Psicologia. [CrossRef]
  2. Baah, C. 1, Govender, I. 2, Subramaniam, P. R. 2 1 I. D., & Governance, U. of K.-N. (2023). Exploring the role of gamification in motivating students to learn. ProQuest. [CrossRef]
  3. Becker, J.-M. , Cheah, J.-H., Gholamzade, R., Ringle, C. M., & Sarstedt, M. (2022). PLS-SEM’s most wanted guidance. International Journal of Contemporary Hospitality Management. [CrossRef]
  4. Bodoff, D. , & Ho, S. Y. (2016). Partial Least Squares Structural Equation Modeling Approach for Analyzing a Model with a Binary Indicator as an Endogenous Variable. Y. ( 38, 400–419. [CrossRef]
  5. Borah, D. , Malik, K., & Massini, S. (2021). Teaching-focused university–industry collaborations: Determinants and impact on graduates’ employability competencies. Research Policy, 4172. [Google Scholar] [CrossRef]
  6. Chin, W. W. (2009). How to Write Up and Report PLS Analyses. Handbook of Partial Least Squares. [CrossRef]
  7. Deci, E. L. , & Ryan, R. M. (2000). The “What” and “Why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry. [CrossRef]
  8. Deci, E. L. , & Ryan, R. M. (2013). Intrinsic Motivation and Self-Determination in Human Behavior. In Google Books. Springer Science & Business Media. https://books.google.com/books?
  9. Enwereuzor, I. K. , Ugwu, L. E., & Ugwu, L. I. (2022). Unlocking the mask: how respectful engagement enhances tacit knowledge sharing among organizational members. International Journal of Manpower. [CrossRef]
  10. Featherstone, M. , & Habgood, J. (2018). UniCraft: Exploring the impact of asynchronous multiplayer game elements in gamification. International Journal of Human-Computer Studies. [CrossRef]
  11. Hair, J. F. , Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: Indeed a Silver Bullet. Journal of Marketing Theory and Practice. [CrossRef]
  12. Hair, J. , Hollingsworth, C. L., Randolph, A. B., & Chong, A. Y. L. (2017). An updated and expanded assessment of PLS-SEM in information systems research. Industrial Management & Data Systems. [CrossRef]
  13. Hassan, L. , Dias, A., & Hamari, J. (2019). How motivational feedback increases user’s benefits and continued use: A study on gamification, quantified-self and social networking. International Journal of Information Management. [CrossRef]
  14. Hassan, L. , Xi, N., Bahadir Gurkan, Koivisto, J., & Juho Hamari. (2020). Gameful Self-Regulation: A Study on How Gamified Self-Tracking Features Evoke Gameful Experiences. Proceedings of the... Annual Hawaii International Conference on System Sciences. [CrossRef]
  15. Hauser, D. J. , & Schwarz, N. (2015). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods. [CrossRef]
  16. Hellín, C. J. , Calles-Esteban, F., Valledor, A., Gómez, J., Otón-Tortosa, S., & Tayebi, A. (2023). Enhancing Student Motivation and Engagement through a Gamified Learning Environment. Sustainability, 4119. [Google Scholar] [CrossRef]
  17. Ho, M. H.-W. , & Chung, H. F. L. (2020). Customer engagement, customer equity and repurchase intention in mobile apps. F. L. ( 121, 13–21. [CrossRef]
  18. Holzinger, A. , Maximilian Errath, Searle, G., Thurnher, B., & Wolfgang Slany. (2006). From Extreme Programming and Usability Engineering to Extreme Usability in Software Engineering Education (XP+UE→XU). [CrossRef]
  19. Holzinger, A. , Plass, M., Kickmeier-Rust, M., Holzinger, K., Crişan, G. C., Pintea, Camelia-M., & Palade, V. (2018). Interactive machine learning: experimental evidence for the human in the algorithmic loop. Applied Intelligence, 2414. [Google Scholar] [CrossRef]
  20. Hult, G. T. M. , Hair, J. F., Proksch, D., Sarstedt, M., Pinkwart, A., & Ringle, C. M. (2018). Addressing Endogeneity in International Marketing Applications of Partial Least Squares Structural Equation Modeling. M. ( 26(3), 1–21. [CrossRef]
  21. Hunt, N. C. , & Scheetz, A. M. (2018). Using MTurk to Distribute a Survey or Experiment: Methodological Considerations. Journal of Information Systems. [CrossRef]
  22. Kamboj, S. , Rana, S., & Drave, V. A. (2020). Factors Driving Consumer Engagement and Intentions with Gamification of Mobile Apps. A. ( 18(2), 17–35. [CrossRef]
  23. Khosa, A. , & Burch, S. (2023). Flipped Classroom: Students’ Cognitive Needs of Relatedness, Competence, and Autonomy in a Fully-Flipped Program. Journal of University Teaching and Learning Practice. [CrossRef]
  24. Knutas, A. (n.d.). Exploring the connection between gamification and student engagement in computer-supported collabo- ration, 2359. [Google Scholar]
  25. Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal of E-Collaboration. [CrossRef]
  26. Koivisto, J. , & Hamari, J. (2019). The rise of motivational information systems: A review of gamification research. International Journal of Information Management. [CrossRef]
  27. Markowitz, A. J. (2017). Associations Between Emotional Engagement With School and Behavioral and Psychological Outcomes Across Adolescence. AERA Open, 2332. [Google Scholar] [CrossRef]
  28. Martin, N. , Kelly, N., & Terry, P. (2018). A framework for self-determination in massive open online courses: Design for autonomy, competence, and relatedness. Australasian Journal of Educational Technology. [CrossRef]
  29. Mulcahy, R. , Russell-Bennett, R., & Iacobucci, D. (2020). Designing gamified apps for sustainable consumption: A field study. Journal of Business Research. [CrossRef]
  30. 30. Neerja Kashive, & Dharini Phanshikar. (2023). Understanding the antecedents of intention for using mobile learning. Smart Learning Environments. [CrossRef]
  31. Papies, D. , Ebbes, P., & Van Heerde, H. J. (2017). Addressing Endogeneity in Marketing Models. J. ( 2017). Addressing Endogeneity in Marketing Models. International Series in Quantitative Marketing, 581–627. [CrossRef]
  32. Pechenkina, E. , Laurence, D., Oates, G., Eldridge, D., & Hunter, D. (2017). Using a gamified mobile app to increase student engagement, retention and academic achievement. International Journal of Educational Technology in Higher Education. [CrossRef]
  33. Peng, D. X. , & Lai, F. (2012). Using partial least squares in operations management research: A practical guideline and summary of past research. Journal of Operations Management. [CrossRef]
  34. Peng, W. , Lin, J.-H., Pfeiffer, K. A., & Winn, B. (2012). Need Satisfaction Supportive Game Features as Motivational Determinants: An Experimental Study of a Self-Determination Theory Guided Exergame. Media Psychology. [CrossRef]
  35. Puig, A. , Inmaculada Rodríguez, Álex Rodríguez, & Gallego, I. (2023). Evaluating Learner Engagement with Gamification in Online Courses. ( 13(3), 1535–1535. [CrossRef]
  36. Racquel, M. , & Cruz, U. (2018). Australian Academy of Business Leadership Entrepreneurial Competencies among Selected Freshmen Business Students in the Academe. In Novotel Sydney Central (pp. 25–26). http://www.aabl.com.au/aablConference/public/documents/pdf/2019_09_30_10_26_01_AABL_P120_Full_Paper.
  37. Rapp, A. , Hopfgartner, F., Hamari, J., Linehan, C., & Cena, F. (2018). Strengthening gamification studies: Current trends and future opportunities of gamification research. International Journal of Human-Computer Studies. [CrossRef]
  38. Ratinho, E. , & Martins, C. (2023). The role of gamified learning strategies in student’s motivation in high school and higher education: A systematic review. Heliyon. [CrossRef]
  39. Richter, N. F. , Hauff, S., Ringle, C. M., & Gudergan, S. P. (2022). The Use of Partial Least Squares Structural Equation Modeling and Complementary Methods in International Management Research. Management International Review. [CrossRef]
  40. Sailer, M. , Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How Gamification motivates: an Experimental Study of the Effects of Specific Game Design Elements on Psychological Need Satisfaction. Computers in Human Behavior. [CrossRef]
  41. Sailer, M. , Hense, J., Mandl, H., & Klevers, M. (2013). Psychological perspectives on motivation through gamification. Opus.bibliothek.uni-Augsburg.de, 1090. [Google Scholar]
  42. Saleem, A. N. , Noori, N. M., & Ozdamli, F. (2021). Gamification Applications in E-learning: A Literature Review. Technology, Knowledge and Learning. [CrossRef]
  43. Samah, L. A. , Ismail, A., & Hasan, M. K. (2022). The Effectiveness of Gamification for Students’ Engagement in Technical and Vocational Education and Training. International Journal of Advanced Computer Science and Applications. [CrossRef]
  44. Sarstedt, M. , Ringle, C. M., & Hair, J. F. (2021). Partial Least Squares Structural Equation Modeling. F. ( 2021). Partial Least Squares Structural Equation Modeling. Handbook of Market Research, 587–632. [CrossRef]
  45. Seaborn, K. , & Fels, D. I. (2015). Gamification in theory and action: A survey. International Journal of Human-Computer Studies. [CrossRef]
  46. Shmueli, G. , Ray, S., Velasquez Estrada, J. M., & Chatla, S. B. (2016). The elephant in the room: Predictive performance of PLS models. Journal of Business Research, 4564. [Google Scholar] [CrossRef]
  47. Standage, M. , Duda, J. L., & Ntoumanis, N. (2005). A test of self-determination theory in school physical education. British Journal of Educational Psychology. [CrossRef]
  48. Sun, J. C.-Y. , & Hsieh, P.-H. (2018). Application of a Gamified Interactive Response System to Enhance the Intrinsic and Extrinsic Motivation, Student Engagement, and Attention of English Learners. Journal of Educational Technology & Society, /: https, 2645. [Google Scholar]
  49. Tian, H. , Akhtar, S., Qureshi, N. A., & Iqbal, S. (2022). Predictors of entrepreneurial intentions: The role of prior business experience, opportunity recognition, and entrepreneurial education. Frontiers in Psychology. [CrossRef]
  50. Tobon, S. , Ruiz-Alba, J. L., & García-Madariaga, J. (2020). Gamification and online consumer decisions: Is the game over? Decision Support Systems. [CrossRef]
  51. Tu, R. , Hsieh, P., & Feng, W. (2019). Walking for fun or for “likes”? The impacts of different gamification orientations of fitness apps on consumers’ physical activities. Sport Management Review. [CrossRef]
  52. van Roy, R. , & Zaman, B. (2019). Unravelling the ambivalent motivational power of gamification: A basic psychological needs perspective. International Journal of Human-Computer Studies. [CrossRef]
  53. Van Zoonen, W. , Sivunen, A., & Treem, J. (2022). Understanding the Psychological and Behavioral Implications of Organizational ICT Affordances. Academy of Management Proceedings. [CrossRef]
  54. Wang, C. K. John., Liu, W. C., Kee, Y. H., & Chian, L. K. (2019). Competence, autonomy, and relatedness in the classroom: understanding students’ motivational processes using the self-determination theory. Heliyon. [CrossRef]
  55. Wang, X. , Goh, D. H.-L., Lim, E.-P., Vu, A. W. L., & Chua, A. Y. K. (2017). Examining the Effectiveness of Gamification in Human Computation. International Journal of Human–Computer Interaction. [CrossRef]
  56. Wasserman, B. A. , Prate, D., Purnell, B., Muse, A., Abdo, K., Day, K., & Boyd, L. (2019). vrSensory: Designing Inclusive Virtual Games with Neurodiverse Children. [CrossRef]
  57. Wee, S.-C. , & Choong, W.-W. (2019). Gamification: Predicting the effectiveness of variety game design elements to intrinsically motivate users’ energy conservation behaviour. Journal of Environmental Management. [CrossRef]
  58. Zainuddin, Z. , Chu, S. K. W., Shujahat, M., & Perera, C. J. (2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 1003. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

Subscribe

© 2024 MDPI (Basel, Switzerland) unless otherwise stated