An empirical study has been conducted to evaluate the platform's ease of adoption by users. This assessment is crucial to gauge the platform's potential for integration into potential tourists’ practices and to cater to their distinct needs. Through the analysis of received feedback and collected data, the aim is to pinpoint the platform's strengths and areas that might require enhancements to optimize its adoption among the target audience.
4.1. Methodology
To gain a comprehensive understanding of the user experience on the VRRO platform, a mixed-methods approach was chosen. This allows for both quantitative assessments of user experience metrics and qualitative insights into user behaviors, motivations, and feedback.
A convenience sampling method was employed, leveraging peers, friends, and acquaintances primarily through social media outreach and direct contact. This approach was chosen due to its feasibility and the potential to gather diverse feedback quickly. While the study did not set strict demographic controls, the participant pool inherently consisted of tech-accustomed users, ensuring a basic level of technological comfort. This demographic is particularly relevant as the study aims to understand the attitude of users who are already curious or prone to adopt such technologies.
The data collection instruments that have been used are:
Quantitative: The short form of the User Engagement Scale (UES) [
31] is employed to gather numerical data on user engagement. Additionally, it included participant ratings of platform modules, self-assessed digital familiarity, and evaluations of future engagement intentions, offering a comprehensive view of user interaction and platform impact.
Qualitative: Open-ended questions were used to collect qualitative data, providing deeper insights into users' experiences and perceptions. This approach captured feedback on specific platform modules, suggestions for improvements, and challenges encountered during interaction. It also included aspects related to technology usage and participants' self-evaluation of their activity on the platform.
Participants were provided with access to the VRRO platform and asked to interact with it for a comfortable duration. Post-interaction, participants completed the feedback questionnaire [
34] consisting of the demographic questions (age, prior engagement with such technologies, interest, platform of use), the short form of the UES (
Table 1), module-specific rating of platform functionalities, self-rating of users’ activity, impressions (open-ended questions).
Rating or self-scoring questions’ answers are in the form of 5-point Likert scales, with values corresponding to the context. For example, ‘How often did you upload virtual tours’ would have the answers ‘Never’, ‘Once’, ‘Twice or thrice’, ‘Multiple times’, or ‘Often’. Their feedback is recorded [
35] and analyzed to draw insights.
4.2. Results
The results included answers from 31 users, 9 female (29%) and 22 male (71%), of ages ranging from 22 to 42 years old, all past or present visitors of Romania, with the intention of traveling within Romania in the future. Users rated their familiarity with web technologies highly, with a mean score of 4.5 out of 5, while immersive technologies familiarity ratings had a mean score of 3.7 out of 5.
Figure 8.
User demographics: age (a) and gender (b), and reported familiarity for web (c) and immersive (d) technologies.
Figure 8.
User demographics: age (a) and gender (b), and reported familiarity for web (c) and immersive (d) technologies.
The devices used for accessing the platform were preponderantly mobile (21), and the most used operating system iOS. Four participants used both mobile and desktop devices in their experience, and 9 only desktop devices.
Figure 9.
Preferred platform access. (a) 70% of surveyed users preferred access by mobile devices; (b) the preferred operating system is iOS (41.9 %).
Figure 9.
Preferred platform access. (a) 70% of surveyed users preferred access by mobile devices; (b) the preferred operating system is iOS (41.9 %).
4.2.1. Activity
We categorized our users based on their level of interaction with the platform from a pool of 31 participants. A total of 10 users, representing 32.3% of the sample, were identified as 'Active creators.' Within this group, 60% (6 users) created tours, 20% (2 users) wrote articles, and another 20% (2 users) contributed to both the gallery and blog posts. These individuals were not only frequent visitors but also actively engaged with the platform's features.
A smaller segment, consisting of 4 users (12.9%), showed 'Low interaction,' participating minimally and preferring to engage through voting or commenting on posts rather than creating content.
Among the active 14 users, 14.3% (2 users) never voted, 71.4% (10 users) voted multiple times, and 14.3% (2 users) voted once. For commenting, 21.4% (3 users) did so once, 28.6% (4 users) commented more than once, and 50% (7 users) never commented at all. 28.6% (4 users) befriended other users and only 14.3% (2 users) of the active users received badges, a reward for high levels of activity.
The 'Visitors,' who primarily used the platform to view content, constituted the majority, with 17 users making up 54.8% of the total participants. This group's interaction was characterized by significantly less content creation and interaction compared to the other groups.
Figure 9.
User classification by activity (a) and user preferences of viewing virtual tours (b).
Figure 9.
User classification by activity (a) and user preferences of viewing virtual tours (b).
29% of the participants favored a traditional approach of VT viewing, engaging exclusively with basic panoramic views on desktop interfaces, aligning with the 9 users that used exclusively desktop devices.
A significant majority, approximately 68%, utilized mobile devices to access the tours, indicating a preference for more immersive experiences. Within this mobile user segment, a small proportion, roughly 10%, experimented with augmented virtuality, which integrates the physical environment with digital overlays through device sensors. A larger portion, 48%, opted for a virtual reality format, suggesting a preference for a fully immersive digital experience.
Notably, 23% of participants were inclined towards a hybrid approach, engaging with both basic panoramas and VR formats. This indicates a non-exclusive, exploratory interaction with the available technological modalities.
4.2.2. Engagement
The platform's commitment to delivering a seamless and engaging user experience is reflected in the ratings provided by our diverse user base. To capture the essence of this experience, we have analyzed the UES scores alongside the ratings for open-access modules and specialized features accessible to active users. This analysis not only sheds light on the overall satisfaction but also allows us to discern patterns across various user segments.
Table 2.
Summary of User Engagement Scale scores.
Table 2.
Summary of User Engagement Scale scores.
Factor |
Focused Attention |
FA 1 |
FA 2 |
FA 3 |
Perceived Usability |
Aesthetic Appeal |
Reward Factor |
UES (Overall) |
Score |
2.97 |
2.29 |
3.23 |
3.39 |
4.56 |
3.70 |
3.97 |
3.81 |
The lower scores in the FA factor of the UES for the VRRO platform align with the platform’s primary function as an informational and educational tool. The platform's minimalist aesthetic, characterized by a simple color scheme and a straightforward interface, facilitates easy navigation and efficient information consumption. However, this design approach, while effective for its intended purpose, does not inherently promote deep immersion or a sense of escapism, which are typical contributors to higher FA scores.
The content and interaction style on VRRO, including article viewing, virtual tour exploration, and chatting, are primarily structured for information exchange rather than immersive engagement. The VR mode, offering static image views, provides a basic level of virtual experience but lacks interactive or dynamic elements that could lead to higher levels of user absorption. This focus on information delivery over immersive interaction is reflected in the FA scores, particularly in the lower ratings for the first statement, "I lost myself in this experience."
Future enhancements to VRRO could aim to increase user immersion and engagement, by incorporating engaging narratives, interactive tutorials, and enhanced VR experiences. However, any such enhancements should be carefully balanced with VRRO's core informational and educational objectives, ensuring that the platform remains true to its primary purpose while enriching the user experience.
The survey questions also assessed users' intent to use the platform for travel-related activities and their likelihood to recommend it to others. These responses were used to create an endurability index, mirroring the concept from the long-form UES, which reflects users' long-term engagement and potential for recommendation.
Table 3.
Engagement endurability scores.
Table 3.
Engagement endurability scores.
Question Categories |
Average Score |
Commemorate Travels |
4.10 |
Connect with Travelers |
3.65 |
Pre-travel Information |
4.06 |
Inform Other Travelers |
4.10 |
Future Platform Use |
3.94 |
Recommend to Friends |
4.23 |
Drive Index |
3.98 |
Endurability Index |
4.05 |
The Drive Index, with a calculated value of 3.98, serves as a quantitative measure of users' immediate motivation to engage with the VRRO platform. This index is derived by averaging responses to four key questions that assess users' intentions to use the platform for specific activities: commemorating travels, connecting with travelers, gathering pre-travel information, and informing other travelers. This high score suggests that the platform successfully meets user needs and interests.
The Endurability Index, with a value of 4.05, reflects the long-term potential of user involvement and the likelihood of users recommending the platform to others. It is calculated by averaging the Drive Index and responses to the next two questions, regarding future platform use and the propensity to recommend the platform to friends. The score underscores the platform's capability to not only attract users initially but also retain them over time, indicating a strong potential for building a loyal user base and fostering community growth.
Another section of the user feedback questionnaire focused on user satisfaction with specific modules of the platform (the Map, the Blog, and the VT Gallery). Overall ratings for the open-access modules—Map, VT Viewer, and Art Viewer—indicate a consistently positive experience, with scores exceeding 4 out of 5 (
Table 4). This suggests a high level of satisfaction with the platform's core functionalities available to all users.
The specialized features, such as Article Creation, VT Creation, and Chat, evaluated only by our 14 active users, also received favorable ratings, with the Article Creation module standing out with the highest scores. This reflects the platform's ability to cater to content creators with tools that are both functional and user-friendly.
When segmenting the data, nuanced variations in ratings based on users' expertise with web and immersive technologies are observed. Notably, participants who reported high technological familiarity tended to rate the modules slightly higher than the others, pointing to a correlation between technological familiarity and user satisfaction.
Table 5.
Overall experience ratings.
Table 5.
Overall experience ratings.
Module |
UES |
Open access modules |
Advanced features |
Endurability |
Rating |
3.81 |
4.06 |
3.76 |
4.05 |
4.2.3. The influence of digital literacy on experience
In the initial stage of our analysis, we concentrated on the distribution of user engagement scores (UES), technology familiarity assessments, and ratings of the open-access modules (accessible for both visitors and authenticated users) —datasets encompassing information from all 31 participants. The selection of these data sets for normality testing was driven by our aim to understand the underlying patterns of user interaction with the platform.
Before presenting the normality of these distributions, we present an overview of overall ratings by user engagement factors and technology familiarity. This summary (
Table 6) provides a baseline understanding of how users with different levels of digital literacy perceive various aspects of the platform, setting the stage for a deeper analysis of distributional characteristics.
Preliminary analysis of the overall ratings suggests a pattern where participants with higher familiarity with web and immersive technologies tend to report slightly higher engagement scores across the individual factors of the User Engagement Scale, as well as overall the UES. Similarly, the ratings for the Virtual Tour (VT) Viewer, Article Viewer, and Map functionalities also appear marginally higher among these users.
By assessing the normality of these distributions, we aim to validate the appropriateness of the statistical methods used for further comparative analysis. The following table encapsulates the results of the Shapiro-Wilk test and key descriptive statistics, providing a snapshot of the data's distribution.
Table 7.
Data Normality and Descriptive Statistics for Key Study Variables.
Table 7.
Data Normality and Descriptive Statistics for Key Study Variables.
Variable |
Shapiro-Wilk W |
p-value |
Average (x̄) |
Std. Deviation (S) |
UES |
0.9749 |
0.6616 |
3.8118 |
0.5137 |
VT Viewer |
0.8253 |
<0.001 |
4.0323 |
1.016 |
Article Viewer |
0.7897 |
<0.001 |
4.129 |
1.0565 |
Map |
0.8174 |
<0.001 |
4.0323 |
1.0796 |
Web Familiarity |
0.5904 |
<0.001 |
4.5484 |
0.85 |
XR Familiarity |
0.875 |
0.0018 |
3.7097 |
1.1603 |
The comparative analysis of user engagement and module ratings across different levels of technology familiarity revealed no statistically significant differences (
Table 8). The UES scores, which reflect user engagement, showed a slight tendency for higher engagement among users familiar with immersive technologies, but this did not reach statistical significance. Similarly, the ratings for the open-access modules did not differ significantly between users with high or low familiarity with web and immersive technologies. The effect sizes were generally small, indicating minimal practical differences in perceptions between the groups. This suggests that familiarity with the technology did not markedly influence the users' engagement or their perception of the platform's modules. This may also imply that these factors may not have a substantial impact on the user experience within the sample studied.
4.2.4. Qualitative insights
To draw qualitative insights into users' experiences, four open-ended questions were posed to assess their candid opinions on the platform's usability and features. Users expressed a strong affinity for the platform's core functionalities, particularly the ease of creating and viewing virtual tours and the informative content provided by the map feature and blog posts. The platform is seen as a valuable tool for travel enthusiasts, especially those interested in virtual reality and photography, with many users indicating they would recommend it to close friends and family who share a passion for travel.
However, users also highlighted areas for improvement, suggesting a need for a more engaging user interface and additional functionalities such as video uploads and social media integration. Technical issues such as slow loading times and navigation difficulties were noted as annoyances that detract from the overall experience. Despite these challenges, the concept of 'pre-visiting' destinations resonated well with users, pointing to the platform's potential as a planning tool for explorers and adventurers. These insights suggest that while the platform's concept is well-received, focusing on design elegance, technical refinement, and community-building features could significantly enhance user satisfaction and engagement.
4.3. Discussion
The empirical study conducted on the VRRO platform offers valuable insights into user engagement and the platform's potential for widespread adoption. The mixed-methods approach, combining the User Engagement Scale with qualitative feedback, has painted a comprehensive picture of the user experience, highlighting the platform's strengths and areas for improvement.
Quantitatively, the platform demonstrates strong engagement, with the Endurability Index suggesting a high likelihood of sustained use and recommendations to others. This is indicative of a successful user interface and experience design that resonates with the target audience. The high ratings for the platform's core functionalities, such as the Map and VT Viewer, underscore the effectiveness of these features in meeting user needs and expectations.
Qualitatively, the feedback points to a user base that values the platform's utility for travel planning and connection with other travelers. The enthusiasm for creating and viewing virtual tours suggests that VRRO has tapped into a niche that is both relevant and exciting for users. However, the call for a more engaging user interface and additional features like video uploads indicates room for enhancement. Addressing technical issues and improving navigation could further refine the user experience.
The study also reveals that while technological familiarity does influence user satisfaction to some extent, it is not a significant barrier to engagement. This suggests that the platform has managed to create an accessible environment that can cater to both tech-oriented users and those less familiar with immersive technologies. It is important to note that the study's small sample size may limit the generalizability of these results.