Preprint
Review
The Extended Mind & Body in Extended Realities: A Scoping Review of XR Applications and Risks in the Metaverse
Altmetrics
Downloads
330
Views
155
Comments
0
A peer-reviewed article of this preprint also exists.
This version is not peer-reviewed
Submitted:
12 September 2024
Posted:
14 September 2024
You are already at the latest version
Alerts
Abstract
This scoping review explores the intersection of Extended Mind Theory and Extended Reality (XR) technologies, focusing on how Virtual Reality, Augmented Reality, and Mixed Reality reshape human cognition and interaction. XR enables users to offload cognitive tasks and engage in embodied experiences, extending cognition beyond the brain into digital environments. The review highlights a wide range of XR applications, from immersive learning in STEM education and medical training, neuropsychological assessment to therapeutic interventions, arts and entertainment, professional skills development, retail and e-commerce, remote work, sports training, architecture and urban planning, and cultural heritage preservation. XR's integration with modalities like haptics, eye-tracking, face- and body-tracking, and brain-computer interfaces further enhances cognitive extension and user engagement. However, alongside these advancements come significant ethical, psychological, and societal challenges, such as data privacy concerns, the psychological effects of prolonged immersion, and social inequality arising from disparate access to XR technologies. This review emphasizes the need for robust ethical frameworks that address these challenges, ensuring that XR technologies enhance human development while maintaining autonomy, privacy, and mental well-being. As XR continues to evolve and integrate with artificial intelligence and other emerging technologies, its role in expanding human cognition will depend on responsible implementation and governance.
Keywords:
Subject: Computer Science and Mathematics - Computer Science
1. Introduction: Theoretical Foundations of the Extended Mind and Embodiment
1.1. The Extended Mind in the Context of the Metaverse
The traditional understanding of human cognition has long confined the workings of the mind within the boundaries of the brain. This perspective, rooted in early neuroscientific studies, views cognitive processes such as memory, thought, and perception as solely functions of the brain’s internal networks of neurons, synapses, and chemicals [1]. In this brain-centric model, the mind is seen as an isolated system, working autonomously to process information from the external world. These processes were believed to be locked within the brain, and the tools we use—such as notebooks, calculators, or computers—were considered external aids, disconnected from the intrinsic workings of cognition itself. Early cognitive science and neuroscience laid a strong emphasis on mapping the neural pathways responsible for specific cognitive functions, assuming that understanding these brain-based mechanisms was the key to understanding cognition as a whole [2].
However, this traditional model of cognition has been challenged by more recent developments in cognitive science, spurred by thinkers like Andy Clark and David Chalmers (1998) [3]. Their Extended Mind Theory proposes a much broader and more inclusive view of cognition, one that extends beyond the confines of the skull. This theory asserts that the mind does not operate in isolation; rather, it is distributed across the brain, body, tools, and environment with which individuals interact. Clark and Chalmers argue that external objects, technologies, and environments play an active role in cognitive processes, functioning as extensions of the mind rather than mere passive aids. This marks a significant departure from traditional views, suggesting that human cognition is not solely internal but also external, involving the tools and environments that shape and support cognitive functions [4].
One of the most influential examples used by Clark and Chalmers to illustrate this theory is the case of Otto, a man with Alzheimer’s disease, who uses a notebook to store information he can no longer reliably keep in memory. According to the traditional model, Otto’s memory would be confined to his brain, and the notebook would be seen as a simple external tool. However, the extended mind theory argues that the notebook is, in fact, part of Otto’s cognitive system—functionally equivalent to his biological memory. In this way, external devices and tools become integrated into our cognitive processes, reshaping how we perceive, think, and remember [3].
In the digital age, particularly within the immersive environments of the metaverse and extended realities (XR), the extended mind theory has become even more relevant. Technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR) are not merely external tools used for cognitive support; they are increasingly integral to shaping how individuals think, learn, and interact with the world around them [5]. These immersive environments allow for real-time interaction and feedback, making the digital tools not just facilitators of cognition but active participants. This convergence of the digital and the physical exemplifies the core idea of the extended mind theory: cognition extends beyond the brain and is distributed across both internal neural processes and external technological systems.
Within AR and VR, users engage with digital objects and environments that augment their cognitive capacities, offering new ways to process information, solve problems, and understand complex concepts. For example, when interacting with a virtual environment, the user’s actions—such as manipulating a virtual object—are not just outputs of internal cognitive processes but are shaped by the feedback from the external environment. In this way, the digital environment becomes part of the cognitive process itself, functioning as a cognitive partner. Technologies like eye-tracking and gesture recognition enable seamless interaction with these environments, allowing the user to offload cognitive tasks onto the external world [6]. This offloading exemplifies how immersive environments in the metaverse blur the line between internal cognition and external augmentation, creating a continuum between the mind and the tools it interacts with [7].
The metaverse, as a convergence of immersive virtual environments, provides a fertile ground for exploring the extended mind. In these environments, virtual worlds are not passive; they are dynamic, interactive spaces that respond to the user’s actions and cognitive state. Through multimodal interfaces—ranging from simple input devices like keyboards and mice to advanced systems such as eye-tracking, haptic feedback, and brain-computer interfaces (BCIs)—users can manipulate and navigate these worlds in ways that extend their cognitive capabilities. These tools do not simply assist cognition; they become integral components of the cognitive process, shaping how information is processed and how experiences are interpreted. Haptic feedback, for instance, allows users to experience tactile sensations in a virtual world, enabling them to engage more deeply with virtual objects, which in turn alters their cognitive engagement with the environment [8,9,10,11]. Similarly, BCIs enable users to control virtual objects directly through neural activity, bypassing the need for physical input devices, further demonstrating the integration of external technology into the cognitive system [12].
These immersive tools and environments in the metaverse mirror the central tenets of the extended mind theory: cognition is not confined to the brain but is distributed across the physical and digital tools that enable and extend our thinking capabilities [3]. As the boundaries between the physical and digital worlds continue to blur, the extended mind theory provides a compelling framework for understanding how our cognitive systems are evolving in response to new technologies. In the context of the metaverse, human cognition is increasingly augmented by digital tools, creating new possibilities for learning, problem-solving, and interaction that transcend the limitations of biological cognition alone [13]. As XR technologies continue to develop, they will play an increasingly central role in expanding the boundaries of human cognition, allowing us to integrate more seamlessly with the external world, both physical and digital..
1.2. Virtual Tools as Cognitive Extensions
Within the metaverse, digital environments and virtual objects do not merely support cognition—they actively shape and transform it. These environments provide users with enhanced cognitive capabilities by integrating external tools and technologies that allow for interactions impossible in the physical world. For example, in VR learning environments, users can engage in highly immersive and interactive tasks, such as exploring the human body at a cellular level, conducting complex physics experiments, or manipulating molecular structures. These tasks, which would be challenging or even impossible in the real world due to constraints of time, space, or resources, become feasible and intuitive within virtual environments. This creates a form of cognitive scaffolding where virtual tools offload information processing tasks from the brain to the environment, enhancing the user’s ability to understand and retain complex concepts [13]. The immersive nature of VR enables learners to interact directly with the content, deepening their engagement and making abstract ideas more tangible and easier to comprehend.
Similarly, BCIs allow users to interact with virtual environments using neural signals, bypassing traditional input methods such as keyboards or controllers. BCIs represent a radical extension of cognition into the digital realm, enabling users to control virtual objects, navigate digital landscapes, and solve complex problems through direct neural activity. This type of interaction not only facilitates a more intuitive and seamless user experience but also creates new avenues for cognitive offloading and enhancement. Cognitive tasks, such as spatial navigation and problem-solving, which were once confined to the brain and body, are now distributed across the brain, body, and virtual tools [12]. In this sense, the metaverse, through the use of BCIs and other advanced interfaces, becomes a cognitive partner, augmenting the brain’s natural abilities and allowing users to perform tasks beyond their biological capabilities.
As technologies in the metaverse evolve, they increasingly take on roles that traditional cognitive tools cannot. For example, haptic feedback systems allow users to feel the texture, weight, and resistance of virtual objects. This form of sensory feedback engages the sensory-motor systems, which are critical to embodied cognition—the idea that cognition is deeply rooted in the body’s interactions with the physical world [8,9,10,11]. In virtual environments, haptic feedback allows users to engage in tactile interactions that simulate real-world experiences, enabling them to "feel" virtual objects as if they were physical. This sensory engagement transforms how users perceive and interact with the digital world, influencing their decision-making processes, motor skills, and learning outcomes.
These sensory experiences are not passive—they actively shape how users interact with virtual environments. By providing tactile and kinesthetic feedback, haptic systems enhance the sense of presence in the metaverse, making users feel as though they are physically present within the digital space. This heightened sense of presence has been shown to improve performance in tasks such as object manipulation, spatial reasoning, and complex problem-solving [8,9,14,15]. The digital environment, through these interfaces, becomes a cognitive partner, working in tandem with the user’s brain and body to extend cognitive capabilities beyond their natural limits. Multimodal interactions, which combine visual, auditory, and tactile inputs, create an enriched environment where cognitive processes such as attention, memory, and learning are enhanced by the simultaneous engagement of multiple senses [16].
Through the use of such interfaces, the digital environment becomes more than just a space for interaction—it becomes a co-constructive agent in the cognitive process. Users can offload tasks onto the virtual environment, allowing them to focus their cognitive resources on higher-level thinking, creativity, and problem-solving. The metaverse, through its integration of multimodal feedback systems, provides users with an unprecedented degree of cognitive augmentation. By extending the user’s mind beyond biological constraints, these technologies foster new forms of cognition that were previously unattainable, offering a glimpse into the future of human-computer interaction and cognitive enhancement [5].
1.3. Embodied Experiences in the Metaverse
The concept of embodiment is crucial for understanding the extended mind within the context of the metaverse. Embodiment refers to the deep integration of the mind, body, and environment in cognitive processes. It suggests that cognition does not occur solely within the brain but is shaped by our bodily interactions with the physical world. In the metaverse, users can embody digital avatars that mimic real-world physicality, creating immersive experiences where cognitive processes such as spatial reasoning, memory, and attention are enhanced [17]. The ability to embody a virtual body in this way blurs the boundaries between internal cognition and external technology, reinforcing the central tenets of the extended mind theory.
The virtual body, or avatar, becomes an extension of the user’s physical self, facilitating interaction with the virtual environment and allowing the user to engage in complex tasks in new and intuitive ways [18,19]. This sense of embodiment in the metaverse creates a profound psychological effect, as users experience cognitive processes—like social interaction, decision-making, and problem-solving—within the virtual world just as they would in the real world. The degree of embodiment experienced by users can deeply influence their behavior and cognitive engagement in virtual environments.
1.3.1. The Role of Illusions in Embodied Experiences
Within the metaverse, the sense of embodiment is not a simple byproduct of interacting with a virtual environment—it is actively constructed through several illusions that shape how users experience their virtual selves and surroundings. According to Slater (2018) [20], four key illusions play an essential role in creating the immersive and embodied experience within virtual worlds: placement illusions, plausibility illusions, embodiment illusions, and time illusions.
- Placement Illusions: The first illusion is the placement illusion, which refers to the sense that the user’s body and consciousness are physically present within the virtual environment. This illusion is critical in fostering immersion, making users feel as though they have truly entered the virtual world. Placement illusions occur when the virtual environment is responsive to the user’s movements and actions in real time, enhancing the feeling of physical presence. For example, when a user reaches out to touch a virtual object, the virtual environment responds in ways that make the action feel natural and aligned with the user’s expectations, creating a convincing sense of being "placed" in the virtual space [20]. Research shows that when placement illusions are strong, users exhibit behavioral and cognitive changes that reflect the same patterns they would display in the real world, reinforcing the integration of the virtual environment into their cognitive processes [5].
- Plausibility Illusions: The second critical illusion is the plausibility illusion, which refers to the degree to which the virtual environment behaves in a believable and coherent manner. For users to feel fully immersed and embodied in the metaverse, the virtual world must respond to their actions in ways that are plausible within the context of the virtual environment. This illusion is vital for maintaining a user’s sense of presence, as it ensures that the virtual world behaves in a way that matches their expectations. For instance, if a user interacts with a virtual object, the object should behave as expected based on its virtual properties—if a virtual glass of water is knocked over, the water should spill accordingly. When plausibility illusions are disrupted (e.g., when virtual objects behave in an unnatural or unexpected manner), it breaks the user’s immersion and diminishes the sense of embodiment [21].
- Embodiment Illusions: The third type, the embodiment illusion, directly concerns the user’s avatar in the virtual world. This illusion refers to the experience of "owning" the virtual body as if it were the user’s own. The avatar is not just a representation on a screen but becomes integrated into the user’s sense of self, such that movements of the virtual body feel as though they are the user’s own physical movements. This phenomenon is particularly evident when users perform motor tasks in the metaverse, such as reaching, grasping, or walking. Embodied cognition theories suggest that the ability to interact with and control a virtual body can lead to changes in how users perceive themselves and others, influencing cognitive functions such as social interaction, empathy, and self-awareness [17,19]. The stronger the embodiment illusion, the more users treat their virtual body as if it were their real one, leading to more naturalistic behaviors and thought processes in the virtual world [20,21].
- Time Illusions: Lastly, time illusions refer to the altered perception of time that users experience within immersive environments. In the metaverse, time may appear to pass more quickly or slowly depending on the user’s level of engagement and the design of the virtual experience. Time illusions occur when the user becomes so absorbed in the virtual world that they lose track of time in the real world. This illusion is particularly important in environments designed for extended interaction, such as educational simulations, where prolonged engagement is desirable. The perception of time can be manipulated through the use of fast-paced tasks, slow-motion effects, or carefully timed feedback to either accelerate or decelerate the user’s sense of time [22]. Time illusions highlight the extent to which the metaverse can alter fundamental aspects of cognition, including attention and decision-making, by reshaping how users experience the passage of time.
1.3.2. Embodied Cognition and the Metaverse
The interplay of these illusions within the metaverse creates an environment where the user’s mind, body, and environment are deeply interconnected. The experience of embodiment allows users to interact with virtual objects and other avatars in ways that feel natural and intuitive, further blurring the line between internal cognition and external technologies. As users engage with virtual objects, their actions and perceptions are transformed, leading to shifts in cognitive processes such as problem-solving, collaboration, and social interaction. In the context of the extended mind theory, these virtual tools and environments act as cognitive extensions, becoming integral to the user’s cognitive system [3].
For example, in a collaborative metaverse simulation, where users must work together to solve a complex problem, their interactions with virtual objects and with each other become part of the collective cognitive process. Each user’s avatar, through the embodiment illusion, allows them to perform tasks that influence not only their own cognitive processes but also those of their collaborators. In this sense, the virtual environment serves as a shared cognitive space where thought, perception, and action are distributed across multiple users and virtual tools. This reflects the principles of the extended mind, where cognition is distributed across individuals, tools, and environments [5].
1.4. Implications and Future Directions
The implications of the extended mind theory in the metaverse are profound, particularly as XR technologies—encompassing VR, AR, and MR—continue to advance. These technologies allow users to offload various cognitive tasks onto external, digital environments, fundamentally reshaping how humans process information, solve problems, and engage with learning. As XR systems become more sophisticated, they offer unprecedented opportunities for cognitive augmentation, where users can extend their cognitive capabilities beyond the natural limits of human biology. The offloading of cognitive functions, such as memory recall, spatial reasoning, and problem-solving, into immersive digital environments marks a significant shift in how cognition is understood and applied across different domains.
Eventually, the metaverse has the potential to become the ultimate manifestation of the extended mind theory, where cognition is not confined to the brain but is distributed across a vast network of physical, digital, and artificial agents. As XR technologies, AI systems, and multimodal interfaces become more integrated and sophisticated, the metaverse will offer users the ability to engage in hyper-augmented cognitive experiences. These experiences will allow individuals to offload, extend, and enhance their cognitive processes in ways that were previously unimaginable. Whether in education, therapy, professional training, or even entertainment, the metaverse represents a profound shift in how human cognition is understood and applied.
In this new cognitive landscape, users are not simply interacting with a digital environment; they are engaging with a dynamic, adaptive, and intelligent system that actively participates in their cognitive processes. The implications of this shift are far-reaching, as it redefines the relationship between humans and technology, enabling new forms of cognition, creativity, and collaboration that transcend the limitations of biological intelligence alone [3]. As XR technologies continue to evolve, the metaverse will become a central space for exploring and expanding the boundaries of the human mind, offering a glimpse into the future of cognitive enhancement and digital interaction.
2. The Virtual Continuum: XR, AR, VR, and MR in Extended Realities
The concept of the Virtual Continuum, first introduced by Milgram and Kishino (1994) [23], provides a framework to understand how physical and virtual realities interact across a spectrum (see Figure 1). This continuum ranges from entirely physical environments at one end to fully virtual environments at the other. Within this spectrum lies a blend of real and virtual experiences facilitated by immersive technologies such as VR, AR, and MR. Collectively referred to as XR, these technologies augment human cognition, perception, and interaction by offering varying levels of immersion and interaction between the digital and physical worlds.
2.1. Virtual Reality in the Virtual Continuum
At the extreme end of the Virtual Continuum is VR, where users are fully immersed in a computer-generated environment, entirely replacing the real world. This immersion is achieved through head-mounted displays (HMDs), controllers, and body-tracking systems that enable users to navigate, manipulate objects, and experience sensory feedback in the virtual space. VR’s ability to create an intense sense of presence has made it a powerful tool for applications such as education, therapy, and entertainment [20].
In education, VR enables students to explore environments that would be otherwise impossible to visit in real life, such as ancient cities or deep ocean habitats. For example, scientific experiments that would be too costly, dangerous, or impractical in a physical setting can be safely and interactively conducted in VR, allowing for deeper engagement and experiential learning [24,25]. VR’s immersive qualities also lend themselves to therapeutic applications, where patients can confront fears in controlled environments (e.g., phobias, PTSD) or practice motor skills in rehabilitation [26].
The total immersion provided by VR allows for significant cognitive offloading, enabling users to delegate tasks like spatial reasoning, memory recall, or problem-solving to the virtual environment [13]. VR’s potential in these areas continues to grow, particularly with advancements in haptic feedback and brain-computer interface (BCI) technologies, which will further extend the scope of cognitive augmentation [27,28].
2.2. Augmented Reality (AR) in the Virtual Continuum
At the opposite end of the Virtual Continuum is AR, which enhances the real world with overlaid digital elements, offering a blend of physical and virtual content. Unlike VR, which replaces the physical world, AR complements it, typically through devices like smartphones, tablets, or AR glasses. This augmentation allows users to interact with digital information while remaining fully engaged with their real-world environment.
AR has found significant use in sectors such as education, healthcare, retail, and industrial applications. In educational settings, AR allows for interactive learning, such as overlaying 3D models onto textbooks or projecting historical events into physical spaces for a more immersive experience [29]. AR’s capacity to provide real-time, context-sensitive information makes it particularly valuable in professional training. For instance, in the medical field, AR can overlay anatomical data or surgical instructions onto patients, enhancing precision during complex procedures [30].
In industrial applications, AR is used to train workers by superimposing assembly instructions onto machinery or guiding maintenance tasks with interactive visual aids [31]. The integration of AR with AI and other technologies has made it a critical tool for enhancing spatial cognition and decision-making, as users can offload tasks like navigation, visualization, and complex spatial reasoning onto the augmented environment [29,32].
2.3. Mixed Reality (MR) in the Virtual Continuum
Sitting between AR and VR on the continuum MR, a hybrid technology that allows real and virtual elements to interact in real-time. In MR, users can manipulate both physical and digital objects, with virtual elements responding to physical actions and vice versa. This creates a seamless blend of reality and simulation, enabling more dynamic interactions than AR or VR alone.
MR applications are particularly valuable in fields such as architecture, engineering, and healthcare. For example, architects can use MR to visualize a building design within its intended physical space, offering clients a walkthrough of a structure before it is built [32,33]. Similarly, in healthcare, MR can be used to simulate surgeries where real and virtual tools interact in real time, providing a safe environment for training medical students and professionals [30,34]. MR’s ability to merge real and digital objects extends the concept of the extended mind, allowing users to offload cognitive tasks onto virtual tools while engaging physically with their environment. This is particularly useful in collaborative scenarios, where teams can work on shared virtual models in real-world settings, enhancing distributed cognition and real-time problem-solving [35].
2.4. Extended Reality (XR): The Integration of AR, VR, and MR
XR)is an umbrella term that encompasses the full range of experiences across the Virtual Continuum, from AR to VR to MR. XR technologies allow users to engage with different levels of immersion and interaction, creating a rich landscape for cognitive augmentation. As XR technologies become more advanced, the lines between physical and virtual worlds will blur, offering seamless transitions between these spaces. XR’s potential extends across diverse fields, including professional training, healthcare, education, and entertainment. For instance, in the entertainment industry, XR enables users to experience interactive virtual concerts or events, blending physical and virtual spaces to create new forms of artistic expression. In healthcare, XR systems can provide real-time medical data overlays during surgeries or create fully immersive environments for therapy and rehabilitation [34].
As these technologies integrate with AI, BCI, and multimodal interfaces, XR will continue to evolve as a powerful tool for cognitive extension. AI-driven XR systems can dynamically adjust to users’ cognitive loads or emotional states, providing real-time personalized experiences that enhance cognitive performance [36,37]. Moreover, the potential for cognitive offloading in XR environments, where tasks such as memory recall, navigation, and problem-solving are distributed across digital and physical systems, creates significant opportunities for enhancing human cognition [38].
However, with this growth comes concerns about data privacy, ethical use, and potential misuse. XR systems collect vast amounts of biometric and behavioral data, raising concerns about how this data is used by corporations and governments [39]. Additionally, the risk of addiction to immersive environments, cyberbullying, and manipulation of public opinion within these spaces necessitates a robust ethical framework to ensure the responsible development of XR technologies [40]. As XR technologies continue to advance and integrate, it becomes crucial to explore how multiple modalities—such as haptic feedback, gaze tracking, and brain signals—play a role in shaping immersive experiences within the Virtual Continuum.
3. Multiple Modalities in XR: Enhancing Immersion in the Metaverse
The rise of XR technologies has ushered in a new era of human-computer interaction that leverages multiple sensory and physiological modalities to create deeply immersive and responsive virtual environments. These multimodal interfaces enhance both cognitive and physical engagement, providing real-time feedback that adjusts the virtual experience based on the user’s actions, emotions, and physiological states (see Table 1). This section delves into the key modalities—eye-tracking, facial tracking, hand and body tracking, haptic feedback, and more—that are shaping the future of XR and the metaverse.
3.1. Eye-Tracking: Precision and Cognitive Insights
Eye-tracking technology is a fundamental modality in XR environments, offering an intuitive and dynamic way for systems to monitor and respond to where users are directing their visual attention experience [41]. By capturing real-time data on the user’s gaze, eye-tracking allows virtual environments to be more responsive, creating more immersive and engaging experiences. The ability to track eye movements enhances the user interface by dynamically adjusting content and providing more intuitive ways to interact with virtual objects and environments. This technology is particularly beneficial in immersive applications like education, professional training, gaming, and even healthcare, where cognitive load and user attention are critical.
One of the primary functions of eye-tracking is dynamic interface adjustment. When a user looks at specific objects or areas within a virtual environment, the system can automatically highlight those elements, bring them into sharper focus, or trigger relevant actions. For example, in a professional training simulation, eye-tracking can be used to detect where a trainee is focusing their attention, and the system can provide real-time feedback or additional information about the object or task being observed. Similarly, in gaming or educational settings, environments can shift scenes or change interactive elements based on the user’s gaze, creating a more engaging and seamless experience [41].
Eye-tracking also provides valuable insights into the user’s cognitive state by analyzing specific gaze patterns, such as fixation duration (how long the user’s eyes remain focused on a particular object) and saccades (rapid movements of the eyes between points of focus) [42,43]. These metrics can be used to estimate cognitive load, or the amount of mental effort the user is exerting at any given time. This data is particularly useful in educational and professional training applications, where an understanding of cognitive load can inform how content is presented. For example, if eye-tracking data suggests that a user is experiencing high cognitive load—such as long fixation durations on a complex object or scene—the system can simplify the environment or reduce distractions to improve comprehension [44]. In contrast, if low cognitive load is detected, the system may introduce more challenges or additional information to maintain engagement and optimize learning outcomes.
Additionally, eye-tracking enhances interaction capabilities within XR environments, enabling users to interact with virtual objects in a more natural and seamless manner [45]. Instead of relying on traditional input devices like controllers or keyboards, users can select or manipulate objects simply by looking at them. This gaze-based interaction creates a highly intuitive and immersive experience, making it easier for users to navigate complex virtual environments. For instance, in a virtual museum tour, eye-tracking could allow visitors to focus on an exhibit, triggering detailed information or a 3D model to appear, enriching the educational experience without the need for physical interaction. In medical applications, eye-tracking could help doctors review digital patient files by highlighting relevant data as they scan through information, thereby improving efficiency and reducing cognitive strain [46].
The future of eye-tracking in XR also holds significant potential for personalized learning and professional training. By continuously monitoring where users direct their attention, systems can tailor experiences to meet individual learning needs or adapt training modules based on a user’s progress. In virtual classrooms or corporate training sessions, eye-tracking could be used to ensure participants are focusing on critical information and to provide real-time guidance or feedback based on their gaze patterns. Moreover, for professionals in high-stress environments, such as pilots or surgeons, eye-tracking could enhance training simulations by monitoring stress levels and cognitive load, ensuring that users maintain focus and achieve high levels of precision during critical tasks.
As XR technologies continue to evolve, the integration of eye-tracking will play a pivotal role in creating more personalized, responsive, and immersive virtual experiences. The ability to track and analyze visual attention in real time not only enhances the usability of XR interfaces but also provides invaluable insights into the cognitive processes of users, making it a cornerstone of future applications in education, training, healthcare, and entertainment. This foundation of visual interaction leads naturally into a broader discussion of how multiple modalities such as facial tracking, hand tracking, and physiological monitoring work in synergy within XR to create a comprehensive and immersive user experience..
3.2. Facial Tracking: Enhancing Emotional and Social Presence
Facial tracking is an increasingly pivotal modality in XR environments, significantly enhancing emotional and social presence within virtual spaces. By capturing users’ facial movements in real-time, this technology enables virtual avatars to replicate and mirror the user’s facial expressions, adding a level of authenticity and emotional depth to social interactions in the metaverse. Whether it’s a smile, a frown, or a raised eyebrow, facial tracking allows virtual avatars to convey the subtleties of human emotion, making interactions in XR environments more natural and engaging. This capability fosters deeper emotional connections, enabling more genuine communication and collaboration in virtual settings, such as virtual workspaces, social hangouts, or even virtual reality games [47].
In social contexts, facial tracking serves to bridge the gap between the real and virtual worlds by replicating the nuances of non-verbal communication that are often lost in traditional online interactions. When avatars reflect real-time facial expressions, users experience heightened social presence, the feeling that they are truly interacting with others in a shared space. This has significant implications for collaborative work environments, where non-verbal cues such as a nod of approval or a quizzical look can inform group dynamics and decision-making processes. Similarly, in virtual social spaces, facial tracking allows users to better express their emotions and intentions, promoting empathy and mutual understanding [48].
Facial tracking also enhances emotional feedback within XR environments by detecting and interpreting changes in the user’s emotional state. For instance, systems equipped with facial tracking can analyze micro-expressions—subtle, involuntary facial movements that often indicate emotions such as stress, frustration, or joy. By continuously monitoring these expressions, the system can adapt the virtual environment in real time to better suit the user’s emotional needs. This capability is particularly valuable in therapeutic and counseling settings, where emotional tracking can help therapists create more personalized and responsive virtual environments. For example, in virtual exposure therapy designed to treat phobias or PTSD, facial tracking can detect signs of anxiety, such as furrowed brows or tensed facial muscles, and automatically adjust the environment to reduce stress or offer calming stimuli [49]. By tailoring the virtual experience to the user’s emotional state, facial tracking enables more effective and supportive therapeutic interventions.
Another important application of facial tracking is in emotionally adaptive virtual environments. As XR technologies advance, virtual spaces are becoming increasingly interactive and responsive, not just to physical actions but also to emotional cues. For instance, in a virtual classroom, facial tracking could detect when a student appears confused or disengaged and offer additional resources, such as a simplified explanation or more engaging content. Similarly, in professional training scenarios, facial tracking could identify when a user is under stress and adjust the task difficulty or provide more guidance, ensuring a better learning experience. These emotionally intelligent environments foster higher levels of user engagement and improve outcomes by creating a more personalized and adaptive experience [49].
In gaming and entertainment, facial tracking introduces new levels of immersion and emotional engagement. Imagine playing a game where the virtual characters respond to your expressions—if you show signs of fear or surprise, the game might adjust its difficulty or alter the storyline in response. By making virtual environments emotionally reactive, facial tracking enhances immersion and creates more compelling, personalized narratives that respond to the player’s emotions in real time. This kind of interactive storytelling can be deeply engaging, allowing users to form emotional connections with virtual characters and environments in ways previously impossible.
Finally, facial tracking holds potential for remote and hybrid work environments, where communication often lacks the emotional richness of face-to-face interactions. By enabling avatars to reflect real-time facial expressions, teams collaborating in virtual environments can communicate more effectively, with the added dimension of non-verbal cues. This can be particularly important in negotiations, brainstorming sessions, or client meetings, where subtle facial expressions convey important information about mood, engagement, and decision-making. With facial tracking, remote interactions can more closely mimic the emotional complexity of in-person meetings, making virtual workspaces more effective and satisfying [47].
As facial tracking becomes more integrated into XR systems, it will contribute to the development of emotionally responsive and socially rich virtual environments. By accurately capturing and reflecting emotional states, facial tracking not only enhances communication but also opens the door to more adaptive and personalized virtual experiences. This capability sets the stage for the broader integration of multiple modalities—such as hand tracking, full-body tracking, and physiological monitoring—which will work in tandem to create more comprehensive, immersive, and emotionally responsive virtual experiences.
3.3. Finger and Hand Tracking: Intuitive Manipulation of Virtual Objects
Finger and hand tracking are transformative modalities in XR environments, offering users the ability to interact with virtual objects in a natural, intuitive manner. By capturing the fine movements of fingers and hands, this technology allows users to engage with digital environments through familiar gestures such as pinching, grabbing, swiping, and pointing. These actions mimic real-world interactions, eliminating the need for traditional controllers, and creating a more immersive experience. Users can reach out to manipulate objects in the virtual world just as they would in physical space, making the interface more seamless and enhancing overall user engagement [50].
This form of interaction is particularly valuable in XR environments designed for education and training. For example, medical trainees can use hand tracking to practice complex surgical procedures, manipulating virtual surgical tools with precision and receiving real-time feedback on their movements. This form of hands-on training allows users to develop muscle memory and technical skills in a risk-free virtual environment. Similarly, in engineering or design fields, hand tracking enables users to interact with virtual prototypes, making adjustments in real time as if they were handling physical models. The tactile-like nature of hand gestures fosters deeper learning and retention, as users are actively engaged in problem-solving through intuitive, embodied interaction [51].
Moreover, hand tracking is essential for enhancing collaborative work in XR environments. In virtual workspaces or design studios, multiple users can interact with the same virtual objects in real time, allowing for simultaneous co-creation of digital models or prototypes. For instance, in a virtual architecture studio, team members from different locations can manipulate building designs together, making adjustments collaboratively as though they were physically present in the same room. The ability to "handle" virtual objects with precision not only makes collaboration more fluid but also fosters creativity and teamwork by enabling real-time interactions [14,14,15,52]. This kind of shared engagement is invaluable in industries like product design, where rapid iterations and adjustments are essential to the creative process.
Finger and hand tracking also provide significant benefits in professional training and simulation-based learning. In environments where precision and accuracy are crucial, such as aviation or military training, real-time hand tracking allows users to interact with control systems, machinery, or tools in virtual simulations, receiving immediate feedback on their actions. For example, an aircraft mechanic can practice assembling or repairing engines in a virtual environment, with hand tracking capturing the precise movements needed for delicate tasks. The system can provide feedback on the accuracy of each step, ensuring that users improve their skills through repeated practice. This combination of hands-on learning and real-time feedback enhances the effectiveness of training and reduces the risks associated with real-world errors [15,52].
The benefits of hand tracking extend beyond technical skills and training into the realm of social and collaborative XR experiences. By enabling real-time, expressive hand gestures, this technology enhances communication and interaction between users. In virtual meetings, for instance, participants can use gestures such as pointing, waving, or offering a thumbs-up to convey non-verbal cues, adding a layer of richness to virtual communication that text or voice chat alone cannot provide. This is particularly important in remote work environments, where non-verbal communication helps build rapport and trust between team members. The ability to see and interpret hand movements in real time strengthens social presence in the metaverse, making interactions feel more authentic and engaging [53].
In addition to its applications in professional and social contexts, hand tracking holds immense potential for creative industries such as art, gaming, and entertainment. In virtual art studios, for example, artists can manipulate digital brushes or sculpt with virtual clay using their hands, offering new possibilities for creative expression. Similarly, in gaming, hand tracking allows players to perform complex in-game actions, such as wielding weapons or casting spells, by simply moving their hands. This kind of interaction not only enhances immersion but also creates more dynamic and personalized gameplay experiences, as players feel physically connected to their virtual actions [8,54].
As hand tracking technologies continue to evolve, they will likely integrate with other modalities such as haptic feedback to create even more immersive experiences. For example, combining hand tracking with tactile sensations will allow users to not only see and move virtual objects but also "feel" them, adding a new dimension of realism and engagement. This synergy of modalities will deepen the user’s connection with the virtual world, making the manipulation of digital objects more lifelike and enhancing the overall immersive experience in XR environments.
In the next section, we will explore full-body tracking, another critical modality that, when combined with hand and finger tracking, enhances the physical embodiment and realism of XR experiences. By mapping the user’s entire body movement, full-body tracking offers even more comprehensive interaction with the virtual world, enabling activities that range from fitness training to immersive social interactions.
3.4. Full-Body Tracking: Immersive Embodiment and Physicality
Full-body tracking is a powerful modality in XR environments, offering users the ability to fully embody their virtual avatars by capturing and mapping the movement of their entire body. This technology provides a heightened sense of immersion by aligning virtual representations with the user’s real-world movements, significantly enhancing the realism and presence within the virtual world. As users move their bodies in physical space, their virtual avatars replicate those movements, leading to more lifelike and dynamic interactions [55,56]. This capability is especially crucial in social and collaborative XR environments, where non-verbal communication, such as body language and gestures, plays a key role in interpersonal interactions and teamwork.
In social and collaborative XR settings, full-body tracking enables more natural and fluid communication between users. For instance, in virtual meetings or group activities, users can convey emotions and intentions not only through speech but also through body movements and posture. This replication of real-world body language enhances the social presence of avatars, making interactions feel more authentic and personal. In virtual events or online multiplayer games, full-body tracking allows users to dance, gesture, or perform physical tasks together, further strengthening the sense of connection and engagement within the virtual world [57,58].
The integration of full-body tracking into fitness and physical therapy applications has also opened up exciting possibilities for health and wellness in XR environments. In virtual fitness programs, for example, users can perform activities such as aerobics, yoga, or sports, with their movements mirrored in the virtual world. This not only makes the workout more engaging but also allows for real-time feedback on posture, form, and intensity, enabling users to optimize their exercise routines [59,60]. For physical therapists, full-body tracking provides a valuable tool for monitoring patients’ rehabilitation exercises. By tracking precise body movements, the system can assess the quality and range of motion, providing feedback to both the therapist and the patient on the progress of recovery. This real-time feedback helps improve the effectiveness of rehabilitation programs and supports remote therapy, making care more accessible [61].
Moreover, full-body tracking has proven to be a game-changer in immersive learning environments, particularly in educational scenarios that require physical engagement. For instance, students learning about anatomy can walk around a three-dimensional model of the human body, exploring it from all angles. In history or geography lessons, learners can walk through virtual reconstructions of historical sites or geographic landscapes, engaging with the content in a more embodied and hands-on manner [62,63]. This physical exploration of virtual environments supports deeper cognitive engagement and retention, as it allows users to interact with complex information in a multisensory way.
Beyond these individual applications, full-body tracking also enhances the sense of embodiment, a key factor in virtual reality’s impact on user experience. The more accurately a user’s movements are replicated in the virtual world, the stronger their sense of "being there" in that environment. This immersive embodiment fosters deeper engagement, particularly in simulations and training scenarios, where users need to practice real-world skills in a safe and controlled virtual setting. For instance, in military or law enforcement training, full-body tracking can be used to simulate real-world physical environments and challenges, allowing trainees to practice movement-based strategies, teamwork, and decision-making under pressure [64]. Similarly, in sports training, athletes can use full-body tracking to simulate game scenarios or practice specific movements, receiving feedback that helps refine their technique [65].
Looking ahead, as haptic feedback technologies continue to evolve, full-body tracking is expected to integrate with these systems to create even more immersive physical experiences in XR. By combining the ability to track full-body movements with tactile sensations, users will not only see their virtual selves move but also feel the resistance, pressure, or texture of objects they interact with. This will enhance not only entertainment and social applications but also professional training and therapy programs by providing users with more comprehensive and lifelike physical experiences.
In the next section, we will explore haptic feedback, another critical modality that complements full-body tracking by providing tactile sensations that enrich the physical engagement with virtual environments. The integration of these modalities will continue to shape the future of XR, creating environments where users can interact with the digital world in ways that mimic real-world physicality, deepening the sense of immersion and engagement.
3.5. Haptic Feedback: Bringing Touch to the Virtual World
Haptic feedback technology plays a pivotal role in enhancing the immersion and realism of XR by introducing the sense of touch into virtual experiences. Unlike visual and auditory modalities, which primarily stimulate sight and hearing, haptic feedback engages the sense of touch, allowing users to physically feel the virtual objects they interact with. This capability is enabled through various devices, such as haptic gloves, vests, or controllers, which provide tactile sensations when users manipulate digital objects. These sensations can include textures, vibrations, resistance, and even temperature, making virtual interactions feel tangible and lifelike ([9,66].
In training simulations, haptic feedback is invaluable, particularly in fields that require fine motor skills and precision. For instance, in medical training, surgeons can practice complex procedures in virtual environments while receiving tactile feedback that mimics the sensation of handling real surgical instruments or interacting with human tissue. This feedback is essential for developing the dexterity and precision required in real-life operations, and it allows medical professionals to practice in a safe, controlled, and repeatable setting without the risk of harm to patients[11]. Similarly, in mechanical training, haptic feedback enables users to feel the resistance and weight of virtual tools or materials, improving their ability to perform tasks like assembling machinery or repairing engines. By simulating the physical properties of objects, haptic feedback helps users refine their motor skills, making training in virtual environments more effective [11,67].
Beyond its utility in professional training, haptic feedback also enhances emotional engagement in XR environments. The ability to physically feel virtual objects can amplify emotional responses, making interactions more meaningful and immersive. For example, in social XR settings, a virtual handshake or hug can be made more impactful by the sensation of touch, fostering deeper emotional connections between users. Similarly, in gaming, haptic feedback allows players to feel the impact of in-game actions, such as the recoil of a virtual weapon or the texture of a virtual surface, increasing the intensity and realism of the experience [67]. The tactile sensations provided by haptic feedback devices make virtual environments feel more immersive and emotionally engaging, leading to a deeper connection between users and the digital world.
In therapy and rehabilitation, haptic feedback has significant applications, particularly in treatments involving motor recovery and emotional regulation. For example, patients undergoing physical therapy for motor impairments can use haptic devices to practice movements in virtual environments, receiving immediate feedback on their performance. This feedback helps patients adjust their movements in real-time, accelerating the rehabilitation process. In emotional therapy, haptic feedback can be used to recreate comforting physical sensations, such as a soothing touch or a relaxing vibration, to help individuals manage anxiety or stress in virtual environments. By creating a multisensory experience, haptic feedback allows therapy to be more personalized and effective [68].
Furthermore, haptic feedback enhances the realism of virtual simulations, where the sense of touch is crucial for user interaction. In architectural or engineering simulations, users can feel the weight and texture of materials as they construct or manipulate virtual models, making the design process more intuitive and interactive. This tactile engagement allows professionals to experiment with designs in a more hands-on way, improving their understanding of how materials behave in real-world scenarios [52]
Looking ahead, as haptic technology evolves, we are likely to see even more sophisticated devices capable of providing highly detailed tactile sensations, including advanced textures, force feedback, and temperature variations. These advancements will further enhance the realism of XR environments, making virtual objects feel indistinguishable from their real-world counterparts. The integration of haptic feedback with other XR modalities, such as full-body tracking and eye-tracking, will create an even more immersive experience, where users can see, hear, and physically interact with digital worlds in a seamless and natural way.
3.6. Galvanic Skin Response (GSR): Monitoring Emotional States
Galvanic Skin Response (GSR) is a powerful physiological modality that measures changes in the skin’s electrical conductance, which is directly linked to the body’s emotional arousal levels. By detecting fluctuations in the skin’s moisture levels, GSR sensors capture real-time data on emotional states such as stress, excitement, or relaxation. In XR environments, this information can be used to create more adaptive and emotionally responsive experiences, where the virtual environment adjusts in real time to the user’s physiological signals [69].
The integration of GSR in XR environments has significant potential for personalizing user experiences. For instance, in a virtual therapy session designed for stress management, if the system detects elevated stress levels via GSR, it could automatically adjust the environment to promote relaxation. This might involve dimming virtual lights, reducing auditory stimuli, or introducing calming elements like soft music or soothing natural sounds. Such adjustments create a more responsive and immersive therapeutic experience, allowing the virtual environment to better support the user’s emotional needs [70,71]. This adaptability makes GSR a valuable tool for applications in mental health and emotional regulation, where tailoring environments to individual emotional states can enhance treatment effectiveness.
Moreover, GSR is essential in educational and training scenarios, particularly those designed to challenge users’ emotional and cognitive limits. For example, in a virtual simulation designed to prepare users for high-stress scenarios—such as emergency response training or flight simulations—the system can monitor GSR levels to gauge emotional responses. If the user shows signs of becoming overly stressed, the simulation can adjust to offer support, perhaps slowing down the pace of the scenario or providing real-time feedback to help manage emotional regulation [36].
In entertainment and gaming, GSR can also enhance user engagement by modifying gameplay based on emotional responses. For example, a horror game could increase or decrease its intensity based on the player’s emotional arousal, detected through GSR, making the experience more thrilling or manageable depending on the user’s preference. Similarly, virtual reality environments designed for relaxation could track the user’s emotional state and continuously adjust the environment to maintain an optimal level of calmness, ensuring a personalized and immersive experience [71].
Beyond individual experiences, GSR data can also provide insights into group dynamics in collaborative virtual environments. For instance, in team-based training exercises or collaborative design projects, the system could monitor the emotional states of all participants. If collective stress levels rise, the system could offer suggestions to ease tension, such as facilitating a break or introducing calming visual elements into the shared space. This ability to gauge emotional responses at both individual and group levels can enhance collaboration by ensuring that participants remain engaged and focused without becoming overwhelmed by stress [26].
Looking ahead, GSR can be further integrated with other physiological modalities, such as heart rate monitoring and facial tracking, to provide even more comprehensive insights into the user’s emotional and physical state. This multimodal approach will allow XR environments to adapt to a broader range of signals, making interactions more nuanced and responsive to the user’s overall experience. As we transition to a deeper exploration of multimodal systems, combining GSR with other forms of physiological data will be key to unlocking richer, more personalized, and emotionally intelligent XR environments.
3.7. Heart Rate Monitoring: Tracking Physiological Engagement
Heart rate monitoring offers valuable insights into both the emotional and physical states of users during XR experiences. By continuously measuring changes in heart rate, XR systems can assess physiological markers of arousal, excitement, or stress, allowing the virtual environment to respond and adapt in real-time. This form of biometric feedback plays a critical role in personalizing experiences, especially in applications such as virtual fitness, stress management, and emotional engagement.
In virtual fitness programs, heart rate monitoring ensures that users are exercising at optimal intensity levels, contributing to safer and more effective workout sessions. For instance, as users engage in physically demanding activities like virtual running, dancing, or other cardiovascular exercises, the system tracks their heart rate and adjusts the workout intensity based on real-time cardiovascular performance [72]. If the user’s heart rate exceeds a certain threshold, the system may prompt the user to reduce the intensity or take breaks, ensuring that workouts remain within safe parameters. Conversely, if the heart rate is too low, the system may encourage the user to increase intensity for maximum fitness benefits. This dynamic adjustment based on heart rate allows virtual fitness programs to be tailored to individual needs, improving both user experience and exercise outcomes.
Beyond fitness, heart rate monitoring is equally important in stress management and emotional regulation within XR environments. In high-stress virtual simulations—such as those designed for military, emergency response, or flight training—heart rate data can be used to detect elevated stress levels. In these scenarios, the system might offer real-time adjustments, such as reducing the pace of the scenario or introducing calming environmental changes, to help the user manage their emotional state and avoid overwhelming stress responses [70]. This is particularly valuable in training simulations where users must maintain focus and make critical decisions under pressure, ensuring that their performance is not compromised by excessive emotional arousal.
Moreover, heart rate monitoring enhances the immersive emotional response in virtual environments by aligning the virtual experience with the user’s physiological state. For example, in virtual reality therapy or mindfulness applications, the system can monitor heart rate as an indicator of the user’s relaxation or stress levels. If the user’s heart rate begins to increase, indicating rising anxiety or stress, the virtual environment can be modified to create a more calming atmosphere, such as softening background sounds, dimming lights, or slowing the pace of the virtual experience. This adaptive feedback based on heart rate enables a more personalized and emotionally intelligent interaction, enhancing the user’s connection with the virtual environment [36].
In gaming and entertainment, heart rate monitoring can be used to heighten immersion and interactivity. For example, in a horror game, the system could increase or decrease the intensity of jump scares or other frightening elements based on the player’s heart rate. If the system detects that the player’s heart rate is too high, it might dial back the intensity to prevent overwhelming the player, while a low heart rate could prompt the game to become more intense. This responsive approach to entertainment, where physiological data directly impacts the gameplay, enhances the player’s emotional engagement and creates a more personalized gaming experience [70].
Heart rate monitoring in XR can also play a crucial role in collaborative environments, where physiological feedback is shared among participants to enhance group dynamics. For example, in team-based virtual simulations or remote collaboration scenarios, the system can monitor heart rate data across all participants. If a member of the team exhibits signs of stress or anxiety (as indicated by an elevated heart rate), the system could alert other team members or suggest strategies to manage the group’s emotional state, fostering a more supportive and effective collaborative environment.
As XR systems continue to evolve, heart rate monitoring, when combined with other physiological modalities such as GSR and facial tracking, will enable even more sophisticated emotional and cognitive responses. Together, these tools create a holistic view of the user’s physical and emotional state, offering the potential for highly adaptive, personalized XR experiences. This physiological insight also sets the stage for our next exploration into brain activation monitoring, which delve deeper into the user’s cognitive states, offering new dimensions of control and interaction within XR environments.
3.8. Electroencephalogram (EEG): Brain-Computer Interfaces and Cognitive Control
Electroencephalogram (EEG) technology, which measures electrical activity in the brain, serves as the foundation for brain-computer interfaces (BCIs). BCIs offer users the ability to control virtual objects or navigate virtual environments directly through brain activity, bypassing traditional input devices like keyboards or controllers. This technology is particularly transformative in XR, providing new avenues for accessibility, cognitive insights, and personalized interaction. Through the integration of EEG with XR, users can engage more deeply with virtual worlds, transforming how they interact, learn, and perform tasks within the metaverse.
One of the most significant contributions of EEG-based BCIs to XR is in enhancing accessibility. For individuals with physical disabilities or limited mobility, BCIs offer a way to interact with the virtual world using only their thoughts. By analyzing specific brainwave patterns, EEG systems can detect user intentions, allowing them to select, move, or manipulate virtual objects without physical movement. This capability opens up the metaverse and other virtual environments to users who may not be able to engage through conventional means, offering a more inclusive and participatory experience [73,74]. The use of BCIs in XR enhances autonomy and independence, empowering users to perform complex tasks without relying on traditional input mechanisms.
Beyond accessibility, EEG in XR provides real-time cognitive insights by monitoring brainwave patterns associated with different mental states. For example, EEG sensors can track changes in focus, engagement, or fatigue, enabling XR systems to adapt dynamically to the user’s cognitive state. When the system detects that a user’s attention is waning, it can adjust the difficulty, pacing, or complexity of tasks to re-engage the user and optimize their performance [28,75]. This is especially useful in educational and training environments, where maintaining an optimal cognitive load is critical for effective learning. By personalizing the user experience based on real-time cognitive feedback, EEG technology enhances both learning outcomes and user satisfaction [76].
EEG also plays a crucial role in creating adaptive environments that respond to the user’s mental state. By continuously analyzing brainwave patterns, EEG systems can gauge the user’s cognitive load, which reflects the amount of mental effort required to perform a task. In high-stakes virtual simulations—such as those used for military training, medical procedures, or emergency response—the system can use EEG data to detect when the user is becoming mentally overwhelmed and adjust the environment accordingly [77,78]. For example, the system might reduce task complexity, slow down the pace of interactions, or provide additional guidance to help the user manage their cognitive load. Conversely, if the user’s EEG data indicates that they are under-stimulated, the system can introduce more challenging tasks or increase the pace of the simulation to maintain engagement [76]. This ability to tailor experiences to individual cognitive needs enhances both performance and immersion in XR environments.
The integration of EEG-based BCIs with XR technologies also opens the door to novel forms of interaction. In addition to controlling virtual objects, users can interact with the environment in ways that extend beyond traditional physical interfaces. For instance, a user could navigate a virtual landscape by simply focusing on a specific location, or they could interact with virtual characters by directing their attention to specific tasks. This kind of cognitive control offers a more fluid and intuitive way of engaging with the virtual world, eliminating the need for physical input devices and creating a more seamless and immersive experience [73,74].
Furthermore, EEG can be used in therapeutic settings, where real-time monitoring of brain activity can support mental health treatments. In virtual therapy sessions, EEG data can be used to track a user’s emotional and cognitive responses to different virtual scenarios, providing therapists with valuable insights into their mental state. For example, EEG could detect heightened anxiety during a virtual exposure therapy session, allowing the system to adjust the environment to help the user manage their stress levels [27,28,79]. This real-time cognitive feedback can make therapeutic interventions more effective, offering personalized treatments based on the user’s mental state.
As EEG technology continues to advance, it is poised to become an integral part of adaptive and personalized XR environments, offering users the ability to engage with virtual worlds in ways that are tailored to their cognitive and emotional needs. The fusion of EEG and XR is not only transforming accessibility and interaction but also redefining the boundaries of cognitive augmentation, paving the way for more immersive and intuitive XR experiences.
4. XR Applications: Expanding Multimodal Interactions across Domains
XR offers groundbreaking opportunities across a wide range of fields. By integrating multiple modalities—such as eye-tracking, hand-tracking, haptic feedback, EEG, and biometric sensors—XR enhances immersion, interaction, and user performance in virtual environments. These multimodal systems are critical in optimizing the applications of XR, making it an essential tool for clinical interventions, education, professional training, entertainment, and beyond (see Table 2). Below is an exploration of XR’s transformative potential across various fields, highlighting the role of multimodal integration in each context.
4.1. Clinical Applications: Therapy, Neuropsychological Assessment, and Rehabilitation
XR technologies are making significant advancements in clinical applications, ranging from mental health therapy and neuropsychological assessment to physical rehabilitation and pain management. Through the integration of multimodal feedback systems, such as eye-tracking, facial tracking, GSR, EEG, and heart rate monitoring, XR environments can provide real-time responses to a patient’s emotional and physiological states, resulting in personalized therapeutic interventions. This adaptability and personalization position XR technologies as transformative tools in clinical interventions, expanding both mental and physical health care.
4.1.1. Therapy
One of the key clinical applications of XR is VR exposure therapy, a method that has shown great promise in treating anxiety disorders, PTSD, and phobias. Patients are immersed in controlled virtual environments where they can gradually confront and manage their fears in a safe setting, allowing for more controlled and personalized treatment compared to traditional exposure therapies. A recent meta-analysis found that VR exposure therapy significantly reduces symptoms of PTSD and anxiety in a range of clinical populations [80,81].
The integration of multimodal sensors enhances this approach by monitoring the patient’s physiological responses. GSR measures stress through changes in skin conductance, while EEG provides insight into brain activity related to anxiety or relaxation. For instance, if the GSR data shows elevated stress levels during exposure to a feared stimulus (such as heights or spiders), the virtual environment can be automatically adjusted, reducing the intensity of the stimuli to encourage gradual desensitization [36,82,83]. This allows for a more dynamic and responsive form of therapy, where the treatment environment adapts in real time based on the patient’s physiological state.
Furthermore, XR-based cognitive-behavioral therapy (CBT) is increasingly being used for the treatment of depression, addictions, and eating disorders [26,84]. In these applications, XR’s ability to simulate real-world environments allows patients to engage in therapeutic exercises that target maladaptive thoughts and behaviors in controlled, immersive settings. The combination of haptic feedback, eye-tracking, and facial recognition enables deeper emotional engagement, making therapy sessions more interactive and personalized.
4.1.2. Neuropsychological Assessment
XR offers enhanced tools for neuropsychological assessments, allowing for a more accurate and engaging evaluation of cognitive functions such as memory, attention, and executive functioning. Traditional assessments often rely on static paper-based tests, which can be limited in ecological validity. XR technologies provide an interactive alternative, where patients can perform tasks in immersive, dynamic environments, with their performance tracked in real time.
For instance, eye-tracking technology allows clinicians to evaluate how patients interact with stimuli during assessments, such as tracking gaze patterns to measure attention or fixations [85,86,87]. In tasks where memory or decision-making is involved, eye-tracking data can indicate cognitive load, revealing whether a patient is struggling to process information or focusing on irrelevant stimuli. Such detailed data allows for a more nuanced understanding of cognitive impairments, such as attention deficits, compared to traditional methods [85,86,87]
In addition to eye-tracking, EEG data provides insights into cognitive engagement and fatigue. Real-time EEG monitoring can detect cognitive overload, lapses in attention, or difficulty with memory retrieval, helping clinicians adjust the task’s complexity during assessments [79,88,89]. By integrating multimodal feedback systems, clinicians can develop a more detailed and accurate cognitive profile for each patient, facilitating tailored treatment plans.
4.1.3. Rehabilitation
Physical rehabilitation is another area where XR is having a substantial impact. With full-body tracking and haptic feedback, patients recovering from injuries, surgeries, or neurological conditions can perform exercises in virtual environments that track and mirror their movements. For example, stroke patients can practice motor skills in a virtual environment where their movements are tracked in real-time, providing therapists with detailed data to monitor progress [61].
A 2020 review found that VR-based rehabilitation leads to significant improvements in motor function for stroke survivors, particularly when combined with multisensory feedback [57,89,90]. By providing tactile sensations, such as resistance when lifting virtual objects, haptic devices enhance motor learning and skill acquisition. Patients can engage in immersive exercises that mimic real-world tasks, such as picking up objects or walking through a virtual environment, encouraging recovery through realistic and engaging activities.
Beyond motor rehabilitation, XR is also used in cognitive rehabilitation for individuals with traumatic brain injury (TBI) or neurodegenerative diseases such as Alzheimer’s. Virtual environments provide patients with tasks that mimic everyday challenges, such as navigating a virtual store or solving a puzzle, while multimodal sensors monitor their cognitive performance. The system can adjust the difficulty level based on the patient’s responses, creating a personalized rehabilitation plan that supports both cognitive recovery and patient engagement [81,91].
4.1.4. Pain Management
XR is proving to be a powerful tool for pain management, particularly in addressing chronic pain. VR distraction therapy involves immersing patients in virtual environments that divert attention away from pain by engaging them in complex, immersive tasks. The use of multisensory feedback, such as visual, auditory, and tactile inputs, helps to reduce the perception of pain, with patients reporting significant pain relief during immersive VR sessions [92].
Additionally, XR-based virtual mirror therapy has shown promise in treating phantom limb pain in amputees. By creating the visual illusion of movement in the missing limb using full-body tracking, patients can "see" their limb moving in the virtual environment, which helps reduce pain [93,94]. The combination of haptic feedback and EEG monitoring allows clinicians to better understand how pain perception can be modulated through virtual experiences, providing new avenues for non-pharmacological pain treatment.
4.2. Educational Applications: Engaging Learning Through XR
XR technologies are transforming the educational landscape by creating interactive and immersive environments that engage multiple sensory modalities, enhancing both learning and retention. By incorporating eye-tracking, hand-tracking, haptic feedback, and multimodal interfaces, XR brings abstract concepts to life and tailors the learning experience to individual needs. These innovations make learning more engaging and adaptable, promoting active participation and deeper comprehension in a wide range of subjects.
4.2.1. STEM Education
XR environments hold immense potential for STEM education, enabling students to visualize and manipulate complex scientific concepts that are often difficult to grasp through traditional learning methods. In subjects such as biology, physics, and chemistry, XR allows learners to interact with digital representations of biological structures, chemical reactions, or physical simulations. For example, students can explore the human body at a cellular level or perform virtual dissections, offering a more interactive approach to understanding anatomical structures [95].
Hand-tracking technologies facilitate this process by allowing students to manipulate digital models with natural gestures like pinching or grabbing, making learning more intuitive. Similarly, eye-tracking can monitor students’ focus, providing feedback to instructors about which parts of the lesson are capturing attention and which are not, allowing real-time adjustments to optimize engagement [44].
In chemistry, for instance, students can perform virtual experiments by interacting with molecular models or balancing chemical equations in a dynamic environment, ensuring a hands-on learning experience without the risks associated with traditional lab settings. Haptic feedback further enriches the learning process by introducing tactile sensations, enabling students to "feel" chemical bonds breaking or forming, or to experience resistance when interacting with virtual objects, which enhances the understanding of abstract concepts.
4.2.2. History and Humanities
XR also plays a crucial role in enhancing learning in the humanities, particularly in history education. Through AR, historical artifacts and cultural landmarks can be superimposed onto physical spaces, allowing students to explore ancient civilizations or historical events from the classroom. For instance, AR applications can project virtual Roman ruins or medieval castles into the school environment, enabling students to walk through and interact with these reconstructions [29].
With full-body tracking, students can physically engage with historical events by participating in reenactments of key moments in history, such as battles or diplomatic negotiations. This immersive, hands-on experience not only enhances historical knowledge but also fosters empathy and critical thinking by placing students in the shoes of historical figures, encouraging them to understand the decisions and actions taken in different cultural and temporal contexts.
Beyond AR, VR simulations can recreate historical cities or cultural sites, such as ancient Athens or Renaissance Florence, offering students the chance to experience history in a way that traditional textbooks and lectures cannot provide. These multimodal experiences are particularly effective in fostering historical empathy and a deeper connection to past events, as students can explore these environments at their own pace, engaging their senses in a more comprehensive learning experience.
4.2.3. Multimodal Learning
Multimodal learning is one of the most powerful features of XR in education. By integrating EEG, eye-tracking, and haptic feedback, XR systems can create learning environments that dynamically adapt to the cognitive state of the student. For example, when students’ brain activity is monitored through EEG, the system can adjust the lesson’s pacing and complexity to match their cognitive load, enhancing both engagement and retention [75].
For instance, if a student’s brainwave data indicates cognitive fatigue or frustration during a particularly challenging task, the XR system can reduce the difficulty or introduce more engaging elements to maintain focus. Conversely, if the system detects high levels of focus, it may introduce more complex tasks to sustain the student’s engagement. Hand-tracking and haptic feedback further enhance this adaptive system by allowing students to physically interact with digital content, making abstract concepts tangible and easier to comprehend.
In a study by Parong and Mayer (2018) [96], students using XR for biology education showed significantly higher retention rates and engagement compared to those using traditional learning methods. The study emphasized how multimodal interactions, particularly through hand-tracking and eye-tracking, allowed students to manipulate virtual biological systems and understand complex processes more effectively.
4.2.4. Language Learning
XR is also making strides in language learning by providing immersive, contextualized environments where students can practice conversational skills in realistic virtual scenarios. By incorporating facial tracking and GSR, virtual tutors can adapt their tone and responses based on the emotional state of the learner, offering a more empathetic and personalized learning experience. For example, a learner struggling with anxiety during a virtual language conversation can be provided with calming cues or simpler dialogue, easing them into more challenging scenarios [72].
Through full-body tracking, students can engage in virtual role-playing scenarios where they interact with other avatars or digital agents, practicing both verbal and non-verbal communication. These virtual interactions foster deeper language comprehension and cultural awareness, as learners experience language use in real-world contexts, such as ordering food at a restaurant or navigating a new city [24].
4.2.5. Special Education
XR technologies also provide significant benefits for special education. By offering customizable and immersive learning environments, XR can cater to students with a wide range of learning disabilities, including dyslexia, ADHD, and autism. XR systems that integrate EEG and eye-tracking can monitor a student’s cognitive load and focus, adapting the content in real-time to suit their individual needs. For example, students with ADHD may benefit from XR environments that adjust to their focus levels by minimizing distractions and providing more engaging, multimodal tasks [90,97].
For students with autism spectrum disorder (ASD), XR environments can be designed to provide controlled social simulations where they can practice social interactions and communication skills in a low-pressure, customizable environment [98,99]. These virtual experiences can be personalized using GSR and facial tracking to monitor the student’s emotional state, ensuring that the scenarios remain supportive and adaptive to their comfort levels [100].
4.3. Professional Training and Skill Development
XR technologies are transforming professional training and skill development by offering highly immersive, realistic simulations that replicate real-world tasks in a safe, controlled environment. By leveraging multimodal technologies such as hand-tracking, haptic feedback, EEG, and GSR, XR environments allow for hands-on experiences, fostering skill development and retention. The ability to simulate complex scenarios, monitor physiological and cognitive states, and provide real-time feedback makes XR a powerful tool across industries, including healthcare, emergency response, manufacturing, and engineering.
4.3.1. Medical Training
In medical education, XR simulations enable students and trainees to practice complex procedures such as surgery with a high level of realism. Hand-tracking allows trainees to interact with virtual surgical tools, while haptic feedback simulates the tactile sensations of cutting tissue, suturing, or manipulating delicate organs. These immersive environments help develop motor skills that are crucial for real-world surgical procedures. Studies have shown that using XR for surgical training enhances learning and reduces the time it takes to acquire practical skills [30,101].
EEG monitoring provides additional insights into the cognitive load of trainees during surgeries, helping instructors optimize training by adjusting the complexity of tasks based on mental fatigue and focus. This is particularly important in surgeries that demand long periods of sustained concentration. Haptic feedback and EEG integration ensure that the physical and cognitive aspects of learning are addressed, enhancing both skill acquisition and cognitive endurance [102].
4.3.2. Emergency Response
XR has also proven invaluable in training first responders for high-pressure situations such as natural disasters, fires, and mass casualty events. These environments simulate crisis scenarios that allow responders to practice search and rescue operations, medical triage, and decision-making in real time. Full-body tracking ensures that trainees’ physical movements within the virtual environment are realistic, which is essential for learning proper techniques in confined or dangerous spaces [103].
To enhance this training, heart rate monitoring and GSR track the physiological stress responses of trainees, helping instructors understand how well they are managing high-stress situations. Studies show that physiological feedback can improve performance by teaching responders how to control stress in real-world scenarios (Andersen et al., 2018). These technologies make it possible to train for complex, dangerous situations in a way that is both safe and effective, helping to build confidence and competency.
4.3.3. Manufacturing and Engineering
In manufacturing and engineering, XR technologies facilitate virtual training in machinery operation, assembly processes, and maintenance, allowing workers to practice without the risk of damaging equipment or causing injury. Hand-tracking enables trainees to interact with complex machinery in virtual environments, simulating real-world gestures like turning knobs, pulling levers, or using specialized tools. Haptic feedback enhances this experience by providing tactile sensations, such as the resistance of materials or the texture of surfaces, offering a more comprehensive training experience [104].
Eye-tracking is used to monitor where trainees focus their attention during assembly or repair tasks. This information helps instructors identify potential areas where further guidance is needed, improving the effectiveness of the training (Kim et al., 2019). EEG sensors can monitor cognitive load, ensuring that tasks are presented in a way that challenges trainees without overwhelming them. Studies have shown that XR environments can reduce training time and improve retention in industrial settings by providing a more interactive and engaging experience [105].
4.3.4. Aviation Training
In aviation, XR simulations offer a realistic and safe environment for pilots to practice maneuvers, emergency responses, and routine operations. Hand-tracking technology allows pilots to interact with virtual cockpit controls, such as throttle levers, instruments, and flight sticks, while haptic feedback simulates the resistance and forces experienced during flight, such as turbulence or control adjustments. This technology provides pilots with the opportunity to practice handling complex situations without the risks associated with real-life flying [103].
EEG and heart rate monitoring are integrated into these systems to track cognitive and physiological responses during high-stress scenarios like engine failure or extreme weather conditions. This data allows instructors to assess a pilot’s decision-making abilities under pressure and adjust training to improve stress management and focus [103]. XR environments also allow for repetitive practice, improving muscle memory and confidence in critical skills.
4.3.5. Corporate and Soft Skills Training
XR is also making a significant impact in corporate training, particularly for developing soft skills such as communication, leadership, and conflict resolution. In virtual environments, users can engage in realistic simulations of business meetings, negotiations, or customer service scenarios. Facial tracking and eye-tracking technologies allow avatars to mirror users’ facial expressions and body language, making interactions more lifelike. This is particularly useful for leadership training, where participants practice managing teams or handling difficult conversations [47,58,105,106].
For example, in customer service training, XR simulations place users in challenging client interactions, where hand-tracking and full-body tracking allow them to practice gestures and body language in response to virtual customers. EEG data is used to monitor emotional engagement and cognitive focus, ensuring that trainees remain engaged and responsive during interactions [79]. Studies have shown that this immersive approach improves interpersonal communication skills and emotional intelligence, which are critical in customer-facing roles [72].
4.3.6. Military and Defense
In military training, XR simulations are increasingly used to prepare soldiers for combat, strategic operations, and equipment handling. By integrating full-body tracking and haptic feedback, soldiers can engage in simulated battlefield environments where they practice maneuvers, weapon handling, and tactical coordination. The tactile sensations of haptic devices help replicate the experience of firing weapons or feeling explosions, enhancing the realism of the training [11,64].
Heart rate monitoring and GSR data track soldiers’ stress levels during these simulations, helping instructors gauge their ability to manage fear and anxiety in high-pressure combat scenarios. This data provides valuable insights into how soldiers perform under stress, allowing for targeted feedback and the development of coping strategies [72,79]. Research shows that XR military simulations significantly improve readiness and performance in real-world combat situations [64,102].
4.4. Arts and Entertainment: Immersive and Interactive Creativity
In the realm of arts and entertainment, XR is enabling new forms of creative expression by integrating multimodal interactions that engage the user on multiple sensory levels. From interactive gaming to virtual galleries, XR offers immersive experiences that merge the physical and digital worlds.
Gaming: Multimodal technologies such as facial tracking, hand-tracking, and haptic feedback allow gamers to experience more realistic and emotionally engaging gameplay. Facial tracking replicates players’ facial expressions on their avatars, enhancing social interaction and emotional immersion in multiplayer environments. Haptic feedback makes virtual actions more tangible, whether players are feeling the impact of an explosion or the sensation of handling objects training [11].
Virtual Art Galleries: XR allows artists to create and display virtual art installations that can be explored by users in fully immersive environments. Hand-tracking enables visitors to interact with artworks, while eye-tracking provides curators with insights into how audiences engage with different pieces [8,41]. Haptic feedback can even simulate the texture of virtual sculptures or paintings, adding a tactile dimension to the art experience [9,10,66].
4.5. Public Health and Safety Training
XR has the potential to revolutionize public health campaigns and safety training, providing an interactive and immersive way to educate the public and professionals about critical safety practices. By integrating multimodal technologies such as eye-tracking, hand-tracking, and haptic feedback, XR provides a comprehensive tool for training professionals and educating the public on critical health and safety issues. These technologies enable simulations that mimic real-world emergency and public health situations, allowing participants to experience scenarios in a controlled yet engaging manner, improving both retention and practical application of learned skills.
4.5.1. Disaster Preparedness
Disaster preparedness is an essential area where XR is proving highly effective. First responders, firefighters, and other emergency personnel can train in virtual disaster environments that simulate scenarios such as earthquakes, floods, or building collapses. These virtual environments are designed to replicate the chaos and unpredictability of real-life disasters, offering professionals the opportunity to practice making quick, informed decisions under pressure.
Full-body tracking plays a critical role in these scenarios by enabling realistic movement through virtual spaces. Trainees can navigate complex environments, such as collapsed buildings or hazardous areas, while haptic feedback provides tactile responses, allowing them to feel the weight of objects like fire hoses, rubble, or tools during search and rescue operations [9,10,66]. The integration of eye-tracking can further assess where trainees focus their attention during an emergency, offering insights into how they prioritize tasks, which can be used to optimize future training programs [41,46].
XR also facilitates collaborative training by enabling multiple participants to interact in the same virtual environment. This capability is especially valuable for team-based emergency responses, where coordination and communication are critical. Real-time facial tracking can capture stress or confusion, offering instructors valuable data to refine their approach to team-based emergency management training[47,105]. Through realistic and immersive training, XR can prepare professionals for a wide range of disaster scenarios, improving readiness and response capabilities [107].
4.5.2. Public Health Campaigns
XR can also be applied to public health campaigns, providing an interactive and engaging way to educate the general public about important health and safety protocols. These virtual environments allow individuals to experience critical health scenarios firsthand, such as responding to pandemics, natural disasters, or hazardous chemical spills. In such scenarios, participants can walk through the necessary steps for containment, protection, or emergency evacuation, all within a safe, simulated environment.
By using facial tracking and eye-tracking, XR platforms can monitor emotional engagement, ensuring that users remain focused and that the training resonates with them on an emotional level [29,41,47]. These features allow for personalized feedback and adaptation during the experience—if a user shows signs of stress or confusion, the system can alter the pacing or complexity of the simulation to ensure the message is effectively communicated.
For instance, during a virtual pandemic response simulation, eye-tracking data can reveal which areas of information (e.g., proper handwashing techniques or mask usage) capture the most attention, allowing public health authorities to focus campaigns more effectively [46]. Similarly, GSR or heart rate monitoring can track physiological responses to stressful scenarios, such as learning about quarantine measures or medical procedures during an outbreak, enabling authorities to design public health messaging that better addresses common anxieties or misconceptions [72,79].
4.5.3. Workplace Safety and Compliance
XR environments also provide interactive workplace safety training for industries such as construction, manufacturing, and chemical processing, where hazards are prevalent, and proper safety measures are crucial. Through multimodal integration, such as hand-tracking and haptic feedback, trainees can practice handling machinery or dangerous materials safely [8,10].
For example, a trainee in a virtual construction site may learn to operate heavy machinery using hand-tracking, while haptic feedback replicates the tactile sensations of interacting with physical controls [108]. These simulations help workers develop muscle memory and gain confidence before operating machinery in real life. Additionally, full-body tracking can simulate realistic physical movements in hazardous environments, such as navigating scaffolding or confined spaces, ensuring that workers are fully prepared to adhere to safety protocols [64,109].
In chemical processing plants, XR can simulate hazardous material spills, guiding employees through containment and cleanup procedures without exposing them to actual danger. Eye-tracking data can be used to assess whether trainees are properly following protocols, while GSR data can measure stress responses to simulate real-world urgency and pressure [46,79].
4.6. Retail and E-Commerce
XR technologies are reshaping the retail landscape by providing immersive, interactive shopping experiences that bridge the gap between physical and online stores. Through the integration of hand-tracking, eye-tracking, haptic feedback, and full-body tracking, XR environments allow users to explore virtual shopping spaces, interact with products in more intuitive ways, and receive personalized, data-driven experiences.
4.6.1. Virtual Shopping
Virtual shopping environments in XR allow customers to explore virtual stores where they can interact with products as if they were physically present. Instead of relying on static images, shoppers can manipulate 3D product models using hand-tracking to rotate items for a 360-degree view or zoom in to inspect details like texture or design. This enhanced interaction provides a more engaging shopping experience, which is especially valuable for products that customers typically prefer to see and touch before purchasing, such as electronics, furniture, or clothing [110].
In virtual clothing stores, full-body tracking enables customers to "try on" outfits using realistic avatars that mirror their movements. This allows customers to visualize how clothing items fit and move on their body, reducing uncertainty about size and appearance before making a purchase. Eye-tracking technology provides retailers with valuable insights into customer behavior, helping them understand which products or parts of the store layout attract the most attention, enabling data-driven optimization of store designs and product placements [32].
4.6.2. Product Customization
XR also enables AR-enhanced retail applications, allowing customers to customize products directly within their own physical environments. For example, AR apps for furniture retailers allow users to visualize virtual furniture in their real-world space, adjusting color, size, or style to see how it fits into their home decor. Haptic feedback further enriches this experience by providing tactile sensations that simulate the texture or weight of virtual items, making the interaction more tangible and realistic [111].
Similarly, in industries like fashion or automotive, users can interact with virtual models of products to customize details such as fabric, color, or features. These virtual models, combined with real-time customization tools, allow customers to create tailored products and preview them in their intended settings, increasing their confidence in purchasing decisions. This personalized approach is key in promoting customer satisfaction and reducing return rates, which is a common challenge in online retail [112].
4.6.3. Enhanced Customer Insights
Beyond improving the user experience, XR technologies like eye-tracking offer significant benefits for retailers by collecting detailed behavioral data. This data provides insights into how customers interact with the store environment and products, revealing patterns in attention, interest, and decision-making. Retailers can use this information to personalize the shopping experience further, adapting product recommendations, store layouts, and marketing strategies based on customer preferences [112].
For example, eye-tracking can reveal which products capture the most attention and how long a customer focuses on certain items. This information enables retailers to optimize product placements or highlight specific features that resonate with customers. The integration of biometric feedback such as GSR can also provide deeper insights into customers’ emotional responses to products, allowing retailers to tailor their messaging and user interface to match the emotional engagement of shoppers [113].
4.7. Architecture and Urban Planning
XR technologies provide architects, urban planners, and construction professionals with the ability to visualize and interact with virtual models of buildings or urban landscapes before they are constructed. These technologies, integrating hand-tracking, haptic feedback, eye-tracking, and full-body tracking, provide architects, urban planners, and construction professionals with more efficient and dynamic ways to design, collaborate, and present their work.
4.7.1. Virtual Walkthroughs
One of the most transformative applications of XR in architecture is the ability to conduct virtual walkthroughs of buildings or urban projects using MR [114].. These virtual walkthroughs allow architects to place digital models of proposed designs directly into physical spaces, giving stakeholders the opportunity to experience the building before it is constructed. Clients can explore the layout, lighting, and spatial arrangement in a way that goes far beyond traditional blueprints or 3D renderings.
Through eye-tracking and facial tracking, architects can gather valuable feedback on client reactions, such as which areas of the design are drawing the most attention or eliciting emotional responses [114]. This data helps architects refine their designs to meet client expectations, creating spaces that are not only functional but also emotionally resonant. For example, if clients spend more time focusing on particular features like open spaces or natural lighting, architects can emphasize or adjust these aspects accordingly.
4.7.2. Collaborative Design
In addition to immersive visualization, XR enhances collaborative design processes, allowing multiple professionals to work together on the same project from different physical locations. Through hand-tracking and full-body tracking, architects and engineers can manipulate 3D models in real-time, using intuitive gestures to adjust designs, place virtual tools, or test structural components [33]. This ability to interact with digital models in a shared virtual space fosters more efficient collaboration and innovation, especially in large-scale urban planning projects that require input from various disciplines.
For example, urban planners can simulate how new buildings will interact with existing infrastructure, analyzing factors such as traffic flow, environmental impact, and public accessibility. With haptic feedback, users can feel the texture and resistance of building materials, providing a tactile layer to the design process [10]. This is particularly valuable in construction, where realistic feedback on materials can help professionals plan how different elements will come together in the real world.
4.7.3. Infrastructure and Sustainability Planning
XR also plays a significant role in infrastructure development and sustainability planning. Architects and urban planners can simulate energy efficiency, material use, and environmental impact before construction begins. By incorporating sensor data into these simulations, they can predict how buildings will perform in various conditions, such as extreme weather events, optimizing designs for sustainability and resilience [115].
Through AR overlays, urban planners can visualize proposed infrastructure changes, such as new roads, bridges, or public transit systems, in real-world environments, allowing for better community engagement and feedback [116]. For instance, residents can use AR applications to see how new developments will affect their neighborhood, giving them the opportunity to provide input before construction starts.
4.7.1. Safety and Risk Assessment
In architecture and construction, safety is a critical concern, and XR provides valuable tools for conducting safety and risk assessments. By creating virtual simulations of construction sites, professionals can identify potential hazards, test safety protocols, and train workers in a controlled environment [108]. For example, full-body tracking can simulate how workers will navigate a site, while haptic feedback allows them to feel the weight and resistance of safety gear or equipment [11,14]. These simulations help prevent accidents and ensure that safety measures are fully understood and implemented before workers enter a real-world site.
4.8. Agriculture and Environmental Monitoring
XR, combined with multimodal interfaces like eye-tracking, hand-tracking, and GSR, offers new opportunities for agriculture and environmental monitoring. These technologies provide immersive data visualization and real-time simulations for better decision-making [115,117].
4.8.1. Agriculture Training
XR is becoming an essential tool for modern agriculture training, where farmers and agricultural professionals can simulate complex scenarios, such as sustainable farming practices, pest management, and crop rotation. Through AR simulations, users can engage with virtual models of crops, soil structures, and machinery, interacting directly with these elements through hand-tracking. This form of engagement helps farmers and agricultural trainees practice techniques without the need for physical resources, reducing costs and minimizing the environmental impact of training programs [117,118].
Additionally, XR offers a deeper understanding of how different environmental factors, such as water use, soil health, and plant nutrition, affect agricultural productivity. For example, trainees can practice applying fertilizers and pesticides in virtual environments, learning how to use these inputs more efficiently and sustainably. Through GSR, instructors can track user engagement, providing real-time feedback and adjusting the difficulty or complexity of tasks to optimize the learning experience [115]. This integration of biometric feedback with XR-based training ensures that learners remain engaged, making the training sessions more effective and personalized.
4.8.2. Environmental Simulations
In the realm of environmental monitoring, XR offers powerful tools for visualizing and simulating the impact of climate change, pollution, deforestation, and other environmental challenges. By combining VR and AR technologies, professionals working in urban planning or environmental science can immerse themselves in virtual projections of cities, forests, or ecosystems affected by climate change. These virtual simulations allow for better analysis of long-term environmental consequences, such as rising sea levels, deforestation, or urban sprawl, and help stakeholders explore potential solutions [119,120].
For example, city planners can use full-body tracking to walk through projected urban landscapes affected by flooding or air pollution, visualizing how different infrastructure decisions might impact environmental resilience. Eye-tracking provides insights into which aspects of the virtual simulation capture attention, while GSR measures users’ emotional responses, helping planners understand how they engage with pressing environmental issues [117,119]. This multimodal integration ensures that users not only see the projected impacts but also interact meaningfully with the data, leading to more informed and emotionally engaged decision-making.
4.8.3. Precision Agriculture and Monitoring
Precision agriculture, which relies on data-driven approaches to optimize crop management, can also benefit from XR technologies. By visualizing large datasets on soil conditions, weather patterns, and crop health through immersive simulations, farmers can make more informed decisions about planting, irrigation, and harvesting. Hand-tracking and haptic feedback enhance the user’s ability to manipulate and explore data in real-time, providing tactile feedback when interacting with virtual models of crop fields or irrigation systems [115].
Moreover, AR applications allow users to overlay real-time data from sensors placed in the field, such as soil moisture or crop health, directly onto physical landscapes. This integration of AR with sensor data helps farmers monitor crops more effectively, respond to potential issues faster, and reduce resource wastage by applying water or fertilizers only where needed. By offering these tools in a hands-on, interactive way, XR promotes more sustainable and efficient farming practices [117].
4.8.4. Conservation and Ecosystem Management
Beyond agriculture, XR technologies are being employed in conservation efforts and ecosystem management. For instance, environmental scientists can use XR simulations to model the effects of deforestation on biodiversity or to visualize the outcomes of conservation projects. Through full-body tracking, users can navigate through projected forest ecosystems or coral reefs, examining the potential impacts of deforestation or pollution on wildlife populations [115,121].
Conservationists can also simulate different intervention strategies in XR environments to explore how various actions, such as reforestation or protected areas, might mitigate environmental degradation. Haptic feedback can simulate the tactile feel of different ecosystems, allowing scientists to engage more deeply with environmental changes [117]. These simulations not only enhance understanding of environmental issues but also help policymakers visualize long-term environmental planning scenarios and engage the public in conservation efforts [121].
4.9. Sports Training and Performance Analysis
XR technologies are increasingly utilized in the sports industry, providing athletes and coaches with immersive tools for performance analysis and skill development. By integrating modalities such as full-body tracking, heart rate monitoring, GSR, and EEG, athletes can receive personalized training experiences. These technologies offer real-time feedback, helping to refine techniques, improve mental focus, and optimize both physical and cognitive performance [78,122].
4.9.1. Skill Development
In XR sports training, athletes can engage in realistic simulations of their sport, allowing them to practice specific skills such as a tennis serve, golf swing, or soccer kick. Hand-tracking and haptic feedback are critical for providing realistic sensations during these exercises. For instance, when practicing a tennis serve in a virtual environment, haptic feedback can replicate the tension of the racket strings and the impact of the ball, thereby helping athletes to improve their precision and consistency [10]. Studies have shown that haptic feedback significantly enhances motor skill development by enabling athletes to "feel" the interaction between themselves and virtual objects, thus creating more naturalistic and effective training environments [8,9].
Moreover, full-body tracking plays a vital role in capturing every aspect of the athlete’s movement [122]. This allows coaches and trainers to analyze motion in great detail, identifying areas for improvement. For instance, motion capture data can be used to identify inefficiencies in an athlete’s technique, such as improper body alignment during a sprint or incorrect posture during a weightlifting movement. By providing instant feedback on movement patterns, XR enables athletes to make real-time adjustments, which accelerates skill acquisition and minimizes the risk of injury [78].
4.9.2. Mental Training
Beyond physical skills, mental training is crucial in sports, particularly in high-pressure situations. XR tools that incorporate GSR, EEG, and eye-tracking technologies help monitor an athlete’s cognitive load and stress levels during training. For instance, GSR sensors detect physiological arousal associated with stress, enabling coaches to understand how athletes respond to intense moments during training sessions or simulated competitions [74]. In sports such as archery, tennis, or golf, where focus and emotional regulation are critical, coaches can use this data to introduce mental conditioning techniques, such as mindfulness exercises or breathing techniques, to help athletes manage their stress more effectively [122].
Additionally, EEG technology is employed to monitor brain activity, which offers insights into an athlete’s level of focus, fatigue, and mental engagement [78]. For instance, athletes training for sports like Formula 1 racing or esports—where reaction time and mental acuity are vital—can benefit from EEG-based feedback systems that track how well they concentrate under high-speed, high-pressure conditions. This data can be used to adjust training strategies, ensuring athletes are mentally sharp when it matters most.
Eye-tracking technology is also highly effective in analyzing athletes’ visual attention and decision-making processes. In team sports, such as soccer or basketball, where spatial awareness and quick decision-making are essential, eye-tracking systems can analyze where players focus their attention during gameplay. For example, an eye-tracking study in soccer might reveal that an athlete is fixating too long on the ball rather than scanning the field for passing options, leading to improved training drills that emphasize peripheral awareness and faster decision-making [50].
4.9.3. Performance Analysis
XR systems not only help athletes improve their skills but also serve as powerful tools for performance analysis. By combining full-body tracking, heart rate monitoring, and biometric data, XR systems can provide a comprehensive understanding of how athletes perform in real-time [122]. For instance, heart rate variability (HRV) can be used to measure an athlete’s physiological response to stress, giving insights into their cardiovascular health and recovery capacity. Coaches can use this data to optimize training regimens by ensuring athletes are exercising at the right intensity levels, adjusting workloads to prevent overtraining and maximize performance.
In endurance sports like cycling or running, XR systems can simulate race conditions, including terrain, wind resistance, and competition, allowing athletes to practice in highly controlled environments. EEG and GSR technologies track mental and emotional engagement during these simulations, ensuring that athletes are not only physically but also mentally prepared for competitions. This holistic approach to training ensures that athletes can refine their technical skills while also enhancing their mental toughness and emotional resilience [74].
4.10. Remote Work and Virtual Offices
As remote work becomes increasingly widespread, XR technologies are reshaping how businesses operate, offering immersive, interactive virtual office spaces. Through the integration of multimodal interfaces—including eye-tracking, hand-tracking, facial tracking, and full-body tracking—virtual offices now provide a high level of engagement, collaboration, and realism [123]. These innovations bridge the gap between physical office spaces and remote work, creating an environment where employees can collaborate as if they were in the same room, regardless of their physical location.
4.10.1. Virtual Meetings
In XR-enabled virtual offices, meetings feel more interactive and natural than traditional video conferencing. Facial tracking allows avatars to replicate the subtle nuances of facial expressions, capturing emotions and enhancing communication [50]. Hand-tracking enables users to manipulate digital objects, making brainstorming sessions more interactive by allowing participants to move virtual sticky notes or interact with 3D models [30,111].
Eye-tracking offers another layer of engagement by allowing facilitators to monitor attention levels and adjust their presentations in real time. For example, in a virtual meeting, the system can track where participants are looking and provide feedback to the speaker on which visual elements are capturing attention, ensuring that presentations remain engaging [41]. Full-body tracking further adds to the immersive experience, allowing users to navigate virtual environments and interact with virtual objects more naturally [104,123].
4.10.2. Remote Collaboration
Beyond meetings, XR offers extensive tools for remote collaboration, particularly in industries requiring detailed interaction with 3D models, such as engineering, architecture, and marketing. Hand-tracking allows team members to manipulate virtual prototypes or CAD models in real time, while haptic feedback provides tactile sensations that mimic the feel of physical materials [10,30]. This technology allows remote collaborators to work on the same virtual project, make adjustments, and provide feedback as though they were physically present together.
Full-body tracking enhances collaborative experiences, allowing users to move freely in the virtual space, which is particularly useful for large-scale projects like building designs or landscape architecture [33]. Additionally, EEG and GSR sensors monitor participants’ cognitive and emotional states, ensuring that workloads and tasks are adjusted to optimize performance [28]. For instance, in high-stress tasks, such as emergency response training, GSR data can prompt instructors to introduce relaxation exercises or modify the task to reduce stress [70].
4.10.3. Training and Onboarding
XR is revolutionizing training and onboarding processes in virtual offices by providing immersive, hands-on learning experiences. New employees can participate in virtual orientation sessions, explore digital replicas of office spaces, and interact with company resources using hand-tracking and eye-tracking technologies [111,123]. Trainers can monitor cognitive engagement through EEG and adjust the training in real time, ensuring that trainees are neither under- nor over-stimulated [75]
Moreover, virtual office tours allow new hires to navigate the workspace and familiarize themselves with team members and company culture, making the transition smoother even when they are not physically present in the office [111]. This virtual onboarding reduces the time and resources traditionally spent on in-person orientations and helps employees feel more connected to the organization from the outset.
4.10.4. Future of Remote Work with XR
As XR technologies evolve, remote work and virtual offices will continue to offer immersive, high-performance environments that replicate and, in some cases, improve upon physical office spaces. Multimodal interfaces—such as haptic feedback, facial tracking, and EEG monitoring—will enhance remote collaboration, allowing for deeper engagement, more intuitive interaction, and improved team dynamics [5,50]. AI-driven XR environments will further personalize the user experience by adjusting to users’ cognitive and emotional states in real-time, ensuring optimal performance and engagement [5].
The seamless integration of physical and virtual workspaces will enable organizations to operate more effectively on a global scale, overcoming the barriers posed by geographical distance and time zones. XR will become a cornerstone of the future workplace, making remote work more accessible, inclusive, and productive. [123]
4.11. Museums and Cultural Heritage Preservation
XR technologies offer groundbreaking solutions for the preservation of cultural heritage and the creation of immersive museum experiences. By integrating multimodal interaction—such as eye-tracking, hand-tracking, and haptic feedback—museums can create more interactive, educational, and engaging experiences for visitors. Moreover, these technologies help preserve and showcase cultural artifacts and sites that may be inaccessible or endangered, allowing broader audiences to appreciate and learn about history and heritage [63,121].
4.11.1. Virtual Museums
Virtual reality enables museums to go beyond traditional displays by creating immersive exhibits where visitors can explore digitized artifacts in ways previously unimaginable. For example, visitors to a virtual museum can use hand-tracking to "pick up" and manipulate 3D replicas of ancient relics, providing a more intimate and interactive experience. This method allows users to inspect objects from all angles, gaining a deeper understanding of the artifact’s craftsmanship and details. Haptic feedback further enhances this experience by simulating the texture, weight, or material of these virtual objects, giving users the sensation of holding these artifacts in their hands, thus bridging the gap between physical and digital engagement [124,125].
This multimodal interaction can significantly enrich educational experiences. For instance, a visitor in a virtual museum could use eye-tracking to interact with dynamic content that adapts based on their focus, offering personalized information or guided tours depending on where they direct their gaze. Such features cater to the user’s learning pace and interests, promoting deeper engagement with the exhibit [63,125].
4.11.2. Cultural Heritage Preservation
In the domain of cultural heritage preservation, XR technologies are being used to create detailed digital records of endangered or inaccessible cultural sites. Through 3D scanning and reconstruction, museums and cultural institutions can offer virtual tours of sites that are either too remote, fragile, or in danger of being lost to time. For example, endangered cultural sites, such as ancient temples or archaeological ruins, can be digitally preserved and reconstructed using XR, providing a detailed and immersive exploration of these landmarks for future generations [62,121,126].
Visitors can walk through digitally reconstructed environments using full-body tracking, exploring ancient cities, pyramids, or even World Heritage sites as they once appeared in their prime. Eye-tracking can be employed to gather data on which aspects of the environment capture the most attention, allowing curators to better understand how visitors engage with these reconstructions. This data can then be used to further refine and enhance the educational value of these digital exhibits [63,126].
Moreover, XR can play a critical role in reconstructing destroyed or damaged cultural heritage. For example, following the destruction of sites due to war, natural disasters, or environmental factors, XR tools can assist in virtually rebuilding these locations, allowing historians, archaeologists, and the public to virtually explore and study them. In many cases, the digital reconstruction may be the only way to experience such sites in the future [121].
4.11.3. Multimodal Engagement in Museums
XR’s multimodal interaction capabilities offer innovative ways for museums to engage with diverse audiences. With hand-tracking, visitors can interact more intuitively with virtual displays, while haptic feedback creates a sensory connection to the digital content, making it feel more tangible and real. Eye-tracking technology enhances educational engagement by identifying the most compelling parts of an exhibit, providing curators with valuable insights into visitor behavior, while simultaneously creating adaptive and personalized tour experiences [127].
The ability of XR to merge physical and digital experiences offers museums the potential to redefine how visitors engage with cultural heritage. The fusion of these multimodal technologies ensures that cultural knowledge is not only preserved but also made more accessible, interactive, and meaningful to future generations.
5. Potential Risks and Ethical Challenges of XR and the Metaverse
While the potential applications of XR technologies and the Metaverse offer groundbreaking advancements in education, healthcare, professional training, and entertainment, their rapid evolution also introduces a range of risks and ethical challenges (see Table 3). As users increasingly integrate their cognitive and physical selves with immersive digital environments, new concerns emerge regarding privacy, safety, and well-being. These technologies, while transformative, pose risks such as cybersickness, addiction, and cyber harassment, alongside deeper concerns about the collection and use of sensitive biometric, behavioral, and emotional data. As XR and the Metaverse continue to evolve, it becomes crucial to address these ethical challenges and ensure responsible development, balancing the immense opportunities with the potential dangers they bring.
5.1. Cybersickness
Cybersickness, a form of motion sickness experienced in XR environments, is a significant challenge for virtual and immersive technologies, especially in VR. The condition occurs due to a sensory conflict between the user’s visual perception of movement and the absence of corresponding physical movement. This mismatch between what the brain expects (based on visual input) and what the body senses (physical stillness) can lead to various symptoms such as dizziness, nausea, headaches, sweating, and disorientation. These symptoms are collectively referred to as cybersickness [128,129].
The risks associated with cybersickness are multifaceted, particularly in professional, educational, and recreational contexts. The most immediate impact is physical discomfort, where users experience dizziness, nausea, and headaches. This discomfort can severely affect the user’s ability to engage with XR environments, limiting the amount of time they can spend in VR or other immersive settings. For instance, users undergoing virtual training for extended periods may struggle with maintaining focus or motivation due to the physical discomfort associated with cybersickness [130,131,132].
In addition to physical discomfort, cybersickness can reduce cognitive performance and productivity in XR applications. In professional settings, such as medical training or virtual collaboration, the symptoms of cybersickness may result in decreased attention, slower reaction times, and poorer decision-making. This is particularly problematic in scenarios that require precise interaction with digital tools or intense cognitive engagement. For example, in a medical training simulation, a surgeon experiencing cybersickness may find it difficult to complete complex tasks, potentially undermining the effectiveness of the training [133].
Cybersickness is also a barrier to broader adoption of XR technologies. If users associate XR experiences with discomfort, they are less likely to engage with the technology frequently or for extended periods. This is particularly concerning in educational or workplace settings, where long-term immersion is often required for learning and collaboration [131,132,134]. The variation in user susceptibility to cybersickness further complicates the issue, as some individuals may experience mild symptoms, while others may be completely unable to use VR technology without significant discomfort [43,85,129,135].
5.1.1. Prevalence and Contributing Factors
Research has shown that a significant percentage of VR users experience cybersickness, with prevalence rates varying between 20% to 80%, depending on the type of VR system, the content being viewed, and individual sensitivity [43,128,129,131,132]. Factors influencing cybersickness include the frame rate and latency of the XR system, the complexity of the virtual environment, and the type of movement depicted in the virtual world. For example, fast-paced or erratic motion within a virtual environment is more likely to induce cybersickness than environments with slower, more controlled movements.
In addition to technological factors, individual differences also play a role in susceptibility to cybersickness. Research indicates that younger users, those prone to motion sickness in real life, and individuals with a history of vestibular disorders are more likely to experience severe symptoms. This variability in how users react to XR environments makes it difficult to develop one-size-fits-all solutions to mitigate cybersickness.
5.1.2. Technological and Design Solutions
To address cybersickness, various technological and design solutions have been proposed. One of the most effective strategies is improving the performance of XR systems by increasing the frame rate and reducing latency, which ensures smoother and more stable visual experiences. A higher frame rate helps reduce the sensory conflict that causes cybersickness by ensuring that virtual movements are closely synchronized with real-world head and body movements [18,131].
Another approach involves optimizing the design of virtual environments to reduce the likelihood of inducing cybersickness. This can include limiting rapid movements, avoiding complex or disorienting camera angles, and designing smoother transitions between different virtual spaces [18,128,131,134]. Additionally, implementing "comfort modes" that slow down the pace of movement or reduce the field of view during fast motion can help minimize symptoms [18,129,131].
Real-time physiological monitoring also offers promise in managing cybersickness. XR systems equipped with multimodal sensors, such as eye-tracking and heart rate monitoring, can detect early signs of discomfort and adjust the virtual experience accordingly. For example, if a system detects that a user’s heart rate is increasing due to motion-induced discomfort, it could automatically slow down the motion or modify the environment to reduce symptoms [130,131]..
5.2. Addiction
The immersive and highly engaging nature of XR technologies and the metaverse presents significant risks of addiction. As virtual environments become increasingly advanced and realistic, users may become overly attached to their virtual lives, games, or social interactions. This engagement can lead to compulsive behaviors similar to those observed in internet and gaming addiction [136,137]. The capacity of XR to provide endless opportunities for exploration, achievement, and social connection can foster an unhealthy reliance on these digital spaces, causing users to prioritize virtual experiences over real-world responsibilities [138,139].
One of the primary risks associated with XR addiction is psychological dependence on virtual environments. As users become deeply invested in their avatars, achievements, or virtual relationships, they may begin to neglect important aspects of their real lives, such as work, education, or personal relationships [140]. This neglect can result in poor academic or job performance, strained familial and social connections, and an overall diminished sense of real-world fulfillment [141]. The immersive nature of XR environments blurs the line between digital and physical spaces, intensifying this dependence.
Another significant concern is the disconnection from real-world social relationships. Users who spend more time interacting with others in virtual environments may reduce their engagement with friends and family in the physical world. While virtual interactions can be stimulating and socially fulfilling to an extent, they often lack the depth and intimacy of face-to-face communication [142]. This social disconnection can exacerbate feelings of loneliness and isolation, which is particularly concerning for vulnerable populations such as adolescents or individuals with preexisting mental health conditions.
A related risk is the potential for "escape behavior," where users turn to the metaverse to avoid confronting real-world problems, such as stress, relationship difficulties, or mental health issues. Instead of addressing these challenges, users may seek solace in the metaverse, exacerbating anxiety, depression, or other psychological issues [139,142,143]. This form of digital escapism mirrors the patterns seen in gaming addiction, where virtual worlds offer an appealing means to avoid real-life difficulties but ultimately worsen mental health. Studies have shown that individuals who use virtual environments to escape reality are more likely to experience mental health challenges, such as increased anxiety and depressive symptoms [144].
The gaming industry, a key component of the metaverse, is particularly prone to fostering addictive behaviors. Many virtual environments and games rely on reward systems, achievements, and social interactions to keep users engaged for extended periods. These game mechanics can become highly addictive, leading to compulsive gaming behaviors that closely resemble those seen in substance-related addictions [142,145]. Features such as loot boxes, in-game currencies, and social competition incentivize users to stay online for long periods, reinforcing addictive patterns [137,146].
5.2.1. Mitigation Strategies
To address the risks of addiction in XR environments, developers and policymakers must implement ethical design principles that discourage excessive use. Designing XR experiences that prioritize balance, implement usage limits, or incorporate "break reminders" can help prevent users from becoming too immersed for unhealthy periods of time. Research suggests that these breaks can help reduce cognitive overload and prevent compulsive behaviors [143]. Additionally, encouraging developers to create content that promotes healthy habits—such as setting limits on playtime or integrating wellness reminders—can mitigate some of the addictive tendencies present in XR environments.
Education and awareness programs about the risks of XR addiction should also be developed to inform the public about responsible use and the importance of maintaining balance between virtual and real-world activities. These programs can target schools, workplaces, and the general population to raise awareness of the potential for addiction and provide strategies to mitigate its effects [138,139]. Moreover, parental controls and age-appropriate content regulations should be implemented to protect younger users, who are particularly vulnerable to the addictive qualities of immersive digital spaces [141].
5.3. Cyber Harassment and Cyberbullying
Cyber harassment and cyberbullying are growing concerns in XR and metaverse environments, particularly in virtual social spaces where users interact through avatars. The perceived anonymity and lack of real-world consequences in these immersive environments foster conditions conducive to harmful behavior. This reduced accountability, coupled with the decentralized nature of many virtual spaces, amplifies the risks of cyber harassment and bullying, making them pressing issues for the future of virtual interactions [147,148]. As XR technologies become more widely adopted, addressing these issues is critical to ensuring the safety and inclusivity of these virtual environments.
5.3.1. Virtual Harassment and Emotional Distress
In XR environments, users are vulnerable to various forms of harassment, including virtual groping, unwanted interactions, and verbal abuse. The immersive nature of XR technology exacerbates these issues, as users often experience such violations as highly personal and immediate [149,150]. This emotional intensity arises because avatars in virtual environments mirror users’ physical movements and actions, making the harassment feel more real and invasive. For example, instances of sexual harassment in VR, where an individual’s avatar is touched without consent, can provoke psychological trauma akin to real-world experiences of assault. The sense of presence in these environments amplifies the emotional impact of such actions, potentially leading to long-term emotional distress [137,149].
5.3.2. Increased Aggression Due to Avatar Anonymity
The anonymity provided by avatars in XR allows individuals to engage in more aggressive behavior than they might in real life. This disinhibition effect leads to more abusive language, threats, and targeted harassment campaigns [148,151]. In these virtual environments, users feel shielded from the repercussions of their actions, which can embolden bullies to act more viciously. The integration of multimodal feedback, such as haptic technology, further heightens the emotional and psychological toll on victims. For example, in VR spaces, the sensation of touch through haptic feedback can make unwanted interactions, such as virtual hitting or touching, feel more tangible and emotionally intrusive [137,152]. This dynamic increases the psychological damage inflicted on victims, potentially leading to anxiety, depression, and other mental health issues [149].
5.3.3. Amplified Bullying Impact Through Multimodal Feedback
The integration of advanced technologies, such as facial tracking and haptic feedback, in XR environments can exacerbate the impact of cyberbullying. In traditional online spaces, harassment may be limited to text or voice interactions, but in XR, multimodal feedback creates a more immersive and realistic experience [119]. The ability to simulate physical sensations through haptics or detect facial expressions makes bullying interactions feel more personal, making it harder for victims to distance themselves emotionally from the experience. As a result, victims of bullying in these immersive environments are more likely to experience long-term psychological trauma, as the emotional weight of the harassment is intensified [151].
5.3.4. Challenges in Moderating XR Spaces
Moderating user behavior in XR environments presents unique challenges. Unlike traditional social media platforms, XR environments are vast, decentralized, and designed for real-time interactions [147]. This scale makes it difficult for moderators or algorithms to detect and respond to incidents of harassment or bullying promptly. Moreover, many XR platforms lack well-established governance structures, making it challenging to enforce rules or impose penalties for inappropriate behavior. The complexity of moderating these virtual spaces increases the likelihood that harmful behavior will go unnoticed or unpunished, leaving victims vulnerable to repeated harassment [148]. Furthermore, the lack of clear legal frameworks or recourse options for victims exacerbates the situation, as many users may not know where to turn for help when they experience abuse in XR environments [147].
5.3.5. Psychological and Social Consequences
The psychological impact of cyberbullying and harassment in XR environments mirrors the effects observed in traditional online bullying, but the immersive nature of XR can intensify these consequences. Victims of virtual harassment often report feelings of isolation, helplessness, and anxiety, which can lead to depression or suicidal ideation in severe cases [149,151]. The emotional intensity of harassment in XR environments, combined with the public nature of many virtual interactions, can also lead to social exclusion. Victims may withdraw from virtual communities out of fear of further harassment, reducing their social engagement both online and in real life [148]. This withdrawal can have far-reaching effects on mental health, as social connections and community involvement are critical to emotional well-being [150].
5.3.6. Real-Time Monitoring and Reporting Systems
To effectively combat cyber harassment and bullying in XR environments, it is essential to implement real-time monitoring systems capable of detecting harmful behavior as it occurs. AI-driven moderation tools can scan conversations for abusive language and flag inappropriate physical interactions between avatars [147]. These systems should be complemented by robust reporting mechanisms that allow users to report incidents of harassment quickly and easily [137]. Such systems will enable moderators to respond to reports promptly and prevent further abuse. Additionally, integrating machine learning algorithms capable of identifying patterns of harassment could help platforms proactively intervene before situations escalate [119].
5.3.7. Establishing Clear Codes of Conduct
A foundational step in addressing harassment and bullying in XR is establishing clear, enforceable codes of conduct that outline acceptable behavior and the consequences for violations [148]. These guidelines should be made available to users upon entering virtual spaces and reinforced regularly to ensure everyone is aware of the expectations. Developers and platform operators should also provide clear instructions on how to report violations and what steps will be taken in response to reported incidents [137]. By promoting transparency and accountability, these codes of conduct can create a safer virtual environment.
5.3.8. Personal Boundaries and Safety Features
XR environments should incorporate personal boundary settings and safety features to give users control over their interactions. Personal boundary settings can prevent avatars from entering another user’s virtual space, thereby reducing the risk of unwanted physical contact or harassment [147]. These features should also include easy-to-use tools that allow users to block, mute, or report offenders, enabling them to protect themselves in real-time. Providing users with the ability to customize their virtual safety settings ensures that they can maintain control over their virtual experiences and reduce their exposure to harmful behavior [137].
5.3.9. Collaboration between Stakeholders
Addressing the issue of cyber harassment and bullying in XR requires collaboration between developers, policymakers, and legal authorities. Establishing governance frameworks that hold individuals accountable for their actions in virtual environments is crucial [148]. Legal recourse options should be available for victims, including the development of international standards for behavior in the metaverse. Governments, corporations, and advocacy groups must work together to create regulations that ensure XR environments are safe, inclusive, and free from harassment [147]. A collaborative approach will help create a framework for addressing the unique challenges posed by harassment in virtual spaces and ensure that those responsible for harmful behavior are held accountable.
5.4. Data Privacy and Security Risks
XR technologies are uniquely positioned at the intersection of physical and virtual worlds, creating immersive experiences by collecting vast amounts of biometric, behavioral, cognitive, and affective data. While these data collection methods are integral to the personalized nature of XR systems, they also present significant privacy and security risks. If such sensitive information is misused or improperly secured, it could be exploited for malicious purposes, leading to ethical, legal, and personal consequences for users [153,154].
5.4.1. Biometric Data Vulnerabilities
One of the key aspects of XR technology is its reliance on biometric data to enhance user immersion and interaction. This includes biometric inputs such as eye-tracking, facial recognition, and EEG (brain activity) that adjust virtual environments and control avatars in real-time. These inputs, while offering personalization, also present severe privacy risks. Unauthorized access to this biometric data could allow third parties to track individuals’ eye movements, facial expressions, or even brain patterns, potentially leading to identity theft or unauthorized surveillance [41,155]. Additionally, biometric data could enable more intrusive profiling of users than is currently possible through other technologies, resulting in the violation of personal autonomy and privacy [28].
5.4.2. Behavioral Data Collection and Profiling
In XR environments, vast amounts of behavioral data are collected, tracking user interactions with objects, movement patterns, and decision-making processes. This data can be used to build comprehensive psychological profiles without users’ consent or knowledge. These profiles could then be sold to third-party advertisers or corporations, or even shared with governments [111]. This behavioral data can be particularly dangerous in its ability to manipulate users’ actions through targeted advertisements or influence behavior in political or social contexts. The potential for behavioral manipulation in XR environments is compounded by the immersive nature of the technology, which offers a deeper level of engagement than traditional online platforms [153,156].
5.4.3. Cognitive and Affective Data Exploitation
XR systems equipped with sensors that measure emotional and physiological responses, such as GSR and heart rate monitors, can track users’ emotional states in real-time [28,70]. These systems offer powerful tools for adapting virtual environments to users’ affective states, enhancing the immersive experience. However, this also raises serious concerns about the exploitation of this data. Cognitive data, such as brain activity measured by EEG, could reveal intimate information about users’ thoughts and emotions [75]. Such data could be sold to advertisers looking to target individuals based on their emotional vulnerabilities or manipulated by malicious actors for nefarious purposes [153]. The commercialization of cognitive and affective data poses significant ethical risks, as users may not be fully aware of how deeply XR systems are penetrating their cognitive processes .
5.4.4. Hacking and Security Breaches
As XR systems grow in complexity and popularity, they become increasingly attractive targets for cyberattacks. Hackers may exploit vulnerabilities in these systems to take control of virtual environments or steal sensitive personal data, such as user location, interaction patterns, and even financial information [153]. In more extreme cases, hackers could manipulate virtual experiences, potentially causing psychological harm to users by creating hostile or traumatic environments. The integration of BCIs in XR adds another layer of risk, as unauthorized access to neural data could lead to unprecedented forms of exploitation, including the manipulation of thoughts or behaviors [75]. Furthermore, the decentralized nature of many XR platforms makes it difficult to secure these environments, leaving users vulnerable to potential cyberattacks [154].
5.4.5. Surveillance and Data Misuse by Corporations
Many XR platforms are operated by large corporations that have extensive control over the data they collect from users. There is increasing concern that these companies could misuse sensitive XR data for corporate gain, such as engaging in surveillance or tracking user behaviors across virtual spaces without explicit consent [156,157]. This data could then be used to refine algorithms for predicting user behavior or influencing decision-making processes, thus blurring the line between user autonomy and corporate exploitation [154]. Additionally, many users are unaware of the granular level of data being collected in XR environments, raising concerns about the scope and transparency of data collection practices [111].
5.4.6. Transparent Data Policies and User Consent
To address the growing privacy risks in XR environments, platform developers must adopt transparent data policies that clearly inform users about what data is being collected, how it will be used, and with whom it will be shared [154]. Robust consent mechanisms should be in place, allowing users to opt in or out of specific data collection practices. Ensuring that users are fully informed about the scope of data collection and their rights can foster trust and reduce the likelihood of misuse [157]. Transparency is critical in creating a responsible data ecosystem where users retain control over their personal information.
5.4.7. Enhanced Data Encryption and Security Protocols
Given the sensitivity of biometric, behavioral, and cognitive data collected by XR systems, enhanced encryption techniques and security protocols must be implemented to protect against unauthorized access [153]. Multi-factor authentication and encryption of sensitive data are essential measures for securing users’ information [154]. Additionally, XR platforms should conduct regular security audits and vulnerability assessments to ensure that their systems are protected against evolving cyber threats. Developers must stay ahead of security challenges by continuously updating their systems to safeguard against hacking and data breaches.
5.4.8. Ethical Guidelines for Data Use
Regulatory frameworks governing the ethical use of data collected by XR technologies are urgently needed. These guidelines should set clear boundaries for how biometric, behavioral, and cognitive data can be used, particularly when it comes to targeted advertising or behavioral manipulation [156]. Companies should be required to follow data minimization principles, collecting only the data necessary for the functionality of their systems while avoiding excessive or invasive data collection practices [154]. Ethical standards should prioritize user autonomy and privacy, ensuring that data collection practices remain transparent and accountable.
5.4.9. International Collaboration for Data Privacy
As XR platforms operate on a global scale, international collaboration is essential to developing standardized data privacy regulations. Governments, industry leaders, and advocacy groups must work together to create cross-border agreements that protect users from data exploitation, regardless of where they are using XR systems [154,157]. These regulations should focus on ensuring that users’ privacy rights are respected and that companies are held accountable for their data collection practices across all jurisdictions. A unified global approach is necessary to address the complex and interconnected privacy risks associated with XR technologies [153].
5.5. Intense Advertising and Commercial Exploitation
The immersive nature of XR technologies and the metaverse opens new avenues for advertisers to embed themselves deeply into the user’s virtual experience. As users navigate digital spaces, they are no longer merely exposed to traditional forms of advertising but rather engage with hyper-targeted and personalized ads embedded seamlessly within their environments [157,158]. This shift presents several significant ethical challenges, ranging from user manipulation to concerns over the commercialization of immersive spaces.
5.5.1. Hyper-Targeted and Manipulative Advertising
One of the most pressing concerns in XR advertising is the use of hyper-targeted ads. XR technologies collect vast amounts of personal data, such as eye-tracking, behavioral patterns, and even cognitive states measured by EEG and GSR sensors [29,46,70]. This data allows advertisers to tailor their messages with unprecedented precision, responding to users’ real-time interactions, interests, and emotional responses. The ability to track where a user looks, how long they focus on a particular item, and even their emotional reactions allows advertisers to create highly personalized experiences [156,158].
While personalization can improve the relevance of ads, it also opens the door to manipulative advertising practices. For example, a user’s emotional state—whether they are excited, stressed, or tired—could be used to strategically deliver ads that exploit their vulnerabilities [75]. This is particularly concerning in scenarios where users may not fully realize the extent to which their behaviors and emotional responses are being monitored and used to influence their decisions. Research has shown that this type of advertising can blur the line between persuasion and manipulation, particularly when ads are delivered during heightened emotional states [156,157].
In XR environments, the manipulation extends beyond static advertisements. Advertisers can create fully immersive virtual experiences that feel natural and non-invasive, allowing users to interact with products or services in ways that foster engagement and emotional attachment [111]. For instance, users might be encouraged to explore virtual stores where they can try on clothes through their avatars or interact with virtual items that mirror their real-world counterparts. While this could enhance the shopping experience, it also raises ethical questions about the manipulation of user preferences and behaviors [110].
5.5.2. Commercial Exploitation of Personal Data
A significant concern surrounding XR advertising is the commercial exploitation of personal data. XR platforms collect detailed biometric and behavioral data, which can provide deep insights into users’ psychological profiles [46,69,70]. For example, a user’s emotional responses to stimuli, attention patterns, and cognitive engagement levels can be tracked and used to inform advertising strategies [46]. This data could potentially be sold to third-party advertisers or used to create detailed consumer profiles, raising concerns about privacy and consent [159].
Moreover, the collection of affective data, such as GSR and heart rate monitoring, allows advertisers to tap into users’ emotional states in real-time, influencing their behaviors at particularly vulnerable moments [70]. For example, when a user is feeling stressed or emotionally charged, an advertisement could be designed to provide comfort or relief, making it more likely that the user will respond positively to the message [28]. The risk here is that users may not fully understand the extent to which their emotional and cognitive states are being used to influence their purchasing decisions, leading to a form of subconscious manipulation [156].
The use of such detailed data also introduces the potential for misuse or exploitation, particularly when users are unaware of the full scope of data collection. In some cases, advertisers may push the boundaries of ethical behavior by using cognitive and emotional data to manipulate user behavior, fostering over-reliance on virtual environments or encouraging compulsive purchasing behaviors [71,157].
5.5.3. Over-Commercialization and "Ad Fatigue"
As XR platforms continue to grow, there is a risk that these spaces will become over-commercialized [158]. The immersive nature of XR means that advertisements are no longer confined to traditional banners or pop-ups; instead, they can be embedded directly into the user’s virtual environment, making them harder to ignore. This constant exposure to immersive ads could lead to "ad fatigue," where users become overwhelmed by the sheer volume and intensity of marketing messages [160].
In traditional online environments, users already face a high level of exposure to advertisements, which can lead to disengagement and frustration. This problem is likely to be magnified in XR, where users are fully immersed in their surroundings and unable to easily escape intrusive ads [156]. The risk is that the over-commercialization of XR spaces could diminish the quality of the user experience, leading to psychological stress, distraction, and a reduction in the overall enjoyment of the immersive environment [29].
Furthermore, as advertisers seek to monetize the metaverse and XR platforms, users may find themselves increasingly bombarded by immersive advertisements that demand their attention. This could lead to a deterioration of trust between users and XR platforms, particularly if users feel that their immersive experiences are being compromised by commercial interests [154]. The challenge for XR developers and advertisers alike will be to strike a balance between monetization and user satisfaction, ensuring that ads are integrated seamlessly and ethically into the virtual environment.
5.5.4. Mitigation Strategies for Ethical Advertising
To mitigate the risks associated with intense advertising and commercial exploitation in XR, several strategies can be employed. One approach is to ensure that advertising is transparent and non-intrusive. Users should have clear control over the types of ads they encounter, as well as the ability to opt out of data collection practices that could be used for hyper-targeted marketing [159]. Another important strategy is the implementation of ethical design principles in the creation of XR advertisements. Advertisers should avoid exploiting users’ emotional states or cognitive vulnerabilities for commercial gain [70,157]. Instead, XR platforms should focus on creating positive, relevant experiences that respect the user’s autonomy and well-being [5]. Finally, regulatory frameworks and data protection laws must evolve to address the unique challenges posed by XR technologies. Governments and policymakers should work with XR developers to ensure that users’ privacy is protected and that the exploitation of biometric, behavioral, and cognitive data is strictly regulated [154,159]. By fostering transparency, accountability, and user control, XR platforms can create a safer and more ethical advertising ecosystem.
5.6. Manipulation of Public Opinion and Information
The immersive and persuasive nature of XR platforms presents new risks for the manipulation of public opinion and the spread of disinformation. As XR technology integrates into everyday life, virtual environments may become increasingly influential in shaping individuals’ beliefs and behaviors. These platforms provide fertile ground for disinformation campaigns, where misleading or fabricated content can be seamlessly woven into immersive experiences, making it harder for users to differentiate between fact and fiction [157,161]. The convergence of AI, virtual worlds, and real-time user interaction heightens the risk of psychological and social manipulation, potentially transforming XR spaces into tools for influencing public opinion on a large scale [162,163].
5.6.1. False Realities and Misinformation
One of the most significant dangers posed by XR is the ability to create entirely fabricated environments that users may accept as real. Misinformation can be embedded within these virtual worlds, where users interact with distorted or false narratives, potentially reinforcing existing biases or creating new ones [157,161]. Virtual experiences are highly immersive, and the emotional and cognitive impact they can have on users often exceeds that of traditional media. In such contexts, individuals may be exposed to "echo chambers," where their beliefs and opinions are reinforced by interacting with only those ideas that align with their pre-existing views, exacerbating political polarization [152,163]. Misinformation in XR environments is more than just textual or visual; it is experiential, making it harder for users to critically evaluate the information presented to them [164].
5.6.2. Influence on Behavior
The immersive nature of XR has a profound psychological effect, allowing users to engage with information on a much deeper level than traditional media [21,26]. In these environments, users may be more vulnerable to manipulation, as immersive experiences often elicit stronger emotional responses, which can significantly influence decision-making processes [17]. This makes XR an ideal platform for orchestrating campaigns designed to sway public opinion or influence individual behaviors [149]. Virtual reality simulations that portray emotionally charged or controversial content could manipulate users’ emotions to create fear, loyalty, or anger, effectively shaping how they view real-world events or political issues [163]. The use of cognitive biases, such as emotional appeal and repetition, in immersive environments makes it easier for disinformation campaigns to alter perceptions and beliefs subtly.
5.6.3. Deepfake Avatars and AI-Generated Content
Another major concern is the use of deepfake technology and AI-generated content within XR spaces. Deepfakes, which use artificial intelligence to create highly realistic but fabricated videos or avatars, can be employed to impersonate public figures or trusted individuals [157,163]. In an XR environment, malicious actors could create convincing deepfake avatars of politicians, celebrities, or community leaders to mislead users or manipulate public discourse. For instance, a deepfake avatar of a political figure could deliver a fabricated speech in a virtual rally, promoting false claims or disinformation [161]. The seamless integration of these fake avatars into XR environments makes it difficult for users to detect the deception. Furthermore, AI-generated content—such as speeches, articles, or social media posts—could be used to flood virtual spaces with misleading information, influencing how users perceive events, public figures, and political issues [157,164](Westerlund, 2019).
5.6.4. Improving Media Literacy and Critical Thinking Skills
One of the most effective ways to combat the manipulation of public opinion in XR environments is by improving media literacy and encouraging critical thinking [165]. Users must be taught how to evaluate the credibility of virtual experiences and be given tools to distinguish between factual information and manipulated content. Educational programs focused on identifying disinformation in XR spaces can help users become more skeptical of immersive content, making them less susceptible to manipulation [162,165]. Additionally, implementing fact-checking mechanisms within XR environments can provide users with real-time feedback on the validity of the information they encounter.
5.6.5. Authentication of Avatars and Content
Developers of XR platforms should prioritize creating systems for verifying the authenticity of avatars and digital content. Authentication tools that use blockchain technology or secure identification processes can help ensure that avatars representing public figures or institutions are legitimate, reducing the risk of deepfake impersonations [166]. These systems could include digital signatures that verify the origin of content and guarantee that public discourse is not being manipulated by AI-generated or fabricated personas. Additionally, moderation tools that detect deepfakes or manipulated content in real-time should be integrated into XR environments to mitigate the spread of disinformation [161].
5.6.6. Regulation and Collaboration
Governments, tech companies, and policymakers need to collaborate to develop regulations that address the risks of disinformation in XR environments. These regulations should focus on preventing the exploitation of immersive technologies for political or financial gain, holding companies accountable for the content shared on their platforms [162,166]. Additionally, platforms should be required to have robust reporting mechanisms for users to flag misleading or harmful content. As XR platforms grow in popularity and influence, clear governance frameworks will be essential to ensure that public opinion is not manipulated through immersive and deceptive virtual experiences [157].
5.7. Physical Health Concerns
The use of XR technologies—especially VR and AR headsets and controllers—offers immersive experiences but also raises concerns about potential physical health impacts. As users spend increasing amounts of time engaging with XR environments, the physical demands of these technologies can lead to various health issues [167,168]. Whether from prolonged use of headsets, repetitive physical actions, or long periods of sedentary behavior, these risks need to be considered carefully in the design and use of XR systems.
5.7.1. Eye Strain and Vision Problems
One of the most commonly reported physical health concerns associated with XR is digital eye strain, also known as computer vision syndrome [167,168]. Users of VR headsets often experience discomfort due to the close proximity of the screens to their eyes and the focus adjustments required to view 3D content. This strain can result in headaches, blurred vision, and difficulty focusing, especially after extended use [167]. The immersion provided by VR also reduces natural blinking rates, leading to dry eyes and exacerbating discomfort [168]. As XR usage becomes more prevalent, particularly in education, entertainment, and professional training, the risk of long-term vision issues may increase if appropriate measures are not taken to limit exposure and implement screen breaks.
5.7.2. Musculoskeletal Problems
In addition to eye strain, repetitive hand, arm, and body movements associated with XR interfaces can lead to musculoskeletal problems [169]. Users who engage in prolonged VR sessions involving repetitive tasks like grabbing, gesturing, or moving their bodies may experience strain in their wrists, shoulders, or neck [170]. The lack of ergonomic consideration in many XR applications, such as gaming, increases the risk of repetitive strain injuries (RSIs) similar to those seen in individuals who use traditional computer interfaces for extended periods [169].
Moreover, VR headsets are typically heavy and can place strain on the neck, especially when worn for long periods [167]. This may lead to neck and back pain, particularly if users adopt poor postures while interacting with virtual environments [170]. These issues are especially concerning for professional training scenarios, where users may be required to perform complex physical tasks repeatedly in a virtual setting, potentially leading to chronic discomfort if proper ergonomic practices are not adopted [167].
5.7.3. Sedentary Lifestyle and Physical Inactivity
While some XR applications, such as fitness programs or rehabilitation exercises, encourage physical movement, many XR experiences—particularly in entertainment and gaming—can promote sedentary behavior [169]. In these virtual environments, users may remain seated for long periods, interacting primarily through hand gestures or head movements rather than full-body activity. This raises concerns about the broader health implications of extended XR use, including the risk of obesity, cardiovascular disease, and metabolic disorders associated with prolonged physical inactivity [170].
The immersive nature of XR can cause users to lose track of time, further exacerbating sedentary behavior. This problem is compounded by the increasing trend of incorporating XR technologies into entertainment and social platforms, where users may spend hours in virtual worlds without taking breaks or engaging in physical movement [167]. The impact of such prolonged inactivity mirrors the concerns already raised by excessive screen time and video gaming, where users experience health complications from extended periods of sitting and reduced physical activity [168].
5.7.4. Breaks and Screen Time Management
To address the risk of eye strain and other vision problems, XR applications should incorporate regular screen breaks and reminders to encourage users to rest their eyes [168]. For example, limiting VR session times or programming the system to prompt users to take periodic breaks can help reduce the risk of digital eye strain. Developers can also optimize display settings to minimize visual discomfort, such as adjusting brightness, contrast, and refresh rates to align with ergonomic best practices [167].
5.7.5. Ergonomic Design and Physical Support
Designers of XR systems must prioritize ergonomic principles in the development of headsets and controllers to reduce the risk of musculoskeletal problems [169,170]. Lightweight headsets, adjustable straps, and properly designed controllers that promote natural hand and arm positions can alleviate strain on users’ bodies. Additionally, incorporating movement variation into XR experiences, such as encouraging full-body engagement or alternating between different physical tasks, can help reduce the risk of repetitive strain injuries [167].
5.7.6. Promoting Physical Activity in XR
To counteract the sedentary nature of many XR applications, developers can integrate movement-based activities and physical challenges into virtual environments, even those designed for entertainment or social interaction [171]. Fitness applications that require users to move, stretch, or exercise while engaging with the virtual environment are an example of how XR can promote physical activity [170]. Additionally, adopting built-in features that monitor inactivity and prompt users to move or take physical breaks could help mitigate the health risks associated with prolonged sitting [169].
5.8. Digital Divide and Inequality
As XR technologies become more integrated into various sectors such as education, healthcare, and industry, concerns are growing that these advancements could exacerbate existing social and economic inequalities [172,173]. The high cost of XR hardware and the necessary infrastructure may create significant barriers for lower-income populations, leading to a widening digital divide [123]. This divide could result in a two-tiered society where only those with the financial means can fully participate in and benefit from the metaverse and XR applications, further entrenching socio-economic disparities [115,166].
5.8.1. Economic Barriers
One of the primary challenges is the substantial cost associated with acquiring XR hardware. High-quality VR headsets, AR glasses, haptic suits, and other peripheral devices often come with a hefty price tag that is unaffordable for many individuals and institutions, especially in low-income communities and developing countries [123,166]. Additionally, the infrastructure required to support these technologies—such as high-speed internet, powerful computers, and ample physical space—adds to the financial burden [172]. This creates a significant barrier to entry, limiting access to those who can afford these expenses.
The economic barriers to accessing XR technologies risk creating a two-tiered system of digital experience [173]. Individuals and communities with financial resources can access enhanced educational tools, healthcare services, and professional opportunities through XR, while those without such means are left behind [172]. This disparity not only limits personal and professional development for those in lower-income brackets but also reinforces existing social and economic inequalities [123]. The unequal distribution of technological resources can lead to a cycle where disadvantaged groups become increasingly marginalized in a digitally-driven society.
5.8.2. Access to Education and Healthcare
XR has the potential to revolutionize education by providing immersive and interactive learning experiences. However, without equitable access, students from lower-income families or underfunded schools may miss out on these advancements, widening the educational gap [174]. Schools in affluent areas may integrate XR into their curricula, offering students cutting-edge tools for learning complex subjects through virtual laboratories, historical simulations, and interactive problem-solving environments [63]. Conversely, schools lacking resources may continue with traditional methods, potentially disadvantaging their students in terms of engagement and skill acquisition [172].
In healthcare, XR technologies are being utilized for patient treatment, medical training, mental health therapies, and rehabilitation programs [60]. These innovations can improve patient outcomes and make healthcare services more efficient [93]. However, the high costs associated with XR technologies may prevent underfunded hospitals, clinics, and healthcare providers in low-income or rural areas from adopting them. This could result in unequal access to high-quality healthcare, where only patients in well-funded systems benefit from the latest XR-enhanced medical treatments and services [60,75].
5.8.3. Impact on Workforce Development
As industries adopt XR technologies for training and operations, workers without access to these tools may find themselves at a disadvantage in the job market [175]. Companies may prefer or require employees who are proficient in XR technologies, creating employment barriers for those from lower socio-economic backgrounds who have not had the opportunity to develop these skills [115]. This can exacerbate unemployment and underemployment in already vulnerable communities, further entrenching economic disparities [172].
5.8.4. Reducing Cost Barriers
To bridge the digital divide, efforts must be made to reduce the cost of XR hardware and make it more accessible [123]. Manufacturers and developers can work towards creating affordable XR devices without significantly compromising quality [174]. Subsidies, grants, and financial assistance programs provided by governments, non-profits, and private organizations can help lower-income individuals and institutions acquire necessary equipment [173]. Bulk purchasing agreements and partnerships with educational and healthcare institutions can also reduce costs [63].
5.8.5. Improving Infrastructure Access
Expanding high-speed internet access and improving technological infrastructure in underserved areas is critical [172]. Investments in broadband expansion projects, particularly in rural and low-income urban areas, can provide the necessary connectivity for XR applications [115]. Public-private partnerships can be instrumental in funding and implementing these infrastructure projects, ensuring that more communities can participate in the digital economy [123].
5.8.6. Public Access and Community Programs
Creating public spaces where XR technologies are available can help mitigate access issues. Libraries, community centers, and schools can serve as hubs where individuals can experience and learn about XR technologies without personal ownership [166]. Educational programs and workshops can introduce XR to broader audiences, fostering digital literacy and skills development across socio-economic groups [173].
5.8.7. Policy and Regulatory Interventions
Government policies can play a significant role in addressing digital inequality [172]. Implementing regulations that promote fair pricing, prevent monopolistic practices, and encourage competition among XR providers can help lower costs [123]. Additionally, policies that support funding for technology in education and healthcare can ensure that institutions serving low-income populations are not left behind. International cooperation may also be necessary to address global disparities in XR access and to develop standards that promote inclusivity [173].
5.8.8. Encouraging Inclusive Design
Developers and content creators should consider the diverse needs of users from different socio-economic backgrounds [63]. Designing XR applications that can operate on lower-spec hardware or that require less bandwidth can make these technologies more accessible [174]. Additionally, providing multilingual support and culturally relevant content can enhance the inclusivity of XR experiences [115,172].
5.8. Psychological Detachment and Dissociation
The hyper-immersive nature of XR technologies, particularly VR, presents significant psychological risks for users. Prolonged and repeated exposure to highly immersive environments can lead to psychological detachment or dissociation, where individuals struggle to distinguish between virtual and real-world experiences [17,20]. As XR technologies continue to become more integrated into daily life, it is crucial to address the psychological risks these technologies pose and ensure the mental well-being of users.
5.8.1. Reality Confusion
One of the primary psychological risks associated with long-term XR use is reality confusion, where the boundaries between virtual and physical environments become blurred. This confusion occurs when users become so immersed in the virtual world that they have difficulty reintegrating into the real world after their virtual session ends [5,26]. Prolonged VR use can disrupt a user’s spatial awareness, leading to a sense of disorientation and altered perception of real-world movements, even after removing the headset [70].
This issue can manifest in cognitive distortions where users may struggle to distinguish between events that occurred in the virtual environment and those in reality [19]. Such experiences can become particularly problematic in scenarios designed to simulate real-world activities, such as driving, flying, or social interactions [26]. The brain’s inability to process the transition from a vivid virtual world to reality can result in physical symptoms such as dizziness, fatigue, and confusion, which can impair decision-making and coordination [21]. These effects are especially concerning for individuals with pre-existing mental health issues, such as anxiety or dissociative disorders, making extended XR exposure potentially dangerous for these populations.
5.8.2. Depersonalization and Dissociation
A more severe psychological effect of XR immersion is depersonalization and dissociation, which refers to feelings of detachment from one’s body or surroundings [48]. In highly immersive VR environments, users may experience a diminished sense of self, feeling disconnected from their physical body or identity [17]. This detachment can be exacerbated by extensive use of avatars or virtual embodiments that do not resemble the user’s real-world appearance [176]. Over time, users may start to identify more with their virtual avatar than their physical self, leading to dissociative symptoms.
Depersonalization occurs when users feel as though they are not in control of their actions or are observing themselves from an external perspective [48]. This sensation is common in VR environments where users control avatars to interact with objects and characters. The heightened sense of presence in these virtual spaces, along with the realism of avatar movements and interactions, can cause users to feel estranged from their real-world selves [26]. Such dissociative experiences can lead to emotional distress and cognitive dissonance, further complicating the user’s ability to return to normal life outside the XR environment [19].
In some cases, prolonged use of XR technologies can result in persistent dissociative states, where users experience ongoing difficulty reintegrating into their physical reality. This can manifest in symptoms such as memory lapses, a sense of unreality, or emotional numbness. XR usage that induces depersonalization is particularly concerning for users already struggling with conditions such as PTSD, anxiety, or depression, as XR experiences may amplify these conditions.
5.8.3. Limiting Exposure Time
A crucial measure for preventing psychological detachment is setting time limits on XR sessions. Developers and platform providers should integrate "break reminders" and automatic time limits to encourage users to take regular breaks, preventing over-immersion in the virtual world [48]. Limiting continuous use of XR technologies can help reduce the likelihood of dissociative experiences and cognitive disorientation [26].
5.8.4. Grounding Techniques
Users should be encouraged to engage in grounding techniques before and after XR sessions to facilitate the transition back to the real world [17]. Grounding exercises, such as focusing on physical sensations, performing breathing exercises, or engaging in physical activities, can help users regain a sense of presence in the real world after extended XR exposure [26].
5.8.5. Informed Usage Guidelines
XR developers and manufacturers should provide clear guidelines on responsible usage, outlining the risks of dissociation and reality confusion. Educating users about potential psychological effects and encouraging moderation in usage can help reduce the risks associated with extended XR exposure [5]. Providing resources and advice on safe usage can empower users to make informed decisions [48].
5.8.6. Monitoring and Psychological Support
XR platforms can implement monitoring systems to track users’ emotional and cognitive states in real time, offering tailored interventions or notifications when signs of psychological distress or dissociation are detected [17]. Additionally, providing access to mental health support or crisis services within XR environments can help users address emerging concerns immediately [26].
5.9. Regulatory Challenges and Governance
The decentralized, evolving nature of the metaverse introduces substantial regulatory and governance challenges, particularly in areas like data privacy, user safety, intellectual property, and content moderation [166,177]. As XR technologies become more sophisticated and pervasive, the need for clear regulations and frameworks to govern virtual environments becomes increasingly urgent. However, the global, decentralized, and immersive nature of these spaces complicates efforts to apply traditional legal and regulatory frameworks [153].
5.9.1. Lack of Regulation
One of the primary challenges facing XR and the metaverse is the lack of robust regulatory frameworks [166,177]. As governments and regulatory bodies struggle to keep up with the rapid pace of technological advancement, many critical areas, such as user data privacy, security, and safety, remain under-regulated. Current data protection laws like the General Data Protection Regulation (GDPR) in the European Union or the California Consumer Privacy Act (CCPA) in the United States, while applicable in some cases, do not fully address the unique complexities of XR environments, where vast amounts of biometric, behavioral, and cognitive data are collected [154,159].
For instance, XR environments routinely collect sensitive biometric data, such as facial expressions, eye movements, and brain activity through EEG, all of which fall into categories of data not fully covered by existing regulations. This regulatory lag leaves significant gaps in protecting users from data misuse, identity theft, or surveillance [159]. As XR technologies become more ingrained in everyday life, regulatory bodies must act swiftly to establish comprehensive policies that protect users while fostering innovation.
5.9.2. Jurisdictional Issues
The global nature of the metaverse presents complex jurisdictional challenges that hinder effective governance [166]. XR platforms allow users from different countries to interact in shared virtual spaces, often transcending traditional national boundaries. This creates confusion over jurisdictional authority, particularly when harmful actions, such as harassment, theft, or fraud, occur within virtual spaces [147].
The cross-border nature of XR raises difficult questions about legal accountability. For example, if a user is harassed by another individual in a virtual world, it may be unclear whether the legal action should be taken in the jurisdiction where the platform operates, the country where the user resides, or the location of the perpetrator. Additionally, different countries have varying standards and laws regarding data privacy, intellectual property, and user rights, making it difficult to enforce a consistent global governance model for XR platforms [157].
Efforts to regulate the metaverse will likely require international collaboration, with governments, corporations, and advocacy groups working together to establish cross-border agreements and shared regulatory standards [157]. Such frameworks must account for the unique properties of XR and metaverse environments, where the line between physical and digital realities is increasingly blurred.
5.9.3. Content Moderation
Another significant regulatory challenge in XR environments is content moderation [150]. Given the vast scale and complexity of the metaverse, where users can create, share, and interact with limitless amounts of digital content, effective moderation is difficult to implement [153]. In traditional social media, companies like Facebook or Twitter have established governance systems to monitor and remove harmful content, but these tools are not always adaptable to the immersive, real-time interactions in XR.
In XR spaces, harmful content can take many forms, ranging from misinformation and hate speech to inappropriate virtual behaviors such as harassment or stalking [147]. The immersive nature of XR can make these experiences even more damaging, as users may experience virtual violations more viscerally than on traditional platforms. The challenge of moderating such vast environments, where content is often created and disseminated in real-time, highlights the limitations of current governance models.
Decentralized governance models in XR platforms further complicate content moderation. Some platforms, like Decentraland, operate through decentralized autonomous organizations (DAOs), where community members vote on governance decisions rather than relying on centralized corporate control [166]. While DAOs can promote transparency and user participation, they also pose challenges in enforcing content standards, as decision-making can be slow or inconsistent across different communities.
5.9.4. Intellectual Property Concerns
Another critical governance issue in XR spaces is the question of intellectual property (IP). In immersive virtual environments, users can create digital assets such as avatars, virtual real estate, or works of art, leading to complex questions about ownership and the protection of intellectual property rights [162,166]. As users and companies increasingly invest in creating and selling digital assets in the metaverse, disputes over ownership, usage rights, and unauthorized reproduction are likely to rise [177].
The decentralized nature of XR platforms means that enforcing IP laws is challenging. Virtual goods may be copied, resold, or modified without the original creator’s consent, raising concerns about how to protect creators’ rights within these spaces. Additionally, the rise of non-fungible tokens (NFTs), which are often used to verify ownership of virtual assets, introduces a new layer of complexity. While NFTs provide a mechanism for proving ownership, they do not always prevent unauthorized duplication or distribution of virtual goods [162,166].
5.9.5. International Legal Frameworks and Collaboration
To mitigate the regulatory gaps and jurisdictional issues in XR, governments, and international organizations must collaborate on developing global legal standards. This could include treaties that establish shared principles for data privacy, user protection, and content moderation within virtual environments [166]. By creating a unified approach, governments can better regulate the cross-border nature of XR spaces while ensuring consistency in legal accountability [157].
5.9.6. Advanced AI for Content Moderation
Developers of XR platforms must invest in AI-driven moderation systems capable of detecting and addressing harmful content in real-time [153]. This could involve machine learning algorithms that flag inappropriate behavior, such as harassment or hate speech, based on a set of predefined rules. AI systems can also monitor user interactions to detect harmful patterns before they escalate, helping to create safer virtual environments [150].
5.9.7. Decentralized Governance with Accountability
Decentralized platforms that rely on DAOs or community-based governance structures should establish clear mechanisms for accountability and enforcement. While decentralization can promote user agency, it also requires checks and balances to ensure that harmful behavior or content is adequately addressed [166]. Transparent governance processes, regular audits, and clear codes of conduct can help foster more responsible community-led moderation [177].
5.9.8. Strengthening IP Laws for the Metaverse
Governments and legal experts must adapt intellectual property laws to address the unique challenges posed by digital assets and virtual goods in the metaverse [166]. This could involve establishing clearer guidelines for the creation, ownership, and distribution of virtual assets, as well as enforcing penalties for IP infringement within these spaces [177]. Additionally, as NFTs become more prevalent, regulatory bodies should establish guidelines that ensure NFTs provide adequate protection for creators while safeguarding against unauthorized replication.
6. General Discussion
This scoping review reveals the transformative potential of XR technologies when analyzed through the lens of the Extended Mind Theory. With the rise of VR, AR, and MR, it becomes increasingly clear that cognitive processes are no longer confined to the biological boundaries of the brain. Instead, they expand into the digital world, integrating with tools, objects, and environments that profoundly shape how we think, interact, learn, and solve problems. While these advances hold the promise of unprecedented cognitive augmentation, they also introduce complex ethical, psychological, and societal challenges that demand rigorous scrutiny and regulation.
6.1. The Role of XR in Cognitive Extension: Expanding the Human Mind
The Extended Mind Theory, which argues that cognition is distributed across internal and external system, finds a natural fit in XR technologies. These immersive environments demonstrate how digital objects and tools are not merely external aids to human cognition but function as integral components of cognitive processes.
6.1.1. From Cognitive Offloading to Cognitive Augmentation
One of the key contributions of XR to the extended mind concept is its capacity to enable and enhance cognitive offloading. In XR environments, users can interact with vast amounts of information, complex simulations, and dynamic digital systems in real time, offloading tasks like memory retention, spatial reasoning, and information processing onto external systems. This allows individuals to transcend the natural limitations of biological cognition [3].
In the context of education, VR and AR systems allow learners to engage with material in ways that stimulate not just visual and auditory senses but also physical and spatial reasoning. Medical students, for instance, can practice surgeries in VR, manipulating virtual organs with haptic feedback and real-time guidance [109], while engineers can explore architectural designs or complex systems in immersive 3D environments [33]. In these scenarios, learners offload cognitive processes—such as understanding spatial relationships or predicting outcomes—onto XR environments, making learning more interactive and immediate.
Moreover, cognitive augmentation through XR extends beyond education. In professional settings such as design, healthcare, and engineering, XR technologies offer real-time feedback and interaction with complex systems, allowing professionals to perform tasks more efficiently and accurately. By interacting with digital twins of machinery, engineers can preemptively diagnose issues, simulate maintenance tasks, or optimize designs before physical prototypes are built [175]. This offloading and extension of cognitive tasks not only boosts productivity but also leads to deeper insights and more innovative solutions [26].
6.1.2. Embodied Cognition and Virtual Bodies
Embodiment is another core concept that XR technologies reinforce, as users not only cognitively interact with virtual environments but also physically and emotionally engage through virtual avatars or digital representations of themselves. In XR environments, the sensation of embodiment—where users feel that they are physically present in the virtual world—enables them to perform tasks as though they are interacting with physical reality. This sensation enhances cognitive immersion and changes the way people perceive and interact with digital objects and other virtual agents [21].
For example, haptic feedback systems enable users to feel resistance when lifting virtual objects or experience tactile sensations when interacting with digital surfaces [9,11,52]. This multisensory engagement deepens the sense of presence and realism, making tasks such as surgical training or mechanical repair feel more authentic. BCIs take this further, allowing users to control digital environments with their neural activity, bypassing traditional input methods like controllers or keyboards [73]. Such interfaces directly link cognition to digital interactions, showcasing how the mind can be extended into virtual spaces [75].
Embodiment also transforms social interactions within XR environments. In collaborative virtual settings, the use of avatars that replicate real-time body movements, facial expressions, and gestures enhances communication and teamwork [19,58]. This blending of physical and virtual presence allows users to collaborate on projects, share ideas, and solve problems in ways that simulate real-world interactions. These embodied experiences are vital for maintaining a sense of connection and shared purpose, particularly in professional and educational settings where collaboration is critical [105].
6.2. Applications of XR in Real-World Contexts
The transformative potential of XR technologies is not only theoretical but is already being realized in practical applications across various fields. These technologies demonstrate significant promise in areas such as education, healthcare, and professional training by expanding cognitive capacities and enhancing user experiences through immersive environments.
6.2.1. Education and Training
One of the most prominent applications of XR technologies is in education, where they have revolutionized learning by offering immersive, interactive experiences that extend the traditional boundaries of instruction. In fields like science, technology, engineering, and mathematics (STEM), XR facilitates hands-on learning in virtual laboratories, where students can engage with complex concepts through virtual dissections, chemical simulations, and architectural designs [108]. These environments allow learners to offload cognitive tasks such as spatial reasoning and problem-solving to the virtual interface, enhancing their capacity to understand and retain information [24]. Medical training has also been significantly enhanced through XR applications. VR enables medical students to practice surgeries in a risk-free, controlled environment, receiving real-time feedback through haptic feedback systems [101,109] This level of interaction, coupled with immersive simulations, has led to improved knowledge retention and skills acquisition, demonstrating the effectiveness of XR in training contexts [30].
6.2.2. Healthcare
Beyond education, XR is increasingly being used in healthcare, particularly for therapeutic interventions and medical training. XR-based cognitive-behavioral therapies (CBT) for anxiety, depression, and phobias allow patients to engage in exposure therapy within controlled virtual environments. These environments are personalized in real-time using biometric data, such as EEG and GSR readings, to monitor the user’s emotional and cognitive states [26]. Such interventions have been shown to be particularly effective in treating specific phobias and PTSD [178]. Additionally, XR is being employed in pain management, where VR distraction therapy helps reduce chronic pain by immersing patients in calming virtual worlds [93]. The ability to manipulate patients’ focus away from pain through immersive visual and auditory experiences provides a promising alternative to traditional pain management techniques. Moreover, XR technologies have significant potential in neuropsychological assessment, offering more ecologically valid testing of cognitive and motor functions. Tools like the Virtual Reality Everyday Assessment Lab (VR-EAL) enable assessments of prospective memory, executive function, and attention in realistic, daily-life environments, improving the accuracy of clinical diagnoses and rehabilitation compared to traditional methods [179,180,181]. The increased ecological validity and engagement of XR assessments make them valuable for both research and clinical practice.
6.2.3. Professional and Industrial Training
XR technologies have also proven to be invaluable in professional and industrial training, particularly in fields that require complex skills and decision-making under pressure. For example, in manufacturing and construction, workers can train in operating machinery and performing intricate tasks in a virtual space, reducing risk while enhancing proficiency [108]. Virtual environments simulate real-world conditions, allowing professionals to practice without the constraints of physical resources or the risks associated with errors. In emergency response training, XR is used to simulate high-stress scenarios, such as natural disasters or large-scale emergencies. First responders can rehearse their roles in virtual environments, enhancing their preparedness and decision-making capabilities. The ability to replicate these environments with real-time feedback using biometric monitoring adds a new dimension to training efficacy [107].
6.2.4. Entertainment and Cultural Heritage
In entertainment, XR has expanded the boundaries of user engagement. Video games and social platforms within the metaverse leverage XR to create fully immersive environments that respond to users’ movements and emotions in real time [149]. These applications utilize advanced tracking systems such as eye-tracking and hand-tracking to mirror users’ actions within the virtual world, creating a heightened sense of presence and interactivity [17]. Cultural heritage preservation has also benefited from XR technologies. Virtual museums allow users to explore historical artifacts and sites through immersive 3D reconstructions, with haptic feedback providing tactile sensations to deepen user engagement [124]. These experiences enhance accessibility to cultural education and provide new ways for users to interact with heritage, fostering a deeper connection to historical content [126].
6.3. Ethical and Psychological Implications of XR
While XR technologies offer significant cognitive and practical advantages, they also introduce several ethical, psychological, and social challenges. These issues require urgent attention, especially as XR technologies become more integrated into daily life and professional environments. The growing ubiquity of XR platforms raises questions about data privacy, psychological safety, social inequality, and the potential for long-term dependency on digital environments.
6.3.1. Privacy, Surveillance, and Data Ethics
The integration of biometric and behavioral data collection within XR environments raises significant privacy concerns. XR technologies often rely on tracking user movements, gaze, facial expressions, and even neural activity, all of which are highly sensitive data points [46,168]. These data can be used to personalize experiences, optimize system performance, or monitor cognitive engagement, but they can also be exploited for commercial or political purposes. For example, eye-tracking data can reveal what users focus on, allowing marketers to target them with precision-based advertising, while brainwave monitoring could be used to manipulate user behavior by adjusting virtual environments to induce certain emotional or cognitive states [44,156].
The integration of biometric and behavioral data collection in XR environments poses significant ethical and privacy challenges. Current frameworks, such as the GDPR, only partially address these issues, as they were not designed with immersive technologies in mind [154]. Eye-tracking, gait analysis, and even neural data, collected BCIs, can provide unprecedented levels of insight into users’ cognitive and emotional states, which could be exploited for commercial or political manipulation [46,76].
Ethical concerns surrounding informed consent in immersive environments must also be addressed. Users need to fully understand how their biometric data is being collected, processed, and potentially used. Without clear regulations, XR platforms risk becoming environments of mass surveillance, where users’ privacy is compromised. Governments and regulatory bodies must work collaboratively to adapt existing laws like GDPR and develop new legal frameworks that account for the nuances of XR environments [153,157]. These frameworks should emphasize user autonomy, ensuring that consent is informed and cognitive privacy is respected [156].
6.3.2. Psychological and Behavioral Impacts
Prolonged exposure to XR environments may have significant psychological and behavioral effects, particularly as users become more immersed in these spaces. One major concern is the potential for XR addiction, where users spend excessive time in virtual worlds at the expense of their real-world responsibilities and relationships [149]. This is especially concerning in the context of entertainment-based XR, such as gaming or social platforms within the metaverse. The deep sense of presence and the compelling nature of these environments can make it difficult for users to disconnect, leading to negative impacts on mental health, such as anxiety, depression, or social isolation [143,149].
Additionally, XR environments can lead to psychological dissociation, where users have difficulty distinguishing between the real world and virtual spaces [84,149]. This blurring of reality can be particularly harmful for vulnerable populations, such as children or individuals with mental health conditions. Research has shown that immersive experiences can lead to heightened anxiety, confusion, or even hallucinations in some cases, especially if users are unable to reset their cognitive boundaries after prolonged use of XR systems [149,182].
Another significant concern is the potential for harassment or inappropriate behavior within virtual environments. The sense of embodiment in XR can make interactions feel very real, and violations of personal space or harassment in virtual worlds can result in psychological trauma similar to that experienced in physical spaces [105,150]. This underscores the need for robust systems of moderation and governance within XR platforms, ensuring that users can feel safe and protected in these environments.
6.4. Societal and Technological Impacts: Bridging Realities and Digital Inequality
6.4.1. Digital Divide and Access
As XR technologies advance, they may exacerbate existing inequalities related to digital access and literacy. While these technologies offer significant cognitive and professional advantages to those who can access them, they may also widen the gap between those with the resources and skills to engage with XR and those without [172]. Access to high-end XR systems often requires substantial financial investment, as well as access to fast internet, specialized hardware, and technological literacy [174]. These barriers could exclude low-income communities, underfunded schools, and developing regions from the cognitive benefits offered by XR [183].
Furthermore, as XR becomes more integrated into professional training, education, and healthcare, those who lack access to these technologies may find themselves at a significant disadvantage. This digital divide could have profound implications for employment, education, and healthcare outcomes, reinforcing existing social and economic inequalities. Policymakers and educators will need to ensure that XR technologies are accessible to a wide range of users and that digital literacy programs are in place to equip individuals with the skills necessary to navigate these environments effectively [25,174].
6.4.2. Future Integration with Artificial Intelligence (AI)
The future of XR technologies lies in their integration with Artificial Intelligence (AI). AI-driven XR environments will have the capability to adapt to users’ cognitive and emotional states in real time, creating personalized experiences that optimize learning, productivity, and entertainment [184]. For instance, AI systems could detect when a user is mentally fatigued and adjust the virtual environment to provide less cognitively demanding tasks [24]. Alternatively, in professional settings, AI could assist users in solving complex problems by analyzing their cognitive patterns and providing real-time feedback [175].
However, the integration of AI into XR also raises concerns about autonomy and decision-making. As AI systems become more sophisticated, there is a risk that users may become overly reliant on these technologies for cognitive support, potentially diminishing their problem-solving abilities and critical thinking skills [184]. The question arises: how much cognitive control should we delegate to AI-driven systems, and at what point does this delegation hinder, rather than augment, human cognition?
7. Conclusion
The convergence of Extended Mind Theory with XR technologies offers a profound framework for understanding the evolution of human cognition in the digital age. XR technologies, including VR, AR, and MR, create immersive environments that enable users to offload cognitive tasks, enhance learning, and engage in complex problem-solving. By seamlessly integrating external digital systems with human cognition, these technologies challenge the traditional view of cognition as confined to the brain, extending it into interactive, external environments.
As demonstrated in this review, XR has vast potential across various domains. In education and training, XR facilitates immersive learning experiences that transcend traditional teaching methods, providing real-time, interactive environments for students and professionals alike. From virtual laboratories in STEM education to medical training that simulates high-stakes surgeries, XR fosters cognitive augmentation by allowing users to interact with complex systems in ways that were previously unimaginable. In healthcare, XR is revolutionizing therapeutic interventions, including cognitive-behavioral therapies and neuropsychological assessments, which provide more ecologically valid assessments of cognitive functions, enhancing clinical diagnosis and rehabilitation efforts. Additionally, XR’s potential in professional and industrial training, as well as in entertainment and cultural heritage, underscores its wide-ranging applicability.
However, the integration of XR with emerging technologies like AI and BCIs introduces significant ethical, psychological, and societal challenges. Issues such as data privacy, the psychological effects of prolonged immersion, and the potential for social inequality must be addressed as XR becomes increasingly embedded in daily life. The risk of overreliance on AI-driven cognitive augmentation further raises questions about balancing human autonomy with technological assistance.
To fully harness the potential of XR technologies while mitigating risks, it is essential to develop robust ethical frameworks. These frameworks must prioritize the protection of privacy, equitable access to technology, and the promotion of mental health. As XR continues to evolve, the balance between cognitive augmentation and ethical responsibility will be crucial in ensuring that these immersive technologies contribute to human development rather than detracting from it. By fostering responsible use and expanding access, XR can play a pivotal role in shaping a future where digital and immersive technologies enhance, rather than hinder, human flourishing across educational, healthcare, professional, and societal contexts..
Funding
This research received no external funding
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.
Conflicts of Interest
The author declare no conflicts of interest.
References
- Kandel, E.R.; Squire, L.R. Neuroscience: Breaking Down Scientific Barriers to the Study of Brain and Mind. Science 2000, 290, 1113–1120. [Google Scholar] [CrossRef] [PubMed]
- Kandel, E.R.; Schwartz, J.H.; Jessell, T.M.; Siegelbaum, S.A.; Hudspeth, A.J. Principles of Neural Science, Fifth Edition; McGraw-Hill’s AccessMedicine; McGraw-Hill Education, 2013; ISBN 978-0-07-139011-8.
- Clark, A.; Chalmers, D. The Extended Mind. Analysis 1998, 58, 7–19. [Google Scholar] [CrossRef]
- Wilson, R.A. Boundaries of the Mind: The Individual in the Fragile Sciences - Cognition; Cambridge University Press: Cambridge, 2004; ISBN 978-0-521-83645-6. [Google Scholar]
- Slater, M.; Sanchez-Vives, M.V. Enhancing Our Lives with Immersive Virtual Reality. Front. Robot. AI 2016, 3. [Google Scholar] [CrossRef]
- Mania, K.; Chalmers, A. The Effects of Levels of Immersion on Memory and Presence in Virtual Environments: A Reality Centered Approach. Cyberpsychol. Behav. 2001, 4, 247–264. [Google Scholar] [CrossRef] [PubMed]
- Vogeley, K.; Bente, G. “Artificial Humans”: Psychology and Neuroscience Perspectives on Embodiment and Nonverbal Communication. Soc. Cogn. Babies Robots 2010, 23, 1077–1090. [Google Scholar] [CrossRef]
- Argelaguet, F.; Hoyet, L.; Trico, M.; Lecuyer, A. The Role of Interaction in Virtual Embodiment: Effects of the Virtual Hand Representation. In Proceedings of the 2016 IEEE Virtual Reality (VR); 2016; pp. 3–10. [Google Scholar]
- Kourtesis, P.; Argelaguet, F.; Vizcay, S.; Marchal, M.; Pacchierotti, C. Electrotactile Feedback Applications for Hand and Arm Interactions: A Systematic Review, Meta-Analysis, and Future Directions. IEEE Trans. Haptics 2022, 15, 479–496. [Google Scholar] [CrossRef]
- Lécuyer, A. Simulating Haptic Feedback Using Vision: A Survey of Research and Applications of Pseudo-Haptic Feedback. Presence Teleoperators Virtual Environ. 2009, 18, 39–53. [Google Scholar] [CrossRef]
- Lécuyer, A.; Burkhardt, J.-M.; Etienne, L. Feeling Bumps and Holes without a Haptic Interface: The Perception of Pseudo-Haptic Textures. In Proceedings of the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; Association for Computing Machinery: New York, NY, USA, 2004; pp. 239–246. [Google Scholar]
- Lebedev, M.A.; Nicolelis, M.A.L. Brain–Machine Interfaces: Past, Present and Future. Trends Neurosci. 2006, 29, 536–546. [Google Scholar] [CrossRef] [PubMed]
- Dede, C. Immersive Interfaces for Engagement and Learning. Science 2009, 323, 66–69. [Google Scholar] [CrossRef]
- Vizcay, S.; Kourtesis, P.; Argelaguet, F.; Pacchierotti, C.; Marchal, M. Electrotactile Feedback For Enhancing Contact Information in Virtual Reality – Best Paper Award. In Proceedings of the ICAT-EGVE 2021 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments; Orlosky, J., Reiners, D., Weyers, B., Eds.; The Eurographics Association; 2021. [Google Scholar]
- Vizcay, S.; Kourtesis, P.; Argelaguet, F.; Pacchierotti, C.; Marchal, M. Design, Evaluation and Calibration of Wearable Electrotacile Interfaces for Enhancing Contact Information in Virtual Reality. Comput. Graph. 2023. [Google Scholar] [CrossRef]
- Loomis, J.M.; Blascovich, J.J.; Beall, A.C. Immersive Virtual Environment Technology as a Basic Research Tool in Psychology. Behav. Res. Methods Instrum. Comput. 1999, 31, 557–564. [Google Scholar] [CrossRef] [PubMed]
- Gonzalez-Franco, M.; Lanier, J. Model of Illusions and Virtual Reality. Front. Psychol. 2017, 8, 1125. [Google Scholar] [CrossRef] [PubMed]
- Kourtesis, P.; Korre, D.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Guidelines for the Development of Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: The Development of Virtual Reality Everyday Assessment Lab (VR-EAL), a Neuropsychological Test Battery in Immersive Virtual Reality. Front. Comput. Sci. 2020, 1. [Google Scholar] [CrossRef]
- Maister, L.; Slater, M.; Sanchez-Vives, M.V.; Tsakiris, M. Changing Bodies Changes Minds: Owning Another Body Affects Social Cognition. Trends Cogn. Sci. 2015, 19, 6–12. [Google Scholar] [CrossRef] [PubMed]
- Slater, M. Immersion and the Illusion of Presence in Virtual Reality. Br. J. Psychol. 2018, 109, 431–433. [Google Scholar] [CrossRef] [PubMed]
- Slater, M. Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments. Philos. Trans. R. Soc. B Biol. Sci. 2009, 364, 3549–3557. [Google Scholar] [CrossRef]
- Schubert, T.; Friedmann, F.; Regenbrecht, H. The Experience of Presence: Factor Analytic Insights. Presence Teleoperators Virtual Environ. 2001, 10, 266–281. [Google Scholar] [CrossRef]
- Milgram, P.; Kishino, F. A Taxonomy of Mixed Reality Visual Displays. IEICE Trans. Inf. Syst. 1994, 77, 1321–1329. [Google Scholar]
- Makransky, G.; Lilleholt, L. A Structural Equation Modeling Investigation of the Emotional Value of Immersive Virtual Reality in Education. Educ. Technol. Res. Dev. 2018, 66, 1141–1164. [Google Scholar] [CrossRef]
- Merchant, Z.; Goetz, E.T.; Cifuentes, L.; Keeney-Kennicutt, W.; Davis, T.J. Effectiveness of Virtual Reality-Based Instruction on Students’ Learning Outcomes in K-12 and Higher Education: A Meta-Analysis. Comput. Educ. 2014, 70, 29–40. [Google Scholar] [CrossRef]
- Riva, G.; Wiederhold, B.K.; Mantovani, F. Neuroscience of Virtual Reality: From Virtual Exposure to Embodied Medicine. Cyberpsychology Behav. Soc. Netw. 2019, 22, 82–96. [Google Scholar] [CrossRef]
- Guger, C.; Ramoser, H.; Pfurtscheller, G. Real-Time EEG Analysis with Subject-Specific Spatial Patterns for a Brain-Computer Interface (BCI). IEEE Trans. Rehabil. Eng. 2000, 8, 447–456. [Google Scholar] [CrossRef] [PubMed]
- Guger, C.; Edlinger, G.; Harkam, W.; Niedermayer, I.; Pfurtscheller, G. How Many People Are Able to Operate an EEG-Based Brain-Computer Interface (BCI)? IEEE Trans. Neural Syst. Rehabil. Eng. 2003, 11, 145–147. [Google Scholar] [CrossRef]
- Billinghurst, M.; Clark, A.; Lee, G. A Survey of Augmented Reality. Found. Trends® Human–Computer Interact. 2015, 8, 73–272. [Google Scholar] [CrossRef]
- Barsom, E.Z.; Graafland, M.; Schijven, M.P. Systematic Review on the Effectiveness of Augmented Reality Applications in Medical Training. Surg. Endosc. 2016, 30, 4174–4183. [Google Scholar] [CrossRef] [PubMed]
- Azuma, R.T. A Survey of Augmented Reality. Presence Teleoperators Virtual Environ. 1997, 6, 355–385. [Google Scholar] [CrossRef]
- Flavián, C.; Ibáñez-Sánchez, S.; Orús, C. The Impact of Virtual, Augmented and Mixed Reality Technologies on the Customer Experience. J. Bus. Res. 2019, 100, 547–560. [Google Scholar] [CrossRef]
- Osorto Carrasco, M.D.; Chen, P.-H. Application of Mixed Reality for Improving Architectural Design Comprehension Effectiveness. Autom. Constr. 2021, 126, 103677. [Google Scholar] [CrossRef]
- Moro, C.; Štromberga, Z.; Raikos, A.; Stirling, A. The Effectiveness of Virtual and Augmented Reality in Health Sciences and Medical Anatomy. Anat. Sci. Educ. 2017, 10, 549–559. [Google Scholar] [CrossRef]
- Kaufmann, H.; Schmalstieg, D. Mathematics and Geometry Education with Collaborative Augmented Reality. Comput. Graph. 2003, 27, 339–345. [Google Scholar] [CrossRef]
- Chiossi, F.; Mayer, S. How Can Mixed Reality Benefit From Physiologically-Adaptive Systems? Challenges and Opportunities for Human Factors Applications 2023.
- Islam, R.; Desai, K.; Quarles, J. Cybersickness Prediction from Integrated HMD’s Sensors: A Multimodal Deep Fusion Approach Using Eye-Tracking and Head-Tracking Data. In Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR); 2021; pp. 31–40. [Google Scholar]
- Kirsh, D. Embodied Cognition and the Magical Future of Interaction Design. ACM Trans Comput-Hum Interact 2013, 20. [Google Scholar] [CrossRef]
- Bailenson, J. Experience on Demand : What Virtual Reality Is, How It Works, and What It Can Do; W.W. Norton & Company, Inc.: New York, NY, 2018; ISBN 978-0-393-25370-2. [Google Scholar]
- Nagy, P.; Koles, B. The Digital Transformation of Human Identity: Towards a Conceptual Model of Virtual Identity in Virtual Worlds. Convergence 2014, 20, 276–292. [Google Scholar] [CrossRef]
- Duchowski, A.T. Eye Tracking Methodology; Springer International Publishing: Cham, 2017; ISBN 978-3-319-57881-1. [Google Scholar]
- Kourtesis, P.; Amir, R.; Linnell, J.; Argelaguet, F.; MacPherson, S.E. Cybersickness, Cognition, & Motor Skills: The Effects of Music, Gender, and Gaming Experience. IEEE Trans. Vis. Comput. Graph. 2023, 29, 2326–2336. [Google Scholar] [CrossRef]
- Kourtesis, P.; Papadopoulou, A.; Roussos, P. Cybersickness in Virtual Reality: The Role of Individual Differences, Its Effects on Cognitive Functions and Motor Skills, and Intensity Differences during and after Immersion. Virtual Worlds 2024, 3, 62–93. [Google Scholar] [CrossRef]
- Marshall, S.P. Identifying Cognitive State from Eye Metrics. Aviat. Space Environ. Med. 2007, 78, B165–B175. [Google Scholar] [PubMed]
- Giatzoglou, E.; Vorias, P.; Kemm, R.; Karayianni, I.; Roussou, M.; Kourtesis, P. The Trail-Making-Test in Virtual Reality (TMT-VR): The Effects of Interaction Modes and Gaming Skills on Cognitive Performance of Young Adults. Preprints 2024. [Google Scholar] [CrossRef]
- Adhanom, I.B.; MacNeilage, P.; Folmer, E. Eye Tracking in Virtual Reality: A Broad Review of Applications and Challenges. Virtual Real. 2023, 27, 1481–1505. [Google Scholar] [CrossRef]
- Zhang, Z.; Fort, J.M.; Giménez Mateu, L. Facial Expression Recognition in Virtual Reality Environments: Challenges and Opportunities. Front. Psychol. 2023, 14. [Google Scholar] [CrossRef] [PubMed]
- Blascovich, J.; Bailenson, J. Infinite Reality: Avatars, Eternal Life, New Worlds, and the Dawn of the Virtual Revolution; William Morrow & Co, 2011; ISBN 0-06-180950-0.
- Kennedy, D.P.; Adolphs, R. Perception of Emotions from Facial Expressions in High-Functioning Adults with Autism. Neuropsychologia 2012, 50, 3313–3319. [Google Scholar] [CrossRef] [PubMed]
- Piumsomboon, T.; Lee, G.; Lindeman, R.W.; Billinghurst, M. Exploring Natural Eye-Gaze-Based Interaction for Immersive Virtual Reality. In Proceedings of the 2017 IEEE Symposium on 3D User Interfaces (3DUI); March 18 2017; pp. 36–39. [Google Scholar]
- Bai, H.; Lee, G.A.; Ramakrishnan, M.; Billinghurst, M. 3D Gesture Interaction for Handheld Augmented Reality. In Proceedings of the SIGGRAPH Asia 2014 Mobile Graphics and Interactive Applications; Association for Computing Machinery: New York, NY, USA; 2014. [Google Scholar]
- Wang, D.; Guo, Y.; Liu, S.; Zhang, Y.; Xu, W.; Xiao, J. Haptic Display for Virtual Reality: Progress and Challenges. Haptic Interact. 2019, 1, 136–162. [Google Scholar] [CrossRef]
- Kreimeier, J.; Hammer, S.; Friedmann, D.; Karg, P.; Bühner, C.; Bankel, L.; Götzelmann, T. Evaluation of Different Types of Haptic Feedback Influencing the Task-Based Presence and Performance in Virtual Reality. In Proceedings of the Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments; Association for Computing Machinery: New York, NY, USA, 2019; pp. 289–298. [Google Scholar]
- Chen, Y.; Tsai, M.-J. Eye-Hand Coordination Strategies during Active Video Game Playing: An Eye-Tracking Study. Comput. Hum. Behav. 2015, 51, 8–14. [Google Scholar] [CrossRef]
- Kilteni, K.; Bergstrom, I.; Slater, M. Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play. IEEE Trans. Vis. Comput. Graph. 2013, 19, 597–605. [Google Scholar] [CrossRef] [PubMed]
- Waltemate, T.; Gall, D.; Roth, D.; Botsch, M.; Latoschik, M.E. The Impact of Avatar Personalization and Immersion on Virtual Body Ownership, Presence, and Emotional Response. IEEE Trans. Vis. Comput. Graph. 2018, 24, 1643–1652. [Google Scholar] [CrossRef] [PubMed]
- Pan, C.Y.; Frey, G.C.; Chuang, T.Y. The Impact of Physical Activity Intervention on Improving Cognition and Motor Skills in Individuals with Autism Spectrum Disorder: A Systematic Review. Autism Res. 2015, 8, 376–390. [Google Scholar] [CrossRef]
- Pan, X.; Hamilton, A.F. de C. Why and How to Use Virtual Reality to Study Human Social Interaction: The Challenges of Exploring a New Research Landscape. Br. J. Psychol. 2018, 109, 395–417. [Google Scholar] [CrossRef] [PubMed]
- Gui, C.; Venema, D.M.; Chien, J.H.; Cochran, T.M.; Siu, K.-C. Quantifying Fear of Falling by Utilizing Objective Body Sway Measures: A 360° Virtual Video Study. Gait Posture 2022, 93, 160–165. [Google Scholar] [CrossRef] [PubMed]
- Ng, Y.-L.; Ma, F.; Ho, F.K.; Ip, P.; Fu, K. Effectiveness of Virtual and Augmented Reality-Enhanced Exercise on Physical Activity, Psychological Outcomes, and Physical Performance: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Comput. Hum. Behav. 2019, 99, 278–291. [Google Scholar] [CrossRef]
- Holden, M.K. Virtual Environments for Motor Rehabilitation: Review. Cyberpsychol. Behav. 2005, 8, 187–211. [Google Scholar] [CrossRef]
- Katifori, A.; Karvounis, M.; Kourtis, V.; Perry, S.; Roussou, M.; Ioanidis, Y. Applying Interactive Storytelling in Cultural Heritage: Opportunities, Challenges and Lessons Learned. In Proceedings of the Interactive Storytelling; Rouse, R., Koenitz, H., Haahr, M., Eds.; Springer International Publishing: Cham, 2018; pp. 603–612. [Google Scholar]
- Liritzis, I.; Volonakis, P.; Vosinakis, S. 3D Reconstruction of Cultural Heritage Sites as an Educational Approach. The Sanctuary of Delphi. Appl. Sci. 2021, 11. [Google Scholar] [CrossRef]
- Lele, A. Virtual Reality and Its Military Utility. J. Ambient Intell. Humaniz. Comput. 2013, 4, 17–26. [Google Scholar] [CrossRef]
- Pastel, S.; Petri, K.; Chen, C.H.; Wiegand Cáceres, A.M.; Stirnatis, M.; Nübel, C.; Schlotter, L.; Witte, K. Training in Virtual Reality Enables Learning of a Complex Sports Movement. Virtual Real. 2023, 27, 523–540. [Google Scholar] [CrossRef]
- Okamura, A.M.; Cutkosky, M.R.; Dennerlein, J.T. Reality-Based Models for Vibration Feedback in Virtual Environments. IEEEASME Trans. Mechatron. 2001, 6, 245–252. [Google Scholar] [CrossRef]
- Zhao, L.; Liu, Y.; Ye, D.; Ma, Z.; Song, W. Implementation and Evaluation of Touch-Based Interaction Using Electrovibration Haptic Feedback in Virtual Environments. In Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR); 2020; pp. 239–247. [Google Scholar]
- Kim, J.; Kim, S.; Lee, J. The Effect of Multisensory Pseudo-Haptic Feedback on Perception of Virtual Weight. IEEE Access 2022, 10, 5129–5140. [Google Scholar] [CrossRef]
- Chen, S.; Jiang, K.; Hu, H.; Kuang, H.; Yang, J.; Luo, J.; Chen, X.; Li, Y. Emotion Recognition Based on Skin Potential Signals with a Portable Wireless Device. Sensors 2021, 21. [Google Scholar] [CrossRef] [PubMed]
- Villani, D.; Repetto, C.; Cipresso, P.; Riva, G. May I Experience More Presence in Doing the Same Thing in Virtual Reality than in Reality? An Answer from a Simulated Job Interview✩. Interact. Comput. 2012, 24, 265–272. [Google Scholar] [CrossRef]
- Villani, D.; Riva, F.; Riva, G. New Technologies for Relaxation: The Role of Presence. Int. J. Stress Manag. 2007, 14, 260. [Google Scholar] [CrossRef]
- Janssen, J.H.; Bailenson, J.N.; IJsselsteijn, W.A.; J. H. D. M. Westerink Intimate Heartbeats: Opportunities for Affective Communication Technology. IEEE Trans. Affect. Comput. 2010, 1, 72–80. [Google Scholar] [CrossRef]
- Leeb, R.; Friedman, D.; Müller-Putz, G.R.; Scherer, R.; Slater, M.; Pfurtscheller, G. Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A Case Study with a Tetraplegic. Comput. Intell. Neurosci. 2007, 2007, 079642. [Google Scholar] [CrossRef] [PubMed]
- Nijholt, A.; Tan, D.; Pfurtscheller, G.; Brunner, C.; Millán, J. d. R.; B. Allison; B. Graimann; F. Popescu; B. Blankertz; K. -R. Müller Brain-Computer Interfacing for Intelligent Systems. IEEE Intell. Syst. 2008, 23, 72–79. [Google Scholar] [CrossRef]
- Mak, J.N.; Wolpaw, J.R. Clinical Applications of Brain-Computer Interfaces: Current State and Future Prospects. IEEE Rev. Biomed. Eng. 2009, 2, 187–199. [Google Scholar] [CrossRef]
- Bamdad, M.; Zarshenas, H.; Auais, M.A. Application of BCI Systems in Neurorehabilitation: A Scoping Review. Disabil. Rehabil. Assist. Technol. 2015, 10, 355–364. [Google Scholar] [CrossRef] [PubMed]
- Mühl, C.; Allison, B.; Nijholt, A.; Chanel, G. A Survey of Affective Brain Computer Interfaces: Principles, State-of-the-Art, and Challenges. Brain-Comput. Interfaces 2014, 1, 66–84. [Google Scholar] [CrossRef]
- Thompson, T.; Steffert, T.; Ros, T.; Leach, J.; Gruzelier, J. EEG Applications for Sport and Performance. Neuroimaging Sports Sci. 2008, 45, 279–288. [Google Scholar] [CrossRef] [PubMed]
- Coffey, E.B.J.; Brouwer, A.-M.; van Erp, J.B.F. Measuring Workload Using a Combination of Electroencephalography and near Infrared Spectroscopy. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2012, 56, 1822–1826. [Google Scholar] [CrossRef]
- Maples-Keller, J.L.; Bunnell, B.E.; Kim, S.-J.; Rothbaum, B.O. The Use of Virtual Reality Technology in the Treatment of Anxiety and Other Psychiatric Disorders. Harv. Rev. Psychiatry 2017, 25. [Google Scholar] [CrossRef] [PubMed]
- Rizzo, A.; Difede, J.; Rothbaum, B.O.; Reger, G.; Spitalnick, J.; Cukor, J.; Mclay, R. Development and Early Evaluation of the Virtual Iraq/Afghanistan Exposure Therapy System for Combat-Related PTSD. Ann. N. Y. Acad. Sci. 2010, 1208, 114–125. [Google Scholar] [CrossRef]
- Bohil, C.J.; Alicea, B.; Biocca, F.A. Virtual Reality in Neuroscience Research and Therapy. Nat. Rev. Neurosci. 2011, 12, 752–762. [Google Scholar] [CrossRef] [PubMed]
- Emmelkamp, P.M.G.; Meyerbröker, K. Virtual Reality Therapy in Mental Health. Annu. Rev. Clin. Psychol. 2021, 17, 495–519. [Google Scholar] [CrossRef]
- Freeman, D.; Bradley, J.; Antley, A.; Bourke, E.; DeWeever, N.; Evans, N.; Černis, E.; Sheaves, B.; Waite, F.; Dunn, G.; et al. Virtual Reality in the Treatment of Persecutory Delusions: Randomised Controlled Experimental Study Testing How to Reduce Delusional Conviction. Br. J. Psychiatry 2016, 209, 62–67. [Google Scholar] [CrossRef]
- Kourtesis, P.; Amir, R.; Linnell, J.; Argelaguet, F.; MacPherson, S.E. Cybersickness, Cognition, & Motor Skills: The Effects of Music, Gender, and Gaming Experience. IEEE Trans. Vis. Comput. Graph. 2023, 29, 2326–2336. [Google Scholar] [CrossRef]
- Kourtesis, P.; MacPherson, S.E. How Immersive Virtual Reality Methods May Meet the Criteria of the National Academy of Neuropsychology and American Academy of Clinical Neuropsychology: A Software Review of the Virtual Reality Everyday Assessment Lab (VR-EAL). Comput. Hum. Behav. Rep. 2021, 4, 100151. [Google Scholar] [CrossRef]
- Kourtesis, P.; MacPherson, S.E. An Ecologically Valid Examination of Event-Based and Time-Based Prospective Memory Using Immersive Virtual Reality: The Influence of Attention, Memory, and Executive Function Processes on Real-World Prospective Memory. Neuropsychol. Rehabil. 2023, 33, 255–280. [Google Scholar] [CrossRef] [PubMed]
- Felton, E.A.; Williams, J.C.; Vanderheiden, G.C.; Radwin, R.G. Mental Workload during Brain–Computer Interface Training. Ergonomics 2012, 55, 526–537. [Google Scholar] [CrossRef]
- Gaggioli, A.; Pallavicini, F.; Morganti, L.; Serino, S.; Scaratti, C.; Briguglio, M.; Crifaci, G.; Vetrano, N.; Giulintano, A.; Bernava, G.; et al. Experiential Virtual Scenarios With Real-Time Monitoring (Interreality) for the Management of Psychological Stress: A Block Randomized Controlled Trial. J Med Internet Res 2014, 16, e167. [Google Scholar] [CrossRef]
- Lorentz, L.; Simone, M.; Zimmermann, M.; Studer, B.; Suchan, B.; Althausen, A.; Estocinova, J.; Müller, K.; Lendt, M. Evaluation of a VR Prototype for Neuropsychological Rehabilitation of Attentional Functions. Virtual Real. 2021. [Google Scholar] [CrossRef]
- Nolin, P.; Stipanicic, A.; Henry, M.; Lachapelle, Y.; Lussier-Desrochers, D.; Rizzo, A. “Skip”; Allain, P. ClinicaVR: Classroom-CPT: A Virtual Reality Tool for Assessing Attention and Inhibition in Children and Adolescents. Comput. Hum. Behav. 2016, 59, 327–333. [Google Scholar] [CrossRef]
- Schmitt, Y.S.; Hoffman, H.G.; Blough, D.K.; Patterson, D.R.; Jensen, M.P.; Soltani, M.; Carrougher, G.J.; Nakamura, D.; Sharar, S.R. A Randomized, Controlled Trial of Immersive Virtual Reality Analgesia, during Physical Therapy for Pediatric Burns. Burns 2011, 37, 61–68. [Google Scholar] [CrossRef]
- Goudman, L.; Jansen, J.; Billot, M.; Vets, N.; De Smedt, A.; Roulaud, M.; Rigoard, P.; Moens, M. Virtual Reality Applications in Chronic Pain Management: Systematic Review and Meta-Analysis. JMIR Serious Games 2022, 10, e34402. [Google Scholar] [CrossRef]
- Sensinger, J.W.; Dosen, S. A Review of Sensory Feedback in Upper-Limb Prostheses From the Perspective of Human Motor Control. Front. Neurosci. 2020, 14. [Google Scholar] [CrossRef] [PubMed]
- Lindgren, R.; Tscholl, M.; Wang, S.; Johnson, E. Enhancing Learning and Engagement through Embodied Interaction within a Mixed Reality Simulation. Comput. Educ. 2016, 95, 174–187. [Google Scholar] [CrossRef]
- Parong, J.; Mayer, R.E. Learning Science in Immersive Virtual Reality. J. Educ. Psychol. 2018, 110, 785. [Google Scholar] [CrossRef]
- Corrigan, N.; Păsărelu, C.-R.; Voinescu, A. Immersive Virtual Reality for Improving Cognitive Deficits in Children with ADHD: A Systematic Review and Meta-Analysis. Virtual Real. 2023. [Google Scholar] [CrossRef] [PubMed]
- Herrero, J.F.; Lorenzo, G. An Immersive Virtual Reality Educational Intervention on People with Autism Spectrum Disorders (ASD) for the Development of Communication Skills and Problem Solving. Educ. Inf. Technol. 2020, 25, 1689–1722. [Google Scholar] [CrossRef]
- Kourtesis, P.; Kouklari, E.-C.; Roussos, P.; Mantas, V.; Papanikolaou, K.; Skaloumbakas, C.; Pehlivanidis, A. Virtual Reality Training of Social Skills in Adults with Autism Spectrum Disorder: An Examination of Acceptability, Usability, User Experience, Social Skills, and Executive Functions. Behav. Sci. 2023, 13. [Google Scholar] [CrossRef] [PubMed]
- Maskey, M.; Lowry, J.; Rodgers, J.; McConachie, H.; Parr, J.R. Reducing Specific Phobia/Fear in Young People with Autism Spectrum Disorders (ASDs) through a Virtual Reality Environment Intervention. PLOS ONE 2014, 9, e100374. [Google Scholar] [CrossRef] [PubMed]
- Andersen, S.A.W.; Mikkelsen, P.T.; Konge, L.; Cayé-Thomasen, P.; Sørensen, M.S. Cognitive Load in Mastoidectomy Skills Training: Virtual Reality Simulation and Traditional Dissection Compared. J. Surg. Educ. 2016, 73, 45–50. [Google Scholar] [CrossRef] [PubMed]
- Andersen, S.A.W.; Mikkelsen, P.T.; Konge, L.; Cayé-Thomasen, P.; Sørensen, M.S. The Effect of Implementing Cognitive Load Theory-Based Design Principles in Virtual Reality Simulation Training of Surgical Skills: A Randomized Controlled Trial. Adv. Simul. 2016, 1, 20. [Google Scholar] [CrossRef] [PubMed]
- William, Irvin; Claire, Goldie; Christopher, O’Brien; Christopher, Aura; Leonard, Temme. Michael Wilson Virtual Reality Aviation Emergency Procedure (EP) Testbed.; April 12 2021; Vol. 11759, p. 1175909.
- Kim, S.; Lee, G.; Huang, W.; Kim, H.; Woo, W.; Billinghurst, M. Evaluating the Combination of Visual Communication Cues for HMD-Based Mixed Reality Remote Collaboration. In Proceedings of the Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems; Association for Computing Machinery: New York, NY, USA, 2019; pp. 1–13. [Google Scholar]
- Rogers, S.L.; Broadbent, R.; Brown, J.; Fraser, A.; Speelman, C.P. Realistic Motion Avatars Are the Future for Social Interaction in Virtual Reality. Front. Virtual Real. 2022, 2. [Google Scholar] [CrossRef]
- Parmar, D.; Isaac, J.; Babu, S.V.; D’Souza, N.; Leonard, A.E.; Jörg, S.; Gundersen, K.; Daily, S.B. Programming Moves: Design and Evaluation of Applying Embodied Interaction in Virtual Environments to Enhance Computational Thinking in Middle School Students. In Proceedings of the 2016 IEEE Virtual Reality (VR); March 19 2016; pp. 131–140. [Google Scholar]
- Alshowair, A.; Bail, J.; AlSuwailem, F.; Mostafa, A.; Abdel-Azeem, A. Use of Virtual Reality Exercises in Disaster Preparedness Training: A Scoping Review. SAGE Open Med. 2024, 12, 20503121241241936. [Google Scholar] [CrossRef]
- Adami, P.; Rodrigues, P.B.; Woods, P.J.; Becerik-Gerber, B.; Soibelman, L.; Copur-Gencturk, Y.; Lucas, G. Effectiveness of VR-Based Training on Improving Construction Workers’ Knowledge, Skills, and Safety Behavior in Robotic Teleoperation. Adv. Eng. Inform. 2021, 50, 101431. [Google Scholar] [CrossRef]
- Alaker, M.; Wynn, G.R.; Arulampalam, T. Virtual Reality Training in Laparoscopic Surgery: A Systematic Review & Meta-Analysis. Int. J. Surg. 2016, 29, 85–94. [Google Scholar] [CrossRef] [PubMed]
- Pantano, E.; Rese, A.; Baier, D. Enhancing the Online Decision-Making Process by Using Augmented Reality: A Two Country Comparison of Youth Markets. J. Retail. Consum. Serv. 2017, 38, 81–95. [Google Scholar] [CrossRef]
- Grewal, D.; Roggeveen, A.L.; Nordfält, J. The Future of Retailing. Future Retail. 2017, 93, 1–6. [Google Scholar] [CrossRef]
- Huang, T.-L.; Liao, S.-L. Creating E-Shopping Multisensory Flow Experience through Augmented-Reality Interactive Technology. Internet Res. 2017, 27, 449–475. [Google Scholar] [CrossRef]
- Blázquez, M. Fashion Shopping in Multichannel Retail: The Role of Technology in Enhancing the Customer Experience. Int. J. Electron. Commer. 2014, 18, 97–116. [Google Scholar] [CrossRef]
- Lukosch, S.; Lukosch, H.; Datcu, D.; Cidota, M. Providing Information on the Spot: Using Augmented Reality for Situational Awareness in the Security Domain. Comput. Support. Coop. Work CSCW 2015, 24, 613–664. [Google Scholar] [CrossRef]
- Santiteerakul, S.; Sopadang, A.; Yaibuathet Tippayawong, K.; Tamvimol, K. The Role of Smart Technology in Sustainable Agriculture: A Case Study of Wangree Plant Factory. Sustainability 2020, 12. [Google Scholar] [CrossRef]
- Kesim, M.; Ozarslan, Y. Augmented Reality in Education: Current Technologies and the Potential for Education. Cyprus Int. Conf. Educ. Res. CY-ICER-2012North Cyprus US08-10 Febr. 2012 2012, 47, 297–302. [Google Scholar] [CrossRef]
- Szilágyi, R.; Herdon, M. Augmented Reality (AR) Applications in Agriculture. In E-Innovation for Sustainable Development of Rural Resources During Global Economic Crisis; IGI Global, 2014; pp. 65–79 ISBN 978-1-4666-8751-6.
- Balducci, F.; Impedovo, D.; Pirlo, G. Machine Learning Applications on Agricultural Datasets for Smart Farm Enhancement. Machines 2018, 6. [Google Scholar] [CrossRef]
- Wang, R.; Sun, Y.; Zong, J.; Wang, Y.; Cao, X.; Wang, Y.; Cheng, X.; Zhang, W. Remote Sensing Application in Ecological Restoration Monitoring: A Systematic Review. Remote Sens. 2024, 16. [Google Scholar] [CrossRef]
- Wang, J.; Pham, T.L.; Dang, V.T. Environmental Consciousness and Organic Food Purchase Intention: A Moderated Mediation Model of Perceived Food Quality and Price Sensitivity. Int. J. Environ. Res. Public. Health 2020, 17. [Google Scholar] [CrossRef] [PubMed]
- Pavlidis, G.; Koutsoudis, A.; Arnaoutoglou, F.; Tsioukas, V.; Chamzas, C. Methods for 3D Digitization of Cultural Heritage. J. Cult. Herit. 2007, 8, 93–98. [Google Scholar] [CrossRef]
- Jeunet, C.; Hauw, D.; Millán, J. del R. Sport Psychology: Technologies Ahead. Front. Sports Act. Living 2020, 2. [Google Scholar] [CrossRef] [PubMed]
- Wu, S.; Stendal, K.; Thapa, D. Emerging Trends in XR-Mediated Virtual Team Collaboration in Digital Workspaces: A Systematic Literature Review. In Advances in Information Systems Development: Information Systems Development, Organizational Aspects, and Societal Trends; Rodrigues da Silva, A., Mira da Silva, M., Estima, J., Barry, C., Lang, M., Linger, H., Schneider, C., Eds.; Springer Nature Switzerland: Cham, 2024; pp. 85–108. ISBN 978-3-031-57189-3. [Google Scholar]
- Bruno, F.; Bruno, S.; De Sensi, G.; Luchi, M.-L.; Mancuso, S.; Muzzupappa, M. From 3D Reconstruction to Virtual Reality: A Complete Methodology for Digital Archaeological Exhibition. J. Cult. Herit. 2010, 11, 42–49. [Google Scholar] [CrossRef]
- Wojciechowski, R.; Walczak, K.; White, M.; Cellary, W. Building Virtual and Augmented Reality Museum Exhibitions. In Proceedings of the Proceedings of the Ninth International Conference on 3D Web Technology; Association for Computing Machinery: New York, NY, USA, 2004; pp. 135–144. [Google Scholar]
- Roussou, M.; Drettakis, G. Photorealism and Non-Photorealism in Virtual Heritage Representation. In Proceedings of the The 4th International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage; Arnold, D., Chalmers, A., Niccolucci, F., Eds.; The Eurographics Association; 2003. [Google Scholar]
- Sylaiou, S.; Mania, K.; Karoulis, A.; White, M. Exploring the Relationship between Presence and Enjoyment in a Virtual Museum. Int. J. Hum.-Comput. Stud. 2010, 68, 243–253. [Google Scholar] [CrossRef]
- LaViola, J.J. A Discussion of Cybersickness in Virtual Environments. SIGCHI Bull 2000, 32, 47–56. [Google Scholar] [CrossRef]
- Rebenitsch, L.; Owen, C. Review on Cybersickness in Applications and Visual Displays. Virtual Real. 2016, 20, 101–125. [Google Scholar] [CrossRef]
- Dennison, M.S.; Wisti, A.Z.; D’Zmura, M. Use of Physiological Signals to Predict Cybersickness. Displays 2016, 44, 42–52. [Google Scholar] [CrossRef]
- Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions Without the Presence of Pertinent Adverse Symptomatology. Front. Hum. Neurosci. 2019, 13. [Google Scholar] [CrossRef]
- Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Technological Competence Is a Pre-Condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Review and Meta-Analysis. Front. Hum. Neurosci. 2019, 13. [Google Scholar] [CrossRef] [PubMed]
- Chang, E.; Kim, H.T.; Yoo, B. Predicting Cybersickness Based on User’s Gaze Behaviors in HMD-Based Virtual Reality. J. Comput. Des. Eng. 2021, 8, 728–739. [Google Scholar] [CrossRef]
- Fernandes, A.S.; Feiner, S.K. Combating VR Sickness through Subtle Dynamic Field-of-View Modification. In Proceedings of the 2016 IEEE Symposium on 3D User Interfaces (3DUI); March 19 2016; pp. 201–210. [Google Scholar]
- Kourtesis, P.; Linnell, J.; Amir, R.; Argelaguet, F.; MacPherson, S.E. Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ. Virtual Worlds 2023, 2, 16–35. [Google Scholar] [CrossRef]
- King, D.; Delfabbro, P.; Griffiths, M. The Convergence of Gambling and Digital Media: Implications for Gambling in Young People. J. Gambl. Stud. 2010, 26, 175–187. [Google Scholar] [CrossRef]
- King, D.L.; Delfabbro, P.H. Video Game Monetization (e.g., ‘Loot Boxes’): A Blueprint for Practical Social Responsibility Measures. Int. J. Ment. Health Addict. 2019, 17, 166–179. [Google Scholar] [CrossRef]
- Starcevic, V. Problematic Internet Use: A Distinct Disorder, a Manifestation of an Underlying Psychopathology, or a Troublesome Behaviour? World Psychiatry 2010, 9, 92–93. [Google Scholar] [CrossRef] [PubMed]
- Starcevic, V.; Billieux, J. Does the Construct of Internet Addiction Reflect a Single Entity or a Spectrum of Disorders? Clin. Neuropsychiatry J. Treat. Eval. 2017, 14, 5–10. [Google Scholar]
- Young, K. Internet Addiction: Diagnosis and Treatment Considerations. J. Contemp. Psychother. 2009, 39, 241–246. [Google Scholar] [CrossRef]
- Przybylski, A.K.; Weinstein, N.; Murayama, K. Internet Gaming Disorder: Investigating the Clinical Relevance of a New Phenomenon. Am. J. Psychiatry 2017, 174, 230–236. [Google Scholar] [CrossRef] [PubMed]
- Kuss, D.J.; Griffiths, M.D. Internet and Gaming Addiction: A Systematic Literature Review of Neuroimaging Studies. Brain Sci. 2012, 2, 347–374. [Google Scholar] [CrossRef] [PubMed]
- Lemmens, J.S.; Valkenburg, P.M.; Peter, J. Psychosocial Causes and Consequences of Pathological Gaming. Curr. Res. Top. Cogn. Load Theory 2011, 27, 144–152. [Google Scholar] [CrossRef]
- Holmgren, H.G.; Coyne, S.M. Can’t Stop Scrolling!: Pathological Use of Social Networking Sites in Emerging Adulthood. Addict. Res. Theory 2017, 25, 375–382. [Google Scholar] [CrossRef]
- Griffiths, M.D.; Parke, J. Adolescent Gambling on the Internet: A Review. Int. J. Adolesc. Med. Health 2010, 22, 59–75. [Google Scholar]
- King, D.L.; Delfabbro, P.H.; Griffiths, M.D. Video Game Addiction. In Principles of Addiction; Elsevier, 2013; pp. 819–825 ISBN 978-0-12-398336-7.
- Upadhyay, U.; Kumar, A.; Sharma, G.; Gupta, B.B.; Alhalabi, W.A.; Arya, V.; Chui, K.T. Cyberbullying in the Metaverse: A Prescriptive Perception on Global Information Systems for User Protection. J. Glob. Inf. Manag. JGIM 2023, 31, 1–25. [Google Scholar] [CrossRef]
- Vandebosch, H.; Van Cleemput, K. Cyberbullying among Youngsters: Profiles of Bullies and Victims. New Media Soc. 2009, 11, 1349–1371. [Google Scholar] [CrossRef]
- Barreda-Ángeles, M.; Hartmann, T. Hooked on the Metaverse? Exploring the Prevalence of Addiction to Virtual Reality Applications. Front. Virtual Real. 2022, 3. [Google Scholar] [CrossRef]
- Fox, J.; Tang, W.Y. Women’s Experiences with General and Sexual Harassment in Online Video Games: Rumination, Organizational Responsiveness, Withdrawal, and Coping Strategies. New Media Soc. 2017, 19, 1290–1307. [Google Scholar] [CrossRef]
- Tokunaga, R.S. Following You Home from School: A Critical Review and Synthesis of Research on Cyberbullying Victimization. Comput. Hum. Behav. 2010, 26, 277–287. [Google Scholar] [CrossRef]
- Rzeszewski, M.; Evans, L. Social Relations and Spatiality in VR - Making Spaces Meaningful in VRChat. Emot. Space Soc. 2024, 53, 101038. [Google Scholar] [CrossRef]
- Odeleye, B.; Loukas, G.; Heartfield, R.; Sakellari, G.; Panaousis, E.; Spyridonis, F. Virtually Secure: A Taxonomic Assessment of Cybersecurity Challenges in Virtual Reality Environments. Comput. Secur. 2023, 124, 102951. [Google Scholar] [CrossRef]
- Zaeem, R.N.; Barber, K.S. The Effect of the GDPR on Privacy Policies: Recent Progress and Future Promise. ACM Trans Manage Inf Syst 2020, 12. [Google Scholar] [CrossRef]
- Moreno-Arjonilla, J.; López-Ruiz, A.; Jiménez-Pérez, J.R.; Callejas-Aguilera, J.E.; Jurado, J.M. Eye-Tracking on Virtual Reality: A Survey. Virtual Real. 2024, 28, 38. [Google Scholar] [CrossRef]
- Dholakia, N.; Darmody, A.; Zwick, D.; Dholakia, R.R.; Fırat, A.F. Consumer Choicemaking and Choicelessness in Hyperdigital Marketspaces. J. Macromarketing 2021, 41, 65–74. [Google Scholar] [CrossRef]
- Westerlund, M. The Emergence of Deepfake Technology: A Review. Technol. Innov. Manag. Rev. 2019, 9, 39–52. [Google Scholar] [CrossRef]
- Alcañiz, M.; Bigné, E.; Guixeres, J. Virtual Reality in Marketing: A Framework, Review, and Research Agenda. Front. Psychol. 2019, 10. [Google Scholar] [CrossRef] [PubMed]
- Arias, O.; Wurm, J.; Hoang, K.; Jin, Y. Privacy and Security in Internet of Things and Wearable Devices. IEEE Trans. Multi-Scale Comput. Syst. 2015, 1, 99–109. [Google Scholar] [CrossRef]
- Çelik, F.; Çam, M.S.; Koseoglu, M.A. Ad Avoidance in the Digital Context: A Systematic Literature Review and Research Agenda. Int. J. Consum. Stud. 2023, 47, 2071–2105. [Google Scholar] [CrossRef]
- Zhou, X.; Zafarani, R.; Shu, K.; Liu, H. Fake News: Fundamental Theories, Detection Strategies and Challenges. In Proceedings of the Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining; Association for Computing Machinery: New York, NY, USA, 2019; pp. 836–837. [Google Scholar]
- Devereaux, A. The Digital Wild West: On Social Entrepreneurship in Extended Reality. J. Entrep. Public Policy 2021, 10, 198–217. [Google Scholar] [CrossRef]
- Ferrara, E. Disinformation and Social Bot Operations in the Run up to the 2017 French Presidential Election. First Monday 2017, 22. [Google Scholar] [CrossRef]
- Ryan Bengtsson, L.; Van Couvering, E. Stretching Immersion in Virtual Reality: How Glitches Reveal Aspects of Presence, Interactivity and Plausibility. Convergence 2023, 29, 432–448. [Google Scholar] [CrossRef]
- McDougall, J. Media Literacy versus Fake News: Critical Thinking, Resilience and Civic Engagement. Medijske Stud. 2019, 10, 29–45. [Google Scholar] [CrossRef]
- Kalyvaki, M. Navigating the Metaverse Business and Legal Challenges: Intellectual Property, Privacy, and Jurisdiction. J. Metaverse 2023, 3, 87–92. [Google Scholar] [CrossRef]
- Guo, J.; Weng, D.; Fang, H.; Zhang, Z.; Ping, J.; Liu, Y.; Wang, Y. Exploring the Differences of Visual Discomfort Caused by Long-Term Immersion between Virtual Environments and Physical Environments. In Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR); March 22 2020; pp. 443–452. [Google Scholar]
- Hirzle, T.; Fischbach, F.; Karlbauer, J.; Jansen, P.; Gugenheimer, J.; Rukzio, E.; Bulling, A. Understanding, Addressing, and Analysing Digital Eye Strain in Virtual Reality Head-Mounted Displays. ACM Trans Comput-Hum Interact 2022, 29. [Google Scholar] [CrossRef]
- Arif, U.; Khan, R.H.; Khan, A.A. Musculoskeletal Disorders and Visual Symptoms Among Virtual Reality Headset Users. In Proceedings of the Ergonomics for Improved Productivity; Muzammil, M., Khan, A.A., Hasan, F., Eds.; Springer Singapore: Singapore, 2021; pp. 821–829. [Google Scholar]
- Hribernik, M.; Umek, A.; Tomažič, S.; Kos, A. Review of Real-Time Biomechanical Feedback Systems in Sport and Rehabilitation. Sensors 2022, 22. [Google Scholar] [CrossRef] [PubMed]
- Putranto, J.S.; Heriyanto, J.; Kenny; Achmad, S. ; Kurniawan, A. Implementation of Virtual Reality Technology for Sports Education and Training: Systematic Literature Review. 7th Int. Conf. Comput. Sci. Comput. Intell. 2022 2023, 216, 293–300. [Google Scholar] [CrossRef]
- Hilbert, M. The Bad News Is That the Digital Access Divide Is Here to Stay: Domestically Installed Bandwidths among 172 Countries for 1986–2014. Telecommun. Policy 2016, 40, 567–581. [Google Scholar] [CrossRef]
- Scheerder, A.J.; van Deursen, A.J.; van Dijk, J.A. Internet Use in the Home: Digital Inequality from a Domestication Perspective. New Media Soc. 2019, 21, 2099–2118. [Google Scholar] [CrossRef]
- Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A Systematic Review of Immersive Virtual Reality Applications for Higher Education: Design Elements, Lessons Learned, and Research Agenda. Comput. Educ. 2020, 147, 103778. [Google Scholar] [CrossRef]
- Attaran, S.; Attaran, M.; Celik, B.G. Digital Twins and Industrial Internet of Things: Uncovering Operational Intelligence in Industry 4.0. Decis. Anal. J. 2024, 10, 100398. [Google Scholar] [CrossRef]
- Slater, M.; Spanlang, B.; Sanchez-Vives, M.V.; Blanke, O. First Person Experience of Body Transfer in Virtual Reality. PLOS ONE 2010, 5, e10564. [Google Scholar] [CrossRef]
- Yadav, A.; Reddy, K.G.N. Virtual Dilemmas: Legal and Ethical Rollercoasters in Immersive Tech Land. In Multidisciplinary Applications of Extended Reality for Human Experience; Kajla, T., Kansra, P., Singh, N., Eds.; IGI Global: Hershey, PA, USA, 2024; pp. 64–80. ISBN 9798369324325. [Google Scholar]
- Freeman, D.; Haselton, P.; Freeman, J.; Spanlang, B.; Kishore, S.; Albery, E.; Denne, M.; Brown, P.; Slater, M.; Nickless, A. Automated Psychological Therapy Using Immersive Virtual Reality for Treatment of Fear of Heights: A Single-Blind, Parallel-Group, Randomised Controlled Trial. Lancet Psychiatry 2018, 5, 625–632. [Google Scholar] [CrossRef] [PubMed]
- Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Validation of the Virtual Reality Everyday Assessment Lab (VR-EAL): An Immersive Virtual Reality Neuropsychological Battery with Enhanced Ecological Validity. J. Int. Neuropsychol. Soc. 2021, 27, 181–196. [Google Scholar] [CrossRef] [PubMed]
- Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. An Ecologically Valid Examination of Event-Based and Time-Based Prospective Memory Using Immersive Virtual Reality: The Effects of Delay and Task Type on Everyday Prospective Memory. Memory 2021, 29, 486–506. [Google Scholar] [CrossRef] [PubMed]
- Kourtesis, P.; MacPherson, S.E. An Ecologically Valid Examination of Event-Based and Time-Based Prospective Memory Using Immersive Virtual Reality: The Influence of Attention, Memory, and Executive Function Processes on Real-World Prospective Memory. Neuropsychol. Rehabil. 2021, 1–26. [Google Scholar] [CrossRef] [PubMed]
- Melo, M.; Vasconcelos-Raposo, J.; Bessa, M. Presence and Cybersickness in Immersive Content: Effects of Content Type, Exposure Time and Gender. Comput. Graph. 2018, 71, 159–165. [Google Scholar] [CrossRef]
- Santos, M.; Sequeira, T.N.; Ferreira-Lopes, A. Income Inequality and Technological Adoption. J. Econ. Issues 2017, 51, 979–1000. [Google Scholar] [CrossRef]
- Chen, X.; Xie, H.; Zou, D.; Hwang, G.-J. Application and Theory Gaps during the Rise of Artificial Intelligence in Education. Comput. Educ. Artif. Intell. 2020, 1, 100002. [Google Scholar] [CrossRef]
Figure 1.
The Virtual Continuum: From the Physical to the Virtual, with AR and MR blending elements of both worlds.
Figure 1.
The Virtual Continuum: From the Physical to the Virtual, with AR and MR blending elements of both worlds.
Table 1.
Multimodal Systems Used in Extended Reality.
| Modality | Primary Function | Applications | Example |
|---|---|---|---|
| Eye-Tracking | Tracks visual attention and cognitive load | Education, professional training, gaming, healthcare | Adapting content based on gaze in training simulations, adjusting difficulty based on cognitive load |
| Facial Tracking | Captures and replicates real-time facial expressions to enhance emotional and social presence | Virtual workspaces, social hangouts, gaming, therapy | Avatars mirroring users’ facial expressions for more natural interaction |
| Hand/Finger Tracking | Allows intuitive manipulation of virtual objects using hand and finger gestures | Professional training, collaborative work, gaming, creative industries | Manipulating virtual objects in real time during design sessions or medical simulations |
| Full-Body Tracking | Tracks full-body movements for realistic avatar replication and physical engagement | Fitness, physical therapy, social interaction, sports training | Performing physical exercises in a virtual fitness program, analyzing posture in physical therapy |
| Haptic Feedback | Provides tactile sensations to simulate the feeling of virtual objects and environments | Training simulations, gaming, therapy, creative industries | Feeling resistance when handling virtual tools in mechanical training or receiving tactile feedback in creative tasks |
| Galvanic Skin Response (GSR) | Monitors emotional states through skin conductance linked to arousal levels | Therapy, stress management, emotional engagement | Adjusting virtual environments in real time based on stress or relaxation levels detected via GSR |
| Heart Rate Monitoring | Tracks physiological engagement through heart rate changes | Virtual fitness, stress management, emotional engagement | Adjusting workout intensity in virtual fitness programs based on heart rate feedback |
| EEG (Electroencephalogram) | Measures brain activity and cognitive states for brain-computer interaction (BCI) | Accessibility, cognitive control, education, professional training | Allowing users to control virtual objects with brain activity or adjusting task difficulty based on cognitive load data |
Table 2.
XR Applications with Multimodal Integration.
| Domain | Key Applications | Multimodal Integration | Example |
|---|---|---|---|
| Clinical | Therapy, neuropsychological assessment, physical rehabilitation, pain management | Eye-tracking, GSR, EEG, haptic feedback | XR exposure therapy for anxiety disorders, stroke rehabilitation with motor skill tracking |
| Education | STEM education, history, language learning, special education | Hand-tracking, eye-tracking, haptic feedback, EEG | Virtual dissections in biology, historical reenactments, personalized learning for students with learning disabilities |
| Professional Training | Medical training, emergency response, manufacturing, corporate skill development | Hand-tracking, haptic feedback, EEG, GSR | XR simulations for surgical training, crisis management in disaster scenarios, leadership training |
| Arts & Entertainment | Gaming, virtual art galleries | Facial tracking, hand-tracking, haptic feedback, eye-tracking | Immersive multiplayer gaming with avatar mirroring, virtual museum experiences with tactile feedback |
| Public Health & Safety | Disaster preparedness, public health campaigns, workplace safety | Eye-tracking, hand-tracking, haptic feedback, full-body tracking | Virtual earthquake simulations for first responders, XR for health safety training |
| Retail & E-commerce | Virtual shopping, product customization, enhanced customer insights | Hand-tracking, eye-tracking, haptic feedback | Virtual clothing try-ons with full-body tracking, real-time eye-tracking for personalized shopping experiences |
| Architecture & Urban Planning | Virtual walkthroughs, collaborative design, safety and risk assessment | Hand-tracking, haptic feedback, eye-tracking, full-body tracking | Virtual tours of proposed buildings with real-time adjustments, AR overlays for community feedback |
| Sports Training | Skill development, mental training, performance analysis | Full-body tracking, heart rate monitoring, GSR, EEG | XR for tennis serve practice with haptic feedback, mental conditioning with GSR and EEG monitoring |
| Agriculture & Environment | Precision agriculture, conservation, environmental monitoring | Hand-tracking, haptic feedback, eye-tracking, GSR | Virtual crop management simulations, immersive environmental impact assessments |
| Remote Work | Virtual meetings, remote collaboration, training, and onboarding | Eye-tracking, hand-tracking, full-body tracking, facial tracking | Interactive virtual office spaces with real-time avatar mirroring and hand-tracking |
| Museums & Cultural Heritage | Virtual museums, cultural preservation, multimodal engagement in museums | Eye-tracking, hand-tracking, haptic feedback | Digital reconstruction of endangered cultural sites, interactive museum exhibits with personalized tours |
Table 3.
Potential Risks and Ethical Challenges in XR and the Metaverse.
| Risk | Primary Concern | Implications | Examples |
|---|---|---|---|
| Cybersickness | Physical discomfort due to sensory conflict between visual input and lack of corresponding physical movement | Decreased engagement, reduced productivity, and hindered adoption of XR technologies in training or education settings | Users experience dizziness, nausea, and headaches during long VR training sessions, limiting their effectiveness |
| Addiction | Psychological dependence on immersive virtual environments | Neglect of real-world responsibilities, social isolation, and potential mental health deterioration | Users spend excessive time in virtual spaces, leading to poor academic or work performance and weakened real-life relationships |
| Cyber Harassment & Bullying | Anonymity and lack of consequences in virtual environments lead to harmful behaviors | Increased emotional distress, anxiety, and psychological harm due to virtual harassment or bullying | Virtual groping or abusive language in social XR spaces, with victims feeling violated and emotionally distressed |
| Data Privacy & Security | Collection of sensitive biometric, behavioral, and emotional data presents risks of misuse or hacking | Unauthorized access to personal data, including identity theft or surveillance, and potential manipulation of user behavior | Biometric data like eye-tracking or EEG used to profile users and influence decision-making without their knowledge |
| Intense Advertising | Hyper-targeted and manipulative advertising based on user data | Manipulation of users’ emotions and cognitive states, commercial exploitation, and over-commercialization of virtual spaces | Real-time emotional responses used to deliver targeted ads that exploit vulnerable moments, leading to compulsive purchases |
| Manipulation of Public Opinion | Misinformation and deepfake content spread through immersive experiences | Influence on political or social views, creating echo chambers and distorting users’ perceptions of reality | Deepfake avatars of political figures delivering fake speeches in virtual environments, confusing users about real-world events |
| Physical Health Concerns | Strain from prolonged use of XR headsets and repetitive movements | Eye strain, musculoskeletal problems, and sedentary lifestyle leading to long-term health issues | Users experience headaches and discomfort due to prolonged headset use or poor posture during VR interactions |
| Digital Divide & Inequality | Economic and infrastructure barriers limit access to XR technologies | Widening social and economic gaps, unequal access to education, healthcare, and workforce opportunities | Lower-income communities unable to access XR-enhanced education, resulting in disadvantaged students |
| Psychological Detachment | Prolonged exposure to immersive environments blurs the line between virtual and real worlds | Reality confusion, dissociation, and emotional distress, particularly in vulnerable populations | Users struggle to distinguish between virtual events and real-world experiences, leading to cognitive disorientation |
| Regulatory Challenges | Lack of comprehensive legal frameworks governing data privacy, content moderation, and intellectual property | Difficulty in enforcing user protections across decentralized and global XR platforms | Jurisdictional issues in handling virtual harassment or disputes over digital assets in cross-border XR environments |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated