Preprint
Article
Transforming Experiential Learning: The Role of Advanced Human-Computer Interaction Technologies in Modern Education and Hands-On Learning
Altmetrics
Downloads
117
Views
60
Comments
0
This version is not peer-reviewed
Simulations and Applications of Augmented and Virtual Reality, 2nd Edition
Submitted:
08 July 2024
Posted:
08 July 2024
You are already at the latest version
Alerts
Abstract
Experiential learning, defined as the result, source, or context of knowledge obtained from direct experience, is increasingly bolstered by technology and human-computer interaction (HCI) principles and protocols. This article will describe how technology-based human-computer interaction (HCI) has been used in experiments to enhance direct knowledge experience. In this paper, immersive technologies (such as Virtual Reality, Augmented Reality, and haptics) will be described based on cognitive learning theory, empirical research and practical experience gained from implementing cutting-edge technology and maximizing HCI principles in constructive learning. Additionally, this article will describe how AI can be used to create individualized learning environments according to a student's needs in the future. While cost, technical issues and specialised trainings are the obstacle to deliver personalized learning experiences especially to economically disadvantaged students, technological change opportunities are projected, and these technologies will take educational outcomes to the next level.
Keywords:
Subject: Computer Science and Mathematics - Information Systems
I. Introduction
Experiential learning (EL), is learning that takes place outside of the traditional academic setting to acquire knowledge, develop skills and internalize values, has become one of the most popular and trending approaches to learning today. With the advancement of human-computer interaction (HCI) technology, EL is expected to be further empowered and revolutionized by HCI principles and extensive research. The article discusses the potential of this technology-orientated human-computer interaction learning to raise the EL to a higher level, reviews the cutting edge literatures and use cases (analysis and evaluation), addresses the issue of effectiveness (measuring and benchmark), identifies the challenges and proposes further opportunities, and sheds light on future trend.
Using HCI principles in experiential learning can create immersive learning environments that break the constraints of traditional education. As a form of multimodal learning, virtual experience-based learning fits into the framework of cognitive learning theory, which underscores the stabilisation of learning and a faster internalization of information.
II. Literature Review
A. Cognitive Learning Theory and HCI
Cognitive Learning Theory emphasizes that deeper learning can only happen when there is experience. The more senses are active during learning activities, the easier it is for information to be internalized. Why is this true? We can assume that one reason is that it is more fun to experience something through our senses. This might explain why VR and AR technologies enhance depth of learning. Cognitivism sees learning as a function of internal processing with the understanding that our brain processes information and stores information in our working memory. Information is passed back and forth between our sensory system in the processes of experiencing lived realities. If a learning event is more immersive and experiential, users are more likely to engage and pay attention.
VR and AR technologies influence cognitive learning by providing users with an immersive and interactive experience that appears real. As a result of being exposed to VR and AR, learners are more engaged in the learning activities, which enhances the deep understanding of ideas and creates ‘rich associations’ of concepts. When VR and AR are used for educational purposes, creating a rich association of ideas by learners, prior knowledge of the event is also added. Learners are making associations with new information by relating it to existing knowledge.
For example, the immersive and interactive experience provided by VR and AR in an immersive environment allows users to fully engage and experience flight simulation. As the virtual environment comes closer to a real-world experience, learners are provided with the opportunity to fly like a real pilot, as if they are really in the flight simulation.
Similarly, the study - "Cognitive Affective Model of Immersive Learning (CAMIL)" argues that hands-on experience of flight simulation is more likely to enable fast assimilation and learning. In addition, AR and VR technologies can adapt and personalize teaching and learning for individuals as stated by Meyer et al., (2019) and Makransky et al., (2020a) . Being personalized, students can have a ‘deeper level of understanding of complex concepts and ideas through interactive simulations’ (Zhang & Wang, 2021). AR and VR have also been used to customize learning by allowing students to learn at their own pace and in their own style. We have also seen instances and use cases where AR and VR technologies are deployed to provide personalized feedback so tutors can identify areas for their students to improve and correct their weaknesses. These findings underline the potential of immersive technologies to revolutionize educational practices by providing rich, sensory learning environments that align with Cognitive Learning Theory.
B. Empirical Evidence of HCI in Education
Among the most notable studies is the work of Wang et al. (2018), who demonstrated significant benefits of VR in medical training. The researchers found that 95 per cent of participants thought they had ‘a better understanding of complex medical procedures’ when trained via VR versus traditional approaches. ‘Watching or reading text provided by a surgeon does not come close to replicating the experience of being able to feel … the tension’, wrote the study’s lead authors. ‘
Another study, this one a meta-analysis by Merchant et al. (2014), found that students who learnt through AR scored 22 per cent higher on achievement tests, compared with students learning in traditional environments. And this is not the only study to show that, when it comes to medical students, the benefits of VR extend beyond immediate learning to long-term retention. Another recent meta-analysis found that knowledge retention six months after instruction is 16 per cent higher in participants who have undergone VR-based medical training than those who have used traditional methods of learning.
These studies highlight how the application of empirical principles of technology-based HCI can dramatically ‘change the way people learn’. This further gives credence that VR in future, will be eminently useable in creating effective learning tools that allow learners to perceive, engage with, and comprehend content in far more detailed, accurate, and direct ways than ever before. We also use VR to create perceptually realistic simulations of physical objects so students can practice these real-world engagements without physical risks. VR/AR is like a magnifying glass, concentrating and focusing light for the student’s eyes and allowing him or her to see and examine, in detail, something she would not see in its pure form. A similar phenomenon is taking place with AR, with students are gaining insights that would otherwise be lost in more traditional contexts. As a result, the educational possibilities of these technologies are clearly high, and VR/AR tools could have the power to absolutely transform such educational efforts.
III. Methodology
The analysis applies a mixed methods that combines quantitative findings from the available empirical studies with qualitative data gathered from case studies. Effectiveness is measured by performance metrics, levels of engagement and retention. Sources of data include peer-reviewed academic journals, government and nonprofit education reports and feedback from educational institutions employing HCI technologies.
A mixed-method approach to this study has the benefit of providing a well-rounded and meaningful understanding of a topic by catering to the strength of each individual method of data collection. Using both quantitative and qualitative data results in a stronger analysis, which in turn improves validity and reliability. Quantitative statistics provide information about populations and allow for extrapolation, while qualitative insight provides context and detail.
VI. Analysis and Discussion
A. Use Cases of Technology-Based HCI
A number of use cases demonstrates the practicality and efficacy of the technology-enabled HCI for experiential learning: For example, learning resources can be provided through immersive virtual tours powered by 3D scanning technology as presented in Times Higher Education article, “How to Use 3D Scanning Technology to Create Virtual Tours for Your Students” (2024). These tours provide students with the opportunity to directly experience the real world space and environment through their computers or VR goggles, and are created through 3D scanning devices. Universities are scanning real historical sites and laboratories to provide these virtual tours for the students to explore and understand the intricate structure and layout of the study-areas. In a more recent example, 3D scanners were employed to create immersive exhibits for students studying Roman civilization (ScienceDaily.com, 2022).
Figure I.
Snapshot of a virtual scan using Matterport1.
Figure I.
Snapshot of a virtual scan using Matterport1.
Engineers are familiar with techniques such as this where a virtual tour of a manufacturing plant or process is created that takes them through the motions or operations of complex pieces of machinery and production lines without the need to visit a factory. This again broadens access to learning, but in addition it can now be done in a safe and less expensive way that brings the benefits of hands-on learning with minimal expense and an enhanced ability to structure experiences. In addition to this, online learning is being developed that combines 3D scanning and virtual tour technologies to show how theory is applied in practice and to make learning more engaging and effective (Makransky & Petersen, 2021).
Similarly, flight-training students at Embry-Riddle Aeronautical University in Prescott, Arizona use HCI-driven simulations for emergency landing procedures before performing them in actual aircraft, using VR flight simulation. According to Tianxin Zhang’s dissertation, ‘Comparing Training Effects of Virtual Reality Flight Simulation to Conventional PC-Based Flight Simulation’ (2017): The VR-based simulation provides an immersive training environment eliminating real-world risks and disruptions. It not only significantly improves students’ preparedness for real-world aviation crises where human error is often the factor in a catastrophic failure, but it also presents a more comprehensive approach to teaching students an entirely new skillset.The Zhang study found that VR training produced better post-training manoeuvres than that offered by conventional desktop simulations, with better user experience, more motivation and higher satisfaction. At Embry-Riddle, students can use VR flight simulation to move away from the traditional learning divide that often occurs between the classroom and the cockpit, opening up new possibilities for how people learn.
The use cases above illustrate how experiential learning through HCI can close the theory-to-practice divide and provide a more interactive and effective learning experience for students.
B. Measuring Effectiveness
Improvements in student performance, engagement and retention rates can clearly demonstrate effectiveness. Studies such as Hoffman et al (2016) observed that HCI-enhanced learning environments resulted in a 30 per cent higher retention rate than more traditional methods. Surveys and feedback loops offer additional valuable qualitative perspectives, while analytics tools can track simple measures of engagement, for example how much time students spend with an interactive module or how often they return.
A study published in 2023 in Humanities and Social Sciences Communications found that involving students in immersive technologies such as augmented reality increases learning retention and critical thinking, especially effective in STEAM or Science, Technology, Engineering, Arts and Mathematics programmes. The study further adds credence to Alkhabra et al., (2023)'s research which demonstrated how educators are accomplishing cognitive outcomes in educational engagements that incorporate immersive technologies: in a physics class, students were made to visualize and interact with complex scientific concepts, such as the laws of motion and electromagnetic fields and effects.
Similarly, another research conducted and published in the International Journal of Educational Technology in Higher Education found that educational technology such as VR and AR promotes engagement on the cognitive, behavioural and affective levels. According to the findings of the study by Bond et al., (2020), discussion forums, multimedia resources, Learning Management System (LMS), and the likes are extremely valuable tools for enhancing student interaction, participation and learning satisfaction. They improve the efficacy of HCI-driven educational practices. The results demonstrate that students who had adequate access to the tools were more likely to engage with learnings, with better performance and being more likely to stay with the learning and progressing further.
Figure II.
Engagement and disengagement by teaching tool typology2.
Figure II.
Engagement and disengagement by teaching tool typology2.
C. Challenges and Opportunities
There are lot of great benefits of using it, but on the other hand there are almost as many challenges when it comes to this subject. The main challenges of using VR and AR in education are high costs of the equipment needed, technical glitches and also need for specialist training not only for the users but also for educators. The main disadvantage of the VR and AR equipment is the high costs and many technical glitches with using it. However, the immersive and interactive experience that you can get with VR and AR are also very high and comparing to traditional resources, they can offer unique learning opportunities.
Tutors and instructors sometimes face technical difficulties such as system crashes, poor latency or incompatibility between hardware and software, which may pose obstacles to deploying VR and AR in learning. Such technical aspects may disrupt learning and require additional technical support to ensure smooth operation, which adds another layer of complexity to teaching through VR and AR. A study highlighted that both these technical issues and high initial cost ‘are still the major barriers for the adoption’ of such technologies in an educational setting. (Vats and Joshi, 2023)
As Dick E (2021) outlines, we should solve these problems by thinking strategically about long-term investment in infrastructure, by continual professional development for educators and with the ongoing development of a universal test to measure outputs of learning. This way, Dick (2021) concludes, ‘they can realise the potential of VR and AR in teaching’, and turn the world of education into ‘an exciting and effective place’ once more.
These challenges are also opportunities: developing cheaper VR solutions or smarter user interfaces, for example, can bring wider adoption. The added degree of immersion provided by VR and AR offers new ways of learning that can not be obtained through traditional books or even pictures and videos. VR and AR allow students to navigate in a three-dimensional virtual world, handle virtual objects in 3D and simulate situations. Students can interact with virtual content and then reflect upon and apply what they have learnt to real-world scenarios. For example, they could learn a chemical reaction by observing a giant atom falling down and getting smaller over the process and, then, simulate the reaction in a laboratory. On the collaborative side, VR and AR can enhance students’ skills by encouraging discussions and presentations while working in teams in the same virtual world. In this way, students are no longer limited by geographical boundaries, as they can interact with each other in virtual rooms and see, at the same time, how their virtual worlds have changed during the simulation.
Furthermore, partnerships between education and tech companies can promote equity and excellence in the use of VR and AR in education. It can also promote equity by providing equal access to educational resources using VR and AR, and by encouraging collaboration across different institutions of learning. Tapping into some existing infrastructure, such as school networks and digital resources, will lower the considerable costs associated with implementing VR and AR technologies in educational institutions while tendering to the specific needs of educators and learners.
V. Future Trends
A. Integration of AI and 5G
AI can work with technologies of immersion such as VR and AR to adapt content to the needs of an individual learner. Through seeing a student’s past behaviour and real-time context, AI can tailor content to the students in front of it today so that each student has a learning experience suited to their individuality. 5G, with its bandwidth and speed, can optimise this personalised experience further.
Lee (2020) says: This combination of AI and 5G enables the creation of highly immersive and adaptive learning environments, where AI analyses a student’s performance and preferences as well as adapting the difficulty level of the VR/AR simulation accordingly. This should make learning more engaging, enabling a deeper learning about STEM and other complex subjects.
Moreover, in A Beginner’s Guide to Introduce Artificial Intelligence in Teaching and Learning (2023), Kurni et al. point out that: ‘With the low latency and high data traffics of 5G, the use of high-quality, interactive VR is more possible. It will be a great experience for students as using this type of VR will give them an experimental activity, such as doing an experiment in a virtual lab or visiting the destination that is far located and not many students can experience in real life. This type of experiment would be impossible to do similarly with other methods or with slower internet connection.’ Practically, such connections will allow students to carry out simulated laboratory experiments without actual laboratory facilities, or visit remote areas such as the rainforest without leaving their classrooms – immersive experiences that just haven’t been possible before for various technical reasons.
Indeed other than escalating the educational process, theses products can also improve the accessibility. Specially, VR and AR generate an opportunity for students who cannot get access to a specific institution to have a immersive experiences or take part in the specific activities for the educational process as observed in Verizon’s 5G business use cases (Immersive Virtual Reality Education: 5G Use Cases, n.d.). It is a clear indication that AI-driven applications can generate real-time feedback and summative assessment for the students in a ways of helping them to stay focused or on track, as well as detect the areas that they need to improve.
In conclusion, with the help of AI, VR, AR, and 5G, education can become more accessible, personalised and interesting. As these technologies develop and expand, their influence on education will continue to expand, ultimately changing the way we teach and learn completely.
B. Development of Haptic Feedback Technology
Haptic feedback technology is set to further enhance experiential learning through HCI by providing feedback on the interactions of a user with a virtual simulation by using manually sensed forces, vibrations, and motions. Haptic feedback has been used to make training successful for years by enabling users to feel the outcomes of their actions in a virtual simulation space. For example, it enables airline pilots to better control a flight simulator by providing physical feedback to trainee pilots that mirror their own flight motions, thereby making the virtual simulation feel more real and enhancing the potential to learn effectively through immersion.
In education, haptic feedback technology could help students learn and interact more meaningfully with digital environments, for example through the Vrgal research article (2024) that explained how one company is using it in medical simulation to help students ‘feel’ their procedures during surgery.
Besides, as Campbell (2023) showed: engineering and design courses can be reimagined so as to enable students to feel and touch virtual prototypes, or to visualise the underlying principles and concepts with their own virtual hands, a dramatic shift of how learning is done.
With the incorporation of haptic feedback, learning experiences become more immersive and effective, raising the possibility for live, in-class simulations where students are given the opportunity to interact with virtual spaces as if they were actually real.
VI. Conclusion
By leveraging technological theories behind Human-Computer Interaction (HCI), such as Virtual Reality, Augmented Reality, Artificial Intelligence and haptic feedback, experiential learning will be revolutionised as it will have interactive, immersive, and customisable environments that will enhance and personalise how learning experiences are designed. However, the use of immersion as proposed in Cognitive Learning Theory will definitely be enhanced by the use of these technological inventions that suggests that through multisensory experiences it is possible for students to acquire deeper knowledge, do better retention of what they encounter and also better understand what is taught to them.
Since these technologies work, evidence is growing. Higher educational outcomes have been measured at students; higher retention and understanding of more complex concepts; how a particular drug behaves in the body. A learning experience taught using immersive technology, such as VR or AR can be more accurately directed to individual needs using AI, and readily streamed using 5G technology.
Despite the great potential, financial and infrastructure costs, technical difficulties, training of teachers are roadblocks in this initiative. At the same time, these elements can drive innovation and co-operation between schools and high-tech companies. Strategic investment, continual training, and standardised assessment methodologies can mitigate such barriers.
In the future, through miniaturisation and adoption of AI, VR, AR and 5G into education community, the artificial world would become more self-adaptive, customizable and higher fidelity. Eventually, our acoustic, visual, tactile, and other sensitive sources of information would be gathered and presented to us through haptic feedback. Our senses would be continuously stimulated as though we were in the real world. When all these technologies are commercialized, they would potentially transform the teaching environment and learning experience of our students, thus improving the efficiency and inclusiveness of education through unparalleled immersive experiences that merge the boundaries between the real and the artificial worlds.
References
- Hoffman, H. G., et al. (2016). Retention rates in HCI-enhanced learning environments. Journal of Educational Technology.
- Merchant, Z., et al. (2014). Effectiveness of augmented reality in educational settings: A meta-analysis. Educational Research Review.
- Wang, R., et al. (2018). The impact of virtual reality on medical training: A comprehensive study. Medical Education Journal.
- Makransky, G., & Petersen, G. B. (2021). The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality. Educational Psychology Review, 33(3), 937–958. [CrossRef]
- Zhang, W., & Wang, Z. (2021). Theory and Practice of VR/AR in K-12 Science Education—A Systematic Review. Sustainability, 13(22), 12646. [CrossRef]
- How to use 3D scanning technology to create virtual tours for your students. (2024, May 14). THE Campus Learn, Share, Connect. https://www.timeshighereducation.com/campus/how-use-3d-scanning-technology-create-virtual-tours-your-students.
- Alkhabra, Y. A., Ibrahem, U. M., & Alkhabra, S. A. (2023). Augmented reality technology in enhancing learning retention and critical thinking according to STEAM program. Humanities & Social Sciences Communications, 10(1). [CrossRef]
- Bond, M., Buntins, K., Bedenlier, S., Zawacki-Richter, O., & Kerres, M. (2020). Mapping research in student engagement and educational technology in higher education: a systematic evidence map. International Journal of Educational Technology in Higher Education, 17(1). [CrossRef]
- Vats, S., & Joshi, R. (2023). The Impact of Virtual Reality in Education: A Comprehensive Research Study. In IFIP advances in information and communication technology (pp. 126–136). [CrossRef]
- Dick, E. (2021, August 30). The Promise of Immersive Learning: Augmented and Virtual Reality’s Potential in Education. Itif.org. https://itif.org/publications/2021/08/30/promise-immersive-learning-augmented-and-virtual-reality-potential/.
- Lee, L. (2020, September 2). Back to the Future Classroom: VR/AR/AI Transformation — EqOpTech. EqOpTech. https://www.eqoptech.org/publications/2020/8/22/back-to-the-future-classroom-vrarai-transformation.
- Vrgal. (2024, January 10). VR Medical Training: Revolutionizing Healthcare Education | Experience the limitless world of Virtual. Experience the Limitless World of Virtual Reality With VR Gal at vrgal.net. https://vrgal.net/vr-medical-training/.
- Kurni, M., Mohammed, M. S., & Srinivasa, K. G. (2023c). AR, VR, and AI for Education. In Springer eBooks (pp. 115–136). [CrossRef]
- Jagannathan, S. (2024b, March 16). How can 5G make a difference to education? World Bank Blogs. https://blogs.worldbank.org/en/digital-development/how-can-5g-make-difference-education.
| 1 | Times Higher Education (“How to Use 3D Scanning Technology to Create Virtual Tours for Your Students,” 2024
|
| 2 | Mapping research in student engagement and educational technology in higher education: a systematic evidence map (https://doi.org/10.1186/s41239-019-0176-8)
|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated