Submitted:
12 December 2024
Posted:
14 December 2024
You are already at the latest version
Abstract
In recent years, the educational field has evolved rapidly owing to the integration of several technologies, especially experiments in remote laboratories in the engineering area. Therefore, this article addresses the development of an innovation system for automatically correcting experiments in remote laboratories in mechatronics using digital twins, convolutional neural networks (CNNs), and generative artificial intelligence technologies. This system was designed to overcome the limitations of the availability of physical laboratories and teachers and to assist in learning, enabling automatic acquisitions at any time. The digital twin captures data from the teacher's and student's experiments, allowing accurate comparisons to identify successes and errors. The application of CNNs serves to validate the results of the experiments through image analysis, whereas generative AI helps to identify patterns. The system was evaluated in a teaching plant, effectively correcting experiments with digital inputs and outputs. In addition, it provides students with detailed feedback on their performance, including specific errors and suggestions for improvement. With a three-layer architecture, i.e., experiments, didactics, and management, the system efficiently processes data from teachers and students, contributing to correcting experiments and optimizing teaching in remote environments.
Keywords:
1. Introduction
2. Related Work
- A.
- Digital Twin in Remote Laboratories
- B.
- Student assessment in laboratories
3. Concept Design
3.1. Experimental layer
- Manager Element:
- Controller:
- Didactic Experiments:
- Camera:
3.2. Didactic Twin Layer
- Learning Algorithm Module:
- Assessment Algorithm:
- Student Dashboard:
3.3. Management layer
- Data teacher:
- Data Student:
- Data Labs
3.4. Data Exchanged Module
4. System implementation
4.1. Implementing the Experimental Layer
- Controller:
- Didactic Experiments:
- Manager Element:
- Camera:
4.2. Implementing the Didactic Twin Layer
4.2.1. Learning Algorithm Module
- Preprocessing:
- Experiment Pattern:
4.2.2. Assessment Algorithm Module:
- Assessment experiment:
- Assessment Results:
- Data Exchange Module:
4.2.3. Student Dashboard Module:
4.3. Implementing the Management Layer
4.3.1. Data Teacher Module:
4.3.2. Data Student Module:
4.3.3. Data Labs Module:
5. Results and tests
- 1st Step:
- 2nd Step:
- 3rd Step:
- 4th Step:
- Correct program:
- Incorrect program:
6. Discussion
7. Conclusions
Acknowledgments
References
- T. C. Caetano, C. C. Moreira, and M. F. R. Junior, "Computer-Aided Experiments (CAE): A Study Regarding a Remote-Controlled Experiment, Video Analysis, and Simulation on Kinematics," in IEEE Transactions on Education, vol. 67, no. 2, pp. 245-255, April 2024. [CrossRef]
- V. Young, S. Cho, C. Talley, R. C. Voicu, C. Tekes, and A. Tekes, "A Modular Control Lab Equipment and Virtual Simulations for Engineering Education," SoutheastCon 2024, Atlanta, GA, USA, 2024, pp. 717-721. [CrossRef]
- G. Mušič, S. Tomažič and V. Logar, "Distance Learning of Programmable Logic Control: An Implementation Example," 2022 29th International Conference on Systems, Signals, and Image Processing (IWSSIP), Sofia, Bulgaria, 2022, pp. 1-4. [CrossRef]
- A. Birk and D. Simunovic, "Robotics Labs and Other Hands-On Teaching During COVID-19: Change Is Here to Stay?," in IEEE Robotics & Automation Magazine, vol. 28, no. 4, pp. 92-102, Dec. 2021. [CrossRef]
- R. J. D. Martínez, "Design Experience of Laboratory Practices in COVID Times.," 2023 Future of Educational Innovation-Workshop Series Data in Action, Monterrey, Mexico, 2023, pp. 1-5. [CrossRef]
- L. Kunzhe and I. Kholodilin, "Developing Virtual Laboratories for Understanding the Automation Course," 2023 Seminar on Electrical Engineering, Automation & Control Systems, Theory and Practical Applications (EEACS), Saint Petersburg, Russian Federation, 2023, pp. 208-213. [CrossRef]
- D. Krūmiņš et al., "Open Remote Web Lab for Learning Robotics and ROS With Physical and Simulated Robots in an Authentic Developer Environment," in IEEE Transactions on Learning Technologies, vol. 17, pp. 1325-1338, 2024. [CrossRef]
- V. Young, S. Cho, C. Talley, R. C. Voicu, C. Tekes, and A. Tekes, "A Modular Control Lab Equipment and Virtual Simulations for Engineering Education," SoutheastCon 2024, Atlanta, GA, USA, 2024, pp. 717-721. [CrossRef]
- A. Gampe, A. Melkonyan, M. Pontual and D. Akopian, "An Assessment of Remote Laboratory Experiments in Radio Communication," in IEEE Transactions on Education, vol. 57, no. 1, pp. 12-19, Feb. 2014. [CrossRef]
- V. F.de Lucena Jr., A. Brito, P. Gönner, N. Jazdi, “A Germany-Brazil experience report on teaching software engineering for electrical engineering undergraduate students,” 2006 Software Engineering Education Conference, Proceedings, pp. 69 - 76. [CrossRef]
- V. F. de Lucena Jr., J. P. de Queiroz-Neto, I. B. Benchimol, A. P. Mendonça, V. R. da Silva, M. Ferreira Filho M. “Teaching software engineering for embedded systems: An experience report from the Manaus research and development pole.” 2007 Proceedings - Frontiers in Education Conference, FIE 2007. [CrossRef]
- G. Carro Fernandez et al., "Mechatronics and robotics as motivational tools in remote laboratories," 2015 IEEE Global Engineering Education Conference (EDUCON), Tallinn, Estonia, 2015, pp. 118-123. [CrossRef]
- R. Rodriguez-Calderón, "In-home laboratory for learning automation at no cost," 2021 IEEE World Conference on Engineering Education (EDUNINE), Guatemala City, Guatemala, 2021, pp. 1-4. [CrossRef]
- S. Verslype, L. Buysse, J. Peuteman, D. Pissoort, and J. Boydens, “Remote laboratory setup for software verification in an embedded systems and mechatronics course,” in 2020 XXIX International Scientific.
- K. Mershad, A. K. Mershad, A. Damaj and A. Hamieh, "Using Internet of Things for Automatic Student Assessment during Laboratory Experiments," 2019 IEEE International Smart Cities Conference (ISC2), Casablanca, Morocco, 2019, pp. [CrossRef]
- P. Orduña et al., "An Extensible Architecture for the Integration of Remote and Virtual Laboratories in Public Learning Tools," in IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, vol. 10, no. 4, pp. 223-233, Nov. 2015. [CrossRef]
- Zine, Othmane & Errouha, Mustapha & Zamzoum, Othmane & Derouich, Aziz & Talbi, Abdennebi. (2019). SEITI RMLab: A costless and effective remote measurement laboratory in electrical engineering. International Journal of Electrical Engineering Education. 56. 3-23. [CrossRef]
- E. W. de Vries and H. J. Wörtche, "Remote Labs and their Didactics in Engineering Education: A Case Study," 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 2021, pp. 1598-1600. [CrossRef]
- S. Grassini and L. Lombardo, "Education in I&M: New Insights in Remote Teaching and Learning of Instrumentation and Measurement: The iHomeX Remote Lab Project," in IEEE Instrumentation & Measurement Magazine, vol. 27, no. 1, pp. 26-30, February 2024. [CrossRef]
- D. P. Reid and T. D. Drysdale, "Student-Facing Learning Analytics Dashboard for Remote Lab Practical Work," in IEEE Transactions on Learning Technologies, vol. 17, pp. 1037-1050, 2024. [CrossRef]
- V. Sarac, "Learning Fundamentals of Electrical Engineering by the Aid of Virtual Laboratories," 2024 23rd International Symposium INFOTEH-JAHORINA (INFOTEH), East Sarajevo, Bosnia, and Herzegovina, 2024, pp. 1-5. [CrossRef]
- F. Schauer, M. Krbecek, P. Beno, M. Gerza, L. Palka and P. Spilaková, "REMLABNET II - Open remote laboratory management system for university and secondary schools research based teaching," Proceedings of 2015 12th International Conference on Remote Engineering and Virtual Instrumentation (REV), Bangkok, Thailand, 2015, pp. 109-112. [CrossRef]
- W. Gutierrez Maroquin, M. Fernandez and W. Mantilla Avila, "The Joint Training, a SENA Learning Model for Latin America," in IEEE Latin America Transactions, vol. 14, no. 6, pp. 2997-3002, June 2016. [CrossRef]
- S. AbuShanab, M. Winzker, R. Brück and A. Schwandt, "A study of integrating remote laboratory and on-site laboratory for low-power education," 2018 IEEE Global Engineering Education Conference (EDUCON), Santa Cruz de Tenerife, Spain, 2018, pp. 405-414. [CrossRef]
- L. F. Zapata-Rivera and C. Aranzazu-Suescun, "Enhanced Virtual Laboratory Experience for Wireless Networks Planning Learning," in IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, vol. 15, no. 2, pp. 105-112, May 2020. [CrossRef]
- J. Nau and K. Henke, "GOLDi Labs as Fully Integrated Learning Environment," 2024 IEEE World Engineering Education Conference (EDUNINE), Guatemala City, Guatemala, 2024, pp. 1-6. [CrossRef]
- S. Alsaleh, A. Tepljakov, A. Köse, J. Belikov, and E. Petlenkov, "ReImagine Lab: Bridging the Gap Between Hands-On, Virtual, and Remote-Control Engineering Laboratories Using Digital Twins and Extended Reality," em IEEE Access, vol. 10, pp. 89924-89943, 2022. [CrossRef]
- N. d. Silva, J. García-Zubía, U. Hernández-Jayo and J. B. D. M. Alves, "Extended Remote Laboratories: A Systematic Review of the Literature From 2000 to 2022," in IEEE Access, vol. 11, pp. 94780-94804, 2023. [CrossRef]
- A. Boltsi, K. Kalovrektis, A. Xenakis, P. Chatzimisios e C. Chaikalis, "Ferramentas digitais, tecnologias e metodologias de aprendizagem para estruturas de educação 4.0: uma pesquisa orientada para STEM", em IEEE Access, vol. 12, pp. 12883-12901, 2024. [CrossRef]
- V. Kuvshinnikov, E. Kovshov and V. Korchagin, "Digital Educational Resources in Industrial Radiography," 2024 7th International Conference on Information Technologies in Engineering Education (Inforino), Moscow, Russian Federation, 2024, pp. 1-5. [CrossRef]
- G. Wang, H. Li, S. Ye, H. Zhao, H. Ding and S. Xie, "RFWNet: A Multiscale Remote Sensing Forest Wildfire Detection Network With Digital Twinning, Adaptive Spatial Aggregation, and Dynamic Sparse Features," in IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1-23, 2024, Art no. 4708523. [CrossRef]
- S. Prohaska and L. Kennes, "Evaluating the Use of Virtual Twins in a Control Systems Course," 2023 IEEE 2nd German Education Conference (GECon), Berlin, Germany, 2023, pp. 1-6. [CrossRef]
- M. C. Elias-Espinosa and E. Bastida-Escamilla, "Evaluating Digital Twins as Training Centers: A Case Study," 2022 10th International Conference on Information and Education Technology (ICIET), Matsue, Japan, 2022, pp. 1-5. [CrossRef]
- S. Mihai et al., "Digital Twins: A Survey on Enabling Technologies, Challenges, Trends and Future Prospects," in IEEE Communications Surveys & Tutorials, vol. 24, no. 4, pp. 2255-2291, Fourthquarter 2022. [CrossRef]
- R. Rayhana, L. Bai, G. Xiao, M. Liao, and Z. Liu, "Digital Twin Models: Functions, Challenges, and Industry Applications," in IEEE Journal of Radio Frequency Identification, vol. 8, pp. 282-321, 2024. [CrossRef]
- A. Barbie and W. Hasselbring, "From Digital Twins to Digital Twin Prototypes: Concepts, Formalization, and Applications," in IEEE Access, vol. 12, pp. 75337-75365, 2024. [CrossRef]
- F. Martell-Chavez, J. M. López-Téllez, C. A. Paredes-Orta and R. Espinosa-Luna, "Virtual laboratories for teaching automation, robotics, and optomechatronics," 2023 IEEE World Engineering Education Conference (EDUNINE), Bogota, Colombia, 2023, pp. 1-4. [CrossRef]
- F. Guc, J. Viola and Y. Chen, "Digital Twins Enabled Remote Laboratory Learning Experience for Mechatronics Education," 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence (DTPI), Beijing, China, 2021, pp. 242-245. [CrossRef]
- L. Buysse, Q. Van den Broucke, S. Verslype, J. Peuteman, J. Boydens and D. Pissoort, "FPGA-based digital twins of microcontroller peripherals for verification of embedded software in a distance learning environment," 2021 XXX International Scientific Conference Electronics (ET), Sozopol, Bulgaria, 2021, pp. 1-4. [CrossRef]
- M. C. Elias-Espinosa and E. Bastida-Escamilla, "Evaluating Digital Twins as Training Centers: A Case Study," 2022 10th International Conference on Information and Education Technology (ICIET), Matsue, Japan, 2022, pp. 1-5. [CrossRef]
- C. A. Bolu et al., "Appropriate Online Laboratories for Engineering Students in Africa," 2022 IEEE IFEES World Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC), Cape Town, South Africa, 2022, pp. 1-5. [CrossRef]
- A. Niedźwiecki, S. Jongebloed, Y. Zhan, M. Kümpel, J. Syrbe and M. Beetz, "Cloud-Based Digital Twin for Cognitive Robotics," 2024 IEEE Global Engineering Education Conference (EDUCON), Kos Island, Greece, 2024, pp. 1-5. [CrossRef]
- N. Papulovskaya and I. Izotov, "Internet of Things Online Laboratory: non-Virtual Training," 2024 7th International Conference on Information Technologies in Engineering Education (Inforino), Moscow, Russian Federation, 2024, pp. 1-4. [CrossRef]
- S. I. Ching Wang, E. Zhi Feng Liu, Y. Y. Huang and H. Yu Sang, "When Drone Meets AI Education: Boosting High School Students’ Computational Thinking and AI Literacy," 2024 Pacific Neighborhood Consortium Annual Conference and Joint Meetings (PNC), Seoul, Korea, Republic of, 2024, pp. 45-52. [CrossRef]
- H. Yu and Y. Guo, "Harnessing the Potential of Chat GPT in Education: Unveiling its Value, Navigating Challenges, and Crafting Mitigation Pathways," 2023 5th International Workshop on Artificial Intelligence and Education (WAIE), Tokyo, Japan, 2023, pp. 48-52. [CrossRef]
- Lundström, N. Maleki and F. Ahlgren, "Online Course Improvement Through GPT-4: Monitoring Student Engagement and Dynamic FAQ Generation," 2024 IEEE Global Engineering Education Conference (EDUCON), Kos Island, Greece, 2024, pp. 1-6. [CrossRef]
- S. Pang, D. Kong, D. Liu, R. Pan, H. Wang and X. Wang, "Evaluation of LoRa Network Link Quality in Complex Urban Environments Based on Static and Dynamic Parameters," in IEEE Access, vol. 12, pp. 125369-125383, 2024. [CrossRef]
- A. Nur Handayani, Muladi, I. Ari Elbaith Zaeni, W. Cahya Kurniawan and R. Andrie Asmara, "Development of e-Collab Classroom for CNN Practice on Electrical Engineering, Universitas Negeri Malang," 2020 4th International Conference on Vocational Education and Training (ICOVET), Malang, Indonesia, 2020, pp. 47-51. [CrossRef]
- Z. Li, F. Liu, W. Yang, S. Peng and J. Zhou, "A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects," in IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 12, pp. 6999-7019, Dec. 2022. [CrossRef]
- M. Saini, S. Dhyani, S. Sharma, M. Awasthi, N. Yamsani and P. Singh, "Machine Learning for Enhanced Student Learning and Engagement in Interactive Classrooms," 2024 2nd International Conference on Advancement in Computation & Computer Technologies (InCACCT), Gharuan, India, 2024, pp. 23-28. [CrossRef]
- A. de la Torre and M. Baldeon-Calisto, "Generative Artificial Intelligence in Latin American Higher Education: A Systematic Literature Review," 2024 12th International Symposium on Digital Forensics and Security (ISDFS), San Antonio, TX, USA, 2024, pp. 1-7. [CrossRef]
- R. Murali, N. Ravi and A. Surendran, "Augmenting Virtual Labs with Artificial Intelligence for Hybrid Learning," 2024 IEEE Global Engineering Education Conference (EDUCON), Kos Island, Greece, 2024, pp. 1-10. [CrossRef]
- M. Mukhammadsidiqov, S. Akmalov and K. Akhmedov, "An Intensive Review and Assessment of Impacts of AI in the Field of Teaching (EdU)," 2024 4th International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), Greater Noida, India, 2024, pp. 1013-1017. [CrossRef]
- S. A. Benson, "Embracing Technologies to Facilitate Student Learning and Future Readiness," 2024 International Symposium on Educational Technology (ISET), Macau, Macao, 2024, pp. 344-346. [CrossRef]
- G. Ren, T. Wang and D. Wang, "Construction and Application of Interactive Assessment Environment Based on Document Cloud," 2021 Tenth International Conference of Educational Innovation through Technology (EITT), Chongqing, China, 2021, pp. 234-239. [CrossRef]
- Elmoazen, Ramy & Saqr, Mohammed & Khalil, Mohammad & Wasson, Barbara. (2023). Learning analytics in virtual laboratories: a systematic literature review of empirical research. Smart Learning Environments. 10. 1-20. 10.1186/s40561-023-00244-y.
- A. Fernández, M. A. Eguía and L. E. Echeverría, "Virtual commissioning of a robotic cell: an educational case study," 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Zaragoza, Spain, 2019, pp. 820-825. [CrossRef]
- M. Jungwirth, P. Hehenberger, S. Merschak, W. -C. Lee and C. -Y. Liao, "Influence of Digitization in Mechatronics Education Programmes: A Case Study between Taiwan and Austria," 2020 21st International Conference on Research and Education in Mechatronics (REM), Cracow, Poland, 2020, pp. 1-7. [CrossRef]
- M. A. G. Martínez, I. Y. Sanchez and F. M. Chávez, "Development of Automated Virtual CNC Router for Application in a Remote Mechatronics Laboratory," 2021 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), Mauritius, Mauritius, 2021, pp. 1-6. [CrossRef]
- L. F. R. Riveros, V. H. B. Tristancho, J. E. S. Sanabria and H. D. V. Amado, "Digital Twins, Didactic Strategy for Teaching Industrial Automation," 2022 Congreso de Tecnología, Aprendizaje y Enseñanza de la Electrónica (XV Technologies Applied to Electronics Teaching Conference), Teruel, Spain, 2022, pp. 1-4. [CrossRef]
- L. Fletcher, J. F. Botero, N. Gaviria, É. F. Aza, and J. Vergara, "RECoNE: A Remote Environment for Computer Networks Education," 2020 IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 2020, pp. 787-791, doi 10.1109/EDUCON45650.2020.9125383.
- Y. -a. Bachiri and H. Mouncif, "Applicable strategy to choose and deploy a MOOC platform with multilingual AQG feature," 2020 21st International Arab Conference on Information Technology (ACIT), Giza, Egypt, 2020, pp. 1-6. [CrossRef]
- R. E. Nalawati and A. Dini Yuntari, "Ratcliff/Obershelp Algorithm as An Automatic Assessment on E-Learning," 2021 4th International Conference of Computer and Informatics Engineering (IC2IE), Depok, Indonesia, 2021, pp. 244-248. [CrossRef]
- H. A. Hassan, M. M. Eid, M. M. Elmesalawy and A. M. Abd El-Haleem, "A New Intelligent System for Evaluating and Assisting Students in Laboratory Learning Management System," 2022 14th International Conference on Computational Intelligence and Communication Networks (CICN), Al-Khobar, Saudi Arabia, 2022, pp. 566-571. [CrossRef]
- S. Disa, Purnamawati and A. M. Idkhan, "Web e-Learning: Automated Essay Assessment Based on Natural Language Processing Using Vector Space Model," 2022 4th International Conference on Cybernetics and Intelligent System (ICORIS), Prapat, Indonesia, 2022, pp.1-4. [CrossRef]
- C. N. Tulha, M. A. G. Carvalho, and L. N. de Castro. “LEDA: A Learning Analytics Based Framework to Analyze Remote Labs Interaction.” 2022, In Proceedings of the Ninth ACM Conference on Learning @ Scale (L@S '22). Association for Computing Machinery, New York, NY, USA, 379–383. [CrossRef]
- A. Protic, Z. Jin, R. Marian, K. Abd, D. Campbell, and J. Chahl, "Implementation of a Bi-Directional Digital Twin for Industry 4 Labs in Academia: A Solution Based on OPC UA," 2020 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), Singapore, Singapore, 2020, pp. 979-983. [CrossRef]
- Heemels and T. Oomen, "Digital Twins in Mechatronics: From Model-based Control to Predictive Maintenance," 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence (DTPI), Beijing, China, 2021, pp. 336 339. [CrossRef]

















| Did you perform activity correction automatically? | Do you use digital twin technology in remote laboratories? | Do you provide feedback to students (digital or video)? | Store student data for their development? | Did you perform automatic corrections for hands-on experiments in remote labs? | |
| Fernández, Eguía e Echeverría [57] | No | Yes | Yes | No | No |
| Jungwirth et al [58] | No | Yes | Yes | No | No |
| Guc, Viola e Chen [38] | No | Yes | Yes | No | No |
| Martínez, Sanchez e Chávez [59] | No | Yes | No | No | No |
| Mušič, Tomažič e Logar [3] | No | Yes | No | No | No |
| Riveros et al. [60] | No | Yes | No | No | No |
| Fletscher et al. [61] | Yes | No | Yes | Yes | No |
| Bachiri e Mouncif [62] | Yes | No | Yes | No | No |
| Nalawati e Yuntari [63] | Yes | No | Yes | No | No |
| Hassan et al [64] | Yes | No | Yes | Yes | No |
| Disa, Purnamawati e Idkhan [65] | Yes | No | Yes | No | No |
| Tulha, Carvalho e Castro [66] | Yes | No | Yes | Yes | Yes |
| Our approach | Yes | Yes | Yes | Yes | Yes |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).