1. Introduction

2. Methodology

2.1. Instruments/Techniques

2.1.1. Instruments/Techniques

To ensure the validity and reliability of the results, standardized tools such as the Usability Scale System (SUS) will be used. [8] and the User Experience Questionnaire (UEQ) [17]. These software engineering metrics measurement tools have proven to be effective in multiple previous studies. [18,19].

The SUS consists of 10 items with responses on a 5-point Likert scale, ranging from “Strongly disagree” to “Strongly agree”. Scores are calculated using the following formula:

$$\mathrm{SUS}\mathrm{Score}=2.5\times \left(\sum _{i=1}^5(Q_{2i-1}-1)+\sum _{i=1}^5(5-Q_{2i})\right)$$

(1)

Where $Q_{2i-1}$ are the odd numbered items and $Q_{2i}$ are the even numbered items. The total SUS score ranges between 0 y 100.

The UEQ measures user experience through 26 items divided into six scales: Attractiveness, Perspicuousness, Efficiency, Dependability, Stimulation and Novelty. The items are also answered on a 7-point Likert scale, ranging from -3 (very negative) to +3 (very positive). The scores for each scale are calculated as the mean of the corresponding items.

Table 1. Description of the scales of the User Experience Questionnaire (UEQ).

Table 1. Description of the scales of the User Experience Questionnaire (UEQ).

Scale	Items	Score Range	Interpretation
Attractive	6 items	-3 a +3	Overall evaluation
Insight	4 items	-3 a +3	Clarity and understanding
Efficiency	4 items	-3 a +3	Speed and organization
Dependability	4 items	-3 a +3	Control and predictability
Stimulation	4 items	-3 a +3	Motivation and interest
New at	4 items	-3 a +3	Innovation and creativity

In addition, data privacy policies will be implemented to protect participants’ personal information, ensuring that all data collected is handled confidentially and securely. According to Følstad, Nordheim and Bjørkli [20], it is crucial to ensure user confidence in data management in interactions with automated systems. To this end, data will be anonymized and stored on secure servers with access restricted to the research team only. Participants will also be informed of their rights and their informed consent will be sought prior to data collection.

2.2. Descriptive Table of Variables

Descriptive tables summarizing the key variables of the study will be presented.

Table 2. Description of questionnaire responses on interest in academic events and use of a bot.

Table 2. Description of questionnaire responses on interest in academic events and use of a bot.

DESCRIPTIVE TABLE	Variables	N° Resp.	Statistics
What type of events academics you are interested in receive information?	Scholarships	12	Mean: 8
	Congresses	7	D.E: 3.37
	Courses	9	Min: 4
	Others	4	Max: 12
Would you be interested in use a bot to inform you automatically on academic events?	Sí	28	Mean: 10.6
	No	1	D.E: 15.04
	Not sure	3	Min: 1
			Max: 28
How often do you think that you would use a this guy?	Several times a week	11	Mean: 8
	Occasionally	9	D.E: 2.45
	Once a week	6	Min: 6
	Daily	6	Max: 11
How would you like to receive updates on academic events?	Push notifications in the application	25	Mean: 10.6
	E-mail address	6	D.E: 12.66
	SMS text messages	1	Min: 1
			Max: 25

3. Results and Discussion

3.1. Predictive Model Results

The logistic regression model was trained to predict students’ interest in using an information bot about academic events. The table below shows the coefficients of the bootstrapped model using 800 observations simulated from the original 33 observations [21]:

Table 3. Coefficients of the bootstrap fitted logistic regression model [22].

Table 3. Coefficients of the bootstrap fitted logistic regression model [22].

Variable	Estimate	Std. Error	z value	Pr($>|$z|)
(Intercept)	5.9502	0.6035	9.860	6.2236e-23
Events_Interest	0.7188	0.1270	5.661	1.5010e-08
Frequency_Use	-1.4915	0.1751	-8.517	1.6333e-17
Preference_Notif	-0.7019	0.2314	-3.033	2.4230e-03

The confusion matrix and model performance metrics are presented below. [23]:

Table 4. Confusion matrix of the predictive model.

Table 4. Confusion matrix of the predictive model.

		Prediction
		No	Sí
Reality	No	19	0
	Sí	71	710

Accuracy (Accuracy):${\displaystyle \frac{19+710}{19+0+71+710}}\approx 91.125\%$

Sensitivity (Recall):${\displaystyle \frac{710}{71+710}}\approx 90.91\%$

Specificity:${\displaystyle \frac{19}{19+0}}=100\%$

These results indicate that the model performs well in predicting students’ interest in using the informational bot.

Figure 1. Confusion matrixgraph.

Figure 1. Confusion matrixgraph.

Figure 2. prediction density.

Figure 2. prediction density.

References

1. Almalki, M.; Ganapathy, V. User Satisfaction with Automated Information Systems in Education. Computers & Education 2021, 158, 104–113. [Google Scholar]
2. Fong, S.; Lee, V. The Impact of Digital Automation in Health and Education. Journal of Technology in Human Services 2018, 36, 200–212. [Google Scholar]
3. Patel, N.; Jones, M. Benefits of Automated Customer Service Systems. Journal of Business and Technology 2019, 24, 150–165. [Google Scholar]
4. Lee, S.; Kim, J. Real-Time Information Systems in Event Management. International Journal of Event and Festival Management 2020, 11, 175–190. [Google Scholar]
5. Tullis, T.; Albert, B. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics; Morgan Kaufmann, 2008.
6. Sauro, J.; Lewis, J.R. Quantifying the User Experience: Practical Statistics for User Research; Morgan Kaufmann, 2016.
7. Nielsen, J. Usability Engineering; Academic Press, 1993.
8. Brooke, J. SUS: A quick and dirty usability scale. Usability evaluation in industry 1996, 189, 4–7. [Google Scholar]
9. Laugwitz, B.; Held, T.; Schrepp, M. Construction and Evaluation of a User Experience Questionnaire. HCI and Usability for Education and Work 2008, 63–76. [Google Scholar]
10. Hassenzahl, M.; Tractinsky, N. User experience - a research agenda. Behaviour & Information Technology 2010, 25, 91–97. [Google Scholar]
11. Tuch, A.N.; Roth, S.P.; Hornbæk, K. Is Usability the Same as User Experience? ACM Transactions on Computer-Human Interaction 2012, 19, 23–32. [Google Scholar]
12. Dhinakaran, A.; Srinivasan, M. Automated Systems in Modern Education: A Review. Journal of Educational Technology 2020, 21, 134–148. [Google Scholar]
13. Kim, Y.K.; Lee, J.Y. Evaluating the Efficiency of Automated Systems in Higher Education. Educational Research Review 2021, 30, 100–115. [Google Scholar]
14. Albert, W.; Tullis, T. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics; Morgan Kaufmann, 2010.
15. Hartson, R.; Pyla, P.S. The UX Book: Process and Guidelines for Ensuring a Quality User Experience; Elsevier, 2012.
16. Ringeval, F.; Fauth, P.; Wissmath, B. The Usability of Automated Information Systems in Various Applications. Proceedings of the 2020 International Conference on Human-Computer Interaction, 2020, pp. 185–198.
17. Schrepp, M.; Hinderks, A.; Thomaschewski, J. Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence 2017, 4, 103–108. [Google Scholar] [CrossRef]
18. Hinderks, A.; Schrepp, M.; Thomaschewski, J.; Hierling, M. Benchmarking user experience questionnaires. Journal of Usability Studies 2018, 13, 159–167. [Google Scholar]
19. McLellan, H.; Thomaschewski, J.; Hinderks, A. The role of the user experience questionnaire (UEQ) in HCI research. Journal of Usability Studies 2012, 8, 41–46. [Google Scholar]
20. Følstad, A.; Nordheim, C.B.; Bjørkli, J.C. Building trust in chatbot implementations: exploring transparency and design features. Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, 2018, pp. 1–10.
21. Gelman, A.; Hill, J. Regression analysis and its application: a data-oriented approach. Journal of Educational Statistics 2008, 33, 554–555. [Google Scholar]
22. Agresti, A.; Franklin, C. Foundations of linear and generalized linear models; John Wiley & Sons, 2015.
23. Powers, D.M. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness & correlation. Journal of Machine Learning Technologies 2020, 2, 37–63. [Google Scholar]
24. Moon, J.Y. Consumer adoption of high-tech products: A meta-analysis of the literature; IEEE Transactions on Engineering Management, 2007.
25. Han, J.; Kamber, M.; Pei, J. Data mining: concepts and techniques. Morgan Kaufmann 2011. [Google Scholar]
26. Sun, S.Y.; Cao, X.; Dai, B. Understanding user acceptance of AI recommendation agents in e-commerce. Computers in Human Behavior 2019, 90, 168–179. [Google Scholar]
27. Kim, J.; Kim, D. Consumer perceptions of chatbot-based interactive services: An extended perspective of technology acceptance model. International Journal of Human-Computer Interaction 2020, 36, 1373–1385. [Google Scholar]
28. Karimi, S.; Walter, Z.; O’Connor, P.; Choi, M. Predicting users’ acceptance of artificial intelligence (AI) speaker devices for purchasing products. Computers in Human Behavior 2018, 84, 268–278. [Google Scholar]
29. Venkatesh, V.; Morris, M.G.; Davis, G.B.; Davis, F.D. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science 2000, 46, 186–204. [Google Scholar] [CrossRef]
30. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 1989, 319–340. [Google Scholar] [CrossRef]
31. Hernandez, J.M.; Mazzon, J.A.; Perez, A. The impact of quality and user experience on the intention to use an online portal for cell phone services. Quality & Quantity 2010, 44, 361–378. [Google Scholar]
32. Park, J.E.; Han, S. Factors affecting the intention to use online learning systems by learners in South Korea. Sustainability 2021, 13, 6214. [Google Scholar]
33. Shin, H.C.; Roth, H.R.; Gao, M.; Lu, L.; Xu, Z.; Nogues, I.; Yao, J.; Mollura, D.J.; Summers, R.M. A review of artificial intelligence in medical imaging: experience, deployment, and performance evaluation. Journal of Digital Imaging 2020, 33, 323–340. [Google Scholar]
34. Tan, H.; Poo, D.C.C.; Hamid, A.W.; Leng, T.T. Acceptance of AI and robotics in healthcare: a human-centric approach. Journal of Healthcare Engineering 2021, 2021, 1–14. [Google Scholar] [CrossRef] [PubMed]
35. Bhattacherjee, A. Understanding information systems continuance: An expectation-confirmation model. MIS Quarterly 2001, 351–370. [Google Scholar] [CrossRef]
36. Ramayah, T.; Ignatius, J.; Suki, N.M.; Patrick, H.; Lo, M.C.; Lee, J. The role of perceived usefulness, perceived ease of use, security and privacy, and customer attitudes to engender customer satisfaction in electronic commerce: A structural equation modeling approach. Asia Pacific Journal of Marketing and Logistics 2006, 18, 103–118. [Google Scholar]