Synergizing Systems Thinking and Technology-Enhanced Learning Using the Flow Theory Framework

Preprint

Article

Synergizing Systems Thinking and Technology-Enhanced Learning Using the Flow Theory Framework

Altmetrics

Downloads

Views

Comments

A peer-reviewed article of this preprint also exists.

Supplementary Material

supplementary.pdf (453.09KB )

This version is not peer-reviewed

Submitted:

01 September 2024

Posted:

03 September 2024

You are already at the latest version

Alerts

Abstract

In an era where digital technologies are integral to daily life, developing higher-order thinking skills with appropriate information and communication technology (ICT) support is crucial. The purpose of this study was to examine synergies of systems thinking and technology-enhanced learning from the perspective of flow theory. For that purpose, we surveyed more than 65 pre-service preschool teachers engaged in a design, technology, and engineering (DTE) course at the University of Ljubljana. Mapping of systems thinking revealed that pre-service preschool teachers needed support with regard to feedback and understanding the interrelationship dimen-sion of systems thinking. Predictive and mediation analyses yielded some interesting results. Those who participated in the ICT-enhanced DTE course showed higher cognitive and social engage-ment, which is correlated with better systems thinking, while their aesthetic engagement was lower. Experiencing a flow state during ICT activities positively influenced systems thinking, par-ticularly in terms of clear goals and autotelic experiences. Flow theory seems to be a solid and ap-propriate framework to use for studying synergies in technology-enhanced systems thinking for sustainable education. These findings underscore the importance of integrating systems thinking into curricula to enhance learning outcomes and prepare students for future challenges toward the achievement of Sustainable Development Goals.

Keywords:

Subject: Social Sciences - Education

1. Introduction

In today’s rapidly changing technological world, the ability to solve problems has become one of the most important skills individuals need in order to live and work successfully [1]. However, along with technological progress, many challenges arise that require innovative approaches and critical thinking. Therefore, it is important to start developing higher-order thinking skills at a young age, which will enable us to effectively deal with complex situations. Higher-order thinking skills such as analysis, synthesis, and creativity enable individuals to adapt to change, critically evaluate information, and find new solutions. In the educational process, it is therefore crucial to give students the opportunity to develop these skills by using appropriate teaching methods [2].

Systems thinking takes a holistic view of problem solving and therefore has the potential, as a new learning approach, to contribute in some way to the transfer of knowledge and skills for a sustainable future [3,4]. It has also been recognized by the OECD to be an important approach for understanding the world we live in and the policies we live under [5]. Additionally, systems thinking offers a process, a set of “technologies” and thinking skills that can improve the understanding required for sustainability education [6]. When using different, novel approaches, student engagement is necessary for a successful process, which depends on many factors, including the use of ICT and digital tools. In today’s world, we can no longer escape digital technologies. They have become an indispensable part of our daily lives. They not only provide means of entertainment but enable many processes to be improved, such as facilitating visualization, preparing materials, watching videos, programming, taking and editing pictures, storing data, and many others [7]. We cannot exclude digital technologies from the learning process, and studies [8,9,10,11,12] show that both teachers (pre-/in-service) and parents have mixed attitudes toward their use in education.

With regard to the use of digital technology in early childhood, Bredekamp [13] states that there is a need to address how to introduce it into the child’s environment in a quality way, rather than whether to include it. Making the leap from seeing digital technology as something to which children (and adults) retreat to a something they can use to learn about life together is important. For this reason, professional support from and for teachers is essential. Furthermore, an optimal way to integrate it into the classroom must be found so that students can develop digital skills while learning to use technology critically [14]. Many models for integrating ICT and digital tools have already been proposed in the literature, such as SAMR, TPACK, the pedagogical wheel, etc., but despite the potential benefits, the applications and outcomes are questionable [15,16]. Both positive and negative effects have been shown, so a clear articulation of the use of ICT and digital tools requires further research. The optimal use of digital technology in the classroom requires an appropriate learning environment that fosters the development of higher-order thinking skills [17]. This means creating a stimulating environment in which students can explore, experiment, and work collaboratively to solve problems. Teachers play a key role, as they need to be able to guide and support students in the use of digital tools [18,19,20]. In addition, choosing the proper digital environment from among all digital applications (e.g., for coding lessons) can be a challenge for both parents and teachers [21].

In terms of sustainable development and achieving the Sustainable Development Goals, it is important that education systems also include content that promotes understanding and solving global challenges. Goal 4 of the Sustainable Development Agenda [4,22], which focuses on quality education for all, is key to realizing a sustainable future. Surveys such as PISA [23] allow us to monitor progress toward this goal and identify where improvements are needed. In Slovenia, the results for mathematics and science were above the average for OECD countries, but the results for reading literacy were below the average. However, the results show a decline in all areas of literacy among young people examined, indicating the need for change [24]. Based on the results, measures are being taken to improve the situation as part of the ongoing curriculum renewal. In the area of reading literacy, action plans for implementing the National Strategy for the Development of Reading Literacy are already in progress, while there are no specific educational strategies for science, mathematics, engineering, arts, and mathematics (STEAM) subjects. On the other hand, these areas are included in the Action Plan for Digital Education until 2027 (ANDI), the National Program for the Development of Artificial Intelligence 2025, and the Digital Slovenia 2030 Strategy [24]. All of these strategies at the national level, as well as in a broader sense, show that it is still necessary to search for new approaches and methods for teaching and ultimately working in today’s world. The importance of exploring modern approaches and stakeholders on the path to literacy is recognized at a higher level. The University of Ljubljana has been heavily involved for quite a long time in many projects to improve learning processes and integrate ICT into educational processes for a digital future and sustainability [25,26].

The aim of this research, which is funded by the Agency of the Republic of Slovenia for Research and Innovation (ARIS), the European Union’s NextGenerationEU [27], and the Ministry of Education of the Republic of Slovenia [28], is to develop educational and training strategies for digital and green transformation for sustainable development in the 21st century. It is part of larger projects, specifically a pilot project, “5.02 Improving the digital skills and competencies of (future) educators for quality educational work with younger children”; a research project, “Developing the twenty-first-century skills needed for sustainable development and quality education in the era of rapid technology-enhanced changes in the economic, social and natural environment”; and a program group, Strategies for Education for Sustainable Development Applying Innovative Student-Centred Educational Approaches [29]. The projects will identify the factors that influence the quality of education for future educators toward technologically driven sustainable development and 21st-century skills. They are based on studying the targeted integration of ICT in pre-service preschool education [28] along the entire educational vertical and through the development of strategies or strategic approaches [29]. The research projects also aim to design, implement, and evaluate learning models and measure the impact on 21st-century skills [29]. In addition, the project objectives cover the increased use of digital tools and ICT by future educators, as it was found that no regular or elective courses in the preschool education degree program are designed to directly promote digital development. The goal will be achieved by renovating existing or designing new study subjects with the didactic use of ICT to achieve a higher level of digital competency among the actors involved [28]. By evaluating current updated learning approaches, models, and curricula, it will be possible to gain insights into the strengths and weaknesses of modern approaches and their stakeholders in order to perfect a learning model focused on the use of ICT to develop sustainable competencies for the 21st century.

However, there is still no clear understanding about how to complement implementing the systems thinking approach through the lens of flow theory with ICT and students’ engagement in activities. Even though there has been research on systems thinking [30,31,32,33], to date, there is no study that has contextualized and conceptualized synergies of systems thinking and technology for an optimal experience as reflected by flow theory. Against this background, the purpose of this study was to investigate the state of the art of systems thinking among pre-service teachers, their perceptions about stimulating higher-order thinking skills, how they engage in DTE activities, and how they experience technology-enhanced DTE through the lens of flow theory. To our knowledge, this is the first study of systems dynamics from the perspective of flow theory, but we still assume with caution that flow might affect systems thinking, especially through collaboration and engagement in DTE activities where ICT is carefully selected for target use and learning goals are clearly defined.

1.1. Systems Thinking and HOTS

Systems thinking, an approach that is used to understand today’s increasingly complex world, refers to interrelationships and dependencies, to the interactions of a system with other systems or subsystems [31]. Different authors define systems thinking differently, including as a discipline to see the whole [34], an important interdisciplinary skill for collaborative work in solving problems [32], and a 21st-century competency for sustainable development and a sustainable society [4]. The importance of systems thinking is also recognized by the International Association for Technical and Engineering Education (ITEEA) [1], which defines it as an important practice of technological and engineering literacy. The general understanding of systems thinking as taking a holistic view is somewhat narrow, as both a holistic and a reductionist view of system dynamics are important [3]. After reviewing several definitions, Arnold and Wade [35] defined systems thinking as “... a set of synergistic analytical skills used to improve the ability to recognize and understand systems, predict their behavior, and design their changes to achieve desired effects.” [35] p. 675.

Moore et al. [36] recognized systems thinking and measured it in individuals using dimensions such as sequence (specifically sequence of events, causal sequence), diversity of causes and variations (specifically multiple possible causes, variations of different kinds (random/special)), and relations and feedback (specifically interrelationships of factors, patterns of relationships, feedback). In their work, they also suggested adding further dimensions and extending the measurement to include elements related to strategic approach orientation, such as personal effort, reliance on authority, and strategic thinking [36]. Several models of systems thinking have been developed, from simple models with three components [30] or four domains or dimensions [37,38] to those with more complex and hierarchical levels [39]. A model for DSRP (Distinctions, Systems, Relationships and Perspectives) rules was defined by Cabrera [3] when he was looking for common features of the many definitions of systems thinking. In the model, the author touches on four so-called simple rules that can be applied to any system in which we distinguish, set boundaries, and identify systems/elements ... we also identify mutual relationships between elements and the system, and look at the system from different perspectives. To support systems thinking, temporal behavior diagrams, causal loops, connecting circles, concept maps, flow state maps, and computer programs, including simulated models for education, have been proposed in the literature [33,40,41]. Other studies [42,43] have shown the positive effects that systems thinking have on the development and improvement of skills in various engineering fields and emphasized the importance of systems thinking as a complement to traditional teaching methods [40]. Systems thinking is an appropriate approach for solving complex problems [33,38,43,44], and therefore its integration into learning programs for current and future teachers is strongly recommended [45].

In his study, Frank [46] found that the systems thinking approach combined with project-based learning effectively promoted the understanding of engineering issues among pre-service teachers who did not have much prior knowledge of technical and technological content. Engström et al. [38] also noted that systems thinking is an appropriate approach for technological and engineering education, as it encourages learners to view technological systems as interconnected parts and to understand relationships, inputs, outputs, processes, etc., holistically. Because systems thinking involves understanding relationships and using multiple perspectives, it promotes interdisciplinary learning and enables a focus on real-world problems. It promotes cooperation, collaborative learning, and the development of higher order thinking skills (HOTS) such as problem solving, analytical thinking, through modeling, predicting, and searching for improvement, all of which is linked to a technology education that emphasizes practical applications and real-world problem solving [38].

The importance of HOTS is widely recognized around the world, and systems thinking is among the approaches with a high potential to foster them. Some authors [47] even include systems thinking among these skills. HOTS generally refers to more complex cognitive processes that go beyond memorization and other lower-level thinking. One of the best-known models is Bloom’s cognitive taxonomy, revised in 2001 [48], which divides cognitive processes into higher and lower levels. The latter include memorizing, understanding, and applying knowledge, and the former include analyzing, evaluating, and creating [48]. HOTS enable critical thinking, decision making, and creativity. They require the use of prior knowledge for reasoning, evaluating, analyzing, synthesizing, and solving problems [2]. Preschool educators use developmental strategies such as comparing, differentiating, identifying, evaluating, classifying, disassembling/assembling, fantasizing, building, planning, and drawing conclusions to promote HOTS in children [49]. After reviewing the work of other authors and the definition of HOTS, Liu et al. [50] proposed five categories: skills in problem solving, metacognition, critical thinking, teamwork, and innovation development. HOTS can also be found among the 21st-century skills that individuals need to be successful in society and play an important role in global education [51].

As there is a relationship between learning ability and HOTS, it is important to use the appropriate approaches and learning methods. Classroom activities that motivate active participation help students develop HOTS [52]. In their study, Lu et al. [53] investigated the relationships between smart classroom preferences, learning motivations, learning strategies, peer interactions, and HOTS. They confirmed that peer interactions and learning motivations in the learning environment have a positive influence on the ability to learn knowledge and skills. They focused on the smart classroom learning environment, which includes technology-enriched classrooms with physical and virtual spaces. The results show that in their work to develop HOTS, teachers must take into account learning motivations and peer interactions, as well as learning strategies and smart class preferences, as the latter also have an influence on HOTS, albeit indirectly [53]. In addition, graphic organizers and concept maps have been found to be good supports for learning strategies to promote HOTS, as they promote reasoning, contrasting ideas, and making connections [54].

Alammary’s research [55] found that in order to increase engagement and promote HOTS, the most effective type of blending learning among the different types (instructor-led face-to-face learning, instructor-led online learning, collaborative face-to-face learning, collaborative online learning, and self-directed online learning) was collaborative face-to-face learning, which was otherwise described as challenging by students. The effect, of course, depends on the size of the group, which in this case must be manageable [55]. For larger groups, online collaboration with greater support from online resources is more suitable. On the other hand, Alammary [55] noted that self-directed online learning is more suitable for lower levels of thinking skills.

In addition to global guidelines for sustainable development [4,51], the importance of developing HOTS can also be seen in educational curricula, even for the preschool years [56]. Modern learning paradigms emphasize the importance of constructivism [57] and social constructivism [58] and aim to create a stimulating environment and provide attentive care, especially for preschool children [59]. According to Piaget [57] and Vygotsky [58], knowledge is built by interactions with the environment and the social context. Piaget focused on existing abilities, while Vygotsky emphasized potential abilities realized with the right support. Interactions between students encourage discussion and critical thinking and support adaptation and problem solving.

In their research, Kumar and Mohamed [60] showed that preschool teachers had high levels of readiness with regard to their knowledge, understanding, and implementation of HOTS. On the other hand, some authors [61] have shown that preschool teachers in rural areas are somewhat less willing to include HOTS in terms of knowledge, skills, and application. As Slunjski [59] noted, the promotion of metacognitive skills among preschool children is influenced by the entire physical environment of the kindergarten. In particular, the positive effects are determined by the opportunities to use thoughtful learning materials that match interests, abilities, and prior knowledge. In addition, the quality of the kindergarten’s social environment, the general atmosphere and democracy in individual groups, the encouragement of children to self-assess, the teachers’ ability to listen to the children and to intervene, and their reflexivity also have a positive influence [59].

Integrating systems thinking and HOTS into education enhances problem solving and critical thinking and sets the stage for deeper engagement based on flow and activity theory. Systems thinking promotes a holistic understanding of interconnected elements, while HOTS empower learners to creatively address challenges [35,48]. This foundation transitions into flow and activity theory, in which individuals achieve optimal engagement by immersing themselves in tasks that match their abilities and interests [62]. By combining these concepts, educators can promote sustained engagement and meaningful learning experiences.

1.2. Using Flow and Activity Theory to Understand Human Engagement

The flow state, otherwise known as the optimal experience, is the state we experience when we are so immersed in a task that we no longer perceive our surroundings or time properly [63]. The flow state was originally described by the researcher Mihály Csíkszentmihályi [64] in the 1960s. Individuals in the flow state usually report being in a state of deep involvement and absorption [63]. People typically lose their sense of time, block out everyday problems, are highly concentrated, merge with the activity, and have a feeling of control over their actions. The activity is later perceived as if the end goal were an excuse for the process itself [14,62]. In a state of flow, the person’s individuality is preserved while they are simultaneously cooperating with the surrounding environment. This cooperation involves changing the environment and, in turn, being changed by it through a complex system of transactions. The flow state is associated not only with pleasure but also with importance and engagement, meaning that the experience itself can be negative, yet a positive feeling often follows its completion [65]. For example, the pressure to finish a task or job can induce a flow state, as it presents a challenge for some individuals [66]. Flow can therefore occur in almost any activity, but this does not mean that the state is easy to achieve [65]. Clear, immediate goals, quick feedback, and an appropriate level of task difficulty are ideal conditions for achieving the state of optimal experience. If the ratio is not determined correctly, the person initially relaxes and then becomes bored (challenge too easy in relation to ability) or feels great anxiety (challenge too demanding in relation to ability) [14,62]. Achieving the flow state is an important strategic effort for novices and experts alike. Habits of attention are important in order to maintain self-confidence during times of uncertainty while conserving energy to effectively adapt to challenges and use one’s skills [62].

Research over the past century, as noted by Whalen [62], has shown that fostering an environment in which an (otherwise complicated) combination of order and freedom is achieved can create a more effective learning context, compared to the classical approach of disciplined attention. In addition, people from environments where flow is encouraged tend to develop an openness to new experiences and the ability to focus and set realistic goals as part of self-regulation [64]. According to Palomäki et al. [67], the literature suggests that there is a connection between the flow state and task performance, but there are also empirical findings that suggest there could be another factor that moderates this link. Palomäki et al. [67] supported other research with their study, showing that when predicting the flow state, a deviation from the performance expectation is an even better predictor than the performance itself. Performance expectancy increases with new experiences in performing the task. They found that an additional factor or moderator associated with flow state and task performance is task experience [67].

A systematic review of flow experiences in game-based learning [68] also reported on individual factors of the flow state, as flow is a subjective experience. Factors that influence flow include interest in the topic, prior knowledge, etc. [68]. A literature review [63] summarized many influential connections of the flow state with cognition, personality, motivation, emotion, behavior, contextual factors, etc. A connection between the flow state and engagement, creative tasks, and creativity, which are important parts of design thinking, has been demonstrated. Furthermore, Yang and Hsu [69] noted that the flow experience could be improved with the help of design thinking. Although design thinking and systems thinking are not the same, they share several similarities (e.g., both require similar cognitive skills, such as analogy, synthesis, and human relations skills, both are applied in design and engineering, etc.), and in some respects, engineering systems thinking is a specific application of design thinking [70]. In addition, self-efficacy was found to improve when the design thinking method [63] was used, even among students who showed lower creativity tendencies [69]. Primus and Sonnenburg [71] showed that design thinking tasks have an influence on the flow experience at both the group and individual level. In addition, an influence was also demonstrated for so-called creative warm-ups and their interaction with design thinking tasks. Creative warm-up exercises have been found to improve individual flow in less stimulating tasks and in longer-lasting design thinking activities [71].

In his work, Pilke [66] highlighted research on flow and information technology, noting that achieving a flow state is likely the main goal of advertising. He found that people often experience flow when using information technology, such as image editing, writing, and programming. This technology facilitates the flow state, allowing for an examination of the quality of the user experience. Although Pilke’s research [66] dates back to 2004, when there were fewer digital applications and tools, the presence of flow in this technology was evident. Pilke emphasized the importance of quick feedback, visual elements, and aesthetics in maintaining a state of flow. In contrast, common challenges include reducing the cognitive load and improving user interface skills [66]. Regarding research on flow states and the use of ICT, there are numerous studies on flow while playing computer games [72,73], which found correlations between flow and motivation, engagement, and outcomes. Rutrecht et al. [72] reported that gamers had a better performance and sense of presence in the VR environment rather than in 2D. In education, the results of one study [74] showed that the factors influencing the flow state in the e-learning environment include the feeling of being in control of the virtual environment [75], attention and focus on the activity, and the feeling of being physically in the environment. Furthermore, the flow state was found to improved learners’ academic performance in an e-learning environment.

Flow theory focuses on the psychological state of the individual in terms of increasing engagement and performance. To provide a holistic overview of how to analyze and promote higher levels of satisfaction and engagement, we can complement flow theory with activity theory. The latter uses a broader sociocultural framework to understand human activity [14,76].

In examining developmental change, activity theory uses several perspectives: historical and situational, as well as individual and systemic. Activity theory was developed in the 1920s and is based on a sociocultural approach [58]. It is an approach to studying human interactions and relationships in specific situations, e.g., the use of digital tools and ICT in preschool. The theory considers the complexity of real-life activities, focuses on social practices, and is often used in the study of topics in teacher education. Key components of activity theory include the following [76,77]:

Subjects, i.e., individuals or groups engaged in an activity/activity (e.g., preschool teachers);
Objects of the activity, i.e., the goal/outcome of the activity (e.g., early robotics teaching);
Tools or mediating artifacts, i.e., instruments or aids for carrying out activities (e.g., teaching materials, drawings, markers, robot sets);
Community, referring to the context of or community involved in the activity (e.g., children in kindergarten);
Rules, i.e., norms and regulations that govern the activity (e.g., curriculum for kindergartens, house rules, kindergarten requirements and rules, educational institutions, ministries);
Division of labor, referring to the distribution of tasks among participants (e.g., which educator does what, preparation and purchase of materials, implementation, evaluation, reporting).

Engeström [78] summarized the main features of activity theory: the key unit of analysis (it is an object-oriented activity system that is multivocal and is related to other activity systems, including multiple views, perspectives, cultures), historicity (referring to the understanding of changes compared to the past), contradictions (representing the main source of change, the driving force of development), and the extension of information (referring to a newly conceptualized object or motive of activity) [78].

The two theories complement each other by emphasizing engagement in activities (social and psychological contexts of engagement) and the role of context, since despite the focus on psychological and internal factors, the environment in the state of flow must not be neglected. Understanding the role of context from both perspectives gives us guidelines for designing elements of the environment to provide engaging activities. In addition, both theories attach greater importance to feedback, which plays an important role in transforming activities from the activity theory perspective and in maintaining the flow state from the flow theory perspective, which is of great importance for improving the quality of activities. With a holistic view, it is therefore possible to create learning environments that meet the needs of individuals in several areas [14,65,76,77].

1.3. The Present Study (Aim, RQs, and Expectations)

The aim of the present study was twofold: (1) to explore the interplay between the systems thinking of pre-service teachers, their attitudes toward stimulating higher-order thinking, and their engagement in technology-enhanced learning from the perspective of flow theory, and (2) to reveal whether having an optimal experience with ICT has an impact on the performance of pre-service teachers in a design and technology course when mediated by systems thinking.

The present study addressed the following research questions (RQs):

RQ1: What is the self-reported level of systems thinking among pre-service teachers, and to what extent are the dimensions of systems thinking evenly developed among them?
RQ2: What are the attitudes of pre-service teachers toward teaching practices aimed at fostering higher-order thinking, and how do these attitudes influence their systems thinking abilities?
RQ3: How do pre-service teachers engage with a design, technology, and engineering course enhanced by ICT and digital tools, and are there particular dimensions of engagement that influence ICT-enhanced systems thinking?
RQ4: How do pre-service teachers perceive their engagement in a technology-enhanced design, technology, and engineering course through the lens of flow theory, and which components of flow impact their development of systems thinking skills?
RQ5: What is the impact of flow on the performance of pre-service teachers when mediated by systems thinking?

2. Materials and Methods

In the present study, the pedagogical focus was centered around technological and engineering literacy as the main outcome of a DTE course for pre-service teachers using ICT and digital tools. A set of practices proposed by ITEEA encompassing systems thinking, design thinking, making and doing, critical thinking, collaboration, communication, and attention to ethics [1] was used as the conceptual framework, reflecting the knowledge, skills, and attitudes toward DTE that pre-service teachers need in order to effectively implement DTE standards and achieve learning objectives defined by the curriculum.

A pre-experimental research design in the form of a one-shot empirical case study was used to (1) explore the potential of using several digital systems to support authentic pedagogical cases, and (2) identify potential relationships and generate hypotheses for further investigation. However, we are aware that due to the higher risk of confounding variables and biases, the results of pre-experimental studies must be treated with caution, and any comparisons should be made with similar care and precision [79].

2.1. Course Format

The aim of the DTE course is to promote and understand the importance of creative transformation and processing of different materials for child development. Students are expected to incorporate design and technology into their work in various areas of the kindergarten education process. They are expected to develop creativity in their work, responsibility for working safely with children, and critical thinking skills and introduce digital technology meaningfully into their work through the knowledge gained in the course. The teaching and learning methods used in the lectures include explanations, demonstrations, and discussions, supported by ICT, interactive content, and virtual reality. Laboratory work takes the form of workshops with active student participation and focuses on projects and curriculum objectives. Both methods integrate technology to promote design and systems thinking and work with interactive content in a collaborative environment [80,81].

Table 1 shows the design of the modules during the semester. The colored parts indicate the periods in which lectures (90 min), laboratory work (90 min), kindergarten (KG) activities (approx. 60 min per group), and consultations with faculty (approx. 30 min per group) took place.

In the lectures, students discussed the content of the course according to the syllabus from a theoretical perspective. They discussed different approaches to learning, as well as the properties of materials and their processing, etc., which they later tried out in practice in the laboratory work. Students learned about different materials, their properties and processing with safe use of working equipment in practice. The semester began with the processing of paper materials (paper, cardboard, corrugated cardboard), continued with artificial materials (EVA foam, PET bottles, plasticine and other artificial modeling materials, PLA filaments), and ended with the processing of wood. At the same tie, other materials were also used in the products, such as cork (stoppers) and metal (staples, rivets, wires). At the end of the semester, the students produced individual products, for which they had to find the necessary materials and manufacturing techniques based on a drawing. They also familiarized themselves with construction collections, in which they produced constructions with different motion transmissions. With regard to kindergarten activities, groups of students prepared activities in which the main activity is the production and processing of materials. Each group was tasked with designing a suitable product to be made in the kindergarten with children. Before the actual activity, they had a consultation, discussed the appropriateness of the activity with the kindergarten teacher, wrote a lesson plan, produced the product, and prepared the required materials [80].

ICT and digital tools were used in the DTE course at all levels according to the substitution, augmentation, modification, redefinition (SAMR) model [16], which is commonly used to categorize educational practices with digital technologies. Examples of the use of ICT and digital tools at the level of enhancement (substitution and augmentation) include classic presentations, real-time review applications that provide immediate feedback, spreadsheets for faster data processing and chart drawing, and simulated applications. However, at the level of transformation (modification and redefinition), digital technology has enabled changes in tasks in terms of creating animation with digital tools, 3D modeling and printing, early programming, etc.

During the lectures, students used various mobile applications for consolidation and assessment (e.g., Plickers, Mentimeter, Kahoot) and collaborative learning environments for online lectures (e.g., Miro, Microsoft Teams). In the laboratory exercises, students made particular use of ICT and digital tools such as smartphones, tablets, and laptops to use spreadsheets and analyze data, draw diagrams, show simulations of a hand mechanism, create stop-motion animation, make technical drawings, do initial programming and block coding, create 3D models and prints, simulate the transmission of movements via gears, etc. ICT and digital tools were also used for writing reports, monitoring the performance of colleagues through an online evaluation questionnaire, and, after their own activity in the kindergarten, creating a summary of the activities and experiences in an online collaborative environment (e.g., Miro), in which they could insert various graphics and graphical elements (Figure 1).

3.1.1. Sustainability in the DTE Course

The goals and criteria of the 2030 Agenda are aimed at reducing poverty, protecting our planet, realizing human rights, and achieving gender equality in the dimensions of sustainable economic, social, and environmental development. The measures and incentives address five important areas [22]:

People: poverty eradication, dignity, equality, healthy environment (SDGs 1–6);
Planet: protection from degradation, sustainable consumption and production, resource management, action on climate change (SDGs 7–10);
Prosperity: economic, social, and technological progress in harmony with nature, people enjoying a successful and fulfilling life (SDGs 11–15);
Peace: promotion of peaceful, just, and inclusive societies free from violence and fear (SDG 16);
Partnership: the mobilization of resources to implement the Agenda, global solidarity, focusing on the needs of the most vulnerable, cooperation among all countries (SDG 17).

The learning model applied in the DTE course primarily followed SDG 4, “Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”, by integrating lectures and internships in kindergartens where students could apply their new knowledge. The subject specifically touches on achieving target 4.2: “By 2030, ensure that all girls and boys have access to quality early childhood development, care and pre-primary education so that they are ready for primary education”. In addition, the DTE course was designed so that students could make products for use in kindergarten and learn to design products so that their manufacture would be suitable for kindergarten children. The materials used in the kindergarten are usually reused waste materials, which is also supported by the course design, e.g., an octopus made from a waste bottle (Figure 2a) and materials reused for kindergarten activities (Figure 2b, c), which is in line with SDG 12 (“Ensure sustainable consumption and production patterns”). In addition, certain lectures and activities in the kindergarten promote creativity and innovation (SDG 9), and in general the course strives for promoting equality, reducing inequality, and promoting gender equality, thus touching on SDGs 5 and 10 [22].

2.2. Sample

The study sample consisted of 65 pre-service teachers enrolled in a regular university course, technical education, during the third semester, conducted in blended learning form. A large majority of activities were carried out in face-to-face mode, and some of them through an online learning management system (synchronous and asynchronous) at the Department of Physics and Technology Education at the University of Ljubljana, the largest Slovenian state university. The 65 pre-service teachers (61 women and 4 men) were informed about the study at the beginning of the 2023–2024 academic year and agreed to participate without expecting any incentives. There were no dropouts, and only those who had incomplete data were excluded from the study (n = 7). The last time these teachers were exposed to any form of technical education or training was in primary school, and they did not attend any design, technology, or engineering courses in high school. Their age ranged from 20 to 22 years, and the mean score for age was calculated as 21.14 (SD = 0.39).

Since the sample reflected diversity in terms of demographics and socioeconomic status, the quality of the sample was assessed through power analysis using G*Power 3.1 (Heinrich Heine Universität, Düsseldorf, Germany) [82] to ensure that the sample size was adequate to provide statistically significant results. A power analysis with input parameters for a two-tailed test (α = 0.05, power(1-β) = 0.85) indicated that a total sample of 62 participants would be needed to detect a moderate effect size f² = 0.15 using the t-test with linear multiple regression and five predictors. The sample of 65 participants was adequate to test and answer the research questions. Participating pre-service teachers had limited practical experience, as the teacher training program included a 4-week school/kindergarten internship in the fourth and sixth semesters, whereas in the third semester, when this study was conducted, they had only 1 performance and 1 observation per methodical course (total of 10 courses). The sample was nationally representative, since it represented nearly 40 % of all pre-service preschool teachers enrolled in the second year of the study in all three Slovenian universities where the study program was organized and is delivered [80].

2.3. Measures

This study examined the variables of students’ systems thinking, their perception of stimulating HOTS, and the engagement and flow in the DTE course. Their grades on the final exam, expressed as percentages, were collected from the official information system at the Faculty of Education, University of Ljubljana. All 4 measures with the corresponding final instruments are attached in the Supplementary Materials, titled “Students’ perceptions of and experiences with the DTE course”.

2.3.1. Systems Thinking and Strategic Approach Orientation Scales

A systems thinking scale (STS) developed by Moore et al. [36] and adapted by Avsec et al. [83], comprising 15 questions, was used as a data collection tool in the present study. A three-factor structure was confirmed using exploratory factor analysis (EFA) with the maximum likelihood estimation method:

ST 1: Multiple causations possible, variation of different types (random/special) (6 items);
ST 2: Interrelations of factors, patterns of relationships, feedback behavior (5 items);
ST 3: Sequence of events and causal sequence (4 items).

As a complement to systems thinking, we added a scale to assess strategic approaches to study and changes, proposed by Moore et al. [36]. Three subscales were revealed using EFA: personal effort (3 items), reliance on authority (3 items), and strategic thinking (5 items). Additionally, Velicer’s MAP test [84] was used to determine the number of components, indicating an additional three factors.

Responses to the items of both scales ranged between 1 (never) and 6 (always) to assess individuals’ perception of their systems thinking or strategic approach orientation. Mean (M) and standard deviation (SD) were calculated for all subjects. All items are ordinal, and the instrument was validated in a previous study [83], confirming the reliability of the scale. The total score for systems thinking was computed by summing up the responses to all items, and scores ranged from 15 to 90, with higher scores indicating better systems thinking ability.

2.3.2. Attitudes toward Stimulating Higher-Order Thinking

Stimulating higher-order thinking (SHOT), a 7-point Likert-type scale, was also used as a data collection instrument in the present study. Responses ranged from 1 (strongly disagree) to 7 (strongly agree). The original SHOT instrument was developed by Wijnen et al. [85], comprising four factors, and for the purposes of this study we extended the original SHOT and adjusted it to our target group, resulting in a seven-factor solution instrument with 30 items, as also suggested by Wijnen et al. [85]. Since the original SHOT was aimed at measuring teachers’ attitudes toward stimulating higher-order thinking in students, we adjusted several items to fit the context of this study. The main focus of the questionnaire was pre-service preschool teachers’ perception of their attitude toward stimulating higher-order thinking in preschool kids. Since preschool education is very sensitive, many preschool teachers tend not to see the relevance of stimulating students’ higher-order thinking, or, even more, may not feel capable of engaging kids in activities to enhance that type of thinking. On the other side, several European Union and national projects and government incentives are operating in the educational sector to boost higher-order thinking skills from kindergarten to university. Thus, new insights into how to effectively integrate ICT and other digital tools in the pedagogical process are necessary. Therefore, we extended the original SHOT with scales of enjoyment, anxiety, and perceived difficulty. The new factor solution structure of SHOT is as follows:

Perceived relevance (PR, 4 items);
Perceived student ability (PSA, 6 items);
Self-efficacy (SE, 4 items);
Context dependency (CD, 4 items);
Enjoyment (ENJ, 4 items);
Anxiety (ANX, 5 items);
Perceived difficulty (PD, 3 items).

2.3.3. Student Engagement

Student engagement in DTE is a multifaceted issue that can be understood through several dimensions. For the purpose of this study, we used Avsec et al.’s [83] adapted survey, based on the original by Naibert and Barbera [86] and Diessner [87]. The instrument consists of 20 items, distributed into five dimensions. Conceptualizing student engagement involves understanding the interplay between behavioral, emotional, cognitive, and social/agentic dimensions. The dimensions used in the study were conceptualized as follows:

Behavioral engagement (BE): participating, making an effort, and having persistence in academic tasks, facing challenges, investing time and energy in learning (4 items);
Cognitive engagement (CE): investing in learning, self-regulating, and using deep learning strategies (4 items);
Emotional engagement (EE): showing interest in the subject matter, having a positive attitude toward the pedagogical process itself, belonging to school or classroom community (5 items);
Social engagement (SE): having a voice and owning the learning process by taking an active role (4 items);
Aesthetic engagement (AE): using sensory perception, having emotional responses through personal connection and aesthetic appreciation, showing creative expression and critical reflection (3 items).

A 6-point Likert scale was used to assess student engagement, from 1 (never) to 6 (always). Negatively worded items were reverse coded before analysis.

2.3.4. Flow

To measure the optimal experience in technology-enhanced DET activities, we used an adapted flow state scale (FSS), originally developed by Jackson and Marsh [88]. The instrument consists of 36 items, and for assessment we used a 7-point Likert-type response format (1 = strongly disagree to 7 = strongly agree). A nine-factor solution of flow with four items per construct is as follows:

Challenge–skill balance (CSB);
Action–awareness merging (AAM);
Clear goals (CG);
Unambiguous feedback (UF);
Concentration on task at hand (CTH);
Paradox of control (PC);
Loss of self-consciousness (LSC);
Transformation of time (TT);
Autotelic experience (AE).

The total score for flow ranged from 36 to 252 points, and the mean (M) and standard deviation (SD) were calculated for distinct components. It is possible that measuring individual components of flow offers a more robust foundation for evaluating the theoretical basis of the FSS than depending solely on an overall score, as argued by Jackson and Marsh [88], Šimleša et al. [89], and Palomäki et al. [67].

2.3.5. Student Performance

The final exam grade was used as a measure of student performance in the DTE course. The final exam was designed to measure students’ technological and engineering literacy as the main outcome. It comprised 15 tasks or questions measuring knowledge and skill on different taxonomic levels. Each test task was worth 1 point, and the maximum score was 15 points (100 %). The design of the final exam tasks followed a valid method for technological literacy testing [90], also relying on ITTEA guidelines and practices [1] and considering course syllabus learning objectives [80]. There were eight multiple-choice items (five options; 1 point for each correct choice), three fill-in-the-blank items (1 point for each correct blank), and four matching answers (1 point for each correct match). The test demonstrated moderate internal consistency (McDonald’s ω = 0.82) [91]. The final exam was created and administered using the Exam.net portal, an official portal used for examinations at the University of Ljubljana. Students attended the examination in situ on the university premises.

2.4. Data Collection and Analysis

2.4.1. Data Collection

Prior to the beginning of the study, ethical approval was obtained by the corresponding university committee. All required information was presented to participants, including the purpose of the study, nature of data collection, analysis and record-keeping, benefits, and the possibility to withdraw from the study, and those who gave consent were given the survey. The data collection was carried out during the 2023–2024 academic year at the end of the winter semester, in January and February 2024, using 1KA (https://1ka.arnes.si/) a portal recommended by the University of Ljubljana for conducting surveys. The survey consisted of four questionnaires, which were delivered in two rounds two weeks apart. In the first round, two instruments were delivered to participants: the STS complemented with strategic approach constructs, and SHOT. In the second round, when all activities in the DTE course were accomplished, participants were asked to rate their perception of engagement and rate their experience with the ICT and digital tools they used in the course (FSS). All four questionnaires were in digital form, and participants completed them as a part of an online course. For those who missed it the first time, we sent two reminders to complete the survey.

2.4.2. Data Analysis

Different software was used to analyze the data in this study descriptively and interpretively. The descriptive and inferential statistics, Shapiro–Wilk normality test, EFA, and reliability estimation were performed using IBM SPSS Statistics (version 25), while the variance-based mediation analysis was conducted using ADANCO 2.4 (https://www.composite-modeling.com/; accessed on 1 May 2024).

As a measure of the effect size, Cohen’s f² was used in mediation analysis for direct effects. The value of Cohen’s f² can be interpreted as an unsubstantial effect (f² < 0.02), weak effect (0.02 ≤ f² < 0.15), moderate effect (0.15 ≤ f² < 0.35), or strong effect (f² ≥ 0.35) [92]. For indirect effects, in this study, we used a mediation effect size measure, υ, developed by Lachowicz et al. [93]: the squared standardized υ is greater than 0.175 for a large effect, 0.075 for a medium effect, and 0.01 for a small effect [93,94].

The reliability of each scale and subscale was estimated using McDonald’s ω (see Table 2). It accommodates varying factor loadings and does not require the stringent assumption of tau-equivalence, making it suitable for a wider range of scales, especially those that may not be strictly unidimensional. Using McDonald’s omega can lead to better-informed decisions about the reliability of educational assessments, as argued by Dunn et al. [91].

3. Results

All participants in the study successfully completed all planned tasks in the DTE course, thereby meeting all requirements to qualify for the final exam. The average test score was M = 69.84 % (SD = 16.11 %), and scores ranged from 40 % (minimum) to 100 % (maximum). The values of skewness = -0.41 (SE = 0.29) and kurtosis = -0.69 (SE = 0.58) were between -2 and +2, which is considered acceptable to prove a normal univariate distribution [95]. The assumption of univariate normality was confirmed with the Shapiro–Wilk test (p > 0.05).

Next, data collected in survey were checked and cleaned until they were suitable for undergoing different analyses. In the next sections, the results are reported according to the research questions we created to guide this study.

3.1. Systems Thinking and Strategic Approach Orientation among Pre-Service Teachers

We analyzed the systems thinking scale separately from the strategic approach factors. Before conducting our analyses, we first evaluated the items of both scales to develop some hypotheses regarding the factor structure already validated in previous studies [36], [83].

3.1.1. Normality Tests and Validity Evidence

In this study, systems thinking is represented by three constructs, and the same for strategic thinking as a complement to systems thinking. The Shapiro–Wilk test results suggest that all six constructs follow a normal distribution (p > 0.05).

Next, we checked for possible multivariate outliers that might violate multivariate normality. For both systems thinking and strategic thinking constructs, the Mahalanobis distance of the cases did not exceed the critical value of 16.27 for three predictors. The probability of the Mahalanobis distance for each case is p > 0.001. In addition, we checked the multicollinearity of independent variables by calculating the variance inflation factor (VIF) for each variable, and all VIFs were less than the stringent threshold of 3, as proposed by Hair et al. [96].

Although the constructs were validated in previous studies [1], [83], we validated all constructs again to ensure that the test or instrument would reliably and accurately measure the theoretical construct it was intended to measure. Since the sample size was rather small, we used a mixed-methods approach to validate constructs.

Firstly, we conducted EFA with maximum likelihood extraction and oblique rotation, as suggested by Fabrigar et al. [97]. Bartlett’s test of sphericity was significant and indicated that the correlation matrix was not random: χ²(105) = 543,86, p < 0.001. The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy was 0.85, which is well above the minimum standard for conducting factor analysis (0.5) [98] and for analyzing the EFA output. The communality of all items extracted with maximum likelihood estimation (MLE) was above 0.55 (all values were between 0.55 and 0.75). The results of EFA suggested that the three-factor solution explained 64.6 % of the variance, which is above the threshold of 0.5–0.6 suggested by Hair et al. [99]. Loadings smaller than 0.5 were excluded from the pattern matrix and from further analysis. The same procedure was followed for strategic thinking constructs, for which Bartlett’s test of sphericity was also significant (p < 0.001) and KMO correlation was 0.73. All extracted communality values were above 0.6. EFA revealed a three-factor solution, and it explained 61.65 % of the variance.

Secondly, for convergent and discriminant validation of the constructs, we used the partial least squares (PLS) approach, which has several advantages when using small sample sizes (e.g., less stringent assumptions, ability to handle complex models, practical application and validation) [100]. For the purpose of the present study, ADANCO v.2.4 software was used [100].

Convergent validity refers to the degree to which multiple indicators of a construct are correlated. It emphasizes the internal consistency of indicators measuring the same construct [101]. Evidence for convergent validity is established when the hypothesized measurement model fits the data adequately, and additional criteria such as standardized factor loadings and average variance extracted (AVE) values are met [101]. As shown in Table 2 and Table 3, all AVE values are above the threshold of 0.5, whereas the square root of AVE (bold diagonal), McDonald’s ω, and composite reliability (CR) is larger than 0.7, which is the threshold suggested by Hair et al. [102].

The values of correlation coefficients (off-diagonal) indicate medium to large convergence [103] for both scales, as shown in Table 2 and Table 3. Thus, the results in Table 2 and Table 3 indicate convergent validity for the adapted constructs, and high convergent validity supports the retention of all dimensions of systems thinking and strategic approach orientation.

Since convergent validity is confirmed for both scales, we can continue with establishing discriminant validity. In other words, if a construct is not accurately represented by its indicators, examining whether it can be distinguished from other constructs is meaningless [101].

Discriminant validity refers to the extent to which a construct is truly distinct from other constructs [104]. It has been shown that constructs that are supposed to be unrelated show low correlations [101]. In this study, discriminant validity was examined using the heterotrait–monotrait (HTMT) approach proposed by Hensler et al. [105] and controlled using the Fornell–Larcker criterion, which checks whether the AVE associated with constructs is greater than the shared variance between constructs [106]. As shown in Table 4 and Table 5, the HTMT ratio of correlations is smaller than the threshold 0.85 [104], while the AVE values (on the diagonal) are larger than average shared variance (ASV) (in parenthesis).

A low ASV indicates that the constructs are distinct and measure different phenomena, while high shared variance suggests that the constructs overlap significantly, which could undermine discriminant validity [107].

Based on the results in Table 4 and Table 5, we conclude that all variables demonstrate discriminant validity. Moreover, validation indicates that each item loads uniquely on only one construct [101].

3.1.2. Levels of Systems Thinking and Strategic Behavior

The first aim of this study was to examine the level of systems thinking of pre-service teachers and their orientation toward their strategic approach for study and work as a complement to systems thinking. The descriptive statistics summarizing the self-reported characteristics of pre-service teachers are shown in Table 6.

The values for skewness and kurtosis shown in Table 6 are in the acceptable range (-2 and +2, -7 and +7, respectively) as proposed by Hair et al. [102], suggesting that the dataset possessed a normal distribution, also confirmed by the Shapiro–Wilk test (p > 0.05). An average total score for systems thinking of 72.4 points (76 %) is comparable with scores for nursing students, public health students [1], and architecture students [108], while medical students and mechanical engineering students scored lower, as reported in studies by Moore at al. [1] and Kurent and Avsec [108], which used the same instrument for data collection.

We also investigated whether systems thinking constructs were evenly developed in pre-service teachers or if there were significant differences among them. For that purpose, we used a repeated measures ANOVA, which met the assumption that the variances of differences between all combinations of related conditions (or construct levels) would be equal. Mauchly’s test indicated that the assumption of sphericity was not violated (χ²(2) = 1.08, p = 0.58 > 0.05). Repeated measures ANOVA determined that systems thinking varied significantly across different dimensions (F(2, 128) = 17.58, p < 0.001), with a partial η² effect size of 0.22. A post hoc analysis using Bonferroni correction for multiple comparisons to keep the type I error at 5 % overall showed a significant difference between ST 1 and ST 2 (p = 0.020) and ST 3 (p < 0.001). A significant difference was also found between ST 3 and ST 2 (p = 0.010). The results indicate that dimension of systems thinking ST3, understanding sequences of events and causal sequences, was most developed, while ST1, understanding the possibility of multiple causations and variations of different types in systems, was less developed.

To find out whether the means of the three strategic approach orientation constructs were different, one-way repeated measures ANOVA was used. Mauchly’s test, which evaluates the sphericity condition, showed that the assumption of sphericity was satisfied (χ²(2) = 0.98, p = 0.61 > p = 0.05) and the test of within-subject effects was significant (F(1,64) = 66.52, p < 0.001), with a partial η² effect size of 0.51. A pairwise comparison of mean differences based on estimated marginal means using Bonferroni adjustment for multiple comparisons to reduce the risk of committing a type I error revealed significant differences between reliance on authority and both personal effort and strategic thinking (p < 0.01), while perceived personal effort in the technology-enhanced DTE course significantly differed from strategic thinking (p = 0.009).

3.2. Pre-Service Teachers’ Attitudes toward Stimulating Higher-Order Thinking

3.2.1. Normality Tests and Validity Evidence

The original SHOT questionnaire developed by Wijnen et al. [85] has four attitudinal factors. For the purpose of this study, we added three new constructs, also suggested by the authors of the original tool. The Shapiro–Wilk test of normality was conducted to determine whether the data for all seven constructs were normally distributed. The results indicate that we failed to reject the null hypothesis for all constructs (p > 0.05), and we concluded that the data were normally distributed.

Next, we checked for possible multivariate outliers that might violate multivariate normality. For both systems thinking and strategic thinking constructs, the Mahalanobis distance of the cases did not exceed the critical value of 24.31 for seven degrees of freedom. The probability of the Mahalanobis distance for each case is greater than p > 0.001. In addition, we checked the multicollinearity of independent variables by calculating the VIF for each variable, and all VIFs were less than the threshold value of 3 proposed by Hair et al. [96].

We conducted EFA to identify the underlying structure of the data and refine the scale if needed. MLE was employed as the extraction method, and oblique rotation was used to allow for correlated factors. After extraction, the communality ranged from 0.56 to 0.92 [109], demonstrating the proportion of variance in each item that was actually explained by the extracted factors. Most items had high extraction communality, suggesting that the extracted factors provided a good representation of the data [109]. The resulting EFA revealed a seven-factor structure, as was anticipated.

In addition to the EFA, we calculated the AVE, average shared variance (ASV), composite reliability (CR), and McDonald’s ω to further explore the convergent and discriminant validity of SHOT subscales.

As shown in Table 7, all constructs of SHOT exhibit convergent validity, since all of the following three conditions are fulfilled: (a) CR and Mc Donald’s ω are greater than 0.7, (b) all standardized factor loadings λ are greater than 0.5, and (c) AVE is greater than 0.5, as suggested by Hair et al. [102].

In this study, we assessed discriminant validity using HTMT correlations. Following the guidelines proposed by Henseler et al. [105], we considered HTMT values below 0.85 as indicative of adequate discriminant validity. We computed HTMT values for each pair of constructs using ADANCO software [100].

An examination of discriminant validity showed that we fulfilled all four criteria: first, there is evidence of convergent validity; second, there are no indicator cross-loads on other constructs; third, AVE is equal or greater than 0.50 and greater than ASV [101,107]; and finally, all correlations between SHOT constructs are less than the threshold of 0.85 (Table 8) [104].

3.2.2. Pre-Service Teachers’ Attitudes toward Stimulating Higher-Order Thinking and the Relationship to Systems Thinking Ability

To address our second research question, first we used descriptive statistics to describe the mean values for pre-service teachers’ perceptions of SHOT subscales, as reported in Table 9.

Most mean values are above the midpoint of the scale (4), except PSA and ANX, which was expected based on the original research of Wijnen et al. [85]. For PSA, the results indicate that pre-service teachers believe that higher-order thinking is suitable for low-achieving kids. Moreover, pre-service teachers do not feel anxious when organizing and conducting activities for kids to enhance their higher-order thinking.

Although the assessment scale we used had 7 points compared to the original 5 points, the rank of subscales based on the mean is the same.

Next, we performed regression analysis to investigate whether scores on the SHOT subscales had predictive value for pre-service teachers’ systems thinking, as measured by the total score, which was represented as the sum of responses for each item on the scale.

Regression analysis revealed that SHOT subscale scores predicted pre-service teachers’ system thinking ability: adjusted R² = 0.35, F (7, 57) = 5.91, p < 0.001. As shown in Table 10, the attitude factors SE, PSA, and ANX are significant predictors for systems thinking (p < 0.05).

Pre-service teachers who considered themselves competent to post challenging questions and prepare tasks for kids to enhance HOTS also reported higher achievement in systems thinking. Moreover, pre-service teachers who were able to provide guidance and support for students during problem-solving activities using various types of supports report higher systems thinking ability. This capability may also enhance their effectiveness in self-regulation and problem-solving [110]. Systems thinking as a holistic approach is particularly useful for understanding and managing classroom dynamics and student behavior; it can help to alleviate feelings of helplessness and stress that often contribute to teacher anxiety, and can help them improve their methods and adapt to changing classroom dynamics, which can further reduce anxiety [31].

3.3. Pre-Service Teachers’ Engagement with Technlogy-Enhanced DTE Course

3.3.1. Normality and Validity Tests

To measure engagement in the technology-enhanced DTE course, we adapted a questionnaire already validated in another study [83]. Since the context, type, and quality of the sample and the setting of the current study differed, this could have affected the participants’ understanding and ability to respond to questions, and potentially impact the reliability and validity of the data. Thus, by reassessing and confirming the questionnaire’s factor structure, internal consistency, reliability, and construct validity, we could ensure that the instrument would function as intended.

The Shapiro–Wilk test of normality was conducted to determine whether the data of all five constructs were normally distributed. The results indicate that we failed to reject the null hypothesis for constructs of social and aesthetic engagement (p > 0.05), while for constructs of behavioral, cognitive, and emotional engagement, the test of the null hypothesis revealed that a set of the data came from a non-normal distribution (p < 0.05). We concluded that all variables did not follow normal distribution. For comprehensive understanding of the distribution characteristics of the data, we also calculated measures of skewness and kurtosis, as shown in Table 13. Skewness and kurtosis offer more nuanced information about the nature of the distribution’s deviation from normality and can inform appropriate data transformations or statistical tests.

Next, we checked for possible multivariate outliers that might violate multivariate normality. The Mahalanobis distance of the cases did not exceed the critical value of 20.51 for the five predictors (max. = 14.43). The probability of the Mahalanobis distance for each case was greater than p > 0.001. In addition, we checked the multicollinearity of independent variables by calculating the VIF for each variable, and all VIFs were less than the threshold value of 3 proposed by Hair et al. [96].

We conducted EFA to identify the underlying structure of our data and refine the scale if needed. MLE was employed as the extraction method, and oblique rotation was used to allow for correlated factors. The KMO value was 0.72, which is above the desirable value proposed by Watkins [98]. The results of Bartlett’s test indicated that the correlation matrix was not random (χ²(190) = 655,598, p < 0.001). After extraction, the communality ranged from 0.50 to 0.85 [31], demonstrating the proportion of variance in each item that was actually explained by the extracted factors. Most items had high extraction communality, suggesting that the extracted factors provided a good representation of the data [31]. Factor loadings λ smaller than 0.5 were excluded from the study. The resulting EFA revealed a five-factor structure explaining 67.76 % of the variance.

In addition to EFA, we calculated AVE, ASV, CR, and McDonald’s ω to further explore the convergent and discriminant validity of the engagement subscales.

As shown in Table 7, all constructs of engagement with the technology-enhanced DTE course exhibited convergent validity, since all of the following three conditions were fulfilled: (a) CR and Mc Donald’s ω greater than 0.7, (b) all standardized factor loadings λ greater than 0.5, and (c) AVE greater than 0.5, as suggested by Hair et al. [102].

Table 11. Reliability of McDonald’s ω, composite reliability (CR), square root of average variance extracted (AVE) (in bold), and correlations among engagement subscales (diagonal).

Latent construct	ω	CR	AVE	BE	CE	EE	SE	AE
BE	0.83	0.84	0.58	0.76
CE	0.77	0.78	0.51	0.58	0.71
EE	0.86	0.85	0.55	0.64	0.43	0.74
SE	0.73	0.75	0.50	0.21	0.02	0.07	0.70
AE	0.80	0.81	0.58	0.16	0.55	0.22	0.32	0.76

An examination of discriminant validity showed that we fulfilled all four criteria: first, there was evidence of convergent validity; second, there were no indicator cross-loads on other constructs; third, AVE was equal or greater than 0.50 and greater than ASV [101,107]; and finally, all correlations between constructs of pre-service teachers’ engagement in the technology-enhanced DTE course were less than the threshold of 0.85 (Table 12) [104].

3.3.2. Pre-Service Teachers’ Engagement in Technology-Enhanced DTE Course and the Relationship to Systems Thinking Ability

To address our third research question, first we used descriptive statistics to describe the mean values for pre-service teachers’ perceptions of engagement subscales, as reported in Table 13. As shown in the table, all mean values are above the scale midpoint of 3.5, supporting that student engagement in DTE was estimated as above average. Participants reported emotional engagement as highest and aesthetic as lowest. This indicates a higher level of satisfaction with the DTE course enriched with technology-related activities.

Table 13. Average values of pre-service teachers’ self-reported scores expressed as mean (M) and standard deviation (SD) across subscales of engagement along with measures of skewness (S) and kurtosis (K); 95 % confidence interval (CI) in brackets.

	Subscales	M	SD	S	K	95 % CI
Engagement with technology-enhanced DTE course	BE	4.97	0.65	-0.61	-0.17	[4.81, 5.13]
	CE	5.10	0.65	-0.67	-0.13	[4.94, 5.26]
	EE	5.38	0.62	-1.21	1.05	[5.23, 5.53]
	SE	4.47	0.75	-0.38	-0.54	[4.28, 4.65]
	AE	4.15	0.95	-0.28	-0.49	[3.92, 4.39]

The values for skewness and kurtosis shown in Table 13 are in the acceptable range (-2 and +2, -7 and +7, respectively) as proposed by Hair et al. [102]. Since the Shapiro–Wilk test gave a p-value indicating the probability that the data were not normally distributed, when interpreting this, we should consider the full context of our analysis. The sample size was rather small, thus was more prone to variability and may not have accurately reflected the population distribution, such that descriptive statistics like skewness and kurtosis can be misleading. In order to find a relationship between the engagement construct and systems thinking ability, we conducted linear regression analysis, which does not assume normality for either predictors or outcomes. Since the assumption of normality in linear regression applies to the residuals, we tested their normality using the Shapiro–Wilk test, and p > 0.05 for unstandardized and standardized residuals indicated normal distribution.

Regression analysis revealed that engagement subscale scores predicted pre-service teachers’ system thinking ability (adjusted R² = 0.36, F (5, 59) = 8.11, p < 0.001). As shown in Table 14, the factors CE, SE, and AE are significant predictors for systems thinking (p < 0.05).

Pre-service teachers who actively involved themselves in the DTE course by answering posted questions, doing conceptual mapping, and overcoming misconceptions scored higher on the systems thinking scale. Similarly, those who understood and built on peers’ ideas, shared their own ideas, and were keen on teamwork reported higher achievement in systems thinking. It seems that peer support might enhance systems thinking. On the contrary, pre-service teachers who focuses on artistic values in the DTE course and integrated an aesthetic vison in technology-enhanced design-based work reported lower achievement in systems thinking.

3.4. Pre-Service Teachers’ Perception of Flow State in DTE Course and the Relationship to Systems Thinking

3.4.1. Normality and Validity Tests

To measure the flow state of students in the technology-enhanced DTE course, which occurs when students are totally connected to the performance and their personal skills are equal to the required challenges, we used an adapted FSS, developed by Jackson and Marsh [88]. Because the context, type, and quality of the sample and the setting of the study, which can affect students’ understanding and responses to questions, were different compared to previous studies, we reassessed the questionnaire’s factor structure, internal consistency, reliability, and construct validity. In this way, we could ensure that the instrument functioned as intended. The Shapiro–Wilk test of normality was conducted to determine whether the data of all nine constructs were normally distributed. The results indicate that we failed to reject the null hypothesis for constructs CSB, UF, CTH, LSC, TT, and AE (p > 0.05), while for AAM, CG, and PC, the test of the null hypothesis revealed that a set of the data did not come from a normal distribution (p < 0.05). We concluded that all variables did not follow a normal distribution.

Next, we checked for possible multivariate outliers that might violate multivariate normality. The Mahalanobis distance of the cases did not exceed the critical value of 27.87 for the nine predictors (max. = 27.23). The probability of the Mahalanobis distance for each case was greater than p > 0.001. In addition, we checked the multicollinearity of independent variables by calculating the VIF for each variable, and all VIFs were less than the liberal threshold value of 5 proposed by Hair et al. [96].

We calculated AVE, CR, and McDonald’s ω to further explore the convergent and discriminant validity of the flow state subscales. As shown in Table 15, all constructs of students’ engagement with the technology-enhanced DTE course from the perspective of flow exhibit convergent validity, since all of the following three conditions are fulfilled: (a) CR and Mc Donald’s ω are greater than 0.7, (b) all standardized factor loadings λ are greater than 0.5, and (c) AVE values are greater than 0.5 as suggested by Hair et al. [102].

The interconstruct correlation values (diagonal) range from 0.28 to 0.85, indicating that the measures have medium to large convergence [103].

Next, we examined discriminant validity using the HTMT approach, as proposed by Hensler et al. [105] and Shaffer et al. [111]. Table 16 shows that a few of the HTMT ratios of the correlations exceed even the liberal threshold of 0.90 [111]. With a small sample and a larger number of constructs, as in our case, establishing discriminant validity using a fixed threshold like 0.90 without considering statistical inference can lead to incorrect conclusions [105]. In this case, the standard approach of Fornell–Larcker has unacceptably low sensitivity, which means that it is largely unable to detect a lack of discriminant validity [112]. Thus, we used as a complement the HTMT_inference criterion, as proposed by Henseler et al. [105]. To implement HTMT_inference we used a bootstrapping procedure (specifying a bootstrap sample of 2000) to generate confidence intervals around the HTMT values. The upper bound of the confidence interval for the HTMT value was below 1, suggesting that the constructs are distinct, which supports discriminant validity, as proposed by Henseler et al. [105].

In Table 16, the results marked in bold indicate a discriminant validity problem according to the HTMT_0.90 criterion, while HTMT_inference does not indicate a discriminant validity problem in these cases (confidence intervals: CTH<->AE [0.78,0.99], CTH<->CG [0.77,0.98], UF<->CG [0.87,0.97], UF<->CSB [0.86,0.99], and UF<->PC [0.87,0.99]).

3.4.2. Flow State and Predictive Analysis

To address our fourth research question, first we used descriptive statistics to describe the mean values for pre-service teachers’ perceptions of flow state subscales, as reported in Table 17. As shown in the table, all mean values are above the scale midpoint of 4, supporting that the student flow state in DTE was estimated as above average. Students scored the autotelic experience highest and time transformation lowest. In general, this indicates a higher level of flow state perceived in the DTE course enriched with well-designed technology-enhanced activities.

The values for skewness and kurtosis, shown in Table 17, were within the acceptable range (-2 and +2, -7 and +7, respectively), as proposed by Hair et al. [102]. In order to find a relationship between FSS constructs and systems thinking ability, we conducted linear regression analysis, which does not assume normality for either predictors or outcomes. Since the assumption of normality in linear regression applies to the residuals, we tested their normality using the Shapiro–Wilk test, and p > 0.05 for unstandardized and standardized residuals indicated normal distribution.

Regression analysis revealed that self-reported flow predicted pre-service teachers’ system thinking ability (adjusted R² = 0.56, F (9, 55) = 7.99, p < 0.001). As shown in Table 18, the factors AAM, CG, CTH, TT, and AE are significant predictors of systems thinking (p < 0.05). Collinearity statistics revealed that all VIFs are smaller than the liberal threshold of 5, proposed by Hair et al. [96], while the tolerance value is less than 1 for all cases.

As shown in Table 18, we found both positive and negative predictors of FSS in systems thinking. When experiencing flow, at least one dimension is present, but typically all are present [64]. The flow dimensions can be classified into three phases, as proposed by Šimleša et al. [89]: First is the pre-experience phase, which involves a balance between challenge and ability, clear goals, and immediate feedback. Second is the experience phase, which is perceived during flow and involves a combination of activity and awareness, concentration, and a sense of control. Third is the effects phase, which describes the individual’s internal experiences, such as loss of self-awareness, altered experience of time, and autotelicity. Autotelicity of experience refers to self-upgrading, where an intrinsic motivation can be seen as a product of satisfying the fundamental needs of perceived competence, perceived autonomy, and relatedness [67].

As shown in Table 18, all phases of flow have predictive value in systems thinking. Students who were so deeply involved in the flow during activities that it became spontaneous or automatic reported higher scores in systems thinking ability. The same was true for students who perceived that the learning goals were clearly defined and engaging in the course was an intrinsically rewarding experience. It seems that autotelic experience as a dimension of flow and as the effect phase might affect systems thinking, while a loss of the sense of time will predict lower scores in systems thinking. The same was found in the experience phase regarding the perception of concentrating on the task at hand. Students who explicitly focused strongly on the task reported lower scores in systems thinking.

3.5. Relationship between Pre-Service Teachers’ Self-Reported Flow and Their Achievement in DTE Course Mediated by Systems Thinking

The exploration of the relationship between flow state, systems thinking, and achievement began with a review of the relevant bivariate correlations (see Table 19). As expected, flow as measured by the self-reported FSS was positively correlated with systems thinking and final exam achievement. This correlation was statistically significant at the 0.01 level.

This preliminary exploration served several important purposes: establishing correlation and causation, assessing the presence of mediation, avoiding misinterpretation due to confounding variables, making an informed decision about sample size, and ensuring that constructs were valid and reliable, as argued in [113].

To examine the relationship between flow and students’ achievements in the DTE course as represented by the final exam score, and the possible mediational role of systems thinking, path analysis was used to test the proposed model and study hypotheses. The analysis was conducted to estimate the direct and indirect paths, as depicted in Figure 3. All analyses were conducted with ADANCO software using variance-based SEM, as proposed with strict guidelines by Nitzl et al. [114].

The path analysis results show a significant direct effect of flow on systems thinking (β = 0.51, p < 0.001, Cohen’s f² = 0.36 (strong effect size)) and a nonsignificant effect on final exam score (p > 0.05). The effect of systems thinking on final exam score was significant (β = 0.48, p < 0.001, Cohen’s f² = 0.25 (moderate effect size)).

The bootstrap estimation procedure (specifying a bootstrap sample of 1999) was used to test the significance of the mediation effect of systems thinking on the relationship between flow and final exam score. The results indicate that systems thinking mediates the relationship between flow and final exam score (indirect effect = 0.25, CI [0.11, 0.38], υ² = 0.053), and the effect size can be classified as medium [94]. Therefore, the self-reported flow pre-service teachers experienced in the technology-enhanced DTE course exerted a significant indirect effect on their academic achievement through systems thinking.

4. Discussion

The following subsections present the findings of our study that address the interplay between pre-service preschool teachers’ systems thinking and their attitudes toward stimulating HOTS and engaging in a technology-enhanced educational environment from the perspective of flow theory. Findings on the potential impact of an optimal ICT experience on pre-service preschool teachers’ performance in a DTE course mediated by systems thinking are also presented.

4.1. Self-Reported Systems Thinking and Strategic Approach Orientation of Pre-Service Preschool Teachers

In the following, we answer RQ 1, which refers to the self-assessed level of systems thinking of pre-service preschool teachers and the even development of its dimensions. The level of systems thinking of pre-service preschool teachers in terms of both the dimensions and the total score was above average and comparable to that of nursing students [36]. The results indicate differences in self-reported levels of development of all dimensions of systems thinking and some differences in the strategically oriented approach. Pre-service teachers gave the highest assessment for items in the dimensions of event sequence and causal sequence (examples include “I think that systems are constantly changing”, “I recognize system problems are influenced by past events”, “I consider the past history and culture of the work unit”, “I consider that the same action can have different effects over time, depending on the state of the system”). This could be because students follow a specific sequence (structure of activities) when preparing for kindergarten activities and learn the importance of children developing a concept of sequence (numerical or otherwise). Sequences form the basis of children’s mathematical thinking, counting, and computational thinking [115], and are increasingly being incorporated into curricula, as lessons on coding and early robotics (concepts of sequences, algorithms, loops, debugging, etc.) are encouraged in the preschool years [21,115,116]. Number sequencing is probably the first abstract concept children learn. Sequencing is also constantly practiced later, e.g., when developing morning and daily routines in kindergarten. Sequence-based activities such as storytelling, cooking/following a recipe, dancing/learning choreography, making ornaments and products, and doing other project-based activities are also very present in the preschool years [56].

Pre-service teachers rated the dimensions of relationships and feedback statistically significantly lower than sequencing, and the dimension of variety of causes and variations lowest. The results align with [117], in which the authors propose paying more attention to feedback, given its lower assessment. In fact, feedback has already been found to be challenging in the education process [118]. De Klejin [119] proposed a model to support the feedback literacy of students and teachers, which comprises four phases: searching for feedback information, making sense of the feedback information, using the feedback information, and responding to feedback information. These phases are nonlinear and interrelated [119]. The concept of sequence may also be easier to understand than the dimensions of relationships and feedback or cause–effect relationships, which were rated lower in this respect, as expected.

As far as the extended dimensions related to strategic approach orientation, there were statistically significant differences between all dimensions. Students rated the dimension of personal effort highest, including items such as “I concentrate on the effort people put into their work”, “I think that lasting change relies on personal effort and motivation”, and “I think people who do not get their desired results/outcomes did not work hard enough.” According to Gray and Mannahan [120], students connect personal effort with academic achievement, as well as other factors such as liking the teacher, liking the subject, and natural ability. Due to the emphasis on personal effort, it is to be expected that reliance on authority (example: “I think the leaders of the organization have the best ideas”) was rated lower in this study. Depending on authority can also be an obstacle to developing one’s own strategies and working methods. In addition, modern approaches to education are changing and becoming more child-centered, moving away from traditional authority-based models. Educators are embracing diversity and adapting to different kinds of children in this respect, so that “blind” following of authority is no longer prevalent and innovative approaches are given more weight [121].

4.2. Pre-Service Preschool Teachers’ Attitudes toward Stimulating Higher-Order Thinking Skills in Relation to Systems Thinking

The subsection answers RQ2, which deals with pre-service teachers’ attitudes toward teaching methods that aim to promote HOTS and how these attitudes might affect their systems thinking. The rating of items was mostly above average, which indicates that the teachers generally consider stimulating HOTS in children to be important and have a rather positive attitude toward it. This supports the findings of Aisyah et al. [49], who also found that in order to promote HOTS, most preschool educators provide different contexts in which children can develop HOTS and optimize situations for the purpose of promoting HOTS. The importance of HOTS was shown by Frausel et al. [122], who established a link between higher-order thinking talk and subsequent higher achievement in HOTS in elementary school. On the other hand, the dimensions of perceived student ability and anxiety were rated lower than the average. Given the items “I think that ’weak’ students cannot handle assignments that require higher-order thinking” and “I think that most assignments that require higher-order thinking are too difficult for ’weak’ students”, pre-service teachers did not see much point in developing HOTS for less gifted children. This could be because preparing such activities is more demanding and requires a lot of energy, which is also related to the dimension of anxiety [123]. For the items in the anxiety dimension, which mainly relate to feelings of discomfort when preparing tasks, activities, and discussions that require HOTS, pre-service teachers indicated lower agreement that they do not experience feelings of anxiety when preparing HOTS activities.

The results show that there is a relationship between certain dimensions of HOTS stimulation and systems thinking, namely, that pre-service teachers who scored higher in the areas of self-efficacy and perceived ability and lower in anxiety rated their systems thinking higher. Pre-service teachers who rated their self-efficacy higher, i.e., they noted that they were able to pose questions to stimulate HOTS, prepare and instruct tasks to stimulate HOTS etc., also rated their systems thinking skills higher. This is a logical consequence considering the close relationship between HOTS and systems thinking [47] and the connection between the positive influence of mastering beliefs and actual performance, as claimed by Brauner [124], who studied the latter in STEM subjects.

Pre-service teachers who believed that activities that require HOTS are also suitable for weaker children and that they can also develop HOTS also rated their systems thinking higher. The perception and understanding of children’s needs and how to stimulate HOTS require a broader understanding in order to identify patterns and relationships, and recognize how various factors influence children’s learning and development, which is consistent with the theory of systems thinking [3,35]. Anxiety is seen as a mental and physical state of negative expectation, and systems thinking helps to reduce it by being able to understand and manage complex situations [3]. The items in the anxiety dimension refer to the teachers’ feelings of being stressed, anxious, and tense when guiding children through assignments, questions, and open-ended discussions to stimulate HOTS. In other words, pre-service teachers who reported higher levels of systems thinking did not report feeling tense, uncomfortable, or stressed when stimulating HOTS by preparing activities, asking questions, etc. This is to be expected, as both HOTS and systems thinking are interdisciplinary and require effort and an understanding of multiple perspectives and aspects [2, 3, 35]. In this regard, it is important to be aware of the interdependence of content when stimulating HOTS and systems thinking.

4.3. Pre-Service Preschool Teachers’ Engagement in ICT and DTE Course Enhanced by Digital Tools

The following describes the engagement of pre-service preschool teachers in the DTE course supported by ICT and digital tools, and specific dimensions of engagement that influence ICT-enhanced systems thinking (RQ3). At all levels of engagement that were reviewed, pre-service teachers generally rated engagement above the median in the technology-enhanced DTE course, but differences emerged in their reporting of emotional and aesthetic engagement. Emotional engagement items addressed the excitement, enjoyment, or boredom regarding activities related to using ICT, while aesthetic items referred to the awareness of creative and interpretive processes and integration of an aesthetic vision in working with ICT and digital tools. Engagement is seen as an important factor, providing a unique insight into student satisfaction. In his research, Deng [125] showed that emotional engagement is the most important factor influencing learner satisfaction in Massive Open Online Courses.

High emotional engagement when working with digital tools and ICT could be related to self-concept with regard to ICT, which was generally reported to be above the median in [117]. The exciting, satisfying component of emotional engagement can also be affected by the difficulty of the task and further optimal experiences with ICT [63,67], while lower aesthetic engagement would not be expected when working with digital tools and ICT, given the preschool teachers’ general perceived importance of aesthetics. In contrast, it is likely that they suppressed or did not pay as much attention to the aesthetic component given the nature of the DTE course, which emphasizes material knowledge, processing technology, and the functionality of products and systems rather than visual appearance [81]. In addition, preschool teachers tend to be creative, since creativity is known to be crucial in early education [126], and this was probably limited given the tasks they worked on with ICT and digital tools, as the applications they used included pre-made elements, avatars, and scenes. On the other hand, it might be less complicated to produce more aesthetically elaborate products (graphic designs, presentations, etc.) with digital support than with hands-on activities, but this strongly depends on the task, purpose, applications, tools, etc.

It was statistically significant that pre-service teachers who were more cognitively and socially engaged and less aesthetically engaged reported higher levels of systems thinking. In this regard, cognitive and social engagement are expected predictors of systems thinking, as applying systems thinking is extremely challenging for individuals in a complex environment, and is easier in a group [127]. Groups with effective collaboration are better able to make decisions because they have more knowledge and experience, along with more opinions and interpretations. This finding is supported by research showing that heterogeneous teams perform better than homogeneous teams. The effective functioning of a group requires the development of group skills, which is considered challenging and can only take place in a group [127]. The aesthetic component is not emphasized in the literature on systems thinking [3, 35], so the results align with expectations based on the theory. In contrast, White [128] emphasized aesthetics in the context of systems research and reflection. Paying attention to aesthetics as a method of cognition is important in understanding systems practice. The author presents aesthetics as a means of challenging systemic thinking through what he calls critical imagination, a way of looking at the emotional and symbolic aspects of decision making and organization in complex environments [128].

4.4. Pre-Service Preschool Teachers’ State of Flow in the Technology-Enhanced DTE Course

This subsection addresses RQ4, which deals with the perception of flow among pre-service preschool teachers engaging in a technology-enhanced DTE course. In addition, the components of flow that may impact their development of systems thinking skills are described. Pre-service preschool teachers rated the state of flow in relation to the use of ICT in the DTE course above average, indicating they did enter the flow state. The autotelic dimension (item example: “The experience left me feel great”) was rated highest, followed by the balance between challenge and skills and concentration on the task (item example: “I had total concentration”). Transformation of time was rated lowest, although still above average. The autotelic experience is closely related to digital entertainment such as games [73], which could be linked to certain tasks that the teachers carried out using ICT as part of the DTE course, and a sense of fun/play in game-based learning (e.g., using Scratch and Scratch Junior for programming, a stop-motion video application to create animations, or Tinker Cad for 3D modeling) [68].

The low score for the time transformation dimension (examples include “The way time passed seemed to be different from normal” and “Time seemed to alter”) can be explained by the fact that the exercises and lectures always lasted a maximum of 90 minutes [81], and pre-service preservice teachers did not interact with ICT the whole time because the introductory part, the explanation of the theory, the instruction, or other preparations were taking place. The time transformation was therefore probably not as pronounced as it could have been due to the already time-limited activities. Furthermore, the awareness of time limitation can significantly impact the ability to enter the flow state. The less one thinks about time, the greater the chances of getting into flow [72]

In general, it was found that pre-service teachers who achieved a higher state of flow also rated systems thinking higher. Clear goals preserve the objective of complex systems, and receiving feedback is already clearly defined in the theory and, therefore, a common attribute of both concepts [31,63], as well as concentration, which was expected to lead to deeper analysis and synthesis. The results show that the dimensions of merged action and awareness, clear goals, concentration on the task at hand, time transformation, and autotelic experience when using ICT and digital tools had statistically significant predictive value for systems thinking; in contrast to the other dimensions, concentration on the current task and time transformation had negative predictive value. This may be the case because systems thinking refers to a broader view and a bigger picture and does not rely too much on details or so-called tunnel vision, which is certainly the case when there is a high focus on the task. A higher focus during complex tasks was found to lead to higher performance [129], but attention to details could lead to difficulty distinguishing between important and unimportant details, failure to understand how the problem of the task fits into the big picture, blurring of the importance of other factors or perspectives, being able to adapt to changing circumstances, and spending a lot of time on unimportant aspects of a problem, which are detrimental to the overall effectiveness of systems thinking [3,31,36].

Given that the flow state and systems thinking both require engagement and involve complexity and challenge, and place importance on feedback and adaptation [63,66], we would expect this dimension of systems thinking to have significant predictive validity. This relation should be investigated further, as the feedback dimension was poorly developed in the sample of pre-service preschool teachers [117].

4.5. Impact of the State of Flow on Pre-Service Preschool Teachers’ Performance Mediated by Systems Thinking

In the following we answer RQ5, which refers to the influence of flow on the performance of pre-service teachers when mediated by systems thinking. Regarding the flow state as it related to the ICT-enhanced DTE course, it had no significant effect on the final exam score (indicating technological and engineering literacy), but it had a significant effect on systems thinking, which in turn had an effect on the final exam score. Thus, the state of flow related to ICT had an indirect effect on the assessment of DTE, and its mediator was systems thinking.

The effect of flow on systems thinking when using ICT aligns with the literature, which suggests the need to support systems thinking with ICT and other digital tools [33,44]. In the educational setting, achieving a flow state can enhance the learning experience by fostering deep engagement and sustained attention. This relationship between flow state and task performance has been documented, indicating that individuals who experience flow are more likely to perform tasks effectively and efficiently [63,67].

Furthermore, systems thinking, defined as the ability to understand and analyze complex systems by recognizing interconnections and patterns, is crucial for developing comprehensive problem-solving skills [31]. The literature suggests that ICT and digital tools can significantly enhance systems thinking by providing interactive and dynamic learning environments [33,44]. In addition, the relation between systems thinking and ICT has already been investigated. Systems thinking improves students’ self-concept regarding ICT [117], which is related to their confidence in using digital tools.

The study’s findings suggest that the flow state experienced during the ICT-enhanced DTE course indirectly affected the final exam scores through its impact on systems thinking. This implies that while flow itself may not directly enhance technological and engineering literacy, it fosters a mindset conducive to systems thinking, which in turn positively influences academic performance [63]. This mediation effect underscores the importance of integrating systems thinking into educational curricula to effectively harness the benefits of the flow state. This is particularly relevant in the context of technology and engineering education, where systems thinking is considered a fundamental practice [1]. Additionally, the combination of systems thinking with project-based learning has proven effective for developing an understanding of engineering problems among pre-service teachers, even those with limited prior knowledge of engineering and technology content [46]. Systems thinking has also been shown to improve skills related to engineering and technology and is considered an important complement to traditional teaching methods [40,42,43]. This highlights the need for educational strategies that not only engage students but also develop their systems thinking, thereby improving their overall academic performance.

4.6. Limitations, Implications, and Future Work

As with all research, especially in the field of education, this study has some limitations. The sample studied consisted of students from the Faculty of Education at the University of Ljubljana. The results could be different with a larger sample. It would therefore make sense to include students of this degree program from other universities in Slovenia in the sample. The sample was not analyzed by gender because, due to the nature of the work, most preschool teachers are women. Another limitation of the study can be seen in the experimental design, as it was a pre-experiment to test the feasibility of the main study to be carried out next year. For example, no randomization, no pre-post test to determine progress, and no control group were used in the study, which should be improved in the main study. An important aspect of the research is also the use of self-assessment questionnaires, which do not necessarily reflect the actual situation, thus more objective methods of measuring characteristics would have to be used. The latter is especially true for assessing the level of systems thinking, in contrast to flow/optimal experience, engagement, and attitude toward stimulating HOTS, where self-assessment questionnaires are useful.

Additionally, in research attention must be paid to the quantity of measurement tools, because when many questionnaires are used, participants may become saturated and not assess the actual situation, but just complete the questionnaires as quickly as possible. Given the flood of surveys, this is an important factor to consider in further research.

There are also some implications of the study. The findings regarding the mediating role of systems thinking in student performance indicate that while the state of flow does not directly enhance technological and engineering literacy, it fosters a mindset conducive to systems thinking, which subsequently positively influences academic performance. This mediating effect highlights the critical importance of incorporating systems thinking into curricula to effectively leverage the benefits of the flow state. The flow state associated with ICT and DTE courses indirectly enhances assessment outcomes by promoting systems thinking. In conjunction with other models for integrating ICT and digital technology, flow has been demonstrated as a suitable complement to systems thinking, enabling the achievement of positive outcomes through appropriate application. This underscores the need for educational strategies that not only engage students but also cultivate their systems thinking skills, thereby enhancing their overall academic performance. Through synergies between systems thinking and flow theory, educators and instructional designers can create technology-enhanced learning environments that promote the flow state and enable learners to engage and achieve in their studies to develop 21st-century sustainable skills.

There are a variety of factors (cognitive, behavioral, motivational, emotional, contextual, cultural, etc.) in educational environments that influence the educational process and all actors involved. It would be advisable to use the findings of this study to improve the learning process to achieve a flow state in multiple dimensions, improve systems thinking, and increase student performance. It would also be advisable to investigate students’ preferences for subjects, their attitudes toward subjects and topics, and their views on the importance of subjects in pre-school education. As Kartal and Taşdemir [130] stated, female pre-service teachers should be encouraged to develop a positive attitude toward engineering and technology. In addition, it would be advisable to investigate students’ self-efficacy, self-direction, self-regulation, etc., when new approaches are introduced. As mentioned in previous research [117], more attention needs to be paid to feedback. In addition, the lack of development of relational understanding should be considered and an optimal environment to improve the learning flow should be found to achieve better learning outcomes. As the study was a pre-experiment prior to the main study, we plan to improve the learning–teaching model to promote technological and engineering literacy using the technology-enhanced systems thinking approach. Our enhancement of the learning mode will be extended using robotic sets suitable for early coding in kindergarten, with the goal of improving preschool teachers’ understanding of functional dependencies, relations, and feedback as critical dimensions of systems thinking. We also aim to include VR content to achieve a better time transformation and thus a flow state [72], in order to develop emotional engagement and empathy as important parts of the design of the DTE course.

5. Conclusions

Finally, we present the most important results of the pre-experiment. It was found that there is still room for improvement in the systems thinking of pre-service preschool teachers, particularly in terms of feedback and understanding interrelationships. With regard to the attitude toward stimulating HOTS, it was found that preschool teachers had a rather positive attitude. A predictive value for systems thinking was shown for the component of self-efficacy and perception of children’s abilities. Pre-service preschool teachers who know children better, recognize their needs, adjust their level, and feel that tasks that stimulate HOTS are not frustrating or too challenging for weaker children have more developed systems thinking according to their self-assessment. The level of systems thinking was also rated higher by teachers who did not feel uncomfortable, stressed, or frustrated when preparing activities and discussions to stimulate HOTS.

Pre-service teachers engaged in the targeted ICT-enhanced DTE course at all levels showed greater engagement with ICT and digital tools at the emotional level and slightly lower engagement at the aesthetic level. The predictive value of such engagement for systems thinking was evident in cognitive, social, and aesthetic engagement, the latter in a negative direction. Pre-service teachers who were more cognitively engaged (e.g., linking knowledge, understanding errors) and socially engaged (understanding others’ ideas, collaborating with others, etc.), and less aesthetically engaged (e.g., contributing an aesthetic vision) with ICT activities reported higher systems thinking. In addition, entering the flow state during activities with ICT was assessed, and it was found that autotelic experience led to flow most frequently, as opposed to the time dimension, suggesting that teachers did not perceive time too differently while using ICT in the DTE course. The predictive validity of the flow dimensions in relation to systems thinking was shown here to be in a positive direction for clear goals, merged action and awareness, and autotelic experience, while teachers who were more concentrated on the tasks and perceived time differently during the ICT tasks rated systems thinking lower.

Lastly, the path analysis pointed to the mediating value of systems thinking between the flow state in the use of ICT and learning performance in DTE. This indicates the useful value of ICT-enhanced systems thinking for the development of learning performance in design, engineering, and technology education. In an increasingly digital world, the convergence of systems thinking, and technology-enhanced learning offers the opportunity to revolutionize educational experiences. This research underscores the applicability of flow theory to exploring the synergy between systems thinking and technology-enhanced teaching. It offers a holistic approach to fostering engaging, immersive, and impactful learning environments and suggests the need for further exploration of systems thinking among students who are not as engaged in technology-enhanced DTE courses to identify factors for suitable approaches that promote effective education for all.

Supplementary Materials

The following supporting information can be downloaded at the website of this paper posted on Preprints.org, Students’ perceptions of and experiences with the DTE course.

Author Contributions

Conceptualization, B.K. and S.A.; methodology, B.K. and S.A.; validation, B.K. and S.A.; formal analysis, B.K. and S.A.; investigation, B.K. and S.A.; resources, B.K. and S.A.; data curation, S.A.; writing—original draft preparation, B.K. and S.A.; writing—review and editing, B.K. and S.A.; visualization, B.K. and S.A.; supervision, B.K. and S.A.; project administration, B.K. and S.A.; funding acquisition, B.K. and S.A. All authors have read and agreed to the published version of the manuscript.

Funding

The authors acknowledge the financial support of the Slovenian Research Agency under the research core funding Strategies for Education for Sustainable Development applying Innovative Student-Centred Educational Approaches (ID: P5-0451) and under the project Developing the Twenty-first-century Skills Needed for Sustainable Development and Quality Education in the Era of Rapid Technology-Enhanced Changes in the Economic, Social and Natural Environment (Grant no. J5-4573) also funded by the Slovenian Research Agency. The authors would also like to thank the pilot project ULTRA 5.02-1554 Improving digital skills and competences of (future) educators for quality educational work with younger children, funded by the Republic of Slovenia, the Ministry of Higher Education, Science and Innovation and the European Union—NextGenerationEU.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and with the ethical principles and integrity in research of the University of Ljubljana, Slovenia. The study was approved by the Department of Physics and Technology Education of the Faculty of Education at the University of Ljubljana (approval number 2022OFT01/09).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available upon request from the author. The data are not publicly available due to privacy issues.

Acknowledgments

The authors thank the participating pre-service teachers at the University of Ljubljana, Faculty of Education Ljubljana, Slovenia.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

International Technology and Engineering Educators Association. Standards for technological and engineering literacy: The role of technology and engineering in STEM education; ITTEA: United States, 2020; www.iteea.org/STEL.aspx.
Bavli, B. Higher Order Thinking Skills (HOTS) in the 21st Century: Challenges and Opportunities for Higher Education. Multidiscip. J. Sci. Educ. Art 2023, 62–71. [Google Scholar]
Cabrera, D.A. Systems thinking. Doctoral Disserttion, Cornel University, New York, May 2006. [Google Scholar]
United Nations Educational, Scientific and Cultural Organization. Education for Sustainable Development Goals, Learning Objectives; UNESCO: France, 2017. [Google Scholar]
OECD and International Institute for Applied Systems Analysis. Systemic Thinking for Policy Making: The Potential of Systems Analysis for Addressing Global Policy Challenges in the 21st Century; Hynes, W., Lees, M., Müller, J.M., Eds.; OECD Publishing: Paris, France, 2020. [Google Scholar]
Soderquist, C.; Overakker, S. Education for sustainable development: A systems thinking approach. Global Environ. Res. 2010, 14, 193–202. [Google Scholar]
Asadova, D.; Soliyeva, S. A Modern Approach to Preschool Education. Role Sci. Innov. Mod. World 2024, 3, 48–57. https://academicsresearch.ru/index.php/trsimw/article/view/2901.
Mertala, P. Teachers’ beliefs about technology integration in early childhood education: A meta-ethnographical synthesis of qualitative research. Comput. Hum. Behav. 2019, 101, 334–349. [Google Scholar] [CrossRef]
Birkollu, S.S.; Yucesoy, Y.; Baglama, B.; Kanbul, S. Investigating the Attitudes of Pre-service Teachers Towards Technology Based on Various Variables. TEM Journal 2017, 6, 578–583. [Google Scholar] [CrossRef]
Merjovaara, O.; Eklund, K.; Nousiainen, T.; Karjalainen, S.; Koivula, M.; Mykkänen, A.; Hämäläinen, R. Early childhood pre-service teachers’ attitudes towards digital technologies and their relation to digital competence. Educ Inf Technol 2024. [Google Scholar] [CrossRef]
Teichert, L. To Digital or not to Digital: How Mothers are Navigating the Digital World with their Young Children. Lang. Liter. 2017, 19, 63–76. [Google Scholar] [CrossRef]
Gillen, J.; Kucirkova, N. Percolating spaces: Creative ways of using digital technologies to connect young children’s school and home lives. Br J Educ Technol 2018, 49, 834–846. [Google Scholar] [CrossRef]
Bredekamp, S. Effective practices in early childhood education: Building a foundation, 4th ed.; Pearson Education: London, England, 2019. [Google Scholar]
Nakamura, J.; Csikszentmihalyi, M. The Expeience of Flow Theory and Research. In The Oxfords Handbook of Positive Psychology, 3rd ed.; Snyder, C., et al., Eds.; 2021; pp. 279–296. [Google Scholar] [CrossRef]
Mohebi, L. Theoretical Models of Integration of Interactive Learning Technologies into Teaching: A Systematic Literature Review. Int. J. Learn. Teach. Educ. Res. 2021, 20. [Google Scholar] [CrossRef]
Blundell, C.N.; Mukherjee, M.; Nykvist, S. A scoping review of the application of the SAMR model in research. Comput. Educ. Open 2022, 3. [Google Scholar] [CrossRef]
Brown, J.P. Teachers’ perspectives of changes in their practice during a technology in mathematics education research project. Teach Teach Educ. 2017, 64, 52–65. [Google Scholar] [CrossRef]
Tezci, E. Factors that influence pre-service teachers’ ICT usage in education. Eur. J. Teach. Educ. 2011, 34, 483–499. [Google Scholar] [CrossRef]
Alelaimat, A.M.; Ihmeideh, F.M.; Alkhawaldeh, M.F. Preparing Preservice Teachers for Technology and Digital Media Integration: Implications for Early Childhood Teacher Education Programs. Int. J. Early Child. 2020, 52, 299–317. [Google Scholar] [CrossRef] [PubMed]
Yang, T.; Hong, X. Early childhood teachers’ professional learning about ICT implementation in kindergarten curriculum: A qualitative exploratory study in China. Front. Psychol. 2022, 13. [Google Scholar] [CrossRef]
Louka, K.; Papadakis, S. Enhancing computational thinking in early childhood education through ScratchJr integration. Heliyon 2024, 10. [Google Scholar] [CrossRef] [PubMed]
United Nations. Transforming our world: the 2030 Agenda for Sustainable Development. Available online: https://sdgs.un.org/2030agenda (accessed on 25 July 2024).
PISA 2022 REULTS (Volume 1). OECD, 5 December 2023. Available online: https://www.oecd.org/en/publications/2023/12/pisa-2022-results-volume-i_76772a36.html (accessed on 26 July 2024).
Ministry of Education. Znani rezultati mednarodne raziskave bralne, matematične in naravoslovne pismenosti PISA 2022. Available online: https://www.gov.si/novice/2023-12-05-znani-rezultati-mednarodne-raziskave-bralne-matematicne-in-naravoslovne-pismenosti-pisa-2022/ (accessed on 28 July 2024).
University of Ljubljana. 1. posvet na projektu “IKT v pedagoških študijskih programih UL, 2017. Available online: http://nio9.uni-lj.si/PrviPosvet.html (accessed on 28 July 2024).
University of Ljubljana. Programi in projekti. Available online: https://www.uni-lj.si/raziskovanje/programi-in-projekti (accessed on 20 July 2024).
Next-GenerationEU. Make it Real. Available online: https://next-generation-eu.europa.eu/index_en (accessed on 23 July 2024).
University of Ljubljana, Faculty of Education. Vključevanje didaktične uporabe IKT v izbrane študijske predmete na VSŠP predšolska vzgoja. Available online: https://www.pef.uni-lj.si/vkljucevanje-didakticne-uporabe-ikt-pv/ (accessed on 23 July 2024).
Univrsity of Ljubljana, Faculty of Education. Projects. Available online: https://www.pef.uni-lj.si/raziskovanje-in-umetnost/projekti/ (accessed on 1 August 2024).
Elsawah, S.; Ho, A.; Ryan, M. Teaching Systems Thinking in Higher Education. INFORMS Trans. Educ. 2021, 22, 66–102. [Google Scholar] [CrossRef]
Gonzales, M. Systems Thinking for Supporting Students with Special Needs and Disabilities; Springer Nature, 2020. [Google Scholar] [CrossRef]
Grohs, J.R.; Kirk, G.R.; Soledad, M.M.; Knight, D.B. Assessing systems thinking: A tool to measure complex reasoning through ill-structured problems. Think Skills Creat 2018, 28, 110–130. [Google Scholar] [CrossRef]
Feriver, S.; Olgan, R.; Teksöz, G.; Barth, M. Impact of early childhood education settings on the systems thinking skills of preschool children through the lens of Brofenbrenner’s theory. Syst Res Behav Sci 2022, 39, 85–103. [Google Scholar] [CrossRef]
Senge, P. The Fifth Discipline, the Art and Practice of the Learning Organization. Doubleday/Currency: New York, 1990. [Google Scholar]
Arnold, R.D.; Wade, J.P. A Definition of Systems Thinking: A Systems Approach. Procedia Comput. Sci 2015, 44, 669–678. [Google Scholar] [CrossRef]
Moore, S.M.; Dolansky, M.A.; Singh, M.; Palmieri, P.; Alemi, F. The Systems Thinking Scale. Unpublished manuscript, 2010. Available online: https://case.edu/nursing/sites/case.edu.nursing/files/2018-04/STS_Manual.pdf.
Arnold, R.; Wade, J. A complete Set of Systems Thinking Skills. In Proceedings of the 27th Annual INCOSE International Symposium, Adelaide; 2017. [Google Scholar]
Engström, S.; Norström, P.; Söderberg, H. A Model for Teachnig Systems Thinking: A Tool for Analysing Technology Teachers’ Conceptualising of Systems Thinking, and How it is Described in Technology Textbooks for Compulsory School. Forsk. slöjdpedagog. slöjdvetensk. 2021, 28, 241–251. Available online: https://journals.oslomet.no/index.php/techneA/article/view/4320.
Ben-Zvi-Assaraf, O.; Orion, N. Four Case Studies, Six Years Later: Developing System Thinking, Skills in Junior High School and Sustaining Them over Time. J. Res. Sci. Teach. 2010, 47, 1253–1280. [Google Scholar] [CrossRef]
Monat, J.; Gannon, T.; Amissah, M. The Case for Systems Thinking in Undergraduate Engineering Education. Int. J. Eng. Pedagogy 2022, 12, 50–88. [Google Scholar] [CrossRef]
Green, C.; Molloy, O.; Duggan, J. An Empirical Study of the Impact of Systems Thinking and Simulation on Sustainability Education. Sustainability 2022, 14, 394. [Google Scholar] [CrossRef]
Keating, C.; Gheorghe, A. Systems Thinking: Foundation for Enhancing System of Systems Engineering. In Proceedings of the 11th System of Systems Engineering Conference, 12–16 June 2016. [Google Scholar] [CrossRef]
Kordova, S.; Frank, M.; Miller, A. Systems Thinking Education—Seeing the Forest through the Trees. Systems 2018, 6, 29. [Google Scholar] [CrossRef]
Monat, J.; Gannon, T. What is Systems Thinking? A Review of Selected Literature Plus Recommendations. Am. J. Syst. Sci. 2015, 4, 11–26. [Google Scholar] [CrossRef]
Feriver, Ş.; Olgan, R.; Teksöz, G.; Barth, M. Systems Thinking Skills of Preschool Children in Early Childhood Education Contexts from Turkey and Germany. Sustainability 2019, 11, 1478. [Google Scholar] [CrossRef]
Frank, M. A systems approach for developing technological literacy. J. Technol. Educ. 2005, 17, 19–34. [Google Scholar] [CrossRef]
Kali, Y.; Orion, N.; Eylon, B. Effect of knowledge integration activities on students’ perception of the earth’s crust as a cyclic system. J. Res. Sci. Teach. 2003, 40, 545–565. [Google Scholar] [CrossRef]
Anderson, L.; Krathwohl, D. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives; Allyn & Bacon: Boston, 2001. [Google Scholar]
Aisyah, S.; Tatminingsih, S.; Chandrawati, T.; Novita, D. Stimulating Strategy Higher Order Thinking Skillsin Early Childhood Education by Utilizing Traditional Games. J. Pendidik. Usia Dini 2024, 18. [Google Scholar] [CrossRef]
Liu, J.; Liu, Z.; Wang, C.; Xu, Y.; Chen, J.; Cheng, Y. K-12 students’ higher-order thinking skills: Conceptualization, components, and evaluation indicators. Think. Ski. Creat. 2024, 52. [Google Scholar] [CrossRef]
Battle for Kids. Framework for 21st Century Learning; Battle for Kids: United States, 2019; https://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf.
Hsu, C.-Y.; Wu, T.-T. Application of Business Simulation Games in Flipped Classrooms to Facilitate Student Engagement and Higher-Order Thinking Skills for Sustainable Learning Practices. Sustainability 2023, 15, 16867. [Google Scholar] [CrossRef]
Lu, K.; Yang, H.; Shi, Y.; Wang, X. Examining the key influencing factors on college students’ higher order thinking skills in the smart classroom environment. Int. J. Educ. Technol. High. Educ. 2021, 18. [Google Scholar] [CrossRef]
Singh, C.; Singh, R.; Singh, T.; Mostafa, N.; Mhtar, T. Developing a Higher Order Thinking Skills Module for Weak ESL Learners. Engl. Lang. Teach. 2018, 11. [Google Scholar] [CrossRef]
Alammary, A.S. Optimizing Components Selection in Blended Learning: Toward Sustainable Students Engagement and Success. Sustainability 2024, 16, 4923. [Google Scholar] [CrossRef]
Ministry of Education, Science and Sport of the Republic of Slovenia. Kurikulum za vrtce; Ministry of Education, Science and Sport of the Republic of Slovenia: Ljubljana, Slovenia, 1999; Available online: https://www.gov.si/assets/ministrstva/MIZS/Dokumenti/Sektor-za-predsolsko-vzgojo/Programi/Kurikulum-za-vrtce.pdf (accessed on 25 July 2024).
Piaget, J. Physical world of the child. Phys. Today 1972, 25, 23–27. [Google Scholar] [CrossRef]
Vygotsky, L. Interaction between learning and development. In Readings on the development of children, 3rd ed.; Gauvain, M., Cole, M., Eds.; Worth, 2001; pp. 22–29, (Original work published 1978). [Google Scholar]
Slunjski, E. Fostering the Developmentof Metacognitive Capacitiesof Preschool Children. Croat. J. Educ. 2022, 25, 977–1000. [Google Scholar] [CrossRef]
Kumar, S.; Mohamed, S. Preschool Teachers’ Knowledge, Understanding and Practice towards Higher Order Thinking Skills. Int. J. Acad. Res. Progress. Educ. Dev 2022, 11, 128–139. [Google Scholar] [CrossRef]
Suffian, S.; Nachiappan, S. Analysis of Teacher Readiness towards Higher Order Thinking Skills (HOTS) Integration in Preschool Teaching and Learning (TNL). Int. J. Acad. Res. Bus. Soc. Sci 2019, 9, 417–423. [Google Scholar] [CrossRef]
Whalen, S.P. Flow and the Engagement of Talent: Implications for Secondary Schooling. National Association of Secondary School Principals 1998, 82, 22–37. [Google Scholar] [CrossRef]
Peifer, C.; Wolters, G.; Harmat, L.; Heutte, J.; Tan, J.; Freire, T.; Tavares, D.; Fonte, C.; Andersen, F.O.; van den Hout, J.; et al. A Scoping Review of Flow Research. Front. Psychol. 2022, 13. [Google Scholar] [CrossRef]
Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience; Harper & Row, 1991. [Google Scholar]
van den Hout, J.J.; Davis, O.C. Flow Team, The psychology of optimal collaboration; Springer Cham, 2019. [Google Scholar] [CrossRef]
Pilke, E.M. Flow experiences in information technology use. Int. J. Hum. Comput. Stud 2004, 61, 347–357. [Google Scholar] [CrossRef]
Palomäki, J.; Tammi, T.; Lehtonen, N.; Seittenranta, N.; Laakasuo, M.; Abuhamdeh, S.; Lappi, O.; Cowley, B.U. The link between flow and performance is moderated by task experience. Comput 2021, 124. [Google Scholar] [CrossRef]
Perttula, A.; Kiili, K.; Lindstedt, A.; Tuomi, P. Flow experience in game based learning—A systematic literature review. Int. J. Serious Games 2017, 4. [Google Scholar] [CrossRef]
Yang, C.-M.; Hsu, T.-F. Integrating Design Thinking into a Packaging DesignCourse to Improve Students’ Creative Self-Efficacy and Flow Experience. Sustainability 2020, 12, 5929. [Google Scholar] [CrossRef]
Greene, M.T.; Gonzales, R.; Papalambros, P.; Mcgowan, A.-M. Design Thinking vs. Systems Thinking for Engineering Design: What’s the Difference? In Proceedings of the 21st International Conference on Engineering Design, Vancouver, Canada, August 2017. [Google Scholar]
Primus, D.J.; Sonnenburg, S. Flow Experience in Design Thinking and Practical Synergies with Lego Serious Play. Creat Res J 2018, 30, 104–112. [Google Scholar] [CrossRef]
Rutrecht, H.; Wittmann, M.; Khoshnoud, S.; Alvarez Igarzábal, F. Time Speeds Up During Flow States: A Study in Virtual Reality with the Video Game Thumper. Timing Time Percept. 2021, 9, 353–376. [Google Scholar] [CrossRef]
Mikicin, M. Autotelic Immersion - Interdependence of the Experience and Autotelic Engagement. Res. Sq 2021. [Google Scholar] [CrossRef]
Rodríguez-Ardura, I.; Meseguer-Artola, A. Flow in e-learning: What drives it and why it matters. Br J Educ Technol 2016, 48, 899–915. [Google Scholar] [CrossRef]
Zaman, M.; Anandarajan, M.; Dai, Q. Experiencing flow with instant messaging and its facilitating role on creative behaviors. Comput 2010, 26, 1009–1018. [Google Scholar] [CrossRef]
Burner, T.; Svendsen, B. Activity Theory—Lev Vygotsky, Aleksei Leont’ev, Yrjö Engeström. In Science Education in Theory and Practice; Springer Nature: Switzerland, 2021; pp. 311–322. [Google Scholar] [CrossRef]
Colasante, M. Five methodological dilemmas when implementing an activity theory transformative intervention in higher education. Teach. High. Educ. 2024, 1–21. [Google Scholar] [CrossRef]
Engeström, Y. Expansive Learning at Work: toward anactivity theoretical reconceptualization. J. Educ. Work 2001, 14, 133–156. [Google Scholar] [CrossRef]
Campbell, D.T. Social Attitudes and Other Acquired Behavioral Dispositions. In Psychology: A study of a science. Study II. Empirical substructure and relations with other sciences; Koch, S., Ed.; McGraw-Hill, 1963; Volume 6, pp. 94–172. [Google Scholar] [CrossRef]
University of Ljubljana, Faculty of Education. Technology Education Curriculum for Preschool Education Students. 2022; (accessed on 6 August 2024). [Google Scholar]
University of Ljubljana, Faculty of Education. Preschool Education. Available online: https://www.pef.uni-lj.si/studij/studijski-programi-prve-stopnje/predsolska-vzgoja/ (accessed on 26 July 2024).
Faul, F.; Erdfelder, E.; Buchner, A.; Lang, A.G. Statistical power analyses using G^*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods 2009, 41, 1149–1160. [Google Scholar] [CrossRef]
Avsec, S.; Jagiełło-Kowalczyk, M.; Żabicka, A.; Gawlak, A.; Gil-Mastalerczyk, J. Leveraging Systems Thinking, Engagement, and Digital Competencies to Enhance First-Year Architecture Students’ Achievement in Design-Based Learning. Sustainability 2023, 15, 15115. [Google Scholar] [CrossRef]
Velicer, W.F. Determining the number of components from the matrix of partial correlations. Psychometrika 1976, 41, 321–327. [Google Scholar] [CrossRef]
Wijnen, F.; van der Molen, J.W.; Voogt, J. Measuring primary school teachers’ attitudes towards stimulating higher-order thinking (SHOT) in students: Development and validation of the SHOT questionnaire. Think Skills Creat 2021, 42, 42. [Google Scholar] [CrossRef]
Naibert, N.; Barbera, J. Investigating Student Enagagement in General Chemistry Active Learning Activities using the Activity Engagement Survey (AcES). J Chem Educ 2022. [Google Scholar] [CrossRef]
Diessner, R.; Solom, R.C.; Frost, N.K.; Parsons, L.; Davidson, J. Engagement With Beauty: Appreciating Natural, Artistic, and Moral Beauty. J. Psychol 2008, 142, 303–329. [Google Scholar] [CrossRef]
Jackson, S.A.; Herbert, M. Development and validation of a scale to measure optimal experience: The Flow State Scale. J Sport Exerc Psychol 1996, 18, 17–35. [Google Scholar] [CrossRef]
Šimleša, M.; Guegan, J.; Blanchard, E.; Tarpin-Bernard, F.; Buisine, S. The Flow Engine Framework: A Cognitive Model of Optimal Human Experience. Eur. Psychol 2018, 14, 232–253. [Google Scholar] [CrossRef]
Avsec, S.; Jamšek, J. A path model of factors affecting secondary school students’ technological literacy. Int. J. Technol. Des. Educ 2018, 28, 145–168. [Google Scholar] [CrossRef]
Dunn, T.J.; Baguley, T.; Brunsden, V. From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. Br. J. Psychol. 2014, 105, 399–412. [Google Scholar] [CrossRef] [PubMed]
Cohen, J. Statistical power analysis for the behavioral sciences, 2nd ed.; Routledge: New York, NY, USA, 2013. [Google Scholar]
Lachowicz, M.J.; Preacher, K.J.; Kelley, K. A novel measure of effect size for mediation analysis. Psychol. Methods 2018, 23, 244–261. [Google Scholar] [CrossRef]
Gaskin, J.; Ogbeibu, S.; Lowry, P.B. Demystifying Prediction in Mediation Research and the Use of Specific Indirect Effects and Indirect Effect Sizes. In Partial Least Squares Path Modeling: Basic Concepts, Methodological Issues, and Applications, 2nd ed.; Latan, H., Hair, J.F., Noonan, R., Eds.; Springer: Cham, Switzerland, 2023. [Google Scholar] [CrossRef]
Hair, J.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate data analysis, 7th ed.; Pearson Educational International: Upper Saddle River, New Jersey, 2010. [Google Scholar]
Hair, J.; Hollingsworth, C.L.; Randolph, A.B.; Chong, A.Y.L. An updated and expanded assessment of PLS-SEM in information systems research. Ind. Manag. Data Syst. 2017, 117, 442–458. [Google Scholar] [CrossRef]
Fabrigar, L.R.; Wegener, D.T.; MacCallum, R.C.; Strahan, E.J. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods 1999, 4, 272–299. [Google Scholar] [CrossRef]
Watkins, M.W. Exploratory Factor Analysis: A Guide to Best Practice. J Black Psychol 2018, 44, 219–246. [Google Scholar] [CrossRef]
Hair, J.F.; Sarstedt, M.; Pieper, T.M.; Ringle, C.M. The Use of Partial Least Squares Structural Equation Modeling in Strategic Management Research: A Review of Past Practices and Recommendations for Future Applications. Long Range Plan 2012, 45, 320–340. [Google Scholar] [CrossRef]
Henseler, J.A. Adanco 2.0.1: User Manual, 1st ed.; Composite Modeling GmbH & Co: Kleve, Germany, 2017. [Google Scholar]
Cheung, G.W.; Cooper-Thomas, H.D.; Lau, R.S.; Wang, L.C. Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations. Asia Pac J Manag 2023. [Google Scholar] [CrossRef]
Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multivariate Data Analysis, 8th ed.; Cengage: Hampshire, UK, 2019. [Google Scholar]
Carlson, K.D.; Herdman, A.O. Understanding the Impact of Convergent Validity on Research Results. Organ Res Methods 2012, 15, 17–32. [Google Scholar] [CrossRef]
Rönkkö, M.; Cho, E. An updated guideline for assessing discriminant validity. Organ Res Methods 2022, 25, 6–14. [Google Scholar] [CrossRef]
Henseler, J.; Ringle, C.M.; Sarstedt, M. A new criterion for assessing discriminant validity in variance-based structural equation modeling. J. Acad. Mark. Sci. 2015, 43, 115–135. [Google Scholar] [CrossRef]
Fornell, C.G.; Larcker, D.F. Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar] [CrossRef]
Carter, C.R. Using confirmatory factor analysis to manage discriminant validity issues in social pharmacy research. Int. J. Clin. Pharm 2016, 38, 731–737. [Google Scholar] [CrossRef]
Kurent, B.; Avsec, S. Interdisciplinary systems thinking and the ICT self-concept in higher education. World Trans. Eng. Technol. Educ. 2024, 22, 70–76. Available online: http://www.wiete.com.au/journals/WTE&TE/Pages/Vol.%2022,%20No.2%20(2024)/02-Avsec-S.pdf.
Goretzko, D.; Pham, T.T.H.; Bühner, M. Exploratory factor analysis: Current use, methodologi-cal developments and recommendations for good practice. Curr Psychol. 2019, 40. [Google Scholar] [CrossRef]
Yilmaz-Na, E.; Sönmez, E. Unfolding the potential of computer-assisted argument mapping practices for promoting self-regulation of learning and problem-solving skills of pre-service teachers and their relationship. Comput Educ 2023, 193. [Google Scholar] [CrossRef]
Shaffer, J.A.; DeGreest, D.; Li, A. Tackling the problem of construct proliferation: A guide to assessing the discriminant validity of conceptually related constructs. Organ Res Methods 2016, 19, 80–110. [Google Scholar] [CrossRef]
Roemer, E.; Schuberth, F.; Henseler, J. HTMT2–an improved criterion for assessing discriminant validity in structural equation modeling. Ind. Manag. Data Syst. 2021, 121, 2637–2650. [Google Scholar] [CrossRef]
Hartley, K.; Bendixen, L.D.; Gianoutsos, D.; Shreve, E. The smartphone in self-regulated learning and student success: clarifying relationships and testing an intervention. Int J Educ Technol High Educ 2020, 17. [Google Scholar] [CrossRef]
Nitzl, C.; Roldan, J.L.; Cepeda, G. Mediation analyses in partial least squares structural equation modeling: Helping researchers discuss more sophisticated models. Ind. Manag. Data Syst. 2016, 116, 1849–1864. [Google Scholar] [CrossRef]
Pérez-Suay, A.; García-Bayona, I.; Van Vaerenbergh, S.; Pascual-Venteo, A.B. Assessing a Didactic Sequence for Computational Thinking Development in Early Education Using Educational Robots. Educ. Sci. 2023, 13, 669. [Google Scholar] [CrossRef]
Bers, M.; Strawhacker, A.; Sullivan, A. The state of the field of computational thinking in early childhood education; OECD Publishing: Paris, France, 2022. [Google Scholar] [CrossRef]
Kurent, B.; Avsec, S. Systems Thinking Skills and the ICT Self-Concept in Preschool Teachers for Sustainable Curriculum Transformation. Sustainability 2023, 15, 15131. [Google Scholar] [CrossRef]
Carless, D.; Boud, D. The development of student feedback literacy: enabling uptake of feedback. Assess Eval High Educ 2018, 43, 1315–1325. [Google Scholar] [CrossRef]
de Kleijn, R.A. Supporting student and teacher feedback literacy: an instructional model for student feedback processes. Assess. Eval. High. Educ. 2023, 48, 186–200. [Google Scholar] [CrossRef]
Gray, J.P.; Mannahan, K.K. How Well do Trait Measures of Achievement Predict Students’ Perceptions of the Link between Personal Effort and Academic Performance? J. Eff. Teach. 2017, 17, 16–27. [Google Scholar]
Aljabreen, H. Montessori, Waldorf, and Reggio Emilia: A Comparative Analysis of Alternative Models of Early Childhood Education. Int J Early Child 2020, 52, 337–353. [Google Scholar] [CrossRef]
Frausel, R.R.; Silvey, C.; Freeman, C.; Dowling, N.; Richland, L.E.; Levine, S.C.; Raudenbush, S.; Goldin-Meadow, S. The origins of higher-order thinking lie in children’s spontaneous talk across the pre-school years. Cognition 2020, 200. [Google Scholar] [CrossRef]
Afifah, I.; Retnawati, H. Is it difficult to teach higher order thinking skills? J. Phys. Conf. Ser 2019, 1320. [Google Scholar] [CrossRef]
Brauner, P.; Leonhardt, T.; Ziefle, M.; Schroeder, U. The Effect of Tangible Artifacts, Gender and Subjective Technical Competence on Teaching Programming to Seventh Graders. In Teaching Fundamentals Concepts of Informatics; Hromkovič, J., Královič, R., Vahrenhold, J., Eds.; Springer: Berlin, Heidelberg, 2010. [Google Scholar] [CrossRef]
Deng, R. Emotionally Engaged Learners Are More Satisfied with Online Courses. Sustainability 2021, 13, 11169. [Google Scholar] [CrossRef]
Leggett, N. Creativity in early childhood: how educators from Australia and Italy are documenting the creative thought processes of young children. SN SS 2024, 4. [Google Scholar] [CrossRef]
Lamb, C.T.; Rhodes, D.H. Collaborative Systems Thinking Research: Exploring systems thinking within teams. 2008. Available online: https://dspace.mit.edu/handle/1721.1/84547. [CrossRef]
White, L. Aesthetics in OR/Systems Practice:Towards a Concept of Critical Imaginationas a Challenge to Systems Thinking. Syst Res Behav Sci 2006, 23, 713–847. [Google Scholar] [CrossRef]
Wati, Y.; Koh, C.; Davis, F. Can You Increase Your Performance in a Technology-Driven Society Full of Distractions? In Proceedings of the ICIS 2014, Auckland, New Zeland, December 2014; https://aisel.aisnet.org/icis2014/proceedings/HCI/6.
Kartal, B.; Taşdemir, A. Pre-service teachers’ attitudes towards STEM: Differences based on multiple variables and the relationship with academic achievement. Int. J. Technol. Educ. 2021, 4, 200–228. [Google Scholar] [CrossRef]

Figure 1. Example of a summary of kindergarten activity at the University of Ljubljana.

Figure 2. Examples of learning in DTE by doing activities at the University of Ljubljana: (a) moving octopus made of a plastic bottle, (b) rocket made from paper cups and wastepaper, and (c) mill made from ice cream sticks, cotton buds, cardboard, and wood.

Figure 3. Mediation model of systems thinking in relation between flow and final exam score. *** p < 0.001; ns, nonsignificant.

Table 1. DTE course design.

	October					November				December				January
Week number	1	2	3	4	5	6	7	8	9	10	11	12	Holidays		13	14
Lectures
Laboratory work
Preparation for KG activities
KG activities

Table 2. Reliability of McDonald’s ω, composite reliability (CR), square root of average variance extracted (AVE) (in bold), and correlations among systems thinking constructs (diagonal).

Latent construct	ω	CR	AVE	ST 1	ST 2	ST 3
ST 1	0.89	0.88	0.54	0.73
ST 2	0.83	0.82	0.56	0.71	0.75
ST 3	0.82	0.81	0.55	0.72	0.64	0.74

Table 3. Reliability of McDonald’s ω, composite reliability (CR), square root of average variance extracted (AVE) (in bold), and correlations among strategic approach orientation constructs (diagonal).

Latent construct	ω	CR	AVE	Personal effort	Reliance on authority	Strategic thinking
Personal effort	0.72	0.71	0.51	0.71
Reliance on authority	0.80	0.79	0.62	0.35	0.75
Strategic thinking	0.76	0.77	0.51	0.25	0.47	0.79

Table 4. Heterotrait–monotrait ratio of correlations (HTMT) and Fornell–Larcker criterion results (in parenthesis) for systems thinking scale. AVE on the diagonal.

Latent construct	ST 1	ST 2	ST 3
ST 1	0.54
ST 2	0.71 (0.51)	0.56
ST 3	0.72 (0.52)	0.64 (0.40)	0.55

Table 5. Heterotrait–monotrait ratio of correlations (HTMT) and Fornell–Larcker criterion results (in parenthesis) for strategic approach orientation scale. AVE on the diagonal.

Latent construct	Personal effort	Reliance on authority	Strategic thinking
Personal effort	0.51
Reliance on authority	0.31 (0.13)	0.62
Strategic thinking	0.24 (0.11)	0.46 (0.24)	0.51

Table 6. Average scores for pre-service teachers’ self-reported data expressed as mean (M) and standard deviation (SD) across subscales of systems thinking and strategic approach along with a measure of skewness (S) and kurtosis (K); 95 % confidence interval (CI) in brackets.

	Variables	M	SD	S	K	95 % CI
Systems thinking	ST 1	4.64	0.74	-0.44	-0.43	[4.46,4.83]
	ST 2	4.85	0.68	-0.26	-0.06	[4.68,5.03]
	ST 3	5.11	0.67	-0.62	-0.02	[4.94,5.27]
	Total ST	72.40	9.23	-0.35	-0.43	[70.11,73.00]
Strategic approach orientation	Personal effort	4.21	0.76	-0.59	0.42	[4.02,4.40]
	Reliance on authority	2.94	0.77	0.06	0.34	[2.75,3.00]
	Strategic thinking	3.86	0.65	0.17	-0.15	[3.69,4.02]

Table 7. Reliability of McDonald’s ω, composite reliability (CR), square root of average variance extracted (AVE) (in bold), and correlations among SHOT’s subscales (diagonal).

Latent construct	ω	CR	AVE	PR	PSA	SE	CD	ENJ	ANX	PD
PR	0.85	0.85	0.59	0.77
PSA	0.90	0.89	0.60	0.22	0.78
SE	0.93	0.83	0.77	0.35	0.14	0.88
CD	0.81	0.80	0.52	0.24	0.70	0.33	0.72
ENJ	0.94	0.95	0.82	0.36	0.02	0.54	0.13	0.90
ANX	0.93	0.93	0.74	0.07	0.51	0.05	0.69	0.11	0.86
PD	0.81	0.80	0.58	0.11	0.29	0.34	0.68	0.31	0.70	0.76

Table 8. Heterotrait–monotrait ratio of correlations (HTMT) and Fornell–Larcker criterion results of ASV (in parenthesis) for SHOT’s subscales. AVE on the diagonal.

Latent construct	PR	PSA	SE	CD	ENJ	ANX	PD
PR	0.59
PSA	0.22 (0.05)	0.60
SE	0.37 (0.13)	0.14 (0.02)	0.77
CD	0.24 (0.07)	0.70 (0.49)	0.33 (0.11)	0.52
ENJ	0.36 (0.14)	0.02 (0.01)	0.54 (0.30)	0.13 (0.02)	0.82
ANX	0.07 (0.01)	0.52 (0.26)	0.05 (0.01)	0.69 (0.48)	0.11 (0.02)	0.74
PD	0.11 (0.01)	0.29 (0.09)	0.34 (0.11)	0.68 (0.47)	0.31 (0.10)	0.70 (0.49)	0.58

Table 9. Average values of pre-service teachers’ self-reported scores expressed as mean (M) and standard deviation (SD) across SHOT subscales along with measures of skewness (S) and kurtosis (K); 95 % confidence interval (CI) in brackets.

	Subscales	M	SD	S	K	95 % CI
SHOT	PR	5.60	0.81	-0.54	1.13	[5.40, 5.80]
	PSA	3.78	1.12	-0.01	0.06	[3.51, 4.06]
	SE	4.59	1.07	0.11	0.08	[4.33, 4.86]
	CD	4.31	0.92	0.26	1.51	[4.08, 4,54]
	ENJ	5.58	1.01	-0.04	-1.11	[5.32, 5.83]
	ANX	3.12	1.28	0.91	0.95	[2.80, 3.44]
	PD	4.31	1.06	0.16	0.21	[4.05, 4.58]

Table 10. Unstandardized and standardized coefficients with p-values in MLR analysis; 95 % confidence interval (CI) in brackets.

Model	Unstandardized Coefficients		Standardized Coefficients	t-statistic	p-value	95 % CI
	β	Std. Error	β
PR	0.88	1.28	0.08	0.68	0.497	[-1.69, 3.46]
PSA	2.42	1.06	0.30	2.29	0.026	[0.30, 4.56]
SE	3.81	1.11	0.44	3.44	0.001	[1.58, 6.02]
CD	-0.56	1.55	-0.06	-0.36	0.724	[-3.65, 2.56]
ENJ	-0.71	1.14	-0.08	-0.62	0.538	[-2.99, 1.57]
ANX	-2.97	1.12	-0.41	-2.63	0.011	[-5.22, -0.71]
PD	-0.21	1.33	-0.03	-0.15	0.876	[-2.88, 2.47]

Table 12. Heterotrait–monotrait ratio of correlations (HTMT) and Fornell–Larcker criterion results of ASV (in parenthesis) for SHOT subscales. AVE on the diagonal.

Latent construct	BE	CE	EE	SE	AE
BE	0.58
CE	0.61(0.33)	0.51
EE	0.65(0.41)	0.44(0.19)	0.55
SE	0.21(0.04)	0.04(0.01)	0.08(0.01)	0.50
AE	0.16(0.02)	0.56(0.31)	0.22(0.05)	0.34(0.11)	0.58

Table 14. Unstandardized and standardized coefficients with p-values in MLR analysis; 95 % confidence interval (CI) in brackets.

Model	Unstandardized Coefficients		Standardized Coefficients	t-statistic	p-value	95 % CI
	β	Std. Error	β
BE	0.83	1.84	0.05	0.45	0.654	[-1.69, 3.46]
CE	7.24	1.76	0.51	4.12	0.000	[0.30, 4.56]
EE	1.71	1.85	0.11	0.92	0.359	[1.58, 6.02]
SE	3.65	1.35	0.29	2.69	0.009	[-5.22, -0.71]
AE	-3.03	1.10	-0.31	-2.74	0.008	[-2.88, 2.47]

Table 15. Reliability of McDonald’s ω, composite reliability (CR), square root of average variance extracted (AVE) (in bold), and correlations among FSS subscales (diagonal).

Latent construct	ω	CR	AVE	AAM	AE	CG	CSB	CTH	LSC	PC	TT	UF
AAM	0.83	0.84	0.67	0.81
AE	0.88	0.89	0.74	0.76	0.86
CG	0.90	0.90	0.77	0.82	0.88	0.88
CSB	0.79	0.80	0.61	0.89	0.89	0.84	0.78
CTH	0.84	0.86	0.67	0.73	0.91	0.91	0.86	0.82
LSC	0.77	0.77	0.60	0.61	0.44	0.54	0.54	0.58	0.77
PC	0.88	0.88	0.73	0.84	0.79	0.88	0.86	0.88	0.50	0.85
TT	0.71	0.73	0.54	0.67	0.74	0.55	0.54	0.58	0.62	0.49	0.74
UF	0.85	0.86	0.69	0.89	0.81	0.92	0.93	0.82	0.52	0.93	0.42	0.83

Table 16. HTMT results.

Latent construct	AAM	AE	CG	CSB	CTH	LSC	PC	TT
AE	0.76
CG	0.82	0.88
CSB	0.89	0.89	0.84
CTH	0.73	0.91	0.91	0.86
LSC	0.61	0.44	0.54	0.54	0.58
PC	0.84	0.79	0.88	0.86	0.88	0.50
TT	0.67	0.74	0.55	0.54	0.58	0.62	0.49
UF	0.89	0.81	0.92	0.93	0.82	0.52	0.93	0.42

Table 17. Average values of pre-service teachers’ self-reported scores expressed as mean (M) and standard deviation (SD) across subscales of flow state along with measures of skewness (S) and kurtosis (K); 95 % confidence interval (CI) in brackets.

	Subscales	M	SD	S	K	95 % CI
Pre-service teachers flow state attechnology-enhanced DTE	CSB	5.61	0.89	-0.69	0.75	[5.39, 5.83]
	AAM	4.97	1.01	-0.51	-0.43	[4.72, 5.22]
	CG	5.54	1.07	-1.11	1.16	[5.27, 5.80]
	UF	5.33	0.81	-0.76	0.29	[5.13, 5.53]
	CTH	5.58	1.01	-0.64	0.07	[5.33, 5.83]
	PC	5.46	0.91	-0.89	0.57	[5.24, 5.69]
	LSC	5.26	1.03	-0.37	-0.35	[5.01, 5.52]
	TT	4.68	0.97	-0.47	0.01	[4.44, 4.92]
	AE	5.86	0.90	-0.57	-0.38	[5.63, 6.07]

Table 18. Unstandardized and standardized coefficients with p-values in MLR analysis; 95 % confidence interval (CI) in brackets.

Model	Unstandardized Coefficients		Standardized Coefficient	t-statistic	p-value	95 % CI
	β	Std. Error	β
CSB	0.98	1.54	0.09	0.63	0.527	[-2.11, 4.08]
AAM	3.36	1.35	0.36	2.48	0.016	[0.65, 6.06]
CG	3.59	1.64	0.41	2.18	0.033	[0.29, 6.89]
UF	-2.39	2.22	-0.20	-1.07	0.286	[-6.85, 2.06]
CTH	-2.70	1.28	-0.29	-2.10	0.040	[-5.28, -0.13]
PC	-1.45	1.89	-0.14	-0.76	0.446	[-5.24, 2.13]
LSC	1.51	1.04	0.16	1.44	0.153	[-0.58, 3.60]
TT	-2.09	1.02	-0.21	-2.03	0.047	[-4.14, -0.03]
AE	3.91	1.30	0.38	2.99	0.004	[1.29, 6.52]

Table 19. Means, standard deviations, correlations, and McDonald’s ω (diagonal values in parentheses) for all variables.

Variable	M	SD	Flow	Systems thinking	Final exam score
Flow	193.45	26.17	(0.92)
Systems thinking	72.40	9.23	0.51 **	(0.94)
Final exam score	69.84	16.11	0.38 **	0.54 **	(0.82)

** Correlation is significant at 0.01 level (2-tailed).

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Synergizing Systems Thinking and Technology-Enhanced Learning Using the Flow Theory Framework

Abstract

1. Introduction

1.1. Systems Thinking and HOTS

1.2. Using Flow and Activity Theory to Understand Human Engagement

1.3. The Present Study (Aim, RQs, and Expectations)

2. Materials and Methods

2.1. Course Format

3.1.1. Sustainability in the DTE Course

2.2. Sample

2.3. Measures

2.3.1. Systems Thinking and Strategic Approach Orientation Scales

2.3.2. Attitudes toward Stimulating Higher-Order Thinking

2.3.3. Student Engagement

2.3.4. Flow

2.3.5. Student Performance

2.4. Data Collection and Analysis

2.4.1. Data Collection

2.4.2. Data Analysis

3. Results

3.1. Systems Thinking and Strategic Approach Orientation among Pre-Service Teachers

3.1.1. Normality Tests and Validity Evidence

3.1.2. Levels of Systems Thinking and Strategic Behavior

3.2. Pre-Service Teachers’ Attitudes toward Stimulating Higher-Order Thinking

3.2.1. Normality Tests and Validity Evidence

3.2.2. Pre-Service Teachers’ Attitudes toward Stimulating Higher-Order Thinking and the Relationship to Systems Thinking Ability

3.3. Pre-Service Teachers’ Engagement with Technlogy-Enhanced DTE Course

3.3.1. Normality and Validity Tests

3.3.2. Pre-Service Teachers’ Engagement in Technology-Enhanced DTE Course and the Relationship to Systems Thinking Ability

3.4. Pre-Service Teachers’ Perception of Flow State in DTE Course and the Relationship to Systems Thinking

3.4.1. Normality and Validity Tests

3.4.2. Flow State and Predictive Analysis

3.5. Relationship between Pre-Service Teachers’ Self-Reported Flow and Their Achievement in DTE Course Mediated by Systems Thinking

4. Discussion

4.1. Self-Reported Systems Thinking and Strategic Approach Orientation of Pre-Service Preschool Teachers

4.2. Pre-Service Preschool Teachers’ Attitudes toward Stimulating Higher-Order Thinking Skills in Relation to Systems Thinking

4.3. Pre-Service Preschool Teachers’ Engagement in ICT and DTE Course Enhanced by Digital Tools

4.4. Pre-Service Preschool Teachers’ State of Flow in the Technology-Enhanced DTE Course

4.5. Impact of the State of Flow on Pre-Service Preschool Teachers’ Performance Mediated by Systems Thinking

4.6. Limitations, Implications, and Future Work

5. Conclusions

Supplementary Materials

Author Contributions

Funding

Institutional Review Board Statement

Informed Consent Statement

Data Availability Statement

Acknowledgments

Conflicts of Interest

References

MDPI Initiatives

Important Links

Subscribe