The application of AI technologies in STEM education: a systematic review from 2011 to 2021
International Journal of STEM Education volume 9, Article number: 59 (2022)
The application of artificial intelligence (AI) in STEM education (AI-STEM), as an emerging field, is confronted with a challenge of integrating diverse AI techniques and complex educational elements to meet instructional and learning needs. To gain a comprehensive understanding of AI applications in STEM education, this study conducted a systematic review to examine 63 empirical AI-STEM research from 2011 to 2021, grounded upon a general system theory (GST) framework.
The results examined the major elements in the AI-STEM system as well as the effects of AI in STEM education. Six categories of AI applications were summarized and the results further showed the distribution relationships of the AI categories with other elements (i.e., information, subject, medium, environment) in AI-STEM. Moreover, the review revealed the educational and technological effects of AI in STEM education.
The application of AI technology in STEM education is confronted with the challenge of integrating diverse AI techniques in the complex STEM educational system. Grounded upon a GST framework, this research reviewed the empirical AI-STEM studies from 2011 to 2021 and proposed educational, technological, and theoretical implications to apply AI techniques in STEM education. Overall, the potential of AI technology for enhancing STEM education is fertile ground to be further explored together with studies aimed at investigating the integration of technology and educational system.
Artificial intelligence in education (AIEd) is an emerging interdisciplinary field that applies AI technologies in education to transform and promote the instructional and learning design, process and assessment (Chen et al., 2020; Holmes et al., 2019; Hwang et al., 2020). The application of AI in STEM education (referred to AI-STEM in this paper), as a sub-branch of AIEd, focuses on the design and implementation of AI applications to support STEM education. Automated AI technologies, e.g., intelligence tutoring, automated assessment, data mining and learning analytics, have been used in STEM education to enhance the instruction and learning quality (Chen et al., 2020; Hwang et al., 2020; McLaren et al., 2010). STEM education is a complex system, from a system perspective, consisting of interdependent elements, including subject, information, medium, and environment (Rapoport, 1986; Von Bertalanffy, 1968). The application of AI, as a critical technology element, should take careful consideration of these complex factors, to achieve a high-quality STEM education (Byrne & Callaghan, 2014; Krasovskiy, 2020; Xu & Ouyang, 2022). This systematic review aims to examine the different elements, including AI technology, subject, information, medium, environment in the AI-STEM system to gain a holistic understanding of the application and integration of AI technologies in the STEM education contexts. Specifically, we collected and reviewed empirical AI-STEM research from 2011 to 2021, summarized the AI techniques and applications, the characteristics of other system elements (i.e., information, subject, medium, environment), the distribution of AI in these elements, and the effects of AI in STEM education. Based on the results, this systematic review provided educational and technological implications for the practice and research in the AI-STEM education.
With the development of computer science and computational technologies, automatic, adaptive, and efficient AI technologies have been widely applied in various academic fields. Artificial Intelligence in Education (AIEd), as an interdisciplinary field, emphasizes applying AI to assist instructor’s instructional process, empower student’s learning process, and promote the transformation of educational system (Chen et al., 2020; Holmes et al., 2019; Hwang et al., 2020; Ouyang & Jiao, 2021). First, AIEd has potential to enhance instructional design and pedagogical development in the teaching processes, such as accessing students’ performance automatically (Wang et al., 2011; Zampirolli et al., 2021), monitoring and tracking students’ learning (Berland et al., 2015; Ji & Han, 2019), and predicting at-risk students (Hellings & Haelermans, 2020; Lamb et al., 2021). Second, AIEd is beneficial for improving student-centered learning, such as providing adaptive tutoring (Kose & Arslan, 2017; Myneni et al., 2013), recommending personalized learning resources (Ledesma & García, 2017; Zhang et al., 2020), and diagnosing students’ learning gaps (Liu et al., 2017). Third, AIEd also brings opportunities to transform the educational system by highlighting the essential role of technology (Hwang et al., 2020), enriching the mediums of knowledge delivery (Holstein et al., 2019; Yannier et al., 2020), and changing the instructor–student relationship (Xu & Ouyang, 2022). Overall, different AI technologies (e.g., machine learning, deep learning) have been deployed in the field of education to enhance instructional and learning process.
The development of AIEd also brought transformations to the field of science, technology, engineering and mathematics (STEM) education, as a sub-branch of AIEd named AI-STEM. STEM education aims to improve students’ interdisciplinary knowledge inquiry and application, as well as their higher-order thinking, critical thinking and problem-solving ability (Bybee, 2013; Pimthong & Williams, 2018). The application of AI in STEM education has advantages to provide adaptive and personalized learning environments or resources, and aid instructors to understand students’ learning behavioral patterns, and automatically assess STEM learning performances (Alabdulhadi & Faisal, 2021; Walker et al., 2014). However, STEM education is a complex system, consisting of interdependent elements, including subject (e.g., instructor, student), information, medium, and environment (Rapoport, 1986; Von Bertalanffy, 1968). Achieving a high quality of STEM education requires a careful consideration of the complex social, pedagogical, environmental factors, rather than merely applying AI technologies in education (Krasovskiy, 2020; Xu & Ouyang, 2022). Therefore, a major challenge in AI-STEM is how to appropriately select and apply AI techniques to adapt to the multiple elements (e.g., subject, information, environment) in STEM education with a goal of high-quality instruction and learning (Castañeda & Selwyn, 2018; Selwyn, 2016). To gain a holistic understanding of the integration of AI technologies in the STEM education contexts, it is crucial to systematically review and examine the complex elements in AI-STEM from a system perspective.
During the past decade, the emerging field of AIEd has gained great attention (Chen et al., 2020; Holmes et al., 2019; Hwang et al., 2020; Ouyang et al., 2022). But existing literature review of AIEd has mainly focused on the trends, applications, and effects of AIEd from a technological perspective (Chen et al., 2020; Tang et al., 2021; Zawacki-Richter et al., 2019). Specifically, we located 18 literature review articles of AIEd published from 2011 to 2021 (see Fig. 1). These AIEd reviews focused on different educational levels, fields, and contexts, including higher education (Zawacki-Richter et al., 2019), e-learning (Tang et al., 2021), mathematics education (Hwang & Tu, 2021), language education (Liang et al., 2021), medical education (Khandelwal et al., 2019; Lee et al., 2021), programming education (Le et al., 2013), and special education (Drigas & Ioannidou, 2012). For example, Zawacki-Richter et. al. (2019) reviewed AIEd in the higher education context and four AI technical applications were classified, namely intelligent tutoring systems, adaptive systems and personalization, profiling and prediction, and assessment and evaluation. Liang et. al. (2021) focused on the application of AI in language education and investigated the roles and research foci (e.g., research methods, research sample groups) of AI techniques in language education. Drigas and Ioannidou (2012) explored AIEd in special education and summarized AI applications based on the student’s disorders, including reading, writing and spelling difficulties, dyslexia, autistic spectrum disorder, etc.
Although various reviews were conducted to understand the field of AIEd, few of them focused on STEM education. Among these 18 literature review articles, we only located two works exploring the application of AI in STEM education. Le et. al. (2013) reviewed the AI-supported tutoring approaches in computer programming education and found that AI techniques were mainly applied to support feedback-based programming tutoring during the student’s individual learning. Hwang and Tu (2021) conducted a bibliometric mapping analysis to systematically review the roles of AI in mathematics education. The results clarified the role of AI in mathematics education into three main types, including intelligent tutoring systems, profiling and prediction, and adaptive systems and personalization. Although some review examined AI in computer science and mathematics education, there is a lack of literature review to investigate the application of AI in general STEM education context. More importantly, due to the complexity of AI-STEM, it is essential to systematically review multiple elements in AI-STEM as well as the effects of AI in the STEM education system.
To fill this gap, this systematic review aims to gain a comprehensive understanding of the integration of AI technologies in the STEM education contexts. Specifically, this review examined and summarized the applications and categories of AI element in the AI-STEM system, the characteristics of other system elements in AI-STEM except AI, the distribution of AI in these elements, and the effects of AI in STEM education. Three research questions (RQs) were proposed:
RQ1: What are the categories of the AI element in the AI-STEM system?
RQ2: What are the characteristics of other system elements (i.e., information, subject, medium, environment element) as well as the distribution of AI in these elements?
RQ3: What are the effects of AI in STEM education?
In order to map the state-of-art of the application of AI techniques in STEM education, we conducted a systematic review from 2011 to 2021, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) principles (Moher et al., 2009).
To locate the empirical studies of AI application in STEM education, the following major publisher databases were selected: Web of Science, Science Direct, Scopus, IEEE, EBSCO, ACM, Taylor & Francis, and Wiley (Guan et al., 2020). Filters were used to the empirical research and peer-reviewed articles in the field of education and educational research from January 2011 to December 2021. After the preliminary screening of articles, snowballing was conducted (Wohlin, 2014) to find the articles that were not extracted using the search strings.
Identification of search terms
Based on the specific requirements of bibliographic databases, we proposed the searching strategies. In terms of the research questions, three types of keywords were used as the search terms. First, keywords related to AIEd and specific AI applications were added (i.e., “artificial intelligence” OR “AI” OR “AIED” OR “machine learning” OR “intelligent tutoring system” OR “expert system” OR “recommended system” OR “recommendation system” OR “feedback system” OR “personalized learning” OR “adaptive learning” OR “prediction system” OR “student model” OR “learner model” OR “data mining” OR “learning analytics” OR “prediction model” OR “automated evaluation” OR “automated assessment” OR “robot” OR “virtual agent” OR “algorithm”). Second, keywords related to STEM were added (i.e., “STEM” OR “science” OR “technology” OR “math” OR “physics” OR “chemistry” OR “biology” OR “geography” OR “engineering” OR “programming” OR “lab”). Third, keywords related to education were added (i.e., “education” OR “learning” OR “course” OR “class” OR “teaching”).
The search criteria were designed to locate the articles that focused on the applications of AI in STEM education. According to the research objectives, inclusion and exclusion criteria were adopted (see Table 1).
The screening process
The screening process involved the following procedures: (1) removing the duplicated articles; (2) reading the titles and abstracts and removing the articles according to the inclusion and exclusion criteria; (3) reading the full texts and removing the articles according to the inclusion and exclusion criteria; (4) using the snowballing to further locate the articles in Google Scholar, and (5) extracting data from the final filtered articles (see Fig. 2). All articles were imported into Mendeley software for screening.
3373 articles were located as the result of the first round of searching. Among these records, 777 duplicates were removed and then 1879 records were excluded because they were not classified under Education & Educational research or journal article. By reviewing the titles and abstracts, the number of articles was reduced to 717 based on the criteria (see Table 1). The selected articles were examined by the first author to determine whether they were suitable for the purpose of this systematic review. The second author independently reviewed approximately 30% of the articles to confirm the reliability. The inter-rater agreement was 92%. Then, the full-text of articles were reviewed by the first author to verify that the articles met all the criteria for inclusion in the review. Finally, a total of 63 articles that met the criteria were identified for the systematic review.
Theoretical framework and analysis procedure
General system theory (GST) is a theoretical framework, arguing that the world is composed of different organic systems, which contain dynamically interacting elements and mutual relationships between them (Rapoport, 1986; Von Bertalanffy, 1950). The main principle of GST is that a system is not simply equal to the sum of its elements, but greater than the sum of its parts (Drack & Pouvreau, 2015; Von Bertalanffy, 1968). To deeply understand the complex nature and general rules of systems, GST highlights the system’s holistic principle to identify the internal elements, functional relationships of them, as well as the external influences upon a system (Crawford, 1974). The theoretical framework of GST has been widely applied in various fields to analyze different types of systems, such as physical, biological, social and educational systems (Drack & Pouvreau, 2015; Kitto, 2014). For example, Chen and Stroup (1993) suggested applying GST as an underpinned theoretical framework to guide the reform of science education and highlighted the integration of science curriculum to avoid the compartmentalized learning of physics, biology, and chemistry. Following this philosophy, we argue that GST can provide a new, holistic perspective to understand the integration of AI technologies and STEM education.
From the perspective of GST, AI-STEM can be viewed as an organic system, which mainly contains five basic elements, namely subject, information, medium, environment, and technology (Von Bertalanffy, 1968) (see Fig. 3). First, subject is defined as people in an educational system and different subjects of people (e.g., instructor, student) can take agency to interact with each other constantly and adaptively. Second, information refers to knowledge spread and constructed between subjects in an educational system, such as learning contents, course materials, knowledge artifacts, etc. Third, medium is the way or carrier to convey information and connect subjects in the system. Fourth, environment serves as an underlying context in an educational system, which influences the function of the whole educational system. Fifth, technology (e.g., AI techniques) is usually appeared as an external element to impact the functions of the educational system. Grounded upon GST, the integration of AI, as an external technology element, in an educational system (such as STEM) is a complex process, that has influences on other system elements (i.e., subject, information, medium, environment) and on the relationships between them. In summary, the framework of GST (see Fig. 3) highlights the multiple elements as well as their mutual relationships in AI-STEM system, which provides us a holistic view for applying AI technologies in STEM education.
We used content analysis method (Cohen et al., 2005; Zupic & Čater, 2015) to classify 63 AI-STEM articles in order to answer the research questions. Based on GST, a coding scheme of educational system elements was developed to systematically examine AI-STEM articles (see Table 2). This coding scheme included the subject of instructor (including instructor involvement and instructor strategy), the subject of learner (including educational level, sample size, and learning outcome), information (i.e., learning content), medium (i.e., educational medium), environment (i.e., educational context), and technology (i.e., AI technique).
63 articles were coded by two raters. The same article can be coded more than one code in one dimension. First, 20% of articles were coded by two coders independently in order to calculate coding reliability. Krippendorff’s (2004) alpha reliability was 0.91 among two raters at this phase. The remaining articles were coded independently by two raters after the reliability was ensured. Consensus was reached by two raters on conflicting coding results. We provided details and examples below to demonstrate how the coding results represented the review data (Graneheim & Lundman, 2004).
To answer the research questions, the results section presents the following three main topics: (1) the categories of the AI element in the AI-STEM system; (2) the characteristics of other system elements (i.e., information, subject, medium, and environment) as well as the distribution of AI in these elements, and (3) the effects and findings of the application of AI in STEM education.
RQ1: What are the categories of the AI element in the AI-STEM system?
Figure 4 demonstrates the trends of empirical studies by year. According to the distribution, the number of publication generally increased along the years. In addition, a majority of reviewed articles (N = 42) were published in the last 4 years from 2018 to 2021. Only 9 of 63 reviewed articles were published in the first 4 years from 2011 to 2014.
Regarding the element of AI technology in AI-STEM, six types of AI applications were identified, namely learning prediction (N = 18, percentage = 29%), intelligent tutoring system (N = 16, percentage = 25%), student behavior detection (N = 13, percentage = 21%), automation (N = 8, percentage = 13%), educational robots (N = 6, percentage = 9%), and others (N = 2, percentage = 3%) (see Fig. 5 and Table 3).
The first category of the AI applications in STEM education was learning prediction, illustrating should be something like systems which predict student learning performance or status in advance through AI algorithms and modeling approaches (Agrawal & Mavani, 2015; Lee et al., 2017). 18 of 63 reviewed articles (29%) focused on learning prediction in STEM education (see Fig. 5). Two sub-categories were summarized under learning prediction: learning performance prediction (N = 14) and at-risk student prediction (N = 4) (see Table 3). First, in the sub-category of learning performance prediction, AI algorithms and modeling techniques were employed in STEM education to help instructors adjust the instructional processes by predicting students’ learning performance (Deo et al., 2020; Hellings & Haelermans, 2020). For example, Buenaño-Fernández et. al. (2019) applied educational data mining and machine learning technique (i.e., decision tree) in computer engineering courses to predict students’ final performance based on their historical grades. Zabriskie et. al. (2019) utilized the random forest model and logistic regression model to predict the physics course outcomes. Another sub-category was predicting at-risk students and dropout factors in STEM education to help instructors intervene in student learning (Vyas et al., 2021; Yang et al., 2020). For example, Lacave et. al. (2018) used Bayesian networks techniques to investigate dropout factors of computer science students in higher education. Yang et. al. (2020) utilized random forest classification to create and examine the prediction models of identifying at-risk students in introductory physics classes. In summary, AI algorithms had been used in STEM education to help instructors or researchers predict students’ final academic performances and learning risks.
Intelligent tutoring system
The second category of the AI applications in STEM education was the intelligent tutoring system (ITS), defined as an AI-enabled system that was designed to provide customized instruction or feedback to students and promote personalized, adaptive learning (Chen et al., 2020; Hooshyar et al., 2015; Murray, 2003). Among the 63 reviewed articles, 16 articles (25%) focused on the applications of ITSs in promoting instruction and learning in STEM education (see Fig. 4). Three sub-categories were identified: instructional content delivery (N = 9), recommendation of personalized learning path (N = 4), and resource recommendation (N = 3) (see Table 3). The first sub-category was using ITSs to deliver instructional content in STEM education. For example, Myneni et. al. (2013) introduced an interactive and intelligent learning system in physics education, where a virtual agent delivered physics concepts to students and decision algorithms were utilized to determine the support level of the virtual agent. Hooshyar et. al. (2018) proposed a novel flowchart-based ITS based on Bayesian networks techniques, which imitated a human instructor to conduct one-to-one instruction with students. The second sub-category of ITS was recommendations of personalized learning path based on student’s profile in STEM education. For example, De-Marcos et. al. (2015) combined genetic algorithm and parliamentary optimization algorithm to create personalized courseware sequencing paths in online STEM learning. Saito and Watanobe (2020) proposed an approach of recommending learning paths that applied recurrent neural network and sequential prediction model to create students’ ability charts and learning paths in programming learning based on their submission history. The third sub-category of ITS was recommending learning resource according to student’s needs in STEM education. For example, Ledesma and García (2017) introduced an expert system as a support tool to tackle mathematical topics, by recommending appropriate mathematical problems in accordance with a student’s learning style. Lin and Chen (2020) proposed a deep learning recommendation based system in programming learning that recommended learning tasks, learning missions and materials according to students’ learning processes and levels. In summary, AI technologies were widely applied in ITSs to enhance personalized and adaptive learning in STEM education through providing one-to-one tutoring and recommending personalized learning paths and resources.
Student behavior detection
The third category of the AI applications in STEM education was student behavior detection, which referred to systems to exploit and track students’ learning behaviors, patterns, and characteristics with AI-enabled data mining and learning analytics in the instructional and learning processes (Chrysafiadi & Virvou, 2013; Ji & Han, 2019; Zheng et al., 2020). Among the 63 reviewed articles, 13 articles (21%) focused on the applications of AI techniques to detect student behaviors in STEM education (see Fig. 5). Two sub-categories were summarized under student behavior detection: student behavior analysis (N = 8) and student behavior monitoring (N = 5) (see Table 3). First, the sub-category of student behavior detection, was applied in STEM education to analyze and reveal students’ latent behaviors. For example, Hsiao et. al. (2020) collected students’ learning data from programming learning platform and examined their learning behaviors through hidden Markov model, and the results revealed the reviewing patterns and reflecting strategies of students in learning programming. Pereira et. al. (2020) used data mining techniques including k-means and association rule algorithm to understand students’ behavior in introductory programming, to help novice programmers promote their learning. Another sub-category of student behavior monitoring was applied to help instructors track students’ learning in STEM education. For example, Balakrishnan (2018) helped instructors to motivate engineering students’ learning through monitoring their learning behaviors such as preferred learning materials and self-directed learning performance. Yannier et. al. (2020) introduced a mixed-reality AI system supported with computer vision algorithms to track children’s active learning behaviors in science education. In summary, student behavior detection had great potential to aid instructors and researchers to analyze, understand, and monitor students’ behaviors in STEM education.
The fourth category of the AI applications in STEM education was automation, which utilized AI technologies to automatically assess students’ performances and generate questions or tasks for instructors (Aldabe & Maritxalar, 2014; Wang et al., 2011; Zampirolli et al., 2021). Among 63 reviewed articles, 8 articles (13%) focused on the AI-supported automated techniques in STEM education (see Fig. 5). Two sub-categories were summarized under automation: automated assessment (N = 7) and automated questions generation (N = 1) (see Table 3). The first sub-category of automated assessment provided instructors and students with convenient assistance in STEM education. For example, Wang et. al. (2011) developed an automated assessment system, AutoLEP, to help novice programmers gain programming skills by providing syntactic and structural checking and immediate feedback automatically. García-Gorrostieta et. al. (2018) introduced a system for automatic argument assessment of computer engineering students’ final reports, to help them improve the abilities of statement and justification in science argumentation. Another sub-category of automated questions generation had potential to reduce instructors’ instructional burdens in STEM education. For example, Aldabe and Maritxalar (2014) proposed an approach to help instructors automatically create multiple-choice tests in science courses through the use of corpora and natural language processing techniques. In summary, AI techniques were used in STEM education to aid instructors and students through automatically generating questions and assessing academic performances.
The fifth category of the AI applications in STEM education was educational robots, which was the adoption of robots in STEM education to facilitate students’ learning experience as well as allow them to acquire knowledge in interactive ways (Atman Uslu et al., 2022; Cao et al., 2021; Yang & Zhang, 2019). It is worth noting that robots are applications that contain various techniques (e.g., mechanical manufacturing, electronic sensors, AI); therefore, considering the research topic, only AI-supported robots were included in this review. Among the 63 reviewed articles, 6 articles (9%) focused on the application of educational robots in STEM education (see Fig. 5). Two sub-categories were identified under educational robots: programming robots (N = 3) and social robots (N = 3) (see Table 3). The first sub-category, programming robots, were specifically designed as learning tools that engaged students to design and operate them with programming languages (Atman Uslu et al., 2022). For example, Rodríguez Corral et. al. (2016) applied a specific ball-shaped robot with sensing, wireless communication and output capabilities in computer courses to teach students object-oriented programming languages. Cao et. al. (2021) introduced an artificial intelligence robot called LEGO MINDSTORMS EV3, to implement instructional tasks in information technology courses to promote students’ innovation and operational ability. Another sub-category of social robots was a kind of intelligent humanoid robots, which could serve as tutors, tutees or learning companions to students and allow students to interact with them orally and physically (Belpaeme et al., 2018; Xu & Ouyang, 2022). For example, Verner et. al. (2020) employed RoboThespian, a life-size humanoid robot, as a tutor to convey science knowledge and concepts to elementary school students. In summary, AI-based educational robots were used in STEM education as instructional tools or educational subjects (e.g., tutor, tutee, companion) to convey knowledge, promote students’ operational skills, and enhance their learning experience.
Among 63 reviewed articles, 2 articles (3%) focused on other applications of AI techniques in STEM education, including AI textbook and group formation. Tehlan et. al. (2020) utilized a genetic algorithm‐based approach to form student groups in collaborative learning based on their skills and personality traits in a programming course. Koć-Januchta et. al. (2020) introduced AI-enriched textbook in biology course to improve students’ engagement by encouraging them to ask questions and receive suggested questions.
RQ2: What are the characteristics of other system elements (i.e., information, subject, medium, environment element) as well as the distribution of AI in these elements?
To further understand how AI techniques have been integrated in STEM education, we examined the other system elements, including information, subject (i.e., instructor, student), medium, and environment in AI-STEM research. In addition, we explored the distribution of AI categories in these elements, to reveal the relationships between AI techniques and these elements.
Information in AI-STEM research
Information (referred to learning content in this study) was described as the subject knowledge and learning contents conveyed in AI-STEM system. In the reviewed 63 studies, all of them mentioned learning content in STEM education, including science, technology, engineering, mathematics, and cross-disciplinary (i.e., more than one discipline) (see Table 4). Among 63 articles, 24 studies focused on technology, followed by articles that focused on science (N = 22) and engineering (N = 7). Mathematics (N = 3) attracted the least attention. In addition, 7 studies contained interdisciplinary subjects, such as computer engineering (Buenaño-Fernández et al., 2019; Tehlan et al., 2020), engineering mathematics (Deo et al., 2020), and integrated STEM education (Suh et al., 2019; Wang, 2016).
Figure 6 shows the frequency of AI application categories in different learning contents. Among all the AI categories, student behavior detection was most frequently applied in the technology domain (N = 8), followed by learning prediction in science (N = 6), and learning prediction in engineering (N = 5) (see Fig. 6).
Instructor in AI-STEM research
Instructor, as a component of subject element in AI-STEM system, played a critical role in conducting instruction, conveying knowledge, and utilizing technologies. In the reviewed 63 studies, 50 of them mentioned the instructor involvement and 50 of them mentioned the instructional strategies, including traditional lecture, problem-based learning, project-based learning, game-based learning, self-learning, and collaborative learning (see Table 5). Regarding the instructor involvement, a majority of instructors would engage in the instructional and learning processes to support students (N = 42), while some studies were conducted without instructors’ involvement and support (N = 8). Additionally, among 50 articles, the traditional lecturing strategy was most frequently used by instructor (N = 27), followed by problem-based learning (N = 10). Also, some studies were carried out through project-based learning (N = 5), self-learning (N = 5), game-based learning (N = 4), and collaborative learning (N = 4).
All AI application categories were mainly applied with the instructor’s support, in which the most frequently used AI category were ITS (N = 11) and learning prediction (N = 11) (see Fig. 7a). In addition, automation was only applied in lecture (N = 8). Learning prediction was most frequently applied in lecture (N = 8) and educational robots were most frequently applied in problem-based learning (N = 4). Compared to other AI technologies, ITS and student behavior detection were integrated with more types of instructional strategies (see Fig. 7b).
Learner in AI-STEM research
Learner, as another component of subject element, could take agency to actively participate in the learning process as to influence the AI-STEM system. In the reviewed 63 studies, 59 of them mentioned the educational levels of learners, from kindergarten to higher education, and 55 of them mentioned sample sizes (see Table 6). Among all the educational levels, 43 focused on higher education (N = 43), followed by elementary school (N = 7), high school (N = 5), and middle school (N = 4). Only one study was conducted in kindergarten. In addition, the number of AI-STEM studies with the medium scale of learners (N = 24) and the large scale of learners (N = 21) were larger than the small-scale study (N = 10).
AI application categories except educational robots were frequently applied in higher education (learning prediction: N = 13, ITS: N = 12, student behavior detection: N = 8, automation: N = 7). The educational robots were frequently applied in elementary school (N = 3) (see Fig. 8a). Moreover, regarding the sample size, learning prediction was most frequently used with a large scale (N = 11), followed by ITS with a medium scale (N = 9), and student behavior detection with a medium scale (N = 7). Additionally, the categories of educational robots and others were not applied in large scale; the category of student behavior detection was not applied in small scale (see Fig. 8b).
Medium in AI-STEM research
Medium (referred to educational medium in this study) was viewed as the way to convey information and connect subjects AI-STEM system. In the reviewed 63 studies, 50 of them mentioned the educational medium, including paper resource, entity resource (i.e., the material object in reality), computer system resource, web open resource, mobile phone resource and E-book resource (see Table 7). Among all educational mediums, computer system was the most frequently used in AI-STEM studies (N = 28), followed by entity resource (N = 10) and web open resource (N = 9). Additionally, mobile phone resource (N = 3), traditional paper resource (N = 1), and E-book resource (N = 1) was the infrequent medium to convey knowledge.
Among all the AI categories, ITS was most frequently used through computer system resource (N = 15), followed by educational robots through entity resource (N = 6), automation through computer system resource (N = 5), and learning prediction through web open resource (see Fig. 9).
Environment in AI-STEM research
Environment (referred to educational context in this study) served as an underlying context to influence the whole AI-STEM system. In the reviewed 63 studies, 51 of them mentioned the educational environment, including face-to-face environment, experimental learning environment, informal learning environment, web-based environment and augmented/virtual reality (see Table 8). Among the 51 studies, 33 studies were implemented in face-to-face environment, followed by web-based environment (N = 11) and experimental environment (N = 6). Two studies conducted in informal learning environment (McLurkin et al., 2013; Verner et al., 2020) and only one study conducted in augmented reality (Lin & Chen, 2020).
All categories of AI techniques were commonly applied in face-to-face environment, in which the most frequently used AI technology category was learning prediction (N = 10), followed by automation (N = 7), ITS (N = 6), and educational robots (N = 5). Moreover, compared to other AI categories, ITS was the most frequently used technique in the web-based environment (N = 7) (see Fig. 10).
RQ3: What are the effects of AI in STEM education?
This review summarized the educational and technological effects of AI applications in AI-STEM research.
Educational effects and findings
From the educational perspective, 42 of the 63 reviewed articles reported the educational effects and findings when applying AI techniques in STEM education. Specifically, 30 out of the 42 articles reported the instruction and learning effects (e.g., learning performance, affective perception, higher-order thinking) of the application of AI techniques in STEM education. 12 articles out of 42 reported students’ learning behaviors and patterns by using AI-enabled data mining and learning analytics techniques.
The effect of learning performance
Among all the reviewed articles, 22 studies revealed the educational effects of AI technologies on students’ learning performance. Most of them showed significantly positive influence of AI techniques on the improvement of students’ learning performances (N = 20). For example, Wu et. al. (2013) investigated the effect of a context-aware ubiquitous learning system in a geosciences course and the results showed that context-aware ubiquitous learning system had significantly positive effects on the learning achievements of students. Thai et. al. (2021) conducted a cluster randomized study to examine the effect of My Math Academy, a digital game-based learning environment that provided personalized content on kindergarten students; the results revealed the significant improvement of learning gains, especially for the moderate-level students. Tehlan et. al. (2020) used a quasi-experiment approach to examine the effects of genetic algorithm-supported pair programming in a programming course; the results found that the students’ learning performances were significantly higher in pair programming than individual programming. Two articles reported insignificant results of the learning performance effects. Koć-Januchta et. al. (2020) used a quasi-experiment to compare the effect of AI-enabled E-book and common E-book in students’ biology learning, and the results showed that there was no significant difference of students’ learning gains between these two types of books. Also, Hellings and Haelermans (2020) conducted a randomized experiment to examine the effect of a learning analytics dashboard with predictive function in a computer programming course, but no significant improvement was found on student performance in the final exam.
The effect of affective perception
Among all the reviewed articles, a majority of studies revealed the educational effects of AI technologies on students’ affective perception, such as attitude, interest, and motivation (N = 17). On the one hand, students showed satisfaction and positive attitude towards the integration of AI technologies and STEM education. For example, Azcona et. al. (2019) used a questionnaire to find students’ positive feedbacks and attitudes towards the application of learning analytics in computer programming classes to detect and warn learning risks. Gavrilović et. al. (2018) evaluated student’s satisfaction of an AI-supported adaptive learning system in Java programming learning through the survey approach; the results revealed the positive feedbacks of students. On the other hand, the application of AI technologies also arouses students’ interests and motivation in STEM learning. For example, Balakrishnan (2018) used a mixed-method approach (i.e., questionnaire and interview) to examine the impact of a computer-based personalized learning environment (PLE) on engineering students’ motivation, and the results revealed the potential of PLE to engage students in learning with a strong sense of interest and motivation. Verner et. al. (2020) investigated students’ perceptions and attitudes towards an interactive robot tutor in science classes and found that the human–robot interaction fostered students’ active learning, maintain their attention and interest in the learning processes.
The effect of higher-order thinking
Among all the reviewed articles, some studies revealed the educational effects of AI technologies on students’ higher-order thinking (N = 7), such as problem-solving ability, computational thinking, and self-regulated learning skills. For example, Hooshyar et. al. (2015) employed a quasi-experimental design to examine the impact of a flowchart-based intelligent tutoring system (FITS) on students’ programming learning and found better improvement of problem-solving abilities in the FITS group than the control group. Lin and Chen (2020) found that students who used a deep learning-based AR system performed significantly better in computational thinking than those using an AR system without deep learning recommendation. Jones and Castellano (2018) utilized adaptive robotic tutors to promote students’ self-regulated learning skills and found that when a robotic tutor provided scaffoldings adaptively, more self-regulated learning behaviors were observed from students over the control condition without scaffoldings. García-Gorrostieta et. al. (2018) used experimental evaluation to test the effect of the automatic argument assessment on students’ computer engineering writing, and the results revealed that the argument assessment system helped students improve argumentation ability in their writing.
The effect of student learning pattern and behavior
Among 42 reviewed articles that mentioned educational effects and findings, 12 articles revealed students’ learning patterns and behaviors in STEM education by using AI-enabled data mining and learning analytics approaches. For example, Sapounidis et. al. (2019) detected 48 children’s preference profiles on tangible and graphical programming through latent class modeling; results found that the graphical programming was preferred by a majority of children, especially children in younger ages. Pereira et. al. (2020) used learning analytics (i.e., k-means, association rule algorithms) in the Amazonas to understand students’ behavior in introductory programming courses and found high heterogeneity among them. Three clusters of novice programmers were detected to explain how student behaviors during programming influenced the learning outcomes. Wang (2016) utilized data mining and learning analytics techniques (i.e., association rule, decision tree) to investigate college students’ course-taking patterns in STEM learning; the results found that the most viable course-taking trajectories is taking mathematics courses after initial exposure to subject courses in STEM.
Technological effects and findings
From the technological perspective, 24 of the 63 reviewed articles reported the technological effect and findings (e.g., efficiency of technology, accuracy of algorithm) when applying AI techniques in STEM education. For example, Çınar et. al. (2020) utilized multiple machine learning algorithms, including Support Vector Machines (SVM), Gini, k-Nearest Neighbors (KNN), Breiman’s Bagging, Freund and Schapire’s Adaboost.M1 algorithms, to automatically grade open-ended physics questions; the results reported that AdaBoost.M1 had the best performance with the highest accuracy of prediction models among all machine learning algorithms. Nehm et. al. (2012) used a corpus of biology evolutionary explanations written by 565 undergraduates to test the efficacy of an automated assessment program, Summarization Integrated Development Environment (SIDE); the results showed that, compared to human expert scoring, SIDE had better performance when scoring models were built and tested at the individual item level, and the performance degraded when suites of items or entire instruments were used to build and test scoring models. Bertolini et. al. (2021) employed five machine learning methods to quantify predictive efficacy of predictive modeling in undergraduate students’ outcome in biology. Results found that individual machine learning methods, especially logistic regression achieved a poor prediction performance while ensemble machine learning methods, in particular the generalized linear model with elastic net (GLMNET), achieved the high accuracy. Deo et. al. (2020) adopted a computationally efficient AI model called extreme learning machines (ELM) to predict weighted score and the examination score in engineering mathematics courses; the results showed that ELM outperformed in prediction with respect to random forest and Volterra.
Discussion and implications
Addressing research questions
Although AIEd has attracted wide attention in educational research and practice, few research works have investigated the applications of AI in STEM education context. To gain a comprehensive understanding of the integration of AI in STEM education, this study conducted a systematic review of AI-STEM empirical research from 2011 to 2021. Grounded upon GST, we examined the AI technologies and applications in STEM education, the characteristics of other system elements (i.e., information, subject, medium, environment), the distribution of AI in these elements, and the effects of AI applications in STEM education. To answer the first question, we found a gradually increasing trend of AI applications in STEM education in the past decade. Furthermore, six categories of AI applications were located, namely learning prediction, ITS, student behavior detection, automation, educational robots, and others (i.e., AI text book, group formation). Regarding the characteristics of elements and the distribution of AI in these elements, first, we found all categories of AI techniques, especially student behavior detection, ITS, and learning prediction, were frequently applied in the learning contents of science and technology. Second, instructors usually involved in STEM education to support students and they used lecturing strategy the most frequently, followed by problem-based learning. Automation was only applied in the lecturing instruction mode and educational robots were most frequently applied in the problem-based learning mode. Third, a majority of AI techniques (except educational robots) were applied in higher education with medium and large scale of learners. The most frequently used AI in higher education were learning prediction and ITSs. Fourth, computer system resource was the most frequently used medium to convey knowledge, particularly when it was applied in ITSs and automation, while paper, mobile phone, and E-book resources were seldom used in AI-STEM research. Fifth, the face-to-face environment was mainly utilized to support all categories of AI applications, and web-based environment was most frequently used supported with ITSs.
Regarding the third question, this review summarized the educational and technological effects and findings of AI applications in STEM. From the educational perspective, the results showed that most of the AI applications had positive effects on students’ academic performance. However, insignificant improvements of learning outcomes were also found in two empirical studies (Hellings & Haelermans, 2020; Koć-Januchta et al., 2020). Moreover, most students held positive attitudes towards the use of AI technology in STEM education, and AI technologies aroused their interest and motivation as well. In other words, the AI applications are beneficial for fostering student’s active learning in STEM education. Moreover, the applications of AI techniques also contributed to the development of students’ higher-order thinking, e.g., computational thinking, problem-solving ability. In addition, AI techniques have great potential to assist instructors by detecting students’ learning patterns and behaviors in STEM education. From the technological perspective, the reviewed articles mainly reported a good efficiency and algorithm accuracy when applying AI in STEM education. Specifically, AI algorithms, especially ensemble machine learning methods, performed well in learning prediction, automation, and personalized recommendation. Overall, underpinned by the GST framework, this review presented an overview of recent trends of the field of AI-STEM, which guided the following educational, technological, and theoretical implications.
The emergence of AI indirectly influences the subject elements (e.g., instructor, learner) in STEM education, which in turn would eventually influence the educational practices and effects. First, AI has potential to transform the instructor–student relationships in STEM education from the instructor-directed to student-centered learning (Cviko et al., 2014). When AI is applied in STEM education, the role of instructor is expected to shift from a leader to a collaborator or a facilitator under the AI-empowered, learner-as-leader paradigm (Ouyang & Jiao, 2021). However, this review found that the instructor-centered lecturing mode was the most frequently used instructional strategy in AI-STEM studies, while other student-centered instructional strategies (e.g., the project-based learning, collaborative learning, game-based learning) appeared infrequently. One of the reasons centers on the complexity of integrating technology and pedagogy in STEM education (Castañeda & Selwyn, 2018; Jiao et al., 2022; Loveless, 2011). For example, ITS and automation techniques are usually designed based on behaviorism (Skinner, 1953) to support instructor’s knowledge delivery and exam evaluation, which may be challenging for instructors to use when integrating it in the student-centered instructional strategies. Recent research has started to balance pedagogical design and technological application in educational practices in order to achieve the goal of AI–instructor collaboration and student-centered learning when AI is integrated (Baker & Smith, 2019; Holmes et al., 2019; Roll & Wylie, 2016). Furthermore, another critical question is: would AI replace instructor responsibilities and roles in STEM education (Segal, 2019)? In this review, we found that the role of instructor was still irreplaceable, because the instructor’s involvement existed in most of the AI-STEM research. Even though AI can free instructors from redundant tasks in STEM education, it still lacks the human ability to convey social emotion, solve critical problems, and implement creative activities (Collinson, 1996; Gary, 2019; Muhisn et al., 2019). Therefore, although AI techniques can bring opportunities to develop STEM education (Hwang et al., 2020), we cannot overstate the role of technology and overlook the essential role of instructor (Selwyn, 2016). Overall, instructors, as important subjects in the educational system, need to take agency to promote the pedagogical designs and strategies when applying AI technologies, in order to achieve a high quality of AI-STEM education (Cantú-Ortiz et al., 2020).
Although AI has the potential to enhance the instruction and learning in STEM education (Chen et al., 2020; Holmes et al., 2019), the development of AI-STEM requires a better fit between AI technologies and other system elements in STEM education. First, regarding the relationships between AI and information element, the results showed that most of the AI applications were used in science and technology learning contents, and educational robots and automation were not applied in engineering and mathematics learning contents. Since STEM education contains interdisciplinary knowledge and learning contents from different subjects, AI is usually restricted in specific learning contents or courses (Douce et al., 2005). Therefore, one of the future directions is to expand the commonality and accessibility of AI techniques in different STEM subjects and courses. Second, the range of AI applications was mainly located in higher education, while few of AI techniques were applied in other educational levels, especially in kindergarten. To some extent, due to the complex function and feedback mechanisms, most of the AI techniques (e.g., ITS, learning prediction) might be appropriate for adult learners. Hence, some interactive AI techniques, e.g., social robots, AI-enabled games, can be designed and developed to support young children’s STEM learning (Belpaeme et al., 2018; Zapata-Cáceres & Martin-Barroso, 2021). Therefore, the ease of use is also one of the important considerations in future development of AI technologies (Law, 2019; Xu & Ouyang, 2022). Third, most of the AI-STEM research was conducted through the traditional mediums (e.g., computer system resource) and contexts (e.g., face-to-face learning environment). A future direction is to create AI-empowered STEM learning environment through combining the advanced educational mediums (e.g., E-book) and contexts (AR/VR), in order to better represent and convey knowledge (Mystakidis et al., 2021).
Due to the complexity of AI-STEM system, this research used a theoretical framework based on GST to examine the multiple elements (i.e., AI technology, information, subject, medium, environment) in AI-STEM research. Compared to previous AIEd reviews that mainly focused on the technological perspective, GST provides a holistic view for us to consider the complex human, pedagogical, environment factors when applying AI in STEM education (Kitto, 2014; Von Bertalanffy, 1968). For example, we found that sometimes instructors did not engage in STEM education to support students, especially when applying ITSs and educational robots. It might reveal a new trend that some AI technologies (e.g., social robot, virtual agent) might have the potential to replace the original role of instructor and work as a new subject to individually convey knowledge (Xu & Ouyang, 2022). Additionally, the results showed the different characteristics of learner’s sample size when applying different AI techniques. Learning prediction was more likely to applied with a large scale of students, and educational robots were inclined to be applied with a small scale of students. The features of AI technologies might explain this phenomenon. For example, a data training process is necessary before learning prediction, which requires the support of algorithmic modeling techniques and large data sets (Agrawal & Mavani, 2015; Lee et al., 2017), while educational robots, as human–machine interaction technologies, seem more suitable and practicable for small-scale STEM learning (Atman Uslu et al., 2022; Belpaeme et al., 2018). Overall, the current study utilized the GST framework to examine the multiple elements in the complex AI-STEM system; it is suggested that different stakeholders, e.g., educators, technical developers, and researchers, can adopt the GST framework as a guide to comprehensively consider the complex elements when applying AI techniques in STEM education (Kitto, 2014; Von Bertalanffy, 1950).
Conclusion, limitation, and future direction
The application of AI technology in STEM education is an emerging trend, which is confronted with the challenge of integrating diverse AI techniques in the complex STEM educational system. Grounded upon a GST framework, this research reviewed the empirical AI-STEM studies from 2011 to 2021. Specifically, this systematic review examined (1) the categories of the AI element in the AI-STEM system; (2) the characteristics of other system elements (i.e., information, subject, medium, and environment) as well as the distribution of AI in these elements, and (3) the effects of AI in STEM education. Based on the results, the current work proposed educational, technological, and theoretical implications for future AI-STEM research, to better aid the educators, researchers and technical developers to integrate the AI techniques and STEM education.
There are three limitations in this systematic review, which lead to future research directions. First, although we searched the best-known scholar databases with the keywords relevant to AI-STEM, some biases might exist in the searching and screening process. Since AI-STEM is a highly technology-dependent field, some studies might only highlight the technology rather than the education context. Therefore, future studies can adjust the searching criteria to solve these problems. Second, from a system perspective, we used a GST framework to examine the multiple elements in the complex AI-STEM system, but we did not investigate the mutual relationships between elements. Therefore, the complex relationships between different elements (e.g., instructor–learner, learner–learner relationship) in AI-STEM system need to be further explored in order to gain a deep understanding of the application of AI in STEM education (e.g., Xu & Ouyang, 2022). Third, the current study only implemented a systematic review, a meta-analysis could be conducted in the future to report the effect sizes of recent empirical studies to gain a deeper understanding of the effects of the AI-STEM integration in an educational system. Overall, the potential of AI technology for enhancing STEM education is fertile ground to be further explored together with studies aimed at investigating the integration of technology and educational system.
Availability of data and materials
The data are available upon request from the corresponding author.
*Reviewed articles (N = 63)
Agrawal, H., & Mavani, H. (2015). Student performance prediction using machine learning. International Journal of Engineering Research and Technology, 4(03), 111–113. https://doi.org/10.17577/IJERTV4IS030127
Alabdulhadi, A., & Faisal, M. (2021). Systematic literature review of STEM self-study related ITSs. Education and Information Technologies, 26, 1549–1588. https://doi.org/10.1007/s10639-020-10315-z
*Aldabe, I., & Maritxalar, M. (2014). Semantic similarity measures for the generation of science tests in Basque. IEEE Transactions on Learning Technologies, 7(4), 375–387. https://doi.org/10.1109/TLT.2014.2355831
*Alemán, J. L. F. (2011). Automated assessment in a programming tools course. IEEE Transactions on Education, 54(4), 576–581. https://doi.org/10.1109/TE.2010.2098442
Atman Uslu, N., Yavuz, G. Ö., & KoçakUsluel, Y. (2022). A systematic review study on educational robotics and robots. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.2023890
*Azcona, D., Hsiao, I. H., & Smeaton, A. F. (2019). Detecting students-at-risk in computer programming classes with learning analytics from students’ digital footprints. User Modeling and User-Adapted Interaction, 29(4), 759–788. https://doi.org/10.1007/s11257-019-09234-7
Baker, T., & Smith, L. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. https://media.nesta.org.uk/documents/Future_of_AI_and_education_v5_WEB.Pdf
*Balakrishnan, B. (2018). Motivating engineering students learning via monitoring in personalized learning environment with tagging system. Computer Applications in Engineering Education, 26(3), 700–710. https://doi.org/10.1002/cae.21924
Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., & Tanaka, F. (2018). Social robots for education: A review. Science Robotics, 3(21), eaat5954. https://doi.org/10.1126/scirobotics.aat5954
*Berland, M., Davis, D., & Smith, C. P. (2015). AMOEBA: Designing for collaboration in computer science classrooms through live learning analytics. International Journal of Computer-Supported Collaborative Learning, 10(4), 425–447. https://doi.org/10.1007/s11412-015-9217-z
*Bertolini, R., Finch, S. J., & Nehm, R. H. (2021). Testing the impact of novel assessment sources and machine learning methods on predictive outcome modeling in undergraduate biology. Journal of Science Education and Technology, 30(2), 193–209. https://doi.org/10.1007/s10956-020-09888-8
*Blikstein, P., Worsley, M., Piech, C., Sahami, M., Cooper, S., & Koller, D. (2014). Programming pluralism: Using learning analytics to detect patterns in the learning of computer programming. Journal of the Learning Sciences, 23(4), 561–599. https://doi.org/10.1080/10508406.2014.954750
*Buenaño-Fernández, D., Gil, D., & Luján-Mora, S. (2019). Application of machine learning in predicting performance for computer engineering students: A case study. Sustainability, 11(10), 1–18. https://doi.org/10.3390/su11102833
Bybee, R. W. (2013). The case for STEM education: Challenges and opportunities. NSTA Press.
Byrne, D., & Callaghan, G. (2014). Complexity theory and the social sciences. Routledge.
Cantú-Ortiz, F. J., Galeano Sánchez, N., Garrido, L., Terashima-Marin, H., & Brena, R. F. (2020). An artificial intelligence educational strategy for the digital transformation. International Journal on Interactive Design and Manufacturing, 14(4), 1195–1209. https://doi.org/10.1007/s12008-020-00702-8
*Cao, X., Li, Z., & Zhang, R. (2021). Analysis on academic benchmark design and teaching method improvement under artificial intelligence robot technology. International Journal of Emerging Technologies in Learning, 16(5), 58–72. https://doi.org/10.3991/ijet.v16i05.20295
Castañeda, L., & Selwyn, N. (2018). More than tools? Making sense of the ongoing digitizations of higher education. International Journal of Educational Technology in Higher Education, 15(1), 1–10. https://doi.org/10.1186/s41239-018-0109-y
Chen, D., & Stroup, W. (1993). General system theory: Toward a conceptual framework for science and technology education for all. Journal of Science Education and Technology, 2(3), 447–459. https://doi.org/10.1007/BF00694427
Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510
*Chrysafiadi, K., & Virvou, M. (2013). PeRSIVA: An empirical evaluation method of a student model of an intelligent e-learning environment for computer programming. Computers & Education, 68, 322–333. https://doi.org/10.1016/j.compedu.2013.05.020
*Çınar, A., Ince, E., Gezer, M., & Yılmaz, Ö. (2020). Machine learning algorithm for grading open-ended physics questions in Turkish. Education and Information Technologies, 25, 3821–3844. https://doi.org/10.1007/s10639-020-10128-0
Cohen, L., Manion, L., & Morrison, K. (2005). Research methods in education. Routledge Falmer.
Collinson, V. (1996). Reaching students: Teachers ways of knowing. Corwin Press.
Crawford, J. L. (1974). A systems approach model for the application of general systems theory principles to education [Doctoral dissertation, University of Houston]. The University of Houston Institutional Repository. https://hdl.handle.net/10657/10661
Cviko, A., McKenney, S., & Voogt, J. (2014). Teacher roles in designing technology-rich learning activities for early literacy: A cross-case analysis. Computers & Education, 72, 68–79. https://doi.org/10.1016/j.compedu.2013.10.014
*De-Marcos, L., Garcia-Cabot, A., Garcia-Lopez, E., & Medina, J. A. (2015). Parliamentary optimization to build personalized learning paths: Case study in web engineering curriculum. International Journal of Engineering Education, 31(4), 1092–1105.
*Deo, R. C., Yaseen, Z. M., Al-Ansari, N., Nguyen-Huy, T., Langlands, T. A. M., & Galligan, L. (2020). Modern artificial intelligence model development for undergraduate student performance prediction: An investigation on engineering mathematics courses. IEEE Access, 8, 136697–136724. https://doi.org/10.1109/ACCESS.2020.3010938
Douce, C., Livingstone, D., & Orwell, J. (2005). Automatic test-based assessment of programming: A review. Journal on Educational Resources in Computing, 5(3), 4-es. https://doi.org/10.1145/1163405.1163409
Drack, M., & Pouvreau, D. (2015). On the history of Ludwig von Bertalanffy’s “general systemology”, and on its relationship to cybernetics—Part III: Convergences and divergences. International Journal of General Systems, 44(5), 523–571. https://doi.org/10.1080/03081079.2014.1000642
Drigas, A. S., & Ioannidou, R. E. (2012). Artificial intelligence in special education: A decade review. International Journal of Engineering Education, 28(6), 1366. http://imm.demokritos.gr/publications/AI_IJEE.pdf
*Ferrarelli, P., & Iocchi, L. (2021). Learning Newtonian physics through programming robot experiments. Technology, Knowledge and Learning, 26, 789–824. https://doi.org/10.1007/s10758-021-09508-3
*Figueiredo, M., Esteves, L., Neves, J., & Vicente, H. (2016). A data mining approach to study the impact of the methodology followed in chemistry lab classes on the weight attributed by the students to the lab work on learning and motivation. Chemistry Education Research and Practice, 17(1), 156–171. https://doi.org/10.1039/c5rp00144g
*García-Gorrostieta, J. M., López-López, A., & González-López, S. (2018). Automatic argument assessment of final project reports of computer engineering students. Computer Applications in Engineering Education, 26(5), 1217–1226. https://doi.org/10.1002/cae.21996
Gary, K. (2019). Pragmatic standards versus saturated phenomenon: Cultivating a love of learning. Journal of Philosophy of Education, 53(3), 477–490. https://doi.org/10.1111/1467-9752.12377
*Gavrilović, N., Arsić, A., Domazet, D., & Mishra, A. (2018). Algorithm for adaptive learning process and improving learners’ skills in Java programming language. Computer Applications in Engineering Education, 26(5), 1362–1382. https://doi.org/10.1002/cae.22043
Graneheim, U. H., & Lundman, B. (2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112. https://doi.org/10.1016/j.nedt.2003.10.001
Guan, C., Mou, J., & Jiang, Z. (2020). Artificial intelligence innovation in education: A twenty-year data-driven historical analysis. International Journal of Innovation Studies, 4(4), 134–147. https://doi.org/10.1016/j.ijis.2020.09.001
*Hellings, J., & Haelermans, C. (2020). The effect of providing learning analytics on student behaviour and performance in programming: A randomised controlled experiment. Higher Education, 83(1), 1–18. https://doi.org/10.1007/s10734-020-00560-z
Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
Holstein, K., McLaren, B. M., & Aleven, V. (2019). Co-designing a real-time classroom orchestration tool to support teacher—AI complementarity. Journal of Learning Analytics, 6(2), 27–52. https://doi.org/10.18608/jla.2019.62.3
*Hooshyar, D., Ahmad, R. B., Yousefi, M., Yusop, F. D., & Horng, S. (2015). A flowchart-based intelligent tutoring system for improving problem-solving skills of novice programmers. Journal of Computer Assisted Learning, 31, 345–361. https://doi.org/10.1111/jcal.12099
*Hooshyar, D., Binti Ahmad, R., Wang, M., Yousefi, M., Fathi, M., & Lim, H. (2018). Development and evaluation of a game-based Bayesian intelligent tutoring system for teaching programming. Journal of Educational Computing Research, 56(6), 775–801. https://doi.org/10.1177/0735633117731872
*Hsiao, I. H., Huang, P. K., & Murphy, H. (2020). Integrating programming learning analytics across physical and digital space. IEEE Transactions on Emerging Topics in Computing, 8(1), 206–217. https://doi.org/10.1109/TETC.2017.2701201
Hwang, G. J., & Tu, Y. F. (2021). Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review. Mathematics, 9(6), 584. https://doi.org/10.3390/math9060584
Hwang, G. J., Xie, H., Wah, B. W., & Gašević, D. (2020). Vision, challenges, roles and research issues of artificial intelligence in education. Computers and Education: Artificial Intelligence, 1, 100001. https://doi.org/10.1016/j.caeai.2020.100001
*Ji, Y., & Han, Y. (2019). Monitoring indicators of the flipped classroom learning process based on data mining—Taking the course of “virtual reality technology” as an example. International Journal of Emerging Technologies in Learning, 14(3), 166–176. https://doi.org/10.3991/ijet.v14i03.10105
Jiao, P., Ouyang, F., Zhang, Q., & Alavi, A. H. (2022). Artificial intelligence-enabled prediction model of student academic performance in online engineering education. Artificial Intelligence Review. https://doi.org/10.1007/s10462-022-10155-y
*Jones, A., Bull, S., & Castellano, G. (2018). “I know that now, I’ m going to learn this next” Promoting self-regulated learning with a robotic tutor. International Journal of Social Robotics, 10(4), 439–454. https://doi.org/10.1007/s12369-017-0430-y
*Jones, A., & Castellano, G. (2018). Adaptive robotic tutors that support self-regulated learning: A longer-term investigation with primary school children. International Journal of Social Robotics, 10(3), 357–370. https://doi.org/10.1007/s12369-017-0458-z
*Khan, I., Ahmad, A. R., Jabeur, N., & Mahdi, M. N. (2021). Machine learning prediction and recommendation framework to support introductory programming course. International Journal of Emerging Technologies in Learning, 16(17), 42–59. https://doi.org/10.3991/ijet.v16i17.18995
Khandelwal, P., Srinivasan, K., & Roy, S. S. (2019). Surgical education using artificial intelligence, augmented reality and machine learning: A review. In A. Sengupta & P. Eng (Eds.), 2019 IEEE international conference on consumer electronics—Taiwan (pp. 1–2). IEEE.
*Kinnebrew, J. S., Killingsworth, S. S., Clark, D. B., Biswas, G., Sengupta, P., Minstrell, J., Martinez-garza, M., & Krinks, K. (2017). Contextual markup and mining in digital games for science learning: Connecting player behaviors to learning goals. IEEE Transactions on Learning Technologies, 10(1), 93–103. https://doi.org/10.1109/TLT.2016.2521372
Kitto, K. (2014). A contextualised general systems theory. Systems, 2(4), 541–565. https://doi.org/10.3390/systems2040541
*Koć-Januchta, M. M., Schönborn, K. J., Tibell, L. A. E., Chaudhri, V. K., & Heller, H. C. (2020). Engaging with biology by asking questions: Investigating students’ interaction and learning with an artificial intelligence-enriched textbook. Journal of Educational Computing Research, 58(6), 1190–1224. https://doi.org/10.1177/0735633120921581
*Kose, U., & Arslan, A. (2017). Optimization of self-learning in computer engineering courses: An intelligent software system supported by artificial neural network and vortex optimization algorithm. Computer Applications in Engineering Education, 25(1), 142–156. https://doi.org/10.1002/cae.21787
*Krämer, N. C., Karacora, B., Lucas, G., Dehghani, M., Rüther, G., & Gratch, J. (2016). Closing the gender gap in STEM with friendly male instructors? On the effects of rapport behavior and gender of a virtual agent in an instructional interaction. Computers & Education, 99, 1–13. https://doi.org/10.1016/j.compedu.2016.04.002
Krasovskiy, D. (2020). The challenges and benefits of adopting AI in STEM education. https://upjourney.com/the-challenges-and-benefits-of-adopting-ai-in-stem-education
Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30(3), 411–433. https://doi.org/10.1093/hcr/30.3.411
*Lacave, C., Molina, A. I., & Cruz-Lemus, J. A. (2018). Learning analytics to identify dropout factors of computer science studies through Bayesian networks. Behaviour and Information Technology, 37(10–11), 993–1007. https://doi.org/10.1080/0144929X.2018.1485053
*Lamb, R., Hand, B., & Kavner, A. (2021). Computational modeling of the effects of the science writing heuristic on student critical thinking in science using machine learning. Journal of Science Education and Technology, 30(2), 283–297. https://doi.org/10.1007/s10956-020-09871-3
Law, N. W. Y. (2019). Human development and augmented intelligence. In The 20th international conference on artificial intelligence in education (AIED 2019), Chicago, IL, USA.
Le, N. T., Strickroth, S., Gross, S., & Pinkwart, N. (2013). A review of AI-supported tutoring approaches for learning programming. In N. Nguyen, T. Van Do, & H. Le Thi (Eds.), Advanced computational methods for knowledge engineering (pp. 267–279). Springer. https://doi.org/10.1007/978-3-319-00293-4_20
*Ledesma, E. F. R., & García, J. J. G. (2017). Selection of mathematical problems in accordance with student’s learning style. International Journal of Advanced Computer Science and Applications, 8(3), 101–105. https://doi.org/10.14569/IJACSA.2017.080316
Lee, J., Jang, D., & Park, S. (2017). Deep learning-based corporate performance prediction model considering technical capability. Sustainability, 9(6), 899. https://doi.org/10.3390/su9060899
Lee, J., Wu, A. S., Li, D., & Kulasegaram, K. M. (2021). Artificial intelligence in undergraduate medical education: A scoping review. Academic Medicine, 96(11S), S62–S70. https://doi.org/10.1097/ACM.0000000000004291
Liang, J. C., Hwang, G. J., Chen, M. R. A., & Darmawansah, D. (2021). Roles and research foci of artificial intelligence in language education: An integrated bibliographic analysis and systematic review approach. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1958348
*Lin, P. H., & Chen, S. Y. (2020). Design and evaluation of a deep learning recommendation based augmented reality system for teaching programming and computational thinking. IEEE Access, 8, 45689–45699. https://doi.org/10.1109/ACCESS.2020.2977679
Liu, M., Li, Y., Xu, W., & Liu, L. (2017). Automated essay feedback generation and its impact on revision. IEEE Transactions on Learning Technologies, 10(4), 502–513. https://doi.org/10.1109/TLT.2016.2612659
Loveless, A. (2011). Technology, pedagogy and education: Reflections on the accomplishment of what teachers know, do and believe in a digital age. Technology, Pedagogy and Education, 20(3), 301–316. https://doi.org/10.1080/1475939X.2011.610931
*Maestrales, S., Zhai, X., Touitou, I., Baker, Q., Schneider, B., & Krajcik, J. (2021). Using machine learning to score multi-dimensional assessments of chemistry and physics. Journal of Science Education and Technology, 30(2), 239–254. https://doi.org/10.1007/s10956-020-09895-9
*Mahboob, K., Ali, S. A., & Laila, U. E. (2020). Investigating learning outcomes in engineering education with data mining. Computer Applications in Engineering Education, 28(6), 1652–1670. https://doi.org/10.1002/cae.22345
*Matthew, F. T., Adepoju, A. I., Ayodele, O., Olumide, O., Olatayo, O., Adebimpe, E., Bolaji, O., & Funmilola, E. (2018). Development of mobile-interfaced machine learning-based predictive models for improving students’ performance in programming courses. International Journal of Advanced Computer Science and Applications, 9(5), 105–115. https://doi.org/10.14569/IJACSA.2018.090514
McLaren, B. M., Scheuer, O., & Mikšátko, J. (2010). Supporting collaborative learning and e-discussions using artificial intelligence techniques. International Journal of Artificial Intelligence in Education, 20(1), 1–46. https://doi.org/10.3233/JAI-2010-0001
McLurkin, J., Rykowski, J., John, M., Kaseman, Q., & Lynch, A. J. (2013). Using multi-robot systems for engineering education: Teaching and outreach with large numbers of an advanced, low-cost robot. IEEE Transactions on Education, 56(1), 24-33. https://doi.org/10.1109/TE.2012.2222646
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., Prisma Group. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097.t001
Muhisn, Z. A. A., Ahmad, M., Omar, M., & Muhisn, S. A. (2019). The Impact of socialization on collaborative learning method in e-learning management system (eLMS). International Journal of Emerging Technologies in Learning, 14(20), 137–148.
Murray, T. (2003). An overview of intelligent tutoring system authoring tools: Updated analysis of the state of the art. In T. Murray, S. B. Blessing, & S. Ainsworth (Eds.), Authoring tools for advanced technology learning environments (pp. 491–544). Springer. https://doi.org/10.1007/978-94-017-0819-7_17
*Myneni, L. S., Narayanan, N. H., & Rebello, S. (2013). An interactive and intelligent learning system for physics education. IEEE Transactions on Learning Technologies, 6(3), 228–239. https://doi.org/10.1109/TLT.2013.26
Mystakidis, S., Christopoulos, A., & Pellas, N. (2021). A systematic mapping review of augmented reality applications to support STEM learning in higher education. Education and Information Technologies, 27, 1883–1927. https://doi.org/10.1007/S10639-021-10682-1/FIGURES/10
*Nehm, R. H., Ha, M., & Mayfield, E. (2012). Transforming biology assessment with machine learning: Automated scoring of written evolutionary explanations. Journal of Science Education and Technology, 21(1), 183–196. https://doi.org/10.1007/s10956-011-9300-9
Ouyang, F., & Jiao, P. (2021). Artificial intelligence in education: The three paradigms. Computers and Education: Artificial Intelligence, 2, 100020. https://doi.org/10.1016/j.caeai.2021.100020
Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies. https://doi.org/10.1007/s10639-022-10925-9
*Pereira, F. D., Oliveira, E. H. T., Oliveira, D. B. F., Cristea, A. I., Carvalho, L. S. G., Fonseca, S. C., Toda, A., & Isotani, S. (2020). Using learning analytics in the Amazonas: Understanding students’ behaviour in introductory programming. British Journal of Educational Technology, 51(4), 955–972. https://doi.org/10.1111/bjet.12953
Pimthong, P., & Williams, J. (2018). Preservice teachers’ understanding of STEM education. Kasetsart Journal of Social Sciences, 41(2), 1–7. https://doi.org/10.1016/j.kjss.2018.07.017
Rapoport, A. (1986). General system theory: Essential concepts & applications. CRC Press.
*Rodríguez Corral, J. M., Morgado-Estévez, A., Cabrera Molina, D., Pérez-Peña, F., Amaya Rodríguez, C. A., & CivitBalcells, A. (2016). Application of robot programming to the teaching of object-oriented computer languages. International Journal of Engineering Education, 32(4), 1823–1832.
Roll, I., & Wylie, R. (2016). Evolution and revolution in artificial intelligence in education. International Journal of Artificial Intelligence in Education, 26(2), 582–599. https://doi.org/10.1007/s40593-016-0110-3
*Saito, T., & Watanobe, Y. (2020). Learning path recommendation system for programming education based on neural networks. International Journal of Distance Education Technologies, 18(1), 36–64. https://doi.org/10.4018/IJDET.2020010103
*Sapounidis, T., Stamovlasis, D., & Demetriadis, S. (2019). Latent class modeling of children’s preference profiles on tangible and graphical robot programming. IEEE Transactions on Education, 62(2), 127–133. https://doi.org/10.1109/TE.2018.2876363
Segal, M. (2019). A more human approach to artificial intelligence. Nature, 571(7766), S18–S18. https://doi.org/10.1038/d41586-019-02213-3
Selwyn, N. (2016). Is technology good for education? Polity Press.
Skinner, B. F. (1953). Science and human behavior. Macmillan.
*Spikol, D., Ruffaldi, E., Dabisias, G., & Cukurova, M. (2018). Supervised machine learning in multimodal learning analytics for estimating success in project-based learning. Journal of Computer Assisted Learning, 34(4), 366–377. https://doi.org/10.1111/jcal.12263
*Suh, S. C., Anusha Upadhyaya, B. N., & Ashwin Nadig, N. V. (2019). Analyzing personality traits and external factors for stem education awareness using machine learning. International Journal of Advanced Computer Science and Applications, 10(5), 1–4. https://doi.org/10.14569/ijacsa.2019.0100501
Tang, K. Y., Chang, C. Y., & Hwang, G. J. (2021). Trends in artificial intelligence supported e-learning: A systematic review and co-citation network analysis (1998–2019). Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.1875001
*Tehlan, K., Chakraverty, S., Chakraborty, P., & Khapra, S. (2020). A genetic algorithm-based approach for making pairs and assigning exercises in a programming course. Computer Applications in Engineering Education, 28(6), 1708–1721. https://doi.org/10.1002/cae.22349
*Thai, K. P., Bang, H. J., & Li, L. (2021). Accelerating early math learning with research-based personalized learning games: A cluster randomized controlled trial. Journal of Research on Educational Effectiveness, 15(1), 28–51. https://doi.org/10.1080/19345747.2021.1969710
*Troussas, C., Krouska, A., & Sgouropoulou, C. (2021). A novel teaching strategy through adaptive learning activities for computer programming. IEEE Transactions on Education, 64(2), 103–109. https://doi.org/10.1109/TE.2020.3012744
*Tüfekçi, A., & Köse, U. (2013). Development of an artificial intelligence based software system on teaching computer programming and evaluation of the system. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 28(2), 469–481.
*Verner, I. M., Cuperman, D., Gamer, S., & Polishuk, A. (2020). Exploring affordances of robot manipulators in an introductory engineering course. International Journal of Engineering Education, 36(5), 1691–1707.
Von Bertalanffy, L. (1950). An outline of general system theory. British Journal for the Philosophy of Science, 1, 134–165. https://doi.org/10.1093/bjps/I.2.134
Von Bertalanffy, L. (1968). General system theory: Foundations, development, applications. George Braziller.
*Vyas, V. S., Kemp, B., & Reid, S. A. (2021). Zeroing in on the best early-course metrics to identify at-risk students in general chemistry: An adaptive learning pre-assessment vs. traditional diagnostic exam. International Journal of Science Education, 43(4), 552–569. https://doi.org/10.1080/09500693.2021.1874071
Walker, E., Rummel, N., & Koedinger, K. R. (2014). Adaptive intelligent support to improve peer tutoring in algebra. International Journal of Artificial Intelligence in Education, 24(1), 33–61. https://doi.org/10.1007/s40593-013-0001-9
*Wang, T., Su, X., Ma, P., Wang, Y., & Wang, K. (2011). Ability-training-oriented automated assessment in introductory programming course. Computers & Education, 56(1), 220–226. https://doi.org/10.1016/j.compedu.2010.08.003
*Wang, X. (2016). Course-taking patterns of community college students beginning in STEM: Using data mining techniques to reveal viable STEM transfer pathways. Research in Higher Education, 57(5), 544–569. https://doi.org/10.1007/s11162-015-9397-4
Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering. In C. Wohlin (Ed.), Proceedings of the 18th international conference on evaluation and assessment in software engineering (pp. 1–10). ACM Press. https://doi.org/10.1145/2601248.2601268
*Wu, P., Hwang, G., & Tsai, W. (2013). An expert system-based context-aware ubiquitous learning approach for conducting science learning activities. Journal of Educational Technology & Society, 16(4), 217–230.
*Xing, W., Pei, B., Li, S., Chen, G., & Xie, C. (2019). Using learning analytics to support students’ engineering design: The angle of prediction. Interactive Learning Environments. https://doi.org/10.1080/10494820.2019.1680391
Xu, W., & Ouyang, F. (2022). A systematic review of AI role in the educational system based on a proposed conceptual framework. Education and Information Technologies, 27, 4195–4223. https://doi.org/10.1007/s10639-021-10774-y
*Yahya, A. A., & Osman, A. (2019). A data-mining-based approach to informed decision-making in engineering education. Computer Applications in Engineering Education, 27(6), 1402–1418. https://doi.org/10.1002/cae.22158
*Yang, J., Devore, S., Hewagallage, D., Miller, P., Ryan, Q. X., & Stewart, J. (2020). Using machine learning to identify the most at-risk students in physics classes. Physical Review Physics Education Research, 16(2), 20130. https://doi.org/10.1103/PhysRevPhysEducRes.16.020130
Yang, J., & Zhang, B. (2019). Artificial intelligence in intelligent tutoring robots: A systematic review and design guidelines. Applied Sciences, 9(10), 2078. https://doi.org/10.3390/app9102078
*Yannier, N., Hudson, S. E., & Koedinger, K. R. (2020). Active learning is about more than hands-on: A mixed-reality AI system to support STEM education. International Journal of Artificial Intelligence in Education, 30(1), 74–96. https://doi.org/10.1007/s40593-020-00194-3
*Yu, Y. (2017). Teaching with a dual-channel classroom feedback system in the digital classroom environment. IEEE Transactions on Learning Technologies, 10(3), 391–402. https://doi.org/10.1109/TLT.2016.2598167
*Zabriskie, C., Yang, J., Devore, S., & Stewart, J. (2019). Using machine learning to predict physics course outcomes. Physical Review Physics Education Research, 15(2), 20120. https://doi.org/10.1103/PhysRevPhysEducRes.15.020120
*Zampirolli, F. A., BorovinaJosko, J. M., Venero, M. L. F., Kobayashi, G., Fraga, F. J., Goya, D., & Savegnago, H. R. (2021). An experience of automated assessment in a large-scale introduction programming course. Computer Applications in Engineering Education, 29(5), 1284–1299. https://doi.org/10.1002/cae.22385
*Zapata-Cáceres, M., & Martin-Barroso, E. (2021). Applying game learning analytics to a voluntary video game: Intrinsic motivation, persistence, and rewards in learning to program at an early age. IEEE Access, 9, 123588–123602. https://doi.org/10.1109/ACCESS.2021.3110475
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education—Where are the educators? International Journal of Educational Technology in Higher Education, 16(39), 1–27. https://doi.org/10.1186/s41239-019-0171-0
*Zhang, Z., Liu, H., Shu, J., Nie, H., & Xiong, N. (2020). On automatic recommender algorithm with regularized convolutional neural network and IR technology in the self-regulated learning process. Infrared Physics and Technology, 105, 103211. https://doi.org/10.1016/j.infrared.2020.103211
Zheng, R., Jiang, F., & Shen, R. (2020). Intelligent student behavior analysis system for real classrooms. In P. C. Center (Ed.), 2020 IEEE international conference on acoustics, speech and signal processing (pp. 9244–9248). IEEE.
*Zulfiani, Z., Suwarna, I. P., & Miranto, S. (2018). Science education adaptive learning system as a computer-based science learning with learning style variations. Journal of Baltic Science Education, 17(4), 711–727. https://doi.org/10.33225/jbse/18.17.711
Zupic, I., & Čater, T. (2015). Bibliometric methods in management and organization. Organizational Research Methods, 18(3), 429–472. https://doi.org/10.1177/1094428114562629
The authors would like to thank Luyi Zheng for her help on preliminary data analysis.
This work was supported by National Natural Science Foundation of China, No. 62177041.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Xu, W., Ouyang, F. The application of AI technologies in STEM education: a systematic review from 2011 to 2021. IJ STEM Ed 9, 59 (2022). https://doi.org/10.1186/s40594-022-00377-5