- Review
- Open access
- Published:
Effects of digital game-based STEM education on students’ learning achievement: a meta-analysis
International Journal of STEM Education volume 9, Article number: 26 (2022)
Abstract
Many researchers have explored the impact of digital games on learning effects in different STEM subjects. The purpose of this meta-analysis is to examine the effect of digital game-based STEM education on the learning achievement of K-12 or higher education students. The analysis results of effect sizes from 33 studies (N = 3894) published from 2010 to 2020 showed that digital games contributed to a moderate overall effect size (ES = 0.667, 95% CI [0.520–0.814], p < 0.001) when compared with other instructional methods. Furthermore, the study explored multiple moderator variables and their potential impacts on learning outcomes such as control treatment, subject discipline, educational level, game type, gaming platform, and intervention duration. The findings suggest that digital games are a promising pedagogical method in STEM education that effectively improves learning gains. Additionally, the study concludes with three recommendations for future research and practices on digital games in STEM education.
Introduction
The growing importance of STEM education is recognized globally, but there are multiple perspectives on its meaning. STEM education can be viewed from a broad perspective, including all STEM disciplines, such as science, mathematics, technology, engineering, and cross-disciplinary combinations of the different STEM disciplines. On the other hand, STEM education also refers to interdisciplinary combinations of the individual STEM subjects (Li, 2014; Li et al., 2020). In our current study, STEM education refers specifically to the STEM subjects of science, mathematics, technology, and engineering. However, learning these disciplines has been considered to have various difficulties and challenges due to the subject's complex, abstract, and multi-dimensional nature (Corredor et al., 2014; Sedig, 2008). Digital games are considered to have a profound potential to meet these challenges and positively impact students' learning gains and attitudes. “Games” perform an essential function of promoting cognitive development (Piaget, 1999), are an immersive, enjoyable, and exciting activity (Papastergiou, 2009), and are widely used in educational contexts (Chu & Chang, 2014; Gunter et al., 2008). The National Science Foundation proposes that STEM learning games have become a new way of learning in K-12 education and are suitable tools for teaching STEM disciplines (National Science Foundation [NSF], 2008). Furthermore, researchers have noted that digital games can achieve diversified STEM learning goals that enhance students' learning motivation, improve their understanding of knowledge concepts, and cultivate their problem-solving abilities (Hwang et al., 2012).
Several studies on the effectiveness of digital games in terms of learning outcomes have been published. However, these studies have reached no uniform conclusions regarding the effects of digital games on students' STEM learning gains. Some studies show that digital games play a significant role in enhancing students' learning performance. For example, Hung et al. (2014) developed a mathematical digital game to help children reduce their anxiety, and indicated that it improved their motivation to learn mathematics and their achievement in the subject. Studies conducted by Chu and Chang (2014), Hwang et al. (2013); (2016) supported this conclusion. Other studies, however, have reported a negative impact on students' STEM learning. For instance, by setting up reviewing as usual in the control group and playing games in the experimental group, Neimeyer (2006) found that educational games had a negative impact on mathematics achievement, as did Ferguson's (2014) study. Additionally, no significant effect on student learning was determined by other researchers (Giannakos, 2013; Khan et al., 2017). The results presented in the above studies are disparate and make it difficult for educators to decide whether to use digital games in STEM course teaching. Therefore, evaluating whether digital games have a positive impact on science, technology, engineering, and mathematics (STEM) education is necessary.
Summary of previous literature reviews
Based on a search of the literature, we found three meta-analyses (Riopel et al., 2019; Tokac et al., 2019; Tsai & Tsai, 2020) and two systematic reviews (Gao et al., 2020; Li & Tsai, 2020) on the application of digital games in STEM subjects. Their analysis methods and research content were quite diverse. Riopel et al. (2019) conducted a meta-analysis of 79 studies on serious games (i.e., digital software explicitly designed for learning purposes) in science education. They reported that knowledge construction and internalization (e.g., declarative and procedural knowledge, knowledge retention) was slightly higher for students. In addition, five moderators' variables (grade level, duration of intervention, level of user control, year of publication, and publication status) showed a significant impact on science learning achievement.
Tokac et al. (2019) analyzed the effects of video games on students' mathematics achievement and found significantly more positive impacts on mathematics achievement than traditional instructional methods. They also studied the moderation effects of grade level, instrument type, length of game-based intervention, country, publication type, and study year characteristics on learning achievement. Tsai and Tsai (2020) examined digital game-based science learning effectiveness. They located 26 research articles from 2000 to 2018 for a meta-analysis and found that gameplay design had a medium effect size, whereas mechanism design had a small-to-medium effect size. Tsai and Tsai also found that students at different educational levels significantly benefitted from game-based science learning. Gao et al. (2020) comprehensively analyzed 30 studies published between 2010 and 2019 on mobile games in STEM education. Their results supported mobile games in STEM education and further called for more research to apply mobile games. Li and Tsai (2020) systematically reviewed 31 studies on GBL in science education from 2000 to 2011. They summarized the fundamental theories supporting GBL and showed that most studies focused on promoting the understanding and learning of scientific knowledge and concepts rather than cultivating problem-solving skills.
Previous literature has provided many insightful conclusions about the effect of digital games on learning achievement in the subject discipline of STEM. However, the studies mentioned above also have their limitations. First, they did not comprehensively analyze the subject disciplines of STEM, but only analyzed a particular subject, such as science or mathematics. Second, they did not treat control treatment as a moderator variable for comparison, and therefore failed to accurately assess the moderator variables that may influence the effectiveness of digital game-based STEM education in terms of students' learning achievement.
Purpose of this meta-analysis
To solve the issue of the inconsistent results among the empirical studies on digital game-based STEM education, the primary purposes of this meta-analysis were to examine the overall effect size of using digital games to promote students' learning achievement in STEM education through integrating studies of different research designs and findings. After all, media comparison research is limited. Mayer (2019) suggested that media comparison research must establish adequate control groups and the potential for publication bias favoring significant media effects. Therefore, we performed a moderator analysis of control treatment, subject discipline, education level, gameplay (game type and gaming platform), and intervention duration to identify critical instructional design principles in the condition of digital game-based STEM Education on learning achievement. To achieve this study purpose, the key research questions that guided this study are as follows:
-
What is the overall effect of digital game-based STEM education on students' learning achievement?
-
Are the learning gains higher when using digital games to support STEM education as compared to non-digital game-based methods?
-
Does the subject discipline impact students' learning achievement in digital game-based learning settings?
-
Does the educational level influence students' learning achievement in digital game-based learning settings?
-
Do the gameplay designs (game types or gaming platforms) affect student achievement in digital game-based learning settings?
-
Does the intervention duration impact students' learning achievement in digital game-based learning settings?
Methods
A meta-analysis is a statistical analysis method for quantitative and comprehensive analysis of a large number of previous research results on a certain topic (Glass, 1976). A meta-analysis is considered a systematic study to answer specific questions or hypotheses. It has more stringent literature screening mechanisms and standards (Noble, 2006). It combines the collected multiple original research results (e.g., R, Mean) into a single effect quantity or effect scale to obtain the comprehensive effect of multiple independent studies, and may better ensure the rigor and effectiveness of the research conclusions. It may allow researchers to resolve disputes arising from conflicting empirical studies and draw more meaningful inferences (Paré et al., 2015).
Data sources and search strategy
Studies were searched for using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) method (Moher et al., 2010, 2015). A total of six web databases were consulted, including the Web of Science Core Collection (WoS), Educational Resources Information Center (ERIC), ProQuest, Springer, Scopus, and Wiley. Eligible studies were published from 2010 to 2020. Simultaneously, these studies must be written in English and published in peer-reviewed journal articles. We referred to the search strategies used in previous meta-analyses (Bai et al., 2020; Gao et al., 2020; Hung et al., 2018; Tsai & Tsai, 2020). Two sets of search keywords were used in the review. The first set consisted of keywords referring to games, including “game”, “gaming”, “gameplay”, “educational game”, and “digital game”. The second set of search keywords contained keywords related to STEM: “science”, “technology”, “engineering”, and “mathematics”. These two sets of keywords were combined with the Boolean operators (AND, OR).
Inclusion and exclusion criteria
According to the following criteria, the research literature that met the meta-analysis's requirements needed to be included and excluded.
The following inclusion criteria were used in the meta-analysis:
-
1.
Studies published between January 2010 and December 2020.
-
2.
Studies that focused on K-12 or higher education settings.
-
3.
Studies that used an intervention research design whereby a group received a digital game treatment, and another group did not.
-
4.
Studies should provide sufficient data to calculate effect sizes.
The following exclusion criteria were used in the meta-analysis:
-
1.
Studies that were not published in English.
-
2.
Studies that had insufficient data to calculate effect sizes.
-
3.
Studies that had no control/comparison groups and did not involve digital games.
-
4.
Studies for which the full text was unavailable.
-
5.
Studies that did not focus on STEM courses.
Figure 1 shows the data collection process, including searching, screening, and selecting qualified articles for inclusion. The 58 studies were included for further consideration in the eligibility phase. We re-read the literature's full text based on the inclusion and exclusion criteria. Finally, a total of 33 studies satisfied the inclusion criteria in this meta-analysis. From these 33 studies, we extracted 36 effect sizes; one study (Su & Cheng, 2013b) included one group for two comparisons, and another study (Chang et al., 2015) compared three different grades, thus contributing five effect sizes.
Moderator variables
Moderator variables referred to characteristics of studies related to the studies' results. Different moderator variables had different results and could cause variance in effect size. The most common moderator variables were subject discipline, educational level, and intervention duration. Furthermore, control treatment, game type, and gaming platform were also considered as moderator variables in this study. These moderator variables were used in previous research to examine what could contribute to the heterogeneity of effect size differences (Chen et al., 2018; Hung et al., 2018; Thompson & Gillern, 2020; Zheng et al., 2016). In order to answer the research questions raised in this meta-analysis, we coded for the following moderator variables. All moderator information of included studies is provided in Table 1 (Additional file 1).
Control treatment
The control treatment analysis allows us to determine whether digital game-based instruction is more conducive to promoting learning than other non-digital game-based methods. Previous studies considered “control treatment” as a moderator variable to compare the experimental treatments with the different control treatments (Garzón & Acevedo, 2019; Merchant et al., 2014; Sitzmann, 2011; Wouters et al., 2013). The two coding categories for control treatment in our meta-analysis were “traditional” and “multimedia”. Traditional instruction was assigned to curricula, traditional teacher introductions, textbooks, physical laboratories, and other resources. Studies that used videos, images, animation, or computer-assisted instruction were classified as multimedia. This category also included software, mobile devices, or web-based resources.
Subject discipline
The definition of “subject” was the name of a discipline or a class in which the STEM enactment occurred (Wahono et al., 2020). Referring to the classification by Wahono et al. (2020), we also coded the studies into three categories: science, mathematics, and technology or engineering. We hope to guide future development by analyzing the impact of digital game-based introduction in different subject disciplines of STEM.
Education level
Students with different knowledge levels would affect the experiment's results, leading to effect size heterogeneity (Fu et al., 2011). Education level is a standard moderator variable in a meta-analysis. It is crucial for educators and developers to identify the educational levels that benefit most from digital games. The education levels were divided into three groups, particularly primary education, secondary education, and higher education, aligning with standard divisions at the school level.
Game type
In the existing literature review, Li and Tsai (2013) divided the games into two types: without a role-play mechanism and with a role-play mechanism. Hung et al. (2018) identified eight game categories, namely immersive games, tutorial games, exer-games, simulation games, adventure games, music games, board games, and alternate reality games. This study adapted the classification framework for game types of Hung et al. (2018) by dividing game types into immersive, tutorial, and board games.
Gaming platform
Computers are considered the most common platforms for gameplay, followed by mobile devices (Hung et al., 2018). In addition, Thompson and Gillern (2020) established that different hardware types might impact how people learn through games. It was also divided into computers, mobile devices, video game consoles (e.g., PlayStation or Xbox), and unspecified devices. The study categorized game platforms as computers and mobile devices based on included articles.
Intervention duration
Based on previous studies (Bai et al., 2020; Chen et al., 2018), this study's intervention durations were coded as one of the following: (a) < 1 week, (b) 1 week-1 month, (c) 1 month-3 months, (d) ≥ 3 months, and (e) Not specified.
Data analysis
We synthesized the effect size and analyzed the moderator variables using the Comprehensive Meta-Analysis 3.0 software. Due to the goal of this study being to examine the effect size of digital game-based instruction in non-digital game-based instruction, the ES for this study was expressed as the standardized mean difference. Homogeneity analysis was computed with the Q statistic and the I2 value to identify homogeneity across studies. A significant Q statistic rejected the null hypothesis of homogeneity and indicated heterogeneity (Lipsey & Wilson, 2001). Thus, the random-effects model was more appropriate (Borenstein et al., 2010), and showed that analysis of the moderator variables was necessary. The ES(d) was calculated by using the following formula (Hedges, 1982):
where \(M_{E}\) and \(M_{C}\) are the estimated means of the experimental and control groups, respectively, with \(N_{E}\) and \(N_{C}\) being the sample sizes of both groups, and \(S_{E}^{2}\) and \(S_{C}^{2}\) the respective standard deviations.
Results and discussion
Analyses of publication bias and heterogeneity
Publication bias from multiple sources may affect the results of meta-analysis studies (Egger et al., 1997). When only positive study results are published, it leads to publication bias (Borenstein et al., 2009). Funnel plots may be useful to assess the validity of meta-analyses. If there is a publication bias in meta-analysis, the funnel plot would be an asymmetrical funnel that both bias and true heterogeneity of potential effects were the reasons for this phenomenon (Egger et al., 1997). Conversely, if the funnel plot is an asymmetrical inverted funnel, it suggests no publication bias. As per Fig. 2, the funnel plot is symmetrical, and most of the studies are in the middle and upper part of the funnel plot, suggesting that there is no publication bias. In the Begg and Mazumdar rank correlation (Kendall's Tau with a continuity correction), the results (Z = 1.457 < 1.96, p = 0.145 > 0.05) indicate that there is insignificant publication bias (Begg & Mazumdar, 1994). We compute the classic fail-safe N test to further confirm that there was no publication bias in the current sample. The classic fail-safe N test is a procedure to evaluate whether publication bias can be ignored (Rosenthal, 1979). If the fail-safe value is larger than Rosenthal's (1979) formula: 5k + 10, where k is the number of effect sizes included in the meta-analysis, it explains that there is no publication bias. The fail-safe value for this meta-analysis was computed to be 3001, which is more than 5(36) + 10. There would need to be a large number of nonsignificant unpublished studies for the effect sizes to be statistically insignificant. Based on this, we conclude the absence of publication bias.
The presence of heterogeneity is examined by using the I2 values. The I2 test supplements the Q-test, where 0%–25% indicates that heterogeneity is considered low, 25%–75% indicates moderate heterogeneity, and 75%–100% indicates substantial heterogeneity (Higgins et al., 2003). The larger the I2 value, the greater the heterogeneity. The statistics (Q = 156.856, I2 = 77.687, p < 0.001) show the presence of statistical heterogeneity. When there is significant heterogeneity, using a random-effects model would better address the differences between research effect sizes (Wang et al., 2019). Therefore, we adopted a random-effects model for the data analysis.
What is the overall effect of digital game-based STEM education on students' learning achievement?
To answer this question, 33 studies with 36 effect sizes and 3894 participants were examined by using a meta-analysis approach. Under the random-effects model, this study reported an overall significant positive effect size (ES = 0.667, 95%CI [0.520–0.814], p < 0.001). According to Cohen (1988), when the effect size was less than 0.2, it indicated a small effect, while when it was between 0.2 and 0.8, it was a moderate effect, and more than 0.8 was classified as a large effect. The results demonstrated that compared to non-digital game learning activities, digital game-based learning had a moderately significant effect on students' STEM learning achievement. In other words, using digital games to improve students' academic performance could be one of the effective methods for STEM education. The forest plot of all included effect sizes in the random-effects model is shown in Fig. 3.
This study compared digital game-based instruction with non-digital game-based instruction, which is the main contribution to the field. There was a significant relationship between digital games and learning achievement, with a moderately positive effect, suggesting that digital game approaches outperformed non-digital games. However, one must be cautious when assuming that computer games are always the most effective form of computer-assisted learning (Mayer, 2019). Like other instructional approaches, educators should adequately consider goals for learning and methods of integrating digital games most effectively into the STEM classroom to promote students' learning achievements.
Possible moderators and analysis of their effects
In addition to knowing that digital games affect students' learning achievement, this study also needs to look for possible moderator variables that affect the effectiveness. Subject discipline, educational level, and intervention duration could contribute to the heterogeneity of effect size differences. Similarly, the game type and gaming platform analysis are essential to the effect of digital game-based STEM education on students' learning outcomes. Under the random-effects model, moderator analyses were performed on these moderator variables. The descriptive analysis of subgroups contributed to answering the research questions of this study. The results and discussion are as follows:
Are the learning gains higher when using the digital game to support STEM education as compared to non-digital game-based methods?
The meta-analysis of the control treatment variable demonstrates no statistically significant difference between the control group treatment of traditional and multimedia approaches (Qb = 3.506, p = 0.061). The effect size of digital game-based instruction compared to multimedia is 0.848 (p < 0.001). When comparing digital game-based instruction with traditional introduction, the effect size was 0.558 (p < 0.001). The results show that digital game-based instruction is more effective than other instruction strategies, indicating that intervention of digital games seems to improve student learning. Additionally, we compared the effect size found in this study with other meta-analyses on game-based STEM subjects. Tsai and Tsai (2020) analyzed the effectiveness of game-based science learning. They considered the gameplay design and game-mechanism design, and the effect sizes found were ES = 0.646 and ES = 0.270, respectively. Tokac et al. (2019) conducted a meta-analysis to measure the impact of learning video games on mathematics achievement compared with traditional methods. They found that mathematics video games contributed to learning compared with traditional methods. In summary, the results of our meta-analysis are consistent with previous meta-analyses, in that digital games were found to have a positive impact on STEM education compared with other methods.
Does the subject disciplines impact students' learning achievement?
For subject discipline, there is no significant difference between the studies for science, mathematics, and technology/engineering (Qb = 2.188, p = 0.335). Digital games have a positive effect on science, mathematics, and technology/engineering. The effect size of science (ES = 0.750, p < 0.001) is higher than that of mathematics (ES = 0.629, p < 0.001) and technology/engineering (ES = 0.367, p = 0.140). However, the effect size of technology/engineering showed no statistically significant difference. The results are consistent with Tsai and Tsai (2020) and Tokac et al. (2019).
Does the educational levels influence students' learning achievement?
Regarding education levels, results in Table 2 suggest no significant difference in the effect sizes for the different education levels (Qb = 5.184, p = 0.075). However, it is most beneficial for primary education (ES = 0.835, p < 0.001) to learn with the assistance of digital games, with results that were significantly better than those in secondary (ES = 0.487, p < 0.001) and higher education (ES = 0.492, p < 0.05). The results fully reflect the relationship of Piagetian theories between playing and cognitive development (Piaget, 1999). Primary school students are in the critical period of cognitive development. Interest is also a key factor. Students in primary education may not be able to fully master the game's rules and are easily attracted by the freshness of digital games. However, secondary and higher education students can understand the game's rules faster, resulting in decreased learning interest.
Do the gameplay designs (game types or gaming platforms) impact learning achievement?
As shown in Table 2, results show that the effect sizes significantly differ among the different game types for students' academic performance (Qb = 11.126, p < 0.01). The effect size of board games (ES = 1.455, p < 0.001) significantly outperformed the other two game types. However, the research on board games (11.11%) was the least. The reason for this phenomenon is the small sample size. Additionally, the research of Chen et al. (2016) and Hwang et al. (2012) also affected the result. Immersive games (ES = 0.583, p < 0.001) and tutorial games (ES = 0.593, p < 0.001) also showed significant results. Reviewing previous literature shows that there has been little exploration of game types. Lamb et al. (2018) found that different game types increased students' achievement, cognition, and affect. In our meta-analysis, even if board games have a more significant impact than the other three game types, only four studies on board games were included in this meta-analysis. Nevertheless, in follow-up research on digital games, it is necessary to compare further the effects of different game types on students' learning achievement in STEM education.
The results of the subgroup analyses of gaming platforms summarize that using computers (ES = 0.625, p < 0.001) and mobile devices (ES = 0.768, p < 0.001) both significantly enhance students' learning, and there is no significant difference between the two subgroups’ effect sizes (Qb = 0.752, p = 0.386). This means that the different gaming platforms tend to have the same learning effectiveness. Gaming platforms are also of interest to education researchers. Different game platforms (e.g., computers and mobile devices) have their own characteristics and limitations, affecting learners' interactions with games (Thompson & Gillern, 2020). Previous literature reviews and meta-analyses have examined the field of gaming platforms (Hung et al., 2018; Thompson & Gillern, 2020). They found that computers were considered the most common gaming platform. Thompson and Gillern (2020) reported a more significant effect size for console-based games in English vocabulary learning than computers and mobile devices. Our results are consistent with these findings. They showed that computers were the most commonly used gaming platforms. In addition, given the rapid increase in mobile technology use (Sung et al., 2016), mobile games have become more popular. It is necessary to further study digital games played on mobile devices and to compare the impact of different game platforms on students' learning achievement. Finally, considering that each game platform can promote academic performance, educators should choose the appropriate game platform according to the learning goals and students' characteristics.
Does the intervention duration impact students' learning achievement?
The effect size of intervention duration was also computed. Results shown in Table 2 suggest that the intervention duration of less than one week has the largest effect size (ES = 0.953, p < 0.001). Consistent with previous discussions, short-term interventions have been associated with better learning achievement than long-term interventions (Riopel et al., 2019). The main reason for this is that the possibility of novelty effects gradually decreases. Learners are often excited about the use of digital games in short-term interventions, which leads to a high degree of curriculum activity participation. However, learners are only curious about new learning methods in the short term. Over time, they experience boredom due to the disappearance of the novelty effect, leading to a reduced desire to use the method. The second largest effect size was for 1 week to 1 month (ES = 0.464, p < 0.05), followed by 1 month to 3 months (ES = 0.579, p < 0.001), greater than or equal to 3 months (ES = 0.597, p < 0.001), and Not specified (ES = 0.676, p < 0.01). Moreover, the results indicate no significant differences among different intervention durations (Qb = 5.070, p = 0.280), which means that digital games have positive effects on learning achievement for different intervention durations.
Conclusions
The results of this study suggest that digital games can effectively promote and enhance students' learning achievement in STEM education, enhancing our understanding of the application and practice of digital games in STEM education. We also examined the different moderator variables that may affect the effect sizes of digital games. Based on the findings in this study, three recommendations for future research on digital games in STEM education are proposed.
First, future research should strengthen the gameplay design and game mechanisms. It should also examine the effects of different types of digital games on student learning in STEM education. Most knowledge content in STEM education involves abstract and multi-dimensional concepts. These are often difficult for students to understand and cause them to lose learning motivation quickly. This in turn hinders their internalization and construction of knowledge. It develops their low academic performance and negative attitudes and may cause students to drop out of courses (Anderson & Barnett, 2011; Khan et al., 2017). Studies by Bai et al. (2020), Chen et al. (2020b), Chen, Huang, et al. (2020a)), and Tsai and Tsai (2020) showed that gameplay and game mechanisms, competition strategies, and gaming platforms significantly increase students' learning achievement, consistent with our research results. Different game types and platforms have promoted students' learning achievement improvement, but educational researchers need to explore the internal mechanism of digital games further to better understand their effects.
Second, future work should carefully examine the influence of the learner’s types and characteristics on their interest in digital games. Brinson (2015) believed that it was crucial to better understand the effectiveness of serious games related to students' grade levels and cognitive or psychological development. In our research, the learning effectiveness of digital games in elementary schools significantly outperformed that in secondary schools and in higher education. Similarly, Riopel et al. (2019) reported that high school students achieved significantly higher science learning gains than older college students and adults, consistent with Wouters et al. (2013). Additionally, personality traits significantly impact the game learning system's design (Jia et al., 2016). Therefore, in the following work, personalized digital games can be further designed according to learner’s types and characteristics to better meet different learners' preferences.
Third, future research should integrate digital games with other technologies to advance the sustainable development of digital games in educational applications. The growth of the Internet and various emerging technologies (e.g., mobile technology, AR/VR, and 3D) provides students with personalized learning support opportunities in their education (Hou et al., 2014; Hwang et al., 2016; Su & Cheng, 2013a). According to Hwang et al. (2016), an AR-based gaming method could improve students' learning attitudes and learning outcomes. Through the implementation of quasi-experiments, Su and Cheng (2013a) found that when compared to traditional teaching, a 3D GBL system with a software engineering curriculum could achieve better learning achievement and motivation through quasi-experiment implementation. Consequently, future research should make full use of various information resources and integrate emerging technology to develop a sense of immersion. The substitution of digital games helps improve students' learning experiences and mobilizes their learning enthusiasm, while also stimulating their learning motivation. Furthermore, extending digital games from computers to mobile devices is necessary. It is beneficial to realize ubiquitous and lifelong learning and to promote digital games' sustainable development in teaching practice applications.
Limitations
While this study highlights digital games as an effective method for promoting students' learning achievement in STEM education compared to non-digital games, a few limitations exist for this meta-analytic research. First, due to the intention of the meta-analysis method, our analysis excludes many empirical studies that have significant value but do not meet the requirements. This study only involves 33 empirical studies and 36 effect sizes. We believe that there may be relevant studies that have not been found. Second, the data analysis uses a random-effects model rather than a more precise fixed-effects model. It will not be comprehensive if we attempt to cover a wide range of internal and external moderator variables. Finally, the current review is not an attempt to be inclusive but rather to provide a systematic overview of digital games in STEM education. There is a paucity of reports on other aspects of DGBL, such as cognitive skills and affective influences. Therefore, it is suggested that some follow-up studies can be conducted to investigate the effects of digital game-based STEM learning from diverse perspectives by taking into account those relevant studies published in the enormous number of academic databases.
Availability of data and materials
Not applicable.
Abbreviations
- CMA:
-
Comprehensive meta-analysis software
- DGBL:
-
Digital game-based learning
- ERIC:
-
Educational Resources Information Center
- GBL:
-
Game-based learning
- NSF:
-
National Science Foundation
- PRISMA:
-
Preferred Reporting Items for Systematic Reviews and Meta-Analysis
- STEM:
-
Science, technology, engineering, and mathematics
- WoS:
-
Web of Science
References
Anderson, J. L., & Barnett, M. (2011). Using video games to support pre-service elementary teachers learning of basic physics principles. Journal of Science Education and Technology, 20(4), 347–362. https://doi.org/10.1007/s10956-010-9257-0
Anderson, J. L., & Barnett, M. (2013). Learning physics with digital game simulations in middle school science. Journal of Science Education and Technology, 22(6), 914–926. https://doi.org/10.1007/s10956-013-9438-8
Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, 100322. https://doi.org/10.1016/j.edurev.2020.100322
Bai, H., Pan, W., Hirumi, A., & Kebritchi, M. (2012). Assessing the effectiveness of a 3-D instructional game on improving mathematics achievement and motivation of middle school students. British Journal of Educational Technology, 43(6), 993–1003. https://doi.org/10.1111/j.1467-8535.2011.01269.x
Begg, C. B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50(4), 1088–1101. https://doi.org/10.2307/2533446
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: Wiley.
Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12
Brinson, J. R. (2015). Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: a review of the empirical research. Computers & Education, 87, 218–237. https://doi.org/10.1016/j.compedu.2015.07.003
Chang, M., Evans, M. A., Kim, S., Norton, A., & Samur, Y. (2015). Differential effects of learning games on mathematics proficiency. Educational Media International, 52(1), 47–57. https://doi.org/10.1080/09523987.2015.1005427
Chen, C. H. (2020). Impacts of augmented reality and a digital game on students’ science learning with reflection prompts in multimedia learning. Educational Technology Research and Development, 68(6), 3057–3076. https://doi.org/10.1007/s11423-020-09834-w
Chen, C. Y., Huang, H. J., Lien, C. J., & Lu, Y. L. (2020a). Effects of multi-genre digital game-based instruction on students’ conceptual understanding, argumentation skills, and learning experiences. IEEE Access, 8, 110643–110655. https://doi.org/10.1109/ACCESS.2020.3000659
Chen, C. H., Liu, G. Z., & Hwang, G. J. (2016). Interaction between gaming and multistage guiding strategies on students’ field trip mobile learning performance and motivation. British Journal of Educational Technolog, 47(6), 1032–1050. https://doi.org/10.1111/bjet.12270
Chen, C. H., Shih, C. C., & Law, V. (2020b). The effects of competition in digital game-based learning (DGBL): A meta-analysis. Educational Technology Research and Development, 68(4), 1855–1873. https://doi.org/10.1007/s11423-020-09794-1
Chen, J., Wang, M., Kirschner, P. A., & Tsai, C. C. (2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: a meta-analysis. Review of Educational Research, 88(6), 799–843. https://doi.org/10.3102/0034654318791584
Cheng, M. T., Su, T. F., Huang, W. Y., & Chen, J. H. (2014). An educational game for learning human immunology: What do students learn and how do they perceive? British Journal of Educational Technology, 45(5), 820–833. https://doi.org/10.1111/bjet.12098
Chu, H. C., & Chang, S. C. (2014). Developing an educational computer game for migratory bird identification based on a two-tier test approach. Educational Technology Research & Development, 62(2), 147–161. https://doi.org/10.1007/s11423-013-9323-4
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates.
Corredor, J., Gaydos, M., & Squire, K. (2014). Seeing change in time: video games to teach about temporal change in scientific phenomena. Journal of Science Education and Technology, 23(3), 324–343. https://doi.org/10.1007/s10956-013-9466-4
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ British Medical Journal, 315, 629–634. https://doi.org/10.1136/bmj.315.7109.629
Ferguson, T. L. K. (2014). Mathematics Achievement with Digital Game-Based Learning in High School Algebra 1 Classes. (Order No. 3613196). Education Database. (1509130712). https://www.proquest.com/dissertations-theses/mathematics-achievement-with-digital-game-based/docview/1509130712/se-2?accountid=47934.
Frauke, V., Segers, E., Takashima, A., & Verhoeven, L. (2017). Effects of a tablet game intervention on simple addition and subtraction fluency in first graders. Computers in Human Behavior, 72, 200–207. https://doi.org/10.1016/j.chb.2017.02.031
Fu, R., Gartlehner, G., Grant, M., Shamliyan, T., Sedrakyan, A., Wilt, T. J., Griffith, L., Oremus, M., Raina, P., Ismaila, A., Santaguida, P., Lau, J., & Trikalinos, T. A. (2011). Conducting quantitative synthesis when comparing medical interventions: AHRQ and the Effective Health Care Program. Journal of Clinical Epidemiology, 64(11), 1187–1197. https://doi.org/10.1016/j.jclinepi.2010.08.010
Gao, F., Li, L., & Sun, Y. (2020). A systematic review of mobile game-based learning in STEM education. Educational Technology Research and Development, 68(4), 1791–1827. https://doi.org/10.1007/s11423-020-09787-0
Garzón, J., & Acevedo, J. (2019). Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educational Research Review, 27, 244–260. https://doi.org/10.1016/j.edurev.2019.04.001
Giannakos, M. N. (2013). Enjoy and learn with educational games: Examining factors affecting learning performance. Computers & Education, 68, 429–439. https://doi.org/10.1016/j.compedu.2013.06.005
Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8. https://doi.org/10.3102/0013189X005010003
Gunter, G. A., Kenny, R. F., & Vick, E. H. (2008). Taking educational games seriously: Using the RETAIN model to design endogenous fantasy into standalone educational games. Educational Technology Research and Development, 56(5–6), 511–537. https://doi.org/10.1007/s11423-007-9073-2
Hedges, L. V. (1982). Estimation of effect size from a series of independent experiments. Psychological Bulletin, 92(2), 490–499. https://doi.org/10.1037/0033-2909.92.2.490
Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. BMJ British Medical Journal, 327(7414), 557–560. https://doi.org/10.1136/bmj.327.7414.557
Hodges, G. W., Flanagan, K., Lee, J., Cohen, A., Krishnan, S., & Ward, C. (2020). A quasi-experimental study comparing learning gains associated with serious educational gameplay and hands-on science in elementary classrooms. Journal of Research in Science Teaching, 57(9), 1460–1489. https://doi.org/10.1002/tea.21661
Hou, H. T., Wu, S. Y., Lin, P. C., Sung, Y. T., Lin, J. W., & Chang, K. E. (2014). A blended mobile learning environment for museum learning. Journal of Educational Technology & Society, 17(2), 207–218.
Hung, C. M., Huang, I., & Hwang, G. J. (2014). Effects of digital game-based learning on students’ self-efficacy, motivation, anxiety, and achievements in learning mathematics. Journal of Computers in Education, 1(2–3), 151–166. https://doi.org/10.1007/s40692-014-0008-8
Hung, H. T., Yang, J. C., Hwang, G. J., Chu, H. C., & Wang, C. C. (2018). A scoping review of research on digital game-based language learning. Computers & Education, 126, 89–104. https://doi.org/10.1016/j.compedu.2018.07.001
Hwang, G. J., Sung, H. Y., Hung, C. M., Yang, L. H., & Huang, I. (2013). A knowledge engineering approach to developing educational computer games for improving students’ differentiating knowledge. British Journal of Educational Technology, 44(2), 183–196. https://doi.org/10.1111/j.1467-8535.2012.01285.x
Hwang, G. J., Wu, P. H., & Chen, C. C. (2012). An online game approach for improving students’ learning performance in web-based problem-solving activities. Computers & Education, 59(4), 1246–1256. https://doi.org/10.1016/j.compedu.2012.05.009
Hwang, G. J., Wu, P. H., Chen, C. C., & Tu, N. T. (2016). Effects of an augmented reality-based educational game on students’ learning achievements and attitudes in real-world observations. Interactive Learning Environments, 24(8), 1895–1906. https://doi.org/10.1080/10494820.2015.1057747
Jia, Y., Xu, B., Karanam, Y., & Voida, S. (2016). Personality-targeted Gamification: A Survey Study on Personality Traits and Motivational Affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. https://doi.org/10.1145/2858036.2858515.
Ke, F. (2019). Mathematical problem solving and learning in an architecture-themed epistemic game. Educational Technology Research and Development, 67(5), 1085–1104. https://doi.org/10.1007/s11423-018-09643-2
Kebritchi, M., Hirumi, A., & Bai, H. (2010). The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & Education, 55(2), 427–443. https://doi.org/10.1016/j.compedu.2010.02.007
Khan, A., Ahmad, F., & Malik, M. M. (2017). Use of digital game based learning and gamification in secondary school science: The effect on student engagement, learning and gender difference. Education & Information Technologies, 22(6), 2767–2804. https://doi.org/10.1007/s10639-017-9622-1
Kim, H., & Ke, F. (2017). Effects of game-based learning in an OpenSim-supported virtual environment on mathematical performance. Interactive Learning Environments, 25(4), 543–557. https://doi.org/10.1080/10494820.2016.1167744
Lamb, R. L., Annetta, L., Firestone, J., & Etopio, E. (2018). A meta-analysis with examination of moderators of student cognition, affect, and learning outcomes while using serious educational games, serious games, and simulations. Computers in Human Behavior, 80, 158–167. https://doi.org/10.1016/j.chb.2017.10.040
Li, Y. (2014). International journal of STEM education - a platform to promote STEM education and research worldwide. International Journal of STEM Education, 1, 1. https://doi.org/10.1186/2196-7822-1-1
Li, M. C., & Tsai, C. C. (2013). Game-based learning in science education: A review of relevant research. Journal of Science Education and Technology, 22(6), 877–898. https://doi.org/10.1007/s10956-013-9436-x
Li, Y., Wang, K., Xiao, Y., & Froyd, J. E. (2020). Research and trends in STEM education: A systematic review of journal publications. International Journal of STEM Education, 7, 11. https://doi.org/10.1186/s40594-020-00207-6
Lin, C. H., Liu, E. Z., Chen, Y., Liou, P., Chang, M., Wu, C., & Yuan, S. (2013). Game-based remedial instruction in mastery learning for upper-primary school students. Educational Technology & Society, 16, 271–281.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. New York: Sage Publications Inc.
Liu, T. Y. (2016). Using educational games and simulation software in a computer science course: Learning achievements and student flow experiences. Interactive Learning Environments, 24(4), 724–744. https://doi.org/10.1080/10494820.2014.917109
Mayer, R. E. (2019). Computer games in education. Annual Review of Psychology, 70, 531–549. https://doi.org/10.1146/annurev-psych-010418-102744
McLaren, B. M., Adams, D. M., Mayer, R. E., & Forlizzi, J. (2017). A computer-based game that promotes mathematics learning more than a conventional approach. International Journal of Game-Based Learning (IJGBL), 7(1), 36–56. https://doi.org/10.4018/IJGBL.2017010103
Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014). Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education, 70, 29–40. https://doi.org/10.1016/j.compedu.2013.07.033
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., The PRISMA Group. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336–341. https://doi.org/10.1016/j.ijsu.2010.02.007
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4, 1. https://doi.org/10.1186/2046-4053-4-1
National Science Foundation [NSF]. (2008). Fostering Learning in the Networked World: The Cyberlearning Opportunity and Challenge. Document Number: NSF 08204. https://www.nsf.gov/pubs/2008/nsf08204/index.jsp. Retrieved August 11, 2008.
Neimeyer, S. A. (2006). An examination of the effects of computer-assisted educational games on student achievement. Master of Science thesis, University of Houston-Clear Lake. https://www.learntechlib.org/p/116207/.
Nejem, K. M., & Muhanna, W. (2013). The effect of using computer games in teaching mathematics on developing the number sense of fourth grade students. Educational Research and Reviews, 8(16), 1477–1482. https://doi.org/10.5897/ERR012.143
Noble, J. (2006). Meta-analysis: Methods, strengths, weaknesses, and political uses. Journal of Laboratory & Clinical Medicine, 147(1), 7–20. https://doi.org/10.1016/j.lab.2005.08.006
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12. https://doi.org/10.1016/j.compedu.2008.06.004
Paré, G., Trudel, M. C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A typology of literature reviews. Information & Management, 52(2), 183–199. https://doi.org/10.1016/j.im.2014.08.008
Piaget, J. (1999). Play, dreams and imitation in childhood. Routledge.
Riopel, M., Nenciovici, L., Potvin, P., Chastenay, P., Patrick, C., Sarrasin, J. B., & Masson, S. (2019). Impact of serious games on science learning achievement compared with more conventional instruction: An overview and a meta-analysis. Studies in Science Education, 55(2), 169–214. https://doi.org/10.1080/03057267.2019.1722420
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
Sedig, K. (2008). From play to thoughtful learning: A design strategy to engage children with mathematical representations. Journal of Computers in Mathematics and Science Teaching, 27(1), 65–101.
Shernoff, D. J., Ryu, J. C., Ruzek, E., Coller, B., & Prantil, V. (2020). The transportability of a game-based learning approach to undergraduate mechanical engineering education: effects on student conceptual understanding, engagement, and experience. Sustainability, 12(17), 6986. https://doi.org/10.3390/su12176986
Sitzmann, T. (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64(2), 489–528. https://doi.org/10.1111/j.1744-6570.2011.01190.x
Stege, L., Lankveld, G. V., & Spronck, P. (2012). Teaching high school physics with a serious game. International Journal of Computer Science in Sport, 11(1).
Su, C. H., & Cheng, C. H. (2013a). 3D game-based learning system for improving learning achievement in software engineering curriculum. Turkish Online Journal of Educational Technology, 12(2), 1–12.
Su, C. H., & Cheng, C. H. (2013b). A mobile game-based insect learning system for improving the learning achievements. Procedia-Social and Behavioral Sciences, 103, 42–50. https://doi.org/10.1016/j.sbspro.2013.10.305
Sung, Y. T., Chang, K. E., & Liu, T. C. (2016). The effects of integrating mobile devices with teaching and learning on students’ learning performance: A meta-analysis and research synthesis. Computers & Education, 94, 252–275. https://doi.org/10.1016/j.compedu.2015.11.008
Thompson, C. G., & Gillern, S. V. (2020). Video-game based instruction for vocabulary acquisition with English language learners: A Bayesian meta-analysis. Educational Research Review, 30, 100332. https://doi.org/10.1016/j.edurev.2020.100332
Tokac, U., Novak, E., & Thompson, C. G. (2019). Effects of game-based learning on students’ mathematics achievement: A meta-analysis. Journal of Computer Assisted Learning, 35(3), 407–420. https://doi.org/10.1111/jcal.12347
Topalli, D., & Cagiltay, N. E. (2018). Improving programming skills in engineering education through problem-based game projects with scratch. Computers & Education, 120, 64–74. https://doi.org/10.1016/j.compedu.2018.01.011
Tsai, Y. L., & Tsai, C. C. (2020). A meta-analysis of research on digital game-based science learning. Journal of Computer Assisted Learning, 36(3), 280–294. https://doi.org/10.1111/jcal.12430
Wahono, B., Lin, P. L., & Chang, C. Y. (2020). Evidence of STEM enactment effectiveness in Asian student learning outcomes. International Journal of STEM Education, 7, 36. https://doi.org/10.1186/s40594-020-00236-1
Wang, C. P., Lan, Y. J., Tseng, W. T., Lin, Y. T., & Gupta, C. L. (2019). On the effects of 3D virtual worlds in language learning—a meta-analysis. Computer Assisted Language Learning, 33(8), 891–915. https://doi.org/10.1080/09588221.2019.1598444
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311
Yallihep, M., & Kutlu, B. (2019). Mobile serious games: Effects on students’ understanding of programming concepts and attitudes towards information technology. Education and Information Technologies, 25(2), 1237–1254. https://doi.org/10.1007/s10639-019-10008-2
Yang, Y. T. C., & Chang, C. H. (2013). Empowering students through digital game authorship: Enhancing concentration, critical thinking, and academic achievement. Computers & Education, 68, 334–344. https://doi.org/10.1016/j.compedu.2013.05.023
Zeng, H., Zhou, S. N., Hong, G. R., Li, Q. Y., & Xu, S. Q. (2020). Evaluation of interactive game-based learning in physics domain. Journal of Baltic Science Education, 19(3), 484–498. https://doi.org/10.33225/jbse/20.19.484
Zhang, L., Shang, J., Pelton, T., & Pelton, L. F. (2020). Supporting primary students’ learning of fraction conceptual knowledge through digital games. Journal of Computer Assisted Learning, 36(4), 540–548. https://doi.org/10.1111/jcal.12422
Zheng, B., Warschauer, M., Lin, C. H., & Chang, C. (2016). Learning in one-to-one laptop environments: A meta-analysis and research synthesis. Review of Educational Research, 86(4), 1052–1084. https://doi.org/10.3102/0034654316628645
Acknowledgements
The authors would like to express the gratefulness to the three anonymous reviewers for providing their precious comments to refine this manuscript.
Funding
Philosophy and Social Science Planning of Zhejiang Province (Grant No. 19NDJC252YB); National Education Science “13th Five-Year” Planning Key Project of Ministry of Education in China (Grant No. DCA160262).
Author information
Authors and Affiliations
Contributions
LHW designed the research, interpreted the data and drafted the work; BC participated in the acquisition, analysis and interpretation of data and drafted the work; GJH gave an important advise and revised the conclusion; JQG and YQW took part in the acquisition and analysis of data. All authors read and approved the final manuscript.
Authors' information
Liang-Hui Wang is an associate professor at the Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University. His research interests include technology enhanced learning, learning analytics and STEM education.
Bing Chen is a graduate student at the Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University. His research interests include digital game-based learning and computational thinking.
Gwo-Jen Hwang is a chair professor at the Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology. His research interests include mobile learning, digital game-based learning, flipped classroom and AI in education.
Jue-Qi Guan is a lecturer at the Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University. Her research interests include technology enhanced learning and smart learning environment.
Yun-Qing Wang is a graduate student at the Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province, Zhejiang Normal University. Her research interests include STEM education and learning analytics.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1:
List of reviewed articles.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Wang, LH., Chen, B., Hwang, GJ. et al. Effects of digital game-based STEM education on students’ learning achievement: a meta-analysis. IJ STEM Ed 9, 26 (2022). https://doi.org/10.1186/s40594-022-00344-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s40594-022-00344-0