Enhancing programming learning performance through a Jigsaw collaborative learning method in a metaverse virtual space

Abstract

Background

Computational thinking (CT) is crucial to fostering critical thinking and problem-solving skills. Many elementary schools have been cultivating students’ CT through block-based programming languages such as Scratch using traditional teacher-centered teaching methods. However, the approach excessively relies on teacher lectures, so the teacher’s teaching load is hefty, and most students cannot receive timely assistance from teachers, thus reducing students’ programming learning performance, interest, and confidence. Therefore, this study designs a Jigsaw collaborative learning method implemented in a metaverse virtual space (JCLM-MVS) for peer-to-peer Scratch programming learning to promote learners’ programming learning performance, CT, and learning attitudes. This study used a quasi-experimental research method, with 48 fifth-grade students from two classes in Taiwan’s elementary school as the research participants. One class of 24 students was randomly assigned to the experimental group using JCLM-MVS to conduct Scratch programming learning, whereas the other class of 24 students was assigned to the control group using the traditional teacher-centered teaching method.

Main findings

The study found that the experimental group of learners showed significantly better Scratch programming learning performance and attitude than the control group, and there was no statistically significant difference in CT between both groups, but both learning approaches effectively promoted CT. Additionally, the interview results showed that most interviewees stated that using JCLM-MVS for Scratch programming learning could receive timely assistance from peers, make collaborative learning more efficient and learning more enjoyable, and more intend to use JCLM-MVS for Scratch programming learning than using traditional teacher-centered teaching method due to the effective collaborative interaction mechanisms and realistic learning space provided in the metaverse virtual space.

Conclusions/potential implications

This study presents a novel and engaging learning approach by integrating a metaverse virtual space with the Jigsaw collaborative learning method, referred to as JCLM-MVS, which enhances the effectiveness of the Jigsaw collaborative learning method in promoting Scratch programming learning performance, CT, and attitudes.

Introduction

STEM education, which encompasses the development of concepts, knowledge, and process understanding in Science, Technology, Engineering, and Mathematics, is considered an essential research domain in educational settings. It fosters important personal and professional skills and competencies, including research inquiry, problem-solving, critical thinking, computational thinking (CT), creative thinking, entrepreneurship, collaboration, teamwork, and communication (Falloon et al., 2020). It is worth noting that Eshach and Fried (2005) proposed six reasons to address why even young children should be exposed to science. They indicate that exposing small children to science can foster their natural curiosity, develop positive attitudes towards scientific inquiry, lay a foundation for better understanding scientific concepts later on, shape their use of scientifically informed language, enable them to comprehend and reason scientifically, and efficiently cultivate scientific thinking. Given the importance of STEM education, it has garnered significant attention and become a global education trend (Tytler, 2020). Among the important skills and competence considered in STEM education, CT is regarded as an ability that can be widely applied to everyday life situations and can facilitate critical thinking and problem-solving skills (Kaur & Chahal, 2023). It is no longer just a professional knowledge skill for computer engineers (Wing, 2006). In recent years, many scholars have recognized the importance of developing CT abilities and consider it as an essential area for education development, and incorporate it into K-12 education curricula (Angeli et al., 2016; Grover & Pea, 2013; Hsu et al., 2018; Voogt et al., 2015). In technology education, CT can be combined with many disciplines to achieve complementary learning effects. Many educators believe programming courses are the simplest and most effective way to cultivate CT abilities (Lye & Koh, 2014; Zhong et al., 2016). A systematic literature review conducted by Jin and Cutumisu (2024) indicated that block-based programming tools, such as Scratch, are the most common intervention used to cultivate students’ CT. This is particularly evident in secondary education, as noted by Marín-Marín et al. (2024). Resnick et al. (2009) also indicated that block-based programming languages, such as Scratch, allow beginners to intuitively design programs through drag-and-drop blocks without memorizing programming language syntax so that students can focus more on the problem-solving process. Therefore, most teachers believe that such block-based programming language is more readily accepted by students (Chen et al., 2017; Grover & Pea, 2013).

In Taiwan’s elementary school programming curriculum, the teaching design primarily relies on teacher-centered instruction, with lectures and demonstrations as the main approaches. Students learn individually and often rely on the teacher’s assistance without peer-to-peer interaction. Most students only learn programming through imitation or memorization. This fails to achieve the teaching goal of improving students’ CT ability through programming and creates a heavy burden and low sense of achievement for teachers. Hanks (2008) pointed out that among different teaching strategies, collaborative learning is considered an effective method to help students develop CT ability. Zhong et al. (2016) has shown that students who learn programming through collaborative learning perform significantly better than students who learn programming alone in completing programming tasks and enhancing CT performance. More importantly, collaboration plays a crucial role in fostering cognitive development, encompassing verbal and social skills during early childhood (Sills et al., 2016). Thus, it is essential to cultivate children’s collaborative learning skills to support cognitive development. However, in collaborative learning, the free-rider effect is the most common phenomenon (Levin, 2003). The free-rider effect refers to the phenomenon where group members believe that their effort is optional or unnecessary for the success of the team task, which leads to a decrease in their personal participation and contribution, causing uneven contributions among members and further lowering the effectiveness of the team (Kerr & Bruun, 1983). This phenomenon is mainly related to the uneven abilities of group members, unfamiliarity with the contribution of other team members, and inadequate personal team awareness (Chen et al., 2023).

The most significant difference between the Jigsaw method and other collaborative learning methods is that each group member is assigned a part of the instructional task, allowing each member to make an equal and meaningful contribution to the group (Yu, 2017). Johnson and Johnson’s (2009) research indicated that the Jigsaw method incorporates practical elements of collaborative learning, particularly active interdependence among members and individual accountability, to promote collaborative learning performance. In other words, the effectiveness of team-based collaborative learning can only be achieved when each team member accomplishes their own goals and supports each other. Therefore, in addition to being responsible for their own instructional tasks, each team member encourages other members to succeed. This positive learning model not only enhances individual learning performance, but also develops a strong sense of responsibility, as team members do not want to bear the responsibility for the team’s failure (Perihan & Kamuran, 2007). Therefore, this study aims to use the Jigsaw method to promote student awareness of the importance of their contributions to the group during small-group collaborative learning, thereby eliminating the problem of unequal contributions among members caused by free riding.

Recently, the metaverse has been recognized as the next generation of social connections and has a very high potential to be applied in educational settings (Hwang & Chien, 2022). Three features of the metaverse (Hwang & Chien, 2022; Lee et al., 2021; Wang et al., 2023) make it quite different from conventional virtual reality (VR) or augmented reality (AR): “shared”, “persistent”, and “de-centralized”. Those features make the metaverse similar to human living physical space (Hwang & Chien, 2022). The feature of “shared” refers to that users can engage in social activities such as discussing an issue, collaborating on a project, and collaboratively solving some problems; the feature of “persistent” refers to offering a persistent world enabling users to “live” in a virtual space to conduct such as working, owning, learning, interacting, creating, and entertaining; the feature of “de-centralized” refers to ensuring that economic activities can be safely conducted and that others will not falsify personal property and logs in the metaverse. To what extent can Gather Town, which allows users to construct a self-designed 2D persistent virtual environment and interact and share with others through their own virtual avatars using voice and video communication functions, be regarded as an online interaction and communication platform with some metaverse features (Hwang & Chien, 2022; Lee et al., 2021Ning et al., 2023). Therefore, if Gather Town’s virtual space can be used as a metaverse learning environment for primary school students to collaborate on programming learning, it can not only break through the limitations of time and space, making it more convenient for learners to collaborate and discuss, but also make the learning process more lively and exciting, thereby helping to enhance students’ interest and learning effectiveness. In other words, incorporating the Jigsaw method into a metaverse learning environment can motivate learners by enhancing collaboration, increasing engagement, and fostering diverse perspectives in a realistic environment without time and space limitations. However, Gather Town is a new online interaction and communication platform, and more research is urgently needed to verify its application in classroom teaching and collaborative learning.

According to the aforementioned research motivations, this study proposes a Jigsaw collaborative learning method implemented in a metaverse virtual space (JCLM-MVS) using Gather Town to assist elementary school students in learning Scratch programming. This study aims to cultivate a sense of responsibility, improve group awareness, and reduce the occurrence of a free-riding phenomenon through mutual teaching of expert groups and mutual discussion of learning groups ruled by the Jigsaw collaborative learning method to facilitate the learning effectiveness of Scratch programming and CT. Additionally, through the interactive discussions in a Gather Town metaverse virtual space, learner participation, collaboration, and engagement will be conducted in a more convenient way than face-to-face physical space, leading to possibly improved learning attitudes in terms of satisfaction, interest, practicality, collaboration, and confidence (Chang, 2009). The meta-analysis study conducted by Ozkan and Uslusoy (2024) to determine the effect of the Jigsaw technique in nursing education indicated that, owing to the characteristics of the digital age we are in, the Jigsaw learning method can also be effectively utilized in online environments and within the metaverse. To the best of our knowledge, no study has been conducted on implementing the Jigsaw learning method in a metaverse. The study addresses a research gap by proposing a novel approach, the JCLM-MVS, which combines the Jigsaw collaborative learning method with a metaverse virtual space, aiming to enhance learning outcomes in elementary school students, particularly in Scratch programming and CT, while also improving engagement and collaboration through interactive discussions in Gather Town. The research questions for this study are as follows:

• (1) Does using the proposed JCLM-MVS for Scratch programming learning among elementary school students result in significantly better learning performance compared to traditional teacher-led instruction?

• (2) Does using the proposed JCLM-MVS for Scratch programming learning among elementary school students result in significantly better CT abilities compared to traditional teacher-led instruction?

• (3) Does using the proposed JCLM-MVS for Scratch programming learning among elementary school students result in significantly better learning attitudes compared to traditional teacher-led instruction?

• (4) Does the proposed JCLM-MVS provide an innovative and compelling learning mode that adapts the Jigsaw collaborative learning method, typically used in physical classrooms, to Scratch programming learning in a metaverse virtual space?

• (5) What are the learning experiences, perceptions, and suggestions of elementary school students who use the proposed JCLM-MVS for Scratch programming learning?

Literature review

Current situation and issues in programming instruction

With the rapid development of information and communication technology (ICT), humanity has entered a highly digitized era, and computer programming is closely related to the development of the current digital age. Therefore, schools are increasingly emphasizing the importance of teaching programming skills to students (de la Hera et al., 2022). Fessakis et al. (2013) pointed out that programming is one of the important literacy for developing higher-order abilities, including creativity, spatial cognition, mathematical ability, and logical reasoning. In addition, many previous studies (Fedorenko et al., 2019; Voogt et al., 2015; Wing, 2006) emphasized that programming has been considered as a new and emerging literacy that needs to be actively developed, just like reading and math. Therefore, many countries have recognized the importance of incorporating programming into school curricula (Brinda et al., 2009; Kalelioğlu, 2015) and have gradually expanded programming courses to younger age groups (Bers et al., 2014; Grout & Houlden, 2014).

However, de la Hera et al. (2022) and Kalelioğlu, (2015) argued that teaching students programming is not an easy task, especially when students lack logical reasoning and CT abilities. This often leads to difficulties in understanding abstract concepts in programming, resulting in frustration and decreased interest in learning (de la Hera et al., 2022; Gandy et al., 2010; Robins et al., 2003). Piteira and Costa’s (2013) research indicated that novice programmers are often deterred by the steep learning curve, which requires them to balance understanding abstract concepts with developing practical skills simultaneously. As programming language syntax becomes longer and more complex, placing too much emphasis on syntax instruction can lead to neglecting the comprehension of abstract concepts (Alvarez & Scott, 2010; Soloway, 1986). Furthermore, current programming environments are primarily designed for direct operation on computers to facilitate learning of concrete programming. However, this approach may not effectively enhance learners’ programming thinking, and is often challenging to simulate and observe program execution due to the much faster processing speed of computers (Zhuang et al., 2023).

To help beginners learn programming more straightforwardly and excitingly, block-based programming languages have emerged (Maloney et al., 2010; Resnick et al., 2009). The central development concept is to transform traditional textual programming syntax into visual forms, allowing beginners to intuitively learn programming through drag-and-drop block programming without having to memorize programming language syntax. This allows more attention to be focused on the problem-solving process, making it particularly suitable for learners learning programming. Papert (1980) believed that a suitable programming language should have an easy-to-use “low threshold” and a “high ceiling” capable of producing complex functions. Scratch is block-based programming software that is quick to learn and offers depth and versatility (Maloney et al., 2010). It includes eight categories of programming blocks, including motion, looks, sound, events, control, sensing, operators, and variables. Learners can create interactive stories, animations, games, and music works with programming control logic through a drag-and-drop block combination approach as well as share and communicate their programming creations with creators from all over the world through the official online community (Sáez-López et al., 2016). Due to its simple interface and drag-and-drop programming, Scratch has been widely used to teach programming logic to students of different age groups from elementary school to university (Maloney et al., 2010). Many studies (Leiva & Salas, 2013; Rizvi et al., 2011) have shown that using Scratch as an introductory tool for teaching other high-level programming languages is an effective way to enhance programming learning performance. In addition, many studies have also shown that Scratch’s programming environment can enhance learners’ motivation for programming courses (Cooper et al., 2003; Howland & Good, 2015) and CT (Grover et al., 2015; Pérez-Marín et al., 2020).

In summary, Scratch can effectively assist learners at different stages of programming education and has a positive impact on learners’ CT and attitudes toward programming learning. However, teaching Scratch programming skills burdens teachers with a significant workload, resulting in a lack of timely assistance for most students, which consequently diminishes their performance, interest, and confidence in programming learning. Therefore, this study proposes a Jigsaw collaborative learning method supported by Gather Town virtual interaction classroom for peer-to-peer learning to assist elementary school students in learning Scratch programming, with the aim of further improving their programming learning performance, CT, and learning attitudes.

Application of collaborative learning strategies to programming learning

Traditional programming learning is mainly conducted through teacher-led instruction in a computer classroom, focusing on the teaching of programming syntax and logical concepts. However, many studies have pointed out that this method limits the effectiveness of learning, primarily because students’ opportunities to practice programming skills are reduced, and teachers cannot adequately assess whether the teaching environment is suitable for student learning. Therefore, it is recommended to integrate collaborative programming activities into the instructional design (Bravo et al., 2005; McDowell et al., 2002). For example, McDowell et al. (2002) found that students using pair programming could produce better code and complete the course more quickly than students who worked independently on programming, but there was no significant difference in learning effectiveness between the two groups of students, indicating that collaborative learning is an effective teaching strategy for programming courses. Iskrenovic-Momcilovic (2019) conducted a study on primary school beginners learning Scratch programming, and the results showed that learners who used pair programming had significantly better learning effectiveness than those who worked alone. Hwang et al. (2008) developed a web-based programming-assisted system (WPAS) that provides students with collaborative tagging and peer evaluation learning activities through the Internet. The results showed that students believed that WPAS helped their cognitive development in programming. Li et al. (2023) proposed a metacognition-based collaborative programming approach (M-CPA) to improve students’ performance in collaborative programming, indicating that M-CPA could significantly improve students’ CT tendency, critical thinking tendency, and metacognition tendency in comparison with the conventional collaborative programming approach (C-CPA) without the guidance of the metacognitive questions module.

The Jigsaw method is a collaborative learning model proposed by Aronson et al. in 1971. According to Yu (2017), the most significant difference between the Jigsaw method and other collaborative learning methods is that each member of the group is assigned a part of the teaching task in the learning task, so each member has the opportunity to contribute to the group. In the Jigsaw method learning process, learners are first assigned to expert groups to discuss expert topics and then return to their original home groups to teach each other about their assigned topics. By assigning group members the responsibility to teach each other, the Jigsaw method aims to improve both individual and group performance, thereby enhancing the effectiveness of collaborative learning (Colosi & Zales, 1998). Johnson and Johnson’s (2009) research indicated that the Jigsaw method possesses elements that make collaborative learning effective, particularly active interdependence between members and individual performance responsibility. In other words, team success is based on each member’s commitment to achieving their goals and supporting each other.

Therefore, in addition to being responsible for their own teaching tasks, team members using the Jigsaw method will encourage other members to succeed, developing a strong sense of responsibility because they do not want to bear the burden of team failure (Perihan & Kamuran, 2007). Many past studies (Perkins & Saris, 2001; Walker & Crogan, 1998) have shown that the Jigsaw method is effective for the development of critical thinking and interpersonal communication skills in elementary, middle, and high school as well as university students. Garcia’s (2021) study applied the Jigsaw method to a programming course to evaluate its impact on programming novices. The study’s results showed that the Jigsaw method is an effective method for teaching programming skills to beginners. After a 14-week programming course, students who used the Jigsaw method for collaborative programming learning had significantly higher learning effectiveness than students who learned programming alone, as well as significant improvements in their learning attitudes and self-efficacy. Furthermore, the study conducted by Anyfanti et al. (2015) investigated the training of Scratch programming skills for primary school students using the Jigsaw puzzle game development methodology. The study confirmed that this teaching approach was both satisfying and rewarding for the children. In addition to the above research, there are currently no other studies applying the Jigsaw method to programming learning and using the Jigsaw method for online collaborative programming learning in a remote environment. However, the study by Li et al. (2024) indicated that participants were more engaged in learning activities when utilizing metaverse platforms such as AltSpace and Gather Town, which offer avatar-mediated communications and collaborations, as well as simulate realistic environments and embodied experiences, thus enhancing the overall learning experience. Therefore, this study uses the Gather Town virtual interaction space to assist Jigsaw-based collaborative learning of Scratch programming for elementary school students, with the hope of improving their programming learning performance, CT, and learning attitudes to fill the current research gap.

Cultivating CT through programming

The term “Computational Thinking (CT)” was first proposed by Papert in 1990, and since then, the definition, teaching, and assessment of CT have become essential research topics (Grover & Pea, 2013). Wing (2006) indicated that CT is the use of basic concepts from computer science to solve problems, design systems, and understand the thinking process of human behavior. It is a necessary skill that can be widely applied to daily situations, not just professional knowledge exclusive to computer engineers. However, despite the exploration of scholars and experts in various fields over the years, there is still no consensus on a universal definition of CT (Brennan & Resnick, 2012a, 2012b; Grover & Pea, 2013). The significance of cultivating CT abilities for students has been widely recognized by educational professionals, but how to successfully promote CT education is a more critical issue (Denning, 2017). Many educational professionals believed that programming is the simplest and most direct way to cultivate CT abilities, but this inherent notion may limit the teaching of CT to specific disciplines (Wing, 2006, 2008). Many previous studies have shown that CT has been widely used in teaching different disciplines, including mathematics (Benakli et al., 2017), biology (Rubinstein & Chor, 2014), computer science (Grover et al., 2015), language (Evia et al., 2015), and programming (Pérez-Marín et al., 2020). Although CT can be integrated with many disciplines, most teachers still use programming as the primary teaching method to cultivate students’ CT abilities (Lye & Koh, 2014; Zhong et al., 2016). For example, Broza et al. (2023) designed a Scratch programming course, “Play with me in Code”, to develop pre-service teachers’ CT skills.

Hsu et al. (2018) pointed out that many programming software tools are currently available to help students develop their CT abilities. These tools use interactive programming functions to train students’ logical thinking abilities. Among many programming languages, Scratch is the most commonly used software for teaching CT, followed by ALICE, Scratch4SL, and LEGO visual programming software. This phenomenon shows that these programming languages that use drag-and-drop blocks for programming are more popular with students compared to traditional syntax-based languages (Chen et al., 2017; Grover & Pea, 2013). In addition to being applied in different subjects, teachers also attempt to assist students in learning CT through various teaching strategies, including problem-based, collaborative, project-based, and game-based learning (Hsu et al., 2018). Furthermore, these teaching methods are not only applied in traditional classroom environments, but have also been applied in distance learning and blended learning environments, providing students with more channels to learn CT through physical teachers and virtual learning environments (Basogain et al., 2018; Grover et al., 2015).

In summary, CT has been widely incorporated into teaching different subjects, but currently, programming is still the main subject used to promote the development of CT. In addition, many researchers have also tried to use different teaching strategies in emerging learning environments to promote the development of students’ CT abilities. This study uses a Gather Town virtual interaction space combined with the Jigsaw collaborative learning method to assist elementary school students in learning Scratch programming and explores whether the innovative teaching method proposed in this study has significant benefits for promoting the development of CT in elementary school students compared to traditional teacher-centered teaching methods.

Research methodology

Research participants

This study recruited 48 fifth-grade students, aged 11 to 12 years old, from two classes at an elementary school in New Taipei City, Taiwan, as research participants. Taiwan’s elementary schools implement the policy of S normal class grouping, also known as heterogeneous grouping. This practice involves organizing students into regular, mixed-ability classes rather than grouping them by ability level. The benefit of this class grouping method is that students in different classes have a similar distribution of learning abilities. The study invited the teacher who taught Scratch programming courses to both classes to participate in the experiment. Both classes consisted of beginners in Scratch programming and followed their school’s formal course arrangement for learning the skills. They used two different learning methods for their Scratch programming courses in a computer classroom. Without affecting the original class composition, one of the classes was randomly assigned to the experimental group of 24 students with 13 boys and 11 girls who received Scratch programming learning through the JCLM-MVS. The other class of 24 students with 12 boys and 12 girls was the control group who received Scratch programming learning through a traditional teacher-led teaching method conducted by the teacher invited by the study. All the research participants signed an informed consent regarding the research purpose, procedures, potential risks, benefits, and rights before experimenting with the study. To assess the difference in prior Scratch programming knowledge levels between the two groups before the experiment, this study conducted a pre-test using a Scratch programming performance test sheet developed for this research, as detailed in the section on research tools, to ensure that both groups had the same level of prior Scratch programming knowledge.

Experimental design and procedures

Experimental design

This study was conducted in two fifth-grade Scratch programming classes at an elementary school. Both the experimental and control group learners were taught using different learning methods over a period of 6 weeks in a computer classroom, due to the limited course time available from the teacher who cooperated with our instructional experiment. The experimental group used the metaverse virtual space of Gather Town, supplemented by collaborative programming learning using the Jigsaw method to learn Scratch programming skills, whereas the control group used a traditional teacher-led teaching method. The learning process of the experimental group involved students being heterogeneously grouped as learning groups based on their pre-test scores on CT. According to the consideration of the Jigsaw collaborative learning method, learners in the experimental group must be designed with learning groups and expert groups in learning processes. Thus, this study designed that each learning group in the experimental group consisted of 4 students, thus forming 6 learning groups. Also, there were 4 expert groups with 6 members each because each programming learning unit was divided into four expert themes. After viewing the discussion guidance materials provided by the teacher in the discussion room constructed using Gather Town for expert groups, expert groups engaged in expert topic discussions to resolve the expert theme assigned through desktop sharing and voice functions. The user interface of a discussion room for an expert group shown in Fig. 1 includes features such as a group privacy discussion area, a computer for playing with teaching materials, a teacher observation area, desktop sharing, and voice discussion functions. After completing the expert topic discussions, students returned to their original discussion rooms for learning groups to engage in peer teaching through desktop sharing and voice functions and complete the programming learning tasks for that unit. The user interface of a discussion room for a learning group shown in Fig. 2 includes features such as a group privacy discussion area, a teacher monitoring area, desktop sharing, and voice discussion functions. The control group, on the other hand, used traditional teacher-centered instruction for individual programming learning. The teacher used a computer classroom broadcasting system to project his computer screen onto students’ screens and demonstrated and explained the steps of Scratch programming. Students then carried out the actual programming operations after the teacher’s demonstration and explanation.

Fig. 1

The user interface of expert discussion rooms constructed by Gather Town for four expert groups

Fig. 2

The user interface of learning discussion rooms constructed by Gather Town for six learning groups

This study selected three-course units, “Magical Birthday Cake”, “The Hunt of the Unicorn”, and “Popcorn Fun”, as programming learning units for conducting teaching experiments. Each unit was divided into four expert themes, and the expert themes were merged into a complete learning unit through collaborative learning using the Jigsaw collaborative learning method. The purpose was to explore whether using the Gather Town online group discussion classroom in combination with collaborative programming learning had a beneficial effect on student learning performance, CT, and learning attitudes in Scratch programming compared to a traditional teacher-led teaching method. The discussion topics and corresponding learning contents of expert groups for each learning unit are shown in Table 1. In order to ensure that students fully understand the basic operation of the Scratch programming software, this study first provided instructions on the Scratch software interface and functionality for both the experimental and control groups. In addition, to avoid the experimental group students being unfamiliar with the Gather Town online discussion classroom interface and functionality and affecting the experimental results, the experimental group students also received additional instruction on the operation of the Gather Town online discussion classroom.

Table 1 The discussion topics and corresponding learning contents of expert groups in a learning unit

Experimental procedures

The experimental procedure of this study includes a pre-test stage, an experimental stage, and a post-test stage, as shown in Fig. 3. Firstly, in the pre-test stage, this study explained the curriculum design and experimental purposes, followed by three pre-tests to understand the Scratch programming ability, CT ability, and learning attitude of the two groups of learners before the experiment. Based on the pre-test scores of CT, the experimental group learners were divided into S-shaped heterogeneous groups of six collaborative learning groups with four members each, and they would undergo collaborative learning using the Gather Town-supported Jigsaw method. S-shaped heterogeneous grouping aims to ensure that the groups had more uniform ability levels; each group included students with low, medium, and high pre-test scores in CT.

Fig. 3

The experimental procedures of this study

Next, in the experimental stage, the operation interface and basic functions of Scratch programming software were introduced to both groups of learners. After the introduction of the Scratch operation interface and basic functions, the experimental group also received additional instructions on the operation interface and online discussion room functions of the Gather Town platform and used the collaborative programming learning function of the Jigsaw method to avoid affecting the experimental results due to unfamiliarity with the functions of Gather Town. Then, both groups of learners underwent three units of Scratch programming learning, each lasting for 2 weeks and one class per week, for a total of 6 weeks. The experimental group used the JCLM-MVS to assist Scratch programming learning. The Jigsaw method has five major components: reading, expert group discussion, team report, testing, and team recognition (Ghaith, 2003). First, a total of six learning groups with 4 members each in the experimental group were randomly and, respectively, selected one from each learning group to form 4 expert groups with 6 members each, and the small discussion rooms constructed by Gather Town for expert groups were used to resolve the expert theme assigned through desktop sharing and voice discussion functions. Before conducting an expert group discussion, each member had to read the learning materials of the course unit constructed in the Gather Town virtual learning space in advance. After that, they returned to their original discussion rooms constructed in Gather Town for team reports in the learning groups to complete the unit learning through peer teaching. After the team report, learners further engaged in their learning and ensured comprehension through peer teaching and discussion within their learning groups. Finally, learners received recognition for their contributions to the team. The learning process was repeatedly conducted with three runs because of three programming course units designed for conducting teaching experiments, while the teacher played an observer to alternately assist each expert and learning group’s discussion in the teacher observation rooms, respectively, constructed by Gather Town for expert groups and learning groups. According to the five major components that the Jigsaw method has, it can cultivate a sense of responsibility, improve group awareness, and reduce the occurrence of a free-ride phenomenon. In contrast, the control group used the teacher-led instruction method for individual Scratch programming learning.

Finally, after the experiment, the two groups of learners took a Scratch programming performance test to understand if there were significant differences in their programming learning performance using the two different learning modes in the post-test stage. They also took a post-test on CT to evaluate if there were significant differences in their development of CT ability using the two different learning modes. Additionally, the learners filled out a Scratch learning attitude scale to understand if there were significant differences in their learning attitudes using the two learning modes. To gain a deeper understanding of the differences in learning experiences and perceptions of the two groups of learners using the JCLM-MVS and traditional teacher-led instruction for Scratch programming, the study selected six learners from each group for semi-structured in-depth interviews. The interviewees were selected based on the highest and lowest three Scratch programming performance test scores from each group, for a total of 12 learners interviewed, to supplement the quantitative analysis with qualitative data support.

Research tools

The research tools used in this study include Gather Town group discussion rooms, Scratch programming performance test sheet, computational thinking test sheet, Scratch learning attitude scale, and semi-structured interview outlines.

Gather Town group discussion room

Gather Town is a synchronous online interaction and communication platform with a part of metaverse features developed by Gather Presence Inc. in April 2021. It combines a 2D virtual environment with video call functionality. In addition, Gather Town allows users to play a role through an avatar that can freely move around the virtual space and interact with other users. In this study, Gather Town was used as an interactive discussion tool for collaborative programming learning using the Jigsaw method with the experimental group of students. The aim was to enhance the learning performance, CT ability, and learning attitude of the learners towards programming courses by constructing small group discussion rooms for collaborative programming learning through sharing desktops and voice communication. The various functions of the Gather Town group discussion room constructed in this study are described in detail below:

Object interaction In the Gather Town space editor, users can freely add various objects to decorate their virtual space. They can also embed text, images, URLs, YouTube video links, and external video conference links as learning objects. Users can interact with learning objects by clicking on the “X” button when they are close to them. The online group discussion room built in this study has learning objects embedded with discussion guide instructional videos, as shown in Fig. 4. The experimental group learners can interact with learning objects and discuss with expert group members after watching the instructional videos. The instructional videos, respectively, designed for three programming learning units can be used as foundation knowledge lectured by the teacher to facilitate the discussion and thinking about how to solve the target programming problem assigned by the teacher in the experimental stage with expert groups ruled by the Jigsaw collaborative learning method.

Fig. 4

A learning object embedded with a discussion guide instructional video

Private discussion area In Gather Town’s space editor, users can create private discussion areas by partitioning specific sections of the open space, as shown on the left side of Figs. 1 and 2. These private areas allow users to share audio and video with each other while blocking communication with the surrounding space and other users. In this study, by dividing each group into separate private discussion areas, members are provided with an environment free from external interference for both expert group discussions and learning group discussions, as dictated by the Jigsaw collaborative learning method. In addition, this function is also used to set up a teacher observation area as shown on the right side of Figs. 1 and 2, allowing teachers to observe and intervene in inter-group discussions during student discussions.

Voice communication and desk sharing Gather Town allows users to interact with each other through features such as voice calls, video calls, and desktop sharing. These functions can facilitate expert group discussions and learning group discussions according to the Jigsaw collaborative learning method. To prevent distraction among elementary school students during the experiment, this study did not allow the use of video calls while using Gather Town for collaborative programming learning. Instead, students were only allowed to use voice calls and desktop sharing for collaborative programming learning.

Scratch programming performance test sheet

To investigate the difference in learning outcomes between utilizing the Jigsaw collaborative learning method with Gather Town support and traditional teacher-led instruction for Scratch programming, this study administered a pre- and post-test using a Scratch programming performance test sheet developed internally. Two primary school programming instructors reviewed the test questions for appropriateness, resulting in expert validity. The test sheet comprises four main sections: multiple-choice questions, fill-in-the-blank questions, debugging questions, and analysis questions, totaling 100 points. It covers four key learning topics: introduction to blocks, block applications, program debugging, and program analysis. Content analysis was conducted to ensure alignment between test questions and learning objectives, ensuring content validity.

The test questions underwent two pre-tests. The first pre-test was conducted on 26 sixth-grade students who had completed a semester of a Scratch programming course at a primary school in New Taipei City, Taiwan. This was done to perform difficulty and discrimination analysis and to remove test items with excessively high or low difficulty and low discrimination based on Classical Test Theory (CTT) (De Ruiter & Bers, 2022). To avoid the memory effect, the second pre-test, modified according to the results of the first pre-test, was conducted on 45 other sixth-grade students from the same school to perform difficulty and discrimination analysis once again. The results of the second pre-test sheet showed that the difficulty index of each item ranged from 0.250 to 0.834, and the discrimination index ranged from 0.233 to 0.750, both meeting the minimum requirements for effective test items (Ebel & Frisbie, 1991). Additionally, the second pre-test results were also examined for internal consistency using SPSS software, and the overall test reliability Cronbach’s α value was 0.822, indicating good reliability. Therefore, this study used the second pre-test items as the Scratch programming performance test questions.

The confirmed Scratch programming performance test sheet includes four sections. The first section is multiple choice, with 4 points for each question (Fig. 5). The second section is fill-in-the-blank, with 4 points for each blank (Fig. 6). The third section is debugging, with 7 points for each question (Fig. 7). Students who can circle the location of the programming error are given 3 points, and those who can correct the program after identifying the error are given 4 points. In other words, completing both of these actions simultaneously can earn a total of 7 points. The fourth section is an analysis question, with 5 points for each question (Fig. 8). One point is given for being able to write the execution command, two points for writing conditional instructions, and two points for writing each action instruction. Completing all of these actions simultaneously can earn a total of 5 points. The total score for the entire test is 100 points.

Fig. 5

An example of a multiple-choice question with 4 points for each question

Fig. 6

An example of a fill-in-the-blank question with 4 points for each blank

Fig. 7

An example of a debugging question with 7 points for each question

Fig. 8

An example of an analysis question with 5 points for each question

Computational thinking test sheet

The computational thinking test sheet used in this study is a translation of the Computational Thinking Test (CTt) developed by González (2014), which serves as the basis for evaluating the CT abilities of students aged between 12 and 13 years old. Therefore, it matches the ages of the research participants in this study. The test consists of 28 questions, including 7 CT concepts: “basic directions and sequences”, “loops-repeat times”, “loops-repeat until”, “if-simple conditional”, “if/else-complex conditional”, “while conditional”, and “simple functions”, which correspond to the concepts that constitute the CT framework proposed by Brennan and Resnick (2012a, 2012b), as shown in Table 2. The test questions are all multiple-choice questions with one correct item in 4 choices and each question with the correct answer gives one point. Therefore, the total score of this test sheet is 28 points. The more questions answered correctly, the better the CT ability, and vice versa. The overall reliability of this test, indicated by a Cronbach’s alpha value of 0.793, suggests that the test has acceptable reliability.

Table 2 The corresponding learning concept of the computational thinking test sheet

Scratch learning attitude scale

To understand whether there are significant differences in learning attitudes between the experimental group and the control group after completing the Scratch programming course using different learning modes, this study invited all participants from the two groups to complete the Scratch learning attitude scale based on their actual learning experience and feelings in the experiment. This study used the Scratch learning attitude scale compiled by Chang (2009), which has a total of 31 questions using a Likert five-point scale, with options ranging from 1 to 5 points, including “strongly agree” (5 points), “agree” (4 points), “neutral” (3 points), “disagree” (2 points), and “strongly disagree” (1 point). The scale is divided into five dimensions based on the attitudes that learners may develop after learning Scratch programming courses, including “satisfaction” (5 questions), “interest” (9 questions), “practicality” (6 questions), “collaboration” (5 questions), and “confidence” (6 questions). The reliability of each dimension of the scale ranges from 0.662 to 0.784, and the overall reliability of the scale measured by Cronbach’s α is 0.87, indicating good reliability of the scale.

Interview outline and protocol

A semi-structured interview, also known as a guided interview, is a method in which the researcher prepares several questions in advance as an interview outline according to the research purpose. During the interview process with the interviewee, the researcher may extend other questions, and the process does not necessarily have to follow the outline completely. This method has the advantage of flexibility, which allows the interviewer to adjust the direction of the interview in response to the interviewee’s feedback and to provide space for the interviewee’s opinions. In this study, after the experiment was completed, semi-structured interview outlines were used to conduct in-depth interviews with the learners in the experimental and control groups. The goal was to understand their feelings, thoughts, and suggestions on using Gather Town to assist in collaborative Scratch programming learning using the Jigsaw method, compared to traditional teacher-led instruction for Scratch programming learning. Specifically, the effectiveness of Gather Town’s small group discussion rooms for online programming learning discussions was investigated to complement the quantitative analysis. The following questions comprise the interview outline for the experimental group:

• 1. Have you encountered any difficulties in using Gather Town to collaborate with your learning partners?

• 2. How do you think the completion of the Scratch programming learning would have been affected if you had not used Gather Town to help you discuss with learning partners?

• 3. What features do you think can be added or improved when using Gather Town for programming learning?

• 4. What do you think is the difference between using Gather Town for online learning and other online meeting platforms, such as Google Meet, Zoom, or Microsoft Teams?

• 5. What do you think is the difference between interacting with learning partners in Gather Town through virtual avatars and interacting with them physically?

The following questions are the interview outline for the control group:

• 1. Have you encountered any problems in the Scratch programming lesson lectured by the teacher?

• 2. Do you like the way the teacher demonstrates before you do it yourself? Why?

• 3. Do you want to collaborate and discuss with your partners like learners in the experimental group to learn Scratch programming skills?

This study interviewed selected participants from both the experimental and control groups. The interviewees were chosen based on the top three and bottom three scores in the post-test of the Scratch learning performance test. Therefore, there were six participants in each group, with a total of 12 learners being interviewed. Although the control group used the traditional teacher-centered approach for Scratch programming learning, in the interviews, the researchers also explained the teaching design philosophy and learning process of the Jigsaw method with Gather Town support to Scratch programming learning to the control group learners. Therefore, during the interviews, learners from both groups provided their viewpoints and feedback on the Jigsaw method with Gather Town support. This study employed a bottom-up approach to analyze interview data, aiming to summarize the benefits and problems associated with the two learning modes in terms of learning experiences, perceptions, and suggestions. The coding process involved fitting the interview data into the corresponding interview questions.

Data analysis method

The “rule of thumb” suggesting that a sample size of 30 is needed for parametric tests is a general guideline, not an absolute requirement (Martinez-Abrain, 2014). The appropriateness of using parametric statistics depends on various factors, including the specific statistical test, the distribution of the data, and the assumptions of the test. Although the sample size of this study is smaller than 30, it was confirmed that the data distribution of both groups satisfied the assumption of normality according to the Kolmogorov–Smirnov test (Drezner et al., 2010). Therefore, the parametric statistical method ANCOVA was used to examine the research questions of this study as well as the effect size, dppc2 (Morris, 2008), and Cohen’s d (Cohen, 1998), were used to verify the significance. Additionally, pre-test scores on the Scratch programming learning performance test, CT test, and Scratch learning attitude scale were considered as covariates to conduct ANCOVA. This was done to examine whether there were significant differences between the groups in terms of the effectiveness of Scratch programming learning, CT, and Scratch learning attitude. These pre-test scores represent prior knowledge, abilities or states of the students that may potentially affect the dependent variables considered in the study. By considering these as control variables, the experimental error can be reduced through statistical methods.

Experimental results

Analysis of differences in Scratch programming learning performance between two groups of learners

To compare the differences in Scratch programming learning performance between two groups of learners, this study used one-way ANCOVA with pre-test scores on the Scratch programming learning performance test as a covariate to compare whether there was a statistically significant difference in post-test performance between the two groups of learners. The analysis is aimed to answer the first research question of the study. Before conducting the ANCOVA, it was necessary to examine a homogeneity of regression coefficients test within the groups, and the results showed that the assumption of homogeneity of regression coefficients was not violated (F = 0.18, p = 0.894 > 0.05), so the ANCOVA could be performed. The results of the ANCOVA with the corresponding effect size (Morris, 2008) are shown in Table 3. The results showed that the adjusted mean post-test scores of the experimental group learners were higher than those of the control group learners and reached statistically significant differences after excluding the influence of the covariate (F = 7.877, p = 0.007 < 0.05).

Table 3 One-way ANCOVA results of Scratch programming learning performance for two groups of learners

Analysis of differences in CT between two groups of learners

To compare the differences in CT between two groups of learners, this study employed a one-way ANCOVA analysis, using pre-test scores of the CT test as the covariate and post-test scores of the computational thinking test as the dependent variable to determine whether there were significant differences in CT performance between the two groups of learners. The analysis is aimed to answer the second research question of the study. Before conducting the ANCOVA analysis, a homogeneity test of regression coefficients within groups was examined, and the results showed that overall (F = 0.092, p = 0.764 > 0.05), “Basic direction and sequence” (F = 1.877, p = 0.178 > 0.05), “Loops-repeat times” (F = 0.505, p = 0.481 > 0.05), “Loops-repeat until” (F = 0.037, p = 0.848 > 0.05), “If-simple conditional” (F = 0.000, p = 0.985 > 0.05), “If/else-complex conditional” (F = 0.984, p = 0.327 > 0.05), “While conditional” (F = 0.131, p = 0.720 > 0.05), and “Simple functions” (F = 0.882, p = 0.353 > 0.05) did not violate the assumption of homogeneity of regression coefficients, allowing the ANCOVA analysis to proceed. The results of the ANCOVA analysis with the corresponding effect size (Morris, 2008) are shown in Table 4. The results indicated that the overall CT of the experimental group learners was higher than that of the control group learners in terms of adjusted mean post-test scores. However, after excluding the influence of the covariate, there were no statistically significant differences in overall (F = 0.756, p = 0.389 > 0.05), “Basic direction and sequence” (F = 1.337, p = 0.254 > 0.05), “Loops-repeat times” (F = 1.281, p = 0.264 > 0.05), “Loops-repeat until” (F = 0.411, p = 0.525 > 0.05), “If-simple conditional” (F = 0.063, p = 0.804 > 0.05), “If/else-complex conditional” (F = 0.583, p = 0.449 > 0.05), and “Simple functions” (F = 1.088, p = 0.302 > 0.05). Only in the “While conditional” dimension, there was a statistically significant difference (F = 4.504, p = 0.039 < 0.05), and the experimental group learners significantly outperformed the control group learners.

Table 4 One-way ANCOVA results of overall and sub-dimensional differences in computational thinking between two groups of learners

As there were no significant differences in the overall and most dimensions of CT performance between the two groups of learners, this study adopted a paired sample t-test to examine whether there were significant differences in the pre-test and post-test scores of CT for the two groups of learners. The results of the paired sample t-test with the corresponding effect size (Cohen, 1998) are shown in Table 5. The results showed that the experimental group learners achieved statistically significant differences in their pre-test and post-test scores of CT (t = − 3.989, p = 0.001 < 0.05); the control group learners also achieved statistically significant differences in their pre-test and post-test scores of CT (t = − 3.045, p = 0.006 < 0.05), indicating that the CT abilities of the two groups of learners, who adopted different learning modes, were significantly improved after learning.

Table 5 Paired-sample t-test results of pre- and post-tests on computational thinking for two groups of learners

Analysis of differences in learning attitude between two groups of learners

To compare the differences in Scratch learning attitudes between the two groups, this study used one-way ANCOVA with pre-test scores of overall Scratch learning attitudes and dimensions such as “satisfaction”, “interest”, “collaboration”, “practicality”, and “confidence” as covariates, and post-test scores as the dependent variable, to compare whether the two groups had significant differences in overall learning attitudes and various dimensions. The analysis is aimed to answer the third research question of the study. Before conducting the covariate analysis, a homogeneity test of regression coefficients within the groups was examined. The results showed that the homogeneity assumption of regression coefficients was not violated for overall (F = 0.200, p = 0.657 > 0.05), “satisfaction” (F = 0.153, p = 0.697 > 0.05), “interest” (F = 0.040, p = 0.843 > 0.05), “collaboration” (F = 0.082, p = 0.776 > 0.05), “practicality” (F = 1.760, p = 0.191 > 0.05), and “confidence” (F = 1.529, p = 0.223 > 0.05), and the covariate analysis could be continued. The results of one-way ANCOVA with the corresponding effect size (Morris, 2008) are shown in Table 6. The results showed that the experimental group had higher adjusted post-test mean scores than the control group in overall and various dimensions. After controlling for the covariates, the experimental group showed significant differences from the control group in overall learning attitudes (F = 28.113, p = 0.000 < 0.05), as well as dimensions such as “satisfaction” (F = 7.891, p = 0.007 < 0.05), “interest” (F = 19.226, p = 0.000 < 0.05), “collaboration” (F = 8.545, p = 0.005 < 0.05), and “practicality” (F = 21.002, p = 0.000 < 0.05), but not in the “confidence” dimension (F = 2.816, p = 0.100 > 0.05).

Table 6 One-way ANCOVA results for the overall learning attitude and each dimension of the two groups of learners

Interview analysis

Appropriateness of Gather Town supplemented with the Jigsaw method to facilitate individual and group accountability for learning Scratch programming

The interview analysis is aimed to answer the fourth research question of the study. According to the interview results, the participants in the experimental group reported that using Gather Town supplemented with the Jigsaw method for learning Scratch programming allowed each group member to exercise group accountability through mutual discussion in an expert group and individual accountability through peer teaching in a learning group, thereby reducing the learning burden. This reduction occurs because members of the same learning group teach each other after alternately playing roles to resolve the expert themes assigned in the expert groups. In addition, the experimental group participants also reported that using Gather Town supplemented with the Jigsaw method for learning Scratch programming helped suppress the free-riding phenomenon in collaborative learning. From the interviews with most of the control group participants, it was found that using traditional teacher-led instruction for learning Scratch programming could effectively help learners complete learning tasks, but some control group participants reported that they could not receive timely assistance from the teacher when they encountered difficulties in the learning process. In addition, some control group participants also reported feeling bored and having difficulty when using traditional teacher-led instruction for learning Scratch programming. Interview results from the experimental and control groups were represented as ES and CS, respectively, along with a student number. Several representative excerpts from the interview results are listed as follows:

ES5: “Sometimes I teach them what to do, and sometimes my partners help me and we help each other.”

ES3: “No, no one is lazy, everyone will discuss how to solve the programming problems assigned.”

ES2” “It’s less burdensome to learn in this way, and my partners will teach me if I don’t know how to do it.”

ES4: “I taught what I learned to my group members, and they helped me complete things I didn’t understand. We all discussed together.”

CS1: “Sometimes I can’t find where the material is or I don’t know how to solve my programming problems, and the teacher is too busy helping other students, and then I get stuck.”

CS5: “Teachers have done it first, and we all do what teachers do, and we all do the same thing, and it’s a little bit boring.”

CS1: “I don’t want to, Scratch is too hard and I’m not interested in programming.”

CS4: “Learning Scratch is a bit boring, it would be more fun if I could discuss it with my classmates at Gather Town.”

Learning experiences, perceptions, and suggestions of Gather Town supplemented with the Jigsaw method to facilitate learning Scratch programming

The interview analysis is aimed to answer the fifth research question of the study. According to the interview results, most of the participants in the experimental group reported that they did not encounter any difficulties in operating the Gather Town platform and did not experience any communication difficulties while interacting with their group members during the collaborative learning process. This indicates that the user interface design of Gather Town is user-friendly. Most of the participants in the experimental group also reported that if they did not use Gather Town to assist in discussions with their group members, it would have affected their collaborative learning process due to not being able to receive peer help in a timely manner. Most of the participants in the experimental group stated that compared to other online interaction and communication platforms, such as Google Meet, Zoom, or Microsoft Teams, the use of avatars in Gather Town allows for more diverse interactions with others. Additionally, Gather Town can construct virtual environments and objects, creating an interactive space similar to reality. Learning Scratch programming in this way is quite interesting. Interviews with the experimental group participants revealed that the main difference between using Gather Town for interactions and physical interactions is that physical interactions can utilize body language and touch to facilitate communication. Otherwise, there is not much difference between both. Moreover, one interviewee suggested from his personal perspective that if Gather Town was changed from a 2D to a 3D virtual space, it would be more immersive and attractive to learners. Several representative excerpts from the interview results are listed as follows:

ES1: “It’s all right, there’s no difficulty when using Gather Town for programming learning with partners.”

ES3: “No, I can understand most of the things my partners tell me.”

ES5: “I can help my partners if they don’t know something; sometimes if I can’t find something or don’t know something, my partners will tell me.”

ES1: “Google Meet can only talk and type something, but Gather Town allows you to walk around with other people, and you can decorate or play games.”

ES1: “If we don’t use Gather Town to help us to have discussions, there are some things that we may not know about.”

ES2: “Apart from talking and video, you can also see other people’s little figures moving around, which is more like reality.”

ES1: “Learning programming through Gather Town is quite interesting and different from the teacher’s lecture.”

ES2: “With Gather Town, you can’t communicate with other people with body language, but except for that there’s no other difference.”

ES4: “Gather Town is a 2D virtual space; it would be more immersive and attractive to me if it was changed to 3D.”

Discussion

This study explores whether using the JCLM-MVS is significantly more effective for learning Scratch programming to elementary school students compared to traditional teacher-led instruction. The results of the covariance analysis showed that, after excluding the influence of pre-test scores, the experimental group’s learning performance was significantly higher than the control group, indicating that learners using the JCLM-MVS for learning Scratch programming had significantly better learning performance than those using traditional teacher-led instruction. Several previous studies (Garcia, 2021; Iskrenovic-Momcilovic, 2019) have indicated that using collaborative learning strategies for programming learning is a more effective teaching method, especially for improving the learning performance of programming beginners, which is consistent with the results of this study. In addition, previous studies have shown that Jigsaw-based learning has elements that make collaborative learning more effective, especially active interdependence among members and individual performance responsibility (Johnson & Johnson, 2009). From the interview data, most experimental group interviewees expressed that using the JCLM-MVS for learning Scratch programming allows each learner to contribute to their team through real-time group discussions and peer assistance and share the learning responsibility, which reduces the learning burden and helps to inhibit free-riding phenomenon in collaborative learning, which echoes the results of Johnson and Johnson’s (2009) research. Most control group interviewees indicated that traditional teacher-led instruction for Scratch programming learning effectively helps them complete learning tasks, but they cannot receive timely assistance from the teacher when encountering difficulties, and they find it less enjoyable. Regardless of whether they were in the experimental or control group, the interviewees believed that peer assistance could reduce the difficulty of learning Scratch, so they were more inclined to use the JCLM-MVS for learning Scratch programming.

This study investigated whether primary school students’ CT abilities are significantly better when using the JCLM-MVS to learn Scratch programming than traditional teacher-led instruction. The results of the covariance analysis showed that after controlling for pre-test scores, there was only a significant difference in the “While conditional” dimension of CT between the experimental and control groups, while there were no significant differences in overall and the other dimensions. This suggests that learners using the JCLM-MVS to learn Scratch programming did not show significantly better CT than those using traditional teacher-led instruction. The main reason for the lack of significant differences in overall CT and most dimensions between the two groups may be due to the short six-week duration of the teaching experiment, which may not have been enough time to produce significant differences in CT abilities or to transfer the concepts from basic programming units to CT in a short time. In addition, the CT test used in this study was difficult for primary school students in terms of both question descriptions and overall difficulty, which may also be one of the possible reasons why there were no significant differences in CT between the two groups. The “While conditional” dimension, often involving loops and iterative processes, may align more closely with the natural progression of students’ understanding, leading to greater proficiency and thus having statistical significance in assessment outcomes compared to the other dimensions of CT. Furthermore, this study further investigated whether the CT performance of the two groups of learners is significantly better after learning with different learning modes than before. The results show that learners in both the experimental and control groups significantly improved their CT performance after learning Scratch programming through two different learning modes. This indicates that regardless of using the JCLM-MVS or traditional teacher-led instruction to learn Scratch programming, both methods have a significant effect on improving learners’ CT abilities. Several previous studies (Grover et al., 2015; Pérez-Marín et al., 2020; Zhang & Nouri, 2019) have shown that the Scratch programming environment can enhance learners’ CT, which is consistent with the results of this study.

This study investigated whether the use of JCLM-MVS by elementary school students for learning Scratch programming results in significantly better learning attitudes compared to traditional teacher-led instruction. The results of the ANCOVA showed that, after controlling for pre-test scores, the experimental group’s learning attitudes are significantly higher than the control group in terms of overall, satisfaction, interest, collaboration, and practicality, but there is no significant difference in confidence. This indicates that learners in the experimental group who used the JCLM-MVS for learning Scratch programming have significantly better learning attitudes than the control group who used traditional teacher-led instruction, but there is no significant difference in confidence. The research findings align with those of Chang and Hwang (2017), suggesting that educational games incorporating mission synchronization-based peer-assistance mechanisms can enhance students’ learning achievements and attitudes. Additionally, the study suggests that this difficulty may arise from the fact that Scratch programming remains challenging for fifth-grade elementary school students, making it quite challenging to foster confidence in learning Scratch programming. Mainly the learning process of programming requires attention to both the understanding of abstract concepts and the development of practical skills, making it difficult for beginners to learn and therefore reducing their interest in learning (Piteira & Costa, 2013). Qualitative interview data from some of the interviewees in this study showed that factors affecting learners’ interest in learning Scratch include the high difficulty of learning Scratch programming, which reduces their interest in learning about the curriculum and Scratch programming software. This result is consistent with the findings of Piteira and Costa (2013). In addition, it was found from the interviews that the experimental group interviewees had a significantly higher preference for the six-week Scratch programming course than the control group interviewees. Most interviewees stated that compared to the traditional teacher-centered approach, using the JCLM-MVS for learning Scratch programming can enhance learners’ interest in learning as well as expressed factors that affect their willingness to continue learning programming are the degree of interest in programming and the belief that whether programming can be applied in daily life or not.

According to the interview results, this study also found that the proposed JCLM-MVS can facilitate the adaptation of the traditional Jigsaw collaborative learning method, typically implemented in a physical classroom space, to Scratch programming learning in a metaverse virtual space with Gather Town support. There were some vital pieces of evidence from the interview results supporting the research findings. Firstly, most interviewees stated that the use of avatars in Gather Town allows for more diverse interactions with others, enabling the construction of virtual environments and objects and creating an interactive space similar to reality. Secondly, each group member was able to contribute to the group and exercise their learning responsibility more easily through Gather Town’s virtual space and interaction functionalities. This approach reduced the learning burden, as members of the same learning group taught each other. They alternated roles to resolve the expert themes assigned in the expert groups. Moreover, using Gather Town in conjunction with the Jigsaw method for learning Scratch programming could reduce the difficulty of learning through peer assistance. Finally, the combination of Gather Town and the Jigsaw method made learning Scratch programming more enjoyable.

While this study has some encouraging research findings, it also has two research limitations. Firstly, this study only recruited 48 fifth-grade students to participate in the experiment, with each learning group having only 24 research subjects. The small sample size may limit the generalizability of the findings to the broader population of primary school students. A larger sample size is needed to generalize the results to the elementary student population. Secondly, Scratch is grounded in Brennan and Resnick’s (2012a, 2012b) CT framework, which also guided the development of Scratch 2.0 (Brennan & Resnick, 2013). A systematic review by Zhang and Nouri (2019) examined the CT skills that can be developed through Scratch for K-9 students based on empirical evidence. The findings indicate that Scratch effectively teaches all CT skills outlined in Brennan and Resnick’s (2012a, 2012b) framework. Additionally, it was found to foster other CT skills, including input/output, reading, interpreting and communicating code, using multimodal media, predictive thinking, and human–computer interaction. Moreover, a systematic review by Wang et al. (2022) indicated that previous research has demonstrated the benefits of integrating STEM content and contexts into CT learning. Similarly, incorporating CT into STEM education enhances students’ understanding of STEM subjects due to the essential role of computation in contemporary STEM fields. Therefore, it is reasonable to infer that Scratch programming can be generalized to other STEM subjects. However, this study specifically focused on using Scratch programming as the learning subject to facilitate CT, which plays a crucial role in STEM education for primary school students. Thus, it is not appropriate to generalize the findings to STEM education for other age groups, such as junior, senior, or university students.

Conclusions and future work

Conclusions

The study aimed to assess the efficacy of the Jigsaw collaborative learning method in a metaverse virtual space (JCLM-MVS) compared to a traditional teacher-centered approach in learning Scratch programming. Results from one-way ANCOVA analyses revealed that learners in the experimental group, utilizing the JCLM-MVS, exhibited significantly better learning performance than those in the control group. Interviews further underscored the benefits of the JCLM-MVS, highlighting its facilitation of real-time collaboration, peer assistance, and shared learning responsibility, leading to reduced learning burden and inhibiting free-riding behavior. However, while the JCLM-MVS did not significantly outperform a traditional teacher-centered approach in overall CT, it did show superiority in the “While conditional” dimension. Interestingly, both learning approaches effectively improved CT abilities. Furthermore, ANCOVA analyses indicated that learners in the experimental group demonstrated significantly better learning attitudes, including satisfaction, interest, collaboration, and practicality, compared to the control group, although no significant difference was observed in confidence. In conclusion, the study proposes a novel approach to learning Scratch programming by integrating a metaverse virtual space with the Jigsaw collaborative learning method. This innovative and engaging learning mode enhances students' learning performance, CT abilities, and learning attitudes.

Instruction suggestions

This study utilized the JCLM-MVS to assist learners in learning Scratch programming. For elementary school students, the Jigsaw collaborative learning model is not a familiar method. The teacher who supported the experiment in the teacher observation area observed that learners’ discussion skills in expert groups, governed by the Jigsaw model, were not quite proficient, despite receiving prior instruction. Therefore, learners need more thorough explanations before engaging in collaborative learning activities. Therefore, learners need to be more deeply explained before applying it for collaborative learning. In addition to explaining the steps of Jigsaw collaborative learning, it is also necessary to explain the responsibility of each learner to discuss and teach each other. Perihan and Kamuran (2007) pointed out that in Jigsaw collaborative learning, group members not only take responsibility for their teaching tasks, but also encourage other members to succeed and develop a strong sense of responsibility because they do not want to bear the responsibility for the team failure. Therefore, helping learners much more understand the meaning of Jigsaw collaborative learning can help them cultivate a sense of responsibility for learning through Jigsaw group discussion activities, thereby promoting learning effectiveness and attitudes while reducing free-riding behavior. Furthermore, when preparing learning materials for the JCLM-MVS for learning Scratch programming, teachers can appropriately intersperse guided questions for the expert group discussion and combine them with guided learning sheets. The purpose of arranging guided questions is to guide expert groups to discuss the key points and maintain learners’ focus on a learning unit during the discussion. Guided learning sheets can also be used as records summarized by the group discussion, and learners can use them as a basis for mutual teaching, which can be beneficial for improving the quality of learners’ mutual teaching.

Future work

This study has shown that using the JCLM-MVS for learning Scratch programming is significantly more effective and positive in learning performance and attitudes than using a traditional teacher-led teaching method. In terms of CT, although there was a significant improvement in CT for both groups of learners before and after learning, there was no significant difference in CT between the two groups. This may be because this study only conducted a six-week Scratch programming learning course, so there was not enough time to produce significant differences in CT in such a short learning period. Wing (2006) pointed out that CT is one of the essential abilities widely applied in daily life situations and requires a long time to develop. Therefore, in the planning of future research, if more extended learning activities can be conducted, it may further verify the use of the JCLM-MVS and a traditional teacher-led teaching method for learning Scratch programming in promoting differences in CT. Moreover, this study analyzed the differences in learning performance, CT, and learning attitudes between the experimental and control groups of Scratch programming learners who used two different learning modes. However, learners with different background variables, such as gender, prior knowledge, and learning styles, may have different effectiveness after the experiment. Furthermore, exploring the learning differences between the two groups of learners under different background variables can help teachers understand the needs of different types of learners and provide direction for course improvement, thereby improving learning performance, CT, and learning attitudes of learners. Also, this study has demonstrated that using the JCLM-MVS for learning Scratch programming is significantly better than using a traditional teacher-led teaching method in terms of learning performance and learning attitudes. However, the present research result is limited to learning Scratch programming for elementary school students. Slavin’s (1985) study indicates that collaborative learning can be widely applied to most subjects and teaching stages. Therefore, exploring the learning effectiveness of the JCLM-MVS in other subjects and teaching stages is also a research direction worth exploring in the future. Furthermore, the experimental group utilized the virtual space of Gather Town within the metaverse, coupled with collaborative programming sessions employing the Jigsaw method, to acquire Scratch programming skills. It is worth analyzing how collaborative learning approaches, supported by learning process analyses, could benefit from integrating behavioral recording technology into Gather Town. Finally, this study has demonstrated that using the JCLM-MVS in learning Scratch programming has significant effects on promoting learning performance, CT, and learning attitudes. However, due to the COVID-19 pandemic in Taiwan during the study period, students often could not come to school for learning due to home quarantine or self-isolation, which prevented the physical Jigsaw collaborative learning from being applied in the programming course designed for this study. Therefore, the study was unable to explore in-depth the impact of the collaborative learning mode of Gather Town virtual classroom and physical classroom Jigsaw learning on elementary school Scratch programming learning. This was regrettable for this study. Therefore, in future research planning, if another experimental group could be planned to use physical classroom Jigsaw learning to assist in learning Scratch programming, it would be more helpful for educators to understand the impact of different learning designs and learning aids on learners’ learning performance in programming teaching. This would enable educators to adopt the most favorable learning mode for learners in different learning situations.