Effects of infusing the engineering design process into STEM project-based learning to develop preservice technology teachers’ engineering design thinking

Abstract

Background

This study focuses on probing preservice technology teachers’ cognitive structures and how they construct engineering design in technology-learning activities and explores the effects of infusing an engineering design process into science, technology, engineering, and mathematics (STEM) project-based learning to develop preservice technology teachers’ cognitive structures for engineering design thinking.

Results

The study employed a quasi-experimental design, and twenty-eight preservice technology teachers participated in the teaching experiment. The flow-map method and metalistening technique were utilized to enable preservice technology teachers to create flow maps of engineering design, and a chi-square test was employed to analyze the data. The results suggest that (1) applying the engineering design process to STEM project-based learning is beneficial for developing preservice technology teachers’ schema of design thinking, especially with respect to clarifying the problem, generating ideas, modeling, and feasibility analysis, and (2) it is important to encourage teachers to further explore the systematic concepts of engineering design thinking and expand their abilities by merging the engineering design process into STEM project-based learning.

Conclusions

The findings of this study provide initial evidence on the effects of infusing the engineering design process into STEM project-based learning to develop preservice technology teachers’ engineering design thinking. However, further work should focus on exploring how to overcome the weaknesses of preservice technology teachers’ engineering design thinking by adding a few elements of engineering design thinking pedagogy, e.g., designing learning activities that are relevant to real life.

Introduction

An increasing number of position papers and empirical studies have focused on exploring the issues of engineering design thinking (Brand, 2020; English & King, 2015). With regard to the cognitive structure of technology teachers in engineering design, Atman et al. (2007) studied the differences between expert practitioners and students during the engineering design process. They proposed that students should pay more attention to problem scoping, information gathering, and decision-making as they develop their cognitive structures and schematic processes in engineering design. Furthermore, when Hynes (2012) investigated how middle school teachers understood and taught the process of engineering design, he found that technology teachers frequently exhibit sophisticated thinking during two particular steps of engineering design: constructing a prototype and redesigning. In a study about the performance of high school students in engineering design subjects, Fan, Yu, and Lou (2018) noted that students’ abilities in predictive analysis and testing/revising are key factors in determining their thinking in engineering design. Sung and Kelley (2018) performed a sequential analysis study on the design thinking of fourth-grade elementary students and found that idea generation plays an important role throughout their design thinking process. These studies suggest that students and teachers at different levels focus on different parts of the design process: idea generation is the most important part of the process for elementary school students; predictive analysis and testing/revising are more important for high school students; and technology teachers tend to focus on prototype construction and redesign. Therefore, the differences between engineering experts and beginners in the engineering design process do not necessarily correspond to what technology teachers emphasize in their engineering design teaching. Furthermore, there is a lack of research on the cognitive structures and engineering design foci of preservice technology teachers, which is an important issue that needs to be addressed.

In addition to engineering design thinking, science, technology, engineering, and mathematics (STEM) education has also received considerable attention in recent years. The notion of STEM itself and how its disciplines should be integrated are open to debate (English & King, 2019), and attracting appropriately qualified people to study and work in STEM areas is an urgent need (Holmes, Gore, Smith, & Lloyd, 2017). Various definitions of STEM education have ranged from disciplinary to transdisciplinary approaches, but from a broad perspective, it can be defined as follows: “STEM education is used to identify activities involving any of the four areas, a STEM-related course, or an interconnected or integrated program of study” (English, 2017). Strimel and Grubbs (2016) noted that technology and engineering are often neglected in secondary-school STEM education, and this neglect perpetuates the educational system’s shortcomings in nurturing technology and engineering talent. In view of this problem, Song et al. (2016) and Brophy, Klein, Portsmore, and Rogers (2008) agreed that STEM education implementation can be improved by planning technology-learning activities (such as hands-on activities) that incorporate engineering design so that students can obtain comprehensive cross-disciplinary experience. That is, students will have more chances to apply STEM knowledge and competency in solving problems or meeting needs instead of focusing on learning specific subject matter and neglecting the application of that knowledge (Lin, Hsiao, Williams, & Chen, 2020). In addition, many studies have found that technology-learning activities based on engineering design enhance learning in STEM. For example, English and King (2019) reported students’ responses to designing and constructing a paper bridge that could withstand an optimal load. Their results showed that students’ sketches indicated an awareness of the problem constraints, an understanding of basic engineering principles, and the application of mathematics and science knowledge.

Based on this line of reasoning, technology-learning activities that incorporate engineering design should be useful for implementing STEM education. However, the implementation of teaching activities oriented toward engineering design requires teachers who have a strong conceptual understanding of engineering design. Therefore, investigations of this aspect are important from an academic and practical perspective. For example, in studies about engineering design, some common concerns are the cognitive structures of expert practitioners in engineering design (Atman et al., 2007) and the characterization of the engineering design process (Hannah, Joshi, & Summers, 2012); the findings of these studies are usually applied in different educational settings (Capobianco & Rupp, 2014; Fan, Yu, & Lin, 2020). Therefore, if technology teachers are to implement STEM education using the processes of technology-learning activities based upon engineering design, the teachers’ own cognitive structure of engineering design is of great significance. A cognitive structure is a hypothetical construct showing the extent of concepts and their relationships in a learner’s long-term memory (Shavelson, 1974). Through probing technology teachers’ cognitive structures, technology teacher educators can understand what knowledge technology teachers have already acquired. In addition, it is a fundamental step toward acquiring evidence of technology teachers’ cognitive structure in order to explore how technology teachers construct engineering design in technology-learning activities (Wu & Tsai, 2005).

One of the primary objectives of the Taiwan Technology Curriculum in 12-year compulsory education is to ensure that secondary school students possess basic competencies in engineering design thinking before they enter university-level engineering institutes (Ministry of Education, 2018). This sounds like a prevocational approach, as engineering design thinking is relevant only to students who progress to engineering education. If the Technology Curriculum is a general curriculum for all students, then the rationale should be that engineering design thinking is good for all students. The aforementioned studies suggest that the cognitive structures of technology teachers in engineering design will affect their use of technological pedagogical approaches oriented toward engineering design and thus determine the quality of STEM education implementation. To address this gap in the literature, this study focuses on the cognitive structure of preservice technology teachers in engineering design to probe their understanding of engineering design thinking. We will describe the current status and issues related to the way preservice technology teachers apply cognitive structure to engineering design. The findings of this study will help preservice technology teachers incorporate engineering design processes into their technology teaching activities. This, in turn, could foster engineering design capabilities in secondary school students and their interest in engineering-related areas. More specifically, the primary research questions of this study are as follows: (1) How does the incorporation of engineering design processes into STEM project-based learning benefit the training of preservice technology teachers in terms of their cognitive structure in engineering design thinking? (2) What are the weaknesses in the cognitive structure of preservice technology teachers in engineering design thinking?

Theoretical framework

To explore the gap in engineering design thinking in the literature, the following literature review focuses on exploring engineering design thinking and related studies. Furthermore, to develop the theoretical basis for STEM project-based learning combined with the engineering design process, the following literature review also focuses on analyzing the related studies and proposing the key elements in developing a STEM project learning curriculum.

Engineering design thinking

Many scholars in the field of technology and engineering education believe that engineering design thinking is a basic competency in engineering and that this mode of thinking should be given priority in secondary and tertiary education (Atman et al., 2007; Dym, Agogino, Eris, Frey, & Leifer, 2005). According to Dym et al. (2005), “Engineering design is a systematic, intelligent process in which designers generate, evaluate, and specify concepts for devices, systems, or processes whose form and function achieve clients’ objectives or users’ needs while satisfying a specified set of constraints.” The cultivation of engineering design thinking can encourage students to develop an inquisitive mindset, approach problems from multiple perspectives, and question existing norms.

Recent studies on engineering design thinking have generally focused on either the processes of this form of thinking or the difference between engineering design experts and university-level engineering students in terms of engineering design thinking. A variety of design thinking processes have been proposed by researchers in this area. Atman et al. (2007) proposed that the engineering design process proceeds as follows: (1) problem definition, (2) information gathering, (3) idea generation, (4) modeling, (5) feasibility analysis, (6) evaluation, (7) decision, (8) communication, (9) implementation, and (10) design revision. Similarly, Hynes (2012) proposed a slightly different engineering design process: (1) identify need or problem, (2) research need or problem, (3) develop possible solutions, (4) select best possible solution, (5) construct a prototype, (6) test and evaluate solution, (7) communicate the solution, and (8) redesign. The main differences between the engineering design thinking process and the problem-solving process are a greater emphasis on modeling and feasibility analysis in the former. In other words, in the problem-solving process, students may not as thoroughly evaluate the viability of each idea when choosing the optimal solution. However, the appropriate use of modeling and feasibility analyses can improve students’ capacity to evaluate ideas, and this is one of the most important features of the engineering design thinking process.

The engineering design thinking processes proposed by Atman, Cardella, Turns, and Adams (2005) and Hynes (2012) include a communication step after the decision step. Even after modeling and feasibility analyses have been conducted and an optimal solution has been selected, it is still necessary to communicate with a client to confirm the final solution and make further adjustments to the solution to address any questions or problems. The purpose of this step is to confirm that a client’s needs are met and to ensure that all team members understand the final solution. The above analysis also shows that there are certain differences between design thinking processes and problem-solving processes.

Research on engineering design thinking

Interested in cultivating engineering design thinking in university and secondary school students (Wind, Alemdar, Lingle, Moore, & Asilkalkan, 2019), many researchers have examined the differences in design thinking processes between expert and novice engineers when faced with an engineering problem. For example, Atman et al. (2005), Atman et al. (2007), and Lammi and Becker (2013) used engineering problems such as the design of a playground for a fictitious neighborhood and the construction of a ping-pong-ball launcher to investigate the engineering design thinking of engineering experts and students. The most important findings of these studies are as follows: (1) expert engineers ask many more questions than students do during the problem-definition step, and some of these questions (15%) are highly specific and based on close scrutiny; (2) during the information-gathering step, expert engineers collect vast amounts of differentiated data, whereas beginners gather only small amounts of localized data and spend a relatively small amount of time on this step; and (3) during the decision step, expert engineers use mathematical calculations and theoretical methods to support their decisions, whereas beginners tend to rely more on intuition.

In a study on the engineering design processes of expert engineers, high school freshmen, and high school seniors, Atman et al. (2007) found that experts spend significantly more time on the information gathering, feasibility analysis, evaluation, and decision-making steps than high school students do. They also found that high school seniors spend more time on idea generation, feasibility analysis, and decision-making than high school freshmen do. Thus, this study revealed differences between experts and high school students in engineering design processes. Although the study focused mainly on analyzing the amount of time spent on each step of the engineering task, it quantified some of the differences between experts and high school students in engineering design processes. In studies about technology and engineering teachers, Hynes (2012) examined the understanding and teaching of the engineering design process by middle school teachers. The most significant finding of this study was that technology teachers often display sophisticated thinking during two particular steps of the engineering design process—constructing a prototype and redesigning.

Overall, expert engineers usually spend more time on each step of the design process than students do; technology teachers generally apply complex thinking while creating prototypes during redesign; and experts ask more questions, perform more research, and conduct more testing than students. The implication is that experience leads to a more mature engineering design thought process. It is important to pay attention to these findings when nurturing future engineering design talent. Despite these related studies, it is not enough for research on engineering design thinking to explore the engineering design process of expert or novice engineers. If we wish to empower teachers to feel more confident in developing and delivering robust engineering-related curricula, we must first explore teachers’ existing cognitive structures. Therefore, the definitions, system concepts, examples, and advanced system-concept explanations of the engineering design process are explored in this study.

STEM project-based learning combined with engineering design

Many engineering design studies have considered how research findings can be converted into tools for engineering pedagogy to provide an education that nurtures the cognitive skills of students and their ability to fuse theory and practice (Borgford-Parnell, Deibel, & Atman, 2010). For the purpose of this study, STEM project-based learning combined with the engineering design process (EDP-STEM-PBL) may be described as a mode of pedagogy that purposefully situates scientific and mathematical knowledge within the context of technological design to create a problem-solving learning environment in which students envision solutions to design challenges, gather information, and solve real problems through the use of the engineering design process (Sanders, 2009; Wahono, Lin, & Chang, 2020). This approach is expected to improve student competitiveness in the burgeoning knowledge economy. STEM education has already been emphasized in education systems in the USA. Bybee (2013) stressed that integrative STEM education should pay attention to globally important issues (e.g., climate change, energy sources) and develop design capabilities through practical technology and engineering activities, paying special attention to theory-based design. The aim of this cross-disciplinary approach is to simultaneously teach rigorous academic concepts and provide experiential, real-world learning opportunities. Therefore, EDP-STEM-PBL can be used to systematically cultivate the scientific, technological, mathematical, and engineering knowledge of students through the engineering design thinking process, thus expanding students’ perspectives and mitigating the lack of practicality in conventional pedagogy, which tends to overemphasize theoretical learning. EDP-STEM-PBL repurposes engineering design studies into pedagogical tools for teaching engineering design.

In conventional pedagogy, intuition is often used to solve problems. However, applying analytical strategies and explicit step-by-step processes to ordinary problems often results in better solutions (Baumann & Kuhl, 2002). Therefore, EDP-STEM-PBL calls for the formation of learning groups that take full responsibility for their learning, and learning goals are achieved through cooperation and sharing among team members (Milentijevic, Ciric, & Vojinovic, 2008). In this study, the engineering design process proposed by Atman et al. (2007) was used to develop STEM project-based learning activities, with an emphasis on the steps of prototype construction and redesign, as suggested by Hynes (2012). In this way, we hope to cultivate the engineering design skills needed by preservice technology teachers and to facilitate sophisticated thinking skills in preservice teachers.

Research methods

Research design

To investigate the effects of EDP-STEM-PBL on the engineering design thinking of preservice technology teachers, we used a pretest-posttest nonequivalent groups design (see Table 1). The STEM project used in this study was the mousetrap car. All preservice technology teachers received 12 periods (50 min/period) of mousetrap car activity training courses over 6 weeks. The courses were intended to improve the preservice teachers’ understanding of STEM knowledge about mousetrap cars, e.g., friction, Newton’s first law of motion, material processing, engineering graphics, the definition and process of engineering design, and the Pythagorean theorem. They were then asked to design and construct a mousetrap car that could travel more than 10 m on a 1-m-wide track. During the activity, the experimental group was taught using an EDP-STEM-PBL curriculum, and the control group was taught using a STEM-PBL curriculum based on the technological problem-solving process (PS-STEM-PBL). The differences between the two curricula are described below.

Table 1 Experimental design of the study

Since this study focused mainly on the cognitive structure used by preservice technology teachers in engineering design thinking, this study followed Tsai and Huang’s (2002) suggestion in using the flow-map method to explore preservice technology teachers’ cognitive structure and avoid the limitations of free word association, controlled work association, tree construction, and concept mapping. The research subjects (preservice technology teachers) were interviewed before and after the experimental teaching course; that is, for the pretest, the preservice teachers were interviewed before the 12 periods of mousetrap car activity training courses, and for the posttest, they were interviewed after the training courses. The flow-map method was then used to analyze their cognitive structures in engineering design thinking, enabling us to identify how the preservice technology teachers developed and changed their cognitive approaches to engineering design thinking through the experimental curriculum. Finally, the learning performance of the preservice technology teachers in the experimental and control groups was examined by employing the flow-map method to analyze the engineering design thinking of the research subjects from each group. In this way, our study provided insight into how different curricula affected the actual performance of preservice technology teachers in a STEM-PBL environment. We used these findings to develop recommendations for EDP-STEM-PBL pedagogy that can serve as a reference for teaching science and technology in the future.

There were similarities and differences between the “engineering design process” and “problem-solving process” curricula as taught to the experimental and control groups.

Curriculum

The researcher was responsible for designing and teaching the EDP group and PS group curriculum. This curriculum is one of the major units of “Introduction to Industrial Technology Education,” but the students’ performances do not influence their final grades. The researcher introduced the background of this study and the definition and application of the engineering design process to both groups to enable them to understand the importance of applying the EDP in activity. The experimental group’s engineering design process curriculum (EDP-STEM-PBL) included such engineering design processes as modeling, feasibility analysis, and group communication; the curriculum began with information gathering according to the problem definition, followed by feasibility analyses based on the problem constraints and then the selection of a solution and construction of a prototype. In contrast, the control group’s technological problem-solving process included problem definition, data collection, feasible idea development, best idea selection, best idea implementation, results evaluation, and design idea revision. The process began with the development of knowledge and problem-solving skills that created a link between the problem and the students’ cognitive structures; this was followed by experimental analysis to verify the students’ hypotheses.

Problem framework

In the experimental group (EDP-STEM-PBL), the problem features were described in a qualitative and highly detailed manner to encourage a search for the most feasible solution. Additionally, this curriculum provided an integrated, cross-disciplinary learning experience (science, technology, engineering, and mathematics) to help the subjects develop high-level cognitive abilities (e.g., design, innovation, and critical thinking) in STEM areas. In the control group (PS-STEM-PBL), the problems were addressed using existing experience, concepts, and techniques in conjunction with a variety of ideas, leading the subjects to find and implement solutions appropriate to the current problem.

Research subjects

The subjects of this study were 28 freshmen/preservice technology teachers who were being trained at the National Taiwan Normal University to specialize in technology education. They had similar engineering-related experience in their pre-engineering technology education courses at the senior high school level due to the national technology curriculum guidelines. All preservice technology teachers must take a required course named “Introduction to Industrial Technology Education.” To ensure that they understood the importance of hands-on learning, a mousetrap car activity was arranged in this course as a key technology-learning activity (see Fig. 1). Fifteen preservice technology teachers were randomly assigned to the experimental group and taught using an EDP-STEM-PBL curriculum, and the thirteen remaining teachers were randomly assigned to the control group and taught using a PS-STEM-PBL curriculum. The National Taiwan Normal University (NTNU) was selected as the site of this study because it was mainly a secondary school technology teacher training university before it diversified its teacher development programs. The NTNU has a long history of teacher education, and it is presently the main source of secondary school technology teachers in Taiwan. Hence, conducting our study at this university could directly improve teacher education curricula and help equip preservice teachers with engineering design thinking capabilities.

Fig. 1

Example of a mousetrap car

Research tools

In representing individual cognitive structure, the flow-map method may be the most useful method for representing the cognitive structure (Tsai & Huang, 2002). Anderson and Demetrius (1993) argued that the flow-map method requires minimal intervention by the interviewer and little inference in its construction (Wu & Tsai, 2005). In this study, we conducted in-depth, semistructured interviews to enable us to use the flow-map method to characterize the cognitive structures of preservice teachers in terms of engineering design thinking. After the audio-recorded interview, a “metalistening” technique was followed for the purpose of exploring the preservice technology teachers’ additional conceptual knowledge. The researchers could immediately replay the interview recordings to provide an opportunity for the preservice technology teachers to recall additional concepts of engineering design that they had not previously disclosed. The preservice technology teachers’ responses to the metalistening technique were also audio recorded by a second recorder. Thus, the interviews could be transcribed verbatim to produce a flow map representing the cognitive structures of the preservice technology teachers in this study (Wu & Tsai, 2005). The interview questions in this study were largely based on the questions designed by Tsai and Huang (2002) for in-depth, semistructured interviews. We used the following questions to probe the subjects’ cognitive structures about engineering design thinking: (1) Could you tell me whether you have used the engineering design thinking process to solve an engineering problem? (2) Please tell me more about the concepts you have just mentioned. (3) Could you explain how the concepts you mentioned are connected to each other? (4) Do you have anything to add to that explanation? To ensure that this research tool was effective, we drafted the interview questions according to the research tools of other, related studies; invited experts on the flow-map method to check our questions; and revised our questions accordingly.

Data analysis

Both quantitative and qualitative data analyses were performed in this study. First, we drafted a flow map based on the interview recordings, and then, after discussion and analysis of the flow map through consensus evaluation, we conducted a quantitative analysis of the flow map to identify gaps and weaknesses in the cognitive structures of preservice technology teachers in engineering design thinking. The data analysis steps of the flow maps were as follows: (1) we interviewed the preservice technology teachers and produced the interview recordings, coupled with the metalistening technique; (2) the interviews were transcribed verbatim into flow maps based on the process proposed by Wu and Tsai (2005); and (3) two researchers adopted the consensus evaluation method in analyzing the flow maps and producing the quantitative data of the preservice technology teachers’ performance in engineering design thinking, which included the following criteria: (1) defining engineering design thinking: a correct definition is assigned a value of 1, and an incorrect or no definition is assigned a value of 0; (2) overall system concepts in engineering design thinking: two researchers discussed the flow maps, decided which steps had been mentioned, and calculated the total number of steps; (3) individual system concepts in engineering design thinking; the two researchers discussed the flow maps and decided which steps had been mentioned. Steps that were mentioned are assigned a value of 1; otherwise, the value is 0; (4) providing examples of engineering design thinking: providing correct examples is assigned a value of 1, and incorrect or no examples are assigned a value of 0; (5) explanations of advanced system concepts (overall): the two researchers discussed the flow maps, decided which steps had further explanations and calculated the total number of steps; (6) explaining system concepts (individual): the two researchers discussed the flow maps and decided which steps had further explanations. Steps that were mentioned are assigned a value of 1; otherwise, the value is 0. By applying the consensus evaluation, the two researchers could come to an exact agreement on the preservice technology teachers’ performance in engineering design thinking and share a common interpretation of the construct (Stemler, 2004).

In summary, the interviews were transcribed verbatim, and a flow map (see Figs. 2, 3, and 4) was drafted using this verbatim transcript (Tsai & Huang, 2002). To analyze the flow maps, the two researchers involved in this work conducted a consensus evaluation on the flow maps and discussed the verbatim transcripts of the preservice technology teachers’ interviews in their flow maps to assess the subjects’ performance in relation to definitions, system concepts, examples, and advanced system-concept explanations of engineering design thinking and generate quantitative data. “Definitions” refers to the ability of a preservice technology teacher (PTT) to clearly explain the meaning of the engineering design process. “System concepts” refers to a PTT’s procedural knowledge (i.e., ability to explain each step of the engineering design process). “Examples” refers to the ability of a PTT to provide examples of each step in the engineering design process and the application of these steps. “Advanced system-concept explanations” refers to the ability of a PTT to use correct conceptual knowledge to explain how the engineering design process is used to solve a problem. In addition, to analyze the differences between the experimental and control groups, we conducted a chi-square test to compare their cognitive structures and suppress differences before the experiment.

Fig. 2

ED9 flow map: pretest interview

Fig. 3

PS6’s flow map: pretest interview

Fig. 4

PS6 flow map: posttest interview

Results

In the following sections, we describe the results of this study in terms of the definitions, system concepts, examples, and advanced system-concept explanations of the preservice technology teachers as they relate to the engineering design thinking process.

Preservice technology teachers’ performance: defining engineering design thinking

As shown in Table 2, the number of preservice technology teachers in the engineering design group (experimental group) who were able to define engineering design thinking increased from two in the pre-experiment test to eight in the post-experiment test. However, the number of preservice technology teachers in the problem-solving group (control group) who were able to provide a definition of engineering design thinking increased only from four in the pretest to eight in the posttest.

Table 2 Preservice technology teachers’ performance: defining engineering design thinking

Preservice technology teachers’ performance: system concepts in engineering design thinking

Overall system concept

An analysis of the performance of the preservice technology teachers in terms of their overall system concepts of engineering design thinking is shown in Table 3. A chi-square test indicated that there was a statistically significant difference between the engineering design group (experimental group) and problem-solving group (control group) in this regard.

Table 3 Preservice technology teachers’ performance: overall system concepts in engineering design thinking

Individual system concepts

An analysis of the performance of the preservice technology teachers in each system concept of engineering design thinking is shown in Table 4. According to the chi-square test, there were statistically significant differences between the experimental and control groups in four system concepts: problem definition, idea generation, modeling, and feasibility analysis.

Table 4 Preservice technology teachers’ performance: individual system concepts in engineering design thinking

Preservice technology teachers’ performance: providing examples of engineering design thinking

As shown in Table 5, the number of preservice technology teachers in the experimental and control groups who were able to provide examples of engineering design thinking increased from 5 and 3 in the pre-experiment test to 15 and 10 in the post-experiment test, respectively. The chi-square test indicated that there was a statistically significant difference between the experimental and control groups.

Table 5 Preservice technology teachers’ performance: providing examples of engineering design thinking

Preservice technology teachers’ performance: explanations of advanced system concepts

Ability to provide explanations for advanced system concepts

An analysis of the abilities of the preservice technology teachers to provide explanations for advanced system concepts is shown in Table 6. According to the chi-square test, there was a statistically significant difference between the experimental and control groups in this aspect.

Table 6 Preservice technology teachers’ performance: explanations of advanced system concepts (overall)

Ability to provide advanced explanations for each system concept

Table 7 shows an analysis of the preservice technology teachers’ ability to provide advanced explanations for each system concept. The chi-square test showed that there was a statistically significant difference between the experimental and control groups in terms of their ability to provide advanced explanations for idea generation.

Table 7 Preservice technology teachers’ performance: explaining system concepts (individual)

Discussion

This section discusses a few important topics in further depth in light of the above results. The topics include the findings of Hynes (2012), who found that technology teachers tend to focus on prototype construction and redesign when they teach engineering design, and whether EDP-STEM-PBL helps to improve the thinking capabilities of preservice teachers in these steps. In addition, we discuss whether preservice teachers are able to exhibit sophisticated thinking during the engineering design thinking process.

Does EDP-STEM-PBL improve preservice technology teachers’ performance during modeling?

In the engineering design process proposed by Hynes (2012), the most important step of the process for preservice teachers is prototype construction. Hynes (2012) further noted that preservice teachers should be able to construct ideas and perform calculations. Idea construction refers to the ability of a preservice teacher to evaluate the feasibility of a model and deeply understand the scope of effects brought about by a modification to the model after the idea generation step in EDP-STEM-PBL. The ability to perform calculations refers to the ability of a PTT to come to terms with the complex or custom dimensions of the prototype constructed according to the idea generation, including the ability to understand the characteristics and measurements of the prototype. In the following sections, we will describe how the preservice teachers in this study reflected on their learning after they were taught the EDP-STEM-PBL curriculum.

The prototype construction step was described by Hynes (2012) as the construction of a working model of an original idea. During this step, an idea may be further developed according to the features of the prototype, and the feasibility of some desired change to the prototype’s specifications may also be assessed. The performance of the preservice technology teachers during the prototype construction step (as per the definition above) is shown in Table 4. When we performed an analysis on the individual system concepts that are key for project-based learning, we found statistically significant differences between the experimental and control groups. This indicated that the preservice teachers in the experimental group were able to clearly and correctly explain procedural knowledge of prototype construction. It also showed that these teachers tended to think that the problems that occur while generating ideas and performing calculations affect the quality of the final implementation. In other words, most of the preservice technology teachers in the experimental group tried to improve their designs after evaluating the prototype they constructed according to the objectives and standards, having gained insights that enabled them to refine the final implementation. In contrast, the performance of the control group in terms of implementation quality was directly affected by the problem-definition and information-gathering steps. The control group was unable to attain the same high level of performance as those using the engineering design process (experimental group) during prototype construction.

Does EDP-STEM-PBL improve preservice technology teachers’ performance during redesign?

In Hynes’s (2012) definition of the engineering design process, redesign refers to the learners’ ability to understand the goals of a project and to realize that the project implementation does not have to be perfect after the first iteration; it is also a measure of the learners’ ability to learn from mistakes and validate their designs. Explanations of redesign may be categorized as in situ explanations, review explanations, and advanced explanations. In situ explanations refer to explanations of redesign provided by a teacher while the student is learning how to build a project (e.g., modifications and suggestions). Review explanations refer to guidance provided by teachers to students after the students have already finished most of the redesign. Advanced explanations refer to advanced interpretations of the redesign, such as designers retracing their work to the problem-definition step (e.g., to identify newly recognized needs or problems) or returning to the modeling step (e.g., to improve their models for a new version of the design). The redesign step can be thought of as preservice technology teachers fundamentally rethinking the engineering design process and overhauling their designs.

As shown in Table 4, the performance of the preservice technology teachers in the experimental group (N = 4, 27%) was better than that of the preservice technology teachers in the control group (N = 1, 8%). This implies that the preservice technology teachers who were taught using the EDP-STEM-PBL curriculum were better than those in the control group at implementing their projects using the engineering design process in an orderly manner. Based on the results of the interviews, teachers should use in situ and review explanations to help students understand redesign and thus expound on the contents of and knowledge within the curriculum. Furthermore, teachers should highlight the actions that must be taken by the students and prevent them from taking ineffective actions. With the teachers’ guidance, the students should think of the engineering design process as a whole instead of rushing to complete the project. Furthermore, students should be guided through advanced explanations to return to the problem-definition or modeling step during redesign to think about why it may be necessary to revise or improve their original design. The overall results suggested that pedagogy based on the engineering design process can significantly improve the redesign capabilities of preservice technology teachers in STEM-PBL.

Does EDP-STEM-PBL improve preservice technology teachers’ ability to use sophisticated thinking?

The cultivation of sophisticated thinking in the engineering design process can be discussed from two perspectives: the ability to provide examples (Table 5) and advanced system concept explanations (Table 6) about the engineering design process. Our analyses showed that the experimental group was superior to the control group in providing examples of engineering design thinking. This suggests that EDP-STEM-PBL helps cultivate the ability of preservice technology teachers to think divergently or convergently. Atman et al. (2007) noted that expert engineers are skilled at making decisions about engineering-related concepts because they have absorbed a broad and diverse range of engineering concepts. Based on their working experience in various jobs, professional engineers are able to easily and accurately evaluate components of the engineering design process (e.g., problem descriptions, prototypes, work plans, independent completion time). They are highly adept at applying their metacognitive skills and understanding engineering design thinking concepts.

The aforementioned qualitative analyses also showed that the experimental and control groups were both unable to provide advanced system concept explanations during project implementation. This is somewhat inconsistent with the findings of previous studies. For example, in a study in which freshmen and senior engineering students were asked to perform an engineering task, Atman et al. (2005) found that the seniors had a higher number of transitions, greater progression to later stages, and longer design times than the freshmen (problem-solving time is an important factor, as it encompasses the cognitive structures of advanced system concepts in engineering design thinking). West, Fensham, and Garrard (1985) noted that “cognitive structures” generally consist of two important components: the knowledge stored within the conceptual structure and the organization of this knowledge. It is possible that the preservice technology teachers in this study were not able to fully express their advanced system-concept explanations during the STEM-PBL curricula, thus resulting in a deviation from the findings of previous studies. In our opinion, the preservice technology teachers in the experimental and control groups may have spent too much time on problem definition and were slow to transition to developing alternative solutions and project implementation. Atman et al. (2005), who used similar teaching materials, found that freshmen and senior engineering students both spent large amounts of time on the problem-definition step, which includes the time spent reading and describing the problem. Therefore, we believe that the preservice technology teachers who were taught using EDP-STEM-PBL were highly proficient in presenting basic descriptions and definitions of engineering design thinking. However, they were far less proficient in the use of high-level deductive reasoning and dialectical thinking, as they generally preferred to use basic definitions, descriptions, and conceptual explanations rather than exhibiting higher-level skills. It is possible that the preservice technology teachers who were taught with the EDP-STEM-PBL curriculum will gradually transition to higher-level skills as they become more familiar with engineering design process pedagogy.

Research limitations

This study focuses on the cognitive structure of preservice technology teachers in engineering design to probe their understanding of engineering design thinking. However, a self-criticism of this study reveals that it was subject to the following limitations. The utilization of the flow-map method with the metalistening technique is very time consuming, making it difficult to increase the number of research subjects. That is, the external validity (generalizability) of this study is the major limitation.

Another limitation is the statistical approach to exploring the effect of applying the engineering design process to STEM project-based learning in developing preservice technology teachers’ schema of design thinking. If the number of research subjects had exceeded 30 preservice technology teachers in each group, we could have utilized analysis of covariance (ANCOVA) in this study. However, there were only 28 preservice technology teachers in this study; thus, we could utilize only the chi-square test to compare their cognitive structures and suppress differences before the experiment.

Finally, we arranged 1 h of interview time for each preservice technology teacher for the purpose of controlling the interview time. That is, the subjects had to express what they wanted to say within 30 min and then listen what they had said and explain what they had forgotten to say for the remaining 30 min. In the pretest interview, this amount of time was sufficient. However, for the posttest interview, this amount of time was sometimes insufficient. Very few preservice technology teachers were anxious to extend the interview, even when we asked them to feel free to finish what they wanted to say.

Conclusions

The main focus of this study was the effectiveness of EDP-STEM-PBL in cultivating the cognitive structures of preservice technology teachers with respect to engineering design thinking. Based on the experiment, we analyzed the effects of EDP-STEM-PBL and compared the results of the experimental and control groups. The results of the experimental teaching activity are as follows:

EDP-STEM-PBL improved problem definition, idea generation, and engineering design thinking

First, preservice technology teachers who were taught using the engineering design thinking process (the experimental group) performed better than the control group in problem definition, idea generation, modeling, and feasibility analysis. In this study, 28 preservice technology teachers were taught from a STEM-PBL curriculum based on the mousetrap car project. The number of preservice technology teachers who were able to describe the problem increased from two (13.3%) to eight (53.3%) in the experimental group and from four (30.8%) to five (38.5%) in the control group. Hence, the ability of the experimental group to define the basic problem (i.e., the ability to clarify the scope and context of the problem) improved significantly after EDP-STEM-PBL. This is consistent with the findings of Atman et al. (2007), whose subjects were asked to design a playground: expert engineers generally read a problem several times and ask for the specifics to be repeated, which helps them identify the constraints of the problem, reconstruct the problem, and summarize effective ideas. In other words, the experimental group was better able to define how the problems of the project activity were linked to the goals of the project after they were taught the EDP-STEM-PBL curriculum.

Atman et al. (2007) noted that expert engineers spend large amounts of time on the engineering design thinking process. However, during the teaching experiment in this study, we found that the subjects either used their intuition or convergent/logical thinking to solve the problem after the steps of problem definition, decision-making, and objective confirmation; this is likely to be a crucial factor in determining the results of the experiment. Furthermore, by analyzing the performance of the experimental group in further detail, we found that the experimental group was better at estimating the influence of each factor during the modeling process and thus produced the best solutions; members of the experimental group were subsequently able to confirm whether a solution met the criteria set by the problem definition and review the general applicability of their solutions. This result is consistent with the findings of Atman et al. (2007), who found that the experimental group was able to determine the relevant causal relationships of a problem and discover the most important factors for solving the problem; on this basis, they identified the focal points of the problem, performed feasibility analyses, reviewed the constraints, and determination criteria of the problem, and provided simple explanations of the results that could be produced by further analysis. Therefore, it is our opinion that teaching goals should be cross-evaluated and explained in advance; additionally, more time should be dedicated to problem scoping and the development of alternative solutions. Furthermore, more effort should be dedicated to feasibility analysis to shorten the time needed by preservice technology teachers to solve engineering problems. On this basis, we expect the incorporation of engineering design process pedagogy to strongly improve teaching effectiveness.

Prioritize procedural learning in modeling and redesign for preservice technology teachers in STEM project-based learning

This approach can improve preservice technology teachers’ capacity for sophisticated thinking. Hynes (2012) noted that the prototype construction and redesign steps of the engineering design process affect the development of sophisticated thinking. Our results showed that the preservice technology teachers who received engineering design process pedagogy outperformed the control group in the prototype construction and redesign steps. Wu and Tsai (2005) proposed that constructivist science activities could provide opportunities for “cognitive apprenticeship,” as these activities allow for the application of metacognition and high-level information-processing strategies to the organization of cognitive structures. Furthermore, during these learning activities, students are provided with opportunities for expression, communication, and consultation, which improve their cognitive structures in engineering design thinking. Therefore, we suggest that the engineering design process be incorporated through engineering-related, project-based learning activities. This will help preservice technology teachers formulate improved designs using their engineering design knowledge and improve their cognitive structures in engineering design thinking.

In this study, the flow-map method was used to acquire and analyze the cognitive structures of preservice technology teachers; quantitative and qualitative analyses were also performed on the collected data. We found that the preservice technology teachers who were taught the engineering design process significantly improved their performance in prototype construction and redesign. Similarly, Atman et al. (2007) found that expert engineers are skilled at making decisions about engineering-related concepts because they understand a broad range of engineering concepts, and their experience enables them to easily and accurately evaluate various aspects of the engineering design process (e.g., problem descriptions, prototypes, work plans, independent completion time). Successful professional engineers are highly adept at applying their metacognitive skills and understanding of engineering design thinking concepts. We believe that incorporating the engineering design process into the training of preservice technology teachers is beneficial for refining their cognitive structure in engineering design thinking. However, the design of these teaching courses should be based on the progress of preservice technology teachers in engineering-related courses to improve their capacity for sophisticated thinking.

Suggestions for future research

One of the main advantages of engineering design process pedagogy is that it cultivates sophisticated thinking. However, there are many difficulties in implementing this mode of pedagogy (Linder, 1999). Many teachers feel that the obstacles to its implementation are too severe, and the issue most frequently cited is insufficient time. If the weaknesses of preservice technology teachers can be discovered through preclass investigation, and a few elements of engineering design thinking pedagogy can be added to address these weaknesses, this approach will undoubtedly enhance teaching effectiveness without wasting too much time on trial and error. Furthermore, as EDP-STEM-PBL involves several skills that are generally applicable to other tasks (Atman et al., 2005), improving the problem scoping and development capabilities of preservice technology teachers will also improve their ability to devise alternative solutions. Engineering design activities prepare preservice technology teachers for realistic problems, provide opportunities for practice, and improve their abilities in various areas, including their ability to integrate knowledge. Therefore, designing learning activities that are relevant to real-life will cultivate practical knowledge and problem-solving capabilities in preservice technology teachers and improve their ability to address real-life subjects.