- Open Access
An investigation of verbal episodes that relate to individual and team performance in engineering student teams
© The Author(s). 2019
- Received: 4 June 2018
- Accepted: 21 January 2019
- Published: 7 February 2019
Being an effective team member is one of the key twenty-first century skills and a fundamental proficiency required for jobs and work settings in an increasingly global economy. Collaborative problem-solving in team settings is a critical practice in science, technology, engineering, and math (STEM) fields as an effective teaching method that is found to promote outcomes associated with individual student learning and the quality of team solutions. However, the social and discursive dynamics of a team can impact these outcomes. The primary goal of this study was to examine the different types of verbal episodes (questions, conflict, and reasoning episodes) in engineering student teams and how these verbal interactions related to the individual and team performances.
Our results showed that different verbal episodes played a significant role on students’ individual success and team performance. Students spent most of their time on question episodes, followed by reasoning episodes, and less frequently so on conflict episodes. The linear combination of question, conflict, and reasoning episodes was significantly related to students’ individual achievement scores. Specifically, the number of question episodes was the best predictor of individual performance. And among six subcategories of question episodes, three (open question, verification answer, and elaborated answer) were significantly correlated with the individual achievement scores. Furthermore, according to the regression model, elaborated answer category was significantly related to individual achievement compared to all other categories. In terms of team performance, results showed that reasoning episodes were significantly related to team performance, and the calculation subcategory of reasoning episodes was the best predictor of team performance. Finally, we found that the teams with more balanced participation among team members performed significantly better than the teams with unequal participation among team members.
These results suggest different verbal episodes are related to individual and team outcomes. Discourse in teams is critical not only for the improvement of academic achievement for individuals but also for the development of better team solutions. Moreover, balanced participation from all team members is important, suggesting teams should be formed, monitored, and supported to ensure balanced participation. Our results recommend that educators should monitor team interactions and promote verbal exchanges that promote student learning and positive team outcomes.
- Team-based learning
- Discourse analysis
- Verbal episodes
- Engineering education
- First-year engineering
Teams play a critical role in organizations, companies, and academic institutions. As businesses and organizations become increasingly global and complex, the required tasks, challenges, and solutions also become more team-based. As a result, engineers are often expected to work in team projects (Borrego, Karlin, McNair, & Beddoes, 2013; Cohen & Bailey, 1997; Dossick & Neff, 2011). They need to communicate with other team members as they exchange their knowledge and insights, give and receive feedback, and negotiate to reach decisions. Similar to other STEM (science, technology, engineering, mathematics) professionals, engineers are required to have a wide range of technical and professional competencies to solve problems collaboratively and efficiently in their work settings. Not surprisingly, communication is one of the most critical skills that many engineers count as important (Katz, 1993; Passow, 2012; National Association of Colleges and Employers, 2014). In fact, engineers spend a significant amount of their work time on communication (Darling & Dannels, 2003). Accordingly, the Accreditation Board for Engineering and Technology (ABET (2017) state that engineering graduates must have “… (d) an ability to function on multidisciplinary teams, (e) an ability to identify, formulate, and solve engineering problems, …(g) an ability to communicate effectively.” Similarly, a survey from more than 5000 engineers reported that teamwork, communication, and problem-solving were the most important skills for engineering in the workplace (Passow, 2012).
To ensure that engineering students develop these competencies, engineering schools have implemented approaches and developed curriculum that promote problem-solving skills in teamwork settings (Anwar, Menekse, Heo, & Kim, 2018; Dym, Agogino, Eris, Frey, & Leifer, 2005; Felder & Brent, 2003; Froyd, Wankat, & Smith, 2012). However, solving problems in a time-constraint situation and often with insufficient data is challenging even for experienced engineers (Katsikopoulos, 2012; Sha & Panchal, 2014). Thus, the problem-solving processes can become even more challenging for team members when they need to make decisions about ill-structured problems. In order to understand what kind of challenges students experience in team settings, there is a need for studies that analyze the process data (or verbal data) while students are working in teams.
How do different types of verbal episodes relate to individual student achievement?
How do different types of verbal episodes relate to team performance on an engineering design project?
How does the distribution of individual contributions among team members relate to team performance?
Collaborative learning and problem-based learning
Recent reform efforts and reviews of high-quality classroom instruction highlight the role of collaboration and student learning (Chi & Menekse, 2015). Recommendations incorporate collaboration during authentic disciplinary activity, which in STEM fields involve argumentation and evidenced-based explanation. Engaging in collaborative groups provides cognitive benefits and deepens conceptual understanding given opportunities to explain, question, justify, and negotiate, with benefits for learning and achievement outcomes. However, there are challenges to obtaining meaningful collaboration, which translate into group variation in learning outcomes. There are challenges to maintaining joint attention and on-task behavior, equitable participation among group members, coordination and regulation of common goals, and responding to disagreement by legitimizing alternative perspectives (Menekse, 2012; Menekse & Chi, 2018).
Collaborative learning is often crucial for achieving conceptual knowledge while solving problems (Van Boxtel, Van der Linden, & Kanselaar, 2000). In the study of Van Boxtel et al. (2000), the authors defined the conceptual knowledge as a learning process where students embellish, organize, and refine the knowledge and develop the ability to use scientific concepts. They argued that collaborative learning settings for team projects facilitate elaborating conceptual understanding.
The collaboration among team members is also the main structure for the problem-based learning (PBL) activities (Hmelo-Silver, Chinn, Chan, & O’Donnell, 2013). PBL is one of the examples that facilitate active learning as students construct knowledge by themselves and solve real-world problems based on their experiences (Barrows, 1986; Cockrell, Caplow, & Donaldson, 2000; Hmelo-Silver, 2004; Jonassen & Hung, 2008; Quinn & Albano, 2008; Savery, 2006; Streveler & Menekse, 2017; Wood, 2006). Barrows’ (2002) definition of PBL included that the learning characteristically occurs in a small group, face to face, with active discussion among the learners. Advocates of PBL point out that “soft skills” such as collaboration and self-directed learning can be acquired through problem-based learning experiences (Hmelo-Silver, Duncan, & Chinn, 2007). Barron (2000) observed that the success of teams of middle school students working on a math-focused problem-solving activity hinged upon collaborative behaviors (or the lack thereof). We might therefore surmise that PBL both depends upon and teaches collaborative work skills.
Problems that are used in PBL settings are ill-structured which means they are vaguely defined and can have multiple solutions. This can especially be challenging for students who do not have much prior knowledge related to the given task, as they often do not know where to start. Thus, collaborative learning is one of the fundamental elements of PBL since students can reduce cognitive load by sharing each team member’s expertise via verbal interaction (Cockrell et al., 2000; Hmelo-Silver et al., 2013). In other words, the interaction with team/group members is a key resource to solve tasks much more efficiently.
Simply having students work in teams, however, is not sufficient to access the benefits of collaborative learning in PBL as the nature of collaborative learning is a complicated mechanism. In fact, some previous research studies found the cases where collaborative learning was not always productive (Kuhn, 2015; Pai, Sears, & Maeda, 2015). For instance, Pai et al. (2015) reported the difference on students’ performance such as learning transferability in their meta-analytic study that investigated the effect sizes of teamwork from previous studies. Chi and Menekse (2015) conducted a research study to explore the reason that discrepancy occurred among effect sizes about collaborative learning in small groups. They studied dialog patterns of dyads and found that students were engaged in three different types of dialog patterns as constructive, active, and passive based on the ICAP framework (Chi, 2009; Menekse, Stump, Krause, & Chi, 2013). Chi and Menekse (2015) argued that students could promote learning when both partners are constructive. On the other hand, there was no benefit of collaboration, beyond individual learning, when partners in dyads were either passive-active, passive-constructive, or active-active.
Educators and administrators in engineering programs have also adapted PBL approaches with team projects in their courses (Prince, 2004; Yadav, Subedi, Lundeberg, & Bunting, 2011). Yadav et al. (2011) compared the effect of PBL on team learning versus traditional lectures on individual learning. They collected data from electrical engineering undergraduate students for their conceptual understanding of certain concepts and perceptions of learning. They found that students’ performance was significantly improved during the two PBL phases and the difference, was vastly greater compared to when students were introduced to the traditional lectures. Accordingly, PBL can be suitable to train engineering students to solve ill-structured problems that they would face in the real work settings (Christensen & Schunn, 2007; Dixon & Johnson, 2011).
Understanding the team interaction is important to design effective engineering course curricula with PBL as it can affect both individual achievement and team performances. Therefore, researchers should investigate team discourse more thoroughly that could hinder or promote successful teamwork as well as individual performance. However, we observed that discourse data have not been investigated as much in the engineering education field regarding team work.
The role of verbal discourse in teams
One key process in engineering is collaboration. Successful collaboration requires effective communication, coordination, trust, and respect among team members, as well as individual and team level responsibility and accountability to achieve certain tasks. Literature has shown the significant role of peer interactions and verbal communication for knowledge construction (Goldman, 2018; Mezirow, 1997). However, some studies have shown that achieving successful collaboration is challenging, and working in small groups is not always beneficial in terms of group performance and individual learning (e.g., Chi & Menekse, 2015; Barron, 2003). Dillenbourg and Hong (2008) argued that the lack of the elaborated explanations, mismatch in mutual regulations of cognitive processes between group members, low quality of arguments, and the nonexistence of negotiation of meanings reduces the effectiveness of collaborative learning.
Verbal discourse characteristics in team settings are critical and often strongly relate to the overall team performance (Chinn, O’donnell, & Jinks, 2000). Prior studies have found that types and frequency of verbal discourse during team interaction affect the collaborative learning and decision-making process (e.g., Salter & Gann, 2003). Saab, Joolingen, and Hout-Wolters (2005) investigated the relationship between communicative activities and discovery learning process. They found a significant relationship between these variables. Indeed, team discourse, captured in the form of verbal episodes, can be seen as the most critical set of data that can be used to predict student learning outcomes. Thus, it is important to accurately analyze verbal episodes so that we could have more thorough understanding of team interactions.
A significant number of research studies have been conducted on engineering students’ team processes and team problem-solving. Some of these studies investigated team-related constructs to find how teamwork influenced learning outcomes or other variables such as engagement without analyzing process or verbal data (Fila & Loui, 2014; Hsiung, 2010; Hsiung, 2012; Huang et al., 2011; Mishra, Ostrovska, & Hacaloglu, 2015; Pasha-Zaidi et al., 2015; Terenzini et al., 2001; Zafft, Adams, & Matkin, 2009; Zhou, Luo, Du, & Kolmos, 2010). On the other hand, a few other studies specifically focused on exploring team interactions through discourse analysis by collecting and analyzing audio and/or video-based verbal data (Donath et al., 2005; Haller, Gallagher, Weldon, & Felder, 2000; Kittleson & Southerland, 2004; Ponsa Asensio, Román Jiménez, Arnó Macià, & Pérez Soriano, 2015; Whitman et al., 2005).
For example, Haller et al. (2000) investigated the social dynamics by discourse analysis to find how team members learned from each other in an engineering class. They observed that students were engaged in two types of interaction in group settings: transfer-of-knowledge sequences and collaborative sequences. They also found that problems in team dynamics mostly occurred during transfer-of-knowledge sequences since team members either discouraged other team members’ contribution or had to assume their roles for the task. In another example, Purzer (2011) performed a mixed-methods study to find the relationship between types of discourse actions, self-efficacy, and student achievement. Purzer collected the data from 22 first-year engineering students in an introduction to engineering design course. Purzer conducted the discourse analysis from six teams to explore the verbal exchanges among team members during the project. The results showed a moderate positive correlation between post self-efficacy and the level of engagement in support-oriented discourse.
While some studies focused on verbal interactions of engineering teams, there is still much to be understood about how engineering students learn and interact in team settings. Verbal data contains rich information that is helpful for researchers to understand students’ discourse patterns in team settings. In prior studies of engineering students’ teams, some researchers have studied verbal discourse but did not explore the relationship among the quality of team interaction, team performance, and individual performance except Purzer’s (2011) study. Thus, more studies are needed to explore team verbal discourse to understand how it is related to engineering students’ individual achievement and team level success.
In this present study, we explored how engineering students’ verbal episodes in team settings relate to their individual achievement as well as their team performances on an engineering design project. In addition, we studied how the distribution of individual team member contributions during teamwork affect their team level success.
Data were collected from seventy-three first-year engineering students at a large public university in the United States. Sixty-four of them were male and nine were female students. Three or four students were assigned to teams at the beginning of the semester based on their availability to allow common meeting times. These students continued to work with the same team members on different assignments and projects throughout the semester. There were nineteen teams in total and the age range of students was 18–19. Sixteen of these teams had four student members and three teams had three student members.
Collection of verbal data
All 19 teams were asked to work on a problem-solving assignment requiring analysis and decision on a system that would reduce the energy consumption of a town library with the least cost. Students were provided with three options as installing solar panels, installing a green roof, or making no changes to the existing design. Based on their decision and justification, all teams were responsible to make a recommendation to the client. The solar panels option represented a solution with the least energy consumption. The green roof option created the least carbon dioxide emissions with an added esthetic benefit. And the no change option represented the lowest cost, and thus the most economical option.
Each team was asked to create a document outlining their recommendation and reasoning behind it. Specifically, teams were asked to describe their problem scoping, specify a plan and/or process, explain the formulae for total system cost, construct a graphical representation of ten-year cost for all current and new systems, and finalize the decisions and justifications in 60 min. All teams were video-recorded during this task, so we collected a total of 19 h of video data. They also produced reports indicating calculations, data for each design option, and their decision with a rationale. The task was graded by the instructional team as part of a team exam, and hence students were incentivized to work on the task and do well. This team problem-solving activity happened during week 11 in a 16-week-long academic semester. Prior to this assignment, the same teams also worked on multiple different assignments; therefore, team members had a history of working together on similar assignments. The “Appendix” section shows an exemplary episode of student discussions.
We decided to video record the team problem-solving activity at week 11 in order to use this verbal data as a predictor for future team performances and individual student performances. Also, the team problem-solving activity was a time-limited assignment and each team had to complete all aspects of the assignment within 60 min. And this was the only time-limited team assignment during the semester. Therefore, it provided equal time and resources to all teams to work on the same assignment. For other assignments, teams had opportunities to meet out of class times and to complete before coming to classes. Therefore, we did not have any control or process data on how teams were interacting while working on other team assignments.
Students’ exam scores were used as a measure of their individual performance. The exam covered specific skills needed for the problem-solving task such as analyzing data using Microsoft Excel and were created by the instructional team and graded by teaching assistants with no involvement from the research team. As we described in the “Collection of verbal data” section, students worked on to choose a system that would reduce the energy consumption of a town library with the least cost as a team assignment. Similarly, the individual exam required students to make decisions based on economic and environmental analysis for different scenarios individually. The individual exam was administrated after the team problem-solving assignment that was videorecorded.
All student teams were asked to work on an engineering design project in the last 4 weeks of the semester. This project was different than the problem-solving assignment that we collected video data from all teams. The focus of the project was to design a commuter system for university campus and surrounding city to reduce the energy consumption of the existing system. In addition, projects needed to identify an innovative approach to reducing energy consumption and greenhouse gas emission. Each team was expected to assess the energy efficiency and usability of the current system and propose a more efficient and innovative system that is feasible in 2020. Each team project was evaluated based on six criteria: (1) problem scoping (10 points), (2) criteria & constraints (10 points), (3) concept generation (20 points), (4) concept reduction (20 points), (5) design & value proposition (20 points), and (6) final design (20 points). Each team received a score on a scale of 100 based on these six criteria. Similarly, the final design project was created by the instructional team and graded by instructors and teaching assistants with no involvement from the research team. There was no video or audio data for this engineering design project.
Question-conflict-reasoning coding protocol
The question-conflict-reasoning coding protocol
Asking team members to verify specific information.
Has everyone had a chance to read this?
Asking team members to find and report information taken from text, criteria, or data.
What was the cost for maintenance? How big is the roof?
Asking others for input on the next course of action or for explanations about concepts to fill the lack of understanding.
I do not understand how you got this answer. Can you please explain?
Yes/no answers to verification questions.
Yes, we need to make a decision now.
Reported information or knowledge derived from text, criteria, or data given in response to questions.
The total cost of solar panels is more than the no change option.
Elaborated explanations to answer any type of questions.
The large drop in energy consumption means a significant decrease in carbon dioxide emissions.
Disagreement with a statement or course of action of another. It also includes elicitation of a counter-argument through questioning.
I do not think that’s the final cost though.
Statements that resolve and clarify the disagreements between team members.
There is a difference between these two calculations because the unit price for a solar panel is different based on their size.
Statements that are obtained directly from given text, information, or data.
The solar panels are 8 by 6 ft.
Statements that refer to calculations.
We just need to divide 1500 by 30, and then multiply that by 90 to find the energy cost.
Statements that indicate propositions about the course of action through reasoning or hypotheses.
First, it is better to convert meters to feet because the given numbers are in square feet.
Two raters scored nine team videos individually for verbal episodes. Each rater worked on identifying the number of different verbal episodes in each of these nine videos and then we compared the number of episodes across two raters. Based on the absolute agreement across raters, the intra-class correlation was .93 for the nine videos that both raters scored, which indicates an excellent agreement between raters (Heyman, Lorber, Eddy, & West, 2014). After this step, two raters discussed the differences in their ratings and finalized the scores. The rest of the videos was scored by one of these two raters. The verbal episodes were calculated at the individual level. Hence, each student had a score for each verbal episode and subcategory.
Segments for question episodes begin with a question and end with an answer. Questions could take the form of a statement when a question is implied. However, the episode may extend if a follow-up question is asked regarding or requesting further information about the previous answer. The question episodes were classified into six categories: (1) verification questions, (2) search questions, (3) open questions, (4) verification answers, (5) short answers, and (6) elaborated answers. For example, questions by any team member to their peers that were asking to verify a specific information or an action were coded as verification questions.
Segments for conflict episodes begin with a response to a statement or action that presented conflicting opinions or understanding. These segments generally end when the conflicting opinions or understandings were resolved and agreed upon. The conflict episodes include two categories as conflicts elicited and conflicts elaborated. Statements for the conflicts elicited category display disagreements with a peer’s statement. And, statements for the conflicts-elaborated category refer to the arguments that clarify and conclude the conflicting ideas.
Segments for reasoning episodes begin with a statement concerning a particular topic or concept that conveys useful information for others. The episode was extended when additional statements from the same person or others served to supplement the initial statement. The segment could end in three ways: (1) no responses from others regarding the topic before a new topic is initiated, (2) general agreement among team members, or (3) with a conflicting statement rather than supplementing statement (which begins a “conflict episode”). The reasoning episodes include three categories as observational, calculation, and procedural. The observational category refers to the statements that are inferred from given text, information, or data. The calculation category refers to the statements that include calculation procedures initiated and completed by team members by using given information or data. And the procedural category refers to the statements that indicate propositions about the course of action through reasoning or hypotheses.
The total count, means, and standard deviations for each verbal episode and related subcategories
Mean per student (N = 73)
Distribution of team participation
We were interested in understanding how the distribution of individual team members’ contributions to teamwork relate to their team performances. In other words, how the equal or unequal individual contributions among team members predict team success for the engineering design project. First, we calculated dispersion scores for each team by using total verbal episodes for each team. We used the variance-to-mean ratio formula (VMR), which is a commonly used index for dispersion scores. The actual formula is the ratio of the variance (σ2) to the mean (μ). Accordingly, the relatively lower VMR index indicates a more equal contribution from all team members. And the relatively higher VMR index indicates an unequal contribution from team members. For example, in a hypothetical team with all four team members having equal number of verbal episodes, the VMR index would be equal to zero since the variance would be zero.
Next, we created two groups as homogenous versus heterogeneous teams by using the median split method on the VMR indices. The homogeneous teams refer to the ones that individual team members have relatively similar number of verbal episodes, whereas the heterogonous teams represent the ones that individual team members have relatively different number of verbal episodes. As a result, we created two groups in which one of the groups included relatively more homogeneous teams in terms of contribution to teamwork, and the other group included relatively more heterogeneous teams.
We conducted three sets of analyses. The first set of analyses focused on the relationship between verbal episodes and individual student performance. The second one focused on the relationship between verbal episodes and team performance. And the third one focused on the dispersion of team members’ participation during collaboration and how the dispersion of teams relates to their team performance.
Verbal episodes and individual student performance on exams
We conducted regression analysis to explore how students’ involvement in different verbal episodes predicts their individual exam performance. First, we used three verbal episodes (question, conflict, reasoning) as the predictors, while the dependent variable was students’ individual exam performance. And the linear combination of the three verbal episodes was significantly related to the students’ individual exam scores, F(3, 69) = 2.85, p < .05. The sample multiple correlation coefficient was .33, indicating 11% of the variance of the students’ individual exam scores can be accounted for by the linear combinations of verbal episodes exchanged in their teams.
Next, we conducted stepwise regression to identify the best predictors among the different types of verbal episodes. The results showed that the best model kept the question episodes and excluded conflict and reasoning episodes. In other words, the number of questions asked and/or answered by students was the best predictor for the students’ individual exam scores, F(1, 71) = 8.57, p < .01, R2 = .11.
The bivariate and partial correlation coefficients of the question episode categories with students’ individual exam scores
Verbal episodes and team performance on final project
As described in the methods section, all teams worked on a final engineering design project in the last four weeks of the semester. The focus of the project was to design a commuter system for the university campus and surrounding city to reduce the energy consumption of the existing system. We used team scores on this design project as the outcome measure to evaluate how students’ verbal episodes relate to their success on team design projects. We first conducted correlation analyses, by using three verbal episodes (question, conflict, reasoning) and team scores on the engineering design project. The correlation showed that there was significant correlation between the number of reasoning episodes and team performance, r(19) = .48, p < .05. Next, we conducted a bivariate linear regression by using the reasoning episodes as the predictor and the team score as the outcome measure. The results showed that two variables were linearly related such that the team scores increased with the increase in the number of reasoning episodes as represented with the regression model:
Team Score = 1.48 Reasoning Episodes + 62.36
Furthermore, we were interested in which categories of the reasoning episodes are the good predictors for team performance. We conducted stepwise regression to identify the predictors among observation, calculation, and procedure categories. The results showed the best regression model kept the calculation category and excluded the observation and procedure categories, F(1, 17) = 6.21, p < .05, R2 = .27. In other words, the number of calculation-related episodes was the best predictor for team performance on the engineering design project. Calculations were important to this task as it took a significant amount of the time for students to make the calculations before making a recommendation based on cost and carbon dioxide emissions from different solutions.
Distribution of team participation and team performance on final project
As we described in section 3.4, we were also interested in understanding how the distribution of individual team members’ contributions to teamwork relate to their team performance on the final project. We used the VMR index for the dispersion scores and we created two groups as homogenous versus heterogeneous teams by using the median split method on the VMR indices. We conducted a one-way ANOVA to understand the differences between these two groups based on their team score on the engineering design project. The results showed the homogeneous teams performed significantly better than the heterogeneous teams, F(1, 17) = 5.49, p < .05. The η2 was .24, indicating the dispersion factor accounting for 24% of the variance of the team scores on the engineering design project.
In this study, we explored the different types of verbal episodes (questions, conflict, and reasoning episodes) in engineering student teams, and how these verbal interactions related to the individual and team performances. For the first research question, we investigated how different verbal episodes relate to individual student achievement and the results showed the question episodes were the best predictor among all three verbal episodes. And among all six subcategories for the question episodes, the elaborated answers subcategory was significantly related to students’ individual exam scores. The second research question focused on examining the relationship between verbal episodes and team performance. Results showed the reasoning episodes were significantly related to team performance, and the calculation subcategory of reasoning episodes was the best predictor of team performance. For the third research question, we focused on how the distribution of individual contributions among team members relates to team performance. We found that the teams with more balanced participation among team members performed significantly better than the teams with unequal participation among team members.
We also found students spent most of their time on question episodes, followed by reasoning episodes, and less frequently so on conflict episodes. This finding is similar to Purzer’s (2011) finding that conflict (labeled as challenge-oriented discourse in Purzer’s (2011) study) was the least observed. These results are not surprising since questions are central to students’ learning processes. Prior studies have shown that ability to generate questions and to provide meaningful answers are crucial to understanding (e.g., Graesser, McMahen, & Johnson, 1994; Watts, Alsop, Gould, & Walsh, 1997). Both generating and answering questions involve monitoring and regulation of knowledge. According to Graesser et al. (1994), there are four types of questions asked in conversations: (1) questions to address knowledge deficit, (2) questions to monitor common ground, (3) questions for coordinating social action, and (4) questions for controlling conversation. While we observed all four types of questions in our dataset, the first category, knowledge deficit, was the most common among all. This can also be explained by the fact that students were given a limited time to work in team settings. And they spent most of their time for seeking related information to solve the ill-structured problem.
The result that the elaborated answer category was the best predictor for individual exam scores indicates students who provided detailed answers to their peers’ questions are performing better than the other students in their teams. In another study, Purzer (2011) found a similar pattern that links elaborated answers to individual achievement through self-efficacy. This result can also be explained by the learning by teaching paradigm (Biswas et al., 2005). Prior studies have shown that teaching others is an effective method to learn (e.g., Menekse & Chi, 2018; Chi, Siler, Jeong, Yamauchi, & Hausmann, 2001; Renkl, 1995; Rosenshine & Meister, 1994). Constructing elaborated responses help students to structure and organize their own knowledge. In addition, students who try to provide elaborated responses typically develop a deeper understanding of the domain. However, this type of other directed learning environment significantly increases the learning outcomes of “teaching/tutoring” students more than “learning/listening” partners (Menekse & Chi, 2018). As our results show, the students that provided detailed responses to the questions from their team members performed significantly better than the other team members. Another explanation is that students who are high performers engage in more elaboration. Our regression model shows a relationship; however, it is possible that students’ prior abilities might also be impacting their discourse in teams.
In terms of team performance, our results showed that the reasoning episodes were significantly related to team performance on the final engineering design project. As previous studies have shown explicit reasoning and time spent on comprehensive data gathering help teams to minimize potential errors (Hong, Doll, Nahm, & Li, 2004). For example, Tschan et al. (2009) explored the role of explicit reasoning in medical teams to reduce the number of incorrect diagnoses. They found teams that spent more time on explicit reasoning performed better than the other teams in terms of avoiding incorrect diagnoses.
Based on our coding scheme, the reasoning episodes included three categories: observational, calculation, and procedural. The calculation category was the best predictor of team performance. So, this finding indicates that the calculation related to coordination and communication within team members had a significant effect on team level success on the engineering design task. Since all teams were asked to justify their decisions based on evidence, results show successful teams spent considerable amount of time on tasks related to calculation, and this effort resulted in fruitful success. Also, calculations were important since students were asked to compare costs, savings, and payback period of the old and proposed new system. In addition, students needed to conduct an environmental analysis to compare carbon dioxide emissions of different systems.
Finally, we were interested in understanding how the dispersion of verbal episodes among team members relates to team level performance. Results showed the teams with more equal participation among team members performed significantly better than the teams that had unequal participation among team members. In prior studies, the equitability of team participation can be evaluated in two common ways: by analyzing turn-taking behaviors and by examining the distribution of verbal episodes. As prior studies argued that the distribution and frequency of turn-taking influence the contributions among team members and hence is important for successful interactions and effective team level outcomes (Chi & Wylie, 2014). More frequent turn-taking allows more equitable contributions from all team members and “make it easier for learners to incorporate their partners’ understanding and adjust their own mental model due to dynamic revisions of knowledge” (Menekse & Chi, 2018, p. 3). Chi and Menekse (2015) argued that ideal case of interactive learning environments requires joint dialog patterns in that all partners make substantive contributions to the topic. Likewise, our results provided evidence that teams with better participation and contribution from all team members performed better than other teams that did not have an equal participation among team members.
Based on the results from this study, and prior team studies, there are some important pedagogical implications. First, results showed the balanced participation among team members is critical for team and individual success. As, other studies also suggested, instructors and/or instructional designers need to provide training and opportunities for team members, so students can realize and experience that their individual contributions to discussions are crucial for team and individual success. Perhaps, this finding is also related to research on team size. While the team size was not considered in this study, one may argue that in large teams (i.e., more than five members) individual contributions may not be as effective as in small teams (e.g., Lowry et al. 2006).
In regard to verbal episodes, we found that question episodes are important for individual performance. So, it is essential for students to be scaffolded to ask meaningful questions, as well as to generate responses to others’ questions in their interactions with their team members. In other words, students need to pay attention to what their team members are asking and how they can best address these questions, which are also related to the idea of balanced participation within team members. So, when all team members participate in generating and responding the questions, this process naturally creates a balanced participation among team members as well. Similar to first pedagogical implication, students need to receive explicit training on the importance of generating and responding to questions within team settings.
We also found the calculation-related episodes were significantly related to team performance. Calculations were important for the team assignment since students were asked to compare costs, savings, and payback period of the old and proposed new systems. In addition, students needed to conduct an environmental analysis to compare carbon dioxide emissions of different systems. As a pedagogical implication, this finding underlines the importance of calculations in engineering. The calculations are at the core of all engineering disciplines, and the fundamental engineering equations and related calculations can facilitate students’ understanding. Therefore, engineering students need continuous instruction on how to apply the knowledge of mathematics and science into engineering problems. In addition, students need opportunities and feedback on how to analyze and interpret data, as well as how to communicate and disseminate their findings based on these calculations.
It is important to note some of the limitations of our study. The first limitation is that we had a fairly small sample size, especially at the team level (i.e., 19 teams). Accordingly, we did not use a multilevel approach for the analysis. Multilevel analysis addresses the problems that researchers encounter in the mixed factorial analysis of variance (Raudenbush & Bryk, 2002). On the other hand, multilevel analysis requires relatively large sample sizes. However, the coding for verbal data is labor and time intensive, and verbal coding for studies with large sample sizes requires resources to hire several well-trained coders. As a result, conducting a multilevel analysis is difficult for team learning studies that involve verbal data coding.
Another limitation was that the verbal data we used was from a separate problem-solving assignment and this assignment was not part of the final engineering design project that students worked on for the last 4 weeks of the semester. While teams received their team performance scores based on the engineering design project, we did not have any verbal data during team members’ collaboration for the engineering design project. On the other hand, this problem-solving assignment was right before the engineering design project, on week 11 in a 16-week-long semester, and it provided a detailed snapshot of how students collaborate in a team environment. Also, we chose to videorecord the team problem-solving activity at week 11 because this activity was a time limited assignment and each team had to complete all aspects of the assignment within 60 min. And this was the only time-limited team assignment during the semester which gave teams equal amount of time to work on it. For other assignments, teams had multiple weeks and they had been working on tasks out-of-class time as well; therefore, it was not feasible for us to keep track of 19 teams in order to have a complete picture of how they collaborate certain aspects of assignments. However, we believe it would be more beneficial to have multiple verbal data points during team collaborations to understand how team dynamics and interactions change over time.
Furthermore, team formation-related factors could be other limitations. In this study, teams were formed at the beginning of the semester based on students’ schedules and their available times, so they can meet out of class times to complete assignments. Based on prior research on team formation, the gender diversity of team members (e.g., Bear & Woolley, 2011), cultural diversity of team members (e.g., Nathan & Lee, 2013), students’ attitudes toward collaboration (e.g., Ruiz Ulloa & Adams, 2004), and social loafing (e.g., Lam, 2015) could play a significant role on team performances. Additionally, instructor and students’ interactions throughout the semester could also play a role on students’ academic success (e.g., Luo, Fan, Menekse, Wang, & Litman, 2015) However, we do not have such data in this study.
Past research indicates that peer interaction is critical not only for the improvement of academic achievement but also for social and emotional development (Hartup, 1989). Teams in general allow students to develop social skills such as helping others, sharing, taking turns, showing respect, and working collaboratively. And students’ social and discursive practices during collaborations could play a substantial role in team harmony and success. Verbal discourse in teams is critical not only for the improvement of academic achievement for individuals in teams but also for the development of better team solutions. It is important that such verbal discourse also includes questions and elaborated responses, which are necessary to promote individual student academic achievement. Student team interactions must also promote verbalization of calculations to promote quality of team outcomes. It is through such verbalization that calculations can be checked and agreed by team members. Engineering educators should monitor discourse and specially reinforce those that are critical for positive performance. Moreover, balanced participation from all team members is critical, suggesting teams should be formed and monitored to support balanced participation in verbal discourse. Future research is needed to explore how the interventions, designed to promote specific verbal episodes, can promote student academic achievement and teamwork outcomes.
We are grateful to the students who participated this research study and allowed us to collect data. We are also grateful to Felicia Bixler and Douglas Stouch for their contributions on verbal coding of video data.
There was no funding for this study.
Availability of data and materials
Research participants’ data for individual achievement and team performance scores can be obtained by emailing the corresponding author. However, video data cannot be shared since video recordings include images and voices of research participants.
MM and SP designed the study. SP collected data, MM cleaned data, and MM conducted statistical analysis, designed verbal data protocol, and supervised verbal data coding. MM, SP, and DH wrote the manuscript collaboratively. All authors read and approved the final manuscript.
Ethics approval and consent to participate
This word adheres to the requirements of the Human Research Protection Program at Purdue University, detailed in IRB protocol #1108011151.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- ABET. (2017). Criteria for accrediting engineering programs. MD: Engineering Accreditation Commission Retrieved from http://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting-engineering-programs-2017-2018/.Google Scholar
- Anwar, S., Menekse, M., Heo, D., & Kim, D. (2018). Work in progress: students’ reflection quality and effective team membership. Salt Lake City: Paper presented at 2018 ASEE Annual Conference & Exposition https://peer.asee.org/work-in-progress-students-reflection-quality-and-effective-team-membership.
- Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. The Journal of the Learning Sciences, 9(4), 403–436.Google Scholar
- Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.Google Scholar
- Barrows, H. S. (1986). A taxonomy of problem-based learning methods. Medical Education, 20(6), 481–486.Google Scholar
- Bear, J. B., & Woolley, A. W. (2011). The role of gender in team collaboration and performance. Interdisciplinary Science Reviews, 36(2), 146–153.Google Scholar
- Biswas, G., Leelawong, K., Schwartz, D., Vye, N., & The Teachable Agents Group at Vanderbilt. (2005). Learning by teaching: a new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363–392.Google Scholar
- Borrego, M., Karlin, J., McNair, L. D., & Beddoes, K. (2013). Team effectiveness theory from industrial and organizational psychology applied to engineering student project teams: a research review. Journal of Engineering Education, 102(4), 472–512.Google Scholar
- Chi, M. T. (2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1(1), 73–105.Google Scholar
- Chi, M. T., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25(4), 471–533.Google Scholar
- Chi, M. T., & Wylie, R. (2014). The ICAP framework: linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243.Google Scholar
- Chi, M. T. H., & Menekse, M. (2015). Dialogue patterns that promote learning. In L. B. Resnick, C. Asterhan, & S. N. Clarke (Eds.), Socializing intelligence through talk and dialogue (Ch. 21 (pp. 263–274). Washington DC: AERA.Google Scholar
- Chinn, C. A., O’donnell, A. M., & Jinks, T. S. (2000). The structure of discourse in collaborative learning. The Journal of Experimental Education, 69(1), 77–97.Google Scholar
- Christensen, B. T., & Schunn, C. D. (2007). The relationship of analogical distance to analogical function and preinventive structure: The case of engineering design. Memory & Cognition, 35(1), 29–38.Google Scholar
- Cockrell, K. S., Caplow, J. A. H., & Donaldson, J. F. (2000). A context for learning: collaborative groups in the problem-based learning environment. The Review of Higher Education, 23(3), 347–363.Google Scholar
- Cohen, S. G., & Bailey, D. E. (1997). What makes teams work: group effectiveness research from the shop floor to the executive suite. Journal of Management, 23(3), 239–290.Google Scholar
- Darling, A. L., & Dannels, D. P. (2003). Practicing engineers talk about the importance of talk: a report on the role of oral communication in the workplace. Communication Education, 52(1), 1–16.Google Scholar
- Dillenbourg, P., & Hong, F. (2008). The mechanics of CSCL macro scripts. International Journal of Computer-Supported Collaborative Learning, 3(1), 5–23.Google Scholar
- Dixon, R. a., & Johnson, S. D. (2011). Experts vs. novices: differences in how mental representations are used in engineering design. Journal of Technology Education, 23(1), 1–11.Google Scholar
- Donath, L., Spray, R., Thompson, N. S., Alford, E. M., Craig, N., & Matthews, M. A. (2005). Characterizing discourse among undergraduate researchers in an inquiry-based community of practice. Journal of Engineering Education, 94(4), 403–417.Google Scholar
- Dossick, C. S., & Neff, G. (2011). Messy talk and clean technology: communication, problem-solving and collaboration using building information modelling. The Engineering Project Organization Journal, 1(2), 83–93.Google Scholar
- Dym, C. L., Agogino, A. M., Eris, O., Frey, D. D., & Leifer, L. J. (2005). Engineering design thinking, teaching, and learning. Journal of Engineering Education, 94(1), 103–120.Google Scholar
- Eggert, R., Joshi, A., Mehrotra, S., Zastavker, Y. V., & Darer, V. (2014). Using discourse analysis to understand “failure modes” of undergraduate engineering teams. In Frontiers in education conference (FIE), 2014 IEEE (pp. 1–5) IEEE.Google Scholar
- Erkens, G., & Janssen, J. (2008). Automatic coding of dialogue acts in collaboration protocols. International Journal of Computer-Supported Collaborative Learning, 3(4), 447–470.Google Scholar
- Felder, R. M., & Brent, R. (2003). Designing and teaching courses to satisfy the ABET engineering criteria. Journal of Engineering Education, 92(1), 7–25.Google Scholar
- Fila, N. D., & Loui, M. C. (2014). Structured pairing in a first-year electrical and computer engineering laboratory: The effects on student retention, attitudes, and teamwork. International Journal of Engineering Education, 30(4), 848-861.Google Scholar
- Froyd, J. E., Wankat, P. C., & Smith, K. A. (2012). Five major shifts in 100 years of engineering education. Proceedings of the IEEE, 100(special centennial issue), 1344–1360.Google Scholar
- Goldman, S. R. (2018). Discourse of learning and the learning of discourse. Discourse Processes, 55(5–6), 434–453. https://doi.org/10.1080/0163853X.2018.1440866.Google Scholar
- Graesser, A. C., McMahen, C. L., & Johnson, B. K. (1994). Question asking and answering. In M. A. Gernsbacher (Ed.), Handbook of psycholinguistics (pp. 517–538). San Diego: Academic Press.Google Scholar
- Haller, C. R., Gallagher, V. J., Weldon, T. L., & Felder, R. M. (2000). Dynamics of peer education in cooperative learning workgroups. Journal of Engineering Education, 89(3), 285–293.Google Scholar
- Hartup, W. W. (1989). Social relationships and their developmental significance. American Psychologist, 44(2), 120–126.Google Scholar
- Heyman, R. E., Lorber, M. F., Eddy, J. M., & West, T. V. (2014). Behavioral observation and coding. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology, 2nd ed (pp. 345–372). New York: Cambridge University Press.Google Scholar
- Hmelo-Silver, C. E. (2004). Problem-based learning: what and how do students learn? Educational Psychology Review, 16(3), 235–266.Google Scholar
- Hmelo-Silver, C. E., Chinn, C. A., Chan, C., & O’Donnell, A. (2013). The international handbook of collaborative learning. UK: Routledge.Google Scholar
- Hmelo-Silver, C. E., Duncan, R. G., & Chinn, C. A. (2007). Scaffolding and achievement in problem-based and inquiry learning: a response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.Google Scholar
- Hong, P., Doll, W. J., Nahm, A. Y., & Li, X. (2004). Knowledge sharing in integrated product development. European Journal of Innovation Management, 7(2), 102–112.Google Scholar
- Hsiung, C. M. (2010). Identification of dysfunctional cooperative learning teams based on students’ academic achievement. Journal of Engineering Education, 99(1), 45–54.Google Scholar
- Hsiung, C. M. (2012). The effectiveness of cooperative learning. Journal of Engineering Education, 101(1), 119–137.Google Scholar
- National Association of Colleges and Employers (2014). Job outlook 2015. PA: NACE. Retrieved from https://www.umuc.edu/upload/NACE-Job-Outlook-2015.pdf. Accessed 04 Sept 2017.
- Huang, H. C., Shih, S.-G., & Lai, W. C. (2011). Cooperative learning in engineering education: a game theory-based approach. International Journal of Engineering Education, 27(4), 875–884.Google Scholar
- Jonassen, D. H., & Hung, W. (2008). All problems are not equal: implications for problem-based learning. Interdisciplinary Journal of Problem-Based Learning, 2(2), 6–28.Google Scholar
- Katsikopoulos, K. V. (2012). Decision methods for design: insights from psychology. Journal of Mechanical Design, 134(8):084504-084504-4. https://doi.org/10.1115/1.4007001.
- Katz, S. M. (1993). The entry-level engineer: problems in transition from student to professional. Journal of Engineering Education, 82(3), 171–174.Google Scholar
- Kittleson, J. M., & Southerland, S. A. (2004). The role of discourse in group knowledge construction: A case study of engineering students. Journal of Research in Science Teaching, 41(3), 267–293.Google Scholar
- Kuhn, D. (2015). Thinking together and alone. Educational Researcher, 44(1), 46–53.Google Scholar
- Lam, C. (2015). The role of communication and cohesion in reducing social loafing in group projects. Business and Professional Communication Quarterly, 78(4), 454–475.Google Scholar
- Lowry, P. B., Roberts, T. L., Romano Jr, N. C., Cheney, P. D., & Hightower, R. T. (2006). The impact of group size and social presence on small-group communication: Does computer-mediated communication make a difference?. Small Group Research, 37(6), 631–661.Google Scholar
- Luo, W., Fan, X., Menekse, M., Wang, J., & Litman, D. (2015, June). Enhancing instructor-student and student-student interactions with mobile interfaces and summarization (pp. 16–20). Denver: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations.Google Scholar
- Menekse, M. (2012). Interactive-constructive-active-passive: The relative effectiveness of differentiated activities on students' learning. (Doctoral dissertation, Arizona State University). ProQuest Dissertations and Theses, 178. Retrieved from https://search.proquest.com/docview/1034283822?accountid=13360.
- Menekse, M., & Chi, M. T. H. (2018). The role of collaborative interactions versus individual construction on students’ learning of engineering concepts. European Journal of Engineering Education. https://doi.org/10.1080/03043797.2018.1538324.
- Menekse, M., Higashi, R., Schunn, C., & Baehr, E. (2017). Exploring the role of robotics teams’ collaboration quality on team performance in a robotics tournament. Journal of Engineering Education, 106(4), 564–584. https://doi.org/10.1002/jee.20178.Google Scholar
- Menekse, M., Stump, G., Krause, S., & Chi, M. T. H. (2013). Differentiated overt learning activities for effective instruction in engineering classrooms. Journal of Engineering Education, 102(3), 346–374. https://doi.org/10.1002/jee.20021.Google Scholar
- Mezirow, J. (1997). Transformative learning: Theory to practice. New Directions for Adult and Continuing Education, 1997(74), 5–12.Google Scholar
- Mishra, D., Ostrovska, S., & Hacaloglu, T. (2015). Assessing team work in engineering projects. International Journal of Engineering Education, 31(2), 627–634.Google Scholar
- Nathan, M., & Lee, N. (2013). Cultural diversity, innovation, and entrepreneurship: firm-level evidence from London. Economic Geography, 89(4), 367–394.Google Scholar
- Pai, H. H., Sears, D. A., & Maeda, Y. (2015). Effects of small-group learning on transfer: a meta-analysis. Educational Psychology Review, 27(1), 79–102.Google Scholar
- Pasha-Zaidi, N., Afari, E., Mohammed, J., Cubero, S., Shoukry, A. M., & El Sokkary, W. (2015). Gender-based teams: perceptions of team satisfaction and effectiveness among engineering students in the United Arab Emirates. International Journal of Engineering Education, 31(4), 953–966.Google Scholar
- Passow, H. J. (2012). Which ABET competencies do engineering graduates find most important in their work? Journal of Engineering Education, 101(1), 95–118.Google Scholar
- Ponsa Asensio, P., Román Jiménez, J. A., Arnó Macià, E., & Pérez Soriano, J. (2015). Professional skills in international multidisciplinary teams. International Journal of Engineering Education, 31(4), 998–1006.Google Scholar
- Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), 223–231.Google Scholar
- Purzer, Ş. (2011). The relationship between team discourse, self‐efficacy, and individual achievement: A sequential mixed‐methods study. Journal of Engineering Education, 100(4), 655–679.Google Scholar
- Quinn, K. A., & Albano, L. D. (2008). Problem-based learning in structural engineering education. Journal of Professional Issues in Engineering Education and Practice, 134(4), 329–334.Google Scholar
- Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: applications and data analysis methods (Vol. 1). CA: Sage.Google Scholar
- Renkl, A. (1995). Learning for later teaching: an exploration of mediational links between teaching expectancy and learning results. Learning and Instruction, 5(1), 21–36.Google Scholar
- Rosenshine, B., & Meister, C. (1994). Reciprocal teaching: a review of the research. Review of Educational Research, 64(4), 479–530.Google Scholar
- Ruiz Ulloa, B. C., & Adams, S. G. (2004). Attitude toward teamwork and effective teaming. Team Performance Management: An International Journal, 10(7/8), 145–151.Google Scholar
- Saab, N., Joolingen, W. R., & Hout-Wolters, B. H. (2005). Communication in collaborative discovery learning. British Journal of Educational Psychology, 75(4), 603–621.Google Scholar
- Salter, A., & Gann, D. (2003). Sources of ideas for innovation in engineering design. Research Policy, 32(8), 1309–1324.Google Scholar
- Savery, J. R. (2006). Oof problem-based learning: definitions and distinctions. Interdisciplinary Journal of Problem-Based Learning, 1(1). https://doi.org/10.7771/1541-5015.1002.
- Sha, Z., & Panchal, J. H. (2014). Estimating local decision-making behavior in complex evolutionary systems. Journal of Mechanical Design, 136(6), 061003-061003-11). https://doi.org/10.1115/1.4026823.
- Streveler, R. A., & Menekse, M. (2017). Taking a closer look at active learning. Journal of Engineering Education, 106(2), 186–190. https://doi.org/10.1002/jee.20160.Google Scholar
- Terenzini, P. T., Cabrera, A. F., Colbeck, C. L., Parente, J. M., & Bjorklund, S. A. (2001). Collaborative learning vs. lecture/discussion: students’ reported learning gains. Journal of Engineering Education, 90(1), 123–130.Google Scholar
- Tonso, K. L. (2006). Teams that work: campus culture, engineer identity, and social interactions. Journal of Engineering Education, 95(1), 25–37.Google Scholar
- Tschan, F., Semmer, N. K., Gurtner, A., Bizzari, L., Spychiger, M., Breuer, M., & Marsch, S. U. (2009). Explicit reasoning, confirmation bias, and illusory transactive memory: a simulation study of group medical decision making. Small Group Research, 40(3), 271–300.Google Scholar
- Van Boxtel, C., Van der Linden, J., & Kanselaar, G. (2000). Collaborative learning tasks and the elaboration of conceptual knowledge. Learning and Instruction, 10(4), 311–330.Google Scholar
- Watts, M., Alsop, S., Gould, G., & Walsh, A. (1997). Prompting teachers’ constructive reflection: pupils’ questions as critical incidents. International Journal of Science Education, 19(9), 1025–1037.Google Scholar
- Whitman, L. E., Malzahn, D. E., Chaparro, B. S., Russell, M., Langrall, R., & Mohler, B. A. (2005). A comparison of group processes, performance, and satisfaction in face-to-face versus computer-mediated engineering student design teams. Journal of Engineering Education, 94(3), 327–337.Google Scholar
- Wood, C. (2006). The development of creative problem solving in chemistry. Chemistry Education Research and Practice, 7(2), 96–113.Google Scholar
- Yadav, A., Subedi, D., Lundeberg, M. A., & Bunting, C. F. (2011). Problem-based learning: influence on students' learning in an electrical engineering course. Journal of Engineering Education, 100(2), 253–280.Google Scholar
- Zafft, C. R., Adams, S. G., & Matkin, G. S. (2009). Measuring leadership in self-managed teams using the competing values framework. Journal of Engineering Education, 98(3), 273–282.Google Scholar
- Zhou, C., Luo, L., Du, X., & Kolmos, A. (2010). Factors influencing group creativity in project-organized teams in engineering education in China. International Journal of Engineering Education, 26(6), 1524–1535.Google Scholar