Vigilante Innovation (VIX): case study on the development of student skills through a team-based design process and environment

Background: Many undergraduate students majoring in science, technology, engineering, and mathematics (STEM) fields lack experience in collaborative thinking, limiting their effectiveness as they enter careers in academic and industrial environments. The SyBBURE Searle Undergraduate Research Program has incorporated a team-based design component into its curriculum to fill this gap in training. This design framework, called Vigilante Innovation (VIX) to highlight its emphasis on self-initiation and action, has evolved into a multi-semester-long group undertaking that combines just-in-time training in entrepreneurship and project design with student-driven collaborations aimed at solving a real-world problem. We hypothesize that this framework provides a hands-on, realistic workplace simulation task through which students can develop an understanding of teamwork. Results: Using a case-study approach, we discuss the development of the VIX design framework since its inception in 2014 and assess the impact of the VIX framework on student learning and growth using a student survey from 2016 to 2017 and student interviews from 2018. Conclusions: A flexible approach, an annualized project timeline, a student-driven prototyping space, and selfselecting project areas emerged as key contributors to the successful implementation of the VIX design and to deepened student learning. The diversity of VIX teams, the self-reported success of student projects, and student interviews indicate that students who participate in VIX possess an in-depth understanding of team-based strategies. These findings support the VIX framework as an effective method of providing undergraduates in STEM fields with efficient and meaningful exposure to the team-based entrepreneurial skills that are vital in their future careers. Additional work is needed to determine if this approach has a long-term impact on student success in team-based environments. The website vigilanteinnovation.com houses a customizable, freely available version of the design guide for educators and innovators alike.


Introduction
Undergraduate education lays the groundwork that prepares students to enter the workforce. From product marketing to research and development at a biotechnology company, teamwork is an essential skill for a successful career in science, technology, engineering, and mathematics (STEM). Effective teamwork does not happen by merely bringing a group of people together, but instead results from learning, experience, and refinement of particular skills. Institutes of higher education must develop approaches to embed collaborative and teambased training exercises in STEM education. Intentionally preparing undergraduates to work in teams has led to demonstrated success in many career outcomes (LePine, Piccolo, Jackson, Mathieu, & Saul, 2008;Rousseau, Aubé, & Savoie, 2006).
The case study presented here utilizes an approach to teamwork training for undergraduates that goes beyond team-based learning. Our approach integrates interprofessional training with a realistic simulation of practical tasks and behavioral skill development employing collaborative problem solving (CPS) that requires set of cognitive and social skills that are different from traditional studies on an individual's problem solving (Graesser et al., 2018;Li et al., 2019). More specifically, we hypothesize that hands-on activities blended with realworld, flexible programming facilitate student understanding of team roles and dynamics. Below, we define teamwork and discuss its benefits, review the current literature on the training of team skills, and present the context for this study.

The need for teams
Collaboration, cooperation, and teamwork all attempt to describe the same concept, although they are sometimes defined differently (Kirschner, 2001). We consider these terms comparable and employ the frequently quoted definition posed by Katzenbach and Smith (revised by Dalcher, 2018) of a team as "a small number of people with complementary skills who are committed to a common purpose, performance goals, and approach for which they hold themselves mutually accountable". We use this definition throughout the paper.
Katzenbach and Smith also wrote that "teams outperform individuals acting alone or in larger organizational groupings, especially when performance requires multiple skills, judgments, and experiences", a conclusion with which we concur (revised by Dalcher, 2018). Moreover, we endorse the assumptions that incentivizing collaborative behaviors for higher education learners will transform classrooms into real-world, action-based learning environments (Hamilton, Ansell, Reynolds, Potenza, & Sinha, 2013), and that giving undergraduates the autonomy to pursue their own interests and project ideas in a group setting makes collaborations more effective (Scager, Boonstra, Peeters, Vulperhorst, & Wiegant, 2016).
Teamwork, team performance, and collaborative thinking span multiple contexts, such as educational setting, laboratories, healthcare, aviation, the military, and academia. A practical example of the importance of teamwork comes from within medical settings, where up to 70% of adverse events are speculated to be independent of individuals' technical errors, but rather a result of teamwork breakdowns (World Health Organization, 2009). Teamwork within an educational setting has been shown to promote deep learning. In this environment, the students engage in productive social interaction, such as discussing contradictory or alternative information (Visschers-Pleijers, Dolmans, De Leng, Wolfhagen, & Van Der Vleuten, 2006). In STEM higher education, a deep-learning approach is crucial for understanding complex ideas (Van Boxtel, Van der Linden, & Kanselaar, 2000). This approach involves a process of conceptual change, which is notably activated in collaborative learning, and requires the students to interact by explaining to and questioning one another critically (Linton, Farmer, & Peterson, 2014;Van Boxtel et al., 2000). Besides these cognitive benefits, collaborative learning through teamwork provides social skills needed for future professional work in a variety of STEM fields (Osborne, 2010).
Studies have demonstrated the positive effects of teamwork across several areas, including healthcare and academic settings (Morey et al., 2002;Padmo Putri, 2013). Focusing on team building has been shown to be beneficial in both new and intact teams. Teamwork even benefits laboratory-based experimental experiences (Holton, 2001). Furthermore, collaborative skills are mandated by multiple organizations, including the Accreditation Board for Engineering and Technology (ABET). Collaboration is often a required component in capstone engineering courses and project-based courses. Research also has demonstrated the positive relationship between collaborative learning and student achievement, effort, persistence, and motivation (Barron, 2003;Johnson & Johnson, 2009;Slavin, 1990;Webb, 2009). However, a study of practicing engineers highlights the need for further development of teamwork skills (Marra, Steege, Tsai, & Tang, 2016). Thus, while the importance of teamwork in both professional and educational settings is well established, the ideal training methods remain in dispute.

Teaching teamwork
Placing students in groups does not innately lead to improved learning and motivation (Gillies, 2004;Khosa & Volet, 2013;Salomon & Globerson, 1989). Although group work can be beneficial, the learning potential of collaboration itself is commonly underused (Johnson & Johnson, 2009). This is particularly true in STEM higher education settings that have not adopted a businesscentered or entrepreneurial mindset (Nokes-Malach, Richey, & Gadgil, 2015). When confronted with the need to provide training in teamwork in higher education, most will think of team-based learning (TBL) as a commonly used curriculum tool. TBL is a clear instructional strategy developed by Larry Michaelson (Marra et al., 2016;Michaelsen & Sweet, 2008). This approach allows a single instructor to lead multiple small groups simultaneously in the same classroom. Learners actively participate through preparation and group discussion, which can occur both inside and outside the classroom environment. The result of this approach is a focus on class time toward the application and integration of information. While this is a useful active learning tool to engage students, it is distinct from teamwork training.
We define teamwork training as a set of tools, methods, and content used to educate the learner around effective team interactions. Teamwork training has one primary objective: to improve crucial team competencies. While students are often expected to work and learn together in groups, specific training in teamwork is a relatively new approach, particularly for professors in STEM fields trained in self-sufficiency (Curran, Sharpe, & Forristall, 2007).
A literature review was conducted on teamwork training using systematic assessment and meta-analysis to quantify the effects of controlled experimental research of teamwork training interventions on team performance outcomes (Mcewan, Ruissen, Eys, Zumbo, & Beauchamp, 2017). One conclusion of this study is that teamwork skills improve through active practice as opposed to lectures on teamwork. For example, a well-known strategy from the aviation industry is crew resource management (CRM) training. Although it is possible to communicate some concepts of behavioral skills and teamwork in a didactic teaching setting, experts in CRM training suggest that they should be practiced simultaneously with technical skills, often with high-fidelity simulators (Baker, Gustafson, Beaubien, Salas, Barach, & Battles, 2003). Therefore, this study expands upon the definition of teamwork training by adding a practice simulation as a component of a successful training program.
Another educational area that employs both simulated and real practice alongside professional learning within teams is undergraduate medical education. This training falls within the realm of inter-professional education (IPE), through which two or more professions learn with, from, and about each other in order to improve collaboration and the quality of practice. The World Health Organization emphasizes the importance of IPE and promotes IPE as a pivotal strategy to enhance patient outcomes by preparing a "collaborative practiceready health workforce" (Berger et al., 2017;WHO, 2010). It has been suggested that IPE helps students to look at a task from multiple disciplinary perspectives, and that this approach enables students to acquire knowledge, skills, and attitudes that they could not acquire in an isolated professional education (Coster et al., 2008;Hallin, Kiessling, Waldner, & Henriksson, 2009;S. S. Hamilton et al., 2008).
Additionally, there have been a few examples of teamwork training within courses for undergraduates in premedical programs. In a pilot course developed to expose undergraduate students who plan to pursue a health profession to an interprofessional, team-based care approach (Yerrapragada & Petersen, 2016), the students engaged in discussions with healthcare professionals, thought-provoking and skill-building exercises, shadowing experiences, and opportunities for reflection. Overall, students gained valuable experience in interprofessionalism and became aware of the importance of teams in shaping the future of healthcare. Another case study from pre-medical education demonstrated how logistical and attitudinal barriers could hinder the integration of teamwork training into curricula (Berger et al., 2017). The approach focused on establishing four interprofessional seminars on team communication, medical error communication, healthcare, and small business management. This study suggested that a structured approach was vital for success. While the aviation and undergraduate medical education realms have led the way in promoting teamwork learning in simulated and practice environments alongside learning technical skills, undergraduate STEM education has not yet employed these methods. Studies are demonstrating that the implementation of teamwork training that incorporates hands-on exercise opportunities for undergraduates is needed to provide evidence of the utility of these training methods for STEM undergraduates. We aim to provide such an example of the effectiveness of teamwork training in a simulated, hands-on environment for undergraduate STEM students.

The SyBBURE Searle Undergraduate Research Program
The SyBBURE Searle Undergraduate Research Program is a year-round program that provides undergraduate students from all disciplinary fields with mentored experiences in an advanced scientific investigation. The program aims to incubate and inspire the next generation of innovators through research, design, and community. Students experience unique and ever-changing opportunities as springboards to future endeavors. This program is not a requirement for any degree or major within the institution. The students apply to take part in the program in exchange for such experiences, receiving a stipend while participating. Since the program began in 2006, the SyBBURE Searle Program has influenced the trajectory of nearly 300 undergraduate students from diverse backgrounds.
A vital component of the SyBBURE Searle Program is our team-based design thread, Vigilante Innovation (VIX), which combines just-in-time training modules covering entrepreneurship and design principles with hands-on project experience. By providing specific training at the critical moment it is needed, students immediately have the opportunity to implement and practice what they have just learned. The VIX framework defines a flexible process and a supportive, student-focused environment to bring together students from any discipline to self-sort into small, multidisciplinary teams (3-4 students) based on a shared interest area around which they contribute their knowledge and expertise. There is no limitation to the area of interest around which teams may form, and students draw inspiration from sources such as their own lives, news, and current events, trends in technology, scientific discoveries, grant calls, and interactions with healthcare providers. These teams work to identify critical problems within their interest areas and prototype solutions. In these student-driven teams, the scientific advisor or instructor acts as a facilitator, taking a backseat to student creativity and team dynamics, which has previously been shown to lead to positive outcomes (Kirschner, 2001).
There are many ways in which higher education and the workforce have approached teaching teamwork and collaborative skill-building, including didactic lectures/ presentations, workshops, simulation training, and review-type activities (Mcewan et al., 2017). Throughout the study, we sought to answer the question, "Does the VIX framework have an impact on the development of student teamwork skills?" This case study expands the current body of knowledge around teamwork and teamwork training approaches as it offers a novel method for teaching teamwork through a realistic workplace simulation task within undergraduate STEM education. To the best of our knowledge, no study has yet assessed the impact of participation in a student-driven design project on teamwork. This case study could serve as a model for teamwork training approaches and may support the argument that hands-on, real-world training opportunities are necessary for effective development of collaborative skills.

Participant recruitment and consenting
Approval from the Vanderbilt Institutional Review Board was obtained in early 2018 (IRB# 180015) to cover retrospective data use, including demographic information and survey data, as well as in-person interviews.
Retrospective student demographic and electronic survey data, collected for other programmatic purposes, was used to assess team diversity and participation. All published student demographic and electronic survey data were de-identified. Data were included from 2006 to summer 2018 for total program demographics and from 2014 to 2018 for VIX participant demographics. Available data on VIX teams (project area, number of students on the team) were gathered and analyzed with Microsoft Excel to determine the percentage of teams within each project area and the average number of students per team.

Electronic survey design and implementation
All students within the SyBBURE Searle Program who participated either during the summer of 2016 or 2017 were asked to complete an electronic survey to provide feedback on the importance of the critical fundamental components that were covered and the degree to which VIX addressed them. The survey consisted of 12 statements, to which students responded using a 5-point Likert scale: "strongly disagree" to "strongly agree" (Strongly Disagree-1; Disagree-2, Neutral-3; Agree-4; Strongly Agree-5) for the importance rating and "not addressed" to "fully addressed" (Not Addressed-1; 2, 3; 4; Fully Addressed-5) for the component coverage rating. The 12 statements prompted students to rate the importance of the following components in creating a valuable VIX experience and degree to which these components were addressed: (1) how to collaborate, (2) how to solve real-world problems, (3) designing a marketable product, (4) meeting with experts, (5) learning a problem-solving method, and (6) answering a scientific question developed by the team. The survey responses were anonymous. Student participation in the electronic survey was voluntary, although students received reminders to complete the survey. Student responses were analyzed using Microsoft Excel to determine response means for each component. At the end of the summer 2016 and 2017 terms, the students were asked to complete a short electronic survey to assess whether participation in VIX addressed critical elements of learning.

Interview design and implementation
In order to gain further insight into their responses and skill development, students were asked to voluntarily participate in a single, 15-min, in-person interview session. Participants were recruited through email and inperson announcements. Students were informed that the interview was voluntary and that neither participation in the interviews nor answers to the interview questions would impact their standing within the program or their relationship with program leadership. All participants interviewed were 18 years of age or older and consented before the interview. Inclusion criteria consisted of a minimum of two semesters or a single summer of participation in the SyBBURE Searle Program. There were no exclusion criteria. Thirteen students elected to participate in the interview. During the interview, students were asked the series of questions listed in Table 1, and their responses were recorded. Responses were coded by hand by a single coder and summarized.

Thematic analytical framework for interview questions
Following data collection, the lead author listened to the recording of the interviews and noted some initial thoughts as was initially suggested by Kohler et al. to begin a thematic analysis (Kohler Riessman, 2001). "Repeated listening," similar to repeated reading (Braun & Clarke, 2006), resulted in data immersion. Following this initial stage and building on the notes and ideas generated through data immersion, a code was created. These codes identified features of the data that the researcher considered pertinent to the research question. The whole data set was given equal attention during this process so that full consideration could be given to establishing repeated patterns within the data. Next, these codes were examined for themes. The themes explained larger sections of the data by combining different codes that may have been very similar or may have been considered the same aspect within the data. All original codes relevant to the research question were incorporated into a theme. Any themes that did not have enough data to support them or were too diverse were discarded. This refinement of the themes took place on two levels. First, the coded data were reanalyzed to ensure that they formed a coherent pattern. Second, the themes were considered with the data set as a whole unit. This step was performed to ensure that the themes accurately reflected the data set as a whole (Braun & Clarke, 2006). The final step was defining and naming the themes. We considered not only the story told within individual themes but also the relation of the individual themes to the overall story within the full data set.

Student and participant populations
The SyBBURE Searle Program has had 282 undergraduate students participate in the program. Ninety-eight (34.8%) of these students were female, and 184 (65.2%) were male. The program's overall ethnic diversity includes 141 (50%) Caucasian, 36 (12.7%) Asian, 20 (7.1%) African, 16 (5.7%) Latino/Hispanic, 15 (5.3%) East Asian,  Fig. 1. These students represented a wide range of majors, as shown in the left panel of Fig. 2. Four students who participated in the program as advanced high school students did not have majors at the time and have not been included as part of the discipline diversity. This data is compounded with the high frequency of double majors in the program (51/282; 18%). The VIX framework began to be developed in the summer of 2014. Since then, the SyBBURE Searle Program has had a total of 94 students who have participated in VIX or an earlier version for varying durations. The population of students since 2014 matches the overall demographics of the SyBBURE Searle Program with 61 (64.9%) male and 33 (35.1%) female students (Fig. 1, right panel). Ethnically, this population also matches the entire cohort of students with 44 (46.8%) Caucasian, 13 (13.8%) Asian, seven (7.4%) African, four (4.6%) East Asian, four (4.2%) Latino/Hispanic, three (3.2%) South Asian, one (1.0%) Caribbean, three (3.2%) students representing two or more ethnicities, two (2.1%) other, and eight (8.5%) unknown or unreported. Finally, this population also matches the overall discipline diversity of the full cohort of students (Fig. 2, right panel). However, since 2014, there has been an increase in the number of double majors (50/94; 53.2%).
There have been nearly 50 VIX teams, examples of which are listed in Table 2. Projects generally spanned a variety of areas, including fabrication, electronics, music, robotics, synthetic biology, chemistry, microfluidics, software, and app development, education, art, fashion, woodworking, sports medicine, 3D printing, and consumer product development. Projects were categorized into one of the following five areas, with the percentage of projects in that area included in parenthesis: Art/Design (11.4%), Food (14.3%), Science (20%), Education (5.7%), Technology (17.14%), Health (20%), and Community (11.4%). On average, there were 3.8 students per team.

Previous iterations building toward the vix framework
The VIX component of the SyBBURE Searle Program was implemented each term (summer, fall, and spring) beginning with the summer of 2014 up until the spring of 2018, when the program switched to an annualized model, as shown in Figs. 3 and 4. While the general intent of VIX over this time period was constant, the curriculum was iterated based on student feedback, both formative and summative, that was collected throughout the program. We describe and discuss the development and iteration of this design approach and environment in detail to give readers a sense for how they might implement and apply these principles to their own enterprises.
In the summer of 2014, out of a desire to improve the teamwork skills of our students, we launched our program's design component by having students participate full-time in a 2-week innovation and entrepreneurship workshop. Through this workshop, we were able to jumpstart the students' ability to think of their research project beyond the walls of the lab and to focus on design and problem-solving. Upon completion of this workshop, the students were introduced to the first version of our team-based design experience. We focused initially on encouraging the students to explore solutions to local or global problems in science, engineering, and medicine. They formed teams around a provided list of problems (based on grant calls) in which they were interested and worked to learn and devise strategies to solve these problems. Each week, they had the option to go deeper or to shift their problem focus. Teams presented updates weekly. The students were given the option to continue on their same project or switch to another in the following fall and spring. Outcomes were mixed, with no teams continuing, and one student pivoting their design ideas to their SyBBURE Searle research project. Students then spent the fall and spring largely exploring other problems, For the second iteration, starting in the summer of 2015, student teams were encouraged to identify their own problem, which required them to lean on their observational and assessment capacities. To make progress, they had to engage in the project, seek out information and experts for help, and develop strategies for successful teamwork. Presenting concise reports was a major portion of this activity, promoting strong oral communication. At the end of the iteration, the students were given the opportunity to assess the activity, which allowed regular improvements to be made. Additionally, during the summer of 2015, two teams of students were selected to engage in a pilot program to explore a series of topics, including developing a problem statement, assessing their idea, determining design requirements, brainstorming solutions, and project management. This Microtonal music has a high barrier of entry, so we'd like to make it easy to learn and understand Education pilot was used to develop materials and lessons that shaped the team-based design experience within the entirety of the SyBBURE Searle Program. Both pilot teams were able to produce and iterate a prototype, leading us to realize that a more defined framework is needed to enable students to make progress toward finding solutions to problems. During the fall and spring semesters, teams completed weekly updates (by way of a form) to help guide them through the design process. Building on the lessons learned through the pilot and in the main design thread, during the spring of 2016, we began to develop a design guidebook with activities to lead teams of multidisciplinary students through the process of identifying their interests, researching those areas, determining problems in those areas, developing solutions to these problems, refining solutions, prototyping, and iteratively improving prototypes. We branded this design guide "Vigilante innovation" to encourage students not to wait for others to make a positive impact on society, but rather to take matters into their own hands. This guide and activity set was then used during the summer of 2016. We focused on a process of "think, create, and innovate" to lead students from generating ideas and finding problems, to building and getting feedback on a prototype. The guide provided students with a roadmap for invention that was particularly evident in the number of functioning prototypes produced over the summer. All student teams, for the first time in the history of our design framework, created prototypes. Many teams chose to continue into the fall and spring (Fig. 4).
Over the next year, we worked to enhance the guide visually and increase the number and type of activities. We also created a flexible track plan with suggested pathways through the guide based on the type of project being pursued (science, consumer product, education/community, health, and technology). We altered our phase-based approach to that of "understand, ideate, prototype, and validate," and implemented the improved guide starting in the summer of 2017. We focused that summer on pushing students toward validation and realized that 10 weeks of roughly quarter-time effort was not sufficient to reach the validation phase. An additional factor contributing to student team success that arose during this summer was the creation of a student prototyping and testing space. For the first time in our program's history, we were able to provide dedicated physical space in which students could work to develop and test their prototypes. By the end of the summer, all student teams again created prototypes, and many teams chose to continue their projects into the fall.
With the realization that time is a critical factor, we began restructuring the VIX process into a multisemester approach. We also recognized that the physical space in which students could work on these projects had to be particularly accessible and, in many ways, driven by student needs. We spent months working closely with students to design and create ideal prototyping and validation space. Then we began a new cycle of VIX in the spring of 2018 under this new timeline and enhanced the physical environment. We formalized exploration and team formation as phases on the front end of the process. Teams were charged with completing the exploration, team formation, and understand phases during the spring semester, individually brainstorming in the break between the end of the semester and the start of the program, and completing the ideate, prototype, and validate phases during the 10-week summer program (still at~25% effort). This change allowed student teams to progress further on projects and to engage more in the design process. By the end of the summer, all student teams had validated their prototypes to some extent. All but one team chose to continue their projects into the fall to continue validation. The remaining students began exploring potential new problems to work on beginning the following spring. The project cycle then continued. The design project aspect of the SyBBURE Searle Program is now a staple of the program. Students now seek out the program as a way to engage not only in research but also in design. While this may skew the outcomes, our lessons learned remain. Although most of our results are anecdotal and gleaned from observation or focus group discussion with students, the implementation of these principles has allowed for successful integration of a design process and environment with the SyBBURE Searle Program: (1) students need a flexible structure when working on team-based design projects; (2) students are more motivated when working in areas of their own interest; (3) physical space with appropriate prototyping and validation tools removes hurdles and enables students to make more progress on their projects; (4) teams work better with a clear, self-selected leader; (5) deadlines with deliverables related to goals are critical; and (6) understand and ideate phases may be concurrent and iterative to more closely mimic real-life innovation and invention.
After sufficient testing, iteration, and learning, we landed on the flexible, phase-based design process shown in the bottom panel of Fig. 3 and top of Fig. 4 along with images representative of the example activities conducted during each phase. The first phase is exploration, during which the students considered their individual interests and related problems. This step has been accomplished through many mechanisms, including a group interest network diagram created with sticky notes, speed dating, and more hands-on approaches. Our group sticky note interest network diagram was particularly useful. In this activity, students write their name and a single interest or problem area on a sticky note, with no limit to the number of ideas they could propose. All of the notes are then compiled and sorted into a word cloud-type diagram, with common or related sticky notes close together in nodes and lines connecting lesser related nodes. Students can review the interest network diagram and self-sort around these common interests into groups. The second phase of the process is team formation. We didactically provide the context for the importance of team roles, cover basic content regarding these roles, and encourage students to take a simple online quiz to gain insights on the roles they typically play in teams. Once students form teams, they decide on an initial area of interest and discuss their team roles and the skills needed to complete the project. After the teams are formed, the students enter the understand phase in which they conduct background research, talk to experts/users, and describe a problem they wish to solve. The students then transition to the ideation phase in which they brainstorm solutions both individually and as a group and then select the features to include in their design. Students move into the prototype phase to create an initial prototype of their solution, and then into the validate phase in which they evaluate this prototype against their design features and constraints. The process ends with each team giving a final presentation and demo of their validated prototype.

VIX electronic survey
Twenty-two of 29 students responded to the survey in 2016, and 32 of 32 students responded in 2017. The results of the survey are shown in Tables 3 and 4, with data from 2016 and 2017 reported separately as the exact VIX program that was implemented differed. The main distinction between these two periods was related to the inclusion of exploration and team formation phases and the additional training and activities that accompanied these phases.
As shown in Table 3, data from both 2016 and 2017 indicate that students score learning how to collaborate with others as the most important component of VIX, with the mean rating from students increasing over the 2-year period. Data between the 2 years also indicates that students rate meeting with experts as the least important component. The order of importance for the remaining components is different for the two terms. Learning a problem-solving method saw the largest positive change in student-rated importance over the 2-year period with a difference in the mean rating of 0.39. Conversely, the largest negative change in mean rating (− 0.23) was observed for answering a scientific question which my team and I developed. It is unclear whether changes in the population of students or the VIX program itself contributed to differences in the importance of each component to the students during the 2-year reporting period. Additional years of data and pre/posttesting of the students could clarify whether we are enrolling students with more interest in certain components or providing training in such a way that they come to value different components.
When looking at how the experience addressed the key program components (Table 4), in 2016, the mean rating of all aspects falls between 3.59 and 3.91, indicating that components were nearly equally addressed, with learning a problem-solving method scoring the highestrated and meeting with experts the lowest rated. In 2017, the mean rating range widened to 3.44 to 4.09, Oliver et al. International Journal of STEM Education (2019) 6:36 and the components that were particularly well addressed based on student response included solving a real-world problem and learning a problem-solving method. These two components had the highest mean rating change from 2016 to 2017 (solving a real-world problem, 0.20; learning a problem-solving method, 0.18).
Meeting with experts again showed to be the leastincluded component as it was indicated by the student responses the previous year and also had the largest negative change in mean rating (− 0.15). While we have discussed the logistics of finding and meeting with experts, formulating questions to ask them, and taking thorough notes in several iterations of the VIX program, we have never required the consultation of experts as the timeline of the program does not often line up with the schedules of in-demand experts. This is perhaps a shortcoming of VIX as the survey results clearly show meeting with experts to be both the least-valued and least-addressed component. Given the iterative nature of VIX and the lack of control groups of students to compare these results, we take the results to indicate that we have included what we sought to include, but that there is always room to improve.
Student comments in an open-ended response around improving the program included making it less productfocused and making the structure of the VIX process more flexible. Students also commented on the fit of the activities in the guide to their actual projects, indicating a need for an approach sufficiently flexible to meet the needs of the variety of projects on which they wanted to work. Many students felt that the requirements were arbitrary and could have been achieved with more general guidelines, again suggesting a need for greater flexibility. The solution included adding different activities for different types of projects, along with recommended activity paths to help guide the variety of student projects. We also altered the guide by allowing students to set their own milestones and timeline; upon implementation, however, we learned that the instructor/facilitator should set major milestone dates, as students at this level need assistance with time management. Despite student desire for greater flexibility, the overall productivity during the time period for which the survey was focused (summer of 2016) was such that all groups achieved a viable prototype. One student commented, "I think that was due in large part to the amount of  Solving a real-world problem 0 3 5 6 8 3.86 0 1 5 17 9 4.06 Designing a marketable product 1 2 7 5 7 3.68 0 6 6 14 6 3.63 Meeting with experts 0 2 11 3 6 3.59 2 4 10 10 6 3.44 Learning a problem-solving method (i.e., the process followed this summer) 1 1 5 7 8 3.91 0 3 3 14 12 4.09 Answering a scientific question which my team and I developed 1 1 5 8 7 3.86 1 4 7 9 11 3.78 Scale: not addressed (1), (2), (3), (4), fully addressed (5) Oliver et al. Overall, findings from this survey elucidated many alterations to the VIX structure to improve its framework and design. These alterations were implemented in subsequent versions of VIX beyond the summer of 2017. As the electronic surveys were initially designed for quality improvement, their rigor and experimental design for the case study we present here is limited. Between 2016 and 2017 when the surveys were conducted, numerous variables were manipulated, including major changes to our design guide and the introduction of a physical space; thus, we are limited in the conclusions we are able to draw from these survey results. Additionally, while we have established specific goals for VIX that include student skill development and comfort with the design process, we have not conducted a pre/post-test to establish whether these goals were met. In order to provide some insights into the perspectives of the students participating during this 2016-2017 period, we provide the following interview results.
Interviews to explore student views on VIX and teamwork skill development Students were asked to voluntarily participate in an inperson interview session to gain further insight into the impact of VIX. In the spring of 2018, 13 students were asked the questions listed in Table 1. For this case study, we utilized student responses to questions 1, 2, 3, and 4 to establish if the survey group was representative of the larger population. The disciplinary diversity of the students included majors in biomedical engineering (4), cell and molecular biology (1), medicine health and society (2), mathematics (3), computer science (1), neuroscience (1), chemical engineering (3), and pre-medicine (1), with 6/13 double majors, a subset of the major distribution of the full program. Questions 8,9,10,11,12,13,14,and 15 were used to evaluate and iterate the VIX framework. The interviewer coded student responses to these questions. Seventy-seven percent of the students reported creating a prototype, which occurs toward the end of the VIX process. When asked to indicate the most important aspect of team success, students responded most commonly with "diversity" (16%) and "organization or strategic approaches" (16%). The next most common answers were "having a clear leader" (11%) and "overall teamwork or team dynamics" (11%). Students also mentioned "creativity" (9%), "work ethic" (9%), "group size" (9%), and "communication" (9%) as critical components. Finally, a few students mentioned "passion for the project" (7%), "research" (2%), and "consistency" (2%). These results, including examples of response wordings, are shown in Fig. 5.

Conclusions
As discussed recently by Yeping et al., traditional STEM education needs to be problematized and reconceptualized with a focus on thinking and building twenty-first century skills (Li et al., 2019). Through this case study, we have explored the VIX framework which seems to support teamwork skill development, particularly collaborative thinking, through a flexible process and environment. This framework is also highly independent and tailored to the student's interests. Independence and autonomy are supported by other studies, such as one that found that it was not necessary to force collaborative interactions or to disturb the autonomy and natural interactions of students (Dillenbourg, 2002). Moreover, structuring the process with too much framework will impede the autonomy crucial for student motivation. A similar conclusion was found in a study looking at the influence of the interdependence of roles, rewards, and structure on student interaction (Brewer & Klein, 2006). In this study, the groups with no structured interdependence had significantly more cognitive interactions based on content discussion than the other groups. These findings indicate that structuring interdependence is not always necessary for university students.
Student autonomy over their projects and idea generation-essentially employing an entrepreneurial mindsetmirrors real-world industry trends. Industries with the most significant opportunities for entrepreneurs are technology (educational software, electronic forensics, technology fashion, artificial intelligence (AI)), health (corporate wellness and healthcare), and community and design (green builders and language translation) (Moutray, 2008). Based on findings from our interviews, students seem to develop a role within the project and maintain a sense of individuality within the team. Autonomy appeared to lead to positive collaboration and teamwork, supporting the notion that positive interdependence is a crucial factor in the effectiveness of collaboration (Johnson & Johnson, 2009). Additionally, the requirements for successful teamwork were echoed in a study that identified three critical factors most important to quality results: a "clear purpose and direction," the "right team members," and "trust" (Ware & Kozlowski, 2017). Student responses from our study touch on these three factors, as well as the importance of organization and strategy, leadership, teamwork, and work ethic (LePine et al., 2008;Marra et al., 2016;Rousseau et al., 2006).
Generalizing that STEM students are experienced in working in groups and in regulating their work, the current case study may support student project autonomy as the driving factor for the success of the program. Many engineering programs around the country support undergraduate research training. Many programs also have highly diverse student populations. However, we believe that only a few of these emphasize team-based training for students. The SyBBURE Searle program uniquely provides students from a diverse background with the opportunity to explore and develop projects that are student-driven. Autonomy, combined with a well-structured framework and challenging program demands, fosters collaboration, and the development of teamwork skills. This program provides exposure to team-based skills.
Overall, we believe that the VIX framework supports the goal of developing teamwork skills through studentdriven design projects, thereby building the skills necessary for their future careers. However, there are a number of problem areas around which we are continuing to innovate, including (1) maintaining project momentum Fig. 5 Interview results. Thirteen students were interviewed. The average size of their VIX groups was 4.1 students. Three out of 13 students did not complete prototypes for their VIX project(s), demonstrating that the majority of students interviewed completed the program as intended until that step in the process. From the interviews, students highlighted some key areas related to successful VIX projects, including teamwork, organization, leadership, creativity, work ethic, small groups, communication, passion, research, and consistency. The code used to create this data and example wording from the interviews are listed in the between terms, (2) ideal project/phase duration, (3) student project prioritization during the academic year, and (4) advancement beyond validated prototype to commercialization while maintaining the educational focus of the experience. The goal of education is to equip students with the skills necessary to solve real-world problems. Using multidisciplinary teams to tackle human-centered design challenges allows students to explore problem-solving in a guided, relatively short-term project. The VIX framework we have iteratively developed supports teamwork skill development through student-driven projects following a flexibly structured guide in a collaborative, supportive environment with the appropriate tools to launch students to success.