Effects of a first-year undergraduate engineering design course: survey study of implications for student self-efficacy and professional skills, with focus on gender/sex and race/ethnicity

Abstract

Background

Students’ academic self-efficacy maximizes likelihood for success and retention, yet prior research suggests that historically underrepresented (minoritized) undergraduate students in higher education and in college-level engineering show lower self-efficacy, which has been linked to histories of systemic exclusion. To address such gaps in student success, this work examines the effect of a new first-year undergraduate engineering design course on students’ self-efficacy, as measured by students’ assessment of their ability to achieve engineering design goals, and their confidence in their professional skills such as teamwork, communication, and leadership. It draws upon two aligned survey studies that examine this development (a) among the students participating in the course during the academic semester and (b) among both course participants and non-participants in the year following the course. Survey results for all students were considered, with attention to specific demographic subgroups traditionally underrepresented in engineering.

Results

Analyses indicate effect of the course on self-efficacy and other examined constructs, such as communication and teamwork, during the course semester and continued effects in engineering design self-efficacy and tinkering self-efficacy in the year following course participation. Results also reveal differences for specific racial/ethnic and gender/sex subgroups in numerous constructs, including suggestion of specific effect for female students.

Conclusions

This study’s focus on the implication of engineering design education on self-efficacy and other critical professional outcomes, as well as its attention to specific demographic subgroups, adds to research on engineering education and the effect of design-focused coursework using project-based learning. The study indicates an increased potential role for such coursework, as early as the first year of a university trajectory, in fostering student growth and increased representation in the field. Findings on differences by gender/sex and by racial/ethnic groups, including clearer positive effect for female students but more complexity in effect for underrepresented racial/ethnic groups, support added research probing experience and outcomes within and across these groups.

Introduction

Recruiting and retaining engineering students is especially important in modern times; technology-related careers employ millions in the U.S. alone, and projections suggest forthcoming critical shortages in the technology workforce globally (da Costa, 2019; United States Department of Labor, 2017). Despite the growing importance and appeal of technology-related careers, engagement and retention in science, technology, engineering, and math (STEM) majors continues to be a challenge and disproportionately affects students of color. For instance, Asian American and White students are more likely to continue into the second year of an engineering program than are their African American, Latino, and American Indian/Alaskan Native peers (American Society for Engineering Education [ASEE], 2017). There has been no notable increase in undergraduate engineering degrees awarded to African American students across the 2010s (ASEE, 2022; National Center for Science and Engineering Statistics, 2020), and a lower number of female-identifying students declare engineering majors as compared to male-identifying students. While rates of retention for female and male engineering students are similar, only 24% of bachelor’s degrees are awarded to female students (ASEE, 2022).

Educational researchers have linked these gaps to historical institutional and structural racism and sexism within education, including STEM higher education, resulting in the systematic exclusion of racial and ethnic minorities and women (Graves et al., 2022; McGee, 2020). Students of historically minoritized identities within STEM fields may be required to “emulate or embody hegemonic values, navigate an environment that is hostile to their identities, or leave the field” (McGee, 2020). These circumstances, and the myriad ways in which they are manifested, lead to specific challenges experienced by students of historically minoritized identities, including lowered sense of belonging and self-efficacy (Cuellar, 2014; Lent et al., 2008; Marra et al., 2009). Despite the significant hurdles, specific efforts have been shown help create a more inclusive climate and thereby start to address these historical inequities. This includes mentorship and relationship development, development of counterspaces, a focus on perspective and growth mindset, and changes in pedagogical approaches (King & Pringle, 2019; Kricorian et al., 2020; Lisberg & Woods, 2018; Ong et al., 2018).

In parallel to these efforts, and not solely to support minoritized students, various strategies have been employed to engage students in their first year of engineering studies (Brannan & Wankat, 2005; Reid et al., 2013). Educators have deployed research-based teaching methods to boost self-efficacy, in order to increase engagement and retention (Beier et al., 2019; Brake & Curry, 2016; Marley & Tougaw, 2019), and evidence of self-efficacy’s effect on academic development and retention suggests the importance of augmented research in this area. For instance, additional research is needed to better understand the nuanced experiences of students from historically underrepresented, or minoritized, racial/ethnic groups to promote retention of these students in STEM fields (Litzler et al., 2014; Sheu et al., 2018).

This study describes the impact of the introduction of a first-year design course on engineering students at a highly selective university in the southeastern United States. Developed to offer mastery experiences with engineering prototyping tools, client-based design projects, and technical communication, the course provides students opportunities to build self-efficacy. Specifically, our study evaluated the effectiveness of the course by tracking the importance of core constructs in the development of self-efficacy, including a focus on engineering/academic engagement and professional skills such as communication and teamwork. The current study incorporated a pre- and post-intervention assessment to examine changes in self-efficacy during the first-year course. It additionally included data collected later in students’ academic trajectories to determine how design-focused coursework may affect development following course enrollment. This study’s focus on the implication of engineering design education on self-efficacy, including its attention to specific demographic subgroups traditionally underrepresented in engineering and its inclusion of data during the semester of participation as well as 1 year following participation, provides a direct contribution to the role of first-year engineering design coursework in student success.

Conceptual background

Self-efficacy in educational development and engineering

Bandura (1994) defines self-efficacy as an individual’s assessment of one’s ability to achieve desired outcomes and describes it as a key factor influencing individual behavior. Perceptions of one’s self-efficacy influence motivation and perseverance “in the face of difficulties through the goals [people] set for themselves, their outcome expectations, and causal attributions for their successes and failures” (Bandura, 2012, p. 13). This creates a cycle by which individuals with higher self-efficacy are more likely to behave in ways that maximize chances for success, thereby further increasing their sense of efficacy. Franzblau and Moore (2001) describe self-efficacy as a social construct, suggesting that large-scale sociocultural marginalization of minoritized groups demonstrably affects perceptions of their self-efficacy.

Self-efficacy in academic context describes a student’s beliefs about his or her ability to achieve educational goals (Elias & MacDonald, 2007). A systematic review demonstrated academic self-efficacy as an important predictor of academic performance among college students, as well as an influence on emotions regarding learning overall (Honicke & Broadbent, 2016). Within the field of engineering, academic self-efficacy is of particular interest because it has been shown to be a predictor of persistence in the field, even after controlling for other factors such as objective math ability and high school achievement (Lent et al., 1986). Beyond general applicability to all students, self-efficacy has been identified as a target of interventions aimed at retaining students who have historically been underrepresented within engineering-related disciplines (Hutchison‐Green et al., 2008).

Strategies to increase student self-efficacy may include intentional creation of “mastery experiences”, which are opportunities for students to experience success in overcoming challenging tasks (Bandura, 1977). Undergraduate engineering students often cite such challenges when asked to identify experiences that influence their self-efficacy or confidence in their engineering abilities. They discuss successes in both their coursework and in applied tasks, such as the design of a functioning device (Hutchison et al., 2006; Marley & Tougaw, 2019; Usher et al., 2015). Other factors identified by engineering students as influencing their self-efficacy within a given course include their understanding and learning of course material, their own motivation, and their course-related abilities (Hutchison et al., 2006).

Research has examined differences in self-efficacy based on student characteristics, such as sex and gender, race, and ethnicity.¹ Litzler et al. (2014) used data in the USA from the nationwide Project to Assess Climate in Engineering (PACE) survey of undergraduate engineering students to examine differences in STEM confidence across student demographics, with particular interest in gender and race/ethnicity. White women reported lower confidence in STEM than their male counterparts. African American and Hispanic men reported higher confidence in STEM compared to White men. In China, Chan et al. (2022) noted the moderating effect of traditional gender role beliefs in their conclusion that girls were more likely to show lower self-efficacy and motivation to pursue a STEM degree than boys. These results were corroborated by a meta-analysis, based on publications from around the globe, related to student self-efficacy in STEM fields that found female respondents experienced greater negative affect (e.g., anxiety) in relation to STEM learning compared to men (Sheu et al., 2018). This study also found that racial/ethnic minorities reported less incidence of negative affect (e.g., lesser incidence of anxiety) in relation to STEM learning than White students; however, the researchers were not able to disaggregate by race/ethnicity to examine differences for specific groups. Regarding first-year engineering, physics, and mathematic courses at a large research university in the USA, sizeable discrepancies between self-efficacy and grades have been noted, with men appearing significantly more confident than women despite small or reverse direction differences in grades (Whitcomb et al., 2020). More recent longitudinal research by Andrews et al. (2021a) evidences differences in self-efficacy by gender in upper-division students, with men reporting higher self-efficacy than women. These differences are based on the aforementioned reality that educational institutions, including STEM education, have been developed to reflect, and have historically stronger enrollment of, White men (McGee, 2020).

Professional competency development

Professional skills and competencies, such as teamwork, communication, and leadership, are posited to benefit learners and performance as practicing engineers. Anwar and Menekse (2020) found teamwork behaviors among engineering students were associated with improved academic performance, noting added self-regulation based on team processes as a mediating factor. Tang (2021) similarly found that teamwork competencies, including commitment, focus, and communication, were a better predictor of engineering students’ academic success than were individual personality traits. Calls for engineers to develop such skills (Accreditation Board for Engineering and Technology, 2011; International Engineering Alliance, 2014; National Academy of Engineering, 2004) reflect an understanding of these skills as improving performance in the workplace. Given this, it is important to examine potential differences in professional skill development based on student characteristics. For example, within the field of medicine, female physicians tend to score higher on ratings of communication skills than their male counterparts (Roter et al., 2002); similar trends may be expected in engineering. Additionally, qualitative data have suggested that a focus on “soft skills” such as communication and development of team relationships may be an asset to minoritized groups as they seek to build careers within engineering (Hodari et al., 2016).

Design-focused and project-based learning pedagogical practices

Within education, pedagogy including design-thinking and client-based work have capacity to contribute to both self-efficacy and professional competency development. Design thinking refers to cognitive processes used to develop solutions to particular contexts, with a focus on understanding end users and ideating to address needs (Wrigley & Straker, 2017). Recent literature has called for an increase in design and design-thinking across STEM education (Li et al., 2019). Within engineering, design-thinking is critically associated with the engineering design process (Lammi & Becker, 2013), defined as “a systematic, intelligent process in which designers generate, evaluate, and specify concepts for devices, systems, or processes whose form and function achieve clients’ objectives or users’ needs while satisfying a specified set of constraints” (Dym et al., 2005). Project-based learning, which can include a client-driven project within an experiential learning framework, allows students to design and develop a project in response to a client or real-world need and with instructional staff serving in an advisory role (Helle et al., 2006), and there is notable precedent for project-based learning within engineering to enhance student learning (de Los Rios et al., 2010).

Project-based learning and design-thinking have been linked conceptually and operationally (Lin et al., 2021; Parmar, 2014), given the aligned focus on design process. There is evidence of design-focused pedagogical practices, which provide iterative opportunities for mastery, as associated with increased self-efficacy (Hilton et al., 2020; Ohly et al., 2017; Wingard et al., 2022); this includes specific evidence of design engineering education experiences positively impacting students’ self-efficacy, including design self-efficacy, with evidence of specific benefit for female students (Siniawski et al., 2016). Recent work presents evidence that students who use a makerspace for a course assignment show significant, positive increases in several measures, including design self-efficacy, belonging to the makerspace, and belonging to the engineering community, although gaps in gender and across racial groups still exist (Andrews et al., 2021b). Moreover, working with an external client enables students to appreciate the meaning and impact of their work in the community (Coyle et al., 2005; Zarske et al., 2011). Additional work has indicated gains in professional skills (e.g., teamwork, leadership, communication) from the use of project-based, experiential learning-informed, and client-based work in engineering coursework and other disciplines (Beier et al., 2019; Gremler et al., 2000; Guo et al., 2020; LaForce et al., 2017).

Study focus

Based on the aforementioned results linking self-efficacy and professional skills with academic success (Anwar & Menekse, 2020; Honicke & Broadbent, 2016; Hutchison‐Green et al., 2008; Lent et al., 1986; Tang, 2021), we begin with the concept that self-efficacy development, combined with development of professional skills, will facilitate academic success and career readiness. We further hypothesize that a design-focused and project-based educational program, which includes a focus on iterative prototyping and team-based work with a client-facing project, would facilitate these self-efficacy and professional skill gains. Of particular interest is the implementation of this intervention in the first year of undergraduate education. This paper examines a project-based and design-focused pedagogical program integrated at the very start of undergraduate education. We assess its impact on student engineering attitudes, with a focus on self-efficacy and professional skills. We additionally pay specific attention to potential for differential effect by gender/sex and race/ethnicity of participants.

This effort expands current scholarship on first-year undergraduate engineering education through its study of an innovative offering of a first-year engineering design course focusing on prototyping and iterative refinement of a client-based project (Reid et al., 2018). In addition, this study advances scholarship based on its specific empirical design and methodology. While there has been increasing attention paid to the role of design-thinking and design self-efficacy in the development of engineering students, much of this work has been cross-sectional in nature (Blizzard et al., 2015; Coleman et al., 2020; Siniawski et al., 2016). Emerging work advocates for movement towards a longitudinal lens (Andrews et al., 2021a). The current study includes data collected a year following course engagement for participants as well as a non-participating comparison group. While this study does not link individual-level data between the course engagement semester and the following year, it does provide added focus on course effect later in an academic trajectory. In addition, as evidenced by findings from Sheu et al. (2018) and Litzler et al. (2014), additional research is needed to better understand the nuanced experiences of students from underrepresented racial/ethnic groups to promote persistence of these students in STEM fields. The present study examines subgroups traditionally underrepresented in engineering, including a focus on African American/Black and Hispanic/Latino students (often reported joint within underrepresented minorities (URM)), as well as examining results by gender/sex.² This focus on subgroups, including underrepresented populations in engineering, represents a critical step in further understanding the role engineering design curricula may play in students’ development.

Below, we describe the focal program/intervention, the specific research questions, and the study methods.

Focal intervention: engineering design first-year course

Engineering design first-year course overview

The focal engineering school provided an opportunity to examine the effect of a design-focused and project-based course on student engineering attitudes and professional competencies. In particular, the engineering school developed a first-year course, Introduction to Engineering Design and Communication (EGR 101), which could serve as a model to test the effect of design-focused, project-based engineering education on self-efficacy and professional skills. This course was implemented in a School of Engineering in a highly selective R1 university in the U.S. South. EGR 101 began with a pilot student group in Fall 2017, with students (~ 50) in the pilot course selected at random by the Associate Dean. Students received credit for the course, so there was no disincentive to participate. The timing of the course and the availability of other required courses (e.g., math) were such that the cohort was not biased toward or against students with particular types (or absence) of AP credits. From 2018 onward, all incoming engineering students (~ 350) were enrolled in EGR 101.

In EGR 101, students learn an engineering design process (Daniels et al., 2018). EGR 101 was designed based on best practices in first-year programs as well as engineering education (Dym et al., 2005; Freeman et al., 2014; Prince, 2004; Prince & Felder, 2006). The one-semester course emphasized collaborative and cooperative learning, engagement in authentic problems, use of near-peer mentors, and active learning. The learning outcomes of EGR 101 were: (1) apply the engineering design process to meet the needs of a client; (2) develop proficiency in two or more prototyping strategies and iteratively prototype a solution; (3) communicate the critical steps in the design process in oral, written, and visual formats; and (4) work collaboratively on a team. In EGR 101, students had considerable opportunity to engage not only with a design process, but also with a range of prototyping tools and strategies as they iteratively refined their solution. Overall, this intervention targeted the development of design and professional skills to build engineering self-efficacy in first-year students.

Each section of the course had two to three faculty instructors who mentored teams through the design and prototyping process and graded all team assignments. Upper-class undergraduate engineering students served as teaching assistants (TA), with some TAs embedded in the class and others available in the makerspace classroom during the evenings and weekends to support prototyping. Starting the first year of required enrollment for all students (2018–2019), given the number of students enrolled, students were divided into seven separate sections of 40–75 students each.

Focus on engineering design process

While first-year engineering courses are common, few focus so intently on students learning an engineering design process through a client-based project (Beier et al., 2019; Daniels et al., 2018). Prior to the start of the course, instructors solicited projects from colleagues within the university and from individuals, non-profits, and companies in the broader community (Table 1); there was no financial cost to the client. By offering a diverse range of problems, students could select a project that met their intellectual and technical interests. Based on student preferences, instructors assembled teams of four to five students. Student teams then worked together for the entire semester on their assigned design project.

Table 1 Sample projects

Student teams completed their client-based projects following an engineering design process (Fig. 1). Similar to other design heuristics taught in universities and used in industry, this model focuses on seven key steps: (1) defining a client’s need, (2) performing relevant background research, (3) establishing quantitative design criteria, (4) generating solution ideas, (5) selecting an appropriate solution using a Pugh matrix, and (6) iteratively prototyping and (7) testing the solution. Following a flipped classroom model (Talbert & Bergmann, 2017), pre-class videos described the key terms and procedural knowledge necessary to complete the steps in the engineering design process and knowledge was assessed via short online quizzes. As each step of the engineering design process was introduced, teams applied that step to their own projects. Teams spent almost all class time (roughly 5.5 h/week) actively working on their client-based projects. The authors estimated that students spent 3–10 h per week outside of class on course requirements, although this has never been tracked at the team or individual level. Out-of-class activities include watching pre-class videos, editing assignments (e.g., technical memos), and prototyping.

Fig. 1

Engineering design process

Prototyping and physically constructing a design solution was a central feature of EGR 101. The classroom design space included various tools and equipment for prototyping, such as 3D printers, laser cutters, power tools, hand tools, sewing machines, and soldering stations, as well as many low- and medium-fidelity materials such as wood, PVC, fasteners, glue, tape, circuit components, tubing, and cardboard. To build prototyping skills, students completed two tools mastery projects concurrent with the Design Analysis Stage (Fig. 1) at the beginning of the semester. Choices included computer-aided design and 3D printing, circuits and microcontrollers, laser cutting and bending, woodworking, and machine shop (mill and lathe). Each tools mastery project had introductory and explanatory material (often via video) to support student learning and was supported by TAs with expertise in prototyping. Because physical prototyping and technical skill development were key learning outcomes, student teams spent more than half of the semester in the prototyping and testing phases of their client-based projects (Fig. 1).

In addition to technical skills, teamwork, project planning, and technical communication were critical for the successful completion of the client-based design project. Like the steps in the design process, these topics were supported using pre-class videos and active in-class support. Student teams documented the results of their journey through the engineering design process through a series of technical memos, oral presentations, and a poster. Support for this emphasis on communication came from the university’s writing program and embedded writing consultants that met with teams during class. Near-peer TAs supported teams to form cohesive, high-performing, self-directed teams.

Research questions

Through our work, we sought to answer the following questions:

(1a) Does a first-year engineering design course using project-based learning pedagogy affect participants’ engineering attitudes (primarily self-efficacy) and professional skills during the course semester? (1b) How might the effects of the course differ based on student specifically race/ethnicity and gender/sex?

(2a) Does a first-year engineering design course using project-based learning pedagogy affect participants’ engineering attitudes (primarily self-efficacy) and professional skills in the year following course enrollment? (2b) How might the effects of the course differ based on student specifically race/ethnicity and gender/sex?

We examine these effects through two related survey studies implemented at distinct timepoints: within the course semester (addressing Q1a,b) and a year following the course (addressing Q2a,b). Our focus on engineering attitudes primarily addresses self-efficacy in engineering, including ability to succeed in engineering overall, in tinkering, and engineering design. These foci were determined based on aims of the engineering design program as well as conceptual research interest.

Methods

These questions were addressed through two related studies described below: Study 1, focused on participants within the first-year course semester, and Study 2, focused on course participants and a non-participant comparison group in their sophomore year. Both studies utilize related surveys examining focal constructs in engineering attitudes and professional skills. These studies are addressed jointly in this paper given their strong conceptual and empirical linkages, and because the two together provide more comprehensive response to our research questions.

Two additional points help to contextualize the empirical designs of Study 1 and Study 2 and the reasons they are described separately. First, during Spring 2018, the engineering school determined to require EGR 101 for all entering engineering students in subsequent years. With this, we were able to examine change within course participants during their semester of EGR 101 of engagement (Study 1), but we were not able to also examine a comparison group (without EGR 101 participation) during their first year. Study 2 thus provided two key elements. First, it provided a longer-term lens on participants. Moreover, it permitted focus on the difference between a comparison group (those enrolling in 2017–2018, the year before the course was required) and a treatment group (those enrolled in 2018–2019, the first year the course was required).

Second, though Studies 1 and 2 are conceptually connected, individual student datapoints could not be directly linked between the two studies. In Study 1, the IRB permitted identifiers that allowed for linking of pre-course and post-course surveys. That, however, did not allow for specific student identification; identifiers were participant-created and were based on students’ telephone numbers and dates of birth. In Study 2, the study team received permission to use university-associated student IDs. Due to the change in IRB permitted identifiers between the two studies, we were not able to determine the number of students who completed the survey in both Study 1 and Study 2, nor were we able to directly link individuals’ data between the two studies.

The campus IRB approved all research processes and informed consent was obtained from all participants. Participants included primarily students who matriculated into the School of Engineering when they entered the university; though, in few cases, participant students transferred into the School of Engineering after matriculation.

Study 1 (Q1): during course semester, focus on participants

To address the research questions in the period during which the course was offered, we developed and utilized a pre-course (Time 1, or T1; start of semester) and post-course (Time 2, or T2; end of semester) survey assessment. Development of the survey was informed by external research and prior work among this article’s study team. First, the research team developed a program logic model that included identification of hypothesized proximal outcome constructs. Second, the team conducted initial exploratory assessment with open-ended qualitative data collection from Fall 2017 pilot course participants, drawing from an exploratory sequential mixed-methods design framework (Creswell & Creswell, 2017; Daniels et al., 2018). With these two processes as bases, we defined specific intended outcome constructs within a central macro-level domain of Engineering Attitudes, primarily addressing engineering self-efficacy; this was the main conceptual focus.

We additionally defined secondary outcome constructs in a Professional Skills domain, including areas such as teamwork, communication, and leadership.³ This resulted in 7 outcome constructs falling within these two domains. The team selected validated instruments (multi-item scales) to address discrete outcome constructs within these domains, where such instruments were available and aligned with intended program outcomes (Table 2), including self-efficacy for general engineering tasks and skill-specific tasks, including design skills (Blizzard et al., 2015; Carberry et al., 2010; Mamaril et al., 2016; Usher et al., 2015). For one construct (Engineering Academic Engagement), the team did not identify an existing specific measure that it felt directly assessed the desired areas; it thus utilized items assessing facets of engagement (interest, enjoyment, career aims alignment) that were identified in review of academic engagement literature and course aims, with these items reviewed by engineering faculty to assess face validity and assessed for internal reliability in analysis. All constructs included were intended to be impacted by the course. The survey also addressed intended major, race/ethnicity, gender (how one self-identifies), and course section. Additional methods, including the full survey instrument, are available as Additional file 1 accompanying the online article.

Table 2 Outcome construct measures

The target sample included all students participating in the course. Data collection occurred in 2018–2019 and 2019–2020, which were the first two years during which the course was implemented for all students. Table 3 provides an overview of the timing of data collection relative to first-year course implementation. Data were collected as an electronic survey using Qualtrics.

Table 3 Overview of data collection implementation, Study 1

In total, 343 respondents completed data collection at both T1 and T2 (of 690 total students in classes for a response rate of 50%). Researchers linked T1 and T2 data at the individual level for paired analysis. Table 4 shows sample characteristics of participating students in Study 1. For the purposes of analysis, researchers grouped students who identified as African American or Black, and/or Hispanic/Latino into a single group (AABHL, akin the how URM is used as underrepresented minority). While the experiences of individuals within this group are undoubtedly unique, they have been historically underrepresented within engineering fields. In addition, combining these students into a single group permitted analyses based on race/ethnicity that otherwise would not have been feasible, given the relatively small number of AABHL individuals participating in the study.⁴ For both studies, we address all racial and ethnic groups, but we note difficulty in inferring meaning from results for the “Multiracial/Other” group (meaning respondents selected more than one racial/ethnic identity or another identity), given the overt diversity of racial/ethnic backgrounds represented.

Table 4 Sample characteristics, Study 1

For analysis, we pooled the students into one sample to increase the sample size overall and for individual demographic groups. For each outcome construct, we created a construct measure. This was based on the mean of the individual variables included in each construct’s multi-item measure. We assessed scale reliability for each scale using Cronbach's a (α ≥ 0.7 for all scales at each time point; see Table 2 for specific values).⁵ Outcome variables were first analyzed in SAS using descriptive statistics, and effect size calculations (Cohen’s d) to determine changes from pre-course to post-course. Effect size was used to address strength of change, to provide added lens on descriptive statistics given the exploratory nature of this work, and to account for cases where small sample sizes may limit evidence based on significance alone (e.g., AABHL students). Effect sizes are considered relatively between groups but with d < 0.3 discussed as small effect, d < 0.6 as medium, and d ≥ 0.6 as large. Effect size was examined for the full sample as well as for gender/sex and racial/ethnic subgroups. In addition, for the full sample, paired t-tests were used to assess significance of change. To further examine differences by subgroup, we used multiple linear regression models accounting for pre-score, gender/sex, race/ethnicity, and gender/sex by race/ethnicity interactions; these models also controlled for year of course enrollment, which was found to be significant in some cases, and models examining just gender/sex or race/ethnicity also examined interactions between these demographic variables and pre-scores. Given the exploratory nature of this work and smaller sample sizes for some groups, we report on results that are approaching significance (0.1 ≤ p < 0.05) as well as those that are significant at the p ≤ 0.05 threshold.

Study 2 (Q2): after course semester, focus on participants and comparison group

To address the research questions considering effect a year following course engagement, a survey was adapted from Study 1. It included all Engineering Attitudes measures as well as the Professional Skills teamwork measure, based on outcomes of core conceptual interest to course designers and based on evidence of effect in Study 1. Additional demographic information was collected and linked to surveys from academic administrative records. The design of Study 2 included sophomore students who participated in EGR 101 and students who did not by distributing the survey to sophomores at two timepoints. In Spring 2019, the survey was distributed to all sophomores enrolled in the engineering school. This survey captured the students who participated in the 2017 pilot the year prior (~ 15% of students entering the engineering undergraduate school in 2017–2018) as well as the majority who had not participated in EGR 101 in that year. In Spring 2020, the survey was distributed to all sophomores enrolled in the engineering school, all of whom all students participated in EGR 101 in their first year. See Table 5 for an overview of data collection timing relative to first-year course implementation.

Table 5 Overview of data collection implementation, Study 2

With this data across these 2 years, we categorized respondents for analysis into two groups: (1) those who had taken the course (majority from 2020 sophomore survey, with smaller numbers of 2019 sophomore respondents who had taken EGR 101 in that first pilot year), and (2) those who had not taken the course (applicable only to respondents to the 2019 sophomore survey; all 2020 sophomores included had taken the course). This permitted a complementary focus using a comparison group, and it allowed us to examine whether any difference between a treatment and comparison group is evident in the year following course participation.

All surveys were administered electronically through Qualtrics. In total, 295 sophomore students completed the survey representing a 53% response rate. Table 6 shows sample characteristics of participating students in Study 2. As with Study 1, for purposes of analysis, researchers again grouped students who identified as African American or Black, and/or Hispanic/Latino into a single group (AABHL).

Table 6 Sample characteristics, Study 2

Demographics for students in Study 2 (N = 295). For Study 2, gender/sex and race/ethnicity data are based on university administrative data, and multiracial identification was not an option. For Study 2, “other” includes race/ethnicity categories that allow respondents to be potentially identifiable based on small sample size if named.

For analysis of Study 2 data, we utilized a process parallel to Study 1 to construct outcome measures. Outcome variables were analyzed in SAS using descriptive statistics for all respondents, by condition (participated in EGR 101 or not), by gender/sex, and by racial/ethnic identity. As with Study 1, we assessed scale reliability for each scale using Cronbach’s a (α ≥ 0.7 for all scales at each time point; see Table 2 for specific values).⁶ Effect size calculations (Cohen’s d) were calculated by participation in EGR 101, as well as by participation within gender/sex and race/ethnicity groups, with the same categorization used in Study 1. Independent t-tests compared respondents overall by participation in EGR 101. Multiple linear regression models were conducted for all outcome constructs controlling for participation in EGR 101, gender/sex, and race/ethnicity, with interaction effects also examining these demographic characteristics by treatment condition. Year of course enrollment was not included because it was not applicable to the control group and because separate analyses examining just the treatment did not find significant differences by year of course participation. As with Study 1, given the exploratory nature of this work and smaller sample sizes for some groups, we report on results that are approaching significance (0.1 ≤ p < 0.05) as well as those that are significant at the p ≤ 0.05 threshold.

Results

Study 1 (Q1): change during course semester

We assessed the degree to which EGR 101 participation affects participants’ reported development on 7 outcome constructs during the course semester. Results integrate paired t-tests, effect sizes, and linear regression models. For clarity in writing, text generally provides specific statistical test results only for the overall group. Where results pertaining to subgroups are discussed, tables should be considered for added information on statistical results.

Intervention effect, change over time

Findings indicated that course participants experienced gains in six of the 7 assessed outcome constructs (Fig. 2). Paired t-tests and effect sizes revealed that students improved, at statistically significant levels and with large effect sizes, in engineering design-self efficacy (t = 12.52, p ≤ 0.01, d = 0.72), communication skills (t = 12.90, p ≤ 0.01, d = 0.74), and teamwork skills (t = 10.89, p ≤ 0.01, d = 0.62) between the beginning and end of the course. Results show that students also reported growth, at statistically significant levels and with medium effect sizes, in their tinkering self-efficacy (t = 9.22, p ≤ 0.01, d = 0.53), general engineering self-efficacy (t = 5.89, p ≤ 0.01, d = 0.34), and leadership skills (t = 7.47, p ≤ 0.01, d = 0.43).

Fig. 2

Survey results reported by EGR 101 students at beginning and end of the semester. Reported values are mean ± standard deviation. *** indicates p ≤ 0.001 using paired t-test. Cohen’s d effect sizes are noted as ^L for large effect size (> 0.60) and ^M for medium effect size (0.30–0.60)

Results do not support evidence of statistically significant gains for engineering academic engagement. However, mean scores suggest that the average student reported relatively high academic engagement on the pre-survey (M = 5.57, SD = 1.06), which persisted through the end of the semester (post survey: M = 5.56, SD = 1.27), suggesting ceiling effect for these measures.

Focus on race/ethnicity

Disaggregating the larger sample by race/ethnicity, we found differences between student demographics in magnitude of gains across constructs (Table 7). White/Caucasian and Multiracial/Other students showed medium or large effect size for all constructs except engineering academic engagement (no group saw medium or high effect size in this), but API students saw large effect size only in engineering design self-efficacy, and AABHL students saw medium effect size for only tinkering self-efficacy and engineering design self-efficacy. Within the Professional Skills constructs, White/Caucasian and Multiracial/Other students show medium effect size for leadership skills while API and AABLH students show small effect size.

Table 7 Pre- and post-mean differences and effect size categories, by race/ethnicity

In multiple linear regression models controlling for initial assessment scores and race/ethnicity as well as year of course enrollment (Table 8), API students show near-significantly lower improvement compared to White/Caucasian students in tinkering self-efficacy and in engineering design self-efficacy; Multiracial/Other students show significant or near-significantly higher improvement compared to White/Caucasian students in tinkering self-efficacy. In addition, we observed near-significant or significant interactions based on pre-score for Multiracial/Other students in tinkering self-efficacy, and for AABHL students in tinkering self-efficacy, with lesser growth among those with higher pre-scores. In multiple linear regression models examining Professional Skills constructs (Table 9), the only significant result by race/ethnicity is evident in teamwork skills, where API students evidence significantly lower improvement when compared to White/Caucasian students.

Table 8 Engineering attitudes regression results, race/ethnicity (β, p)

Table 9 Professional skills regression results, race/ethnicity (β, p)

Focus on gender

Within Engineering Attitudes constructs (Table 10), female respondents show higher effect size than male respondents in general engineering self-efficacy (medium for female respondents, small for male respondents) and tinkering self-efficacy (large for female respondents, medium for male respondents). Within Professional Skills constructs, female respondents showed large effect size for teamwork while male respondents showed medium effect size.

Table 10 Pre- and post-mean differences and effect size categories, by gender

Multiple linear regression models examining Engineering Attitudes constructs and Professional Skills constructs, controlling for initial assessment scores and gender as well as year of course enrollment, do not detect significant differences between male and female respondents or significant interactions by pre/baseline score, though beta results for Engineering Attitudes constructs generally indicate lower scores for male respondents compared to female respondents (Tables 11, 12).

Table 11 Engineering attitudes regression results, gender (β, p)

Table 12 Professional skills regressions, gender (β, p)

Focus on race/ethnicity and gender

Finally, we considered race/ethnicity and gender jointly. When examining effect size (Table 13), results show male White/Caucasian respondents improving with large effect size while female counterparts show medium effect size across multiple constructs. For all other racial/ethnic groups, female respondents show greater effect size than male respondents; this is starkest for AABHL participants, where female respondents show large effect size, but male respondents show no change across multiple constructs. Results further indicated that the gains experienced by students of different races/ethnicities differ in some cases by gender. This is most notable with API and AABHL respondents, with more medium-to-large effect size gains for female respondents in these groups compared to their male counterparts. For instance, female AABHL respondents saw gains in their engineering design self-efficacy with large effect sizes, a finding that was masked when we examined the combined mean scores of male and female students. In fact, male AABHL respondents were the only group of students that did not experience significant gains in engineering design self-efficacy.

Table 13 Pre- and post-mean differences and effect size categories, by race/ethnicity and gender

Effect sizes for the Professional Skills constructs showed male AABHL and Multiracial/Other respondents evidencing greater effect size than female respondent counterparts in communication and leadership skills; we find the opposite gender trend for API students. In addition, male White/Caucasian respondents show higher leadership skills effect size than do their female counterparts. Only Multiracial/Other students show a gender difference in teamwork effect size category, with female respondents showing larger effect size than male respondents.

Multiple linear regression models further reflect benefit for female respondents within certain racial/ethnic groups. In models examining Engineering Attitudes constructs incorporating gender by race/ethnicity interactions (Table 14), we see significant gender by AABHL interactions in engineering design self-efficacy; the interaction is near-significant for engineering academic engagement. Pairwise comparisons of least squares means show female AABHL respondents reporting greater gains than male respondent counterparts. Results also show near-significant interactions in engineering academic engagement for Multiracial/Other students, with female respondents reporting higher scores than male respondents. In models examining Professional Skills constructs (Table 15), we see a significant interaction for API by gender in communication skills, with female API respondents reporting higher scores than male respondent counterparts.

Table 14 Engineering attitude regression results, race/ethnicity*gender interactions (β, p)

Table 15 Professional skills regression results, race/ethnicity*gender interactions (β, p)

Study 2 (Q2): evidence of effect in sophomore year

As with Study 1, results for Study 2 draw from independent t-tests, effect sizes, and linear regression models. For clarity in writing, text below generally provides specific statistical test results only for the overall group; where results pertaining to subgroups are discussed, tables can be used for added information on statistical results.

Intervention effect, treatment vs. comparison

Independent t-tests and effect sizes (Fig. 3) shows that students in the treatment condition reported greater engineering design self-efficacy (t = 5.05, p ≤ 0.001; d = 0.64) and tinkering self-efficacy (t = 2.24, p = 0.03, d = 0.27) compared to students that did not take the course. The effect of EGR 101 participation for outcome variables remained significant in regression models controlling for gender and race/ethnicity demographic characteristics (engineering design self-efficacy: b = 0.70, p ≤ 0.001; tinkering self-efficacy b = 0.40, p ≤ 0.01).⁷ Analysis results did not evidence similar treatment effects for three other constructs: general engineering self-efficacy, engineering academic engagement, and teamwork skills.

Fig. 3

Survey results reported by students who did and did not take EGR 101. The survey was taken at the close of their sophomore year. Reported values are mean ± standard deviation. p value shown as t (< 0.1), * (< 0.05), ** (< 0.01), and *** (< 0.001). Cohen’s d effect sizes are noted as high for > 0.60 and medium for 0.30–0.60

Focus on race/ethnicity

Study 2 analyses also examined differences by race/ethnicity, focused on students who are White/Caucasian, API, and AABHL (Table 16).⁸ Effect size calculations show the treatment group reporting higher scores than the comparison group in engineering design self-efficacy across all racial/ethnic groups, with large effect size for White/Caucasian and API students and medium effect size for AABHL students. In addition, in tinkering self-efficacy, the API and AABHL treatment groups report higher scores than their peer comparison group with medium effect size, whereas this difference is of small effect size for White/Caucasian students. In teamwork, the sole Professional Skill measured, we find a medium effect size for the API treatment group compared to API students who did not take the course; for White/Caucasian respondents, the comparison group actually had higher scores than treatment, though with small effect size. “Other” race students (n = 2) are not included because there were none enrolled in the engineering design course.

Table 16 Mean differences and effect size categories comparing those enrolled/not enrolled in EGR 101, by race/ethnicity

In multiple linear regression models examining Engineering Attitudes constructs and controlling for initial assessment scores and race/ethnicity (Table 17), AABHL students show near-significantly higher results compared to White/Caucasian students in engineering academic engagement and lower results in teamwork. API students show significantly lower reported teamwork skills results compared to White/Caucasian students. However, and notably, results also show significant or near-significant teamwork interaction terms for AABHL by course enrollment and API students by course enrollment. For each, estimated marginal means show the treatment group of these racial/ethnic groups has significantly higher mean scores on teamwork than for those in the comparison group. Multiracial/Other students show significant differences from White/Caucasian students, but the small sample size (n = 2) should be considered in interpretation.

Table 17 Regression results, focus on race/ethnicity (β, p)

Focus on sex⁹

Table 18 shows results by sex for mean differences and effect size. We find that the treatment condition reports higher scores than the comparison for both female and male respondents in engineering design self-efficacy, with an especially large effect size for female students (male: d = 0.51; female: d = 0.97). For tinkering self-efficacy, female students also show higher effect size, although the difference is actually slight (male: d = 0.28, female: d = 0.32) when compared to male respondents.

Table 18 Mean differences and effect size categories comparing those enrolled/not enrolled in EGR 101, by sex

In multiple linear regression models (Table 19), male respondents show higher tinkering and engineering design self-efficacy scores compared to female students. Of note to our study, regression results did not show sex by condition interactions to be significant for any of the outcome variables examined.

Table 19 Regression results, focus on sex (β, p)

Discussion

This paper quantitatively explores the effect of a first-year undergraduate engineering design course on a suite of student-reported outcomes, with a primary focus on Engineering Attitudes (self-efficacy and engagement) as well as Professional Skills, through two related studies. As noted previously, greater self-efficacy has been linked to persistence and achievement of goals within an academic context (Honicke & Broadbent, 2016; Lent et al., 1986). This paper provides a critical lens in assessing short-term gain as well as longer-term effect, and this dual-timeframe approach augments prior work examining outcomes of first-year interventions in engineering programs.

Overall, results provide evidence that a design-focused and project-based engineering course, implemented in a first undergraduate year, positively affects engineering attitudes (focused on self-efficacy) and professional skills development. When examining overall student gains during EGR 101, students in Study 1 reported significant gain from pre- to post-course in a majority of constructs assessed, including in constructs such as engineering design and tinkering self-efficacy, teamwork, communication, and leadership skills—that were most closely related to and targeted by the course content and structure. Furthermore, there was a clear ongoing effect on students’ engineering design self-efficacy: per Study 2, students who participated in the course demonstrated greater engineering design self-efficacy at the end of their sophomore year compared to those who did not participate. This provides evidence of efficacy for the course’s primary intention of self-efficacy development being sustained more than a year beyond course engagement, expanding upon prior evidence of effect within one semester (Siniawski et al., 2016).

In addition, findings regarding differences in gains based on students’ race/ethnicity and gender/sex provide an important step in providing a more nuanced understanding of where disparities exist in students’ experiences of engineering coursework. This lens can identify whether there is opportunity for the course design or delivery to more equitably support gain for all students; it is particularly salient given how dominant institutions, including education systems, have centered relatively advantaged groups (McGee, 2020). Given that African American/Black and Hispanic/Latino (AABHL) students are historically underrepresented in engineering disciplines, their performance in outcomes assessed, relative to White peers, was particularly relevant. Overall, we see evidence of within-semester gains for AABHL respondents in the majority of constructs assessed. At the end of the course, AABHL students’ gains generally showed lower effect sizes than their White peers, indicating that the course may not benefit this group to the degree it does others. This could suggest an opportunity for an additional intervention, or adjustments in the current intervention, to ensure equitable growth and support among these students. While prior research (Gasman et al., 2017; Rodriguez et al., 2021) may help to inform such intervention, additional needs assessment efforts that directly engage students at the focal university (e.g., in added qualitative data collection) would be important in understanding the unique context and experiences of the target population(s). Regression results controlling for baseline, although not statistically significant, show evidence of lesser growth in Professional Skills specifically compared to White/Caucasian students, suggesting specific areas of focus. Yet, by the end of their sophomore year, effect size results show AABHL students in the treatment group demonstrated greater engineering design self-efficacy than their AABHL peers in the comparison group, and regression interactions show higher performance of the treatment group in teamwork skills; this suggests potential ongoing value for this group.

In addition, and complicating AABHL findings, we observed that this study’s baseline assessments, which provide a lens on perspective and ability at course and university entry, showed AABHL students as having the highest baseline scores among the different racial and ethnic groups examined for over half (4 of the 7) of outcome constructs examined, including central constructs of general engineering self-efficacy and engineering design self-efficacy. It is thus possible that their higher baseline scores may relate to lesser growth. It also speaks to complexity of considering self-efficacy change in relation to measures such as retention, particularly because research speaking to persistence in STEM fields and engineering among AABHL and other minoritized students suggests self-efficacy development as a facilitator of retention but also speaks to retention challenges for minoritized groups (Adedokun et al., 2013; Chang et al., 2014; Marra et al., 2009; Raelin et al., 2014). This study’s baseline findings for AABHL students suggest value in further understanding the progression of self-efficacy across an academic trajectory as related to retention and per specific subgroups, and prompts potential value in further understanding how self-efficacy at the start of a higher education trajectory can be best supported and sustained.

We also found intriguing and complicating trends among API students relative to White students. While API students demonstrated growth during the course across nearly all examined outcome constructs, both effect size and regression results indicate reported lesser improvement than White/Caucasian peers across a large number of constructs measured. This is notable particularly given that API students are often assumed to be well-represented and high-performing in engineering disciplines, so they may not be viewed as benefiting from additional supports. Interestingly, Study 2 provided evidence of greater value for API students in the treatment condition compared to those who did not take the course. This suggests a potential difference in shorter- and longer-term effects, as well as difference when examining results within racial groups versus between racial groups, and merits further study.

The Study 1 results, in many cases, also indicate particular gain for students with multiracial or “other” racial identities. This result can be complicated to interpret, as this group by definition is composed of a heterogenous mix of multiracial or other racial identities. However, this may suggest evidence of the particular experience of multiracial students; this would support other research (Campbell, 2009; Mitchell & Warren, 2022) indicating that multiracial groups have difference associators with academic performance and distinct educational outcomes from monoracial peers.

Overall, our focus on race/ethnicity can expand upon Sheu et al. (2018)’s findings, including their observation that STEM is associated with great affect for students from at least some racial/ethnic minority groups; they call for added research on STEM self-efficacy development among by racial/ethnic minority groups who are both under- and overrepresented in STEM and hypothesize that greater affect among minorities may be driven by Asian Americans, as overrepresented and perhaps particularly encouraged in STEM areas, or perhaps by those with multiracial identities or those outside of the specific groups represented here. Our findings support this hypothesis in terms of evidence of greater gain for Asian American and Multiracial/Other students than for AABHL students, which may reflect the multitude of ways that AABHL students experience othering within higher education and STEM spaces and speak to the limiting effect of one specific course in addressing this (Cuellar, 2014; Kricorian et al., 2020). However, this is complicated and possibly countered by evidence of higher baseline self-efficacy scores for AABHL students relative to other ethnic and racial groups; this may reflect a change from prior context to less hospitable departmental and university norms. These complex results further speak to the need to understand effect differences among racial/ethnic groups and consider how those differences may change across an educational trajectory.

Differences in effect by gender/sex were noteworthy, generally showing greater benefit for female respondents than male respondents. This is a critical finding given gender/sex disparity in engineering (ASEE, ), and it indicates potential value of this course model. For instance, during the course semester, we saw greater effect size for female students compared to male students in Engineering Attitudes and in Professional Skills constructs (e.g., tinkering self-efficacy, general engineering self-efficacy, teamwork skills), and we saw greater gains in female API and AABHL respondents in particular compared to their male respondent counterparts. In addition, Study 2 engineering design self-efficacy effect size results showed female students in the treatment group outperforming comparison group peers, though such results were not reflected in regression results significance. This potential gender/sex effect would underscore the value of team-based, design-focused courses for supporting female students in engineering (Coleman et al., 2020; Siniawski et al., 2016). Efforts that increase female students’ self-efficacy, given the role of self-efficacy in academic perseverance and achievement (Honicke & Broadbent, 2016; Lent et al., 1986), may facilitate improved retention. Results suggesting that this course intervention increases female students’ engineering tinkering self-efficacy may help reduce gender- or sex-based differences in engineering engagement.

This study intentionally does not directly compare results across Study 1 and Study 2 due to the inability to identify repeated respondents between the two studies; however, we did note that assessment of general engineering self-efficacy is lower for Study 2, during a sophomore year (both treatment and comparison), relative to Study 1’s first-year post-course data. We observe evidence of similar findings in other work (Marra et al., 2009). Reasons for this are not directly explained by this study, but they may be related to the trajectory of coursework in the program’s second year, which is largely composed of analytical courses that emphasize closed-ended calculations and where performance assessment is largely based upon homework and exams. This presents a potential future area of focus. We note a general lack of effect on engineering academic engagement in both Study 1 and Study 2. For Study 1, we note that students began with relatively high academic engagement, thus potentially limiting evidence of growth. For Study 2 data, however, EGR 101 participants did not outperform comparison students in this construct; results are quite similar for the two groups. Given stronger findings in self-efficacy, it seems that the course increases confidence but not overarching engagement and interest. Finally, Study 1 regression analyses controlling for the year in which the course was taken (2018–2019 or 2019–2020) indicated non-significant differences by year in all core constructs except teamwork skills, with 2019–2020 participants outperforming 2018–2019 participants. While the course was parallel between the two years, the teamwork finding may indicate changes to instructor preparation in the second iteration.

Though our study adds to understanding of engineering instruction and the effects of early undergraduate engineering design coursework, it has several limitations that may help to inform and augment similar research efforts in the future. First, and unsurprisingly, our analyses were challenged by relatively small sample size among students from underrepresented minority groups, particularly students who identified as African American/Black or Hispanic/Latino. These students’ underrepresented minority (URM) status meant that they are by definition smaller in number among engineering students, and this necessitated the grouping of minority students into a single underrepresented minority category (AABHL) for analytical purposes. However, existing literature speaks to difference in social and educational experience and attainment within and between ethnic groups comprising this category (Farley & Alba, 2002; Thomas et al., 2009). A larger and/or more diverse cohort of participants would ideally allow for larger sample size even among underrepresented minority groups, which would permit the disaggregation of AABHL students to examine differences in experience among these groups. This is also relevant to the API subgroup, as students in this group may come from diverse socioeconomic, cultural, and ethnic backgrounds, which may lead to differences in educational and STEM engagement and experience (Baker et al., 2007; Eng et al., 2008; Kang et al., 2023; Lee, 2006); per data available, we were not able to disaggregate among this group. Our sample size was additionally affected by the survey response rate among participating students. While a larger number of students completed the Study 1 pre- and post-course surveys, the number of students with data at both timepoints (thus allowing for pre/post-analyses) was smaller. Additional efforts could be made to encourage and potentially incentivize participation among students to increase the response rate and reduce opportunity for potential bias among respondents relative to the population. We also acknowledge that combining data sets across years (e.g., 2017–2018 with 2018–2019) may have some unintended and unknown impact, but this decision was necessary to feasibly evaluate the impact on students from underrepresented minority groups.

Additional aspects of study design due to constraints and realities of the context can also be seen as limitations and inform future work. While our study benefitted from the incorporation of multiple timepoints across 2 years of students’ academic trajectories, we were unable to link Study 1 and Study 2 data for individual students. This was due to differences in data collection processes between the two years and further complicated by the lack of a comparison group in Study 1. Study 2’s comparison group also included only one cohort, while participants in the treatment drew from two different cohorts: a smaller number taking the course the same year as the comparison group and a larger number taking the course the following year, when it became a first-year requirement. While examinations of the treatment group by year do not indicate difference by year, this still presents an added variable to consider. Finally, one cohort of Study 2 respondents completed the sophomore year survey during Spring 2020, during the early stages of the COVID-19 pandemic. While it is possible that the context of general turmoil facing college students at this time may have affected students’ perspectives on their confidence and future plans, it may be all the more noteworthy that we continued to see treatment students outperform the comparison group at this time point.

Based on findings from Study 1 and Study 2, current study limitations, and additional relevant areas of work, we envision several opportunities for future research building upon this study; this understands this study as taking an exploratory lens. Overall, we would encourage further explorations of the effect of engineering design courses on self-efficacy and professional skills; our work provides suggestion of effect, but further work could explore this in a different context or with different instructional models. One area of exploration might be which aspects of EGR 101 (e.g., success following iterative prototyping, autonomy in choice of project, etc.) contribute most significantly to development of self-efficacy.

Added research, perhaps building on this study’s design, could be extended to other institutions that are considering introducing similar first-year design courses. Such research could explore effect of distinct university contexts, shed light on which different aspects of a course (e.g., teamwork, working with a real-world client, mastery of technical skills) are most impactful in related constructs, and help to address issues of smaller sample size for certain subgroups or provide more nuanced variables for of racial and ethnic identities.

In addition, when examining student trajectory over time, an additional lens on retention (i.e., whether students remain within engineering-related majors and ultimately obtain a degree) would be valuable given the known discrepancies in retention of students from varying backgrounds; we are taking specific steps currently. Regarding datapoints examined, ability to link data directly across first-year and sophomore year should also be built into future work, and an additional timepoint even later in an academic trajectory could provide a valuable lens on effect throughout university enrollment. This could be examined in conjunction with retention data to further examine longer-term associations between self-efficacy and persistence in engineering.

Further work should also explore the specific experiences or processes underlying gender/sex and race/ethnicity differences shown in this study. Our study was designed to identify differences, but its design did not permit understanding of specific factors prompting these differences. Added qualitative research, moving towards an explanatory mixed-methods design, could provide added understanding of mechanism underlying patterns addressed in this paper. This is under discussion as an extension of the current study and could include specific experimentation regarding adjustments that may support more equitable gain, such as an iteration including project foci addressing issues directly germane to underrepresented groups. Other recent work has indicated academic hope and STEM belonging are associated with persistence for underrepresented students (Hansen et al., 2023), providing a lens on other critical psychological and attitudinal factors that could be explored in tandem with a focus on self-efficacy. Within a global context, we encourage a similar design model beyond the US but would foresee this work as necessarily situated within the context. For instance, we view this study’s focus on historically marginalized and minoritized subgroups as relevant broadly, though the specific subgroups of focus would likely vary by national context; we would value future research taking this subgroup lens as contextualized within in other national settings. This study’s note of relatively high baseline self-efficacy for AABHL students, coupled with prior similar evidence (Litzler et al., 2014), could also support a broader continued inquiry into the link between self-efficacy and educational outcomes for distinct ethnic and racial groups, both within and beyond the US.

Conclusions

Overall, results provide evidence that a design-focused and project-based engineering course, implemented in a first undergraduate year, positively affects engineering attitudes (focused on self-efficacy) and professional skills development during the course and positively affect engineering design self-efficacy even into the year following the course. Results also show difference in effect by gender/sex, with trends toward greater benefit for female participants compared to male counterparts in the course semester and thereafter. There is evidence of additional differences by race and ethnicity, though with more complexity. We see evidence of lesser gains for African American and Latinx respondents during the course as compared to White peers, but data collected a year later show these students demonstrating greater engineering design self-efficacy than their peer African American and Latinx students who did not participate in the course. These results support the value of, and can inform, further curricular efforts to integrate team-based engineering design projects into undergraduate curriculum in students’ early years. In addition, the design of the studies discussed here, including the focus on student subgroups and on effect a year following course engagement, can inform ongoing research seeking to understand mechanisms for addressing student self-efficacy. This project’s findings on differences by gender/sex and by ethnic and racial groups support added research probing experience and outcomes within and across these groups, and the variation in results across years supports the value of ongoing research examining students at different stages in, and as they progress through, an educational trajectory.