Self-efficacy and belonging: the impact of a university makerspace

Abstract

Background

In recent years, makerspaces have become increasingly common venues of STEM education and are rapidly being incorporated into undergraduate programs. These spaces give students and instructors access to advanced design technology and facilitate the incorporation of a wide variety of projects into the curriculum; however, their impacts on students are not yet fully understood. Using matched survey responses (i.e., repeated measures) from undergraduate students enrolled in engineering courses that assigned a makerspace-based project, we evaluate how the use of a university makerspace impacts students’ attitudes towards design, engineering, and technology. Further, we examine whether there are differences based on students’ year in program, gender, and race.

Results

Paired t-tests were used to analyze whether and how nine factors changed within individual students over one semester. Analyses revealed that students who visited the facility showed significant gains in measures of innovation orientation, design self-efficacy, innovation self-efficacy, technology self-efficacy, belonging to the makerspace, and belonging to the engineering community. Subsequently, repeated measures analyses of variance (RMANOVAs) on the students who visited the makerspace revealed significant main effects of students’ year in program, gender, and race, as well as interactional effects of both year in program and race with time.

Conclusions

These results affirm the value of incorporating makerspace-based projects into STEM curricula, especially during early coursework. However, our analyses revealed consistent gender gaps in measures of self-efficacy before and after using the makerspace. Similarly, gains in belonging to the makerspace were not equal across racial groups. We conclude that while makerspaces are fulfilling some of their promise for educating innovative problem solvers, more attention needs to be paid to avoid reproducing disparities in STEM education that are already experienced by female students and racial minorities.

Introduction

Brought to the public eye through Maker Media and the Maker Movement, a makerspace is broadly defined as a facility that “enables making;” typically, these spaces include cutting edge technology and a variety of traditional hand tools, but the available equipment and layout of the spaces vary greatly between facilities (Barrett et al. 2015; Dougherty 2012). These spaces have become a recent topic of interest and have been widely lauded as a disruption with the potential to increase student access to and engagement with STEM (Hoople et al. 2020; Martin 2015; Roldan et al. 2017). Consequently, makerspaces have become increasingly commonplace throughout K-16 STEM education, and especially so in undergraduate engineering degree programs. While more traditional machine shops have long been a resource available to these students, many higher education institutions are now renovating or replacing these facilities with makerspaces that offer advanced technology, like 3D printing (Wilczynski 2015), and provide open work areas for students to meet and create with both their peers and supervisors (Forest et al. 2014). As these facilities expand and begin to be incorporated into both formal and informal elements of STEM education, the need to examine their impact on students becomes apparent.

This study investigates how the use of a university makerspace in engineering course projects impacts students’ attitudes towards design, engineering, and technology. Specifically, we examine students’ (1) affect towards elements of professional practice, or an individual’s affinity towards the skills and tasks that are involved in engineering work; (2) self-efficacy, or an individual’s confidence in their ability to succeed in a venture, and (3) sense of belonging, or an individual’s self-perceptions of fit, within engineering spaces. We focus specifically on these variables given their relevance to activities involved in making, in addition to their well-documented relationships with persistence in STEM (e.g., Brainard and Carlin 1998; Marra et al. 2012).

University makerspaces not only give students access to tools and workspaces not previously incorporated into engineering degree programs, but also act as a technology-rich learning environment that facilitates innovative design experiences not yet fully understood by the literature. In doing so, they provide students the opportunity to practice tasks associated with professional practice, experiences which we hypothesize may increase students’ affect towards, self-efficacy in, and sense of belonging to engineering spaces. However, recalling that 20.9% of bachelor’s degrees in engineering are earned by women, and 21.6% by underrepresented minorities (National Science Foundation and National Center for Science and Engineering Statistics 2019), we suspect these impacts may not be equal across student groups. By examining these outcomes with respect to students’ year in program, gender, and race, we make a unique contribution to the growing body of literature on makerspaces.

Makerspaces

The broad definition of makerspaces is matched by the breadth of their applications in both formal and informal STEM learning, as well as by the variety of locations they are housed in, including museums, libraries, commercial workshops, K-12 classrooms, and state-of-the-art university facilities (e.g., Barrett et al. 2015; Johnson et al. 2015). In all, these technology-rich environments are said to provide opportunities for engaging STEM learners of all ages. Here, we specifically examine their impacts on undergraduate engineering students.

As design technology has advanced over the last few decades, the creation and expansion of makerspaces have paralleled this growth, especially on college campuses. A 2014 systematic review found that 40 of the 127 highest ranked US colleges and universities had documentation of a makerspace facility publicly available on their institution websites (Barrett et al. 2015). This increase in accessibility, in conjunction with the speed and relative ease of advanced manufacturing technology, has been matched by an overwhelming expansion of the applications of these technologies by students who visit for a variety of academic, personal, and extracurricular projects (Ali et al. 2016; Forest et al. 2014; Josiam et al. 2019; Wilczynski et al. 2016).

Conceptual framework

As a result, makerspaces have been a topic of interest in education research and have been widely praised as a disruption that may increase student access to STEM (Martin 2015; Roldan et al. 2017). In engineering, the American Society for Engineering Education has promoted the importance of including making in the curriculum and the potential of makerspaces to lead to new technologies and innovations; align informal and formal learning; restructure methods of teaching, evaluation, and assessment; and create opportunities for diversity, accessibility, and inclusion ( 2016; 2017). Echoing this call, Foster et al. (2015) claim that making experiences develop practical skills that are essential to a comprehensive engineering education. Others simply argue for the potential of the unique makerspace culture that may profoundly shape the education of students, in engineering and beyond (Forest et al. 2014).

Broadly speaking, makerspaces have been extolled as centers of innovation, entrepreneurship, and design. In the context of engineering education, extant research on these facilities mirrors this ideology and largely focuses on makerspaces’ improvements to aspects of the design process and prototyping. Many argue that prototyping, or the development of a physical product during the design process, is an essential step to allow students to identify design flaws that would have otherwise gone unnoticed and ultimately results in better final project outcomes (Forest et al. 2014; Wilczynski et al. 2016). Similarly, Kim and Maher (2008) argue that prototyping processes activate students’ ability to connect education to its applications in the real world and in industry. Makerspaces remove the obstacles to accessing these benefits, through advanced design and manufacturing equipment that does not require extensive training (Wong and Partridge 2016).

Gaps in the literature

Beyond asserting the benefits of prototyping itself, however, the literature lacks a comprehensive exploration of the impact of makerspace projects on student learning outcomes and student experiences, especially specific to engineering. In one study on an interdisciplinary undergraduate STEM course where students used a makerspace, students reported gains in collaborative skills and creativity, as well as an appreciation of experiential learning (Ludwig et al. 2017). In engineering, research suggests that makerspace usage can increase confidence, creativity, and entrepreneurial thinking in students (Longo et al. 2017). In interviews, engineering students articulate many benefits of makerspaces, including offering them authentic design experiences that expose them to a diverse set of topics (Jalal and Anis 2020) and increasing their access to equipment, their creative thinking, their confidence, and their collaborative skills (Dogan et al. 2020). Thus, we hypothesize that experiences in makerspaces directly impact students’ attitudes towards design.

In survey-based research, Brubaker et al. (2019) identified participation in one introductory makerspace course led to gains in design, innovation, engineering self-efficacy, and perceived closeness to the maker community. Similarly, a pilot version of the current study, which used a subset of the sample examined here, showed students reported significant gains in several metrics of self-efficacy, in their affect towards design, and in their sense of belonging to the makerspace (Carbonell et al. 2019). Consequently, we believe the relationship between these attitudes and makerspace experiences warrant further exploration.

Building upon this research, we hypothesized that a variety of factors may be impacted by a student’s participation in a makerspace. Specifically, we examine students’ (1) affect towards elements of professional practice, or an individual’s affinity towards the skills and tasks that are involved in engineering work; (2) self-efficacy, or an individual’s confidence in their ability to succeed in a task, and (3) sense of belonging, or self-perceptions of fit, within engineering spaces. The relevance of these constructs to activities involved in making motivates our investigation into their change over a semester in which students complete a project in the makerspace. In addition, several of these constructs have well-documented relationships with persistence and career choice in STEM (e.g., Brainard and Carlin 1998; Marra et al. 2012); next, we explore the literature surrounding each component of our framework.

Literature review

Affect towards engineering practice

Affect towards elements of engineering practice refers to students’ affinity towards the skills and tasks that are involved in engineering work (Patrick et al. 2017). The literature suggests that exposure to these types of activities (e.g., building things, taking things apart, being interested in how things work) can positively impact engineering-related outcomes (Pierrakos et al. 2010). The predominant model for measuring students’ affect towards professional practice was developed from ABET’s EC2000 Criterion 3a-k and inductive interviews with recent engineering graduates and engineering students across multiple disciplines (Patrick et al. 2017). The model identifies six key practices: design, tinkering, analysis, framing and solving problems, project management, and collaboration. Several of these factors have been shown to predict a student’s engineering identity (Choe et al. 2019), which has been further linked to retention (Patrick et al. 2018).

We examine students’ affect towards professional engineering because makerspaces give students access to equipment that allows them to practice tasks and processes common in the workplace. For instance, many capstone design projects are tailored to mimic the design process a professional engineering team would undergo; in utilizing the makerspace, students are able to work through a version of this experience. While there are many differences, working in the makerspace may be the closest students come to modeling professional engineering practices in their undergraduate experience. Further, these mastery experiences, such as successfully designing and 3D printing a prototype, have been shown to be influential developers of self-efficacy in both men and women (Mamaril and Royal 2008).

Self-efficacy

Broadly speaking, there is substantial evidence of the importance of self-efficacy, or an individual’s confidence in their ability to succeed in a venture, to education outcomes such as persistence, particularly in STEM (e.g., Betz and Hackett 1986; Eccles and Wigfield 2002; Lent et al. 1984; Rottinghaus et al. 2003). Yet, it is important to note that self-efficacy beliefs are neither stable over time, nor generalizable across disciplines or tasks. Instead, research articulates that domain- or task-specific self-efficacies are better suited at measuring confidence in related domains or tasks than the general construct (Wang and Richarde 1988). Specific to engineering education, domain-specific self-efficacy has been found to be a significant predictor of persistence (Bandura 1986; Brainard and Carlin 1998; Marra et al. 2012; Sheppard et al. 2010) and engineering career choice (Cass et al. 2011; Godwin et al. 2013). Further, while readers may guess that students would gain self-efficacy over time, longitudinal research has largely found that engineering self-efficacy beliefs actually decrease in undergraduates (Andrews et al. 2021; Jones et al. 2010).

As previously stated, makerspaces provide students with opportunities to practice skills and tasks that are commonly associated with engineering. Moreover, students gain exposure to novel design and prototyping technologies they may have never used before. As such, there is ample motivation to examine these makerspace-specific self-efficacies, such as design self-efficacy, innovation self-efficacy and technology self-efficacy that may inform students’ engineering academic and career choices (Carbonell et al. 2019; Lucas et al. 2009).

Further, these constructs may be of particular importance to certain student groups. Research consistently shows not only that self-efficacy is important in persistence for women, but also that gender differences in self-efficacy contribute to gendered academic and career outcomes in STEM (e.g., Brainard and Carlin 1998; Lazarides and Lauermann 2019; Stout et al. 2011; Wegemer and Eccles 2019) and in engineering specifically (Marra et al. 2009). For example, Brainard and Carlin (1998) found that undergraduate women who persisted in engineering and science had higher perceived self-confidence, or self-efficacy, in their mathematics and science skills than those who did not. Marra et al. (2012) confirmed these findings, supporting the conclusion that self-efficacy is a key factor in women’s decisions to persist in engineering degree programs.

Sense of belonging

Research also indicates the importance of a sense of belonging in STEM. Sense of belonging generally relates to self-perceptions of fit within a given context and has been well established as a theoretical construct throughout the literature (Hurtado and Carter 1997; Osterman 2000; Schar et al. 2017). The context in question can be formal, such as an educational setting or STEM discipline, or informal, such as friendships or affinity groups. Most importantly, the positive impacts of a strong sense of belonging on academic achievement and persistence in STEM majors are well documented (Good et al. 2012; Rainey et al. 2018; Seymour and Hewitt 1997; Strayhorn 2012; Tate and Linn 2005). In engineering specifically, students’ sense of belonging has been found to have a significant impact on student performance, and a lack of belonging has been identified as a contributing factor in students’ decisions to leave engineering (Marra et al. 2012), especially women (Brainard and Carlin 1998). As such, the importance of understanding experiences that may play a part in developing students’ sense of belonging, such as engaging with a university makerspace, cannot be understated.

Unfortunately, underrepresented students’ lack of a sense of belonging, often due to a chilly climate ingrained in the departmental culture, is well documented in STEM (Johnson 2012; Lee et al. 2020; Rainey et al. 2018) and in engineering specifically (Banda and Flowers 2016; Garriott et al. 2019; Godbole et al. 2018; Tate and Linn 2005). Given that makerspaces can act as anticipatory socialization for future engineers, where students may model professional engineering practice and learn associated norms, it is necessary to assume there is a risk that undesirable elements of engineering culture may be reproduced there (Vossoughi et al. 2016). While we hypothesized that students would gain a sense of belonging to the makerspace and to their engineering community by actually being in the makerspace and practicing engineering, given prior research, we suspect that sense of belonging in particular may vary by student demographics. Thus, claims about belonging and diversity in these increasingly prevalent engineering spaces should be further explored.

Aims of this study

In sum, facility stakeholders, including students, faculty, and administrators, hold many assumptions about the benefits and importance of makerspaces, but little research has quantified these impacts or examined how they may be disparate for various student groups. This study seeks to understand the following:

• 1. To what extent are there differences in students’ affect towards elements of professional engineering practice, task-specific self-efficacies, and sense of belonging after usage of a makerspace as a part of a course project?

• 2. Are there variations in these relationships by student characteristics, such as students’ year in program, gender, and race?

Based on prior literature and the types of activities typically conducted in university makerspaces, we hypothesized that these attitudes will be impacted by a student’s participation in a makerspace as a part of a course project. Specifically, we hypothesized that these experiences may increase students’ affect towards, self-efficacy in, and sense of belonging to engineering spaces. Given prior literature, however, we suspect we suspect that gains in these metrics may not be equal across student groups.

Here, we build upon our pilot study (Carbonell et al. 2019) by using a larger sample size, examining additional variables, and employing a stronger methodology to identify differences between student year, gender, and race. This research addresses crucial gaps in the literature by disaggregating our data by student groups and utilizing a matched, longitudinal design (i.e., repeated measures) to ensure observed differences cannot be due to different students taking the surveys but instead reflect changes within individuals over time. Further, we sample students from a variety of engineering courses in order to gain a broad understanding of how students benefit from using the space.

Methods

To address our research questions, we surveyed undergraduate engineering students enrolled in a selection of courses that included substantial projects designed to be completed with the support of the makerspace facilities. To account for variation between courses and instructors, we sampled courses that varied by department, student level, and complexity of the course assignments. The online survey, which took approximately 15 min to complete, was administered to students in either the Fall 2018 or the Spring 2019 semester, at the beginning and end of the semester. The survey asked participants to respond to a series of Likert-type, multiple-choice, and open-ended questions about their attitudes towards engineering, design, and technology.

The invention space

The Invention Space (a pseudonym) is a makerspace at a large public university in the Southwest, which enrolls approximately 6000 undergraduate engineering students. The facility is the centerpiece of the largest engineering building on campus, both of which opened to engineering students in the Fall of 2017. Spanning more than 23,000 square feet, the Invention Space occupies two floors, with floor-to-ceiling windows that divide the facility from the building’s atrium. The majority of the makerspace is dedicated to its digital fabrication lab and open worktables for students. However, students have access to a wide breadth of equipment, including Filament, SLA, SLS and Metal 3D Printers, Full Spectrum Laser Cutters, CNC Milling Machines, Desktop CNC Machines, VirtualBench, and myRIO, and an assortment of relevant handheld tools, such as manual mills and lathes. The variety of equipment available to undergraduate and graduate students through The Invention Space is reflected in the variety of projects completed there.

The makerspace is open to all engineering undergraduate and graduate students, as well as engineering faculty. Over a 2-year period, at least 4320 undergraduate students visited The Invention Space a combined 29,500 times; among other reasons, these students came to the makerspace to work collaboratively with their classmates, to tinker on personal projects, and to meet with their extracurricular organizations, such as Tau Beta Pi and the Society of Women Engineers (Josiam et al. 2019). Moreover, since its establishment at the institution, more than 30 courses have included curricular components that prompt students to utilize the makerspace in their academic work. While completing their projects, these students had the opportunity to collaborate with their instructor, The Invention Space staff, and the student workers who support the space.

Courses surveyed

Similarly, there is much variation between the courses we selected to survey for this analysis. Beginning from a list of 33 courses identified by the Invention Space leadership as having a substantial project that could be completed in the makerspace, our research team identified the 24 courses that were offered during either Fall 2018 or Spring 2019 semesters. We then contacted the instructors of these courses to request permission to visit their classrooms twice that semester; we visited every class that allowed us. These eight courses we surveyed were offered across five different departments, two of which are interdisciplinary. Five of these courses targeted lower-division students and three targeted upper-division students. Each course spanned only one semester, either Fall 2018 or Spring 2019. Table 1 shows a breakdown of the courses visited.

Table 1 Courses surveyed

Beyond simply representing a wide range of departments and student years, these courses vary in the scope and characteristics of the projects assigned by instructors, who each adapt the capabilities of the facility to best meet their needs. Students may have worked individually or in groups, may have been required or only strongly encouraged to visit The Invention Space, and may have visited once that semester or once a week. In some cases, students may have only interacted with The Invention Space for one homework assignment that required them to 3D print a part they designed in Solidworks. Other courses, like Mechanical Engineering Design Methodology, guide students through a semester-long, iterative design process that likely involved numerous visits and multiple pieces of the equipment available to them in the makerspace. The current analysis aims to generalize across that variability, instead of controlling for it, in order to gain a broad understanding of how a variety of students and faculty interact with the space.

Instrument administration

To address our research questions, we surveyed undergraduate engineering students enrolled in courses that incorporate the makerspace into their curriculum. We surveyed a selection of courses that varied by department, student year, and complexity of the course assignments; instructor demographics were not a factor in our sampling decisions. The survey asked participants to respond to a series of Likert-style, multiple-choice, and open-ended questions about their attitudes towards engineering, design, and technology. The online survey took approximately 15 min to complete and was administered in class during the first weeks of each semester, and again during the last weeks of each semester. Surveys were administered to students in courses offered during either the Fall 2018 or the Spring 2019 semesters. Students were not given any incentive to complete the survey at either timepoint, since in our experience providing class time to complete the survey is sufficient to ensure a high response rate.

Survey metrics

Drawing on prior literature and our conceptual framework, we compiled a survey instrument comprised of nine previously validated scales, each measuring factors that we hypothesized may be impacted by a student’s participation in a makerspace. Four scales measured students’ affect towards elements of professional engineering practices, including affect towards tinkering (Patrick et al. 2017), affect towards analysis (Patrick et al. 2017), affect towards design (Patrick et al. 2017), and innovation orientation (Jin et al. 2015). Three scales measured students’ self-efficacy in design (Carbonell et al. 2019), innovation (Carbonell et al. 2019), and technology (Lucas et al. 2009). Finally, two scales measured students’ sense of belonging within the makerspace and sense of belonging within engineering (Hurtado and Carter 1997); this scale was modified from its original form. No other scales were modified. Table 4 provides an overview of the scales’ alphas and number of items. Further, a table detailing the question stems, Likert ranges, and the number of items measuring each factor in their entirety is included in the Table 12 in Appendix. In all, these nine factors were measured by 32 items, attached to fourteen question stems.

Additionally, the instrument asked whether students had visited The Invention Space at both timepoints; students were not asked how often or how many times they visited the facility. All other data points (e.g., student demographics, GPA) were generated from the university’s administrative records.

Elements of professional engineering practice

All items had a 5-point Likert response scale. For Design, Tinkering, and Analysis, the question stem asked students to rate the extent to which they would enjoy a profession or career that usually requires a series of tasks (Patrick et al. 2017). Design, indicated by seven items, measured students’ affect towards creative elements of engineering professional practice. This scale has a Cronbach’s alpha of 0.84. Tinkering was measured by two items that asked students to rate their interest in fixing things and taking things apart. This scale has Cronbach’s alpha of 0.61; it is important to note that this is lower than the generally accepted threshold of 0.70. Analysis was measured by three items that asked about students’ interest in applying their technical knowledge to an engineering problem. This scale has Cronbach’s alpha of 0.70.

Students’ Innovation Orientation, borrowed from the Young Entrepreneurs Survey, was measured through six 5-point, Likert scale items that asked students to rate the extent to which they partook in a series of innovative behaviors (Jin et al. 2015). This scale has Cronbach’s alpha of 0.80.

Self-efficacy

Students were asked to use a 5-point Likert scale to rate their confidence in their ability to perform a series of tasks in order to measure their Design Self-Efficacy and Innovation Self-Efficacy, initially hypothesized to be one factor. These factors were validated first through exploratory factor analyses (EFA) with a subset of the sample from one semester of survey data, which indicated that the four items load onto two separate factors, design self-efficacy and innovation self-efficacy (Carbonell et al. 2019); subsequent confirmatory factor analyses (CFA) with the remainder of that same dataset indicated an excellent fit (RMSEA = 0.000, CFI = 1.000, TFI = 1.054, and χ² = 0.145; df = 1; p = 0.70). To further validate the scale, a secondary CFA was conducted on students’ survey response at timepoint 2 (i.e., post-surveys), which confirmed the validity is stable (RMSEA = 0.000, CFI = 1.000, TFI = 1.015, and χ² = 0.142; df = 1; p = 0.71). Design self-efficacy has Cronbach’s alpha of 0.83, and innovation self-efficacy has an alpha of 0.78. Students’ Technology Self-Efficacy was measured through four 10-point, Likert scale items that asked students to rate their confidence in their ability to perform a skill at that time (Lucas et al. 2009). This scale has Cronbach’s alpha of 0.87.

Sense of belonging

Students’ sense of belonging was measured through the adaptation of a previously validated scale that analyzed students’ perceptions of the campus climate and its impact on their sense of belonging (Hurtado and Carter 1997). The original scale asked students to evaluate their sense of belonging to their institution; this survey modified the item wording to ask students about the makerspace and the engineering community, rather than a campus. For instance, instead of being asked to what extent students saw themselves as part of the campus community, one item measuring Sense of Belonging to the Makerspace asks students to rate the extent to which they saw themselves as a part of The Invention Space. We then conducted an exploratory factor analysis to validate this construct. This analysis showed that all three items loaded onto the same factor. A follow-up CFA is discussed below. This factor has Cronbach’s alpha of 0.96.

Students were also asked to evaluate their Sense of Belonging to the Engineering Community at their institution, using the same wording modification process. An exploratory factor analysis validated that all three items loaded onto the same factor. Sense of belonging within engineering has Cronbach’s alpha of 0.94. A confirmatory factor analysis with all six sense of belonging items indicated a two-factor solution was a good fit (RMSEA = 0.050, CFI = 0.998, TFI = 0.996, and χ² = 12.28; df = 8; p = 0.139).

Research design

We conducted all data analyses using StataCorp. 2015 Stata Statistical Software: Release 17. A total of 610 students responded to at least one of the four surveys administered over two semesters. From the initial pool, individuals’ responses to the pre- and post-surveys were matched across timepoints using their unique student IDs. The sample was narrowed to only include students who had responded to both the pre- and post-surveys in the same semester. While this matching significantly reduces the sample size, it is a stronger research design that ensures observed differences cannot be due to different students taking the pre- and post-surveys. Subsequently, we removed respondents by listwise deletion if they were missing values for any of the nine factors of interest. Student responses were then matched to university records to retrieve demographic information about the sample and removed if they could not be matched.

This left an analytical sample of n = 213; 87.3% of these students, n = 186, visited the makerspace and 12.7% of these students, n = 27, enrolled in the same courses did not. To clarify, some projects required students to visit the space, whereas others only strongly encouraged it. In both cases, students were able to choose whether to visit the makerspace or not. Readers should note that because of this, it is possible that there may be inherent differences between these groups and comparisons between them should be interpreted carefully. However, the non-visitor group, albeit small, is included for reference to contextualize the results for those students who did visit the makerspace. Additionally, their data sheds light on the attitudes of students from every course in our sample who, for whatever reason, did not visit and highlights an avenue for future research.

Descriptive statistics were generated to describe the characteristics of students in the sample. Subsequently, matched (i.e., paired samples) t-tests were conducted separately on the samples of students who did visit the makerspace, n = 186, and those who did not, n = 27, for each of the nine factors of interest. We used a Bonferroni correction of p = 0.0028 to account for the eighteen t-tests run and calculated effect sizes for each significant factor to further quantify the difference between pre- and post-groups. Subsequently, we used repeated measures ANOVAs to understand how these gains differ across student characteristics within our experimental group, n = 186. When applicable, we conducted post hoc tests using Tukey’s HSD for characteristics with more than one level (e.g., first-year, second-year, third-year, and fourth-year students) and split the data file by factors of interest to conduct follow-up one-way ANOVAs.

Results

Table 2 provides an overview of the academic characteristics of the analytical sample. The sample consists of 93% engineering students and 7% non-engineering majors. Most students in the sample were lower-division students (e.g., first- and second- years). The majority of students were enrolled in either the Mechanical, Civil, or Electrical Engineering departments. In some cases, university records of major were not available; for this reason, the n varies and is included for reference.

Table 2 Overview of student characteristics

Table 3 provides a demographic overview of the analytical sample, generated from university records; a more detailed cross-tabulation of students’ race and gender is included in the Table 13 in Appendix. However, the institutional data reports international students’ race as “foreign”; while this is not a particularly meaningful classification, we maintain this data to differentiate from students who left questions blank or selected “prefer not to answer” on the demographic questions upon enrollment at the institution. Additionally, this designation is commonly used in reports by the National Science Foundation (e.g., Khan et al. 2020).

Table 3 Overview of student demographics

Compared to national statistics on engineering degree attainment, our sample shows an overrepresentation of students who identify as women, Asian, and Hispanic or Latinx, and an underrepresentation of all other student groups (National Science Foundation and National Center for Science and Engineering Statistics 2019). For analyses by race, we group students into four groups: white, Asian, Hispanic/Latinx, and all others, due only to their representation in our sample. While it is not particularly meaningful to group international students with racial minorities, we group these students to gain statistical power and to avoid removing them from our sample altogether.

Table 4 shows Pearson’s bivariate correlation matrix of the nine factors of interest; this test measures the strength and direction of linear relationships between variables, where a result of 1 indicates a perfectly linear, positive association. All correlation coefficients were less than .70, indicating the constructs do not overlap with one another at a problematic level (Meyers et al. 2006); in general, correlations below 0.3 indicate weak associations, between 0.3 and 0.5 indicate a moderate association, and those greater than 0.5 indicate strong associations. The significance level of correlation is α = .001. In addition, we report the means, standard deviations, and alphas for each factor, calculated at the first survey timepoint.

Table 4 Means, standard deviations, and correlations between analytical variables, n = 213

Matched longitudinal analyses

At the time that the pre-survey was administered, 66.2% of participants reported that they had never visited or worked within The Invention Space facilities before. By the end of the semester, only 12.7% of the students, n = 27, had not visited the space. The 87.3% of students, n = 186, who had visited the makerspace had an average GPA of 3.48, which was 0.41 points higher than those who had not.

Matched t-tests were conducted separately on students who did visit the makerspace, n = 186, and those who did not, n = 27, for each of the nine factors of interest. A Bonferroni correction of p < 0.0028 was used to account for the eighteen matched t-tests conducted. Table 5 details the results of the matched t-tests, as well as the effect size, or standardized mean difference, for each factor that showed statistically significant differences. Following Cohen (2013), effect sizes from 0 to 0.05 indicate a small difference, effect sizes from 0.06 to 0.15 indicate a moderate difference and effect sizes above 0.25 indicate a large difference.

Table 5 Matched T-test results and effect sizes

For students who visited the makerspace, six of the nine factors showed statistically significant increases within persons over a one semester period: innovation orientation, design self-efficacy, innovation self-efficacy, technology self-efficacy, sense of belonging within the makerspace, and sense of belonging within engineering. Affect towards design, tinkering, and analysis did not show statistically significant gains. Students who did not visit the makerspace only reported statistically significant increases in technology self-efficacy.

Comparisons between student groups

We then used repeated measures ANOVAs to model differences due to student characteristics, while accounting for within-subject variance over time. These analyses were conducted only on the group of students who reported that they visited the makerspace that semester, n = 186. Hypothesizing that the greatest differences between student groups would exist for measures of self-efficacy and belonging, we only conducted these analyses for the following five factors: design self-efficacy, innovation self-efficacy, technology self-efficacy, belonging to the makerspace, and belonging to the engineering community. We used three student groupings to run these analyses: student year, gender, and race. While examining differences across student majors or courses is potentially interesting, the distribution of students is such that the statistical significance would be overinflated by the small number of students in several majors and courses.

For each of the five factors of interest, we ran three analyses: (1) a 2 × 4 repeated measures ANOVA with student year and time predicting factor mean; (2) a 2 × 2 repeated measures ANOVA with gender and time predicting factor mean; and (3) a 2 × 4 repeated measures ANOVA with race and time predicting factor mean. Table 6 summarizes where significance was found in each of these analyses. As expected from the prior results of the matched t-tests, there is a significant main effect of time for nearly every factor. These analyses confirmed our hypotheses that there are variations in these relationships by student characteristics.

Table 6 Significant differences by student groupings, n = 186

Year in program

Table 7 summarizes the main effects of the 2 × 4 repeated measures ANOVA with student year and time predicting factor value. Analyses revealed two statistically significant main effects of students’ year in program: design self-efficacy (F(3,182) = 3.79, p = 0.0173) and sense of belonging to the engineering community (F(3,182) = 2.96, p = 0.0336).

Table 7 Main effects of student year, n = 186

Post hoc testing using Tukey’s HSD revealed the following differences in student responses. On design self-efficacy, fourth-year students had statistically significantly higher means than second- and first-year students; no other combination of student groups showed significance. Second-years, however, had a statistically significantly higher sense of belonging to the engineering community than any other group; again, no other combination of student groups showed significance.

The 2 × 4 repeated measures ANOVA with student year in program and time predicting factor value also revealed one significant interaction in belonging to the makerspace (F(3,182) = 6.67, p = 0.003). Follow-up one-way ANOVAs where the data file was split by student year revealed that only first-years experienced statistically significant gains in their sense of belonging to the makerspace. Table 8 details the pre- and post-means for each year, as well as the corresponding effect sizes for significant results.

Table 8 One-way ANOVAs, by student year, n = 186

Gender

Table 9 below summarizes the results of the 2 × 2 repeated measures ANOVA with gender and time predicting factor value. Analyses revealed a significant main effect of gender for design self-efficacy (F(1,184) = 9.73, p = 0.0021), innovation self-efficacy (F(1,184) = 5.19, p = 0.0239), and technology self-efficacy (F(1,184) = 10.12, p = 0.0017). Male students in the sample had higher mean scores for each form of self-efficacy. No additional post hoc testing was required for analysis by gender because there were only two categories of gender in university records.

Table 9 Main effects of gender, n = 186

The 2 × 2 repeated measures ANOVA with gender and time predicting factor value did not reveal any significant interactional effects. In other words, both male and female students demonstrated gains in self-efficacy over the semester, but the benefits were equal between groups; the gaps in self-efficacy between genders persisted.

Race

Due to low representation for 3 of the 6 reported racial groups in this sample, for this analysis, students were grouped into four categories in order to gain statistical power: white only, Asian only, Hispanic/Latinx only, and all others. Table 10 summarizes the results of The 2×4 repeated measures ANOVA with student race and time predicting factor value, which revealed a significant main effect of race on students’ innovation self-efficacy (F(3,176) = 3.83, p = 0.0108). Post hoc testing using Tukey’s HSD revealed that Asian students and students in the “All others” category (who identified as Foreign, multiracial, or Black) had statistically significantly lower means than white students; no other combination of student groups showed significance.

Table 10 Main effects of race, n = 186

The 2 × 4 ANOVA also revealed a significant interaction of time with belonging to the makerspace (F(3,176) = 3.56, p = 0.0154). Follow-up one-way ANOVAs where the data file was split by race revealed statistically significant gains in belonging to the makerspace for only those students who identified as white, Asian, or Hispanic/Latinx. Table 11 details the pre- and post-means for each group, as well as the effect sizes for significant results.

Table 11 One-way ANOVAs, by race, n = 186

Discussion

In this study, we sought to understand how the use of a university makerspace in a course project impacts students. In partnership with a makerspace at a large, public institution in the southwest, our research team surveyed undergraduate students in courses that incorporated a makerspace-based project into their curriculum in order to better understand the ways in which incorporating these spaces into a class project impacts students’ affect towards engineering practice, their self-efficacy, and their sense of belonging. We will first discuss our matched longitudinal analyses, which confirmed our hypotheses and revealed that students who used the makerspace as a part of a course assignment showed significant, positive increases in measures of their innovation orientation, design self-efficacy, innovation self-efficacy, technology self-efficacy, belonging to the makerspace, and belonging to the engineering community. Our findings highlight the potential benefits of makerspaces to not only final design products, as previously documented (Forest et al. 2014; Wilczynski et al. 2016), but also to students’ self-efficacy and sense of belonging.

Then, comparisons across student groups revealed statistically significant main effects of student year, gender, and race, as well as interactional effects of both student year and race with time, suggesting that different students benefit differently from the makerspace. By year in program, first-year students show the most dramatic—and the only statistically significant—gains in sense of belonging to the makerspace. Our analyses also revealed persistent gender gaps in self-efficacy and unequal gains in belonging to the makerspace across racial groups that warrant further investigation. In all, we note that although makerspaces are fulfilling some of their promise for educating innovative engineers and designers, more attention needs to be paid to avoid reproducing disparities in engineering education that are already experienced by female students and racial minorities.

Differences over time

Students who used the facility showed statistically significant gains on six of the nine factors of interest. Apart from sense of belonging in engineering, the effect sizes for each of these significant gains range from 0.57 to 0.92, indicating that exposure to the makerspace had vast positive impacts on students. The effect size for sense of belonging in engineering was 0.19. These results go beyond merely affirming the value of prototyping with respect to the final design product itself and instead focus on prototyping’s direct benefits to students.

Further, we remind readers that our sample included students from courses that varied by department, student level, and scope of course project. Across this variation, students showed gains in engineering attitudes that are not only sizable for one semester, but also stable across our sample, even in a few cases when course assignments suggest students may have visited only once. This broad impact suggests that makerspaces like The Invention Space offer immense potential benefits to a wide variety of students when integrated into their engineering curriculum. Engineering instructors should assign students course projects that could use these makerspaces, thereby increasing the likelihood that more students receive early exposure to the equipment, experiences, and benefits available to them there.

Lack of significant change in affect towards engineering practice

While each of the metrics of students’ affect towards elements of professional engineering practice showed gains over a one semester period, only one of the four, innovation orientation, was statistically significant (Table 5). It is noteworthy that none of the other three (affect towards design, tinkering, and analysis) were significant, since these factors have been previously shown to predict a student’s engineering identity (Choe et al. 2019), which has been further linked to retention (Patrick et al. 2018).

However, these results may be understandable with respect to the parameters of the projects being completed for these courses. For instance, undergraduate engineering design projects often involve more design generation than analysis of prototype efficacy, a tension that is echoed in the literature via a thorough exploration of the relationships between making, tinkering, and engineering (Vossoughi and Bevan 2014). Similarly, these patterns are mirrored in our survey results; student gains in affect towards design were close to statistical significance and were much higher than those in analysis or tinkering. Tinkering gains were minimal and far from significant. This may indicate that the structure of course projects, which often emphasize design optimization over exploration, may not be conducive to developing a tinkering mentality nor analytical skills. It may also reflect uncaptured variance in the rigor of projects between courses in our sample.

Significant gains in self-efficacy

Each of the metrics of self-efficacy showed statistically significant gains, with effect sizes ranging from 0.54 to 0.92 (Table 5). These effect sizes, which indicate standardized mean differences in each factor between timepoints, are not only quite large by statistical standards (Cohen 2013), but also occurred across only a one semester time period. Further, the fact that students are showing gains in these metrics at all stands in stark contrast to prior longitudinal research which show engineering attitudes, such as self-efficacy, typically decreases over time (Andrews et al. 2021; Jones et al. 2010). This indicates that completing a course project in a makerspace, no matter how small, could potentially counteract the drop in self-efficacy typical for engineering undergraduate students.

The simplicity of these results makes them especially compelling—as students complete tasks associated with design, innovation, and technology, they feel measurably more adept at doing so, even after a matter of weeks. The long-proven links between students’ self-efficacy beliefs and retention (Bandura 1986; Brainard and Carlin 1998; Marra et al. 2012; Sheppard et al. 2010) offer a strong motivation for the integration of skill-building in a makerspace into engineering classes.

Increased sense of belonging

Students’ sense of belonging to the makerspace increased over half a standard deviation over the course of one semester, whereas sense of belonging to engineering only increased about one fifth of a standard deviation. Such significant gains are especially important when recalling not only that some have hypothesized that not all students feel welcome in makerspaces (Vossoughi et al. 2016), but also that the impacts of sense of belonging on persistence are well-documented (Good et al. 2012; Hausmann et al. 2007; Rainey et al. 2018; Seymour and Hewitt 1997; Tate and Linn 2005). These results suggest that requiring students to visit the space as a part of an assignment or course project may play a role in mitigating student hesitations about the space; this is especially important given research that suggests students who are required to visit the makerspace once are more likely to return in the future (Josiam et al. 2019). Further, requiring all students to visit a makerspace as a class assignment early in their engineering education could be a way to help make the space more inclusive and to ensure that more students are receiving the benefits a makerspace can offer.

Differences across student groups

However, demographic analyses indicated noteworthy, significant differences between student groups. Here, we discuss only the 87.3% of students surveyed who visited the makerspace, n = 186. The following paragraphs discuss mean differences in design self-efficacy, innovation self-efficacy, technology self-efficacy, belonging to the makerspace, and belonging to the engineering community across three student characteristics: student year, gender, and race. Each section first discusses the main effects of the characteristic and then the interactional effects of this characteristic with time.

Differences by student year

Analyses revealed two statistically significant main effects of student year for design self-efficacy and belonging to the engineering community, as well as a significant interaction of time and student year for belonging to the makerspace.

Fourth-year students reported statistically significantly higher levels of design self-efficacy than first- and second-year students. This result is not surprising, as upper-division students have likely had more exposure to the design process and are typically enrolled in courses with more complex, design-focused course projects, such as capstone design; still it contradicts research that finds engineering self-efficacy beliefs decrease over time (Andrews et al. 2021; Jones et al. 2010). Descriptively, these cross-sectional results indicate growth in design self-efficacy throughout students’ time as undergraduates, with the exception of first-year students, whose mean was higher than second-year students.

Second-year students felt a statistically significantly higher sense of belonging to the engineering community than any other group; future research should investigate how this belief might fluctuate throughout students’ undergraduate experiences. While both sense of belonging to engineering and design self-efficacy showed differences across student years, these differences had effect sizes of only 0.05 (Table 7), indicating a small difference. In contrast, the effect of time (i.e., the impact of participating in a course project that uses the makerspace; Table 5) had effect sizes of 0.19 and 0.92, respectively. In other words, while the effects of student year are statistically significant, these effect sizes indicate the impact of time in the makerspace is much more dramatic than that of student year.

Examining the interaction effects of time and student year revealed interesting trends between student groups. Analyses revealed an interaction of student year and time with respect to students’ belonging to the makerspace. In this case, only first-year students experienced a statistically significant gain in sense of belonging to the makerspace over the semester, jumping from an average of 4.3 to 6.7 (on a scale of 0–10). First-year students were also, by far, the most represented student group in the sample, comprising 110 of the 186 students. This representation indicates that the significant gains shown by first-year students are driving the significant increase seen in the entire sample in the longitudinal analyses. These results do not discount the gains shown by all students, but instead, reaffirm the value of incorporating the makerspace into courses early in the educational experiences of students.

Gendered differences in self-efficacy

Analyses broken into gender groups revealed main effects of gender across the three measures of self-efficacy, indicating that male students surveyed had statistically significantly higher means than female students on design self-efficacy, innovation self-efficacy, and technology self-efficacy. The lack of significant interactional effects of time and gender indicates that while both groups reported significant gains in each metric, the benefits were equal between groups. In other words, the gender gaps in self-efficacy were not narrowed (nor widened) over the course of the semester. This result is not surprising, given the vast body of research documenting gender gaps in engineering-related self-efficacy (Brainard and Carlin 1998; Marra et al. 2012; Sheppard et al. 2010). However, this study extends prior research by demonstrating that gender gaps in self-efficacy persist in makerspaces. Interventions focused on building self-efficacy for female students in particular would be important to extend into makerspaces, as self-efficacy beliefs are malleable, and they can be influenced based on interactions with others or changes in the environment (Bandura 1997).

Four proven builders of self-efficacy beliefs are performance/mastery experiences, vicarious experiences, social persuasion, and physiological states (Bandura 1997; Mamaril and Royal 2008; Marra et al. 2009). Mastery experiences would include learning to use makerspace equipment and successfully completing assignments and projects using the space. Vicarious experiences include seeing others with whom one identifies being successful, such as student workers and more fourth-year students working in the space. Makerspace membership is multi-faceted, and it is important for individuals from all backgrounds to feel accepted into the community. To achieve this, makerspaces can offer physical supports and staffing to create a more inclusive makerspace culture (Roldan et al. 2017); physical supports can include access to protective equipment and clothing in a variety of sizes to support makerspace usage. Makerspaces can also hire female and LGBTQ+-identifying professional and student staff from multiple backgrounds to offer a counternarrative to the predominantly masculine culture of engineering.

Differences between racial groups

It is important to recall that compared to national statistics on engineering degree attainment, our sample shows an overrepresentation of students who identify as both Asian and Hispanic/Latinx, and an underrepresentation of all other student groups, most notably Black students (National Science Foundation and National Center for Science and Engineering Statistics 2019). To gain statistical power and retain all participants for these analyses, we grouped students into four categories by their reported race: white only, Asian only, Hispanic/Latinx only, and all others. Analyses revealed a significant main effect of race on students’ innovation self-efficacy, where students who identified as white reported statistically higher means than Asian students and students who identified as foreign, multiracial, or Black. With the small sample size and low representation of particular groups, it is unclear whether the differences are being driven by foreign, Black, multiracial students, or a combination of groups. Further study is needed.

Similarly, with respect to the significant interaction of time and race for belonging to the makerspace, students who identified as foreign, multiracial, or Black did not demonstrate statistically significant gains in their sense of belonging to the makerspace, whereas each of the other three groups did. While it is not particularly meaningful nor just to group international students with racial minorities, it is important to note that this combined group of individuals, who are likely already marginalized in multiple aspects of their education, are not seeing the benefits to sense of belonging to a makerspace their peers experienced simply by taking a class which requires makerspace use. While more research is needed to replicate and understand this finding, the important implication is that more attention needs to be paid to helping marginalized groups feel like they belong and reap the full benefits of makerspace participation, especially given the historically chilly climates for underrepresented groups in both documented STEM (Johnson 2012; Lee et al. 2020; Rainey et al. 2018) and engineering specifically (Banda and Flowers 2016; Garriott et al. 2019; Godbole et al. 2018; Tate and Linn 2005). Further, it is important to consider members of historically marginalized groups as individuals. Students who identified as Hispanic/Latinx experienced the largest gains in sense of belonging to the makerspace (Table 11). One or both of these findings would have been obscured if we had combined Black and Hispanic/Latinx participants into one underrepresented minority group. When sample sizes are not large enough, qualitative research methods would be important to augment understanding.

Limitations

There are several limitations of our study to consider when interpreting these results. Data collection took place at only one institution and about only one makerspace facility; institutional and makerspace characteristics specific to this university or the instructors who allowed us to visit their classrooms could account for some variance in our dataset and limit the generalizability of our study. There are benefits and limitations to using institutional demographic data as we have in the current study (e.g., the institution reports some students’ race as “foreign”), but this level of detail is less common in larger, multi-institution studies.

Further, there are a variety of factors not captured by our survey instrument that could contribute to variation in student responses (e.g., other coursework, interactions with makerspace staff, number of visits); future research should focus on more nuanced analyses of what components of makerspace participation may be most impactful in shaping students’ engineering attitudes. However, by sampling broadly from courses that vary by department and student year, we gain a credible evidence of how a variety of students benefit from the makerspace.

Further, while we include longitudinal analyses of a group of students who did not visit the makerspace for reference, making comparisons between these groups is not the aim of this study. Rather, we encourage readers to focus attention on the vast gains within students over one semester and the differences in these gains by student year, gender, and race. The number of students who did not visit (n = 27) is relatively small, and the descriptive statistics in Table 3 suggest there may be inherent differences between the group of students who chose to visit the makerspace as a part of their course and those that did not. Hence, comparisons between these two groups should be interpreted carefully. Future research should investigate why students choose to visit or not visit makerspaces when assigned to a course project that requires it, especially given the demographic differences seen between these two groups in our sample.

Conclusions

The study aimed to explore the ways in which students’ attitudes towards engineering, design, and technology change after completing a makerspace-based project. In partnership with a makerspace at a large, public institution in the southwest, our research team surveyed 213 undergraduate students in eight unique courses. Each course incorporated a makerspace-based project into its curriculum, but courses varied by department, student year, subject matter, and project complexity. Analysis revealed statistically significant positive gains across six of the nine factors of interest for students who visited the facility: design self-efficacy, technology self-efficacy, innovation orientation, innovation self-efficacy, sense of belonging within the makerspace, and sense of belonging within the engineering community. Students who did not visit the facility showed significant improvements for only technology self-efficacy. Moreover, students who visited the makerspace during the semester they were surveyed had a cumulative GPA nearly half a point higher than those who did not. Our findings highlight the potential benefits of makerspaces to not only final design products, as previously documented, but also to students’ self-efficacy and sense of belonging.

However, subsequent demographic analyses revealed statistically significant main effects of student year in program, gender, and race, as well as interactional effects of both student year and race with time. By student year, first-year students show the most dramatic—and the only statistically significant—gains in sense of belonging to the makerspace. These results reaffirm the value of incorporating the makerspace into the early educational experiences of students. Gender gaps in self-efficacy persist, as all students show gains from completing a makerspace-related course project. Students who identified as foreign, multiracial, or Black did not show significant gains in belonging to the makerspace, whereas white, Asian, and Hispanic/Latinx groups all did. More research is needed to replicate these initial results. We conclude that although makerspaces are fulfilling some of their promise for educating innovative engineers and designers, more attention needs to be paid to avoid reproducing disparities in engineering education that are already experienced by female students and racial minorities. Future work should examine the generalizability of these findings across other STEM disciplines and makerspace contexts.

Appendix

Table 12 Factors by item

Table 13 Student demographics