The influences of student- and school-level factors on engineering undergraduate student success outcomes: A multi-level multi-school study
International Journal of STEM Education volume 9, Article number: 23 (2022)
Given the relatively low graduation and retention rate in undergraduate engineering programs in the United States, the factors that influence student success outcomes need to be examined. However, limited research systematically studied both student- and school-level factors and how they influenced undergraduate engineering student success outcomes. We gathered responses from 458 engineering undergraduate students in a cross-sectional multilevel multi-school (14 schools) survey. These 14 schools included both large state universities and liberal arts colleges. The survey measured various student-level factors, including demographic, skills, and personality variables, along with seven school-level factors, such as student–faculty ratio and school type (i.e., public versus private). The data were analyzed using the hierarchical multilevel modeling approach.
The results showed that female students reported better outcomes than male students, racial minority students reported better outcomes than White students, but first-generation students reported poorer outcomes. Communication competency was associated with student learning outcomes, GPA, and program satisfaction, whereas conflict management preferences were not significantly correlated with any student success outcomes. The results of the school-level factors’ influences on student success outcomes were not consistent, but some factors, such as student–faculty ratios and diversity rate, were significantly related to some student outcomes.
Engineering education is a complex, multi-faceted issue that requires more collaborative and systematic research. We hope our findings help educators understand the different factors that could potentially influence engineering students and inform better program design and policymaking.
Student success outcome is arguably one of the important topics in education research. Student success outcome is largely defined as a variable that assesses how well students are prepared to accomplish their current and future academic, personal, and professional goals (Kuh et al., 2006). In recent years, more engineering education research has studied student success outcomes given the relatively low graduation and retention rate in engineering programs compared to the national rate of students across all majors in the United States (Marra et al., 2012). Various types of higher education organizations have made extensive efforts in recruiting more students to enroll in engineering education (Van den Bogaard, 2012). As a result, undergraduate engineering enrollment has been on the rise in the past 10 years, with 622,502 full-time undergraduate engineering students enrolled in 2018 (Roy, 2019). However, a large number of students transfer out of engineering majors or drop out of the university prior to graduation. Over the last 60 years, the average engineering program graduation rates have consistently hovered around a critically low 50% in the United States (Geisinger & Raman, 2013).
Previous studies in engineering education identified a set of factors that could directly or indirectly influence student success outcomes, such as gender, race, and family educational background (Bossart & Bharti, 2017; Fletcher et al., 2021; Smith & Lucena, 2016). However, those factors are mainly at the student level, and school-level factors such as school type (e.g., private vs. public) and diversity rate are not well studied. In addition, many studies recruited students from a limited range of schools by only including students from one type of school. Although some research studies on students from multiple schools (Marbouti et al., 2021), there is a sparse representation of school diversity, such as the inclusion of engineering students from liberal arts schools. However, the number of liberal arts engineering students has been rapidly growing in the United States (Koshland, 2010). To expand the current literature, this study systematically examines the influences of student- and school-level factors on student success outcomes by using a multi-school multilevel approach, and we further include liberal arts engineering students in our multi-school approach.
Given the complex nature of engineering education, previous research used a variety of ways to conceptualize and operationalize student success outcomes in engineering education, such as college grades, satisfaction, and learning outcomes (Bean & Eaton, 2001; Kuh et al., 2006; Marbouti et al., 2021). Although the conceptualization of student success outcomes varies greatly, there is consistent consensus on the importance of understanding them (Kuh et al., 2006). The current study conceptualizes student success outcomes as academic achievement, satisfaction, and learning outcomes, and the study further examines the potential influences of the school and student factors on these outcomes. To the best of our knowledge, the current study conceptualized student success outcomes these ways as they were the most common conceptualizations in the current higher education literature. To do so, the current study uses instructive perspectives (Tinto, 1987) as the theoretical framework to gain a more comprehensive examination of potentially relevant factors at both student and school levels. Influences on student success outcomes could exist at sociological, organizational, psychological, cultural, and economic levels (Tinto, 1987), which could be categorized into two levels: student and school.
Student-level factors in the current study are identified from the sociological, economic, psychological, and cultural perspectives that could influence student success outcomes in the instructive perspectives (Tinto, 1987). First, students’ demographic characteristics could influence student success outcomes, and previous research often has operationalized demographic characteristics as age, gender, race, household income, and family educational background (French et al., 2005; Kuh et al., 2006). For example, Black students, who are highly underrepresented in engineering education, face significant challenges compared to students from other racial/ethnic groups (Fletcher et al., 2021). In addition to race and gender, first-generation students from families where no parents or guardians have earned a baccalaureate degree tend to perform poorer on student success outcomes than those who are not (Martin et al., 2020). In-depth interviews revealed that first-generation students majoring in engineering felt that they did not belong in engineering because engineering coursework is distancing them from their family members, specifically parents (Smith & Lucena, 2016). First-generation students also reported that they did not have the same social, cultural, or economic capital as their peers (Martin et al., 2020). They felt that none of their funds of knowledge, bodies of knowledge and skills such as car or plumbing repairs that working-class families possess to survive and make a living (González et al., 2006), were validated in their engineering education. This made their sense of isolation worse (Smith & Lucena, 2016).
In addition to demographic characteristics, certain relevant skills and personality styles might also influence student success outcomes in undergraduate engineering education, namely, communication competence and conflict management styles. Students’ skills and personality styles are critical for their success in their undergraduate education (Bean & Eaton, 2001). Communication competence refers to the skills of effective and appropriate communication and the ability to use and adapt that knowledge in various contexts (McCroskey & McCroskey, 1988). Different from rudimentary communication skills, communication competence is a more complex concept that reflects a student’s physiological and psychological characteristics and ability to adapt to various social and cultural contexts (McCroskey & McCroskey, 1988). Student success outcomes in all academic settings are closely related to communication competence because students are frequently required to interact with faculty and peers, especially when seeking information, having in-class discussions, and asking questions (Goldman, 2019). Recent research has shed some light on the importance of communication competency in STEM education (Wilkins et al., 2015; Yeke & Semerciöz, 2016), so the study further advances the understanding by examining how communication competence impacts student success outcomes.
Another important student-level factor that could influence student success outcomes is conflict management style. Teamwork has been recognized as an essential skill for engineers as teamwork is regularly integrated into engineering fields (Van den Beemt et al., 2020). The differences in goals, perspectives, and emotions in teamwork could lead to conflicts that negatively affect team performance and productivity (Liu et al., 2008). How the conflict is handled is more important to the teamwork success than the conflict itself (Paul et al., 2004). Five styles of handling conflict behaviors have been identified in the current literature: collaboration, competition, avoidance, accommodation, and compromise (Liu et al., 2008). The previous research indicates that more collaborative conflict management styles are more likely to be associated with positive team performance outcomes (Montoya-Weiss et al., 2001). Avoidance is more likely to hurt team performance because it means that the team cannot bring its full range of resources in decision-making (Chang & Lee, 2013). Satisfactory and successful team performance could motivate students to be more engaged and further enhance students’ academic integration (Liu et al., 2008). Thus, certain conflict management styles could either be positively or negatively associated with student success outcomes. Based on the previous literature and the instructive perspectives, the current study asks the following research question about student-level factors:
RQ1: How do student-level factors, including (a) demographic characteristics, (b) communication competency, and (c) conflict management styles influence undergraduate engineering students’ success outcomes, including academic achievement, learning outcomes, and satisfaction?
The school-level factors include factors from the organizational and economic levels based on the instructive perspectives (Tinto, 1987). Organizational and economic levels emphasize the school’s structural-demographic characteristics and processes that are thought to influence student success outcomes (Kuh et al., 2006). Based on the previous literature, the school-level factors that could influence these experiences include school size, school type, student–faculty ratio, tuition, graduation and retention rate, and diversity rate. First, the interaction between faculty and students, often measured by the student–faculty ratio, could often influence students’ experiences and success (Marra et al., 2012). Many schools often use a low student–faculty ratio as a selling point to those choosing schools for tertiary education. Second, student peer group characteristics such as racial/ethnic diversity, average socioeconomic status (SES), and age could influence peer experiences, which eventually could influence student success outcomes (Titus, 2004). Many studies have suggested that diverse peers in the learning environment could improve intergroup relationships and mutual understanding by challenging students to refine their thinking (Morales et al., 2021). These interactions and experiences would influence the way students think, behave, as well as the overall satisfaction with the college experience and perceptions of the campus climate.
Lastly, school structural-demographic characteristics could potentially impact student success outcomes. The school size and type have been specifically linked to student success (Titus, 2004). For example, different types of higher education schools (e.g., public versus private; liberal arts versus large state schools) have drastically different education missions and policies supporting those missions. Each type of school’s value could determine the overall school environment for students and eventually impact their success outcomes (Terenzini & Reason, 2008). Specifically, studies found that students from private liberal arts colleges have a higher level of engagement, whereas students at large public universities were more satisfied with their college experiences (Hu & Kuh, 2002; Kuh & Siegel, 2000). Thus, based on the previous literature, the current study asks the following research question about school-level factors:
RQ2: How do school-level factors, including (a) school type, (b) school size, (c) student–faculty ratio, (d) estimated cost of yearly attendance, (e) the 4-year retention rate, (f) the 4-year graduation rate, and (g) the diversity rate, influence undergraduate engineering students’ success outcomes including academic achievement, learning outcomes, and satisfaction?
The current study and multilevel approach
The factors that could influence student success outcomes are complex and multi-faceted, and the current literature tends to focus on student factors. However, as one would assume, the student characteristics and the school environment would concurrently influence student success outcomes. As many previous education research projects have examined, understanding the different levels of factors would more accurately account for variances in student success outcomes (e.g., Ma & Klinger, 2000; Ma & Ma, 2014; Titus, 2004). However, to the best of our knowledge, no previous study has examined the concurrent influences of both individual determinants and the academic environment on undergraduate engineering students, particularly those in liberal arts colleges. Thus, the current study uses a multilevel approach to address this gap in the current research, which is particularly suitable for the following reasons. For one, the multilevel approach allows a comprehensive analysis of how student-level factors affect student success outcomes while accounting for the school-level variance and vice versa (Ma & Ma, 2014), for example, if we want to examine how the household income affects student GPA. However, students are clustered within different schools where average household incomes are significantly different between schools. Thus, the multilevel approach accounts for both within-group and between-group variances where the average household income at that specific school is part of the analysis. Such a linear hierarchical approach could more effectively analyze the naturally nested structure in education data (Raudenbush & Bryk, 2002). Moreover, the multilevel approach can examine the magnitudes of influences from each level and would answer the practical question of whether individual student or school environment matters more. Thus, we further ask the following research question:
RQ3: How much do student-level factors versus school-level factors comparatively influence undergraduate engineering students’ success outcomes?
The current study conducted a multi-school survey with undergraduate engineering students in the Midwestern and Northeastern United States. The study recruited students from 14 higher education organizations in two regions of the United States, ranging from large state universities to liberal arts colleges. Based on the current literature and our research questions, the survey assessed student success outcomes and student-level demographic factors, and the research team gathered relevant school-level information by contacting each school.
We recruited undergraduate students who were enrolled in an engineering program in the Midwestern and Northeastern United States by using convenience sampling. Convenience sampling is a non-probability sampling method that recruits a conveniently available group of participants. In this study, a recruitment email was sent to a total of 921 full-time instructors and faculty members at 18 higher education organizations, and we asked the instructor/faculty to share the survey link with their current students. A total of 532 students from 18 different schools responded to the survey, but we removed responses from four schools from the dataset due to a low response count (n < 10) from each school.
After collecting consent to participate, the survey collected demographic information including age, sex, race, classification (also known as year level; i.e., freshman, sophomore, junior, senior), household income, the name of the school, and whether the student was a first-generation college student (i.e., nobody from the core family has ever graduated from college). Then the survey assessed students’ communication competency, conflict management styles, self-reported learning outcomes, current GPA, satisfaction with the university, and satisfaction with the engineering program. Lastly, we collected the student’s school email addresses to send each participant an Amazon gift card as compensation and prevent duplicate responses. After the survey, we collected information about the 18 schools which had student responses. We used the information published by National Center for Education Statistics, and the information included the school type (i.e., public or private), university size (i.e., current number of students enrolled), the student–faculty ratio at the school, estimated yearly cost of attendance (i.e., published tuition based on 24 credit hours per academic year), the 4-year retention rate of the school, the 4-year graduation rate of the school, and diversity rate of the school (i.e., percentage of students who identified as non-White/Caucasian).
We recruited a total of 514 undergraduate engineering students from 14 different schools. We removed 56 cases with a significant amount of missing data (i.e., 50% of the survey response was missing). The final sample consisted of 458 undergraduate engineering students from 14 different higher education schools in the Midwestern and Northeastern United States. The age of the students ranged from 18 to 49 (M = 20.93, SD = 2.99). Most participants were male (n = 297, 64.8%). Most participants identified as White/Caucasian (n = 368, 80.3%); 22 (4.8%) identified as Black/African American; 34 (7.4%) identified as Asian; 18 (3.9%) identified as Latinx; five participants (1.1%) identified as Middle Eastern; 11 (2.4%) participants identified as multiracial. The final sample had a relatively even distribution among freshmen (n = 112, 24.5%), sophomores (n = 93, 20.3%), juniors (n = 111, 24.2%), and seniors (n = 141, 30.8%). Out of the 458 participants, 118 (25.75%) identified as first-generation students. Lastly, participants reported a wide range of household income from less than $10,000 a year to more than $150,000 a year, with the median household income being between $70,000 and $80,000 a year.
We measured learning outcomes using self-reported GPA and a 16-item survey instrument created based on the Associations of American Colleges and Universities (AACU) guidelines on important learning outcomes for engineering students. The student’s GPA ranged from 1.88 to 4.33 on a four-point system (M = 3.45, SD = 0.45). AACU listed four categories of essential learning outcomes, including intellectual and practical skills, communication and collaboration skills, personal and social responsibility skills, and advanced learning skills. Following the established tool of assessment (Ma & Klinger, 2000), we used 16 items to assess how often a student applied each skill in their engineering education on a 4-point Likert-type scale (1 = never; 4 = frequently). Higher scores indicated more frequent applications of essential skills learned in engineering education. The items formed a measure (M = 2.98, SD = 0.89) with acceptable reliability (α = 0.87).
We measured students’ satisfaction with their program and university. We used five pairs of opposite adjectives (i.e., bad–good, harmful–beneficial, unimportant–important, invaluable–valuable, uninspiring–inspiring) on a 7-point semantic differential scale to evaluate students’ satisfaction with their program and university. Higher scores indicated more favorable evaluations of their program and university. The items formed a program satisfaction measure (M = 5.71, SD = 1.05) with acceptable reliability (α = 0.85) and a university satisfaction measure (M = 5.44, SD = 1.32) with great reliability (α = 0.93).
We collected students’ age, sex, race, classification, household income, and whether the student was a first-generation college student. The descriptive statistics of these variables were reported in the Participants section. In addition, we used 12 items from a previously validated scale (McCroskey & McCroskey, 1988) to measure participant’s self-perceived communication competence with “various communication contexts (i.e., public, meeting, group, dyad) and receivers (strangers, acquaintance, friend)” (McCroskey & McCroskey, 1988, p. 4). The items measured participants’ self-evaluations of communication competence on a 10-point Likert-type scale (1 = completely Incompetent, 10 = competent). The items formed a measure (M = 7.52, SD = 1.52) with great reliability (α = 0.92).
The survey measured five different conflict management styles (i.e., collaboration, competition, avoidance, accommodation, and compromise) using 15 items. The measures were adopted from previous research on conflict management styles (Liu et al., 2008). Three items were used to measure each conflict style on a 4-point Likert-type scale (1 = rarely, 4 = always). A higher score on a conflict management style sub-scale indicated stronger associations with that style. Three items (e.g., “I discuss issues with others to try to find solutions that meet everyone’s needs.”) formed the collaboration sub-scale (M = 1.88, SD = 0.57) with acceptable reliability (α = 0.72); three items (e.g., “I would argue my case and insist on the advantages of my point of view.”) formed the competition sub-scale (M = 2.46, SD = 0.61) with acceptable reliability (α = 0.69); three items (e.g., “When I find myself in an argument, I usually say very little and try to leave as soon as possible.”) formed the avoidance sub-scale (M = 2.72, SD = 0.73) with acceptable reliability (α = 0.82); three items (e.g., “I try to meet the expectations of others.”) formed the accommodation sub-scale (M = 2.02, SD = 0.61) with great reliability (α = 0.90); three items (e.g., “I try to negotiate and use a give-and-take approach to problem situations.”) formed the compromise sub-scale (M = 2.20, SD = 0.57) with great reliability (α = 0.91).
We gathered the information about whether the school was a public or private school, university size, student–faculty ratio at the school, estimated cost of yearly attendance, the 4-year retention rate, the 4-year graduation rate, and the diversity rate. Out of the 14 schools with participants who had more than 10 responses to the survey, 7 (50%) were private schools. The student enrollment size (i.e., university size) ranged from 1,141 to 46,900 (M = 15,336, SD = 14,907). The student–faculty ratio of the school ranged from eight-students to one-faculty to 19-students to one-faculty (M = 13.27, SD = 3.86). The cost of yearly tuition ranged from $8,900 to $58,200 (M = 32,186, SD = 19,093). The 4-year retention rate ranged from 72 to 99% (M = 88.80%, SD = 7.54%). The 4-year graduation rate ranged from 53 to 97% (M = 77.87%, SD = 12.74%). The diversity rate ranged from 37 to 79% (M = 64.93%, SD = 13.14%). The full list of information for each school is presented in Table 1.
We first dummy-coded race, sex, first-generation college student status, and whether the school was private or not. For race, the value 0 indicated that the participant identified as White/Caucasian, and 1 indicated that the participants identified as a race other than White/Caucasian. For sex, the value 0 indicated that the participant identified as male, and 1 indicated the participant identified as female. For student classification, it was measured and treated as an ordinal variable where a higher score indicated a higher classification, ranging from freshman to senior. For the first-generation college student status, the value 0 indicated that the participant did not identify as a first-generation college student, and 1 indicated the participant identified as a first-generation college student. For private school status, the value of 0 indicated the school was not a private school, and 1 indicated the school was private. We then ran a series of bivariate correlations between all variables measured at the student level to check any potential issues of multicollinearity, and no issue of multicollinearity was identified (all r < 0.70). The data were structured to be students’ responses nested within the school. We then used HLM 8.1.4 (by Scientific Software International, INC.; Raudenbush & Bryk, 2002) to build four hierarchical linear models (HLMs) to estimate the effects of both student and school-level factors on student success outcomes. Each HLM has a student outcome listed as the dependent variable, and the outcomes included self-reported learning outcomes, current grade point average (GPA), satisfaction with the university, and satisfaction with the engineering program. For each HLM, the first-level variables included age, classification (treated as an ordinal variable), household income (treated as an ordinal variable), dummy-coded sex, dummy-coded race, dummy-coded first-generation college student status, communication competency, and five conflict management styles. All continuous variables (i.e., age, communication competency, conflict management styles) were entered as group mean-centered to avoid loss of the integrity of group comparisons. The second level variables included the dummy-coded private school status, university size, students-to-faculty ratio in the engineering program, estimated cost of yearly attendance, the 4-year retention rate of the engineering program, the 4-year graduation rate of the engineering program, and the diversity rate of the engineering program. All variables besides the dummy-coded private school status were grand mean-centered for more meaningful interpretations.
We first built a null model with only the dependent variable entered. Then, we added the student-level variables and then used the estimation of fixed effects with robust standard errors to determine the significance level of each variable. We then used data-driven backward deletion to remove the first-level variables one by one until all variables in the model were significant (Ma & Klinger, 2000). The same data-driven backward deletion and iteration processes were repeated for the school-level variables. We used backward deletion as it was the most accurate approach in HLM when the multilevel models had no clear theoretical guidance (Ma & Klinger, 2000). We present the results of the final models after the backward deletion with the unstandardized coefficient (B) and p-value of each significant factor.
Learning outcomes model
At the student-level, being a female student (B = 1.75, p < 0.001), having a higher household income (B = 0.11, p < 0.001), having a higher classification (B = 0.19, p < 0.01), not being a first-generation student (B = − 1.27, p < 0.001), having better communication competency (B = 0.10, p < 0.05), identified more as a conflict avoider (B = 1.09, p < 0.001), and identified less as a conflict accommodator (B = − 0.31, p < 0.01) were all positively associated with learning outcomes, while holding all other variables statistically constant. At the school level, being in a private school (B = 1.26, p < 0.001) and having higher tuition (per every $10 k; B = 0.28, p < 0.001) were all positively associated with learning outcomes while holding all other variables statistically constant. The B value is unstandardized coefficients between two variables and can be interpreted as the number of units that would change in the dependent variable when there is one unit of increase in the independent variable while controlling for all other variables (Raudenbush & Bryk, 2002). For example, our results showed that one unit increase in students’ classification (i.e., one level higher) would lead to a 0.19 unit increase in the learning outcomes scores. The proportion of overall variance explained by the model was 28.70%; for student level, the proportion of variance has been explained by the model was 11.99%; for school level, the proportion of variance has been explained by the model was 16.72%.
At the student-level, being a female student (B = 0.25, p < 0.001), not being a first-generation student (B = − 0.86, p < 0.001), having better communication competency (B = 0.26, p < 0.001) were all positively associated with higher GPA, while holding all other variables statistically constant. At the school level, being in a private school (B = 1.06, p < 0.001), having a lower student–faculty ratio (B = − 0.42, p < 0.001), and having a higher diversity rate (B = 2.06, p < 0.001) were all positively associated with higher GPA, while holding all other variables statistically constant. The proportion of overall variance explained by the model was 42.59%; for student level, the proportion of variance explained by the model was 24.61%; for school level, the proportion of variance explained by the model was 17.98%.
Program satisfaction model
The Chi-square test of variance components indicated there was not a significant amount (p > 0.50) of variance explained by the school-level factors. Thus, the second level of the model was consequently not specified. At the student-level, being a female student (B = 2.56, p < 0.05), having a higher classification (B = 0.70, p < 0.05), being a racial minority student (B = 2.55, p < 0.001), having better communication competency (B = 0.22, p < 0.01), identified more as a conflict collaborator (B = 0.99, p < 0.05), identified less as a conflict avoider (B = − 1.54, p < 0.001), and identified more as a conflict accommodator (B = 1.44, p < 0.001) were all positively associated with higher program satisfaction, while holding all other variables statistically constant. The proportion of variance explained by the model was 17.88%.
University satisfaction model
At the student-level, being a first-generation student (B = 1.22, p < 0.001), being a racial minority student (B = 4.63, p < 0.001), identified more as a conflict compromiser (B = 2.55, p < 0.001) were all positively associated with high university satisfaction, while holding all other variables statistically constant. At the school level, being in a school with more students (per 10 k students; B = 0.63, p < 0.001) and having a higher retention rate (B = 0.66, p < 0.05) were all positively associated with higher university satisfaction while holding all other variables statistically constant. The proportion of overall variance explained by the model was 26.21%; at the student level, the proportion of variance explained by the model was 13.98%; at the school level, the proportion of variance explained by the model was 11.25%. The full results of all four models can be found in Table 2.
The current study seeks to understand how several relevant student- and school-level factors influence student success outcomes among undergraduate engineering students. We conducted a multi-school multilevel survey project to answer the research questions. Overall, both factors were significantly associated with the student success outcomes, except that the school-level factors were not correlated with program satisfaction. Some student demographic characteristics and individual skills/traits, particularly students’ gender, race, first-generation student status, and communication competency, were all significantly related to student success outcomes. Some school-level factors indeed influenced different student success outcomes, but the results remained inconsistent.
The current study contributes to the literature in the following ways. First, the factors analyzed and tested in the previous studies mainly existed at the student level, and school-level factors, such as school type (e.g., private vs. public) and diversity rate were not well studied. Our study expands the understanding by including the school-level factors, which indeed had significant impacts on student success outcomes based on our findings. Second, many studies recruited students from a limited range of schools by only including students from one type of school (e.g., large public research-intensive universities). Although some research studies students from multiple schools (Marbouti et al., 2021), there is a sparse representation of school diversity, such as the inclusion of engineering students from liberal arts schools. Our study makes novel attempts to include a wide variety of different types of schools. Lastly, to the best of our knowledge, we are the first project that examined student success outcomes among undergraduate engineering students using the multilevel analytical approach, which more accurately reflected the theorized data structure of ecological influences on student success. We further expand on some of the intriguing findings and their practical implications in this discussion.
RQ1 asked how the student-level factors would influence student success outcomes. First and foremost, communication competency was consistently associated with all four student success outcomes. Today, communication is an essential component in engineering education, recognized by academicians and industry professionals because engineers need to communicate through a growing array of ways to an increasing range of audiences (Yeke & Semerciöz, 2016). The accreditation criteria established by the Accreditation Board for Engineering and Technology (ABET) specifically highlight the importance of communication skills (Hussain et al., 2021). In addition, the development of new communications technologies, an increasingly global marketplace, and an increased emphasis on teamwork all illustrate the need to improve the communication competence of engineers (Tenopir & King, 2004). In engineering education, the primary focus has been on communication competencies related to technical speaking and writing skills. However, succeeding in an engineering job also requires the ability to communicate, pursue information, maintain relationships, and regulate one’s actions and beliefs in various contexts (Sageev & Romanowski, 2001). Thus, we emphasize the importance of collaborative instructions with social science disciplines such as communication and education to improve communication competence among future engineers.
Second, female students outperformed male students in GPA and learning outcomes, and they also reported higher university and program satisfaction. These findings are consistent with previous higher and K-12 education research (French et al., 2005; Van den Bogaard, 2012). Many female students reported that their male peers gave their ideas less credit and failed to trust them with technical work in group projects (Wolfe et al., 2016). This situation might lead to unequal learning opportunities, which potentially leads female students to work harder to gain acceptance and equity. Consequently, female students might study and work harder to stay competitive in a male-dominated field (Subri, 2018). Moreover, women show greater interest than men in solving societal issues. Female students have engaged more actively than male students in non-engineering extracurricular activities (Chachra & Kilgore, 2009). Those activities may improve some non-technical skills in leadership and communication, which would positively contribute to student success outcomes (Ro & Knight, 2016).
Third, racial minority students reported higher university and program satisfaction but did not report better GPA and learning outcomes. Contrary to findings of previous studies, racial minority students in our sample did not have statistically significantly lower GPAs or learning outcomes. However, our results could be related to the limited number of racial minority students in our sample. Minority students, especially Black students, are less likely to participate in a voluntary survey compared to their White peers (Jang & Vorderstrasse, 2019). The underrepresentation of racial minority students has been a common issue in engineering education and research (Roy, 2019). Future engineering education research should explore and employ persuasive recruiting strategies to more effectively recruit minority students in STEM. More interestingly, minority students reported higher university and program satisfaction than their White peers. This could be accredited to some of the minority student mentoring programs, in which many universities have made active efforts to better serve minority students. Minority students may have limited professional, social, and financial supports for their study and career life from their families. Plus, they are more likely to encounter more challenges when they enter a college environment where the predominant racial, ethnic, or religious culture differs from their own (Foor et al., 2007). Effective mentoring could assist racial and ethnic minority students to advance socially, politically, and economically, especially for those who are also first-generation students (Atkins et al., 2020).
Lastly, first-generation students reported poorer learning outcomes and lower GPA than students whose parents had obtained at least some college education. These findings are consistent with previous research, where first-generation students are less engaged and less likely to integrate into their college experiences successfully (Pike & Kuh, 2005). In addition, first-generation students perceive the college environment as less supportive and have slower progress in their intellectual development due to a lack of educational aspirations (Pike & Kuh, 2005). Previous research found that educational aspiration from family is one of the strongest influencers of the first-semester GPA among first-generation students, yet they often did not receive proper support and understanding from their families (Smith & Lucena, 2016). First-generation students have been a growing population in higher education over the last two decades, but they are less equipped with resources, social support, social capital, and coping tools (Martin et al., 2020).
The current study reflected the nested structure of factors that influenced student outcomes through study recruitment and analytical approaches. RQ3 sought to compare the differences in influences from student-level factors versus school-level factors on student success outcomes. The findings showed that school-level factors explained less (compared to student-level), but still a significant portion of variances in student success outcomes. School-level factors reflect faculty engagement, cultural atmosphere, and the social environment (Kuley et al., 2015). It might not be surprising that the environment around us affects how we function in various contexts, and the education environment is no exception (Pike et al., 2003). School-level characteristics are the underlying mechanisms that can either exacerbate or supersede student-level factors. Moreover, student success outcomes should be viewed as the result of a series of long-term intentional school actions, policies, and practices that gradually influence the larger environment.
RQ2 asked how organizational characteristics would influence undergraduate engineering students’ outcomes. For one, being in a school that has a higher racial diversity rate was positively associated with a higher average GPA. Such evidence indicates the importance of diversity in a higher education school and calls for more school actions, policies, and practices that actively include diversity-oriented mentorship and support. Such actions, policies, and practices do not only encourage the success of racial minority students in engineering but could also potentially enhance success for all students.
Interestingly, the results showed that having higher tuition was positively associated with learning outcomes. Even though previous research has pointed out how students are particularly concerned with the affordability of their education, engineering students, even those from low-income households, are willing to invest more in their undergraduate education due to expected high financial returns upon graduation (Geiger, 2004). A previous study found that students in engineering were less sensitive to the price of tuition (Shin & Milton, 2008). These students are more sensitive to the quality of engineering programs, which is visible to students through more advanced equipment and the availability of resources often paid through higher tuition. However, this does not mean that tuition-related policies will not influence student success outcomes as college students in the United States today face considerable and historically unprecedented challenges in financing college education (Wolniak et al., 2018). It explains why financial aid policies still appear to play a central role in promoting student outcomes among students from any income level (Tinto & Pusser, 2006). However, the resources and advanced equipment can often come with a pricey tuition fee. Tuition policymaking is complex and is sensitive to local, state, and school culture and political contexts (Pusser, 2003). Our findings potentially suggest the viable and effective practice of simultaneously keeping high tuition policies and increasing financial aid as a method to maintain and improve student success outcomes, especially for low SES students in engineering.
To our surprise, the findings showed that the student–faculty ratio was negatively associated with student GPA. In other words, students from a school with a higher student -faculty ratio reported higher GPAs. However, the moderator of this relationship might be faculty distance, where higher faculty distance lowered self-efficacy, academic confidence, and GPA among students (Vogt, 2008). Simply because a school has a lower student–faculty ratio does not automatically mean faculty have “closer” interactions with students. For example, larger research-intensive schools now hire more non-tenure and/or part-time instructors to teach undergraduate courses, while more tenure-track faculty members focus on research and graduate education (American Association of University Professors, 2018). Thus, even the student–faculty ratios could potentially be higher at these larger public research-intensive universities, some of these faculty members might have limited interactions with undergraduate students.
Student–faculty distance is an individual perception-based variable and could even be faculty-specific. Thus student–faculty distance might be a more accurate indicator of student outcomes, rather than solely relying on the student–faculty ratio. Moreover, large state universities sampled in the current study often have more research faculty members who might have fewer interactions with undergraduate students than smaller liberal arts colleges. Policymakers should pay more attention to interactions, relationships, and distance between students and faculty, as student–faculty ratios might not accurately indicate the quality of faculty-and-student interactions. Hiring “closer” faculty members that would actively interact, engage, and support their students might be more effective than hiring more faculty.
Based on our findings, we make the following suggestions for engineering program development and policymaking. First, we recommend that engineering programs consider establishing mentorship programs with higher-performing female and racial minority students as peer mentors to other underrepresented engineering students. Our findings suggested that some of these female and racial minority students reported better learning outcomes and GPAs compared to their counterparts. These higher-performing students might have school and life skills that might be valuable to their peers and other underrepresented engineering students, such as first-generation students. Students will select their peer mentors based on their preferred identity concordance rather than be paired by the program. Representation and concordance are proven to effectively motivate both female and minority students in undergraduate mentorship programs (Morales et al., 2021; Zaniewski & Reinholz, 2016). The peer mentorship program will focus on helping students increase social capital and learn coping skills, such as difficulties with identity-specific challenges and extracurricular affairs (Puccia et al., 2021).
Second, we suggest that engineering programs consider establishing a social network-based mentorship program that includes first-generation students’ family members. As our findings and previous studies have shown, motivating first-generation students has to consider their interpersonal networks and social capitals (Martin et al., 2020). The program can host a series of “family visit” events, in which families and important interpersonal contacts of first-generation students will meet and interact with faculty and other students. The program’s main purposes would be to educate families and interpersonal contacts about engineering basics so these people could better understand students’ engineering majors and curriculum and teach family members practical tools and resources to support their first-generation students. The goals of the programs are to create a common understanding and shared social capital between first-generation students and their families through interactive workshops, fun activities, and peer interactions.
Third, we recommend the development of hybrid courses that better integrate communication into engineering. Although many engineering curricula offer project-based courses, they mainly focus on engineering design and field-specific communications skills outlined by the ABET criteria. It is critical to improving students’ overall communication competence and conflict management skills to face real-life team challenges, business demands, and interdisciplinary tasks. Faculty from engineering and social science disciplines could collaborate on developing courses and workshops that focus on improving these skills. For example, social science faculty could give lectures related to leadership, communication, and business management strategies, while engineering faculty could develop engineering projects that reflect current industry demands that require such skills. Two groups of faculty should collaborate on designing real-life engineering case studies and interactive activities that could improve communication competency and other social skills.
In addition, some schools offer engineering leadership certificate programs that aim to train undergraduate engineering students’ professional skills, so students can be better prepared for the corporate world. These programs have shown to have great potentials in improving six dimensions of competency among undergraduate engineering students, including “communication, innovation, creativity, execution, personal drive, and teamwork” (Paul & Cowe Falls, 2015, p. 1). Further integrating the aforementioned hybrid courses into the leadership certificate programs could enhance the effectiveness of achieving the goals of these programs.
Limitations and future research
The findings of the current study should be interpreted within its limitations. First, the data were collected from students in only the Midwestern and Northeastern United States to control for the variance related to regional differences. For example, students from California might have drastically different educational experiences than those from Michigan. In order to more accurately account for such regional variance, a much larger nationwide sample of students is needed, but such a project, unfortunately, was beyond the scope of the current study.
Second, there might be limited randomness in our sample as we did not (also were unable to) recruit students via direct communication, and the survey was forwarded by faculty only. Thus, there could be potential selection bias related to faculty’s willingness to share the survey with their students.
Third, GPA is an important student success outcome that was measured in our current study, but the average GPAs for programs and schools were not controlled for in the current analyses. After repeated requests for such information, many department chairs and program directors were not able to provide such information due to privacy concerns. Consequently, we were not able to include the average GPAs for programs and schools as school-level variables in our analyses. The lack of such information means we had less control over potential self-selection bias, where students who had higher GPAs might be more likely to participate in a survey about their academic performance.
Lastly, there could be cross-factor effects on student success outcomes that we did not explore in the current analyses. For example, a student might be a first-generation female student or a minority student with poor communication competency. Furthermore, student-level factors, such as racial minority status, could interact with school-level factors, such as diversity rate, to have significant impacts on student success outcomes. Exploring and analyzing these more complex moderations address separate sets of research questions from the current manuscript, and it would require substantially more space beyond the length of this manuscript. Thus, we plan to explore such questions in a different manuscript as part of the future efforts of this project, and our future project would use supervised machine learning algorithms to better understand the classifications of school- and student-level factors.
The current study sought to understand how both student- and school-level factors influence student success outcomes through a multi-school multilevel approach. Engineering education is a complex, multi-faceted issue that requires more collaborative and systematic research. We hope our findings help educators understand the different factors that could potentially influence engineering students and inform better program design and policymaking.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Science, technology, engineering, and mathematics
Grade point average
Associations of American Colleges and Universities
Hierarchical linear models
Accreditation Board for Engineering and Technology
American Association of University Professors. (2018). Data snapshot: Contingent faculty in U.S. higher ed. Retrieved from https://www.aaup.org/sites/default/files/10112018%20Data%20Snapshot%20Tenure.pdf
Atkins, K., Dougan, B. M., Dromgold-Sermen, M. S., Potter, H., Sathy, V., & Panter, A. T. (2020). “Looking at Myself in the Future”: How mentoring shapes scientific identity for STEM students from underrepresented groups. International Journal of STEM Education, 7(1), 42. https://doi.org/10.1186/s40594-020-00242-3
Bean, J., & Eaton, S. B. (2001). The psychology underlying successful retention practices. Journal of College Student Retention: Research, Theory and Practice, 3(1), 73–89. https://doi.org/10.2190/6R55-4B30-28XG-L8U0
Bossart, J., & Bharti, N. (2017). Women in engineering: Insight into why some engineering departments have more success in recruiting and graduating women. American Journal of Engineering Education., 8(2), 127–140. https://doi.org/10.19030/ajee.v8i2.10070.
Chachra, D., & Kilgore, D. (2009). Exploring gender and self-confidence in engineering students: A multi-method approach. In Center for the Advancement of Engineering Education (NJ1). Center for the Advancement of Engineering Education. https://eric.ed.gov/?id=ED542116
Chang, W. L., & Lee, C. Y. (2013). Virtual team e-leadership: The effects of leadership style and conflict management mode on the online learning performance of students in a business-planning course. British Journal of Educational Technology, 44(6), 986–999. https://doi.org/10.1111/bjet.12037
Fletcher, T. L., Jefferson, J. P., Boyd, B. N., & Cross, K. J. (2021). Missed opportunity for diversity in engineering: Black women and undergraduate engineering degree attainment. Journal of College Student Retention: Research, Theory and Practice. https://doi.org/10.1177/1521025120986918
Foor, C. E., Walden, S. E., & Trytten, D. A. (2007). I wish that I belonged more in this whole engineering group: Achieving individual diversity. Journal of Engineering Education, 96(2), 103–115. https://doi.org/10.1002/j.2168-9830.2007.tb00921.x
French, B. F., Immekus, J. C., & Oakes, W. C. (2005). An examination of indicators of engineering students’ success and persistence. Journal of Engineering Education, 94(4), 419–425. https://doi.org/10.1002/j.2168-9830.2005.tb00869.x
Geiger, R. L. (2004). Knowledge and money: Research universities and the paradox of the marketplace. USA: Stanford University Press.
Geisinger, B. N., & Raman, D. R. (2013). Why they leave: Understanding student attrition from engineering majors. International Journal of Engineering Education, 29(4), 914–925. https://lib.dr.iastate.edu/abe_eng_pubs/607/
Goldman, A. M. (2019). Interpreting rural students’ stories of access to a flagship university. The Rural Educator, 40(1). https://doi.org/10.35608/RURALED.V40I1.530
González, N., Moll, L. C., & Amanti, C. (Eds.). (2006). Funds of knowledge: Theorizing practices in households, communities, and classrooms. Routledge. ISBN: 978-0805849189
Hu, S., & Kuh, G. D. (2002). Being (dis) engaged in educationally purposeful activities: The influences of student and institutional characteristics. Research in Higher Education, 43(5), 555–575. https://www.jstor.org/stable/40197272
Hussain, W., Spady, W., Khan, S., Khawaja, B., Naqash, T., & Conner, L. (2021). Impact evaluations of engineering programs using ABET student outcomes. IEEE Access, 1–9. https://doi.org/10.1109/ACCESS.2021.3066921.
Jang, M., & Vorderstrasse, A. (2019). Socioeconomic status and racial or ethnic differences in participation: Web-based survey. JMIR Research Protocols, 8(4), e11865. https://doi.org/10.2196/11865
Koshland, C. P. (2010). Liberal arts and engineering. In D. Grasso & M. B. Burkins (Eds.), Holistic engineering education (pp. 53–67). Springer.
Kuh, G. D., Kinzie, J. L., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2006). What matters to student success: A review of the literature. In commissioned report for the national symposium on postsecondary student success. National Postsecondary Education Cooperative. https://nces.ed.gov/npec/pdf/Kuh_Team_Report.pdf
Kuh, G. D., & Siegel, M. J. (2000). College student experiences questionnaire: Tentative norms for the (4th ed.). Indiana University.
Kuley, E., Maw, S., Fonstad, T., & Kuley, L. (2015, August 7). Engineering student retention and attrition literature review. In Proceedings of the Canadian Engineering Education Association. Canadian Engineering Education Association. https://doi.org/10.24908/pceea.v0i0.5813
Liu, X., Magjuka, R. J., & Lee, S. H. (2008). The effects of cognitive thinking styles, trust, conflict management on online students’ learning and virtual team performance. British Journal of Educational Technology, 39(5), 829–846. https://doi.org/10.1111/j.1467-8535.2007.00775.x
Ma, V. J., & Ma, X. (2014). A comparative analysis of the relationship between learning styles and mathematics performance. International Journal of STEM Education, 1(1), 3. https://doi.org/10.1186/2196-7822-1-3
Ma, X., & Klinger, D. A. (2000). Hierarchical linear modeling of student and school effects on academic achievement. Canadian Journal of Education, 25(1), 41–55. https://doi.org/10.2307/1585867
Marbouti, F., Ulas, J., & Wang, C. H. (2021). Academic and demographic cluster analysis of engineering student success. IEEE Transactions on Education, 64(3), 261–266. https://doi.org/10.1109/TE.2020.3036824
Marra, R. M., Rodgers, K. A., Shen, D., & Bogue, B. (2012). Leaving engineering: A multi-year single institution study. Journal of Engineering Education, 101(1), 6–27. https://doi.org/10.1002/j.2168-9830.2012.tb00039.x
Martin, J. P., Stefl, S. K., Cain, L. W., & Pfirman, A. L. (2020). Understanding first-generation undergraduate engineering students’ entry and persistence through social capital theory. International Journal of STEM Education, 7(1), 37. https://doi.org/10.1186/s40594-020-00237-0
McCroskey, J. C., & McCroskey, L. L. (1988). Self-perceived communication competence scale (SPCC). Communication Research Reports, 5, 108–133.
Montoya-Weiss, M. M., Massey, A. P., & Song, M. (2001). Getting it together: Temporal coordination and conflict management in global virtual teams. Academy of Management Journal, 44(6), 1251–1262. https://doi.org/10.2307/3069399
Morales, D. X., Grineski, S. E., & Collins, T. W. (2021). Effects of mentoring relationship heterogeneity on student outcomes in summer undergraduate research. Studies in Higher Education, 46(3), 423–436. https://doi.org/10.1080/03075079.2019.1639041
Paul, R., & Cowe Falls, L. G. (2015). Engineering leadership education: A review of best practices. Proceedings of the American Society for Engineering Education Annual Conference & Exposition, 1–11. https://peer.asee.org/23972
Paul, S., Samarah, I. M., Seetharaman, P., & Mykytyn, P. P. (2004). An empirical investigation of collaborative conflict management style in group support system-based global virtual teams. Journal of Management Information Systems, 21(3), 185–222. https://doi.org/10.1080/07421222.2004.11045809
Pike, G. R., & Kuh, G. D. (2005). First-and second-generation college students: A comparison of their engagement and intellectual development. The Journal of Higher Education, 76(3), 276–300. https://www.jstor.org/stable/3838799
Pike, G. R., Kuh, G. D., & Gonyea, R. M. (2003). The relationship between institutional mission and students’ involvement and educational outcomes. Research in Higher Education, 44(2), 241–261. https://doi.org/10.1023/A:1022055829783
Puccia, E., Martin, J. P., Smith, C. A. S., Kersaint, G., Campbell-Montalvo, R., Wao, H., Lee, R., Skvoretz, J., & MacDonald, G. (2021). The influence of expressive and instrumental social capital from parents on women and underrepresented minority students’ declaration and persistence in engineering majors. International Journal of STEM Education, 8(1), 20. https://doi.org/10.1186/s40594-021-00277-0
Pusser, B. (2003). Beyond Baldridge: Extending the political model of higher education organization and governance. Educational Policy, 17(1), 121–140. https://doi.org/10.1177/0895904802239289
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). USA: Sage.
Ro, H. K., & Knight, D. B. (2016). Gender differences in learning outcomes from the college experiences of engineering students. Journal of Engineering Education, 105(3), 478–507. https://doi.org/10.1002/jee.20125
Roy, J. (2019). Engineering by the numbers. American Society for Engineering Education. https://news.engineering.pitt.edu/download/1026047/2018-Engineering-by-Numbers-Engineering-Statistics-UPDATED-15-July-2019.pdf
Sageev, P., & Romanowski, C. J. (2001). A message from recent engineering graduates in the workplace: Results of a survey on technical communication skills. Journal of Engineering Education, 90(4), 685–693. https://doi.org/10.1002/j.2168-9830.2001.tb00660.x
Shin, J. C., & Milton, S. (2008). Student response to tuition increase by academic majors: Empirical grounds for a cost-related tuition policy. Higher Education, 55(6), 719–734. https://doi.org/10.1007/s10734-007-9085-1
Smith, J. M., & Lucena, J. C. (2016). Invisible innovators: How low-income, first-generation students use their funds of knowledge to belong in engineering. Engineering Studies, 8(1), 1–26. https://doi.org/10.1080/19378629.2016.1155593
Subri, U. S. (2018). A review of job challenges factors for women engineer. International Journal of Academic Research in Business and Social Sciences, 8(9), 1450–1455. https://doi.org/10.6007/IJARBSS/v8-i9/4854
Tenopir, C., & King, D. W. (2004). Communication Patterns of Engineers. Wiley.
Terenzini, P., & Reason, R. (2008). Parsing the first year of college year 3 progress report. Report to Spencer Foundation (Major Grant No. 200600114). http://www.uwyo.edu/accreditation/_files/archive/Parsing_Yr3_Progress_Rep.pdf
Tinto, V., & Pusser, B. (2006). Moving from theory to action: Building a model of institutional action for student success. National Postsecondary Education Cooperative. https://nces.ed.gov/npec/pdf/Tinto_Pusser_Report.pdf
Tinto, V. (1987). Leaving college: Rethinking the Causes and Cures of Student Attrition. Chicago: University of Chicago Press.
Titus, M. A. (2004). An examination of the influence of institutional context on student persistence at 4-year colleges and universities: A multilevel approach. Research in Higher Education, 45(7), 673–699. https://doi.org/10.1023/B:RIHE.0000044227.17161.fa
Van den Beemt, A., MacLeod, M., Van der Veen, J., Van de Ven, A., van Baalen, S., Klaassen, R., & Boon, M. (2020). Interdisciplinary engineering education: A review of vision, teaching, and support. Journal of Engineering Education, 109(3), 508–555. https://doi.org/10.1002/jee.20347
Van den Bogaard, M. (2012). Explaining student success in engineering education at Delft University of Technology: A literature synthesis. European Journal of Engineering Education, 37(1), 59–82. https://doi.org/10.1080/03043797.2012.658507
Vogt, C. M. (2008). Faculty as a critical juncture in student retention and performance in engineering programs. Journal of Engineering Education, 97(1), 27–36. https://doi.org/10.1002/j.2168-9830.2008.tb00951.x
Wilkins, K. G., Bernstein, B. L., & Bekki, J. M. (2015). Measuring communication skills: The STEM interpersonal communication skills assessment battery. Journal of Engineering Education, 104(4), 433–453. https://doi.org/10.1002/jee.20100
Wolfe, J., Powell, E. A., Schlisserman, S., & Kirshon, A. (2016, June). Teamwork in engineering undergraduate classes: What problems do students experience? ASEE Annual Conference and Exposition. https://monolith.asee.org/public/conferences/64/papers/16447/view
Wolniak, G., George, C., & Nelson, G. (2018). The emerging differential tuition era among U.S. public Universite. In Under pressure (pp. 191–214). https://doi.org/10.1163/9789004398481_012
Yeke, S., & Semerciöz, F. (2016). Relationships between personality traits, cultural intelligence and intercultural communication competence. Procedia-Social and Behavioral Sciences, 235, 313–319. https://doi.org/10.1016/j.sbspro.2016.11.036
Zaniewski, A. M., & Reinholz, D. (2016). Increasing STEM success: A near-peer mentoring program in the physical sciences. International Journal of STEM Education, 3(1), 14. https://doi.org/10.1186/s40594-016-0043-2
The current project received funding support from an internal grant from University of Mount Union. The open-access publishing charge was supported by an internal grant from University of Mount Union and another internal grant from Kennesaw State University.
Ethics approval and consent to participate
This study was reviewed and approved by the University Institutional Review Board (IRB 2021-482) at University of Mount Union.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wang, X., Dai, M. & Mathis, R. The influences of student- and school-level factors on engineering undergraduate student success outcomes: A multi-level multi-school study. IJ STEM Ed 9, 23 (2022). https://doi.org/10.1186/s40594-022-00338-y