Motivational climate predicts effort and achievement in a large computer science course: examining differences across sexes, races/ethnicities, and academic majors
International Journal of STEM Education volume 10, Article number: 65 (2023)
The motivational climate within a course has been shown to be an important predictor of students’ engagement and course ratings. Because little is known about how students’ perceptions of the motivational climate in a computer science (CS) course vary by sex, race/ethnicity, and academic major, we investigated these questions: (1) To what extent do students’ achievement and perceptions of motivational climate, cost, ease, and effort vary by sex, race/ethnicity, or major? and (2) To what extent do the relationships between students’ achievement and perceptions of motivational climate, cost, and effort vary by sex, race/ethnicity, and major? Participants were enrolled in a large CS course at a large public university in the southeastern U.S. A survey was administered to 981 students in the course over three years. Path analyses and one-way MANOVAs and ANOVAs were conducted to examine differences between groups.
Students’ perceptions of empowerment, usefulness, interest, and caring were similar across sexes and races/ethnicities. However, women and Asian students reported lower success expectancies. Students in the same academic major as the course topic (i.e., CS) generally reported higher perceptions of the motivational climate than students who did not major or minor in the course topic. Final grades in the course did not vary by sex or race/ethnicity, except that the White and Asian students obtained higher grades than the Black students. Across sex, race/ethnicity, and major, students’ perceptions of the motivational climate were positively related to effort, which was positively related to achievement.
One implication is that females, Asian students, and non-CS students may need more support, or different types of support, to help them believe that they can succeed in computer science courses. On average, these students were less confident in their abilities to succeed in the course and were more likely to report that they did not have the time needed to do well in the course. A second implication for instructors is that it may be possible to increase students’ effort and achievement by increasing students’ perceptions of the five key constructs in the MUSIC Model of Motivation: eMpowerment, Usefulness, Success, Interest, and Caring.
The motivational climate in a course has been defined as “the aspects of the psychological environment that affect students’ motivation and engagement within a course” (Jones et al., 2022b, p. 1). Understanding students’ perceptions of the motivational climate is important because positive motivational climate perceptions have been associated with many positive outcomes, such as increased student engagement (Christenson et al., 2012; Jones et al., 2021, 2023a), and higher instructor and course ratings in undergraduate courses (Jones & Skaggs, 2016; Jones & Wilkins, 2023; Wilkins et al., 2021). The motivational climate in a course is perceived by each student individually, and therefore, students’ perceptions of the motivational climate can differ significantly in some cases (Jones et al., 2022; Lam et al., 2015; Schweig & Martínez, 2021). Furthermore, these differences in perceptions can vary by sex/gender and race/ethnicity (Harper et al., 2019; Schenke et al., 2017). Understanding differences in perceived motivational climates within courses by sex/gender and race/ethnicity could provide insights into how teaching practices can have differential effects on students’ engagement, persistence, and achievement in courses.
The purposes of this study were to examine students’ achievement and perceptions of motivational climate in a computer science (CS) course to determine the extent to which (1) students’ achievement and perceptions of motivational climate, cost, ease, and effort vary by sex, race/ethnicity, or major, and (2) the relationships between students’ achievement and perceptions of motivational climate, cost, and effort vary by sex, race/ethnicity, and major. Having a better understanding of how these motivational climate constructs vary by sex, race/ethnicity, and major could be useful because these constructs can be manipulated by the instructor (Hulleman et al., 2017; McGinley & Jones, 2014). Consequently, it may be possible to develop interventions that focus on certain aspects of the motivational climate to increase the motivation and engagement of the targeted groups. Targeting students in lower-level CS courses can be especially important because students’ perceptions of lower-level courses can contribute to their decision to remain in their major (Jones et al., 2014, 2016; Nelson et al., 2015). Because engineering students are often required to take these lower-level CS courses, and CS courses are sometimes located within engineering departments, the findings of this study may be relevant beyond CS to the field of engineering more broadly. Women and racially minoritized students are also a key population to target for recruitment and retention in CS and engineering because they are underrepresented in these professions and earn fewer degrees compared to represented White and Asian students (Meiksins & Layne, 2022; National Center for Science & Engineering Statistics, 2021). Ultimately, instructional strategies could be employed to engage all engineering students more deeply in the course activities to create an equitable and inclusive climate that helps to recruit and retain them within CS and engineering majors, which could lead to more employee diversity in these disciplines.
The MUSIC Model of Motivation Theory
We used the MUSIC Model of Motivation Theory (Jones, 2009, 2018, 2020) as our theoretical framework because it has a strong theoretical and empirical basis, and it can be applied within CS and engineering courses to better understand students’ motivations and course engagement (e.g., Jones et al., 2013, 2014, 2016; Hall et al., 2013; Mora et al., 2017; Tendhar et al., 2017). The MUSIC model includes five course perceptions that comprise the motivational climate within a course: eMpowerment, Usefulness, Success, Interest, and Caring (the initial sounds of the words form the acronym MUSIC). Empowerment refers to students’ perceptions of control (e.g., choice, freedom) within their learning environment. Usefulness refers to students’ perceptions of the usefulness of the course content or activities in relation to their goals, either in the short- or long-term. The Success component focuses on the extent to which students believe that they can succeed in the course activities. Interest is the extent to which students perceive the course to be interesting and enjoyable. Lastly, Caring refers to students’ perceptions that other people in their learning environment (i.e., the instructor, other students) care about their learning and well-being. The MUSIC model was developed based on decades of motivation research in the social sciences. For example, Jones (2018, 2020) lists 28 theories that were used as evidence for different aspects of the MUSIC model, including, but not limited to, arousal theories (Berlyne, 1960; Duffy, 1957), attribution theory (Weiner, 2000), competence theories (Elliot & Dweck, 2005; White, 1959), expectancy-value theory (Wigfield & Eccles, 2000), flow theory (Csikszentmihalyi, 1990), goal theories (Ford, 1992; Locke & Latham, 2002), interest theories (Renninger & Hidi, 2015; Schraw & Lehman, 2001), self-determination theory (Ryan & Deci, 2020), self-theories of intelligence (Dweck, 1999), self-worth theories (Covington, 1992), and social cognitive theory (Bandura, 1986).
An overview of the MUSIC model is presented in Fig. 1, and it includes several elements that are important to the present study. The left side of the figure shows how variables that are external to students (e.g., instructor’s teaching strategies, ease of course, and environmental conditions) interact with those that are internal to students (e.g., cognition, affect, needs/desires, identity beliefs, personality characteristics, abilities). These external and internal variables then affect students’ perceptions of the motivational climate (i.e., MUSIC perceptions) within a course. For example, a Black female student may perceive that she can succeed in a CS course and find the course activities interesting because of a combination of external factors (e.g., she believes that the course will be relatively easy because the teacher incorporates cooperative learning and she usually succeeds in classes that use this approach) and internal factors (e.g., she generally enjoys CS activities; she believes that the knowledge she gained in prior classes will help her in this class). As a result (moving from left to right in Fig. 1), she has high perceptions of Success and Interest (and possibly eMpowerment, Usefulness, and Caring), which can lead to increased motivation and engagement/effort in the course, which may result in a high course grade. This outcome can cycle back (as depicted by the arrow from right to left at the bottom of Fig. 1) to further strengthen her ability beliefs and interest in CS. Thus, the MUSIC model is a dynamic cycle that explains how students’ motivations in a course can change over time, which can be measured in minutes or in months or years depending on the purpose of the measurement.
Motivation is defined as “the extent to which one intends to engage in an activity” (Jones, 2018, p. 5); and therefore, students can be motivated to do many things, but may not engage in all of them. This possibility is depicted by the “cost/benefit decisions” rectangle in Fig. 1. For example, students may not engage in courses even when they are motivated to do so because they are more motivated by other activities (e.g., other courses, extracurricular activities).
Although it is possible for students to be motivated based on only one of the MUSIC model perceptions (such as Success), students’ perceptions of the other MUSIC model components can also affect their motivation and the motivational climate in the course. For instance, despite high success expectations, a female student can perceive that the instructor is uncaring towards female students in the class and/or that cooperative learning activities are uninteresting or useless in helping her to learn the concepts. These low perceptions of caring, interest, and usefulness interact with her high success perceptions to affect her motivation and engagement.
Relationships between MUSIC perceptions and other variables
It is impossible to statistically test the entire MUSIC model as one whole model because of the large number of variables involved and the limits of statistical analyses. Therefore, researchers have tested various parts of the model by investigating relationships among some of the variables. In CS courses, researchers have documented relationships between students’ MUSIC perceptions and their broader CS beliefs and goals, but this research is only beginning (Jones et al., 2023b). More research has been conducted within engineering courses where researchers have documented relationships among students’ MUSIC perceptions and a variety of outcomes, such as course effort and some of the internal variables shown in Fig. 1 (e.g., engineering identification). For example, when students’ MUSIC perceptions of an engineering course were included in a structural equation model with course effort, engineering identification, and other variables, all of the MUSIC perceptions predicted engineering identification (Jones et al., 2014). However, only Empowerment, Usefulness, and Interest were significantly related to course effort, and only Success was associated with estimated course grade when all of the MUSIC variables were included in the model. Subsequent studies of students’ engineering courses also documented significant associations between at least some MUSIC perceptions and students’ engineering identification (in Jones et al., 2016, all MUSIC perceptions except Interest were significantly associated with engineering identification; in Tendhar et al., 2017, Success, Interest, and Caring were significantly associated with engineering identification). These findings demonstrate that while at least some MUSIC perceptions are related to course effort and engineering identification, the significant relationships can vary as the context changes and when different variables are included in the model. For these reasons, it is important for researchers to include all five MUSIC variables in studies to ensure that the study controls for a range of students’ course perceptions.
Differences across MUSIC perceptions by sex and race/ethnicity
In this section, we highlight some of the research that has documented differences between sexes and races/ethnicities for the variables included in this study. We begin with the Success variable because students’ success expectancies (a.k.a., self-efficacy; Bandura, 1986) in a course (i.e., students’ confidence in their ability to succeed in the course) has been shown to predict their persistence, achievement, and major/career choices in CS (Beyer et al., 2003) and engineering (Jones et al., 2016; Lent et al., 1986). Some studies documented that females’ self-efficacy in CS (Pirttinen et al., 2020; Tellhed et al., 2023), engineering (Besterfield-Sacre et al., 2001; Jones et al., 2010), and STEM domains (Chan, 2022; Young et al., 2018) are lower than those of males. Yet, other studies found no differences in engineering students’ self-efficacy beliefs by sex/gender (Concannon & Barrow, 2012; Lent et al., 2005; Mamaril et al., 2016). A more consistent finding related to Success is that Asian students tend to report lower self-efficacy in courses than students of other races/ethnicities because they have a higher fear of failure and anxiety, often due to parental pressure (D’Lima et al., 2014; Ma et al., 2018; Wilson et al., 2015; Zusho et al., 2005). Finally, in one study that investigated the differences in self-efficacy among minorities in STEM, researchers found that Hispanic students reported higher levels of self-efficacy compared to White and Asian students (Wilson et al., 2015).
Related to the Caring variable, a substantial number of studies have demonstrated that female engineering students perceive a lack of inclusion or acceptance, and greater discrimination or disrespect from faculty and fellow students, which is often reflected in their low caring ratings (Brainard & Carlin, 1998; Marra et al., 2009; Vogt et al., 2007). Women of color can face a “double bind” (Malcom et al., 1976) as they experience hostility and microaggressions within the environment that are associated with their identities as a woman and as a person of color (Blosser, 2020; Cross et al., 2021). Regardless of their sex, students of color in higher education can endure emotional injury as a result of racial stereotypes and microaggressions (McGee, 2016).
Related to the Effort variable, females have reported putting forth more effort than males in engineering courses and courses in similar fields (Hawks & Spade, 1998; Vogt et al., 2007). Similarly, Asian American students tend to spend more time on homework than White American students (Mau, 1997). Other researchers studying first-generation Latina women have found that when these women attend a predominantly White institution, they attribute their academic achievement and outcomes to the effort they put forth (Gutierrez-Serrano et al., 2022). They also reported that the high personal value those students placed on being in college, and the appreciation they received from their parents were two major contributing factors for their high motivation to improve performance, achieve, and persist.
Most studies have documented that male and female students’ engineering course grades are not significantly different (Hawks & Spade, 1998; Jones, 2010). However, Asian students tend to receive higher grades than non-Asian students (Chen & Stevenson, 1995), and racially minoritized students tend to earn lower grades than Asian and White students as undergraduates (Whitcomb & Singh, 2021). Although some studies showed that Black students were more likely to report higher self-perceived abilities and confidence in learning CS than White and Hispanic students (Wang et al., 2017), other studies documented that Black students had the lowest STEM persistence rates (Pedraza & Chen, 2022), and had lower achievement in certain STEM courses compared to their White peers (Tyson, 2011).
We chose to study a lower-level CS course (CS2) that is the second course in the software engineering track of courses required for all CS majors and minors. It is the minimum requirement for acceptance to the graduate programs in CS at the participating university. Approximately 70% of the students in this course tend to be engineering majors (Hooshangi et al., 2022), many of whom intend to major or minor in CS. Some students are required to take the course for a non-CS or non-engineering major such as Discrete Mathematics, Computational Neuroscience, or Computational Modeling and Data Analytics, while others may be simply exploring the CS discipline. Consequently, this course can help to not only retain CS and engineering students in their major, but also to attract non-CS and non-engineering students to consider a minor or master’s degree in CS.
Students can enroll in this CS2 course as early as their first semester at the university if they earn computer science (CS1) credit in high school. First-year CS and engineering students in the course have a high pass rate and a low DFW rate relative to other students (Hooshangi et al., 2022). To improve the overall course climate and to recruit and retain students in CS majors and minors, we studied the motivational climate for both the students who plan to continue in the discipline as a major or minor (CS students) and those who do not plan to continue (non-CS students). We were also interested in how student demographics were related to student motivation, which would assist us in targeting course designs that can broaden participation within CS.
Within the context of this lower-level, undergraduate CS2 course, our two research questions were the following: (1) To what extent do students’ achievement and perceptions of motivational climate, cost, ease, and effort vary by sex, race/ethnicity, or major? and (2) To what extent do the relationships between students’ achievement and perceptions of motivational climate, cost, and effort vary by sex, race/ethnicity, and major?
We provide this author positionality statement to increase the transparency and trustworthiness of the study (Darwin Holmes, 2020). We conducted the present study with the hopes that the findings could be used to improve the motivational climate within a CS2 course in ways that would motivate and engage all students in the course, and potentially retain and attract students to the CS discipline. Although it has been disappointing that we were unable to include all students in the present study (as explained in the following section), we believe that this study is a beginning step that can lead to other studies that do include all students. The second author was an “insider” (Herr & Anderson, 2005) because she was the instructor for the CS2 course included in the present study. The other authors were “outsiders” in that they were not associated with the CS2 course or department; however, they were employed by the same university. The first author identifies as a White male educational psychology professor, the second author as a White female CS professor, the third author as an Asian male research methodologist, and the fourth author as a White female educational psychology doctoral student. Although our perspectives are limited by our experiences and identities, we believe our diverse perspectives have strengthened this study in meaningful ways.
Participants and procedures
Participants were enrolled in a CS2 course within an engineering college at a large public university in the southeastern United States. A total of 981 students completed a survey assignment for the course and agreed to participate in the study. The survey assignment was administered in three different sections of a course that was offered in three consecutive years. Students completed the survey near the end of the course as part of their regular coursework and received full credit if they completed it, regardless of whether they agreed to participate in the study or not. This study was approved by the Institutional Review Board at the university (IRB #17-057) and students completed a consent form to participate. The number of students that consented to participate were as follows: 229 of 368 students (62.2%) consented to participate in Year 1 (Fall 2019), 395 of 550 students (71.8%) consented to participate in Year 2 (Fall 2020), and 357 of 521 students (68.5%) consented to participate in Year 3 (Spring 2021).
Students self-reported their sex as either male (n = 750, 76.5%), female (n = 225, 22.9%), or other (n = 6, 0.6%), and their race/ethnicity as White or Caucasian (not Hispanic; n = 454, 46.3%), Asian or Pacific Islander (n = 393, 40.1%), Black or African American (n = 40, 4.1%), Hispanic (n = 39, 4.0%), Native American (n = 1, 0.1%), more than one of the options provided (n = 45, 4.6%), or another race/ethnicity not provided as an option (n = 9, 0.9%). Most of the students were undergraduates, with 249 (25.4%) first year students, 506 (51.6%) sophomores, 193 (19.7%) juniors, 26 (2.7%) seniors, six master’s students (0.6%), and one doctoral student (0.1%). Students selected the major or minor that best described them using the following categories: CS major (n = 328, 33.4%), intending to be a CS major (n = 271, 27.6%), CS minor (n = 163, 16.6%), computational modeling and data analytics (CMDA) major (n = 158, 16.1%), mathematics major (n = 13, 1.3%), or another major (n = 48, 4.9%). Throughout this article, we sometimes combine some of these students to form two groups: “CS students” for those committed to the discipline (CS majors, intending to be a CS major, and CS minor) and “non-CS students” (CMDA majors, math majors, and other majors).
The six students who self-reported their sex as non-binary were not included in the analyses that involved comparing sexes because there were not enough students in this group to provide the statistical power needed for those analyses. Future studies are needed to better understand possible sex and gender identity differences within non-binary groups. We use the term “sex” in this study because we are referring to students’ sex assigned at birth. We did not collect data that would allow us to categorize students by their gender, which is a socially constructed psychological construct that refers to the roles, behaviors, activities, and attributes that are considered appropriate for women and men in society (American Psychological Association, 2014).
With respect to race/ethnicity, the one Native American student and the nine students who indicated that their race/ethnicity was not listed as an option were not included in the analyses that involved comparing races/ethnicities because there were not enough participants to provide adequate statistical power. In addition, we did not include the 45 students who reported their race/ethnicity as more than one of the options provided. It was inappropriate to place these 45 students in a group together because these students may not share similar racial/ethnic identities. Therefore, our analyses that involved comparisons for race/ethnicity included four groups of students that we refer to as: White (or Caucasian, not Hispanic), Asian (or Pacific Islander), Black (or African American), and Hispanic. We acknowledge that Asia and the Pacific Islands are composed of many different races/ethnicities and that there may be differences among students who we categorized in the Asian subgroup. However, we did not examine these differences because we did not provide students with options on the survey for different racial/ethnic subgroups within the Asian/Pacific Islander category. Future studies could examine whether it is possible to separate Asian/Pacific Islander students into meaningful subgroups based on the analyses conducted.
Course and instructor
The course was offered through the CS department and was a required course for all CS majors, all CS minors, all computational modeling and data analytics (CMDA) majors, and some math and neuroscience majors. The CS department was part of the College of Engineering, and many of the CS minors tended to major in one of the other 12 engineering disciplines offered at the university. The course was an intensive computer programming course that focused on data structures and software design. The course format varied over the years (e.g., face-to-face, online, a hybrid version that was partially face-to-face and partially online), but the categories of assignments and their percentage of the final course grade remained consistent across years. The structure of the course included content delivery, formative quizzes, short-form coding practice, weekly lab programming assignments, and five larger programming projects. The final project was designed and completed by students who worked together in teams.
The same instructor taught all three sections of this CS course. The instructor self-identified as a White female and had taught the course for four years prior to the semester in which the data were collected for the present study.
As part of the course survey, students rated items on a 6-point Likert-format scale with descriptors at each point (1 = Strongly disagree, 2 = Disagree, 3 = Somewhat disagree, 4 = Somewhat agree, 5 = Agree, 6 = Strongly agree). All of the measures described in this section (except for achievement) used the same Likert-format scale and the items on the scales were combined into one section and presented to each student in a different, random order to reduce order bias (Schwartz, 1999).
Motivational climate. Students’ perceptions of the motivational climate were assessed using the 26-item, College Student version of the MUSIC Model of Academic Motivation Inventory (Jones, 2012/2022). The MUSIC Inventory is composed of five scales that measure the extent to which students perceive that: they have control of their learning environment in the course (Empowerment scale, 5 items), the coursework is useful to their future (Usefulness scale, 5 items), they can succeed at the coursework (Success scale, 4 items), the instructional methods and coursework are interesting (Interest scale, 6 items), and the instructor cares about whether the student succeeds in the coursework and cares about the student’s well-being (Caring scale, 6 items). Example items include the following: “I have the freedom to complete the coursework my own way” (Empowerment), “In general, the coursework is useful to me” (Usefulness), “I am confident that I can succeed in the coursework” (Success), “The coursework is interesting to me” (Interest), and “The instructor cares about how well I do in this course” (Caring). The User Guide (Jones, 2012/2022) includes the complete inventory, administration instructions, and validity information. The internal consistency reliability of the scale scores was very good in other studies of undergraduate students (α ranged from 0.82 to 0.87 in Chittum et al., 2019, and from 0.91 to 0.96 in Jones & Skaggs, 2016).
Ease of the course. The three-item Ease of Course scale (Jones et al., 2021) was used to assess the extent to which students perceived the course to be easy. The items are: (1) “This course is very easy for me”, (2) “I don’t need to work my hardest to get a high grade in this course”, and (3) “In this course, I can get the grade I want with very little effort.” This scale has demonstrated acceptable internal consistency reliability in other studies with undergraduate students (α = 0.73, Jones et al., 2021; α = 0.82, Jones et al., 2022b).
Effort in the course. The four-item Course Effort scale (Jones, 2019) was used to measure the amount of effort that students believe that they are putting forth in a course. An example item is, “In this course, I put forth my maximum effort” (the complete scale is available at Jones, 2012/2022). The internal consistency reliability for the scores was good in other undergraduate courses (α = 0.93, 0.87, 0.94, 0.83, and 0.79 in Jones, 2019; α = 0.87 in Jones et al., 2021).
Cost of engaging in the course. A three-item Time Cost scale was used to measure the extent to which students did not have the time to put into the course. This scale was presented in Jones et al., (2021, α = 0.86) and was based on a scale developed by Kosovich et al. (2015). The items in the scale were (1) “This course requires too much time”, (2) “Because of other things that I do, I don’t have time to put into this course”, and (3) “I’m unable to put in the time needed to do well in this course.”
Achievement in the course. Students’ achievement was measured with their final end-of-course grade. The grade was calculated similarly for all three courses with approximately one-third of the grade based on exams, one-third based on projects, and one-third based on labs, homework, and participation. The grades are reported in this study as percentages that could range from 0 to 100%.
We used SPSS (version 27) to compute the descriptive statistics, correlations, and Cronbach’s alpha values for all of the study variables. We used SAS (Version 9.4) for all of the other statistical analyses (i.e., MANOVAs, ANOVAs, path analyses). We set the significance level at 0.05 for all of the analyses.
To answer our first research question (To what extent do students’ achievement and perceptions of motivational climate, cost, ease, and effort vary by sex, race/ethnicity, or major?), we conducted three one-way MANOVAs to compare the means between groups. The null hypothesis in a one-way MANOVA states that there is no difference in the means of all dependent variables across groups, which is evaluated by Wilk’s Λ. When this null hypothesis was rejected (as indicated by Wilk’s Λ), we identified the source of the difference(s) by conducting a one-way ANOVA for each dependent variable. The null hypothesis in a one-way ANOVA states that there is no difference in the means of the dependent variable across groups, which is evaluated by the F test. When this null hypothesis was rejected, we identified the source of difference(s) by conducting multiple comparisons in which we used the Tukey–Kramer adjustment to account for the unbalanced cell sizes and to control the overall Type I error rate (Kramer, 1956; Tukey, 1953).
To answer our second research question (To what extent do the relationships between students’ achievement and perceptions of motivational climate, cost, and effort vary by sex, race/ethnicity, and major?), we conducted a path analysis using the model shown in Fig. 2. We included a path from Success to Grade in the model because although students who believe they can succeed may put forth more effort (as depicted by the arrow in Fig. 2 from Success to Effort), students’ success expectancies may also directly affect their grades without increasing or decreasing their effort. For example, this may occur when students have high success expectancies but do not need to put forth high effort because they believe the coursework is easy, perhaps because they already have many of the abilities needed to attain a high grade. The covariances between the five MUSIC variables and the cost variable were all freely estimated because they are exogenous variables. We fit the model using students who did not believe that the course was easy because when students believe that the course is easy, they do not need to put forth effort (Jones et al., 2021). Students who rated course ease less than 4.0 were identified as those who did not believe that the course was easy (n = 801; 82% of the participants). For parameter estimation, we used the MLSB/MLM estimation method (i.e., maximum likelihood parameter estimates with standard errors and a mean-adjusted Chi-square test statistic) because it can accommodate data from nonnormal distributions and generate scaled fit indices (Satorra & Bentler, 1994).
To compare the results across sex (females and males), race/ethnicity (White, Asian, Black, and Hispanic), and major (CS majors, CS minors, CMDA major, and Other majors), we conducted multi-group analyses. Within each comparison (e.g., comparison between sexes), we first fit an unrestricted model in which the path model in Fig. 2 was fitted to all the groups (e.g., the female group and the male group) simultaneously and all the parameters were freely estimated. Then, we fit a restricted model, in which eight path parameters (i.e., the five MUSIC variables to Effort, Cost to Effort, Success to Grade, and Effort to Grade) were equated between the groups (e.g., the female group and the male group) and all other parameters were freely estimated. Finally, we computed the scaled Chi-square difference (Satorra & Bentler, 2010) to compare the unrestricted and restricted models.
Descriptive statistics, correlations, and reliabilities
We provide the descriptive statistics for all of the study variables in Table 1. The means for students’ MUSIC perceptions were fairly high and ranged from 4.3 to 5.2. The mean for Cost was about average (M = 3.4), the mean was below average for Ease (M = 2.8), and the mean was fairly high for Effort (M = 4.8). The grades ranged from failing (54.4%) to almost perfect (99.3%) with a mean value of 87.9% (SD = 6.8). The skewness and kurtosis values were acceptable for all of the variables (|< 2|) except for Usefulness and Caring, which had slightly higher kurtosis values of 3.02 and 2.23, respectively, because some students rated both of these constructs with the highest scale value.
We determined that the internal consistency reliability, as assessed by Cronbach’s alpha (α), was good for all of the scales (George & Mallery, 2019) and ranged from 0.85 to 0.90 for the MUSIC Inventory scales. The MUSIC variables were correlated moderately (r ranged from 0.45 to 0.67), which is typical for undergraduate samples of students (e.g., Jones & Skaggs, 2016; Jones et al., 2021).
Results for Research Question 1
We conducted three one-way MANOVAs to answer Research Question 1 to determine the extent to which students’ achievement and perceptions of motivational climate, cost, ease, and effort varied by sex, race/ethnicity, or major. All three of the MANOVA analyses were statistically significant (for sex, Wilk’s Λ = 0.910, p < 0.001; for race/ethnicity, Wilk’s Λ = 0.888, p < 0.001; for major, Wilk’s Λ = 0.830, p < 0.001). Given the significant results from all three MANOVAs, we examined the one-way ANOVA results for each dependent variable and presented the results in Table 2. Compared to males, females reported lower values for Empowerment, Success, and Ease, and higher values for Cost and Effort. These differences (and those for race/ethnicity and major) are explained in detail in the “Discussion” section.
Table 3 includes the results of the one-way ANOVAs by race/ethnicity and shows the comparison by variable for White and Caucasian (not Hispanic), Asian or Pacific Islander, Black or African American, and Hispanic students. We documented differences between the four groups for Empowerment, Success, Interest, Cost, Ease, and Grade. Hispanic students reported higher Empowerment than White and Black students. White students had higher Success expectancies than Asian students. Asian and Hispanic students reported higher Interest than White and Black students. Asian students reported higher values for Cost than White students, and lower values for Ease than White students. Black students obtained lower grades than White and Asian students.
To check for a possible interaction between sex and race/ethnicity, we also conducted a two-way MANOVA with sex, race/ethnicity, and their interaction. The MANOVA test indicated that the interaction effect between sex and race/ethnicity was not significant (Wilk’s Λ = 0.964, p > 0.05).
The results of the one-way ANOVAs by academic major are provided in Table 4. We combined current CS majors with students intending to become CS majors because we assumed that they would have similar motivations and goals. We also created an “Other” group that consisted of all students who were neither CS majors, CS minors, nor CMDA majors. We identified differences between the groups for all of the variables except Effort, which was fairly high for all groups (M ranged from 4.6 to 4.8). The grades were also similar across groups except that the CMDA majors earned lower grades than the other three groups. As a trend, CS majors reported higher MUSIC perceptions than the other groups, although they were similar to CS minors for Success, Interest, and Caring. The CS minors reported values similar to the non-CS students (i.e., the CMDA majors and the “Other” group) for many of the variables. The two groups that comprised the non-CS students (i.e., the CMDA majors and the “Other” group) were not statistically significantly different on any of the variables except that the CMDA majors earned lower grades. Table 5 presents a summary of all the findings related to Research Question 1.
Results for Research Question 2
We began answering our second research question (To what extent do the relationships between students’ achievement and perceptions of motivational climate, cost, and effort vary by sex, race/ethnicity, and major?) by fitting the model in Fig. 2 to data from the 801 students who did not perceive the course to be easy. The standardized path estimates are provided in Fig. 3.
The commonly reported fit indices are presented in Table 6 in the “All” column and indicated good to excellent fit of the model because the SRMR and RMSEA were less than 0.08, and the CFI and NNFI/TLI were greater than or equal to 0.95 (Hu & Bentler, 1999; Kline, 2013). The MUSIC variables and cost explained 16% of the variance in Effort, whereas Effort and Success explained 19% of the variance in Grades.
For all three multi-group analyses (i.e., sex, race/ethnicity, and major), the fit indices indicated adequate fit of all the unrestricted models, and the scaled Chi-square difference tests indicated no significant difference between the unrestricted and restricted models. These findings indicate that the model in Fig. 2 does not vary by sex, race/ethnicity, or major.
Table 7 provides the unstandardized and standardized estimates for the path parameters in the model shown in Fig. 2 and for the restricted models for sex, race/ethnicity, and major. The significant path estimates in all three restricted models (i.e., for sex, race/ethnicity, and major) were the same paths as those that were significant in the model that included all students: Empowerment to Effort (except for the race/ethnicity model in which case it is close to significance at p = 0.066), Usefulness to Effort, Interest to Effort, Effort to Grade, and Success to Grade. All of these significant path estimates were positive as anticipated, except that the path between Empowerment and Effort was negative. The path estimates from Success to Effort, Caring to Effort, and Cost to Effort were not statistically significant.
RQ1: variations by sex, race/ethnicity, and major
Our first research question asked: In an undergraduate CS course, to what extent do students’ achievement and perceptions of motivational climate, cost, ease, and effort vary by sex, race/ethnicity, or major? The answer to this question is important because if differences between groups exist, they may create a motivational climate that is unmotivating or unengaging to some students, which could lead to course conditions that are inequitable or not inclusive of all students. It is important to note that some of the differences we documented are relatively small; therefore, we do not want to exaggerate the possible importance of those findings. For example, although Asian students reported statistically significantly higher scores for Interest than White students, the practical significance may not be overly meaningful given that the means only differed by 0.2 on the six-point scale (the mean was 4.5 for Asian students and 4.3 for White students). Therefore, we caution readers not to make hasty generalizations from all of these findings, and rather, to consider whether the differences are practically meaningful. As an example of findings that are fairly substantial and may have practical significance, consider the differences in Grades. The Black students obtained significantly lower grades (M = 83.6%) than the White (M = 88.1%) and Asian (M = 88.0%) students. Understanding the sources of these differences could lead to solutions that could help Black students to obtain higher grades.
We first examined the extent to which females and males perceived the motivational climate (i.e., the MUSIC constructs) in the course differently. We found that both sexes had similar perceptions of Usefulness, Interest, and Caring. This finding indicates that there are many similarities in how the overall motivational climate in this course is perceived by females and males. The differences in MUSIC perceptions were that females reported lower values for Empowerment and Success. Other differences were that females reported the course was harder, they put forth more effort, and the time cost was higher than males. The final grade earned by females was not statistically significantly different from males, which is consistent with the findings of previous studies (Huang & Brainard, 2001; Takahira et al., 1998).
The fact that females reported lower Success expectancies than males indicates that females were less likely to perceive that they could succeed in the course. These findings are consistent with the results of other studies that have documented lower Success expectancies (self-efficacy) for females (e.g., Besterfield-Sacre et al., 2001; Jones et al., 2010; Marshman et al., 2018; Pirttinen et al., 2020; Young et al., 2018). Combined with the results for Ease, Effort, and Cost, these findings provide a picture that females do not expect to do as well as males because they find the course more difficult, which requires them to put forth more effort and takes more time. These struggles could lead to many undesirable outcomes, such as having less time available for other coursework and non-academic activities, and could cause females to question whether they are “good enough” to succeed in CS and pursue a CS career (Margolis et al., 2000). These findings could also shed some light on the types of psychological struggles that female students are more likely to endure, such as anxiety and depression, which can drastically impact their persistence (Seymour, 1995).
The fact that the lead instructor of the courses was a woman may have had a positive effect on women’s perceptions because role models can influence female students’ attitudes towards engineering (Casad et al., 2018; Wallace & Haines, 2004) and other STEM fields (Stout et al., 2011). Future studies could compare how students’ perceptions of the motivational climate differ in courses taught by men.
Differences by race/ethnicity varied somewhat across the five MUSIC perceptions, but we identified a few patterns that we present in this section in the order of the size of the participant groups in this study from largest to smallest (i.e., White, Asian, Black, Hispanic). White students reported higher values than Asian students for Success and Ease, and lower values than Asian students for cost. White students also obtained higher grades than the Black students. Finally, the White students reported lower values on Empowerment than Hispanic students and lower values on Interest than the Asian and Hispanic students. We discuss all of these findings within the context of the other race/ethnic groups in the following paragraphs.
The findings that Asian students reported lower Success expectances and higher time Cost than White students are consistent with other studies in which Asian students have reported lower Success expectations than other students (D’Lima et al., 2014; Wilson et al., 2015). For instance, in studies of college students, Asian American students displayed significantly higher levels of anxiety than their Anglo-American counterparts because they feared that their failure could result in severe consequences (Steinberg et al., 1992; Zusho et al., 2005). Asian American students can feel pressure from their parents’ high academic standards (D’Lima et al., 2014; Steinberg et al., 1992). In some studies, this increased pressure and fear of failure has led Asian American students to put forth increased effort and achieve higher than other students (Ma et al., 2018). However, that was not the case in the present study. Asian students put forth the same amount of effort as the White, Black, and Hispanic students, and they earned the same final grades as the White and Hispanic students.
One possible reason that Asian students may report higher time Cost than White students is that English is the second language for some of these students; and therefore, it may take them more time to comprehend and produce materials in English. Campbell and Li (2007) noted that in New Zealand, Asian students faced language barriers that caused them to spend a lot of time preparing for class, looking up new words in the dictionary, and improving their English language skills. We did not ask students if English was their second language, but future studies could include this variable to better understand how language may be a barrier to students’ success. It is also possible that there are other differences between the Asian and White students that led the Asian students to report higher time Cost, such as they enrolled in more courses or were engaged in more extracurricular activities. Future students could also examine these possibilities.
Black students tended to respond similarly to students in one or more of the other racial/ethnic groups. The exceptions were that Black students reported lower Empowerment than Hispanic students, and lower Interest than Asian and Hispanic students; and Black students obtained lower grades than White and Asian students. Black students were not statistically different from White students on any of the motivation-related variables, they only differed in that their grades were lower. This finding is consistent with previous studies which reported that Black students as a racially minoritized group tended to have lower grades (Whitcomb & Singh, 2021) and lower achievement than their White and Asian peers (Tyson, 2011). One reason for this trend could be that, on average, Black students did not have the prerequisite skills to do as well in the course because K-12 schools that serve high populations of Black students lack accessibility to quality and advanced learning experiences in and exposure to STEM (London et al., 2021).
One consistent finding was that the perceptions of the Hispanic students were as high or higher than any of the other three races/ethnicities for all five MUSIC variables. Hispanic students reported higher Empowerment than White and Black students and higher Interest than White and Black students. Hispanic students reported values for Cost, Ease, and Effort that were similar to all three of the other racial/ethnic groups, and their grades were not statistically different from the other three groups. These findings may be attributed to a variety of factors. Hispanic students may have reported higher values for the five MUSIC variables because, compared to White and Asian students, Hispanic students in other studies have reported more positive perceptions of CS, have been more likely to report that they would have a job in which they would need to know CS (i.e., CS is useful to them), and have been more likely than White and Black students to report that they were very interested in learning CS (Wang et al., 2017).
The CS majors reported higher values than the non-CS students on all of the MUSIC variables except Empowerment (Empowerment was the same for CS majors and the “Other” group), whereas the mean values for the CS minors tended to be between the CS majors and the non-CS students. The CS majors (and the CS minors to a lesser extent) had chosen the CS field as a focus of their studies and likely have an affinity for programming and an interest in software development. All CMDA students were required to enroll in this course and some of them may be interested in some aspects of the course (e.g., modeling and analysis) but not other aspects (e.g., software development). Compared to CS students, the CMDA students may have less experience and confidence in their programming skills. Because this course is focused on building programs, it is likely to be more appealing to CS students who tend to be interested in designing, building, and creating.
Typically, the CMDA majors reported lower values for the MUSIC variables than the other students and they received lower grades than the other students. Because they reported putting forth a similar amount of effort as the other groups, it is possible that they had less prior knowledge and skills related to the topics in this course. Because the CMDA students are not in the College of Engineering and the requirements for admittance into the CMDA major is less rigorous, these students may have a history of lower academic achievement overall compared to the engineering students. In addition, the CMDA students may have had less programming experience and been less prepared to manage a CS course with a heavy workload and firm deadlines. Furthermore, although some CMDA majors had completed a previous course that taught the Java knowledge needed in the current course, many CMDA majors enrolled in a 2-course sequence in Python that was designed to support their academic path, but may not have prepared them adequately for the present course. Consequently, some CMDA students may need a stronger support system and preparation than was available.
RQ2: relationships between MUSIC perceptions, cost, effort, and achievement
Our second primary research question was the following: In an undergraduate CS course, to what extent do the relationships between students’ achievement and perceptions of motivational climate, cost, and effort vary by sex, race/ethnicity, and major? We tested the model in Fig. 2 and the results provided evidence that the model was valid for use in this CS course. That is, students’ MUSIC perceptions (with the exception of Empowerment, as discussed in the following paragraph) and cost were positively related to their effort and their effort was positively related to their achievement. These findings add to the evidence produced in other studies of engineering courses that demonstrate that students’ MUSIC perceptions are related to their effort and achievement (e.g., Jones et al., 2014).
Although all of the MUSIC components were significantly positively correlated with Effort (see Table 1), when all five MUSIC components were included in the model shown in Fig. 2, only Usefulness and Interest were positively significant predictors of Effort, whereas Empowerment was negatively predictive of Effort. These findings occurred because the MUSIC components are correlated, and only when they are all included in the model at the same time is it possible to determine which variables are the best predictors of Effort. Usefulness and Interest were also found to be significant predictors of course effort in a study of students in an engineering course that included all of the MUSIC variables in the model at once (Jones et al., 2014). These findings in engineering courses are somewhat different from those in studies of undergraduate students in non-engineering courses. As examples of studies that included all of the MUSIC variables in the model, Interest and Caring were significant predictors of Effort in an undergraduate geography course (Jones et al., 2021), and Success and Interest (and to a lesser extent, Empowerment and Caring) were significant predictors in undergraduate language courses (Jones et al., 2023; Li et al., 2022). Future research could examine whether Usefulness and Interest are the most significant predictors of Effort in all types of engineering courses or if the predictors vary by engineering course.
We are not sure why Empowerment negatively predicted Effort in the path analyses (except in the race/ethnicity model). One plausible explanation is that the part of the variance for the Empowerment construct that predicts Effort overlaps with the Usefulness and Interest constructs, and the remaining part of the variance for the Empowerment construct is negatively related to Effort. Empowerment refers to the amount of autonomy and control that students have within the course. Having some autonomy may increase students’ perceptions of interest in and usefulness of the course topics and activities. However, this autonomy may also allow students to use strategies that require less Effort, in which case increased Empowerment could lead to decreased Effort. These speculations need to be examined more thoroughly in future research to understand the role of Empowerment in relation to Effort and the other MUSIC variables.
The relationships between the variables depicted in Fig. 2 were consistent across sex, race/ethnicity, and major, which provides evidence that the part of the MUSIC model shown in Fig. 2 can be used with a variety of CS and non-CS students. This adds to a growing body of research supporting the relationships between variables in the MUSIC Model of Motivation Theory (e.g., Jones, 2010, 2019; Jones et al., 2021). The fact that time Cost was not a significant predictor of Effort is inconsistent with the MUSIC model theory because Cost is considered to be one of the variables that can affect students’ motivation and engagement in a course (see Fig. 1). For the students in this course, other factors are more important than time Cost in students’ decisions to put forth effort. It may be that students view this course as one of their first important courses in CS; and therefore, they want to put forth the effort needed to get a high grade regardless of the time Cost involved.
The findings of this study must be interpreted within the study limitations. First, the course included in this study may not be representative of all CS courses. However, courses similar to this course are offered at most public universities that offer engineering and/or CS degrees. Second, we were unable to include students who self-identified as more than one race/ethnicity option provided (because we could not assume that these students would form a homogenous group) or as another race/ethnicity not provided (because they did not report their race/ethnicity). Furthermore, we were unable to include any students who were either not White or Asian, or non-binary in the path analyses because there were not enough students in these groups to provide the statistical power needed to conduct robust analyses. Future studies could include enough students from a variety of races/ethnicities and sexes/genders to permit statistical comparisons across these groups. Third, because students were surveyed near the end of the course, students who dropped out of the course earlier in the semester were not included in the study. It is possible that students who dropped out of the course had different course perceptions and grades than those who completed the course. Finally, students in Year 2 and Year 3 completed the surveys during the COVID-19 pandemic, which may have caused undue stress on the instructor and students, thus limiting the generalizability of these findings to courses during non-pandemic years.
Implications and conclusion
One implication of this study for instructors is that females, Asian students, and non-CS students may need more support, or different types of support, to help them believe that they can succeed in CS courses. On average, these students were less confident in their abilities to succeed in the course and were more likely to report that they did not have the time needed to do well in the course. Because the female and Asian students earned grades that were statistically similar to the other students, there does not seem to be a problem with respect to how much students are learning and achieving, except for the CMDA students. However, having lower success expectancies could lead these students to question whether they want to persist in CS-related disciplines (Bandura, 1986). Asian students may have external pressure from parents to motivate them to continue in CS-related disciplines (Ma et al., 2018), and non-CS students may not need to have high CS success expectancies to succeed in their major. However, female CS and engineering students may question their CS ability and begin to think that CS is not a good career fit for them, which could lead to them losing interest or disidentifying with CS (Jones et al., 2015; Margolis et al., 2000). Thus, having a lower course self-efficacy could be especially detrimental to females. In addition, if non-CS majors (e.g., other engineering students) have lower success expectancies, they may be less likely to consider pursuing a minor or graduate degree in CS. And, even with parental or other external pressures to continue in CS, Asian students may benefit from higher success expectancies. By being aware of students’ lower success expectancies, instructors may be more likely to identify and remediate situations or issues in ways that can help these students believe they can succeed, resulting in a more equitable and inclusive learning environment. Further research could examine whether increasing students’ success expectancies can lead to positive outcomes besides increased achievement.
A second implication for instructors is that it may be possible to increase students’ effort and achievement if they increase students’ MUSIC perceptions. Studies indicate that students (regardless of their sex/gender, race/ethnicity, or background) are likely to take more CS courses when they have excellent instructors who use pedagogically sound practices (Beyer, 2014). In the present study, students’ perceptions of Usefulness and Interest were the most significant predictors of their Effort, which predicted their grades. In addition, Success predicted course grades directly. Although correlation does not imply causation, it is reasonable to suspect (until experimental studies can be conducted) that increasing students’ Usefulness perceptions and Interest in the course content and activities could lead to increased effort. Or, considered from a different vantage point, poor course designs can decrease interest in science and mathematics courses, especially for women (Seymour, 1995). Many different strategies are available to instructors to increase usefulness perceptions (e.g., connecting the course activities to the “real-world” and students’ future goals [Spence et al., 2022]) and situational interest (e.g., incorporating novelty, piquing students’ curiosity, stimulating emotional arousal, varying activities [see Jones, 2018]). A variety of strategies can be used by instructors to increase students’ success expectancies, including those that help them to succeed (e.g., divide long assignments into shorter, more manageable assignments) and those that provide supportive verbal feedback (Bandura, 1986). Another benefit to increasing students’ MUSIC perceptions is that these perceptions are highly predictive of the instructor and course ratings provided by students (Jones et al., 2022b; Wilkins et al., 2021), and these ratings are often important to faculty because they can affect personnel decisions, such as annual reviews, merit raises, and promotion decisions (Stroebe, 2020).
Compared to CS majors, non-CS students reported that the course was less useful and interesting to them, which could lead to less effort in the course given that Usefulness and Interest were significant predictors of Effort. It may be possible to increase students’ effort and achievement (especially for the CMDA majors) by identifying teaching strategies that could be used to improve their perceptions of Usefulness and Interest. Doing so may also increase the likelihood that they would choose CS as a minor, as students studying other disciplines who perceive CS to be useful to their disciplines are more likely to be attracted to CS (Carter, 2006).
A final implication is that CS instructors can assess students’ MUSIC perceptions of the motivational climate using the MUSIC Inventory. In the present study, we used the MUSIC Inventory results to examine the motivational climate within a class and within subgroups (e.g., females vs. males). However, recent studies have demonstrated that these data can also be used to assess individual students’ perceptions, which could provide further insight into the variety of perceptions within the class (Jones et al., 2022a). Analyzing data at the individual level could help instructors to provide more equitable and inclusive learning experiences to all students. Data can also be collected over time within the same course to provide instructors with feedback about how different course activities affect the motivational climate.
In sum, the results of this study are potentially useful to instructors, course designers, and administrators interested in creating equitable motivational climates within courses. In this study, a positive motivational climate was associated with increased student effort and achievement across sex, race/ethnicity, and academic major. By attending to the various aspects of the motivational climate (i.e., students’ perceptions of empowerment, usefulness, success, interest, and caring), instructors can also increase the likelihood that their course is equitable and inclusive of all students. We hope that this study will inform future studies and that CS and engineering departments will take these findings into account in their efforts to expand and diversify their fields and attract and retain women and racially minoritized students in STEM fields.
Availability of data and materials
The data are available upon request by contacting the first author.
American Psychological Association. (2023). Understanding transgender people, gender identity and gender expression. Retrieved October 26, 2023, from https://www.apa.org/topics/lgbtq/transgender-people-gender-identity-gender-expression
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
Berlyne, D. E. (1960). Conflict, arousal, and curiosity. McGraw-Hill.
Besterfield-Sacre, M. B., Moreno, M., Shuman, L. J., & Atman, C. J. (2001). Gender and ethnicity differences in freshman engineering student attitudes: A cross-institutional study. Journal of Engineering Education, 90(4), 477–490. https://doi.org/10.1002/j.2168-9830.2001.tb00629.x
Beyer, S. (2014). Why are women underrepresented in computer science? Gender differences in stereotypes, self-efficacy, values, and interests and predictors of future CS course-taking and grades. Computer Science Education, 24(2–3), 153–192. https://doi.org/10.1080/08993408.2014.963363
Beyer, S., Rynes, K., Perrault, J., Hay, K., & Haller, S. (2003). Gender differences in computer science students. In Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education (pp. 49–53). https://doi.org/10.1145/611892.611930
Blosser, E. (2020). An examination of Black women’s experiences in undergraduate engineering on a primarily white campus: Considering institutional strategies for change. Journal of Engineering Education, 109(1), 52–71. https://doi.org/10.1002/jee.20304
Brainard, S., & Carlin, L. (1998). A six-year longitudinal study of undergraduate women in engineering and science. Journal of Engineering Education, 87(4), 369–375. https://doi.org/10.1002/j.2168-9830.1998.tb00367.x
Campbell, J., & Li, M. (2007). Asian students’ voices: An empirical study of Asian students’ learning experiences at a New Zealand University. Journal of Studies in International Education, 12(4), 375–396. https://doi.org/10.1177/1028315307299422
Carter, L. (2006). Why students with an apparent aptitude for computer science don’t choose to major in computer science [Paper presentation]. 37th SIGCSE Technical Symposium on Computer Science Education, Houston, TX, United States. https://doi.org/10.1145/1124706.1121352
Casad, B. J., Oyler, D. L., Sullivan, E. T., McClellan, E. M., Tierney, D. N., Anderson, D. A., Greeley, P. A., Fague, M. A., & Flammang, B. J. (2018). Psychological interventions to improve gender and racial equality in STEM. Group Processes & Intergroup Relations, 21(5), 767–787. https://doi.org/10.1177/1368430218767034
Chan, R. C. H. (2022). A social cognitive perspective on gender disparities in self-efficacy, interest, and aspirations in science, technology, engineering, and mathematics (STEM): The influence of cultural and gender norms. International Journal of STEM Education. https://doi.org/10.1186/s40594-022-00352-0
Chen, C., & Stevenson, H. W. (1995). Motivation and mathematics achievement: A comparative study of Asian-American, Caucasian-American, and East Asian high school students. Child Development, 66, 1213–1234.
Chittum, J. R., Jones, B. D., & Carter, D. M. (2019). A person-centered investigation of patterns in college students’ perceptions of motivation in a course. Learning and Individual Differences, 69, 94–107. https://doi.org/10.1016/j.lindif.2018.11.007
Christenson, S. L., Reschly, A. L., & Wylie, C. (Eds.). (2012). Handbook of research on student engagement. Springer.
Concannon, J. P., & Barrow, L. H. (2012). A reanalysis of engineering majors’ self-efficacy beliefs. Journal of Science Education and Technology, 21(6), 742–753. https://doi.org/10.1007/s10956-011-9362-8
Covington, M. V. (1992). Making the grade: A self-worth perspective on motivation and school reform. Cambridge University Press.
Cross, K. J., Mendenhall, R., Clancy, K. B. H., Imoukhuede, P., & Amos, J. (2021). The pieces of me: The double bind of race and gender in engineering. Journal of Women and Minorities in Science and Engineering, 27(3), 79–105. https://doi.org/10.1615/JWomenMinorScienEng.2021034902
Csikszentmihalyi, M. (1990). Flow: The psychology of optimal experience. Harper Perennial.
Darwin Holmes, A. G. (2020). Researcher positionality—A consideration of its influence and place in qualitative research—A new researcher guide. Shanlax International Journal of Education, 8(4), 1–10. https://doi.org/10.34293/education.v8i4.3232
D’Lima, G. M., Winsler, A., & Kitsantas, A. (2014). Ethnic and gender differences in first-year college students’ goal orientation, self-efficacy, and extrinsic and intrinsic motivation. The Journal of Educational Research, 107(5), 341–356. https://doi.org/10.1080/00220671.2013.823366
Duffy, E. (1957). The psychological significance of the concept of “arousal” or “activation.” The Psychological Review, 64(5), 265–275. https://doi.org/10.1037/h0048837
Dweck, C. S. (1999). Self-theories: Their role in motivation, personality, and development. Psychology Press.
Elliot, A. J., & Dweck, C. S. (2005). Competence and motivation: Competence as the core of achievement motivation. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of competence and motivation (pp. 3–12). Guilford.
Ford, M. (1992). Motivating humans: Goals, emotions, and personal agency beliefs. Sage.
George, D., & Mallery, P. (2019). IBM SPSS statistics 25 step by step (15th ed.). Taylor & Francis.
Gutierrez-Serrano, G., Romo, L. F., & Chagolla, D. (2022). Latina first-generation college students’ motivation to persist: An attribution theory and self-determination theory perspective. Journal of Latinos and Education. https://doi.org/10.1080/15348431.2022.2096611
Hall, S., Jones, B. D., Amelink, C., & Hu, D. (2013). Educational innovation in the design of an online nuclear engineering curriculum. The Journal of Effective Teaching, 13(2), 58–72.
Harper, R. P., Weston, T. J., & Seymour, E. (2019). Student responses to problematic STEM teaching methods. In E. Seymour & A. B. Hunter (Eds.), Talking about leaving revisited (pp. 149–195). Springer. https://doi.org/10.1007/978-3-030-25304-2_6
Hawks, B. K., & Spade, J. Z. (1998). Women and men engineering students: Anticipation of family and work roles. Journal of Engineering Education, 87(3), 249–256. https://doi.org/10.1002/j.2168-9830.1998.tb00351.x
Herr, K., & Anderson, G. (2005). The action research dissertation. Sage.
Hooshangi, S., Ellis, M., & Edwards, S. H. (2022, February). Factors influencing student performance and persistence in CS2. In SIGSE 2022: Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V.1 (pp. 286–292). Association for Computing Machinery. https://doi.org/10.1145/3478431.3499272
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
Huang, P. M., & Brainard, S. G. (2001). Identifying determinants of academic self-confidence among science, math, engineering, and technology students. Journal of Women and Minorities in Science and Engineering, 7(4), 315–337.
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2017). Making connections: Replicating and extending the utility value intervention in the classroom. Journal of Educational Psychology, 109(3), 387–404. https://doi.org/10.1037/edu0000146
Jones, B. D. (2009). Motivating students to engage in learning: The MUSIC model of academic motivation. International Journal of Teaching and Learning in Higher Education, 21(2), 272–285.
Jones, B. D. (2010). An examination of motivation model components in face-to-face and online instruction. Electronic Journal of Research in Educational Psychology, 8(3), 915–944.
Jones, B. D. (2012/2022). User guide for assessing the components of the MUSIC® Model of Motivation. http://www.theMUSICmodel.com
Jones, B. D. (2018). Motivating students by design: Practical strategies for professors (2nd ed.). CreateSpace.
Jones, B. D. (2019). Testing the MUSIC Model of Motivation Theory: Relationships between students’ perceptions, engagement, and overall ratings. The Canadian Journal for the Scholarship of Teaching and Learning, 10(3), 1–15. https://doi.org/10.5206/cjsotl-rcacea.2019.3.9471
Jones, B. D. (2020). Motivating and engaging students using educational technologies. In M. J. Bishop, E. Boling, J. Elen, & V. Svihla (Eds.), Handbook of research in educational communications and technology learning design (5th ed., pp. 9–35). Springer. https://doi.org/10.1007/978-3-030-36119-8_2
Jones, B. D., Epler, C. M., Mokri, P., Bryant, L. H., & Paretti, M. C. (2013). The effects of a collaborative problem-based learning experience on students’ motivation in engineering capstone courses. Interdisciplinary Journal of Problem-based Learning, 7(2), 34–71. https://doi.org/10.7771/1541-5015.1344
Jones, B. D., Fenerci-Soysal, H., & Wilkins, J. L. M. (2022a). Measuring the motivational climate in an online course: A case study using an online survey tool to promote data-driven decisions. Project Leadership & Society, 3, 100046. https://doi.org/10.1016/j.plas.2022.100046
Jones, B. D., Khajavy, G. H., Li, M., Mohamed, H. E., & Reilly, P. (2023a). Examining the cross-cultural validity of the MUSIC Model of Academic Motivation Inventory in English language courses. SAGE Open, 13(1), 1–16. https://doi.org/10.1177/21582440231156583
Jones, B. D., Krost, K., & Jones, M. W. (2021). Relationships between students’ course perceptions, effort, and achievement in an online course. Computers and Education Open, 2, 100051. https://doi.org/10.1016/j.caeo.2021.100051
Jones, B. D., Miyazaki, Y., Li, M., & Biscotte, S. (2022b). Motivational climate predicts student evaluations of teaching: Relationships between students’ course perceptions, ease of course, and evaluations of teaching. AERA Open, 8(1), 1–17. https://doi.org/10.1177/23328584211073167
Jones, B. D., Osborne, J. W., Paretti, M. C., & Matusovich, H. M. (2014). Relationships among students’ perceptions of a first-year engineering design course and their engineering identification, motivational beliefs, course effort, and academic outcomes. International Journal of Engineering Education, 30(6A), 1340–1356.
Jones, B. D., Paretti, M. C., Hein, S. F., & Knott, T. W. (2010). An analysis of motivation constructs with first-year engineering students: Relationships among expectancies, values, achievement, and career plans. Journal of Engineering Education, 99(4), 319–336. https://doi.org/10.1002/j.2168-9830.2010.tb01066.x
Jones, B. D., Ruff, C., & Osborne, J. W. (2015). Fostering students’ identification with mathematics and science. In K. A. Renninger, M. Nieswandt, & S. Hidi (Eds.), Interest in mathematics and science learning (pp. 331–352). American Educational Research Association. https://doi.org/10.3102/978-0-935302-42-4_19
Jones, B. D., & Skaggs, G. E. (2016). Measuring students’ motivation: Validity evidence for the MUSIC Model of Academic Motivation Inventory. International Journal for the Scholarship of Teaching and Learning, 10(1). http://digitalcommons.georgiasouthern.edu/ij-sotl/vol10/iss1/7
Jones, B. D., Tendhar, C., & Paretti, M. C. (2016). The effects of students’ course perceptions on their domain identification, motivational beliefs, and goals. Journal of Career Development, 43(5), 383–397. https://doi.org/10.1177/0894845315603821
Jones, B. D., & Wilkins, J. L. M. (2023). Validating the MUSIC Model of Academic Motivation Inventory: Evidence for the short forms of the college student version. Journal of Psychoeducational Assessment, 41(1), 22–35. https://doi.org/10.1177/07342829221121695
Jones, B. D., Zhu, X., Ellis, M., Fenerci, H., & Ambarkutuk, Z. (2023). Relationships between course motivational climate and students’ computer science beliefs and goals. Research presented at the annual convention of the American Psychological Association, Washington DC.
Kline, R. B. (2013). Exploratory and confirmatory factor analysis. In Y. Petscher & C. Schatsschneider (Eds.), Applied quantitative analysis in the social sciences (pp. 171–207). Routledge.
Kosovich, J. J., Hulleman, C. S., Barron, K. E., & Getty, S. (2015). A practical measure of student motivation: Establishing validity evidence for the expectancy-value-cost scale in middle school. Journal of Early Adolescence, 35(5–6), 790–816. https://doi.org/10.1177/0272431614556890
Kramer, C. Y. (1956). Extension of multiple range tests to group means with unequal number of replications. Biometrics, 12, 307–310. https://doi.org/10.2307/3001469
Lam, A. C., Ruzek, E. A., Schenke, K., Conley, A. M., & Karabenick, S. A. (2015). Student perceptions of classroom achievement goal structure: Is it appropriate to aggregate? Journal of Educational Psychology, 107(4), 1102. https://doi.org/10.1037/edu0000028
Lent, R. W., Brown, S. D., & Larkin, K. C. (1986). Self-efficacy in the prediction of academic performance and perceived career options. Journal of Counseling Psychology, 33(3), 265. https://doi.org/10.1037/0022-0184.108.40.2065
Lent, R. W., Brown, S. D., Sheu, H.-B., Schmidt, J., Brenner, B. R., Gloster, C. S., Wilkins, G., Schmidt, L. C., Lyons, H., & Treistman, D. (2005). Social cognitive predictors of academic interests and goals in engineering: Utility for women and students at historically black universities. Journal of Counseling Psychology, 52(1), 84–92. https://doi.org/10.1037/0022-0220.127.116.11
Li, M., Jones, B. D., Williams, T. O., & Guo, Y. (2022). Chinese students’ perceptions of the motivational climate in college English courses: Relationships between course perceptions, engagement, and achievement. Frontiers in Psychology, 13, 853221. https://doi.org/10.3389/fpsyg.2022.853221
Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57, 705–717. https://doi.org/10.1037/0003-066X.57.9.705
London, J. S., Lee, W. C., & Hawkins Ash, C. D. (2021). Potential engineers: A systematic literature review exploring black children’s access to and experiences with stem. Journal of Engineering Education, 110(4), 1003–1026. https://doi.org/10.1002/jee.20426
Ma, Y., Siu, A., & Tse, W. S. (2018). The role of high parental expectations in adolescents’ academic performance and depression in Hong Kong. Journal of Family Issues, 39(9), 2505–2522. https://doi.org/10.1177/0192513X18755194
Malcom, S. M., Hall, P. Q., & Brown, J. W. (1976). The double bind: The price of being a minority woman in science (AAAS Report No. 76-R-3). American Association for the Advancement of Science.
Mamaril, N. A., Usher, E. L., Li, C. R., Economy, D. R., & Kennedy, M. S. (2016). Measuring undergraduate students’ engineering self-efficacy: A validation study. Journal of Engineering Education, 105(2), 366–395. https://doi.org/10.1002/jee.20121
Margolis, J., Fisher, A., & Miller, F. (2000). The anatomy of interest: Women in undergraduate computer science. Women’s Studies Quarterly, 28(1/2), 104–127.
Marra, R. M., Rodgers, K. A., Shen, D., & Bogue, B. (2009). Women engineering students and self-efficacy: A multi-year, multi-institution study of women engineering student self-efficacy. Journal of Engineering Education, 98(1), 27–38. https://doi.org/10.1002/j.2168-9830.2009.tb01003.x
Marshman, E. M., Kalender, Z. Y., Nokes-Malach, T., Schunn, C., & Singh, C. (2018). Female students with A’s have similar physics self-efficacy as male students with C’s in introductory courses: A cause for alarm? Physical Review Physics Education Research, 14(2), 020123.
Mau, W.-C. (1997). Parental influences on the high school students’ academic achievement: A comparison of Asian immigrants, Asian-Americans, and White-Americans. Psychology in the Schools, 34(3), 267–277.
McGee, E. O. (2016). Devalued black and Latino racial identities: A by-product of STEM college culture? American Educational Research Journal, 53(6), 1626–1662. https://doi.org/10.3102/0002831216676572
McGinley, J., & Jones, B. D. (2014). A brief instructional intervention to increase students’ motivation on the first day of class. Teaching of Psychology, 41(2), 158–162. https://doi.org/10.1177/0098628314530350
Meiksins, P., & Layne, P. (2022). Women in engineering: Analyzing 20 years of social science literature. SWE Magazine, 68(3). https://magazine.swe.org/lit-review-22/
Mora, C. E., Anorbe-Diaz, B., Gonzalez-Marrero, A. M., Martin-Gutierrez, J., & Jones, B. D. (2017). Motivational factors to consider when introducing problem-based learning in engineering education courses. International Journal of Engineering Education, 33(3), 1000–1017.
National Center for Science and Engineering Statistics. (2021). Women, minorities, and persons with disabilities in science and engineering: 2021 (Special Report NSF 21–321). National Science Foundation. https://ncses.nsf.gov/wmpd
Nelson, K. G., Shell, D. F., Husman, J., Fishman, E. J., & Soh, L. (2015). Motivational and self-regulated learning profiles of students taking a foundational engineering course. Journal of Engineering Education, 104(1), 74–100. https://doi.org/10.1002/jee.20066
Pedraza, L., & Chen, R. (2022). Examining motivator factors of STEM undergraduate persistence through two-factor theory. The Journal of Higher Education, 93(4), 532–558. https://doi.org/10.1080/00221546.2021.1999722
Pirttinen, N., Hellas, A., Haaranen, L., & Duran, R. (2020). Study major, gender, and confidence gap: Effects on experience, performance, and self-efficacy in introductory programming. In Proceedings of the 2020 IEEE Frontiers in Education Conference (FIE) (pp. 1–7). https://doi.org/10.1109/FIE44824.2020.9273884
Renninger, K. A., & Hidi, S. E. (2015). The power of interest for motivation and engagement. Routledge. https://doi.org/10.4324/9781315771045
Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, 1–11. https://doi.org/10.1016/j.cedpsych.2020.101860
Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Sage Publications.
Satorra, A., & Bentler, P. M. (2010). Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika, 75, 243–248. https://doi.org/10.1007/S11336-009-9135-Y
Schenke, K., Nguyen, T., Watts, T. W., Sarama, J. H., & Clements, D. H. (2017). Differential effects of the classroom on African American and non-African American’s mathematical achievement. Journal of Educational Psychology, 109(6), 794–811. https://doi.org/10.1037/edu0000165
Schraw, G., & Lehman, S. (2001). Situational interest: A review of the literature and directions for future research. Educational Psychology Review, 13(1), 23–52. https://doi.org/10.1023/A:1009004801455
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54, 93–105. https://doi.org/10.1037/0003-066X.54.2.93
Schweig, J. D., & Martínez, J. F., et al. (2021). Understanding (dis)agreement in student ratings of teaching and the quality of the learning environment. In W. Rollett (Ed.), Student feedback on teaching in schools (pp. 91–110). Springer. https://doi.org/10.1007/978-3-030-75150-0_6
Seymour, E. (1995). The loss of women from science, mathematics, and engineering undergraduate majors: An explanatory account. Science Education, 79(4), 437–473. https://doi.org/10.1002/sce.3730790406
Spence, C. M., Kirn, A., & Benson, L. (2022). Perceptions of future careers for middle year engineering students. Journal of Engineering Education, 111(3), 598–615. https://doi.org/10.1002/jee.20455
Steinberg, L., Dornbusch, S. M., & Brown, B. B. (1992). Ethnic differences in adolescent achievement: An ecological perspective. American Psychologist, 47(6), 723–729.
Stout, J. G., Dasgupta, N., Hunsinger, M., & McManus, M. A. (2011). STEMing the tide: Using ingroup experts to inoculate women’s self-concept in science, technology, engineering, and mathematics (STEM). Journal of Personality and Social Psychology, 100, 255–270. https://doi.org/10.1037/a0021385
Stroebe, W. (2020). Student evaluations of teaching encourages poor teaching and contributes to grade inflation: A theoretical and empirical analysis. Basic and Applied Social Psychology, 42(4), 276–294. https://doi.org/10.1080/01973533.2020.1756817
Takahira, S., Goodings, D. J., & Byrnes, J. P. (1998). Retention and performance of male and female engineering students: An examination of academic and environmental variables. Journal of Engineering Education, 87(3), 297–304. https://doi.org/10.1002/j.2168-9830.1998.tb00357.x
Tellhed, U., Björklund, F., Kallio Strand, K., & Schöttelndreier, K. (2023). “Programming is not that hard!” When a science center visit increases young women’s programming ability beliefs. Journal for STEM Education Research, 6, 252–274. https://doi.org/10.1007/s41979-023-00094-w
Tendhar, C., Singh, K., & Jones, B. D. (2017). Using the domain identification model to study major and career decision-making processes. European Journal of Engineering Education, 43(2), 235–246. https://doi.org/10.1080/03043797.2017.1329280
Tukey, J. W. (1953). The problem of multiple comparisons. In H.I. Braun (Ed.), Multiple comparisons, 1948–1983, (Vol. 8 of The Collected Works of John W. Tukey, pp. 1–300). Chapman & Hall
Tyson, W. (2011). Modeling engineering degree attainment using high school and college physics and calculus coursetaking and achievement. Journal of Engineering Education, 100(4), 760–777. https://doi.org/10.1002/j.2168-9830.2011.tb00035.x
Vogt, C. M., Hocevar, D., & Hagedorn, L. S. (2007). A Social cognitive construct validation: Determining women’s and men’s success in engineering programs. The Journal of Higher Education, 78(3), 337–364. https://doi.org/10.1080/00221546.2007.11772319
Wallace, J. E., & Haines, V. A. (2004). The benefits of mentoring for engineering students. Journal of Women and Minorities in Science and Engineering, 10, 377391.
Wang, J., Hejazi Moghadam, S., & Tiffany-Morales, J. (2017, August). Social perceptions in computer science and implications for diverse students. In Proceedings of the ICER ‘17: 2017 ACM Conference on International Computing Education Research (pp. 47–55). https://doi.org/10.1145/3105726.3106175
Weiner, B. (2000). Intrapersonal and interpersonal theories of motivation from an attributional perspective. Educational Psychology Review, 12(1), 1–14. https://doi.org/10.1023/A:1009017532121
Whitcomb, K. M., & Singh, C. (2021). Underrepresented minority students receive lower grades and have higher rates of attrition across STEM disciplines: A sign of inequity? International Journal of Science Education, 43(7), 1054–1089. https://doi.org/10.1080/09500693.2021.1900623
Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68–81. https://doi.org/10.1006/ceps.1999.1015
Wilkins, J. L. M., Jones, B. D., & Rakes, L. (2021). Students’ class perceptions and ratings of instruction: Variability across undergraduate mathematics courses. Frontiers in Psychology, 12, 576282. https://doi.org/10.3389/fpsyg.2021.576282
Wilson, D., Bates, R., Scott, E., Painter, S., & Shaffer, J. (2015). Differences in self-efficacy among women and minorities in STEM. Journal of Women and Minorities in Science and Engineering, 21(1), 27–45. https://doi.org/10.1615/JWomenMinorScienEng.2014005111
White, R. W. (1959). Motivation reconsidered: The concept of competence. Psychological Review, 66(5), 297–333. https://doi.org/10.1037/h0040934
Young, A. M., Wendel, P. J., Esson, J. M., & Plank, K. M. (2018). Motivational decline and recovery in higher education STEM courses. International Journal of Science Education, 40(9), 1016–1033. https://doi.org/10.1080/09500693.2018.1460773
Zusho, A., Pintrich, P. R., & Cortina, K. S. (2005). Motives, goals, and adaptive patterns of performance in Asian American and Anglo American students. Learning and Individual Differences, 15(2), 141–158. https://doi.org/10.1016/j.lindif.2004.11.003
The authors would like to thank Virginia Tech’s Open Access Subvention Fund for paying the publication fees for this article.
This study was funded in part by the Pathways Grant Program, Provost’s Office, Virginia Tech.
The authors declare that there they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jones, B.D., Ellis, M., Gu, F. et al. Motivational climate predicts effort and achievement in a large computer science course: examining differences across sexes, races/ethnicities, and academic majors. IJ STEM Ed 10, 65 (2023). https://doi.org/10.1186/s40594-023-00457-0