Gendered patterns in students’ motivation profiles regarding iSTEM and STEM test scores: a cluster analysis

Abstract

Background

Promoting and improving STEM education is being driven by economic concerns as modern economies have a rising demand for qualified researchers, technicians, and other STEM professionals. In addition, women remain under-represented in STEM-related fields, with significant economic and societal consequences. Abundant research has shown that gendered pathways into and away from STEM are mediated through motivation, but there is paucity of knowledge regarding gendered patterns in high school students’ motivation profiles, especially in transdisciplinary domains like integrated STEM (iSTEM). This study addresses these gaps by examining the interconnection between patterns in motivation profiles towards integrated STEM (iSTEM), gender and STEM test scores.

Results

Using cluster analysis in a sample of N = 755 eighth grade students, we established four distinct motivation profiles. Subsequently, a multinomial logistic regression was performed to calculate predicted probabilities for cluster membership based on gender and test scores. Cluster distributions indicate significant differences based on gender and test score. Although our analysis shows no difference in average test scores, significant gender differences can be found in and between motivation profiles. For instance, girls are more likely to belong to a less favorable profile cluster than boys. In that cluster, girls have on average a significantly higher test score compared to boys, indicating a differential effect of motivation profiles.

Conclusions

The concept of motivational co-expression emphasizes a need for instructors to move past the simple high or low motivation labels, and toward an appraisal that recognizes how students adopt a complex interplay of motivation types. Moreover, the gender analyses raise questions about how we can move towards more equitable approaches.

Introduction

Internationally, consensus can be found on the importance of students’ participation in Science, Technology, Engineering, and Mathematics (STEM, Dewitt & Archer, 2015, p. 4). With STEM career fields expanding at a rapid rate, there is a growing shortage of STEM professionals (Keith, 2018; OECD, 2008). As such, promoting and improving STEM education is increasingly being driven by economic concerns as modern economies have a rising demand for qualified researchers, technicians, and other STEM professionals. Despite these needs and subsequent positive prospects on the labor market for people with a STEM background, insufficient numbers of students choose a STEM profession or career (Keith, 2018). In particular, girls seem to disengage from STEM (Card & Payne, 2021; Ing, 2014; Wang & Degol, 2017) as female students and employees are under-represented in STEM-related fields. According to UNESCO, only 35% of STEM students in higher education globally are women (Chavatzia, 2017). Not only is female participation in STEM education and employment low, but the attrition rate is also particularly high. Women leave STEM disciplines in disproportionate numbers during their studies and even during their careers (Fernández Polcuch et al., 2018). This underrepresentation has important economic and societal consequences. Morais Maceira (2017) indicates that equal gender participation in STEM has a strong positive impact on a country’s gross domestic product (GDP) and helps to reduce disparity in economic status (e.g., wage gap) between men and women.

Given the significance of the observed gender gap in STEM, the issue has received widespread attention. Literature offers several theoretical frameworks to explain and address the persistent underrepresentation of women in STEM fields, with social cognitive career theory (Lent et al., 1994), expectancy-value theory (Eccles & Wigfield, 2002), and theories regarding personal interest (Hidi & Renninger, 2006) being the most influential. More recently, Wang and Degol (2017) adopted a multiple theoretical perspective and attributed female underrepresentation in STEM fields to a cultural phenomenon brought about by the complex interaction of six underlying factors: (1) absolute ability differences, (2) relative ability strengths, (3) career preferences, (4) lifestyle preferences, (5) field-specific ability beliefs, and (6) gender stereotypes and bias. Hence, career pathways encompass the ability to pursue a career as well as the motivation to employ that ability and devote time to it. Many studies on motivation and gender have focused on learners’ self-efficacy, goals, interests, and values (e.g., Liu et al., 2009; Marshman et al., 2018; Ratelle et al., 2007), but findings are inconsistent. Stolk et al., (2021, p. 4) described several studies that showed that women reported higher autonomous motivations and lower controlled motivations compared to men (Ratelle et al., 2007; Vallerand et al., 1992), but it also discussed studies that reported no gender differences in situational or contextual level motivations (Liu et al., 2009; Vecchione et al., 2014) or less positive motivations among women (Hakan & Münire, 2014). This highlights the need for further exploration of gender and motivation towards STEM.

Motivation and motivation profiles

Motivation is fundamental to human agency and volitional behavior, and several influential theories have been proposed to explain why individuals choose or persist in a specific course of action (Hattie et al., 2020, p. 1). Moreover, motivation-related self-concepts (e.g., self-efficacy and academic self-concept) are considered important precursors for students’ academic and career aspirations (Eccles & Wigfield, 2002; Jiang et al., 2020). According to expectancy-value theory, students are more motivated to achieve in areas in which they expect to succeed and that they value (Leaper et al., 2012, p. 269). The expectation of success refers to the individuals' ability beliefs. This is comparable to self-efficacy in social cognitive theory (Bandura, 1997) and perceived competence in self-perception theory (Harter, 1992).

Through the lens of self-determination theory (SDT), Ryan and Deci (2020) specify three types of motivation (i.e., amotivation, extrinsic motivation, and intrinsic motivation), arranged along a continuum reflecting the degree to which the regulation of behavior is internalized (i.e., non-, external, introjected, identified, integrated, and intrinsic regulation). An overview can be found in Fig. 1. As both identified motivation and intrinsic motivation are characterized by a sense of personal choice, they are referred to as autonomous types of motivation. On the other hand, external and introjected regulation can be classified as controlled types of motivation since both are associated with a sense of pressure that can originate from an external source (i.e., external regulation) or the students themselves (i.e., introjected regulation). Each motivation type is defined by unique characteristics (i.e., enjoyment, meaningfulness, ego involvement, external pressures) and will therefore have different consequences (Howard et al., 2021; Vansteenkiste et al., 2009). For instance, more autonomous motivation types lead to better academic achievement, effort and engagement compared to external types of motivation (Howard et al., 2021). In addition, the category of amotivation, which refers to a state in which neither intrinsic nor extrinsic factors energize action (Ryan & Deci, 2020), shows a strong relation with poor outcomes such as absenteeism and dropout intention (Howard et al., 2021).

Fig. 1

(adapted from Ryan and Deci (2020))

Representation of motivation in self-determination theory

According to SDT, individuals will adopt more internalized, or autonomous types of motivation when three basic needs are satisfied: (1) competence, a sense of mastery or self-efficacy; (2) relatedness, a sense of positive connections; and (3) autonomy, a sense of choice and control (Ryan & Deci, 2000). Although motivation forms an extensively studied and conceptualized concept, several enduring questions remain unanswered and several theoretical frameworks regarding motivation can be found (Koenka, 2020). In their effort to unify five theoretical frameworks regarding motivation, Hattie et al. (2020) emphasizes the interplay of self or internal processes and external influences (e.g., our perception of others, teachers, bosses) that determine our motivations. They used the metaphor of the rope aimed to emphasize that there is no single strand underlying our motivation, but that there are many overlapping internal processes and external influences; and the strength in the rope ‘lies not in one fibre running throughout its length, but in the overlapping of many fibres’ (Wittgenstein, 2010: Sect. 67). For instance, high levels of controlled motivation do not exclude high levels of autonomous motivation or vice versa. As contemporary studies indicate the complex interplay of different motivational processes (Hattie et al., 2020), a more holistic approach (e.g., motivation profiles) is needed.

The relationship between distinct motivational variables and achievement is not new and studies often investigate motivational constructs in isolation using a variable-centered approach. These studies rely on the use of linear models using different kinds of regression and correlation techniques to study student achievement and its relation to motivational variables. Meanwhile, there is less emphasis on the interplay of individual differences in key motivational indicators to look for meaningful groups of students that share similar profiles (e.g., motivation profiles) in which the student is regarded as the unit of analysis. We therefore propose the use of an alternative approach to examine student motivation: the person-centered approach.

Cluster analysis has previously been used to examine students’ motivational profiles in educational settings (Kong & Liu, 2020; Liu et al., 2009; Ng et al., 2016; Ratelle et al., 2007; Vansteenkiste et al., 2009). It allows individuals to be allocated to subgroups which hold a particular motivation profile. This facilitates a person-centered approach which identifies homogenous groups of students based on their responses to variables, instead of the usual variable-centered approach that typically groups variables on common underlying dimensions or factors (Wang & Biddle, 2001). Therefore, cluster analysis is complementary to factor analysis as it focusses on how individuals (cases) group together based on behaviors, beliefs, or other characteristics of interest (Antonenko et al., 2012). A cluster analysis approach would allow us to apply the SDT framework and represent motivation in a way that captures the multi-dimensionality of the construct.

In literature, the strength of motivation profiles has been established and lies predominantly in its ability to identify intraindividual qualities that characterize meaningful classifiable groups of students. Nevertheless, the use of cluster analysis to identify high school students’ motivation profiles with a specific emphasis on motivation towards STEM is less substantial. Stolk et al. (2021) examined the motivation profiles of women and men in college STEM courses and were able to identify seven motivational response profiles (i.e., autonomous, high autonomous-high external, high identified-high external, moderate identified, neutral, external, and high amotivation). In line with Vansteenkiste et al. (2009) and Ratelle et al. (2007), the autonomous cluster represented the most positive motivation response profile. Additionally, Stolk et al. (2021) focused on motivational differences related to different pedagogical approaches and identified strong gender-based differences in motivation during lecture-based learning. In courses with lecture-based learning activities, female students registered higher controlled motivation and lower autonomous motivation compared to male students (Stolk et al., 2021). As such gender differences can be observed in motivation profiles regarding pedagogy, but little research meaningfully connects high school students’ motivation profiles towards integrated STEM (iSTEM) with gender. In contrast to ‘segregated’ STEM, iSTEM requires the application of knowledge and practices from across STEM disciplines to solve authentic problems (Nadelson & Seifert, 2017).

Gender gap

Abundant research has shown that gendered pathways into and away from STEM are mediated through motivation (Dietrich & Lazarides, 2019; Eccles & Wang, 2016) and gender gaps have been observed in both STEM interest and self-efficacy (Ertl et al., 2017; Tzu-Ling, 2019). From a SDT point of view, individual interest (i.e., a relatively enduring preference for certain topics, subject areas, or activities; Ainley et al., 2002) and self-efficacy (i.e., confidence in being able to orchestrate and execute actions required for achieving intended results such as mastering a task; Bandura, 1986, p. 369) form two precursors for individuals to adopt more internalized or autonomous types of motivation (Ryan & Deci, 2000). It should be emphasized that self-efficacy “is not a unitary or global trait, like self-esteem. Rather, self-efficacy is conceived as a dynamic set of self-beliefs that are linked to performance domains and activities” (Lent & Brown, 2006, p. 15). A student may be firmly convinced that he or she can perform well in a math class, but not feel competent to ask questions. Although this individual’s self-esteem may be stable across both domains, his or her self-efficacy can be very different for the different contexts (Sawtelle et al., 2012, p. 1099).

Most studies have identified gender differences in STEM self-efficacy that favor men (e.g., Marshman et al., 2018; Nissen & Shemwell, 2016; Yerdelen-Damar & Peşman, 2013), but some studies also reported no significant gender differences in self-efficacy (Britner & Pajares, 2006; Concannon & Barrow, 2009; Kalender et al., 2019). For instance, Yerdelen-Damar and Peşman (2013) concluded that boys showed higher levels of physics self-efficacy than girls, while boys’ achievement in physics was lower. On the other hand, Kalender et al., (2019, p. 10) found no direct relation between gender and competency beliefs but pointed out that the relationship flows through recognition by others.

As career pathways encompass the ability to pursue a career as well as the motivation to employ that ability (Wang & Degol, 2017, p. 119), this study aims to bridge research regarding gender and motivation profiles and skill.

Integrated STEM

Most studies regarding motivation focus on ‘segregated’ STEM disciplines (e.g., Mathematics or Science). Even when labeled STEM, researchers often focus on specific domains such as Mathematics or Science. The current international focus in STEM education, however, moves towards integrating the separate STEM disciplines through ‘integrated STEM’ (iSTEM) (Koul et al., 2018; Roehrig et al., 2021). Thibaut et al. (2018) identified in their review of literature the following five categories of instructional elements essential for teaching integrated STEM: (1) the explicit assimilation of learning goals, content and practices from different STEM disciplines; (2) a problem-centered learning environment that involves students in authentic, open-ended, ill-structured, real-world problems; (3) an inquiry-based learning environment that engages students in questioning, experimental learning and hands-on activities; (4) design-based learning that uses open-ended, hands-on design challenges; and (5) cooperative learning where students get the opportunity to communicate and collaborate with each other. This supports the claim that iSTEM conceptually differs from its separate subdisciplines and requires a unique pedagogical approach (De Meester et al., 2020; Roehrig et al., 2021). Therefore, this study focuses on motivation profiles in iSTEM courses.

Goals of this study

Despite abundant research on gender, motivation and ability, there is paucity of knowledge regarding gendered patterns in high school students’ motivation profiles. Especially in transdisciplinary domains like iSTEM, publications are scant.

We previously established the importance of the quality of motivation (Vansteenkiste et al., 2009) and observed gender differences in both STEM interest and self-efficacy (Eccles, 2011; Ertl et al., 2017; Tzu-Ling, 2019). As iSTEM differs from ‘segregated’ STEM and requires a unique pedagogical approach (De Meester et al., 2020; Roehrig et al., 2021), more knowledge on motivation profiles towards iSTEM is needed. To better understand how motivation profiles in iSTEM relate to STEM test scores and to identify possible gender differences, the following three research questions were developed to guide this study:

• 1. What student profiles regarding iSTEM motivation can be identified?

• 2. How do these profiles relate to STEM test scores?

• 3. To what extent can we distinguish gendered patterns in student profiles regarding iSTEM motivation and test scores?

Methods

This study is part of a larger project on iSTEM in which several academic partners joined forces to help teachers develop iSTEM teaching modules in teacher design teams. The iSTEM methodology used is based on research conducted during the STEM@school project (Knipprath et al., 2018). To measure the effects of the developed iSTEM teaching modules, researchers adopted a quasi-experimental design. Current data collection is part of the pre-test at the beginning of the school year before any intervention took place. Student motivation variables were measured using individual self-report and a cognitive STEM test. Both were administered through an online questionnaire.

Participants

Participants in the study were 755 grade eight students enrolled in STEM courses across 28 institutions. STEM courses in grade eight are part of an optional package (e.g., classical languages, STEM, economics, physical education, arts, society and well-being, catering and hospitality). Students in the Flemish education system choose a more specialized study track when transitioning to the ninth grade. Students who identified as boys provided a total of 538 responses (71%), and those who identified as girls provided 217 responses (29%). This sample is representative for the total Flemish (the Dutch speaking community of Belgium) grade eight student population enrolled in STEM courses (i.e., boys: 73%, girls: 27%; Verhaegen et al., 2020). Information regarding gender, institution and motivation was acquired from the self-reports of students via an online questionnaire. Informed consent from the participating students was obtained online before starting the questionnaire. Students completed the online questionnaires during normal school hours under supervision of the schools’ contact person. Individual self-report questionnaires on motivation were administered first, followed by a cognitive STEM test.

Instruments

Motivation

To measure motivation, we used individual self-report questionnaires. Twenty items from the Self-Regulation Questionnaire (SRQ; Ryan & Connell, 1989) were adjusted to assess students’ motivation for studying iSTEM (De Loof, 2019). Participants indicated the importance of their study behavior motivation towards iSTEM on a four-point Likert scale ranging from 1 = strongly disagree to 4 = strongly agree. As Howard et al. (2021) concluded that more autonomous motivation types lead to better academic achievement, we chose this classification structure for our constructs. This is also in line with previous work on motivation profiles (Ratelle et al., 2007; Stolk et al., 2021; Vansteenkiste et al., 2009). Three constructs were measured (i.e., amotivation, controlled motivation and autonomous motivation) based on underlying subscales (see Appendix A). Controlled motivation was composed of the subscales of external regulation (e.g., “During STEM classes I do my best because others expect this from me”) and introjected regulation (e.g., “During STEM classes I do my best because I want others to think I’m smart.”). Autonomous motivation was constructed from subscales of identified regulation (e.g., “During STEM classes I do my best because STEM is important to me”) and intrinsic regulation (e.g., “I try to do my best during STEM classes because STEM is fun”). Four items questioned amotivation (e.g., “I’m wasting my time during STEM lessons”). It is based on a two-level model using the three constructs and five subscales (see Appendix A). A confirmatory factor analysis (CFA) validation resulted in an acceptable fit, as CFI = 0.94, RMSEA = 0.07 and standardized factor loadings were uniformly significant. Given the acceptable fit, we proceeded with this model and calculated the participants’ mean scores on the tree scales. Additionally, a differential item functioning analysis was conducted to see if items functioned differently for boys and girls. At the overall construct level, there were minimal differences in the total expected score and therefore all items were retained (see Appendix D).

Cognitive STEM test

The instrument was constructed based on the curriculum for Physics, Mathematics, and technological concepts of seventh and eighth grade. Items from existing iSTEM instruments (e.g.,De Loof, 2019), were selected by pedagogical and subject matter experts. Subsequently, we piloted the test with a smaller group of 187 students to examine internal consistency, item difficulty and discriminatory power. Items with a discrimination value below 0.15 were removed from the item battery. The psychometric qualities of the test were investigated, using latent trait models under Item Response Theory (IRT). The ltm-package (Rizopoulos, 2006) of R (open-source software for statistical computing) was used. This resulted in a 23-item multiple choice test. Item characteristics (i.e., difficulty and discrimination) were analyzed. Analysis of variance (ANOVA) showed that the 2-PL model fitted the data best based on Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC) and Log-Likelihood. The discrimination values (α) for all the included items were above 0.15, which was in line with our pilot test, and this indicated that all items were able to differentiate between student skill. The test information function showed a sharp peak around 0 (see Appendix B), which means that the test is very informative for students with average skills. An overview of item topics and correlating STEM subdiscipline can be found in Appendix C. We also looked at gender bias on an item level. Although five out of the 23 items (see Appendix C) display a potential bias, three favored girls and two favored boys. Therefore, the overall instrument with 23 items was deemed suitable.

Analyses

STEM motivation profiles. To identify iSTEM motivation profiles, we used a two-step cluster analysis (CA). Cluster analysis is used to detect groups of students with similar patterns of variation across sets of variable characteristics of the observations (Bartholomew et al., 2008; Sarstedt & Mooi, 2014). An important note is that the grouping is not known in advance and that the identification of homogenous students is in essence a taxonomy analysis. Quantitative clustering techniques have proven to be successful before in characterizing motivations by revealing how individuals express a combination of different motivation types (Bråten & Olaussen, 2005; Csizér & Dörnyei, 2005; Ratelle et al., 2007; Stolk et al., 2021).

While several techniques exist to perform cluster analysis, we opted for a TwoStep Cluster analysis. The algorithm used in this type of cluster analysis provides several desirable features that differentiate it from traditional clustering techniques like k-means clustering and hierarchical clustering. In two-step cluster analysis both categorical and continuous variables can be used to generate a solution based on very large datasets like the one in this study. As is implied by the name, TwoStep clustering is composed by two separate steps with the first step being the pre-clustering step. In this step, all cases are scanned one by one to construct a cluster features (CF) tree (Zhang et al., 1996). In this pre-clustering phase, the algorithm applies a log-likelihood distance measure to determine whether a specific case should form a new pre-cluster on its own and wait for more similar cases later in the process, or whether it should be merged with other cases. When all cases have been scanned, the resulting pre-clusters are treated as entities on their own and serve as the raw data for the next step. Here the advantage of TwoStep clustering and the use of pre-clustering becomes apparent as the pre-clustering reduces the size of the matrix that contains the distance between all plausible pairs of cases in that it now depends on the number of pre-clusters as opposed to the total number of cases. Further, during the pre-clustering step, all continuous variables are standardized automatically, so no data transformation must be conducted in separate steps. After the first step, an agglomerative algorithm is used to complete the clustering procedure. For a more comprehensive description, consult Meila and Heckerman (2013) and Banfield and Raftery (1993). Another advantage of the TwoStep Cluster analysis is that it empirically identifies important combinations in the data rather than imposing them from a prior scheme. Hence, we can look for inconsistent motivational profiles and nuance our understanding of student motivation in secondary education STEM using a technique that helps us reveal the multifaceted nature of their motivation. In other words, we will try to identify students’ simultaneous expression of different forms of motivation at that time (Ratelle et al., 2007). Furthermore, we opted to use the TwoStep Cluster analysis technique since it has proven successful in earlier research on motivational profiles concerning eighth graders within the context of Trends in International Mathematics and Science Study (TIMSS) (Michaelides et al., 2019).

Since cluster analysis techniques are explorative in nature, a different number of clusters may be extracted and interpreted. This is especially true for TwoStep clustering. Therefore, during our preliminary analysis, only a small number of clusters were extracted when the optimal number of clusters was automatically determined. In this solution, the clusters lacked additional information due to their consistency with respect to the input variables. More concretely, one cluster included students scoring high on all variables, another one grouped students with moderate scores, and the third one was composed of students scoring low on all variables. Since the aim of this study is to identify inconsistent profiles across several motivational constructs, we opted to increment the number of clusters between four and six. This range of clusters was selected for a few reasons, the most important one being that the clustering solution would produce more than just clusters with consistent motivational responses. Since we incremented the number of clusters between four and six, selection of the competing number of clusters was not done automatically. In choosing the final number of clusters, statistical criteria such as the silhouette measure of cohesion and separation (Tinsley & Brown, 2000) were used. The value of this score ranges from − 1 to 1 with a high value indicating that the object matches well with its own cluster, while poorly matching neighboring clusters. In addition, the relative size of the smallest cluster resulted in > 7% of the sample. Lastly, the interpretation of the number of derived clusters was considered by two independent researchers and agreement was reached (Sarstedt & Mooi, 2014). The scale variables were extracted from the earlier mentioned Self-Regulation Questionnaire (SRQ; Ryan & Connell, 1989). Cluster analyses were carried out in SPSS (Version 27.0). The following measures were entered as input variables:

• 1. Autonomous motivation

• 2. Controlled motivation

• 3. Amotivation.

Motivational profiles and their relationship with cognitive test score and gender.

Next, we investigated the correlation between STEM motivation profile, STEM cognitive test scores and gender. Test and motivational scores were collected at the level of students; therefore, every student is a data point. Given that students learn together in a school, students are nested within schools. To adequately model the data, the hierarchical structure of the student data must be considered, we computed the intra-cluster correlation coefficients (ICC) and modeled our data using multilevel modeling. Multilevel modeling allows data to be clustered in groups and have a hierarchical structure. Regarding the test score, the ICC of school was 0.11, meaning that 11% of the variance in test scores can be attributed to the school. RStudio v4.0.0 was used for exploratory data analysis, modeling, and model diagnostics. Analyses were conducted using the lme4 package (Bates et al., 2014).

To predict cluster membership based on gender and test score; and to visualize our data, multinomial logistic regression analysis (Starkweather, 2011), the nnet-package (Venables & Ripley, 2002) of R was used. This allowed us to predict categorical placement in or the probability of category membership on a dependent variable (i.e., motivation profile) based on multiple variables (i.e., gender and STEM test score). To detect outliers and influential data points, we ran separate logit models and used diagnostics on each model. After assessing the nature of these outliers and their effect on the analysis, no data points were removed or altered.

Results

Cognitive STEM test

Although the integrated STEM test incorporates items from different STEM fields, a general score was calculated for each student (max score = 10). The mean score (µ) (n = 755) was 4.84, with a standard deviation (σ) of 3.3. Skewness (0.08) and kurtosis (2.36) were within an acceptable range, so we can conclude that test scores follow a normal distribution (see Fig. 2). Analysis indicated no significant score differences, t(753) = 1.29, p = 0.20 between girls (M = 4.92, SD = 1.84) and boys (M = 4.81, SD = 1.81).

Fig. 2

STEM motivation profiles identified in this study—mean and standard deviation bars. AU = autonomous motivation, CON = controlled motivation, AM = amotivation

iSTEM motivation and test scores

Prior to conducting a cluster analysis, we removed 23 incomplete responses, resulting in a sample of 732 students. Two-step cluster analysis was used to determine the number of profiles regarding iSTEM motivation. A model with four clusters was considered most suitable given the statistical criteria that each separate cluster should not contain fewer than 7% of the total number of respondents, and a multivariate test should indicate that the cluster solution explains at least 50% of the total variance (Tinsley & Brown, 2000). The number of respondents belonging to each of the clusters exceeded 7% of the total number of respondents (see Table 1). Table 1 also shows the cluster membership for boys and girls. Moreover, the data contain enough cases (N = 732) to satisfy the cases to variables assumption, as guidelines indicate a minimum of 10 cases per independent variable (Schwab, 2002). The four-cluster solution explains 56% of the variance in amotivation, 66% of the variance in autonomous motivation, and 50% of the variance in controlled motivation.

Table 1 Cluster profiles and gender

Table 2 provides descriptive statistics for each cluster centroid in terms of motivation scales and STEM test score. A graphic representation of the identified iSTEM motivation profiles can be found in Fig. 2. Students with cluster profile 4 score significantly lower on the STEM test than students with cluster profile 1, cluster profile 2 and even cluster profile 3. Although no significant differences were observed between girls’ and boys’ STEM test scores, we do see differences between boys and girls within certain motivation clusters (i.e., cluster 4).

Table 2 Comparison of motivation profiles based on motivation scales and STEM test scores

In the following section, we will present the characteristics of each cluster in detail, and we will identify the specific groups of students, based on STEM test score and gender.

Cluster 1 (n = 193, 26%) reported the highest levels of autonomous motivation (i.e., 3.60) while also showing the lowest amounts of controlled motivation (i.e., 1.55) and amotivation (i.e., 1.07). No significant difference in STEM test score was observed between boys and girls in this cluster (see Table 2).

Cluster 2 (n = 175, 24%) indicated high levels of autonomous motivation (i.e., 3.28), but differs from cluster 1 with regard to a much higher score on controlled motivation (i.e., 2.43). Scores on amotivation (i.e., 1.19) are similar to those in cluster 1. Also, no significant difference in STEM test scores was observed between boys and girls in this cluster (see Table 2).

Cluster 3 (n = 199, 27%) showed lower levels of amotivation (i.e., 1.64) in comparison with cluster 4, but higher than those in cluster 1 and cluster 2. Student in this cluster scored higher on autonomous motivation (i.e., 2.92) than those in cluster 4, but still lower than those in cluster 1 and cluster 2. Also, controlled motivation in this cluster (i.e., 1.64) was considerably lower than in cluster 2 and cluster 4, but slightly higher than in cluster 1. No significant difference in STEM test score was observed between boys and girls in this cluster (see Table 2).

Cluster 4 (n = 165, 23%) indicated the highest levels of amotivation (i.e., 2.25) while also scoring the lowest on autonomous motivation (i.e., 2.30). Controlled motivation scores (i.e., 2.17) were higher than in cluster 1 and cluster 3 but lower than in cluster 2. Moreover, standard deviations in this cluster were the highest ones out of all clusters on all motivational scores, indicating that students in this cluster might differ more from each other than students in other clusters. Girls in this cluster on average have a higher STEM test score than boys (see Table 2). This significant gender difference in STEM test score was only observed in this cluster.

To further investigate how these profiles relate to STEM test score and to identify gendered patterns in student profiles regarding iSTEM motivation and STEM test scores, we calculated predicted probabilities for membership to one of the four clusters. A multinomial logistic regression was performed to create a model of the relationship between the predictor variables (i.e., gender and test score) and membership to one of the four clusters. The fit of the model containing only the intercept improved with the addition of the predictor variables, X²(6, N = 732) = 25.3, Nagelkerke R² = 0.04, p = 0.0003. Parameter estimates can be found in Table 3. The logistic regression coefficient (B) associated with the predictor (i.e., ability and gender) is the expected change in log odds of having the outcome per unit change. Increasing the predictor by one unit multiplies the odds of having the outcome by the coefficient (B). Correlation between ability and cluster membership is statistically significant for cluster 4 in relation to cluster 1, cluster 2 and cluster 3. Correlation between gender and cluster membership is statistically significant for cluster 4 in relation to cluster 1. This difference between boys and girls can also be observed in Table 1.

Table 3 Parameter estimates—multinomial logistic regression analysis

A boy with an average STEM test score (z-score = 0) has a 26% chance to belong to motivation cluster 1, a 24% chance to belong to motivation cluster 2, a 27% chance to belong to motivation cluster 3, and a 23% chance to belong to motivation cluster 4. A girl with the same test score (z-score = 0) has a 24% chance to belong to motivation cluster 1, a 19% chance to belong to motivation cluster 2, a 24% chance to belong to motivation cluster 3, and a 34% chance to belong to motivation cluster 4. A higher STEM test score correlates with a higher possibility to belong to cluster 1, cluster 2 or cluster 3, over the chance to belong to cluster 4. A graphic representation of the multinomial logistic regression is plotted in Fig. 3.

Fig. 3

Plot predicted probabilities across (standardized) STEM test score values for each gender

Significant gender differences can be observed between cluster 1 and cluster 4. Cluster 4 can be considered a less favorable motivation profile as students in this cluster have the highest levels of amotivation (i.e., 2.25) while also scoring the lowest on autonomous motivation (i.e., 2.30). This is in steep contrast with cluster 1, where students show the highest levels of autonomous motivation (i.e., 3.60) and the lowest levels of amotivation (i.e., 1.07). Girls have a higher chance to belong to cluster 4 compared to boys. However, girls in this cluster on average have a higher score on the STEM test than boys (see Table 2).

Discussion

The main purpose of this study was to identify gendered patterns in motivation profiles towards integrated STEM (iSTEM) and to examine how these relate to STEM test scores. The study identified discernible patterns of motivational related variables in grade eight STEM courses students in Flanders. Four clusters of motivational profiles were identified (see Fig. 2).

Cluster 3 accounted for the largest percentage of the sample (27%) while cluster 4 was the smallest (23%). The four-cluster solution explains 56% of the variance in amotivation, 66% of the variance in autonomous motivation, and 50% of the variance in external motivation. Our analysis showed that girls are more likely to belong to the high amotivation profile cluster (i.e., cluster 4) than boys.

When looking at STEM test scores, overall, no differences were found between boys and girls. Higher scores correlate with a higher probability of being placed in a more favorable profile cluster, which is in line with previous research on motivation and self-efficacy. Individuals who feel competent, have a sense of mastery or high self-efficacy will adopt more internalized, or autonomous types of motivation (Ryan & Deci, 2000). This effect was similar for boys and girls. Although girls had a higher chance to belong to the high amotivation profile cluster, girls in this cluster also had on average a significantly higher test score compared to boys. The fact that gendered patterns can be predominantly found in the clusters 4 profile, is interesting. Student motivation levels can influence their level of effort on test performance and thus their test score. Current study suggests that this motivation effect might be differential by gender. That is, girls might have done their best on the test even when having a less favorable (i.e., cluster 4) motivation profile while boys might have put less effort in the test when having the same profile. Although higher levels of amotivation, students in cluster 4 also have on average higher levels of controlled motivation and lower levels of autonomous motivation. This interplay of different types of motivation seems to affect boys and girls differently and supports the idea of using motivation profiles. It is therefore imperative to better understand and further investigate this interplay of different motivation types. The concept of motivational co-expression emphasizes a need for instructors to move past the simple high or low motivation labels, and toward an appraisal that recognizes how students adopt a complex interplay of motivation types. Moreover, the gender-based patterns in students’ motivations regarding iSTEM and how they relate to student performance could impact pedagogical choices.

Although the person-centered approach has been used to investigate motivational profiles of students before in other contexts, one of the interesting findings of this study is the fact that motivational profiles can be homogeneous (i.e., consistently low, medium, or high on all dimensions) or heterogeneous. When looking at studies adopting the self-determination theory framework, Ratelle et al. (2007) found distinct groups of high school students in which levels of autonomous, controlled, and amotivation varied, which differed in their academic achievement. When they used a college sample of students, grouping was different, pointing to the fact that context or developmental factors seem to matter. Looking at the results from this study, we found results similar to the study of Ratelle et al. (2007), with most groups being heterogeneous regarding their levels of autonomous, controlled, and amotivation. This means that motivation is state-like and dynamic in response to context and developmental factors and not always homogeneous within students across time or context. Future studies should therefore take this stake-like view of motivation into consideration by, for example, employing more longitudinal designs to detect possible shifts over clusters by students over time. An important note regarding this implication for further research is that special attention needs to be considered regarding contextual and developmental factors. Examples of these factors are for instance exposure to new learning environments, psychological maturation, and as highlighted by this study, a different school context. The data show that schools are significant predictors for students’ cognitive scores. This study, however, does not consider school variables (e.g., size, location, culture) or teaching styles. A mixed method approach could be adopted to better understand how schools make a difference regarding their students iSTEM motivation profiles and why gendered patterns can predominantly be found in two of the identified motivation profiles.

The present analysis only assessed gender as a dichotomous variable (boys vs. girls). Although dichotomous genders are associated with academic stereotypes and have been included in associated research efforts regarding gender differences in motivation, non-binary students are not adequately represented in this assessment. In the future, additional categories or an open text response could more accurately capture student gender.

Conclusions

To close the gender gap in STEM and develop more equitable approaches, better understanding of motivational differences regarding iSTEM is essential. The results of our study show significant gender differences in motivation profiles regarding iSTEM and STEM test scores within those profiles. Girls in the eighth grade, currently enrolled in STEM courses, have a higher chance compared to boys with equal ability to have a less favorable iSTEM motivation profile (i.e., cluster 4, the high amotivation profile cluster). Although girls have a higher chance to belong to the high amotivation profile cluster, girls in this cluster score higher on the cognitive STEM test compared to boys. These patterns in motivation profiles and the correlation with STEM test scores were detected in grade eight students. Therefore, motivational aspects related to iSTEM need to be further examined and addressed at an early age. Approaches and motivation styles might have different outcomes depending on gender.

Appendices

Appendix A: Questionnaire factor structure, subscale, and items characteristics

Factor, subscale, and items	M	SD	Factor loading	α
Autonomous motivation				0.91
Intrinsic regulation			0.94	0.92
I try to do my best during STEM classes because STEM is fun	3.22	0.74	0.88
During STEM classes I do my best because STEM interests me	3.24	0.79	0.87
During STEM classes I do my best because I enjoy the lessons	3.11	0.76	0.82
During STEM classes I do my best because I find the lessons interesting. *	3.15	0.76	0.88
Identified regulation			0.97	0.76
During STEM classes I do my best because I want to learn new things. *	3.27	0.71	0.79
During STEM classes I do my best because I find learning STEM important. *	2.90	0.75	0.52
During STEM classes I do my best because STEM is important to me. *	2.82	0.79	0.71
During STEM classes I do my best because I want to do something with STEM later. *	2.90	0.87	0.67
Controlled motivation				0.86
Introjected regulation			0.95	0.80
During STEM classes I do my best because I want others to think I’m smart	1.66	0.72	0.52
During STEM classes I do my best because I’ll feel guilty if I don’t	1.94	0.84	0.77
During STEM classes I do my best because I’ll feel embarrassed if I don’t	1.72	0.79	0.80
During STEM classes I do my best because I want others to think I’m a good student	1.93	0.84	0.74
External regulation			0.73	0.82
During STEM classes I do my best because others expect this from me	2.17	0.91	0.78
During STEM classes I do my best because others say I must	1.71	0.79	0.73
During STEM classes I do my best because it is expected of me	2.49	0.95	0.70
During STEM classes I do my best because others want me to	1.85	0.85	0.74
Amotivation				0.86
Amotivation			1	0.86
I waste my time during STEM lessons	1.52	0.68	0.77
I don’t know why I take STEM classes	1.52	0.74	0.81
I wonder why we get STEM at school	1.53	0.74	0.79
I don’t know why I should do my best during STEM lessons. *	1.49	0.70	0.74

1. Ratio Chi-squared by the degrees of freedom (X²/df) = 3.98; Comparative of FIT Index (CFI) = 0.94; root mean square error of approximation (RMSEA) = 0.065; SRMR = 0.056; α = standardized Cronbach’s alpha; M = average value; SD = standard deviation. *Gender DIF items

Appendix B: Cognitive test information

Test information function cognitive STEM test

Distribution STEM test scores

Appendix C: Cognitive test items, item difficulty and discrimination by gender

Items	Sub-discipline	Practices	Core idea	Gender	Item difficulty	Item discrimination	P (x = 1|z = 0)
Item 1	Biology	Knowledge question	Photosynthesis	Male	0.03	0.25	0.50
				Female	0.21	0.23	0.49
Item 2	Physics/Mathematics	Using mathematics and computational thinking	Volume	Male	5.17	0.21	0.25
				Female	2.01	0.05	0.25
Item 3	Physics/Mathematics	Analyzing, and interpreting data, Using mathematics and computational thinking	State of matter	Male	2.35	0.4	0.28
				Female	2.58	0.37	0.28
Item 4	Physics/Mathematics	Analyzing, and interpreting data, Using mathematics and computational thinking	Optics	Male	2.21	0.02	0.41
				Female	2.09	0.24	0.38
Item 5	Biology	Knowledge question	Micro-organisms	Male	0.4	0.39	0.46
				Female	0.58	0.34	0.45
Item 6	Technology/Engineering	Analyzing, and interpreting data	Logic—Operators	Male	0.19	0.46	0.48
				Female	0.92	0.21	0.45
Item 7	STEM	Analyzing and interpreting data	System of Units	Male	1.27	0.53	0.34
				Female	2.77	0.36	0.27
Item 8	Engineering	Analyzing and interpreting data	Gears	Male	5.12	0.18	0.29
				Female	7.51	0.18	0.21
Item 9	Technology/Engineering	Analyzing and interpreting data, Using mathematics and computational thinking	Electric circuit	Male	1.86	0.51	0.28
				Female	4.64	0.29	0.21
Item 10	Technology/Engineering	Analyzing and interpreting data, Using mathematics and computational thinking	Logic—Operators	Male	3.8	0.13	0.38
				Female	0.2	0.42	0.48*
Item 11	Biology/Mathematics	Analyzing and interpreting data	Body Mass Index	Male	0.04	0.30	0.50
				Female	− 0.86	0.39	0.58
Item 12	Engineering	Analyzing and interpreting data	Schematics	Male	0.05	0.78	0.49
				Female	− 0.09	0.98	0.52
Item 13	Mathematics	Using mathematics and computational thinking	Pythagoras	Male	0.32	0.46	0.46
				Female	0.41	0.54	0.44
Item 14	Information	Analyzing and interpreting data	Schematics	Male	− 0.4	0.64	0.56*
				Female	0.2	0.71	0.46
Item 15	Physics	Analyzing and interpreting data	Temperature	Male	0.32	1.34	0.40
				Female	0.21	1.6	0.42
Item 16	Science	Planning and carrying out investigations	Scientific method	Male	0.03	1.91	0.48
				Female	0.03	1.78	0.48
Item 17	Physics	Asking questions and defining problems	Heat transfer	Male	− 0.15	1.07	0.54
				Female	− 0.27	0.87	0.56
Item 18	Chemistry	Analyzing and interpreting data	Toxic gasses	Male	0.11	1.39	0.46
				Female	− 0.55	1.15	0.65*
Item 19	Biology	Analyzing and interpreting data	Micro-organisms	Male	0.6	0.75	0.39
				Female	− 0.02	0.67	0.50*
Item 20	Physics/Mathematics	Developing and using models	Acceleration	Male	− 0.63	0.94	0.64
				Female	− 0.29	1.16	0.58
Item 21	Engineering/Physics	Analyzing and interpreting data	Isolation materials	Male	0.56	0.76	0.40
				Female	0.77	0.51	0.40
Item 22	Technology/Engineering	Using mathematics and computational thinking	Coding	Male	0.42	0.82	0.42
				Female	0.51	0.75	0.41
Item 23	Engineering/Mathematics	Analyzing and interpreting data, using mathematics	Scale	Male	0.33	0.95	0.42*
				Female	0.82	1.03	0.30

1. *Items that shows significant gender bias

Appendix D: Impact of DIF items on test characteristic curves

Autonomous motivation (5 DIF items)

Controlled motivation (no DIF items)

Amotivation (1 DIF item)