Validation of the Mathematics Motivation Questionnaire (MMQ) for secondary school students

Abstract

Background

Motivation is critical for supporting persistence and achievement in science, technology, engineering, and mathematics (STEM) disciplines. In this study, we focus on the assessment of mathematics motivation among secondary school students. We provide validity and reliability evidence for the Mathematics Motivation Questionnaire (MMQ)—adapted from the Science Motivation Questionnaire designed for college students—using data from 2551 secondary students from seven states across the United States.

Results

Exploratory and confirmatory factor analyses confirmed five latent factors of the MMQ indicated by 19 items: intrinsic value, self-regulation, self-efficacy, utility value, and test anxiety. The nonlinear SEM reliability coefficients of the five constructs ranged from 0.76 to 0.91. To assess criterion validity, analyses using a subset of the data that included students’ mathematics standardized scores (n = 536) indicated that intrinsic value, self-regulation, and self-efficacy were significantly positively correlated with mathematics achievement, whereas test anxiety was significantly negatively correlated with mathematics achievement.

Conclusions

The MMQ provides a reliable, valid, and feasible measure of the specific factors underlying mathematics motivation among secondary students.

Designing instruction that motivates students to learn is a fundamental challenge among educators in science, technology, engineering, and mathematics (STEM) disciplines (Hernandez et al., 2013; Rosenzweig & Wigfield, 2016). Although motivation is important across all disciplines, research suggests mathematics imposes unique motivational barriers, including feelings of anxiety (Dowker et al., 2016) and beliefs that mathematics is not personally interesting or valuable in one’s life (Peterson & Hyde, 2017). Low motivation in mathematics is especially prevalent as students transition to secondary school—a critical time during which students develop their identities as learners (Hogheim & Reber, 2015).

Unfortunately, many secondary school students show declining mathematics motivation and achievement, in part due to differences in the school context and instructional practices, as well as the increased complexity of the learning material (Eccles et al., 1993). Motivating students to learn is critical because motivated students are more likely to invest effort toward mastering the material, employ effective self-regulation strategies, persist in the face of challenges, and demonstrate higher levels of achievement (Pintrich & De Groot, 1990; Renninger & Hidi, 2019). In contrast, unmotivated students tend not to engage in challenging academic tasks or use effortful learning strategies, due to unproductive beliefs they hold about their own capability or the value of the learning material (Wigfield et al., 2016).

Identifying and supporting unmotivated students at critical periods, such as the transition to secondary education, is important for preventing academic downfalls in the years that follow. This is especially crucial in STEM disciplines, for which issues related to equity, persistence, and achievement are of particular interest to STEM educators and researchers and affect the future global STEM workforce (Cromley et al., 2016; Hernandez et al., 2013; Jackson et al., 2021; Jiang et al., 2020; Wang, 2013). Mathematics in particular may serve as critical filter of which students ultimately decide to pursue STEM majors and careers (Watt et al., 2017). Dabney et al. (2012) found that interest in mathematics during middle school was positively associated with later career interest in STEM. Other research has found that adolescents’ beliefs about one’s ability in mathematics predict career attainment in STEM (Blotnicky et al., 2018; Seo et al., 2019). Thus, understanding the factors that contribute to secondary school students’ mathematics motivation can contribute toward increasing student participation in STEM disciplines.

Prior research on academic motivation has produced several theoretical frameworks, each positing distinct yet closely related constructs (Graham & Weiner, 2012). This has created challenges in developing feasible, reliable, and valid measures that capture a range of core constructs important for mathematics achievement (Renninger & Hidi, 2019; Wentzel & Miele, 2016). As Marsh et al. (2019) recently noted, “researchers tend to focus on their preferred measures, sometimes paying relatively little attention to testing how (or if) they differ from other, apparently related constructs” (p. 332). In response, we created the Mathematics Motivation Questionnaire (MMQ) for secondary students, which was adapted from the popular and well-validated Science Motivation Questionnaire (SMQ; Glynn et al., 2009) for college students. The MMQ targets students’ beliefs, values, self-regulation strategies, and anxiety toward learning and performing in mathematics. We focused on secondary students because decreases in mathematics motivation at this age are more pronounced compared to other academic domains (Wigfield et al., 2016). The SMQ was developed based on prominent theories of academic motivation to explore the relationships among important motivational factors and science achievement for college students. It has been revised and validated with both science and nonscience majors (Glynn et al, 2009, 2011).

As the MMQ was adapted for use in different contexts and grade levels, it is necessary to explore validity and reliability evidence of the MMQ. Thus, this study examined the construct validity, reliability, and criterion validity evidence of the MMQ applicable for secondary school students to understand the motivational factors underlying learning in mathematics.

Theoretical framework

Motivation is an internal state that initiates and maintains goal-directed behavior (Pintrich, 2003). According to expectancy-value theory (Wigfield et al., 2016), motivation depends on students’ beliefs about themselves (expectancies) and about the task (values). Expectancies refer to students’ expectancies for success, or the belief in their ability to succeed within a domain. Expectancies for success are closely related to what other theories of motivation refer to as self-efficacy (Marsh et al., 2019; Schunk & DiBenedetto, 2016). For example, self-efficacy items might ask students how confident they are that can understand the concepts taught in a course or that they can turn complete their assignments on time (Zimmerman et al., 1992).

Expectancy-value theory also distinguishes among three types of values: intrinsic value, utility value, and attainment value (Rosenzweig et al., 2019). Intrinsic value refers to the enjoyment experienced by performing a particular academic task (e.g., “I enjoy doing things in math”); utility value refers to the extent to which an academic task fits within a person’s current or future goals (e.g., “Math is useful for my future”); and attainment value refers to the importance to the individual of performing well on an academic task (e.g., “For me, being good at math is important”, Weidinger et al., 2020). Intrinsic value and utility value are somewhat related to aspects of what self-determination theory (Ryan & Deci, 2017) refers to as intrinsic motivation (i.e., acting for internal or personal reasons) and extrinsic motivation (i.e., acting to receive external rewards), respectively. Despite some discrepancies across theoretical constructs, there is consensus that beliefs about oneself and the task are fundamental components of academic motivation.

Students’ beliefs and values are closely connected to their willingness and ability to regulate their own learning. Self-regulation theory (Zimmerman & Labuhn, 2012) refers to a metacognitive system that regulates students’ learning strategies. Self-regulated learners set goals, select learning strategies, structure a supportive learning environment, monitor their performance, and allocate their effort accordingly. For example, the popular Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993) asks students to report the extent to which they relate learning material to what they already know (elaboration strategies), make diagrams or tables to organize the learning material (organizing strategies), or study in a place free of distractions (resource management strategies). Research indicates that use of self-regulated strategies is positively associated with important motivational factors such as self-efficacy (Pintrich & De Groot, 1990; Schunk & DiBenedetto, 2016).

Finally, an important additional consideration is students’ feelings of anxiety, which are particularly problematic for students in mathematics (Dowker et al., 2016; Ramirez et al., 2018). Mathematics anxiety is a negative emotional reaction to learning in mathematics or to the prospect of solving mathematics problems (Ramirez et al., 2018). One component of mathematics anxiety is mathematics test anxiety, or a negative emotional reaction to being evaluated in mathematics (Hembree, 1988; Kazelskis et al., 2000). Students with higher levels of mathematics anxiety or test anxiety generally have negative views of mathematics, show a tendency to avoid mathematics, and exhibit lower mathematics achievement (Dowker et al., 2016).

Limitations of existing mathematics motivation measures

Several measures have been developed to assess the myriad constructs underlying mathematics motivation. However, the existing measures tend to be long, outdated, and assess similar constructs. For example, the Fennema–Sherman Mathematics Attitude Scale (Fennema & Sherman, 1976) is a popular instrument designed to assess nine aspects relevant to secondary students’ mathematics motivation, including attitudes toward success in mathematics, stereotyping of mathematics as a male domain, perceived attitude of mother, father, and teacher towards one as a learner of mathematics, effectance motivation in mathematics, confidence in learning mathematics, and usefulness of mathematics. The Attitude Towards Mathematics Inventory (ATMI; Tapia & Marsh, 2004) has four constructs of enjoyment, motivation, self-confidence, and value, indicated by 40 items, which has content validity evidence from high school students. Yet these and other popular measures are often not clearly linked to a coherent theoretical framework that specifies distinct motivational constructs.

There are also limitations of the popular MSLQ for assessing mathematics motivation among secondary students. First, the MSLQ is not situated in the context of mathematics learning but for learning in general. Second, the original MSLQ (Pintrich & de Groot, 1990) has 44 items for 5 constructs (intrinsic value, self-efficacy, test anxiety, cognitive strategy use, and self-regulation). Among the 5 constructs of the original MSLQ, the items for cognitive strategy use look similar to the items for self-regulation. Pintrich et al. (1993) came up with the revised and extended version of the MSLQ with 81 items for 15 constructs: 31 items for 6 constructs regarding motivation and 50 items for 9 constructs regarding learning strategies. In sum, the revised MSLQ is (a) not specifically contextualized for motivation in mathematics learning, (b) too long for secondary students, and (c) contains constructs that are not quite relevant to our purpose of study to assess multifaceted mathematics motivation.

Finally, popular measures of mathematics anxiety, such as the Mathematics Anxiety Rating Scale (MARS; Richardson & Suinn, 1972; Suinn & Winston, 2003), the Revised Mathematics Anxiety Scale (RMAS; Alexander & Martray, 1989; Bowd & Brady, 2002), the Mathematics Anxiety Questionnaire (MAQ; Wigfield & Meece, 1988), include items that assess students’ fears of both learning and being evaluated in mathematics (Kazelskis et al., 2000; Pletzer et al., 2016). However, existing mathematics anxiety measures tend to be long and/or lack validity evidence for secondary students.

Taken together, past research indicates mathematics achievement is influenced by (a) beliefs about one’s competence (e.g., self-efficacy); (b) the perceived value of mathematics (e.g., intrinsic value and utility value); (c) self-regulated strategies to learn and perform, and (d) feelings of anxiety toward learning or being assessed in mathematics. However, prior research often only examines one or two factors (i.e., mathematics anxiety or self-regulation) using individual researcher-developed or borrowed subscales of measurement instruments that target a single construct (Chamberlin, 2010; Marsh et al., 2019). In addition, the existing measures in mathematics education tended to be too long for secondary students (e.g., the MARS originally had 98 items and was shortened to 30 items). The MMQ will fill this gap as it was designed to measure multifaceted motivational constructs simultaneously for secondary students.

Method

Creating the Mathematics Motivation Questionnaire

We created the initial MMQ by replacing the word “science” with “math” on the SMQ. The SMQ (Glynn et al., 2009) was originally developed to assess six factors underlying student motivation in science: intrinsic motivation, extrinsic motivation, self-determination, personal relevance, self-efficacy, and anxiety. The factors chosen for the SMQ were inspired by multiple theories of academic motivation, including expectancy-value theory and self-determination theory. In a series of studies, Glynn et al., (2009, 2011) identified an initial factor structure of the SMQ and established validity evidence with both science and nonscience undergraduates. The initial analysis with nonscience majors identified five factors of the SMQ: intrinsic motivation and personal relevance, self-efficacy and assessment anxiety, self-determination, career motivation, and grade motivation. Each of these factors was significantly associated with high school science preparation and college science GPA. Despite this validity evidence, the internal structure of the SMQ did not produce a clear one-to-one mapping between naming each factor and its corresponding theoretical construct on the SMQ. For example, some of the factors involved grouping multiple constructs widely viewed as distinct in the literature (e.g., self-efficacy and assessment anxiety).

Therefore, to clarify the theoretical mapping of the items, we assessed the face and content validity of the items based on their degree of correspondence to constructs from motivation and self-regulation theories in mathematics education. As shown in Appendix: Table 9, this resulted in six constructs: intrinsic value (e.g., “I enjoy learning math”), attainment value (e.g., “Earning a good math grade is important to me), self-regulation (e.g., “I use strategies to ensure I learn math well”), self-efficacy (e.g., “I am confident I will do well on math assignments and projects”), utility value (e.g., “The math I learn is relevant to my life”), and test anxiety (e.g., “I am nervous about how I will do on math tests”). Each of the 30 total items asks students to use a five-point scale to select “never”, “rarely”, “sometimes”, “usually”, or “always”.

Participants

Participants were 2551 secondary students in grades 7–9 across 25 different public and private schools from seven states in the United States. Participating mathematics and science teachers administered the MMQ to their students during the 2015–2018 school years, near the beginning of the school year for seventh graders and towards the end of the school year for eighth and ninth graders. Missing responses revealed some systematic patterns, such as no responses on the second page of the MMQ. Thus, we excluded missing responses pairwise to maximize the use of available categorical data in the factor analyses, using robust weighted least squares (WLSMV) in Mplus (Muthén & Muthén, 2017).

For criterion validity evidence, we utilized available state standardized mathematics achievement test scores of 536 seventh grade students from seven schools in a single state in the Midwest United States. Table 1 provides demographic characteristics for all students in the sample and the subset of students with standardized test scores.

Table 1 Demographic characteristics of secondary school student participants

Data analyses

The five-point Likert-type items used in the draft MMQ are categorical and the distributions of item responses were all skewed, mostly negatively (80%), revealing non-normal distributions. Therefore, WLSMV employed in Mplus 8.4 (Muthén & Muthén, 2017) was applied to obtain parameter estimates for factor analyses with categorical data. First, an exploratory factor analysis (EFA) was conducted by randomly splitting the data in half (n = 1275) to identify the underlying factor structure and irrelevant items that did not fit into any factors in the MMQ. For the EFA, eigenvalues and factor loadings after oblique rotation of GEOMIN were calculated to judge the number of factors and items within each factor. For example, we extracted the number of factors underlying the data based on the number of eigenvalues greater than one using the Kaiser–Guttman rule (Kaiser, 1991) and the point of inflection of the curve in the scree plot (Cattell, 1966). According to Stevens’ (2002) guideline about the relationship between the sample size and cutoff factor loading, we considered items with a loading greater than 0.40 significant for the designated factor. This cutoff suppressed any irrelevant items that did not fit into the designated factor. In addition, if an item loaded onto more than one factor with “salient” secondary factor loading over 0.30 (Brown, 2015, p. 27), that item was excluded.

Second, after identifying the factor structure and relevant items, we conducted confirmatory factor analyses (CFAs) using the other half of the data (n = 1276) to confirm and refine the factor structure identified through the EFA. Based on the fit indexes that Mplus provides, the Chi-square, root-mean-square error of approximation (RMSEA), comparative fit index (CFI), Tucker–Lewis index (TLI), and standardized root mean square residual (SRMR) were used to judge CFA model fits (Brown, 2015). We attempted various confirmatory factor structure models with the results of the EFA to refine the model fits of the CFAs using modification indices (i.e., specific areas of the model misfit that show items with a discrepancy between the data and the proposed model) until all goodness-of-fit indexes resided in the good-fit range. We considered the model fit indexes in the good-fit range when RMSEA is close to 0.06 or below, CFI and TLI values are close to 0.95 or greater, and SRMR is close to 0.08 or below (Brown, 2015).

CFA model respecification was conducted to optimize parsimony and improve interpretability. As we finalized a factor structure and items for the MMQ, we utilized nonlinear SEM methods to calculate the internal consistency reliability coefficient of ω for ordinal data, using Mplus and SAS, following the two-step approaches in Yang and Green (2015). In addition, Cronbach’s α, using SPSS Statistics 24.0 (IBM Corp., 2016) was calculated for comparison with ω, as the coefficient alpha is known for underestimating the reliability evidence of categorical data.

For criterion validity evidence, we calculated Pearson correlation coefficients between the scores averaged from the items loaded for the identified factors and the state standardized mathematics achievement test scores (AERA, APA, & NCME, 2014). We also calculated Point-biserial coefficients between each of the identified factors and students’ demographic background variables to explore to what extent the identified factors are operationalizable in capturing differences in student motivation factors by demographics. Here, student demographic background variables were coded as dichotomous for gender (0 = male, 1 = female), underrepresented minority (0 = White and Asian, 1 = Others), and socioeconomic status (0 = no free/reduced lunch, 1 = free/reduced lunch).

Results

Exploratory factor analysis

Polychoric correlation coefficients among the 30 items, which are ordered categorical variables, revealed that the coefficients ranged from –0.643 to 0.809, meaning that putative factors identified through EFA are not independent. The negative correlation coefficients were due to items intended to assess mathematics test anxiety, which were worded such that high scores indicate more anxiety. All the aforementioned methods to extract the number of factors indicated the same number of six factors, so we considered it for inclusion in a putative factor structure for the modified MMQ. Table 2 presents factor pattern (i.e., factor loading) and structure coefficients from the six-factor model.

Table 2 Factor pattern and structure coefficients for the six-factor model from EFA

The exclusion criteria for cutoff factor loading yielded one factor that was poorly defined with less than three items, so we eliminated those two items: Item 3 (“I like to do better than the other students on the math tests”) and Item 12 (“I expect to do as well as or better than other students in the math course”) (Brown, 2015). Note that the two items were dropped because they did not clearly map to a theoretical construct. Item 2 (“The math I learn relates to my personal goals”), Item 23 (“The math I learn is relevant to my life”), and Item 25 (“The math I learn has practical value for me”) were all significantly cross-loaded to both Factor 3 (i.e., Utility Value) and Factor 4 (i.e., Intrinsic Value). Item 18 (“I hate taking the math tests”), which was designed to be an indicator of test anxiety, was dropped because even a student without test anxiety may hate taking a math test. Four items (Item 7, Item 16, Item 20, and Item 30) with smaller factor loadings across all six factors than the cutoff of 0.40 were excluded. Those exclusions resulted in 20 items out of 30 that had significant factor loadings onto one of five factors, indicating each item’s unique contribution to one of the factors (see Table 3).

Table 3 Exploratory factor analysis results of the MMQ (n = 1275)

Based on the constructs named in the initial MMQ, we matched the constructs to the factors clustered with a group of items. The five factors were renamed based on the motivation theories and self-regulation: intrinsic value, self-regulation, self-efficacy, utility value, and test anxiety. The overall reliability of the 20-item MMQ was Cronbach’s α = 0.86. Each construct included in the MMQ had acceptable Cronbach’s αs ranging from 0.71 to 0.89. All items were worthy of retention because removal of any item did not significantly increase Cronbach’s α for any factor (Field, 2009).

Confirmatory factor analysis modeling

Several CFAs were conducted to confirm and refine the factor structure for the 20-item MMQ using the other half of the data (n = 1276). We evaluated each CFA model through three steps: (a) checking the consistency of multiple goodness-of-fit indexes and judging the fit of the model to the data; (b) examining localized areas of poor fit; and (c) inspecting parameter estimates, such as factor loadings, factor variances, and residual variances to ensure reliability on each item to the latent factor (Brown, 2015). Even though all items had loadings that met the minimum criteria of 0.40, Model 1 created from the EFA with five factors and 20 items yielded a poor fit (see Model 1 in Table 4). Because a CFA requires more constraints in relationships between items and factors than a model identified though an EFA, we modified the initial factor model by checking items that behaved poorly based on the modification indices (MIs) over 4.0 (Brown, 2015).

Table 4 Confirmatory factor analysis (CFA) models with goodness-of-fit indexes (n = 1276)

The modification indices indicated that test anxiety Item 15 (“I think about how my math grade will affect my overall grade point average”) might be a source of misfits (i.e., potential indicators of error covariances). Item 15 showed the largest modification index (MI) of 947.0, suggesting a free estimate of a parameter loaded to utility value, followed by another MI of 946.8, suggesting another free estimate of a parameter loaded to self-regulation. As indicated by MI, Item 15 might be also true for students who care about their grades, regardless of their test anxiety. Because the purpose of this study was to confirm and refine the factor structure of the MMQ, we did not free the parameters for Item 15 with the source of misfits. Therefore, a CFA with the removal of Item 15 yielded an improved model fit as shown in Model 2 in Table 4.

As presented in Table 4, the five-factor Model 2 indicated by 19 items showed improved model fit. All model fit indices resided in the good-fit-range. However, some items had large MIs (over 4.0) including anxiety Item 13 (“I worry about failing math tests.”) cross-loading on other factors (intrinsic value, self-regulation, self-efficacy, and utility value), indicating areas for improvement in the model fits. Nonetheless, we decided to hold Model 2 as the final model after an examination of the items and their designated constructs revealed they were theoretically meaningful (Brown, 2015). Table 5 presents the parameter estimates of the finalized MMQ with a five-factor model defined by 19 items (see Appendix: Table 10). Correlation coefficients among the four factors were all significant and ranged from − 0.370 to 0.733 as shown in Table 6, implying no multicollinearity over 0.85 between constructs, indicating “problematic discriminant validity” (Brown, 2015, p. 146).

Table 5 Parameter estimates of the final CFA Model (Model 2) with items as listed in Appendix: Table 10

Table 6 Standardized factor correlation coefficients among the five factors

In Table 7, the five constructs for each factor in Model 2 are defined along with two types of internal consistency reliability evidence of the finalized MMQ with 19 items from n = 1276. The Cronbach’s αs ranged from 0.73 to 0.89, with the overall Cronbach’s α of 0.85. The nonlinear SEM reliability coefficients ranged from 0.76 to 0.91, with the overall ω = 0.90. Therefore, both types of coefficients indicate acceptable internal consistency reliability in participants’ responses on the MMQ (Field, 2009). As expected, the nonlinear SEM reliability coefficients were slightly higher than the Cronbach’s αs (Yang & Green, 2015).

Table 7 Definition of the five constructs of the finalized MMQ with reliability evidence (n = 1276)

Criterion validity evidence

To assess criterion validity, we used a subset of the data (N = 536) for which we obtained scores on students’ seventh-grade mathematics standardized test. Table 8 presents the correlation coefficients among each of the finalized five MMQ factors, mathematics achievement, and demographic background. Intrinsic value, self-regulation, and self-efficacy were significantly positively correlated with mathematics achievement, and test anxiety was significantly negatively correlated with mathematics achievement. Utility value was not significantly correlated with mathematics achievement. Female students tended to show higher self-regulation and test anxiety than male students. In general, underrepresented minority students tended to show lower intrinsic value, self-efficacy, utility value, and mathematics achievement than majority (i.e., White and Asian) students. Students with a free/reduced lunch status tended to have lower intrinsic value, self-regulation, self-efficacy, and mathematics achievement than their counterparts.

Table 8 Correlation matrix among the five MMQ constructs, mathematics achievement, and demographic background

Discussion

This study contributes to our understanding of what motivates students to persist and learn in STEM disciplines (Atit et al., 2020; Jiang et al., 2020; Wang, 2013), particularly the factors underlying mathematics motivation among secondary school students. Secondary school is a critical time at which students establish beliefs about their abilities and interests in STEM (Hogheim & Reber, 2015), and mathematics may impose unique motivational barriers that steer students away from pursuing STEM degrees and careers (Dowker et al., 2016). Mathematics motivation is particularly important to examine because mathematics may serve as a filter of which students pursue and persist in STEM disciplines (Blotnicky et al., 2018; Dabney et al., 2012; Seo et al., 2019; Watt et al., 2017). To capture the multidimensional nature of mathematics motivation, we created the MMQ and examined its psychometric properties. Below we discuss sources of the validity evidence of the MMQ based on Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014).

Validity evidence on the MMQ content

The content of the MMQ is broadly consistent with prominent theories of academic motivation, which emphasizes the multidimensional nature of motivation (Bandura, 2001; Pintrich, 2003; Ryan & Deci, 2017; Wigfield et al., 2016). For example, according to expectancy-value theory (Wigfield et al., 2016), motivation depends on students’ expectancies for success (closely related to self-efficacy) and their perceived task values (intrinsic, utility, and attainment value). In line with this framework, our analyses revealed distinct factors that align with the constructs of intrinsic value, utility value, and self-efficacy. Note that self-efficacy closely relates to expectancies for success in expectancy-value theory, yet we decided to retain the self-efficacy label used by Glynn et al. (2009), which was based on social cognitive theory (Bandura, 2001). The factor we defined as intrinsic value also corresponds somewhat to intrinsic motivation in self-determination theory (Ryan & Deci, 2017). However, we determined intrinsic value is a more appropriate characterization because a task with intrinsic value may be completed for intrinsic and/or extrinsic reasons (Rosenzweig et al., 2019).

Next, the test anxiety factor is consistent with Glynn et al. (2009) and with past research linking anxiety to mathematics performance (Maloney & Beilock, 2012). The self-regulation factor consisted of items that Glynn et al. (2009) originally grouped as self-determination. However, inspection of these items suggests the items do not best reflect self-determination, nor is self-determination appropriately characterized as a single construct (Ryan & Deci, 2017). According to self-determination theory, self-determination comprises the three basic needs of competence, autonomy, and relatedness. Considering the items under this construct refer to the investment of effort, the use of learning strategies, and preparation for exams, we reasoned that self-regulation in the use of behavioral learning strategies is a more appropriate characterization of this construct. This is also in line with items labeled self-regulation within the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993).

Validity evidence on the internal structure of the MMQ

Our factor analyses identified five factors underlying mathematics motivation based on 19 items: intrinsic value (3 items), self-regulation (4 items), self-efficacy (4 items), utility value (4 items), and test anxiety (4 items) (see Appendix: Table 10). This factor structure has several distinctions from the SMQ factor structure identified by Glynn et al. (2009), which also identified five factors from 30 items using principal component analysis: intrinsic motivation and personal relevance (10 items), self-efficacy and assessment anxiety (9 items), self-determination (4 items), career motivation (2 items), and grade motivation (5 items). Compared to the SMQ, the finalized MMQ presents five constructs that map onto distinct constructs identified by theories of motivation such as expectancy-value theory. For example, two factors on the SMQ, intrinsic motivation and personal relevance (items 1, 2, 11, 16, 19, 22, 23, 25, 27, and 30) and career motivation (items 10 and 17) were regrouped and combined for two distinctive factors: intrinsic value (items 1, 22, and 27) and utility value (items 10, 11, 17, and 19). Interestingly, our analysis did not identify factors corresponding to grade motivation or attainment value. From our content analysis of the initial MMQ (Appendix: Table 9), we deemed only three items to potentially target attainment value, whereas all other constructs contained at least five items. In our analysis, the three attainment value items did not significantly load together or onto one of the five factors identified in our analysis. Future iterations of the MMQ may benefit from adding more items specifically designed to target attainment value.

Validity evidence of the MMQ on relations to other variables

The criterion validity of the MMQ from the correlation matrix in Table 8 indicated predictive validity evidence in that intrinsic value, self-efficacy and self-regulation of seventh-grade students were positively associated with mathematics achievement, whereas test anxiety was negatively associated with mathematics achievement. Further, while there was no gender difference in mathematics achievement, female students tended to engage in more self-regulation and experience higher text anxiety than male students. This is consistent with prior research on gender differences in self-regulated learning behavior (Weis et al., 2013) and mathematics test anxiety (Dowker et al., 2016). On the other hand, underrepresented minority students tended to show lower intrinsic value, self-efficacy, and utility value, and students with a free/reduced lunch status tended to have lower intrinsic value, self-regulation, and self-efficacy. While the result from the small sample size of the underrepresented minority students might not be generalizable, this result is consistent with prior research showing gaps in motivation among students from underrepresented groups in STEM disciplines (Estrada et al., 2016).

These findings are in line with past research demonstrating the importance of positive self-efficacy beliefs (Parker et al., 2014) and effortful self-regulation strategies (Schunk & Greene, 2017) for mathematics achievement and STEM achievement more broadly (Jiang et al., 2020; Wang, 2013), as well as the detrimental role of feeling anxious towards learning and being assessed in mathematics (Ashcraft, 2002). Past research also suggests female students tend to experience higher levels of mathematics and test anxiety than male students (Devine et al., 2012; Dowker et al., 2016). Interestingly, utility value was not significantly associated with mathematics achievement. One explanation is that items targeting the utility value of mathematics to one’s future career may not be as meaningful yet to middle school or junior high school students. Another explanation is that other criterion measures may better reflect the role of utility value on student motivation. For example, other research indicates that utility-value interventions show promise for motivating students to persist or pursue more coursework in STEM (Harackiewicz & Priniski, 2018; Harackiewicz et al., 2012).

Intended use of the MMQ

This paper provides validity and reliability evidence of the MMQ to assess the five constructs with only 19 items. Thus, the MMQ requires a shorter time for secondary students to complete than the existing instruments, like the MSLQ, preventing survey fatigue (e.g., abandoning a survey in the middle of the survey responses). The MMQ contributes to the literature on mathematics and STEM education in several ways. First, the MMQ can serve to diagnose and clarify secondary students’ motivation profile in learning mathematics and understanding their mathematics performance in class. Second, the MMQ can be used as one evaluation tool to assess the effects of motivation-based interventions in STEM (Rosenzweig & Wigfield, 2016). Third, after diagnosing the current status of student motivation, the MMQ can help teachers determine the best approaches to increase student motivation according to their specific motivational profile. Finally, the MMQ can be used to investigate the relationship between students’ mathematics motivation and their STEM achievement, while also considering other plausible factors that may affect students’ performance.

Limitations of the study and future research

Future research should examine a broader diversity of students to establish validity generalization of the MMQ. Our study focused on secondary students, and our validity evidence was constrained to mostly White students in Michigan. The validity of the MMQ should be assessed across other grade levels in mathematics, racial and ethnic groups, and across other states in the United States. Future research should also check for differential item functioning or measurement invariance across subgroups of participants, such as different gender groups and racial or ethnic groups, to ensure the fair use of the MMQ (AERA, APA, & NCME, 2014). Finally, longitudinal work is needed to test how the different motivational constructs identified in the present study relate to key long-term outcomes in mathematics and STEM disciplines more broadly. There is also further need to verify other types of validity evidence that were not explored in this study, including convergent and discriminant validity.

Conclusion

Overall, the finalized MMQ provides a feasible measure for capturing secondary students’ motivation to learn in mathematics, presenting sound validity and reliability evidence and revealing varied motivation by demographic background. In revealing the multidimensional nature of mathematics motivation, the MMQ provides implications for the development of future motivation-based interventions in STEM (Rosenzweig & Wigfield, 2016), such as those designed to promote self-regulation and self-efficacy or to reduce test anxiety.

Appendices

Appendices

Table 9 Initial version of the Math Motivation Questionnaire (MMQ)

Table 10 The finalized version of MMQ