 Research
 Open Access
 Published:
Validation of the Mathematics Motivation Questionnaire (MMQ) for secondary school students
International Journal of STEM Education volume 8, Article number: 52 (2021)
Abstract
Background
Motivation is critical for supporting persistence and achievement in science, technology, engineering, and mathematics (STEM) disciplines. In this study, we focus on the assessment of mathematics motivation among secondary school students. We provide validity and reliability evidence for the Mathematics Motivation Questionnaire (MMQ)—adapted from the Science Motivation Questionnaire designed for college students—using data from 2551 secondary students from seven states across the United States.
Results
Exploratory and confirmatory factor analyses confirmed five latent factors of the MMQ indicated by 19 items: intrinsic value, selfregulation, selfefficacy, utility value, and test anxiety. The nonlinear SEM reliability coefficients of the five constructs ranged from 0.76 to 0.91. To assess criterion validity, analyses using a subset of the data that included students’ mathematics standardized scores (n = 536) indicated that intrinsic value, selfregulation, and selfefficacy were significantly positively correlated with mathematics achievement, whereas test anxiety was significantly negatively correlated with mathematics achievement.
Conclusions
The MMQ provides a reliable, valid, and feasible measure of the specific factors underlying mathematics motivation among secondary students.
Designing instruction that motivates students to learn is a fundamental challenge among educators in science, technology, engineering, and mathematics (STEM) disciplines (Hernandez et al., 2013; Rosenzweig & Wigfield, 2016). Although motivation is important across all disciplines, research suggests mathematics imposes unique motivational barriers, including feelings of anxiety (Dowker et al., 2016) and beliefs that mathematics is not personally interesting or valuable in one’s life (Peterson & Hyde, 2017). Low motivation in mathematics is especially prevalent as students transition to secondary school—a critical time during which students develop their identities as learners (Hogheim & Reber, 2015).
Unfortunately, many secondary school students show declining mathematics motivation and achievement, in part due to differences in the school context and instructional practices, as well as the increased complexity of the learning material (Eccles et al., 1993). Motivating students to learn is critical because motivated students are more likely to invest effort toward mastering the material, employ effective selfregulation strategies, persist in the face of challenges, and demonstrate higher levels of achievement (Pintrich & De Groot, 1990; Renninger & Hidi, 2019). In contrast, unmotivated students tend not to engage in challenging academic tasks or use effortful learning strategies, due to unproductive beliefs they hold about their own capability or the value of the learning material (Wigfield et al., 2016).
Identifying and supporting unmotivated students at critical periods, such as the transition to secondary education, is important for preventing academic downfalls in the years that follow. This is especially crucial in STEM disciplines, for which issues related to equity, persistence, and achievement are of particular interest to STEM educators and researchers and affect the future global STEM workforce (Cromley et al., 2016; Hernandez et al., 2013; Jackson et al., 2021; Jiang et al., 2020; Wang, 2013). Mathematics in particular may serve as critical filter of which students ultimately decide to pursue STEM majors and careers (Watt et al., 2017). Dabney et al. (2012) found that interest in mathematics during middle school was positively associated with later career interest in STEM. Other research has found that adolescents’ beliefs about one’s ability in mathematics predict career attainment in STEM (Blotnicky et al., 2018; Seo et al., 2019). Thus, understanding the factors that contribute to secondary school students’ mathematics motivation can contribute toward increasing student participation in STEM disciplines.
Prior research on academic motivation has produced several theoretical frameworks, each positing distinct yet closely related constructs (Graham & Weiner, 2012). This has created challenges in developing feasible, reliable, and valid measures that capture a range of core constructs important for mathematics achievement (Renninger & Hidi, 2019; Wentzel & Miele, 2016). As Marsh et al. (2019) recently noted, “researchers tend to focus on their preferred measures, sometimes paying relatively little attention to testing how (or if) they differ from other, apparently related constructs” (p. 332). In response, we created the Mathematics Motivation Questionnaire (MMQ) for secondary students, which was adapted from the popular and wellvalidated Science Motivation Questionnaire (SMQ; Glynn et al., 2009) for college students. The MMQ targets students’ beliefs, values, selfregulation strategies, and anxiety toward learning and performing in mathematics. We focused on secondary students because decreases in mathematics motivation at this age are more pronounced compared to other academic domains (Wigfield et al., 2016). The SMQ was developed based on prominent theories of academic motivation to explore the relationships among important motivational factors and science achievement for college students. It has been revised and validated with both science and nonscience majors (Glynn et al, 2009, 2011).
As the MMQ was adapted for use in different contexts and grade levels, it is necessary to explore validity and reliability evidence of the MMQ. Thus, this study examined the construct validity, reliability, and criterion validity evidence of the MMQ applicable for secondary school students to understand the motivational factors underlying learning in mathematics.
Theoretical framework
Motivation is an internal state that initiates and maintains goaldirected behavior (Pintrich, 2003). According to expectancyvalue theory (Wigfield et al., 2016), motivation depends on students’ beliefs about themselves (expectancies) and about the task (values). Expectancies refer to students’ expectancies for success, or the belief in their ability to succeed within a domain. Expectancies for success are closely related to what other theories of motivation refer to as selfefficacy (Marsh et al., 2019; Schunk & DiBenedetto, 2016). For example, selfefficacy items might ask students how confident they are that can understand the concepts taught in a course or that they can turn complete their assignments on time (Zimmerman et al., 1992).
Expectancyvalue theory also distinguishes among three types of values: intrinsic value, utility value, and attainment value (Rosenzweig et al., 2019). Intrinsic value refers to the enjoyment experienced by performing a particular academic task (e.g., “I enjoy doing things in math”); utility value refers to the extent to which an academic task fits within a person’s current or future goals (e.g., “Math is useful for my future”); and attainment value refers to the importance to the individual of performing well on an academic task (e.g., “For me, being good at math is important”, Weidinger et al., 2020). Intrinsic value and utility value are somewhat related to aspects of what selfdetermination theory (Ryan & Deci, 2017) refers to as intrinsic motivation (i.e., acting for internal or personal reasons) and extrinsic motivation (i.e., acting to receive external rewards), respectively. Despite some discrepancies across theoretical constructs, there is consensus that beliefs about oneself and the task are fundamental components of academic motivation.
Students’ beliefs and values are closely connected to their willingness and ability to regulate their own learning. Selfregulation theory (Zimmerman & Labuhn, 2012) refers to a metacognitive system that regulates students’ learning strategies. Selfregulated learners set goals, select learning strategies, structure a supportive learning environment, monitor their performance, and allocate their effort accordingly. For example, the popular Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993) asks students to report the extent to which they relate learning material to what they already know (elaboration strategies), make diagrams or tables to organize the learning material (organizing strategies), or study in a place free of distractions (resource management strategies). Research indicates that use of selfregulated strategies is positively associated with important motivational factors such as selfefficacy (Pintrich & De Groot, 1990; Schunk & DiBenedetto, 2016).
Finally, an important additional consideration is students’ feelings of anxiety, which are particularly problematic for students in mathematics (Dowker et al., 2016; Ramirez et al., 2018). Mathematics anxiety is a negative emotional reaction to learning in mathematics or to the prospect of solving mathematics problems (Ramirez et al., 2018). One component of mathematics anxiety is mathematics test anxiety, or a negative emotional reaction to being evaluated in mathematics (Hembree, 1988; Kazelskis et al., 2000). Students with higher levels of mathematics anxiety or test anxiety generally have negative views of mathematics, show a tendency to avoid mathematics, and exhibit lower mathematics achievement (Dowker et al., 2016).
Limitations of existing mathematics motivation measures
Several measures have been developed to assess the myriad constructs underlying mathematics motivation. However, the existing measures tend to be long, outdated, and assess similar constructs. For example, the Fennema–Sherman Mathematics Attitude Scale (Fennema & Sherman, 1976) is a popular instrument designed to assess nine aspects relevant to secondary students’ mathematics motivation, including attitudes toward success in mathematics, stereotyping of mathematics as a male domain, perceived attitude of mother, father, and teacher towards one as a learner of mathematics, effectance motivation in mathematics, confidence in learning mathematics, and usefulness of mathematics. The Attitude Towards Mathematics Inventory (ATMI; Tapia & Marsh, 2004) has four constructs of enjoyment, motivation, selfconfidence, and value, indicated by 40 items, which has content validity evidence from high school students. Yet these and other popular measures are often not clearly linked to a coherent theoretical framework that specifies distinct motivational constructs.
There are also limitations of the popular MSLQ for assessing mathematics motivation among secondary students. First, the MSLQ is not situated in the context of mathematics learning but for learning in general. Second, the original MSLQ (Pintrich & de Groot, 1990) has 44 items for 5 constructs (intrinsic value, selfefficacy, test anxiety, cognitive strategy use, and selfregulation). Among the 5 constructs of the original MSLQ, the items for cognitive strategy use look similar to the items for selfregulation. Pintrich et al. (1993) came up with the revised and extended version of the MSLQ with 81 items for 15 constructs: 31 items for 6 constructs regarding motivation and 50 items for 9 constructs regarding learning strategies. In sum, the revised MSLQ is (a) not specifically contextualized for motivation in mathematics learning, (b) too long for secondary students, and (c) contains constructs that are not quite relevant to our purpose of study to assess multifaceted mathematics motivation.
Finally, popular measures of mathematics anxiety, such as the Mathematics Anxiety Rating Scale (MARS; Richardson & Suinn, 1972; Suinn & Winston, 2003), the Revised Mathematics Anxiety Scale (RMAS; Alexander & Martray, 1989; Bowd & Brady, 2002), the Mathematics Anxiety Questionnaire (MAQ; Wigfield & Meece, 1988), include items that assess students’ fears of both learning and being evaluated in mathematics (Kazelskis et al., 2000; Pletzer et al., 2016). However, existing mathematics anxiety measures tend to be long and/or lack validity evidence for secondary students.
Taken together, past research indicates mathematics achievement is influenced by (a) beliefs about one’s competence (e.g., selfefficacy); (b) the perceived value of mathematics (e.g., intrinsic value and utility value); (c) selfregulated strategies to learn and perform, and (d) feelings of anxiety toward learning or being assessed in mathematics. However, prior research often only examines one or two factors (i.e., mathematics anxiety or selfregulation) using individual researcherdeveloped or borrowed subscales of measurement instruments that target a single construct (Chamberlin, 2010; Marsh et al., 2019). In addition, the existing measures in mathematics education tended to be too long for secondary students (e.g., the MARS originally had 98 items and was shortened to 30 items). The MMQ will fill this gap as it was designed to measure multifaceted motivational constructs simultaneously for secondary students.
Method
Creating the Mathematics Motivation Questionnaire
We created the initial MMQ by replacing the word “science” with “math” on the SMQ. The SMQ (Glynn et al., 2009) was originally developed to assess six factors underlying student motivation in science: intrinsic motivation, extrinsic motivation, selfdetermination, personal relevance, selfefficacy, and anxiety. The factors chosen for the SMQ were inspired by multiple theories of academic motivation, including expectancyvalue theory and selfdetermination theory. In a series of studies, Glynn et al., (2009, 2011) identified an initial factor structure of the SMQ and established validity evidence with both science and nonscience undergraduates. The initial analysis with nonscience majors identified five factors of the SMQ: intrinsic motivation and personal relevance, selfefficacy and assessment anxiety, selfdetermination, career motivation, and grade motivation. Each of these factors was significantly associated with high school science preparation and college science GPA. Despite this validity evidence, the internal structure of the SMQ did not produce a clear onetoone mapping between naming each factor and its corresponding theoretical construct on the SMQ. For example, some of the factors involved grouping multiple constructs widely viewed as distinct in the literature (e.g., selfefficacy and assessment anxiety).
Therefore, to clarify the theoretical mapping of the items, we assessed the face and content validity of the items based on their degree of correspondence to constructs from motivation and selfregulation theories in mathematics education. As shown in Appendix: Table 9, this resulted in six constructs: intrinsic value (e.g., “I enjoy learning math”), attainment value (e.g., “Earning a good math grade is important to me), selfregulation (e.g., “I use strategies to ensure I learn math well”), selfefficacy (e.g., “I am confident I will do well on math assignments and projects”), utility value (e.g., “The math I learn is relevant to my life”), and test anxiety (e.g., “I am nervous about how I will do on math tests”). Each of the 30 total items asks students to use a fivepoint scale to select “never”, “rarely”, “sometimes”, “usually”, or “always”.
Participants
Participants were 2551 secondary students in grades 7–9 across 25 different public and private schools from seven states in the United States. Participating mathematics and science teachers administered the MMQ to their students during the 2015–2018 school years, near the beginning of the school year for seventh graders and towards the end of the school year for eighth and ninth graders. Missing responses revealed some systematic patterns, such as no responses on the second page of the MMQ. Thus, we excluded missing responses pairwise to maximize the use of available categorical data in the factor analyses, using robust weighted least squares (WLSMV) in Mplus (Muthén & Muthén, 2017).
For criterion validity evidence, we utilized available state standardized mathematics achievement test scores of 536 seventh grade students from seven schools in a single state in the Midwest United States. Table 1 provides demographic characteristics for all students in the sample and the subset of students with standardized test scores.
Data analyses
The fivepoint Likerttype items used in the draft MMQ are categorical and the distributions of item responses were all skewed, mostly negatively (80%), revealing nonnormal distributions. Therefore, WLSMV employed in Mplus 8.4 (Muthén & Muthén, 2017) was applied to obtain parameter estimates for factor analyses with categorical data. First, an exploratory factor analysis (EFA) was conducted by randomly splitting the data in half (n = 1275) to identify the underlying factor structure and irrelevant items that did not fit into any factors in the MMQ. For the EFA, eigenvalues and factor loadings after oblique rotation of GEOMIN were calculated to judge the number of factors and items within each factor. For example, we extracted the number of factors underlying the data based on the number of eigenvalues greater than one using the Kaiser–Guttman rule (Kaiser, 1991) and the point of inflection of the curve in the scree plot (Cattell, 1966). According to Stevens’ (2002) guideline about the relationship between the sample size and cutoff factor loading, we considered items with a loading greater than 0.40 significant for the designated factor. This cutoff suppressed any irrelevant items that did not fit into the designated factor. In addition, if an item loaded onto more than one factor with “salient” secondary factor loading over 0.30 (Brown, 2015, p. 27), that item was excluded.
Second, after identifying the factor structure and relevant items, we conducted confirmatory factor analyses (CFAs) using the other half of the data (n = 1276) to confirm and refine the factor structure identified through the EFA. Based on the fit indexes that Mplus provides, the Chisquare, rootmeansquare error of approximation (RMSEA), comparative fit index (CFI), Tucker–Lewis index (TLI), and standardized root mean square residual (SRMR) were used to judge CFA model fits (Brown, 2015). We attempted various confirmatory factor structure models with the results of the EFA to refine the model fits of the CFAs using modification indices (i.e., specific areas of the model misfit that show items with a discrepancy between the data and the proposed model) until all goodnessoffit indexes resided in the goodfit range. We considered the model fit indexes in the goodfit range when RMSEA is close to 0.06 or below, CFI and TLI values are close to 0.95 or greater, and SRMR is close to 0.08 or below (Brown, 2015).
CFA model respecification was conducted to optimize parsimony and improve interpretability. As we finalized a factor structure and items for the MMQ, we utilized nonlinear SEM methods to calculate the internal consistency reliability coefficient of ω for ordinal data, using Mplus and SAS, following the twostep approaches in Yang and Green (2015). In addition, Cronbach’s α, using SPSS Statistics 24.0 (IBM Corp., 2016) was calculated for comparison with ω, as the coefficient alpha is known for underestimating the reliability evidence of categorical data.
For criterion validity evidence, we calculated Pearson correlation coefficients between the scores averaged from the items loaded for the identified factors and the state standardized mathematics achievement test scores (AERA, APA, & NCME, 2014). We also calculated Pointbiserial coefficients between each of the identified factors and students’ demographic background variables to explore to what extent the identified factors are operationalizable in capturing differences in student motivation factors by demographics. Here, student demographic background variables were coded as dichotomous for gender (0 = male, 1 = female), underrepresented minority (0 = White and Asian, 1 = Others), and socioeconomic status (0 = no free/reduced lunch, 1 = free/reduced lunch).
Results
Exploratory factor analysis
Polychoric correlation coefficients among the 30 items, which are ordered categorical variables, revealed that the coefficients ranged from –0.643 to 0.809, meaning that putative factors identified through EFA are not independent. The negative correlation coefficients were due to items intended to assess mathematics test anxiety, which were worded such that high scores indicate more anxiety. All the aforementioned methods to extract the number of factors indicated the same number of six factors, so we considered it for inclusion in a putative factor structure for the modified MMQ. Table 2 presents factor pattern (i.e., factor loading) and structure coefficients from the sixfactor model.
The exclusion criteria for cutoff factor loading yielded one factor that was poorly defined with less than three items, so we eliminated those two items: Item 3 (“I like to do better than the other students on the math tests”) and Item 12 (“I expect to do as well as or better than other students in the math course”) (Brown, 2015). Note that the two items were dropped because they did not clearly map to a theoretical construct. Item 2 (“The math I learn relates to my personal goals”), Item 23 (“The math I learn is relevant to my life”), and Item 25 (“The math I learn has practical value for me”) were all significantly crossloaded to both Factor 3 (i.e., Utility Value) and Factor 4 (i.e., Intrinsic Value). Item 18 (“I hate taking the math tests”), which was designed to be an indicator of test anxiety, was dropped because even a student without test anxiety may hate taking a math test. Four items (Item 7, Item 16, Item 20, and Item 30) with smaller factor loadings across all six factors than the cutoff of 0.40 were excluded. Those exclusions resulted in 20 items out of 30 that had significant factor loadings onto one of five factors, indicating each item’s unique contribution to one of the factors (see Table 3).
Based on the constructs named in the initial MMQ, we matched the constructs to the factors clustered with a group of items. The five factors were renamed based on the motivation theories and selfregulation: intrinsic value, selfregulation, selfefficacy, utility value, and test anxiety. The overall reliability of the 20item MMQ was Cronbach’s α = 0.86. Each construct included in the MMQ had acceptable Cronbach’s αs ranging from 0.71 to 0.89. All items were worthy of retention because removal of any item did not significantly increase Cronbach’s α for any factor (Field, 2009).
Confirmatory factor analysis modeling
Several CFAs were conducted to confirm and refine the factor structure for the 20item MMQ using the other half of the data (n = 1276). We evaluated each CFA model through three steps: (a) checking the consistency of multiple goodnessoffit indexes and judging the fit of the model to the data; (b) examining localized areas of poor fit; and (c) inspecting parameter estimates, such as factor loadings, factor variances, and residual variances to ensure reliability on each item to the latent factor (Brown, 2015). Even though all items had loadings that met the minimum criteria of 0.40, Model 1 created from the EFA with five factors and 20 items yielded a poor fit (see Model 1 in Table 4). Because a CFA requires more constraints in relationships between items and factors than a model identified though an EFA, we modified the initial factor model by checking items that behaved poorly based on the modification indices (MIs) over 4.0 (Brown, 2015).
The modification indices indicated that test anxiety Item 15 (“I think about how my math grade will affect my overall grade point average”) might be a source of misfits (i.e., potential indicators of error covariances). Item 15 showed the largest modification index (MI) of 947.0, suggesting a free estimate of a parameter loaded to utility value, followed by another MI of 946.8, suggesting another free estimate of a parameter loaded to selfregulation. As indicated by MI, Item 15 might be also true for students who care about their grades, regardless of their test anxiety. Because the purpose of this study was to confirm and refine the factor structure of the MMQ, we did not free the parameters for Item 15 with the source of misfits. Therefore, a CFA with the removal of Item 15 yielded an improved model fit as shown in Model 2 in Table 4.
As presented in Table 4, the fivefactor Model 2 indicated by 19 items showed improved model fit. All model fit indices resided in the goodfitrange. However, some items had large MIs (over 4.0) including anxiety Item 13 (“I worry about failing math tests.”) crossloading on other factors (intrinsic value, selfregulation, selfefficacy, and utility value), indicating areas for improvement in the model fits. Nonetheless, we decided to hold Model 2 as the final model after an examination of the items and their designated constructs revealed they were theoretically meaningful (Brown, 2015). Table 5 presents the parameter estimates of the finalized MMQ with a fivefactor model defined by 19 items (see Appendix: Table 10). Correlation coefficients among the four factors were all significant and ranged from − 0.370 to 0.733 as shown in Table 6, implying no multicollinearity over 0.85 between constructs, indicating “problematic discriminant validity” (Brown, 2015, p. 146).
In Table 7, the five constructs for each factor in Model 2 are defined along with two types of internal consistency reliability evidence of the finalized MMQ with 19 items from n = 1276. The Cronbach’s αs ranged from 0.73 to 0.89, with the overall Cronbach’s α of 0.85. The nonlinear SEM reliability coefficients ranged from 0.76 to 0.91, with the overall ω = 0.90. Therefore, both types of coefficients indicate acceptable internal consistency reliability in participants’ responses on the MMQ (Field, 2009). As expected, the nonlinear SEM reliability coefficients were slightly higher than the Cronbach’s αs (Yang & Green, 2015).
Criterion validity evidence
To assess criterion validity, we used a subset of the data (N = 536) for which we obtained scores on students’ seventhgrade mathematics standardized test. Table 8 presents the correlation coefficients among each of the finalized five MMQ factors, mathematics achievement, and demographic background. Intrinsic value, selfregulation, and selfefficacy were significantly positively correlated with mathematics achievement, and test anxiety was significantly negatively correlated with mathematics achievement. Utility value was not significantly correlated with mathematics achievement. Female students tended to show higher selfregulation and test anxiety than male students. In general, underrepresented minority students tended to show lower intrinsic value, selfefficacy, utility value, and mathematics achievement than majority (i.e., White and Asian) students. Students with a free/reduced lunch status tended to have lower intrinsic value, selfregulation, selfefficacy, and mathematics achievement than their counterparts.
Discussion
This study contributes to our understanding of what motivates students to persist and learn in STEM disciplines (Atit et al., 2020; Jiang et al., 2020; Wang, 2013), particularly the factors underlying mathematics motivation among secondary school students. Secondary school is a critical time at which students establish beliefs about their abilities and interests in STEM (Hogheim & Reber, 2015), and mathematics may impose unique motivational barriers that steer students away from pursuing STEM degrees and careers (Dowker et al., 2016). Mathematics motivation is particularly important to examine because mathematics may serve as a filter of which students pursue and persist in STEM disciplines (Blotnicky et al., 2018; Dabney et al., 2012; Seo et al., 2019; Watt et al., 2017). To capture the multidimensional nature of mathematics motivation, we created the MMQ and examined its psychometric properties. Below we discuss sources of the validity evidence of the MMQ based on Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014).
Validity evidence on the MMQ content
The content of the MMQ is broadly consistent with prominent theories of academic motivation, which emphasizes the multidimensional nature of motivation (Bandura, 2001; Pintrich, 2003; Ryan & Deci, 2017; Wigfield et al., 2016). For example, according to expectancyvalue theory (Wigfield et al., 2016), motivation depends on students’ expectancies for success (closely related to selfefficacy) and their perceived task values (intrinsic, utility, and attainment value). In line with this framework, our analyses revealed distinct factors that align with the constructs of intrinsic value, utility value, and selfefficacy. Note that selfefficacy closely relates to expectancies for success in expectancyvalue theory, yet we decided to retain the selfefficacy label used by Glynn et al. (2009), which was based on social cognitive theory (Bandura, 2001). The factor we defined as intrinsic value also corresponds somewhat to intrinsic motivation in selfdetermination theory (Ryan & Deci, 2017). However, we determined intrinsic value is a more appropriate characterization because a task with intrinsic value may be completed for intrinsic and/or extrinsic reasons (Rosenzweig et al., 2019).
Next, the test anxiety factor is consistent with Glynn et al. (2009) and with past research linking anxiety to mathematics performance (Maloney & Beilock, 2012). The selfregulation factor consisted of items that Glynn et al. (2009) originally grouped as selfdetermination. However, inspection of these items suggests the items do not best reflect selfdetermination, nor is selfdetermination appropriately characterized as a single construct (Ryan & Deci, 2017). According to selfdetermination theory, selfdetermination comprises the three basic needs of competence, autonomy, and relatedness. Considering the items under this construct refer to the investment of effort, the use of learning strategies, and preparation for exams, we reasoned that selfregulation in the use of behavioral learning strategies is a more appropriate characterization of this construct. This is also in line with items labeled selfregulation within the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993).
Validity evidence on the internal structure of the MMQ
Our factor analyses identified five factors underlying mathematics motivation based on 19 items: intrinsic value (3 items), selfregulation (4 items), selfefficacy (4 items), utility value (4 items), and test anxiety (4 items) (see Appendix: Table 10). This factor structure has several distinctions from the SMQ factor structure identified by Glynn et al. (2009), which also identified five factors from 30 items using principal component analysis: intrinsic motivation and personal relevance (10 items), selfefficacy and assessment anxiety (9 items), selfdetermination (4 items), career motivation (2 items), and grade motivation (5 items). Compared to the SMQ, the finalized MMQ presents five constructs that map onto distinct constructs identified by theories of motivation such as expectancyvalue theory. For example, two factors on the SMQ, intrinsic motivation and personal relevance (items 1, 2, 11, 16, 19, 22, 23, 25, 27, and 30) and career motivation (items 10 and 17) were regrouped and combined for two distinctive factors: intrinsic value (items 1, 22, and 27) and utility value (items 10, 11, 17, and 19). Interestingly, our analysis did not identify factors corresponding to grade motivation or attainment value. From our content analysis of the initial MMQ (Appendix: Table 9), we deemed only three items to potentially target attainment value, whereas all other constructs contained at least five items. In our analysis, the three attainment value items did not significantly load together or onto one of the five factors identified in our analysis. Future iterations of the MMQ may benefit from adding more items specifically designed to target attainment value.
Validity evidence of the MMQ on relations to other variables
The criterion validity of the MMQ from the correlation matrix in Table 8 indicated predictive validity evidence in that intrinsic value, selfefficacy and selfregulation of seventhgrade students were positively associated with mathematics achievement, whereas test anxiety was negatively associated with mathematics achievement. Further, while there was no gender difference in mathematics achievement, female students tended to engage in more selfregulation and experience higher text anxiety than male students. This is consistent with prior research on gender differences in selfregulated learning behavior (Weis et al., 2013) and mathematics test anxiety (Dowker et al., 2016). On the other hand, underrepresented minority students tended to show lower intrinsic value, selfefficacy, and utility value, and students with a free/reduced lunch status tended to have lower intrinsic value, selfregulation, and selfefficacy. While the result from the small sample size of the underrepresented minority students might not be generalizable, this result is consistent with prior research showing gaps in motivation among students from underrepresented groups in STEM disciplines (Estrada et al., 2016).
These findings are in line with past research demonstrating the importance of positive selfefficacy beliefs (Parker et al., 2014) and effortful selfregulation strategies (Schunk & Greene, 2017) for mathematics achievement and STEM achievement more broadly (Jiang et al., 2020; Wang, 2013), as well as the detrimental role of feeling anxious towards learning and being assessed in mathematics (Ashcraft, 2002). Past research also suggests female students tend to experience higher levels of mathematics and test anxiety than male students (Devine et al., 2012; Dowker et al., 2016). Interestingly, utility value was not significantly associated with mathematics achievement. One explanation is that items targeting the utility value of mathematics to one’s future career may not be as meaningful yet to middle school or junior high school students. Another explanation is that other criterion measures may better reflect the role of utility value on student motivation. For example, other research indicates that utilityvalue interventions show promise for motivating students to persist or pursue more coursework in STEM (Harackiewicz & Priniski, 2018; Harackiewicz et al., 2012).
Intended use of the MMQ
This paper provides validity and reliability evidence of the MMQ to assess the five constructs with only 19 items. Thus, the MMQ requires a shorter time for secondary students to complete than the existing instruments, like the MSLQ, preventing survey fatigue (e.g., abandoning a survey in the middle of the survey responses). The MMQ contributes to the literature on mathematics and STEM education in several ways. First, the MMQ can serve to diagnose and clarify secondary students’ motivation profile in learning mathematics and understanding their mathematics performance in class. Second, the MMQ can be used as one evaluation tool to assess the effects of motivationbased interventions in STEM (Rosenzweig & Wigfield, 2016). Third, after diagnosing the current status of student motivation, the MMQ can help teachers determine the best approaches to increase student motivation according to their specific motivational profile. Finally, the MMQ can be used to investigate the relationship between students’ mathematics motivation and their STEM achievement, while also considering other plausible factors that may affect students’ performance.
Limitations of the study and future research
Future research should examine a broader diversity of students to establish validity generalization of the MMQ. Our study focused on secondary students, and our validity evidence was constrained to mostly White students in Michigan. The validity of the MMQ should be assessed across other grade levels in mathematics, racial and ethnic groups, and across other states in the United States. Future research should also check for differential item functioning or measurement invariance across subgroups of participants, such as different gender groups and racial or ethnic groups, to ensure the fair use of the MMQ (AERA, APA, & NCME, 2014). Finally, longitudinal work is needed to test how the different motivational constructs identified in the present study relate to key longterm outcomes in mathematics and STEM disciplines more broadly. There is also further need to verify other types of validity evidence that were not explored in this study, including convergent and discriminant validity.
Conclusion
Overall, the finalized MMQ provides a feasible measure for capturing secondary students’ motivation to learn in mathematics, presenting sound validity and reliability evidence and revealing varied motivation by demographic background. In revealing the multidimensional nature of mathematics motivation, the MMQ provides implications for the development of future motivationbased interventions in STEM (Rosenzweig & Wigfield, 2016), such as those designed to promote selfregulation and selfefficacy or to reduce test anxiety.
Availability of data and materials
The datasets generated and analyzed for the current study are available from the corresponding author upon reasonable request.
Abbreviations
 CFA:

Confirmatory factor analysis
 CFI:

Comparative fit index
 EFA:

Exploratory factor analysis
 MI:

Modification index
 MMQ:

Mathematics Motivation Questionnaire
 WLSMV:

Weighted least squares
 RMSEA:

Root mean square error of approximation
 SEM:

Structural equation modeling
 SMQ:

Science Motivation Questionnaire
 SRMR:

Standardized root mean square residual
 STEM:

Science, Technology, Engineering, and Mathematics
 TLI:

Tucker–Lewis index
References
Alexander, L., & Martray, C. (1989). The development of an abbreviated version of the Mathematics Anxiety Rating Scale. Measurement and Evaluation in Counseling and Development, 22, 143–150.
American Educational Research Association [AERA], American Psychological Association [APA], National Council on Measurement in Education [NCME]. (2014). Standards for educational and psychological testing. American Educational Research Association.
Ashcraft, M. H. (2002). Math anxiety: Personal, educational, and cognitive consequences. Current Directions in Psychological Science, 11(5), 181–185.
Atit, K., Power, J. R., Veurink, N., Uttal, D. H., Sorby, S., Panther, G., Msall, C., Fiorella, L., & Carr, M. (2020). Examining the role of spatial skills and mathematics of motivation on middle school achievement. International Journal of STEM Education. https://doi.org/10.1186/s40594020002343
Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1–26.
Blotnicky, K. A., FranzOdendaal, T., French, F., & Joy, P. (2018). A study of the correlation between STEM career knowledge, mathematics selfefficacy, career interests, and career activities on the likelihood of pursuing a STEM career among middle school students. International Journal of STEM Education, 5(1), 1–15.
Bowd, A. D., & Brady, P. H. (2002). Factorial structure of the Revised Mathematics Anxiety Rating Scale for undergraduate education majors. Psychological Reports, 91, 199–200.
Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.
Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.
Chamberlin, S. A. (2010). A review of instruments created to assess affect in mathematics. Journal of Mathematics Education, 3(1), 167–182.
Cromley, J. G., Perez, T., & Kaplan, A. (2016). Undergraduate STEM achievement and retention: Cognitive, motivational, and institutional factors and solutions. Policy Insights from the Behavioral and Brain Sciences, 3(1), 4–11.
Dabney, K. P., Tai, R. H., Almarode, J. T., MillerFriedmann, J. L., Sonnert, G., Sadler, P. M., & Hazari, Z. (2012). Outofschool time science activities and their association with career interest in STEM. International Journal of Science Education, Part B, 2(1), 63–79.
Devine, A., Fawcett, K., Szucs, D., & Dowker, A. (2012). Gender differences in mathematics anxiety and the relation to mathematics performance while controlling for test anxiety. Behavioral and Brain Functions, 8(33), 1–9.
Dowker, A., Sarkar, A., & Looi, C. Y. (2016). Mathematics anxiety: What have we learned in 60 years? Frontiers in Psychology, 7, 508.
Eccles, J. S., Wigfield, A., Midgley, C., Reuman, D., Iver, D. M., & Feldlaufer, H. (1993). Negative effects of traditional middle schools on students’ motivation. The Elementary School Journal, 93(5), 553–574.
Estrada, M., Burnett, M., Campbell, A. G., Campbell, P. B., Denetclaw, W. F., Gutiérrez, C. G., Hurtado, S., John, G. H., Matsui, J., McGee, R., & Okpodu, C. M. (2016). Improving underrepresented minority student persistence in STEM. CBE—Life Sciences Education. https://doi.org/10.1187/cbe.16010038
Fennema, E., & Sherman, J. A. (1976). FennemaSherman mathematics attitudes scales: Instruments designed to measure attitudes toward the learning of mathematics by females and males. Journal for Research in Mathematics Education, 7(5), 324–326.
Field, A. (2009). Discovering statistics using SPSS (3rd ed.). SAGE Publications Ltd.
Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159–1176.
Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science motivation questionnaire: Construct validation with nonscience majors. Journal of Research in Science Teaching, 46(2), 127–146.
Graham, S., & Weiner, B. (2012). Motivation: past, present, and future. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook, Theories, constructs, and critical issues (Vol. 1, pp. 367–397). The American Psychological Association.
Harackiewicz, J. M., & Priniski, S. J. (2018). Improving student outcomes in higher education: The science of targeted intervention. Annual Review of Psychology, 69, 409–435.
Harackiewicz, J. M., Rozek, C. S., Hulleman, C. S., & Hyde, J. S. (2012). Helping parents to motivate adolescents in mathematics and science: An experimental test of a utilityvalue intervention. Psychological Science, 23(8), 899–906.
Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58(1), 47–77.
Hernandez, P. R., Schultz, P., Estrada, M., Woodcock, A., & Chance, R. C. (2013). Sustaining optimal motivation: A longitudinal analysis of interventions to broaden participation of underrepresented students in STEM. Journal of Educational Psychology, 105(1), 89–107.
Hogheim, S., & Reber, R. (2015). Supporting interest of middle school students in mathematics through context personalization and example choice. Contemporary Educational Psychology, 42, 17–25.
IBM Corp. (2016). IBM SPSS Statistics Version 240. IBM Corp.
Jackson, C., MohrSchroeder, M. J., Bush, S. B., Maiorca, C., Roberts, T., Yost, C., & Fowler, A. (2021). Equityoriented conceptual framework for K12 STEM literacy. International Journal of STEM Education. https://doi.org/10.1186/s4059402100294z
Jiang, S., Simpkins, S. D., & Eccles, J. S. (2020). Individuals’ math and science motivation and their subsequent STEM choices and achievement in high school and college: A longitudinal study of gender and college generation status differences. Developmental Psychology, 56(11), 2137–2151.
Kaiser, H. F. (1991). Coefficient alpha for a principal component and the KaiserGuttman rule. Psychological Reports, 68(3), 855–858.
Kazelskis, R., Reeves, C., Kersh, M. E., Bailey, G., Cole, K., Larmon, M., Hall, L., & Holliday, D. C. (2000). Mathematics anxiety and test anxiety: Separate constructs? The Journal of Experimental Education, 68(2), 137–146.
Maloney, E. A., & Beilock, S. L. (2012). Math anxiety: Who has it, why it develops, and how to guard against it. Trends in Cognitive Sciences, 16(8), 404–406.
Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Arens, A. K. (2019). The murky distinction between selfconcept and selfefficacy. Beware the lurking jinglejangle fallacies. Journal of Educational Psychology, 111(2), 331–353.
Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th ed.). Muthén & Muthén.
Parker, P. D., Marsh, H. W., Ciarrochi, J., Marshall, S., & Adjuljabbar, A. S. (2014). Juxtaposing math selfefficacy and selfconcept as predictors of longterm achievement outcomes. Educational Psychology, 34(1), 29–48.
Peterson, J. L., & Hyde, J. S. (2017). Trajectories of selfperceived math ability, utility value and interest across middle school as predictors of high school math performance. Educational Psychology, 37(4), 438–456.
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 4, 667–686.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and selfregulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40.
Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813.
Pletzer, B., Wood, G., Scherndl, T., Kerschbaum, H. H., & Nuerk, H. C. (2016). Components of mathematics anxiety: Factor modeling of the MARS30Brief. Frontiers in Psychology, 7, 91.
Ramirez, G., Shaw, S. T., & Maloney, E. A. (2018). Math anxiety: Past research, promising interventions, and a new interpretation framework. Educational Psychologist, 53(3), 145–164.
Renninger, K. A., & Hidi, S. (Eds.). (2019). The Cambridge handbook of motivation and learning. Cambridge University Press.
Richardson, F. C., & Suinn, R. M. (1972). The Mathematics Anxiety Rating Scale: Psychometric data. Journal of Counseling Psychology, 19, 551–554.
Rosenzweig, E. Q., & Wigfield, A. (2016). STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist, 51(2), 146–163.
Rosenzweig, E. Q., Wigfield, A., & Eccles, J. (2019). Expectancies, values, and its relevance for student motivation and learning. In K. A. Renninger & S. Hidi (Eds.), The Cambridge handbook of motivation and learning (pp. 617–644). Cambridge University Press.
Ryan, R. M., & Deci, E. L. (2017). Selfdetermination theory: Basic psychological needs in motivation, development, and wellness. Guilford Press.
Schunk, D. H., & DiBenedetto, M. K. (2016). Selfefficacy theory in education. In K. R. Wentzel & D. B. Miele (Eds.), Handbook of motivation at school (2nd ed., pp. 34–54). Routledge.
Schunk, D. H., & Greene, J. A. (2017). Handbook of selfregulation of learning and performance (2nd ed.). Routledge.
Seo, E., Shen, Y., & Alfaro, E. C. (2019). Adolescents’ beliefs about math ability and their relations to STEM career attainment: Joint consideration of race/ethnicity and gender. Journal of Youth and Adolescence, 48(2), 306–325.
Stevens, J. P. (2002). Applied multivariate statistics for the social sciences (4th ed.). Erlbaum.
Suinn, R. M., & Winston, E. H. (2003). The Mathematics Anxiety Rating Scale, a brief version: Psychometric data. Psychological Reports, 92(1), 167–173.
Tapia, M., & Marsh, G. (2004). An instrument to measure mathematics attitudes. Academic Exchange Quarterly, 8, 16–21.
Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50(5), 1081–1121.
Watt, H. M., Hyde, J. S., Petersen, J., Morris, Z. A., Rozek, C. S., & Harackiewicz, J. M. (2017). Mathematics—A critical filter for STEMrelated career choices? A longitudinal examination among Australian and US adolescents. Sex Roles, 77(3), 254–271.
Weidinger, A. F., Spinath, B., & Steinmayr, R. (2020). The value of valuing math: Longitudinal links between students’ intrinsic, attainment, and utility values and grades in math. Motivation Science, 6(4), 413–422.
Weis, M., Heikamp, T., & Trommsdorff, G. (2013). Gender differences in school achievement: The role of selfregulation. Frontiers in Psychology, 4, 442.
Wentzel, K. R., & Miele, D. B. (2016). Handbook of motivation at school (2nd ed.). Routledge.
Wigfield, A., & Meece, J. L. (1988). Math anxiety in elementary and secondary school students. Journal of Educational Psychology, 80(2), 210–216.
Wigfield, A., Tonks, S. M., & Klauda, S. L. (2016). Expectancyvalue theory. In K. R. Wentzel & D. B. Miele (Eds.), Handbook of motivation of school (2nd ed., pp. 55–74). Routledge.
Yang, Y., & Green, S. B. (2015). Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items. European Journal of Research Methods for the Behavioral and Social Sciences, 11(1), 23–34.
Zimmerman, B. J., Bandura, A., & MartinezPons, M. (1992). Selfmotivation for academic attainment: The role of selfefficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663–676.
Zimmerman, B. J., & Labuhn, A. S. (2012). Selfregulation of learning: Process approaches to personal development. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook: Theories, constructs, and critical issues (Vol. 1, pp. 399–425). The American Psychological Association.
Acknowledgements
We would like to thank all the teachers and students who participated in the study.
Funding
This research was supported by a grant from the Institute of Education Sciences (Award No. R305A150365).
Author information
Authors and Affiliations
Contributions
LF completed the majority of the writing for the manuscript. SY completed the majority of the data analysis for the manuscript and assisted with writing the manuscript. KA assisted with data analysis and writing the manuscript. JP, GP, SS, DU, and NV also assisted with writing the manuscript. NV helped with teacher recruitment, training, and data collection for this manuscript. SS, DU, and LF are the principal investigators on the grant funding this project and thus were in charge with the design and execution of the study. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendices
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fiorella, L., Yoon, S.Y., Atit, K. et al. Validation of the Mathematics Motivation Questionnaire (MMQ) for secondary school students. IJ STEM Ed 8, 52 (2021). https://doi.org/10.1186/s4059402100307x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s4059402100307x
Keywords
 Mathematics
 Motivation
 Secondary students
 Construct validity
 Criterion validity