Skip to main content

Validation of the Mathematics Motivation Questionnaire (MMQ) for secondary school students

Abstract

Background

Motivation is critical for supporting persistence and achievement in science, technology, engineering, and mathematics (STEM) disciplines. In this study, we focus on the assessment of mathematics motivation among secondary school students. We provide validity and reliability evidence for the Mathematics Motivation Questionnaire (MMQ)—adapted from the Science Motivation Questionnaire designed for college students—using data from 2551 secondary students from seven states across the United States.

Results

Exploratory and confirmatory factor analyses confirmed five latent factors of the MMQ indicated by 19 items: intrinsic value, self-regulation, self-efficacy, utility value, and test anxiety. The nonlinear SEM reliability coefficients of the five constructs ranged from 0.76 to 0.91. To assess criterion validity, analyses using a subset of the data that included students’ mathematics standardized scores (n = 536) indicated that intrinsic value, self-regulation, and self-efficacy were significantly positively correlated with mathematics achievement, whereas test anxiety was significantly negatively correlated with mathematics achievement.

Conclusions

The MMQ provides a reliable, valid, and feasible measure of the specific factors underlying mathematics motivation among secondary students.

Designing instruction that motivates students to learn is a fundamental challenge among educators in science, technology, engineering, and mathematics (STEM) disciplines (Hernandez et al., 2013; Rosenzweig & Wigfield, 2016). Although motivation is important across all disciplines, research suggests mathematics imposes unique motivational barriers, including feelings of anxiety (Dowker et al., 2016) and beliefs that mathematics is not personally interesting or valuable in one’s life (Peterson & Hyde, 2017). Low motivation in mathematics is especially prevalent as students transition to secondary school—a critical time during which students develop their identities as learners (Hogheim & Reber, 2015).

Unfortunately, many secondary school students show declining mathematics motivation and achievement, in part due to differences in the school context and instructional practices, as well as the increased complexity of the learning material (Eccles et al., 1993). Motivating students to learn is critical because motivated students are more likely to invest effort toward mastering the material, employ effective self-regulation strategies, persist in the face of challenges, and demonstrate higher levels of achievement (Pintrich & De Groot, 1990; Renninger & Hidi, 2019). In contrast, unmotivated students tend not to engage in challenging academic tasks or use effortful learning strategies, due to unproductive beliefs they hold about their own capability or the value of the learning material (Wigfield et al., 2016).

Identifying and supporting unmotivated students at critical periods, such as the transition to secondary education, is important for preventing academic downfalls in the years that follow. This is especially crucial in STEM disciplines, for which issues related to equity, persistence, and achievement are of particular interest to STEM educators and researchers and affect the future global STEM workforce (Cromley et al., 2016; Hernandez et al., 2013; Jackson et al., 2021; Jiang et al., 2020; Wang, 2013). Mathematics in particular may serve as critical filter of which students ultimately decide to pursue STEM majors and careers (Watt et al., 2017). Dabney et al. (2012) found that interest in mathematics during middle school was positively associated with later career interest in STEM. Other research has found that adolescents’ beliefs about one’s ability in mathematics predict career attainment in STEM (Blotnicky et al., 2018; Seo et al., 2019). Thus, understanding the factors that contribute to secondary school students’ mathematics motivation can contribute toward increasing student participation in STEM disciplines.

Prior research on academic motivation has produced several theoretical frameworks, each positing distinct yet closely related constructs (Graham & Weiner, 2012). This has created challenges in developing feasible, reliable, and valid measures that capture a range of core constructs important for mathematics achievement (Renninger & Hidi, 2019; Wentzel & Miele, 2016). As Marsh et al. (2019) recently noted, “researchers tend to focus on their preferred measures, sometimes paying relatively little attention to testing how (or if) they differ from other, apparently related constructs” (p. 332). In response, we created the Mathematics Motivation Questionnaire (MMQ) for secondary students, which was adapted from the popular and well-validated Science Motivation Questionnaire (SMQ; Glynn et al., 2009) for college students. The MMQ targets students’ beliefs, values, self-regulation strategies, and anxiety toward learning and performing in mathematics. We focused on secondary students because decreases in mathematics motivation at this age are more pronounced compared to other academic domains (Wigfield et al., 2016). The SMQ was developed based on prominent theories of academic motivation to explore the relationships among important motivational factors and science achievement for college students. It has been revised and validated with both science and nonscience majors (Glynn et al, 2009, 2011).

As the MMQ was adapted for use in different contexts and grade levels, it is necessary to explore validity and reliability evidence of the MMQ. Thus, this study examined the construct validity, reliability, and criterion validity evidence of the MMQ applicable for secondary school students to understand the motivational factors underlying learning in mathematics.

Theoretical framework

Motivation is an internal state that initiates and maintains goal-directed behavior (Pintrich, 2003). According to expectancy-value theory (Wigfield et al., 2016), motivation depends on students’ beliefs about themselves (expectancies) and about the task (values). Expectancies refer to students’ expectancies for success, or the belief in their ability to succeed within a domain. Expectancies for success are closely related to what other theories of motivation refer to as self-efficacy (Marsh et al., 2019; Schunk & DiBenedetto, 2016). For example, self-efficacy items might ask students how confident they are that can understand the concepts taught in a course or that they can turn complete their assignments on time (Zimmerman et al., 1992).

Expectancy-value theory also distinguishes among three types of values: intrinsic value, utility value, and attainment value (Rosenzweig et al., 2019). Intrinsic value refers to the enjoyment experienced by performing a particular academic task (e.g., “I enjoy doing things in math”); utility value refers to the extent to which an academic task fits within a person’s current or future goals (e.g., “Math is useful for my future”); and attainment value refers to the importance to the individual of performing well on an academic task (e.g., “For me, being good at math is important”, Weidinger et al., 2020). Intrinsic value and utility value are somewhat related to aspects of what self-determination theory (Ryan & Deci, 2017) refers to as intrinsic motivation (i.e., acting for internal or personal reasons) and extrinsic motivation (i.e., acting to receive external rewards), respectively. Despite some discrepancies across theoretical constructs, there is consensus that beliefs about oneself and the task are fundamental components of academic motivation.

Students’ beliefs and values are closely connected to their willingness and ability to regulate their own learning. Self-regulation theory (Zimmerman & Labuhn, 2012) refers to a metacognitive system that regulates students’ learning strategies. Self-regulated learners set goals, select learning strategies, structure a supportive learning environment, monitor their performance, and allocate their effort accordingly. For example, the popular Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993) asks students to report the extent to which they relate learning material to what they already know (elaboration strategies), make diagrams or tables to organize the learning material (organizing strategies), or study in a place free of distractions (resource management strategies). Research indicates that use of self-regulated strategies is positively associated with important motivational factors such as self-efficacy (Pintrich & De Groot, 1990; Schunk & DiBenedetto, 2016).

Finally, an important additional consideration is students’ feelings of anxiety, which are particularly problematic for students in mathematics (Dowker et al., 2016; Ramirez et al., 2018). Mathematics anxiety is a negative emotional reaction to learning in mathematics or to the prospect of solving mathematics problems (Ramirez et al., 2018). One component of mathematics anxiety is mathematics test anxiety, or a negative emotional reaction to being evaluated in mathematics (Hembree, 1988; Kazelskis et al., 2000). Students with higher levels of mathematics anxiety or test anxiety generally have negative views of mathematics, show a tendency to avoid mathematics, and exhibit lower mathematics achievement (Dowker et al., 2016).

Limitations of existing mathematics motivation measures

Several measures have been developed to assess the myriad constructs underlying mathematics motivation. However, the existing measures tend to be long, outdated, and assess similar constructs. For example, the Fennema–Sherman Mathematics Attitude Scale (Fennema & Sherman, 1976) is a popular instrument designed to assess nine aspects relevant to secondary students’ mathematics motivation, including attitudes toward success in mathematics, stereotyping of mathematics as a male domain, perceived attitude of mother, father, and teacher towards one as a learner of mathematics, effectance motivation in mathematics, confidence in learning mathematics, and usefulness of mathematics. The Attitude Towards Mathematics Inventory (ATMI; Tapia & Marsh, 2004) has four constructs of enjoyment, motivation, self-confidence, and value, indicated by 40 items, which has content validity evidence from high school students. Yet these and other popular measures are often not clearly linked to a coherent theoretical framework that specifies distinct motivational constructs.

There are also limitations of the popular MSLQ for assessing mathematics motivation among secondary students. First, the MSLQ is not situated in the context of mathematics learning but for learning in general. Second, the original MSLQ (Pintrich & de Groot, 1990) has 44 items for 5 constructs (intrinsic value, self-efficacy, test anxiety, cognitive strategy use, and self-regulation). Among the 5 constructs of the original MSLQ, the items for cognitive strategy use look similar to the items for self-regulation. Pintrich et al. (1993) came up with the revised and extended version of the MSLQ with 81 items for 15 constructs: 31 items for 6 constructs regarding motivation and 50 items for 9 constructs regarding learning strategies. In sum, the revised MSLQ is (a) not specifically contextualized for motivation in mathematics learning, (b) too long for secondary students, and (c) contains constructs that are not quite relevant to our purpose of study to assess multifaceted mathematics motivation.

Finally, popular measures of mathematics anxiety, such as the Mathematics Anxiety Rating Scale (MARS; Richardson & Suinn, 1972; Suinn & Winston, 2003), the Revised Mathematics Anxiety Scale (RMAS; Alexander & Martray, 1989; Bowd & Brady, 2002), the Mathematics Anxiety Questionnaire (MAQ; Wigfield & Meece, 1988), include items that assess students’ fears of both learning and being evaluated in mathematics (Kazelskis et al., 2000; Pletzer et al., 2016). However, existing mathematics anxiety measures tend to be long and/or lack validity evidence for secondary students.

Taken together, past research indicates mathematics achievement is influenced by (a) beliefs about one’s competence (e.g., self-efficacy); (b) the perceived value of mathematics (e.g., intrinsic value and utility value); (c) self-regulated strategies to learn and perform, and (d) feelings of anxiety toward learning or being assessed in mathematics. However, prior research often only examines one or two factors (i.e., mathematics anxiety or self-regulation) using individual researcher-developed or borrowed subscales of measurement instruments that target a single construct (Chamberlin, 2010; Marsh et al., 2019). In addition, the existing measures in mathematics education tended to be too long for secondary students (e.g., the MARS originally had 98 items and was shortened to 30 items). The MMQ will fill this gap as it was designed to measure multifaceted motivational constructs simultaneously for secondary students.

Method

Creating the Mathematics Motivation Questionnaire

We created the initial MMQ by replacing the word “science” with “math” on the SMQ. The SMQ (Glynn et al., 2009) was originally developed to assess six factors underlying student motivation in science: intrinsic motivation, extrinsic motivation, self-determination, personal relevance, self-efficacy, and anxiety. The factors chosen for the SMQ were inspired by multiple theories of academic motivation, including expectancy-value theory and self-determination theory. In a series of studies, Glynn et al., (2009, 2011) identified an initial factor structure of the SMQ and established validity evidence with both science and nonscience undergraduates. The initial analysis with nonscience majors identified five factors of the SMQ: intrinsic motivation and personal relevance, self-efficacy and assessment anxiety, self-determination, career motivation, and grade motivation. Each of these factors was significantly associated with high school science preparation and college science GPA. Despite this validity evidence, the internal structure of the SMQ did not produce a clear one-to-one mapping between naming each factor and its corresponding theoretical construct on the SMQ. For example, some of the factors involved grouping multiple constructs widely viewed as distinct in the literature (e.g., self-efficacy and assessment anxiety).

Therefore, to clarify the theoretical mapping of the items, we assessed the face and content validity of the items based on their degree of correspondence to constructs from motivation and self-regulation theories in mathematics education. As shown in Appendix: Table 9, this resulted in six constructs: intrinsic value (e.g., “I enjoy learning math”), attainment value (e.g., “Earning a good math grade is important to me), self-regulation (e.g., “I use strategies to ensure I learn math well”), self-efficacy (e.g., “I am confident I will do well on math assignments and projects”), utility value (e.g., “The math I learn is relevant to my life”), and test anxiety (e.g., “I am nervous about how I will do on math tests”). Each of the 30 total items asks students to use a five-point scale to select “never”, “rarely”, “sometimes”, “usually”, or “always”.

Participants

Participants were 2551 secondary students in grades 7–9 across 25 different public and private schools from seven states in the United States. Participating mathematics and science teachers administered the MMQ to their students during the 2015–2018 school years, near the beginning of the school year for seventh graders and towards the end of the school year for eighth and ninth graders. Missing responses revealed some systematic patterns, such as no responses on the second page of the MMQ. Thus, we excluded missing responses pairwise to maximize the use of available categorical data in the factor analyses, using robust weighted least squares (WLSMV) in Mplus (Muthén & Muthén, 2017).

For criterion validity evidence, we utilized available state standardized mathematics achievement test scores of 536 seventh grade students from seven schools in a single state in the Midwest United States. Table 1 provides demographic characteristics for all students in the sample and the subset of students with standardized test scores.

Table 1 Demographic characteristics of secondary school student participants

Data analyses

The five-point Likert-type items used in the draft MMQ are categorical and the distributions of item responses were all skewed, mostly negatively (80%), revealing non-normal distributions. Therefore, WLSMV employed in Mplus 8.4 (Muthén & Muthén, 2017) was applied to obtain parameter estimates for factor analyses with categorical data. First, an exploratory factor analysis (EFA) was conducted by randomly splitting the data in half (n = 1275) to identify the underlying factor structure and irrelevant items that did not fit into any factors in the MMQ. For the EFA, eigenvalues and factor loadings after oblique rotation of GEOMIN were calculated to judge the number of factors and items within each factor. For example, we extracted the number of factors underlying the data based on the number of eigenvalues greater than one using the Kaiser–Guttman rule (Kaiser, 1991) and the point of inflection of the curve in the scree plot (Cattell, 1966). According to Stevens’ (2002) guideline about the relationship between the sample size and cutoff factor loading, we considered items with a loading greater than 0.40 significant for the designated factor. This cutoff suppressed any irrelevant items that did not fit into the designated factor. In addition, if an item loaded onto more than one factor with “salient” secondary factor loading over 0.30 (Brown, 2015, p. 27), that item was excluded.

Second, after identifying the factor structure and relevant items, we conducted confirmatory factor analyses (CFAs) using the other half of the data (n = 1276) to confirm and refine the factor structure identified through the EFA. Based on the fit indexes that Mplus provides, the Chi-square, root-mean-square error of approximation (RMSEA), comparative fit index (CFI), Tucker–Lewis index (TLI), and standardized root mean square residual (SRMR) were used to judge CFA model fits (Brown, 2015). We attempted various confirmatory factor structure models with the results of the EFA to refine the model fits of the CFAs using modification indices (i.e., specific areas of the model misfit that show items with a discrepancy between the data and the proposed model) until all goodness-of-fit indexes resided in the good-fit range. We considered the model fit indexes in the good-fit range when RMSEA is close to 0.06 or below, CFI and TLI values are close to 0.95 or greater, and SRMR is close to 0.08 or below (Brown, 2015).

CFA model respecification was conducted to optimize parsimony and improve interpretability. As we finalized a factor structure and items for the MMQ, we utilized nonlinear SEM methods to calculate the internal consistency reliability coefficient of ω for ordinal data, using Mplus and SAS, following the two-step approaches in Yang and Green (2015). In addition, Cronbach’s α, using SPSS Statistics 24.0 (IBM Corp., 2016) was calculated for comparison with ω, as the coefficient alpha is known for underestimating the reliability evidence of categorical data.

For criterion validity evidence, we calculated Pearson correlation coefficients between the scores averaged from the items loaded for the identified factors and the state standardized mathematics achievement test scores (AERA, APA, & NCME, 2014). We also calculated Point-biserial coefficients between each of the identified factors and students’ demographic background variables to explore to what extent the identified factors are operationalizable in capturing differences in student motivation factors by demographics. Here, student demographic background variables were coded as dichotomous for gender (0 = male, 1 = female), underrepresented minority (0 = White and Asian, 1 = Others), and socioeconomic status (0 = no free/reduced lunch, 1 = free/reduced lunch).

Results

Exploratory factor analysis

Polychoric correlation coefficients among the 30 items, which are ordered categorical variables, revealed that the coefficients ranged from –0.643 to 0.809, meaning that putative factors identified through EFA are not independent. The negative correlation coefficients were due to items intended to assess mathematics test anxiety, which were worded such that high scores indicate more anxiety. All the aforementioned methods to extract the number of factors indicated the same number of six factors, so we considered it for inclusion in a putative factor structure for the modified MMQ. Table 2 presents factor pattern (i.e., factor loading) and structure coefficients from the six-factor model.

Table 2 Factor pattern and structure coefficients for the six-factor model from EFA

The exclusion criteria for cutoff factor loading yielded one factor that was poorly defined with less than three items, so we eliminated those two items: Item 3 (“I like to do better than the other students on the math tests”) and Item 12 (“I expect to do as well as or better than other students in the math course”) (Brown, 2015). Note that the two items were dropped because they did not clearly map to a theoretical construct. Item 2 (“The math I learn relates to my personal goals”), Item 23 (“The math I learn is relevant to my life”), and Item 25 (“The math I learn has practical value for me”) were all significantly cross-loaded to both Factor 3 (i.e., Utility Value) and Factor 4 (i.e., Intrinsic Value). Item 18 (“I hate taking the math tests”), which was designed to be an indicator of test anxiety, was dropped because even a student without test anxiety may hate taking a math test. Four items (Item 7, Item 16, Item 20, and Item 30) with smaller factor loadings across all six factors than the cutoff of 0.40 were excluded. Those exclusions resulted in 20 items out of 30 that had significant factor loadings onto one of five factors, indicating each item’s unique contribution to one of the factors (see Table 3).

Table 3 Exploratory factor analysis results of the MMQ (n = 1275)

Based on the constructs named in the initial MMQ, we matched the constructs to the factors clustered with a group of items. The five factors were renamed based on the motivation theories and self-regulation: intrinsic value, self-regulation, self-efficacy, utility value, and test anxiety. The overall reliability of the 20-item MMQ was Cronbach’s α = 0.86. Each construct included in the MMQ had acceptable Cronbach’s αs ranging from 0.71 to 0.89. All items were worthy of retention because removal of any item did not significantly increase Cronbach’s α for any factor (Field, 2009).

Confirmatory factor analysis modeling

Several CFAs were conducted to confirm and refine the factor structure for the 20-item MMQ using the other half of the data (n = 1276). We evaluated each CFA model through three steps: (a) checking the consistency of multiple goodness-of-fit indexes and judging the fit of the model to the data; (b) examining localized areas of poor fit; and (c) inspecting parameter estimates, such as factor loadings, factor variances, and residual variances to ensure reliability on each item to the latent factor (Brown, 2015). Even though all items had loadings that met the minimum criteria of 0.40, Model 1 created from the EFA with five factors and 20 items yielded a poor fit (see Model 1 in Table 4). Because a CFA requires more constraints in relationships between items and factors than a model identified though an EFA, we modified the initial factor model by checking items that behaved poorly based on the modification indices (MIs) over 4.0 (Brown, 2015).

Table 4 Confirmatory factor analysis (CFA) models with goodness-of-fit indexes (n = 1276)

The modification indices indicated that test anxiety Item 15 (“I think about how my math grade will affect my overall grade point average”) might be a source of misfits (i.e., potential indicators of error covariances). Item 15 showed the largest modification index (MI) of 947.0, suggesting a free estimate of a parameter loaded to utility value, followed by another MI of 946.8, suggesting another free estimate of a parameter loaded to self-regulation. As indicated by MI, Item 15 might be also true for students who care about their grades, regardless of their test anxiety. Because the purpose of this study was to confirm and refine the factor structure of the MMQ, we did not free the parameters for Item 15 with the source of misfits. Therefore, a CFA with the removal of Item 15 yielded an improved model fit as shown in Model 2 in Table 4.

As presented in Table 4, the five-factor Model 2 indicated by 19 items showed improved model fit. All model fit indices resided in the good-fit-range. However, some items had large MIs (over 4.0) including anxiety Item 13 (“I worry about failing math tests.”) cross-loading on other factors (intrinsic value, self-regulation, self-efficacy, and utility value), indicating areas for improvement in the model fits. Nonetheless, we decided to hold Model 2 as the final model after an examination of the items and their designated constructs revealed they were theoretically meaningful (Brown, 2015). Table 5 presents the parameter estimates of the finalized MMQ with a five-factor model defined by 19 items (see Appendix: Table 10). Correlation coefficients among the four factors were all significant and ranged from − 0.370 to 0.733 as shown in Table 6, implying no multicollinearity over 0.85 between constructs, indicating “problematic discriminant validity” (Brown, 2015, p. 146).

Table 5 Parameter estimates of the final CFA Model (Model 2) with items as listed in Appendix: Table 10
Table 6 Standardized factor correlation coefficients among the five factors

In Table 7, the five constructs for each factor in Model 2 are defined along with two types of internal consistency reliability evidence of the finalized MMQ with 19 items from n = 1276. The Cronbach’s αs ranged from 0.73 to 0.89, with the overall Cronbach’s α of 0.85. The nonlinear SEM reliability coefficients ranged from 0.76 to 0.91, with the overall ω = 0.90. Therefore, both types of coefficients indicate acceptable internal consistency reliability in participants’ responses on the MMQ (Field, 2009). As expected, the nonlinear SEM reliability coefficients were slightly higher than the Cronbach’s αs (Yang & Green, 2015).

Table 7 Definition of the five constructs of the finalized MMQ with reliability evidence (n = 1276)

Criterion validity evidence

To assess criterion validity, we used a subset of the data (N = 536) for which we obtained scores on students’ seventh-grade mathematics standardized test. Table 8 presents the correlation coefficients among each of the finalized five MMQ factors, mathematics achievement, and demographic background. Intrinsic value, self-regulation, and self-efficacy were significantly positively correlated with mathematics achievement, and test anxiety was significantly negatively correlated with mathematics achievement. Utility value was not significantly correlated with mathematics achievement. Female students tended to show higher self-regulation and test anxiety than male students. In general, underrepresented minority students tended to show lower intrinsic value, self-efficacy, utility value, and mathematics achievement than majority (i.e., White and Asian) students. Students with a free/reduced lunch status tended to have lower intrinsic value, self-regulation, self-efficacy, and mathematics achievement than their counterparts.

Table 8 Correlation matrix among the five MMQ constructs, mathematics achievement, and demographic background

Discussion

This study contributes to our understanding of what motivates students to persist and learn in STEM disciplines (Atit et al., 2020; Jiang et al., 2020; Wang, 2013), particularly the factors underlying mathematics motivation among secondary school students. Secondary school is a critical time at which students establish beliefs about their abilities and interests in STEM (Hogheim & Reber, 2015), and mathematics may impose unique motivational barriers that steer students away from pursuing STEM degrees and careers (Dowker et al., 2016). Mathematics motivation is particularly important to examine because mathematics may serve as a filter of which students pursue and persist in STEM disciplines (Blotnicky et al., 2018; Dabney et al., 2012; Seo et al., 2019; Watt et al., 2017). To capture the multidimensional nature of mathematics motivation, we created the MMQ and examined its psychometric properties. Below we discuss sources of the validity evidence of the MMQ based on Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014).

Validity evidence on the MMQ content

The content of the MMQ is broadly consistent with prominent theories of academic motivation, which emphasizes the multidimensional nature of motivation (Bandura, 2001; Pintrich, 2003; Ryan & Deci, 2017; Wigfield et al., 2016). For example, according to expectancy-value theory (Wigfield et al., 2016), motivation depends on students’ expectancies for success (closely related to self-efficacy) and their perceived task values (intrinsic, utility, and attainment value). In line with this framework, our analyses revealed distinct factors that align with the constructs of intrinsic value, utility value, and self-efficacy. Note that self-efficacy closely relates to expectancies for success in expectancy-value theory, yet we decided to retain the self-efficacy label used by Glynn et al. (2009), which was based on social cognitive theory (Bandura, 2001). The factor we defined as intrinsic value also corresponds somewhat to intrinsic motivation in self-determination theory (Ryan & Deci, 2017). However, we determined intrinsic value is a more appropriate characterization because a task with intrinsic value may be completed for intrinsic and/or extrinsic reasons (Rosenzweig et al., 2019).

Next, the test anxiety factor is consistent with Glynn et al. (2009) and with past research linking anxiety to mathematics performance (Maloney & Beilock, 2012). The self-regulation factor consisted of items that Glynn et al. (2009) originally grouped as self-determination. However, inspection of these items suggests the items do not best reflect self-determination, nor is self-determination appropriately characterized as a single construct (Ryan & Deci, 2017). According to self-determination theory, self-determination comprises the three basic needs of competence, autonomy, and relatedness. Considering the items under this construct refer to the investment of effort, the use of learning strategies, and preparation for exams, we reasoned that self-regulation in the use of behavioral learning strategies is a more appropriate characterization of this construct. This is also in line with items labeled self-regulation within the Motivated Strategies for Learning Questionnaire (MSLQ; Pintrich et al., 1993).

Validity evidence on the internal structure of the MMQ

Our factor analyses identified five factors underlying mathematics motivation based on 19 items: intrinsic value (3 items), self-regulation (4 items), self-efficacy (4 items), utility value (4 items), and test anxiety (4 items) (see Appendix: Table 10). This factor structure has several distinctions from the SMQ factor structure identified by Glynn et al. (2009), which also identified five factors from 30 items using principal component analysis: intrinsic motivation and personal relevance (10 items), self-efficacy and assessment anxiety (9 items), self-determination (4 items), career motivation (2 items), and grade motivation (5 items). Compared to the SMQ, the finalized MMQ presents five constructs that map onto distinct constructs identified by theories of motivation such as expectancy-value theory. For example, two factors on the SMQ, intrinsic motivation and personal relevance (items 1, 2, 11, 16, 19, 22, 23, 25, 27, and 30) and career motivation (items 10 and 17) were regrouped and combined for two distinctive factors: intrinsic value (items 1, 22, and 27) and utility value (items 10, 11, 17, and 19). Interestingly, our analysis did not identify factors corresponding to grade motivation or attainment value. From our content analysis of the initial MMQ (Appendix: Table 9), we deemed only three items to potentially target attainment value, whereas all other constructs contained at least five items. In our analysis, the three attainment value items did not significantly load together or onto one of the five factors identified in our analysis. Future iterations of the MMQ may benefit from adding more items specifically designed to target attainment value.

Validity evidence of the MMQ on relations to other variables

The criterion validity of the MMQ from the correlation matrix in Table 8 indicated predictive validity evidence in that intrinsic value, self-efficacy and self-regulation of seventh-grade students were positively associated with mathematics achievement, whereas test anxiety was negatively associated with mathematics achievement. Further, while there was no gender difference in mathematics achievement, female students tended to engage in more self-regulation and experience higher text anxiety than male students. This is consistent with prior research on gender differences in self-regulated learning behavior (Weis et al., 2013) and mathematics test anxiety (Dowker et al., 2016). On the other hand, underrepresented minority students tended to show lower intrinsic value, self-efficacy, and utility value, and students with a free/reduced lunch status tended to have lower intrinsic value, self-regulation, and self-efficacy. While the result from the small sample size of the underrepresented minority students might not be generalizable, this result is consistent with prior research showing gaps in motivation among students from underrepresented groups in STEM disciplines (Estrada et al., 2016).

These findings are in line with past research demonstrating the importance of positive self-efficacy beliefs (Parker et al., 2014) and effortful self-regulation strategies (Schunk & Greene, 2017) for mathematics achievement and STEM achievement more broadly (Jiang et al., 2020; Wang, 2013), as well as the detrimental role of feeling anxious towards learning and being assessed in mathematics (Ashcraft, 2002). Past research also suggests female students tend to experience higher levels of mathematics and test anxiety than male students (Devine et al., 2012; Dowker et al., 2016). Interestingly, utility value was not significantly associated with mathematics achievement. One explanation is that items targeting the utility value of mathematics to one’s future career may not be as meaningful yet to middle school or junior high school students. Another explanation is that other criterion measures may better reflect the role of utility value on student motivation. For example, other research indicates that utility-value interventions show promise for motivating students to persist or pursue more coursework in STEM (Harackiewicz & Priniski, 2018; Harackiewicz et al., 2012).

Intended use of the MMQ

This paper provides validity and reliability evidence of the MMQ to assess the five constructs with only 19 items. Thus, the MMQ requires a shorter time for secondary students to complete than the existing instruments, like the MSLQ, preventing survey fatigue (e.g., abandoning a survey in the middle of the survey responses). The MMQ contributes to the literature on mathematics and STEM education in several ways. First, the MMQ can serve to diagnose and clarify secondary students’ motivation profile in learning mathematics and understanding their mathematics performance in class. Second, the MMQ can be used as one evaluation tool to assess the effects of motivation-based interventions in STEM (Rosenzweig & Wigfield, 2016). Third, after diagnosing the current status of student motivation, the MMQ can help teachers determine the best approaches to increase student motivation according to their specific motivational profile. Finally, the MMQ can be used to investigate the relationship between students’ mathematics motivation and their STEM achievement, while also considering other plausible factors that may affect students’ performance.

Limitations of the study and future research

Future research should examine a broader diversity of students to establish validity generalization of the MMQ. Our study focused on secondary students, and our validity evidence was constrained to mostly White students in Michigan. The validity of the MMQ should be assessed across other grade levels in mathematics, racial and ethnic groups, and across other states in the United States. Future research should also check for differential item functioning or measurement invariance across subgroups of participants, such as different gender groups and racial or ethnic groups, to ensure the fair use of the MMQ (AERA, APA, & NCME, 2014). Finally, longitudinal work is needed to test how the different motivational constructs identified in the present study relate to key long-term outcomes in mathematics and STEM disciplines more broadly. There is also further need to verify other types of validity evidence that were not explored in this study, including convergent and discriminant validity.

Conclusion

Overall, the finalized MMQ provides a feasible measure for capturing secondary students’ motivation to learn in mathematics, presenting sound validity and reliability evidence and revealing varied motivation by demographic background. In revealing the multidimensional nature of mathematics motivation, the MMQ provides implications for the development of future motivation-based interventions in STEM (Rosenzweig & Wigfield, 2016), such as those designed to promote self-regulation and self-efficacy or to reduce test anxiety.

Availability of data and materials

The datasets generated and analyzed for the current study are available from the corresponding author upon reasonable request.

Abbreviations

CFA:

Confirmatory factor analysis

CFI:

Comparative fit index

EFA:

Exploratory factor analysis

MI:

Modification index

MMQ:

Mathematics Motivation Questionnaire

WLSMV:

Weighted least squares

RMSEA:

Root mean square error of approximation

SEM:

Structural equation modeling

SMQ:

Science Motivation Questionnaire

SRMR:

Standardized root mean square residual

STEM:

Science, Technology, Engineering, and Mathematics

TLI:

Tucker–Lewis index

References

  1. Alexander, L., & Martray, C. (1989). The development of an abbreviated version of the Mathematics Anxiety Rating Scale. Measurement and Evaluation in Counseling and Development, 22, 143–150.

    Article  Google Scholar 

  2. American Educational Research Association [AERA], American Psychological Association [APA], National Council on Measurement in Education [NCME]. (2014). Standards for educational and psychological testing. American Educational Research Association.

    Google Scholar 

  3. Ashcraft, M. H. (2002). Math anxiety: Personal, educational, and cognitive consequences. Current Directions in Psychological Science, 11(5), 181–185.

    Article  Google Scholar 

  4. Atit, K., Power, J. R., Veurink, N., Uttal, D. H., Sorby, S., Panther, G., Msall, C., Fiorella, L., & Carr, M. (2020). Examining the role of spatial skills and mathematics of motivation on middle school achievement. International Journal of STEM Education. https://doi.org/10.1186/s40594-020-00234-3

    Article  Google Scholar 

  5. Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52(1), 1–26.

    Article  Google Scholar 

  6. Blotnicky, K. A., Franz-Odendaal, T., French, F., & Joy, P. (2018). A study of the correlation between STEM career knowledge, mathematics self-efficacy, career interests, and career activities on the likelihood of pursuing a STEM career among middle school students. International Journal of STEM Education, 5(1), 1–15.

    Article  Google Scholar 

  7. Bowd, A. D., & Brady, P. H. (2002). Factorial structure of the Revised Mathematics Anxiety Rating Scale for undergraduate education majors. Psychological Reports, 91, 199–200.

    Article  Google Scholar 

  8. Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). Guilford Press.

    Google Scholar 

  9. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.

    Article  Google Scholar 

  10. Chamberlin, S. A. (2010). A review of instruments created to assess affect in mathematics. Journal of Mathematics Education, 3(1), 167–182.

    Google Scholar 

  11. Cromley, J. G., Perez, T., & Kaplan, A. (2016). Undergraduate STEM achievement and retention: Cognitive, motivational, and institutional factors and solutions. Policy Insights from the Behavioral and Brain Sciences, 3(1), 4–11.

    Article  Google Scholar 

  12. Dabney, K. P., Tai, R. H., Almarode, J. T., Miller-Friedmann, J. L., Sonnert, G., Sadler, P. M., & Hazari, Z. (2012). Out-of-school time science activities and their association with career interest in STEM. International Journal of Science Education, Part B, 2(1), 63–79.

    Article  Google Scholar 

  13. Devine, A., Fawcett, K., Szucs, D., & Dowker, A. (2012). Gender differences in mathematics anxiety and the relation to mathematics performance while controlling for test anxiety. Behavioral and Brain Functions, 8(33), 1–9.

    Google Scholar 

  14. Dowker, A., Sarkar, A., & Looi, C. Y. (2016). Mathematics anxiety: What have we learned in 60 years? Frontiers in Psychology, 7, 508.

    Google Scholar 

  15. Eccles, J. S., Wigfield, A., Midgley, C., Reuman, D., Iver, D. M., & Feldlaufer, H. (1993). Negative effects of traditional middle schools on students’ motivation. The Elementary School Journal, 93(5), 553–574.

    Article  Google Scholar 

  16. Estrada, M., Burnett, M., Campbell, A. G., Campbell, P. B., Denetclaw, W. F., Gutiérrez, C. G., Hurtado, S., John, G. H., Matsui, J., McGee, R., & Okpodu, C. M. (2016). Improving underrepresented minority student persistence in STEM. CBE—Life Sciences Education. https://doi.org/10.1187/cbe.16-01-0038

    Article  Google Scholar 

  17. Fennema, E., & Sherman, J. A. (1976). Fennema-Sherman mathematics attitudes scales: Instruments designed to measure attitudes toward the learning of mathematics by females and males. Journal for Research in Mathematics Education, 7(5), 324–326.

    Google Scholar 

  18. Field, A. (2009). Discovering statistics using SPSS (3rd ed.). SAGE Publications Ltd.

    Google Scholar 

  19. Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159–1176.

    Article  Google Scholar 

  20. Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science motivation questionnaire: Construct validation with nonscience majors. Journal of Research in Science Teaching, 46(2), 127–146.

    Article  Google Scholar 

  21. Graham, S., & Weiner, B. (2012). Motivation: past, present, and future. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook, Theories, constructs, and critical issues (Vol. 1, pp. 367–397). The American Psychological Association.

    Chapter  Google Scholar 

  22. Harackiewicz, J. M., & Priniski, S. J. (2018). Improving student outcomes in higher education: The science of targeted intervention. Annual Review of Psychology, 69, 409–435.

    Article  Google Scholar 

  23. Harackiewicz, J. M., Rozek, C. S., Hulleman, C. S., & Hyde, J. S. (2012). Helping parents to motivate adolescents in mathematics and science: An experimental test of a utility-value intervention. Psychological Science, 23(8), 899–906.

    Article  Google Scholar 

  24. Hembree, R. (1988). Correlates, causes, effects, and treatment of test anxiety. Review of Educational Research, 58(1), 47–77.

    Article  Google Scholar 

  25. Hernandez, P. R., Schultz, P., Estrada, M., Woodcock, A., & Chance, R. C. (2013). Sustaining optimal motivation: A longitudinal analysis of interventions to broaden participation of underrepresented students in STEM. Journal of Educational Psychology, 105(1), 89–107.

    Article  Google Scholar 

  26. Hogheim, S., & Reber, R. (2015). Supporting interest of middle school students in mathematics through context personalization and example choice. Contemporary Educational Psychology, 42, 17–25.

    Article  Google Scholar 

  27. IBM Corp. (2016). IBM SPSS Statistics Version 240. IBM Corp.

    Google Scholar 

  28. Jackson, C., Mohr-Schroeder, M. J., Bush, S. B., Maiorca, C., Roberts, T., Yost, C., & Fowler, A. (2021). Equity-oriented conceptual framework for K-12 STEM literacy. International Journal of STEM Education. https://doi.org/10.1186/s40594-021-00294-z

    Article  Google Scholar 

  29. Jiang, S., Simpkins, S. D., & Eccles, J. S. (2020). Individuals’ math and science motivation and their subsequent STEM choices and achievement in high school and college: A longitudinal study of gender and college generation status differences. Developmental Psychology, 56(11), 2137–2151.

    Article  Google Scholar 

  30. Kaiser, H. F. (1991). Coefficient alpha for a principal component and the Kaiser-Guttman rule. Psychological Reports, 68(3), 855–858.

    Article  Google Scholar 

  31. Kazelskis, R., Reeves, C., Kersh, M. E., Bailey, G., Cole, K., Larmon, M., Hall, L., & Holliday, D. C. (2000). Mathematics anxiety and test anxiety: Separate constructs? The Journal of Experimental Education, 68(2), 137–146.

    Article  Google Scholar 

  32. Maloney, E. A., & Beilock, S. L. (2012). Math anxiety: Who has it, why it develops, and how to guard against it. Trends in Cognitive Sciences, 16(8), 404–406.

    Article  Google Scholar 

  33. Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Arens, A. K. (2019). The murky distinction between self-concept and self-efficacy. Beware the lurking jingle-jangle fallacies. Journal of Educational Psychology, 111(2), 331–353.

    Article  Google Scholar 

  34. Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th ed.). Muthén & Muthén.

    Google Scholar 

  35. Parker, P. D., Marsh, H. W., Ciarrochi, J., Marshall, S., & Adjuljabbar, A. S. (2014). Juxtaposing math self-efficacy and self-concept as predictors of long-term achievement outcomes. Educational Psychology, 34(1), 29–48.

    Article  Google Scholar 

  36. Peterson, J. L., & Hyde, J. S. (2017). Trajectories of self-perceived math ability, utility value and interest across middle school as predictors of high school math performance. Educational Psychology, 37(4), 438–456.

    Article  Google Scholar 

  37. Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 4, 667–686.

    Article  Google Scholar 

  38. Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40.

    Article  Google Scholar 

  39. Pintrich, P. R., Smith, D. A., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813.

    Article  Google Scholar 

  40. Pletzer, B., Wood, G., Scherndl, T., Kerschbaum, H. H., & Nuerk, H. C. (2016). Components of mathematics anxiety: Factor modeling of the MARS30-Brief. Frontiers in Psychology, 7, 91.

    Article  Google Scholar 

  41. Ramirez, G., Shaw, S. T., & Maloney, E. A. (2018). Math anxiety: Past research, promising interventions, and a new interpretation framework. Educational Psychologist, 53(3), 145–164.

    Article  Google Scholar 

  42. Renninger, K. A., & Hidi, S. (Eds.). (2019). The Cambridge handbook of motivation and learning. Cambridge University Press.

    Google Scholar 

  43. Richardson, F. C., & Suinn, R. M. (1972). The Mathematics Anxiety Rating Scale: Psychometric data. Journal of Counseling Psychology, 19, 551–554.

    Article  Google Scholar 

  44. Rosenzweig, E. Q., & Wigfield, A. (2016). STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist, 51(2), 146–163.

    Article  Google Scholar 

  45. Rosenzweig, E. Q., Wigfield, A., & Eccles, J. (2019). Expectancies, values, and its relevance for student motivation and learning. In K. A. Renninger & S. Hidi (Eds.), The Cambridge handbook of motivation and learning (pp. 617–644). Cambridge University Press.

    Chapter  Google Scholar 

  46. Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Press.

    Book  Google Scholar 

  47. Schunk, D. H., & DiBenedetto, M. K. (2016). Self-efficacy theory in education. In K. R. Wentzel & D. B. Miele (Eds.), Handbook of motivation at school (2nd ed., pp. 34–54). Routledge.

    Google Scholar 

  48. Schunk, D. H., & Greene, J. A. (2017). Handbook of self-regulation of learning and performance (2nd ed.). Routledge.

    Book  Google Scholar 

  49. Seo, E., Shen, Y., & Alfaro, E. C. (2019). Adolescents’ beliefs about math ability and their relations to STEM career attainment: Joint consideration of race/ethnicity and gender. Journal of Youth and Adolescence, 48(2), 306–325.

    Article  Google Scholar 

  50. Stevens, J. P. (2002). Applied multivariate statistics for the social sciences (4th ed.). Erlbaum.

    Google Scholar 

  51. Suinn, R. M., & Winston, E. H. (2003). The Mathematics Anxiety Rating Scale, a brief version: Psychometric data. Psychological Reports, 92(1), 167–173.

    Article  Google Scholar 

  52. Tapia, M., & Marsh, G. (2004). An instrument to measure mathematics attitudes. Academic Exchange Quarterly, 8, 16–21.

    Google Scholar 

  53. Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50(5), 1081–1121.

    Article  Google Scholar 

  54. Watt, H. M., Hyde, J. S., Petersen, J., Morris, Z. A., Rozek, C. S., & Harackiewicz, J. M. (2017). Mathematics—A critical filter for STEM-related career choices? A longitudinal examination among Australian and US adolescents. Sex Roles, 77(3), 254–271.

    Article  Google Scholar 

  55. Weidinger, A. F., Spinath, B., & Steinmayr, R. (2020). The value of valuing math: Longitudinal links between students’ intrinsic, attainment, and utility values and grades in math. Motivation Science, 6(4), 413–422.

    Article  Google Scholar 

  56. Weis, M., Heikamp, T., & Trommsdorff, G. (2013). Gender differences in school achievement: The role of self-regulation. Frontiers in Psychology, 4, 442.

    Article  Google Scholar 

  57. Wentzel, K. R., & Miele, D. B. (2016). Handbook of motivation at school (2nd ed.). Routledge.

    Book  Google Scholar 

  58. Wigfield, A., & Meece, J. L. (1988). Math anxiety in elementary and secondary school students. Journal of Educational Psychology, 80(2), 210–216.

    Article  Google Scholar 

  59. Wigfield, A., Tonks, S. M., & Klauda, S. L. (2016). Expectancy-value theory. In K. R. Wentzel & D. B. Miele (Eds.), Handbook of motivation of school (2nd ed., pp. 55–74). Routledge.

    Google Scholar 

  60. Yang, Y., & Green, S. B. (2015). Evaluation of structural equation modeling estimates of reliability for scales with ordered categorical items. European Journal of Research Methods for the Behavioral and Social Sciences, 11(1), 23–34.

    Article  Google Scholar 

  61. Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663–676.

    Article  Google Scholar 

  62. Zimmerman, B. J., & Labuhn, A. S. (2012). Self-regulation of learning: Process approaches to personal development. In K. R. Harris, S. Graham, & T. Urdan (Eds.), APA educational psychology handbook: Theories, constructs, and critical issues (Vol. 1, pp. 399–425). The American Psychological Association.

    Chapter  Google Scholar 

Download references

Acknowledgements

We would like to thank all the teachers and students who participated in the study.

Funding

This research was supported by a grant from the Institute of Education Sciences (Award No. R305A150365).

Author information

Affiliations

Authors

Contributions

LF completed the majority of the writing for the manuscript. SY completed the majority of the data analysis for the manuscript and assisted with writing the manuscript. KA assisted with data analysis and writing the manuscript. JP, GP, SS, DU, and NV also assisted with writing the manuscript. NV helped with teacher recruitment, training, and data collection for this manuscript. SS, DU, and LF are the principal investigators on the grant funding this project and thus were in charge with the design and execution of the study. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Logan Fiorella.

Ethics declarations

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendices

Table 9 Initial version of the Math Motivation Questionnaire (MMQ)
Table 10 The finalized version of MMQ

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Fiorella, L., Yoon, S.Y., Atit, K. et al. Validation of the Mathematics Motivation Questionnaire (MMQ) for secondary school students. IJ STEM Ed 8, 52 (2021). https://doi.org/10.1186/s40594-021-00307-x

Download citation

Keywords

  • Mathematics
  • Motivation
  • Secondary students
  • Construct validity
  • Criterion validity