Skip to main content

Internet use at and outside of school in relation to low- and high-stakes mathematics test scores across 3 years

Abstract

Background

The excessive use of Internet-based technologies has received a considerable attention over the past years. Despite this, there is relatively little research on how general Internet usage patterns at and outside of school as well as on weekends may be associated with mathematics achievement. Moreover, only a handful of studies have implemented a longitudinal or repeated-measures approach on this research question. The aim of the current study was to fill that gap. Specifically, we investigated the potential associations of Internet use at and outside of school as well as on weekends with mathematics test performance in both high- and low-stakes testing conditions over a period of 3 years in a representative sample of Estonian teenagers.

Methods

PISA 2015 survey data in conjunction with national educational registry data were used for the current study. Specifically, Internet use at and outside of school as well as on weekends were queried during the PISA 2015 survey. In addition, the data set included PISA mathematics test results from 4113 Estonian 9th-grade students. Furthermore, 3758 of these students also had a 9th-grade national mathematics exam score from a couple of months after the PISA survey. Finally, of these students, the results of 12th-grade mathematics national exam scores were available for 1612 and 1174 students for “wide” (comprehensive) and “narrow” (less comprehensive) mathematics exams, respectively.

Results

The results showed that the rather low-stakes PISA mathematics test scores correlated well with the high-stakes national mathematics exam scores obtained from the 9th (completed a couple of months after the PISA survey) and 12th grade (completed approximately 3 years after the PISA survey), with correlation values ranging from r = .438 to .557. Furthermore, socioeconomic status index was positively correlated with all mathematics scores (ranging from r = .162 to .305). Controlled for age and gender, the results also showed that students who reported using Internet the longest tended to have, on average, the lowest mathematics scores in all tests across 3 years. Although effect sizes were generally small, they seemed to be more pronounced in Internet use at school.

Conclusions

Based on these results, one may notice that significantly longer time spent on Internet use at and outside of school as well as on weekends may be associated with poorer mathematics performance. These results are somewhat in line with research outlining the potentially negative associations between longer time spent on digital technology use and daily life outcomes.

Introduction

The relationship and possible impact of information and communication technology (ICT) on academic outcomes has received considerable attention over the past years. Diffusion of various devices, such as personal computers and laptops, smartphones, and tablets have the potential of enhancing productivity and efficiency of learning. For instance, it is possible to have a library of materials in one’s personal device, replacing a heavy backpack filled with textbooks. Information-seeking is easier than ever—popular search engines, such as Google, and online encyclopedias (e.g., Wikipedia) allow to browse for up-to-date knowledge, supplementing or even replacing textbooks. Various applications may be useful in visualizing mathematical concepts (e.g., the nature of a regression equation, geometry, etc), and simulations could help with understanding scientific concepts (Atit et al., 2020). Paper-and-pencil testing could be supplemented or substituted by computer-based assessment (Nissen, Jariwala, Close, & Dusen, 2018). There is also evidence that educational technology applications may have a positive effect on mathematics achievement (Cheung & Slavin, 2013; Miller, 2018).

On the other hand, it has also been shown that ICT use may not necessarily enhance academic experience as expected (Star et al., 2014). Furthermore, several studies have reported that excessive use of Internet-based digital technologiesFootnote 1 is associated with impairments in daily life. Not only has it been demonstrated to inversely correlate with psychological well-being (Twenge & Campbell, 2018), it has also been shown that educational factors have negative associations with excessive Internet-based digital device use (Kates, Wu, & Coryn, 2018; Lepp, Barkley, & Karpinski, 2015; Rozgonjuk, Saal, and Täht, 2018). From an instructor’s point of view, it may be relevant to understand the association between online- and offline-teaching (Yang, 2017) as well as study habits that could be influenced by ICTs (Hora & Oleson, 2017; Rozgonjuk, Kattago, and Täht, 2018). In addition, it has been found that the potential benefits of ICTs in mathematics education context may not be fully harnessed if the educator lacks ICT-related knowledge, has minimal training as well as learning opportunities regarding ICT, and if the technical support of ICT is limited (Zakaria & Khalid, 2016). Therefore, it would be relevant to take a closer look at Internet use in relation to mathematics achievement.

In the current study, we focus on students’ general self-reported duration of Internet use at and outside of school, as well as on weekends, in relation to PISA 2015 mathematics test results and 9th- and 12th-grade national mathematics exam scores. The results could be helpful, as they may provide insights into the association between ICT and mathematics outcomes; this knowledge, in turn, could be potentially useful in designing cyber-hygiene practices that may be helpful in improved academic achievement.

Literature overview

Mathematics has been demonstrated to be a key factor in future academic success (Konvalina, Wileman, & Stephens, 1983). The results of mathematics exams could play an important role in students’ future educational path: for instance, in Estonia, a students’ admission to university may heavily depend on how well one performs on a mandatory national mathematics exam. Although there are many studies that have investigated the interplay between academic outcomes and Internet-based digital technology use, studies have not generally focused on mathematics or have focused on very specific educational technology applications or platforms (Cheung & Slavin, 2013; Fabian & Topping, 2019). Nevertheless, it has been suggested that these digital technologies may shape the future of mathematics education (Engelbrecht, Llinares, & Borba, 2020).

Even though the usage of ICTs has the potential to improve the efficiency of a classroom experience (e.g., by substituting paper-based books with digital resources, as well as by allowing the implementation of distance learning), previous studies have demonstrated that the effects of ICT use are not necessarily learning-enhancing. In fact, some studies have found that ICT use may even impair learning—this also in the context of mathematics (Bulut & Cutumisu, 2017; Zhang & Liu, 2016). Ravizza, Hambrick, & Fenn (2014) demonstrated that non-academic Internet use was negatively related to learning even when the students’ ability was controlled for.

Several factors may explain this association (Hu, Gong, Lai, & Leung, 2018). First, school-level indicators, such as the availability of Internet-based technologies as well as the size of school could play a large role (Eickelmann, Gerick, & Koop, 2017; Luu & Freeman, 2011). Second, several factors regarding individual differences in students’ perception and attitudes towards ICT use may be relevant (Hu, Gong, Lai, & Leung, 2018). Third, where (e.g., at school vs outside of school) and for what purposes (e.g., purely for learning vs entertainment) the ICTs are used may also be helpful in explaining the association between ICT use and academic outcomes (Petko, Cantieni, & Prasse, 2017; Skryabin, Zhang, Liu, & Zhang, 2015). Finally, individual differences in predisposing factors, such as personality traits, emotion regulation, and tendency to procrastinate have been shown to be relevant in developing problematic ICT use patterns (Brand, Young, Laier, Wölfling, & Potenza, 2016, Brand et al., 2019; Rozgonjuk, 2019) .

While there are studies that have investigated the relationship between Internet and other ICT use and academic achievement, research on general ICT usage patterns in relation to mathematics outcomes is rather scarce. Moreover, most of these studies tend to rely on cross-sectional data. Of note, however, it should be mentioned that Zhang and Liu (2016), for instance, have also provided evidence over a longer period of time, finding that ICT use at school is negatively correlated to academic achievement.

In the current study, we aim to present a more comprehensive set of empirical evidence on the association between Internet use and mathematics achievement. Specifically, we take into account the differences in Internet use at and outside of school and on weekends, and we include both low- and high-stakes mathematics achievement test scores. The former includes the PISA 2015 (see the Sample and procedure section) mathematics test scores, while the latter data are from two additional time points: from 9th- and 12th-grade national mathematics exams. Importantly, while PISA 2015 mathematics test could be considered as a low-stakes test—since the outcome of this test does not affect a student’s future significantly (Mägi, Adov, Täht, & Must, 2013; Silm, Must, & Täht, 2013)—the national mathematics exams can be considered as high-stakes tests within the Estonian educational system, because the results of these tests could play a proportional role in a student’s admission scores for the next stage of education.

Additionally, students’ gender and socioeconomic status (SES) may play a role in academic achievement and mathematics results. According to a meta-analysis by Voyer and Voyer (2014), female students tend to achieve better grades in school—this is a common finding, regardless of culture, school subject, or time period (based on literature from 1914 to 2011). But in addition to better academic achievement, female students also achieve better rates in higher education graduation and post-secondary school enrollments, and demonstrate better overall retention when compared with male students (Clark, Sang Min, Goodman, & Yacco, 2008). On the other hand, meta-analysis that included standardized test performance results indicates that male students tend to achieve better results in mathematics (Lindberg, Hyde, Petersen, & Linn, 2010) and natural sciences (Hedges & Nowell, 1995). Men also tend to have a higher propensity towards choosing a science, technology, engineering, and mathematics (STEM)-related career (Ketenci, Leroux, & Renken, 2020). On the other hand, female students have been found to have more positive mathematics attitudes compared with male students (Zuo, Ferris, & LaForce, 2020).

The positive relationship between students’ SES and academic achievement has been shown in many studies (e.g., Jimerson, Egeland, Sroufe, & Carlson, 2000; Liu & Schunn, 2020; Sirin, 2005; White, 1982). For instance, a meta-analysis by White (1982) demonstrated that the relationship yielded an effect size of r = .343, while a meta-analysis by Sirin (2005) found the effect size of r = .299. Therefore, one may conclude that SES generally has a medium-sized positive effect on academic achievement, including in mathematics and natural sciences. Therefore, it would be a necessary covariate in investigating links between mathematics performance and other variables.

The data set of the current study is based on an Estonian student sample. This may be of interest for mathematics as well as educational scientists in general for two reasons. Firstly, according to PISA 2015 results, the achievements of Estonian students in mathematics (as well as functional reading and science) were among the highest within the countries participating in PISA 2015 survey (OECD, 2016). Second, Estonia is also well-known for its high levels of digitalization and diffusion of e-governance (Solvak et al., 2019) that may also promote higher digital technology implementation in education. The results of this study could shed light on whether these aspects could be relevant when compared with previous findings in the field of ICT use and academic achievement (in mathematics).

Conceptual framework

One of the explanations to these findings is the so-called displacement hypothesis, according to which the negative effects of ICT use are directly proportional to time spent on one’s device. This is because time spent on a digital device decreases the potential time one could spend on reading books, exercising, and/or socializing in non-digital settings (Neuman, 1988). According to another explanation, the digital Goldilocks hypothesis, engaging too little or too much in digital technology use could result in poorer outcomes (Przybylski & Weinstein, 2017). On one hand, not using the Internet may result in a student’s inability to seek for additional information and to discuss homeworks via social networking sites. On the other hand, too much Internet use could result in a student not paying attention to learning relevant materials—especially when it is being communicated orally by a teacher. In fact, some ICT use has been shown to be related to more favorable outcomes than no or too much use in children’s subjective well-being (Przybylski & Weinstein, 2017; Twenge & Campbell, 2018) and cognitive test results (Rozgonjuk & Täht, 2017). In general, the lowest test scores were associated with individuals reporting highest time spent online.

The current study could add to this theoretical framework by providing empirical evidence that could further help the discussion on the association between ICT use and academic outcomes. Specifically, we show how groups based on self-reported Internet use at and outside of school as well as on weekends differ from each other in mathematics performance. Furthermore, the associations are controlled for students’ SES and gender, variables that are typically associated with academic performance, including STEM subjects (Sirin, 2005; Voyer & Voyer, 2014). Therefore, these results could potentially show if any of these conceptual frameworks could be used as a potential explanation for the results.

Aim of this work

The general aim is to investigate if self-reported Internet use at and outside of school and on weekends that was queried in 9th grade is associated with mathematics test performance across both low- and high-stakes mathematics testing conditions over a period of approximately 3 years. It should be noted, however, that in the current study, Internet use at school means not only using it for schoolwork, but also for leisurely activities.

Based on previous findings, there is reason to believe that the lowest mathematics achievement scores are associated with the highest self-reported Internet use. However, in addition to providing insights into Internet use at school in association with mathematics performance, we also provide evidence for how mathematics performance is correlated with Internet use outside of school as well as on weekends.

The findings of this study may be informative regarding how general Internet use could be a predictive factor for mathematics achievement. For instance, large-effect associations would mean that Internet use at school could play a pivotal role either in improving or hindering students’ achievement in mathematics. On the other hand, small or non-significant associations between Internet use and mathematics test scores may be helpful for recalibrating people’s expectations towards technology’s effects on mathematics achievement. As has been shown recently, the potential effects of digital technology use may be smaller (yet negative) than could be expected from sensationalized headlines in mass media (Orben & Przybylski, 2019) which may unnecessarily fuel technology use-related panics (Orben, 2020; Segool, Goforth, Bowman, & Pham, 2016). In other words, due to sensationalized headlines in the media, teachers, parents, and other stakeholders may be led to believe that digital technology use is more harmful than it actually may be. In the context of the current study, for instance, mathematics teachers may choose not to use digital technology-based teaching methods because of the fear that digital technology use may be detrimental for students’ achievements. Similarly, small or non-significant associations could also be helpful in curbing one’s enthusiasm regarding the hype typically associated with new technological advances. Accordingly, this study may have the potential to provide input into policymaking regarding ICT use in education (e.g., whether the general public should be concerned or not).

In addition, if the results demonstrate the associations between Internet use (e.g., the more Internet is used, the lower the mathematics results) outside of school or on weekends, there could be a reason to believe that by targeting the Internet use on one’s time outside of school could have potential to improve a student’s mathematics performance.

Method

Sample and procedure

The data that we used was combined from two sources: (1) the Economic Co-operation and Development (OECD) public PISA 2015 data repository and (2) the National Exam Data Repository managed by the Innove Foundation in Estonia. Variables regarding Internet use at school as well as sociodemographics were retrieved from the former, and national mathematics exam scores from the latter. Since the Innove Foundation was responsible for administering both PISA survey as well as national mathematics exams, the data were merged in-house based on students’ personal identification code. The data set shared with the reviewers did not include personal identifiers and was therefore fully anonymous. Merging these datasets allows to use self-report measures in combination with high-stakes exam scores from nationally standardized settings in repeated-measures study design.

The PISA is a triennial international survey which attempts to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. In the current study, we focused on the PISA 2015 Estonian sample. The PISA 2015 used a stratified systematic sample where sampling probabilities had to be proportional to the estimated number of 15-year-old students (OECD, 2016). Sampling consisted of two stages: first, schools were sampled from all schools of Estonia (which included 15-year-old students); second, students were sampled from these schools. This method should provide a high-standard sample that is representative of 15-year-old students in Estonia. The PISA 2015 survey was carried out in Estonia in April 2015. Data for all of the participating countries is available on the OECD website (OECD, 2017a). The Estonian sample used in the current study included 6147 15-year-old students participating, 49.3% of whom were boys. Students were assessed in science, mathematics, and reading comprehension. These data were merged with national mathematics exam scores administered at the end of secondary school.

As national exams are different across each year, and students who took the PISA test while in 8th grade took their exams a year later than students who were 9th graders at the time of the PISA survey, we excluded 8th graders from the effective sample so that we could consistently compare the results (remaining n = 4709). Then, we included only students who had responded to the Internet use, socioeconomic status, and gender variables (remaining n = 4113). The 4113 students (48.23% male, 51.76% female) had the PISA mathematics test scores. Out of them, 3758 students (48.32% male, 51.68% female) had a 9th-grade mathematics exam results from approximately 2 months after the PISA survey. Finally, 1612 students (51.24% male, 48.76% female) had a “wide” mathematics exam score, while 1174 students (32.45% male, 67.55% female) had a “narrow” mathematics exam score from 12th grade. Sample breakdown by Internet response variables and gender, along with each group’s average SES, are in Supplementary Table 1.

It is necessary to comment that the reason for sample attrition over time is not completely clear; it may be that some students decided not to pursue their educational path beyond 9th grade. It could also be that some of the students decided not to take the national mathematics exam in 2019 for other reasons (e.g., because of studying abroad, choosing to continue their education in a vocational school, etc.).

Another important issue necessary to be mentioned is the somewhat nested nature of the data. While the PISA data set does include some school-level information, and it may be accounted for in analyses involving the PISA mathematics test, students in Estonia may change their school when they pursue their educational path in a secondary school. In addition, many Estonian schools only provide education until 9th grade, meaning that students who would like to pursue studying need to change their school for secondary education.

Measures

In the current work, we focused on Internet use and mathematics achievement variables. Nevertheless, the PISA survey data set also includes some sociodemographic variables. We used participants’ gender (coded as 1 = female, 2 = male), and socioeconomic status (SES) index which consituted standardized z-scores computed from different relevant SES-related items queried in the PISA 2015 survey. SES index variable is standardized across all OECD countries that participated in the PISA study.

Internet use items

The study participants provided self-reported estimation for the duration of their Internet use (a) at school, (b) outside of school, and (c) on weekends. The scale for all variables was the same: 1 = No time; 2 = 1–30 min per day; 3 = 31–60 min per day; 4 = Between 1 and 2 h per day; 5 = Between 2 and 4 h per day; 6 = Between 4 and 6 h per day; 7 = More than 6 h per day.

PISA mathematics test

The PISA 2015 dataset (OECD, 2017a) includes mathematics achievement test results along with other educational test results. As each student responded only to a fraction of the entire assessment, other answers were imputed. The PISA student achievement dataset provides so-called “plausible values” for secondary analysis. Plausible values are multiple imputations of unobservable latent achievement for each student; details about the imputation procedure could be found in Wu (2005). Ten plausible values were given to all subjects within the framework of PISA 2015 (OECD, 2017b).

Wu (2005) states that it is possible to use one of the plausible values in order to recover population parameters. However, as the current study is not modeling population parameters, it is more appropriate to use all plausible values in order to retrieve more accurate parameter estimates. Therefore, we followed the guidelines by OECD (2018) and computed the results for PISA mathematics test scores across all ten plausible values in all relevant analyses.

The mathematics achievement test is meant to measure mathematics literacy (defined as students’ capacity to formulate, employ, and interpret mathematics in a variety of contexts). It includes reasoning mathematically and using mathematical concepts, procedures, facts, and tools to describe, explain, and predict phenomena (OECD, 2016).

Mathematics national exam scores for 9th graders (2018)

The mandatory 9th-grade mathematics national exams are administered to students in Estonia in spring during the last period of the 9-grade education program. In the current case, the students took the exam in spring 2015. In Estonia, a mathematics exam is compulsory for graduating from primary school (9-grade education). This exam covers all the topics taught in primary school. The 9th grade mathematics exams are scored on the scale of 0 to 100 points.

Mathematics national exam scores for 12th graders (2018)

The mandatory mathematics national exams at the end of 12th grade are administered to students in Estonia in spring during the last period of the secondary education program. In the context of the current study, the students took the exam in spring 2018. In Estonia, a mathematics exam is compulsory for graduating from secondary school. However, students can choose whether to take the simplified (“narrow”) or more comprehensive (“wide”) mathematics exam. The latter may have more relevance in continuing one’s education on the next, higher education stage. Both exams cover all topics taught in secondary school (e.g., geometry and trigonometry, among others), while tasks in the narrow mathematics exam are less comprehensive. Both exams are scored on the scale of 0 to 100 points.

Analysis

The data were analyzed in RStudio version 4.0.3 (R Core Team, 2020). In order to compute the results that include PISA 2015 mathematics test scores, we followed the guidelines by OECD (2018) and estimated all statistics across ten plausible values of those mathematics scores.

We conducted Pearson correlation analysis (p-values adjusted with the Holm’s method) to investigate the relationship between 9th- and 12th-grade national mathematics exam scores and SES using the rcorr.adjust() function from the RcmdrMisc package v 2.7.1 (Fox, 2020).

Welch’s two sample t tests were used to compare average 9th- and 12th-grade national mathematics exam scores across gender from the R’s base package. Cohen’s d–s as group difference effect size measures were computed using the cohensD() function from the lsr package v 0.5 (Navarro, 2015).

In order to compare groups of students based on their self-reported Internet use at and outside of school and on weekends, we computed a series of analyses of covariances (ANCOVAs) using the ancova() function from the jmv package v 1.2.23 (Selker, Love, & Dropmann 2020), controlling for gender and SES effects. These ANCOVAs were computed with 9th- and 12th-grade national mathematics exam scores as dependent variables in each model. In addition, Holm’s post hoc tests were used for computing the differences between each pair of groups, complemented with effect size estimates (Cohen’s d–s) for group differences.

Commonly used effect size benchmarks according to Cohen (1988) and Sawilowsky (2009) are reported for post hoc comparisons, where d = 0.01, 0.20, 0.50, 0.80, 1.20, and 2.00 denote “very small”, “small”, “medium”, “large”, “very large”, and “huge” effect sizes, respectively. This said, we also report the recently proposed benchmarks by Kraft (2020) who argues that, in educational context, it may be more justified to interpret d < 0.05, d < 0.20, and d > 0.50 as “small”, “medium”, and “large” effect sizes, respectively. Partial eta-squared (η2p) are presented as effect size measures in ANCOVA models. Commonly used benchmarks are η2p = .010, .059, and .138 for small, medium, and large effects, respectively (Cohen, 1988; Richardson, 2011).

Results

Descriptive statistics for and correlations between SES and mathematics scores

In Table 1 below, we present the descriptive statistics and correlations for SES and mathematics scores.

Table 1 Descriptive statistics and correlations between SES and mathematics test scores

As can be seen from Table 1, SES is positively correlated with all mathematics variables, yielding small-to-medium effect sizes. PISA mathematics test scores have rather medium-to-large positive associations with 9th- and 12th-grade exam scores. Ninth-grade national mathematics exam scores have strong positive correlations with 12th-grade exam scores.

The results from Welch’s two sample t tests showed that male students (M = 534.72, SD = 80.55) had higher scores in PISA 2015 mathematics test (estimated across all ten plausible values) than female students (M = 525.56, SD = 76.31), t(4047.95) = − 3.737, p = .002, d = .117. Girls (M = 66.36, SD = 22.62) scored higher in the 9th-grade mathematics exam than boys (M = 63.66, SD = 24.03), t(3696.2) = 3.54, p > .001, d = 0.12. While there were no gender differences in the “wide” mathematics exam scores, t(1608.5) = 0.77, p = .444, d = 0.04, girls (M = 43.02, SD = 24.54) had higher scores than boys (M = 37.92, SD = 23.78) in the “narrow” mathematics exam, t(771.51) = 3.403, p < .001, d = 0.21.

Group differences in mathematics scores based on internet use

Below, the results for ANCOVAs are presented. Figure 1 depicts group differences based on self-reported Internet use at and outside of school and on weekends across the PISA mathematics test (estimated across ten plausible values), 9th-grade national mathematics exam, and 12th-grade (both “wide” and “narrow”) mathematics exam scores, controlled for potential SES and gender effects (estimated marginal means are depicted on y-axis). Holm’s post hoc comparisons for 9th- and 12th-grade exam scores are presented in Supplementary Table 2.

Fig. 1
figure1

Estimated marginal means of mathematics scores based on Internet use at and outside of school and on weekends, controlled for SES and gender. Notes: On x-axis (Self-reported Internet Use in 2015): 1 = “No time”; 2 = “1–30 min per day”; 3 = “31–60 min per day”; 4 = “Between 1 and 2 h per day”; 5 = “Between 2 and 4 h per day”; 6 = “Between 4 and 6 h per day”; 7 = “More than 6 h per day”. G12 (Narrow) = Grade 12 national mathematics exam (narrow) score; G12 (Wide) = Grade 12 national mathematics exam (wide) score; G9 = Grade 9 national mathematics exam score; PISA PV = PISA 2015 mathematics test scores (estimated across ten plausible values as suggested in OECD (2018). The 95% CIs are depicted.

Overall, the results indicate to either curvilinear or linearly decreasing scores with the increase of self-reported Internet usage. As can be observed below in Tables 2, 3, 4, and 5, SES was a significant covariate in all models predicting mathematics results. In addition, significant group differences due to Internet use yielded small (based on Cohen, 1988, and Sawilowsky, 2009) or medium (based on Kraft, 2020) effects. Below, the results for each model are presented.

PISA mathematics test scores and internet use

Table 2 shows that Internet use variables as well as SES and gender were all significant predictors of PISA mathematics test scores in all models. It could be observed that the effects of SES are consistent throughout all models, yielding medium size. While gender has consistently small effects, Internet use at school has medium, while Internet use outside of school and on weekends have small-sized associations with PISA mathematics test scores.

Table 2 Results of ANCOVA analyses for PISA 2015 mathematics test scores (estimated across ten plausible values) and Internet use variables, controlled for SES and gender

Post hoc test results (in Supplementary Table 2) show that, in general, some Internet use at school has positive associations with PISA mathematics test scores, whereas the no or 6+ h per day of Internet use at school tend to be negatively associated with PISA mathematics test scores. In general, one may also notice that there may be a sweet spot of Internet use at school after which there seems to be a linear decline in PISA mathematics scores.

Within Internet use outside of school, somewhat similar patterns could be observed: students who reported no or 6+ h per day of Internet use outside of school had lower PISA mathematics test scores than students who reported some Internet use outside of school.

Finally, similar patterns described above tended to be also present in relation to Internet use on weekends. Here, too, the significant group differences tended to emerge either between students who reported not using Internet on weekends and students who reported using Internet on weekends between 2 and 4 h a day, or between students who reported using the Internet for 6+ h per day on weekends and those who tended to use the Internet on weekends for a couple of hours per day.

In general, therefore, students who either reported that they did not use Internet or used it for 6+ h per day tended to score lower than other students.

Ninth-grade mathematics national exam scores and internet use

The results in Table 3 show that, as with PISA mathematics scores, 9th-grade mathematics national exam scores were predicted by Internet use variables as well as SES and gender in all models. While potential gender effects were small, SES yielded small-to-medium effects. Internet use variables, too, had small associations with 9th-grade mathematics national exam scores. Highest partial eta-squared for Internet use was in the model regarding Internet use at school.

Table 3 Results of ANCOVA analyses for 9th-grade mathematics national exam scores and Internet use variables, controlled for SES and gender

Holm’s post hoc test results (in Supplementary Table 2) indicated to similar patterns in 9th-grade national exam scores as observed in PISA mathematics test: in general, moderate Internet use was associated with better outcomes than no or 6+ h per day of Internet use at school. Somewhat similar patterns could be observed for Internet use outside of school. Interestingly, when looking into Internet use on weekends, only some of the differences that were significant included 6+ h per day of Internet use—indicating to poorer outcomes than when Internet was used for a couple of hours per day on weekends.

Twelfth-grade “wide” mathematics national exam scores and internet use

The results in Table 4 are somewhat different than in the case of PISA and 9th-grade mathematics national exam scores. While SES remains to be a significant covariate predicting 12th-grade “wide” mathematics exam scores (with small effect sizes), gender is not a significant covariate in any models. While Internet use variables were significant predictors, the effect sizes were small, with highest partial eta-squared in the model for Internet use at school as predictor.

Table 4 Results of ANCOVA analyses for 12th-grade “wide” mathematics national exam scores and Internet use variables, controlled for SES and gender

Results of Holm’s post hoc tests (in Supplementary Table 2) show that across Internet use at school, there seems to be a relatively linear decline in mathematics scores with the increase of Internet use. Students who reported using the Internet at school for 6+ h per day scored lower on the “wide” mathematics national exam than students who reported not using the Internet at school. In general, students reporting using the Internet at school for 6+ h per day at school tended to have the lowest scores in mathematics exams.

While there were no group differences in “wide” mathematics scores between students who reported not or moderately using the Internet outside of school, 6+ h per day of Internet use outside of school was associated with poorer mathematics exam scores.

Finally, there were no group differences in the “wide” mathematics exam scores across Internet use on weekends.

Twelfth-grade “narrow” mathematics national exam scores and internet use

According to Table 5, SES and gender were significant covariates predicting 12th-grade national “narrow” mathematics exam scores (with small-sized effects). While Internet use at and outside of school had small yet significant associations, Internet use on weekends was not a significant predictor of these mathematics exam scores.

Table 5 Results of ANCOVA analyses for 12th-grade “narrow” mathematics national exam scores and Internet use variables, controlled for SES and gender

Across these exam scores, the Holm’s post hoc test results (in Supplementary Table 2) showed that using the Internet at school for up to 30 min tended to be associated with better “narrow” mathematics exam outcomes than using the Internet for more than 2 h per day at school. Students reporting using the Internet for 6+ h per day outside of school scored lower than students who reported using the Internet for between 12 h per day. There were no group differences in “narrow” mathematics scores based on Internet use on weekends.

Discussion

The main aim of the present study was to investigate how students’ Internet use at and outside of school and on weekends predicts mathematics performance across several years in low- and high-stakes testing conditions. Below are some of the insights into the findings as well as the contribution and limitations of this study.

We expected to find that students who report using Internet the most (at and outside of school, on weekends) to have the lowest average scores in mathematics tests. This was generally the case in all exams across all Internet use conditions—with the exception of Internet use on weekends as a predictor in 12th-grade mathematics exam scores. The results of this study are mostly in line with some previous, more general findings that have demonstrated the small negative relationships between Internet-based technology use and academic achievement (Kates, Wu, & Coryn, 2018; Lepp, Barkley, & Karpinski, 2015; Rozgonjuk, Saal, et al., 2018), as well as in the domain of mathematics education (Bulut & Cutumisu, 2017; Eickelmann, Gerick, & Koop, 2017; Hu,Gong, Lai, & Leung, 2018; Skryabin, Zhang, Liu, & Zhang, 2015). Yet—and especially in the case of Internet use at school—it also seemed that both the students who reported using the Internet for 6+ h per day at school as well as students who reported not using the Internet at all had lower mathematics test scores than students who reported some Internet use. This finding is in line with the digital Goldilocks hypothesis which states that there may be an optimal time of ICT use which may be associated with favorable outcomes (Przybylski & Weinstein, 2017). Moreover, these patterns seemed to be highly similar across different mathematics tests over a 3-year period.

What do these findings tell us? From one hand, these results support the notion that perhaps general Internet use, especially for a very long duration, may not be unidirectionally beneficial to mathematics performance. Perhaps too much Internet use may lead to activities which are irrelevant to learning objectives and outcomes, therefore resulting in poorer performance on mathematics tests. Using Internet during classes could potentially contribute to multitasking behavior while studying, which in turn could lead to distraction and be detrimental for understanding class context (Sana, Weston, & Cepeda, 2013). In fact, a recent study demonstrated that interruptions due to pop-up notifications are associated with more surface approach to learning (Rozgonjuk, Elhai, Ryan, and Scott, 2019).

On the other hand, it should also be stressed that these associations, although negative, are relatively small. Of course, it also depends on what exactly constitute small effects. Recently, Kraft (2020) argued that perhaps research in the field of education should not rely on common benchmarks proposed by Cohen (1988), since effect sizes in educational research may be small yet still pragmatically relevant. By benchmarks proposed in Kraft (2020), several group differences yielded medium-sized effects. This said, while the negative associations of Internet use seem to be present in some conditions, there are probably other more important aspects of education that drive the mathematics performance of a student. Mapping these variables should be in focus of subsequent studies.

Although it may be hardly plausible to assume that a teenager’s Internet use may affect their mathematics performance after 3 years, the results seem to suggest that there may be some substance to it that deserves further attention. On one hand, it could be that Internet use patterns do not change dramatically over time, potentially explaining these findings. Of course, although the current study has a repeated-measures approach, this causal hypothesis cannot be answered with a high degree of validity based on the data used in this study. On the other hand, there may be an alternative explanation for these findings. Importantly, it should be stressed that a student’s socioeconomic status (SES) was positively associated with all mathematics scores both in bi- and multivariate analyses. Furthermore, with the exception of “wide” mathematics exam scores, gender was also a significant covariate in models predicting mathematics performance. While boys scored higher in PISA mathematics test, girls had higher scores in 9th- and 12th-grade “narrow” mathematics exam scores. These findings are in line with common results in educational research, including mathematics, where SES and gender are associated with academic performance (Lindberg, Hyde, Petersen, & Linn, 2010; Sirin, 2005; Voyer & Voyer, 2014; White, 1982). It is nevertheless curious that even when mathematics results were controlled for these covariates in the current study, some of the relationships between Internet use variables and mathematics test scores remained significant, albeit small. In general, no Internet use or using the Internet for a very long time (e.g., 6+ h per day) was associated with poorer outcomes.

The main contributions of the current study regard the nature of the data, and some novel findings. First, the PISA 2015 sampling standards are very high and resemble the whole population of 15-year-old students in Estonia. Therefore, this sampling method provides more generalizability with regards to results, because one of the more common limitations in survey-based studies—convenience sampling and self-selection bias—are mitigated. Second, in addition to self-reported Internet use, the data included actual (that is, not self-reported) results for mathematics performance. Furthermore, these results were obtained for both low-stakes (PISA 2015 mathematics test) as well as high-stakes tests (9th- and 12th-grade national mathematics exams). This should support the validity of findings. While PISA 2015 mathematics test scores correlated strongly with national mathematics exam scores, the patterns in the relationships between Internet use variables and mathematics scores were similar, regardless of the low- or high-stakes testing conditions, and even when the possible effects of students’ SES and gender were taken into account. Third, mathematics results from three time points were used, further increasing validity and reliability of findings. While repeated-measures design provides a stronger case for causality in the reported relationships (Cole & Maxwell, 2003; Gollob & Reichardt, 1987), in order to establish a causal link, chronological order of measurements may not be sufficient, and a (quasi-)experimental study should be conducted to replicate these findings with more robust confidence. Therefore, we also want to explicitly state that our results do not support strong causal interpretation.

Of more theoretical contributions, the results show that the association between Internet use and mathematics achievement could be predicted over a period of three years, both across low- and high-stakes testing conditions. Furthermore, the results also seem to indicate that it may be necessary to distinguish Internet use at and outside of school (also Internet use on weekends) in this line of research, since the relationship patterns with mathematics achievement may vary due to that. This is also in line with some previous findings (Petko, Cantieni, & Prasse, 2017; Skryabin et al., 2015).

The limitations of the study ought to be addressed as well. Firstly, although the mathematics scores were not self-reported, Internet use variables did rely on self-reports. Studies have shown that people are not very accurate in estimating their digital technology use duration and frequency in relation to objectively measured device use (Boase & Ling, 2013; Kobayashi & Boase, 2012; Loid, Täht, & Rozgonjuk, 2020; Rozgonjuk, Levine, Hall, & Elhai, 2018). Therefore, future studies should aim towards documenting, or tracking, the actual digital device use in classroom, e.g., as done in learning analytics (Schneider, Reilly, & Radu, 2020) and digital phenotyping in other fields (Baumeister & Montag, 2019; Rozgonjuk, Elhai, & Hall, 2019). This could provide a more valid picture regarding the potential effects of digital technology use in relation to mathematics achievement.

Secondly, as mentioned earlier, the causal interpretation regarding the effects of Internet use at school is largely based on chronology of measurements—this, of course, does not necessarily mean that there is a direct causal effect. It could also be that other factors influence both Internet use (or how the student recalls their typical Internet use) and mathematics performance. For instance, individual differences could influence both (self-reported) technology use and academic outcomes. For example, more conscientious students could engage in less non-purposeful digital technology use and are also more disciplined to attain to learning tasks. In addition, one may also hypothesize that mathematics anxiety hinders learning mathematics and may motivate using ICTs instead of learning. It has recently been demonstrated that mathematics anxiety is associated with more surface approach to learning (Rozgonjuk et al., 2020), a factor also associated with problematic ICT use (Alt & Boniel-Nissim, 2018).

A third limitation is using rather general Internet use measures and correlating these self-reported estimates with mathematics scores. In order to gain insights into the potential effects of digital technology on mathematics, it would be highly informative to include data about digital technology use in mathematics learning, specifically. It could be questionable whether students really spend 6+ h per school day online. The question asking about the duration of Internet use at school does not specify whether the activities spent on the Internet are school-related or not. Therefore, it could be that students who reported using 6+ h of Internet per school day also included Internet use outside of coursework (e.g., during recess, etc) in their estimation. Clearly, this is a limitation of the study, and should be taken into account in future investigations.

Finally, it should be noted that it was not possible to account for the nested nature of the data for the national exam scores, and the reason for sample attrition over time was not completely clear. Further research should address these limitations.

In conclusion, this study demonstrates that the relationship between Internet use and mathematics achievement is rather curvilinear, but students who reported using the Internet for 6+ h per day tended to have, on average, lower mathematics test scores than students reporting less Internet use. This said, it is also interesting that these patterns may have variations depending on where and when Internet is used—either at or outside of school, or on weekends.

Availability of data and materials

The data as well as analysis script are available upon scholarly request.

Notes

  1. 1.

    Of note, there are no clear criteria for what is "excessive" digital technology use. While the World Health Organization (2019) has provided guidelines for how much screen time is suggested for children who are 3–4 years old (no more than 1 h of screen time); it is hard to set a threshold for screen time for older children and adults, since more and more aspects of everyday life (e.g., studying, working) are affected by overall digitalization. Often, therefore, "excessive" is subjective.

References

  1. Alt, D., & Boniel-Nissim, M. (2018). Links between adolescents’ deep and surface learning approaches, problematic internet use, and fear of missing out (FoMO). Internet Interventions, 13, 30–39. https://doi.org/10.1016/j.invent.2018.05.002.

    Article  Google Scholar 

  2. Atit, K., Power, J. R., Veurink, N., Uttal, D. H., Sorby, S., Panther, G., … Carr, M. (2020). Examining the role of spatial skills and mathematics motivation on middle school mathematics achievement. International Journal of STEM Education, 7(1), 38. https://doi.org/10.1186/s40594-020-00234-3.

    Article  Google Scholar 

  3. Baumeister, H., & Montag, C. (2019). Digital phenotyping and mobile sensing: New developments in psychoinformatics. Springer International Publishing. https://doi.org/10.1007/978-3-030-31620-4.

  4. Boase, J., & Ling, R. (2013). Measuring mobile phone use: Self-report versus log data. Journal of Computer-Mediated Communication, 18(4), 508–519. https://doi.org/10.1111/jcc4.12021.

    Article  Google Scholar 

  5. Brand, M., Wegmann, E., Stark, R., Muller, A., Wolfling, K., Robbins, T. W., & Potenza, M. N. (2019). The interaction of person-affect-cognition-execution (I-PACE) model for addictive behaviors: Update, generalization to addictive behaviors beyond internet-use disorders, and specification of the process character of addictive behaviors. Neuroscience & Biobehavioral Reviews, 104, 1–10. https://doi.org/10.1016/j.neubiorev.2019.06.032.

    Article  Google Scholar 

  6. Brand, M., Young, K. S., Laier, C., Wölfling, K., & Potenza, M. N. (2016). Integrating psychological and neurobiological considerations regarding the development and maintenance of specific internet-use disorders: An interaction of person-affect-cognition-execution (I-PACE) model. Neuroscience & Biobehavioral Reviews, 71, 252–266. https://doi.org/10.1016/j.neubiorev.2016.08.033.

    Article  Google Scholar 

  7. Bulut, O., & Cutumisu, M. (2017). When technology does not add up: ICT use negatively predicts mathematics and science achievement for Finnish and Turkish students in PISA 2012. In J. Johnston (Ed.), Proceedings of EdMedia 2017, (pp. 935–945) https://www.learntechlib.org/primary/p/178407/.

    Google Scholar 

  8. Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113. https://doi.org/10.1016/j.edurev.2013.01.001.

    Article  Google Scholar 

  9. Clark, M. A., Sang Min, L., Goodman, W., & Yacco, S. (2008). Examining male underachievement in public education. NASSP Bulletin, 92(2), 111–132. https://doi.org/10.1177/0192636508321155.

    Article  Google Scholar 

  10. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. L. Erlbaum Associates. https://doi.org/10.4324/9780203771587.

  11. Cole, D. A., & Maxwell, S. E. (2003). Testing mediational models with longitudinal data: Questions and tips in the use of structural equation modeling. Journal of Abnormal Psychology, 112(4), 558–577. https://doi.org/10.1037/0021-843x.112.4.558.

  12. Eickelmann, B., Gerick, J., & Koop, C. (2017). ICT use in mathematics lessons and the mathematics achievement of secondary school students by international comparison: Which role do school level factors play? Education and Information Technologies, 22(4), 1527–1551. https://doi.org/10.1007/s10639-016-9498-5.

    Article  Google Scholar 

  13. Engelbrecht, J., Llinares, S., & Borba, M. C. (2020). Transformation of the mathematics classroom with the internet. ZDM, 52(5), 825–841. https://doi.org/10.1007/s11858-020-01176-4.

    Article  Google Scholar 

  14. Fabian, K., & Topping, K. J. (2019). Putting “mobile” into mathematics: Results of a randomised controlled trial. Contemporary Educational Psychology, 59, 101783. https://doi.org/10.1016/j.cedpsych.2019.101783.

    Article  Google Scholar 

  15. Fox, J. (2020). RcmdrMisc: R Commander miscellaneous functions (2.7-0) [Computer software]. https://CRAN.R-project.org/package=RcmdrMisc

  16. Gollob, H. F., & Reichardt, C. S. (1987). Taking account of time lags in causal models. Child Development, 58(1), 80–92.

    Article  Google Scholar 

  17. Hedges, L., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269(5220), 41–45. https://doi.org/10.1126/science.7604277.

    Article  Google Scholar 

  18. Hora, M. T., & Oleson, A. K. (2017). Examining study habits in undergraduate STEM courses from a situative perspective. International Journal of STEM Education, 4(1), 1. https://doi.org/10.1186/s40594-017-0055-6.

    Article  Google Scholar 

  19. Hu, X., Gong, Y., Lai, C., & Leung, F. K. S. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers & Education, 125, 1–13. https://doi.org/10.1016/j.compedu.2018.05.021.

    Article  Google Scholar 

  20. Jimerson, S., Egeland, B., Sroufe, L. A., & Carlson, B. (2000). A prospective longitudinal study of high school dropouts examining multiple predictors across development. Journal of School Psychology, 38(6), 525–549. https://doi.org/10.1016/s0022-4405(00)00051-0.

    Article  Google Scholar 

  21. Kates, A. W., Wu, H., & Coryn, C. L. S. (2018). The effects of mobile phone use on academic performance: A meta-analysis. Computers & Education, 127, 107–112. https://doi.org/10.1016/j.compedu.2018.08.012.

    Article  Google Scholar 

  22. Ketenci, T., Leroux, A., & Renken, M. (2020). Beyond student factors: A study of the impact on STEM career attainment. Journal for STEM Education Research, 3(3), 368–386. https://doi.org/10.1007/s41979-020-00037-9.

    Article  Google Scholar 

  23. Kobayashi, T., & Boase, J. (2012). No such effect? The implications of measurement error in self-report measures of mobile communication use. Communication Methods and Measures, 6(2), 126–143. https://doi.org/10.1080/19312458.2012.679243.

    Article  Google Scholar 

  24. Konvalina, J., Wileman, S. A., & Stephens, L. J. (1983). Math proficiency: A key to success for computer science students. Communications of the ACM, 26(5), 377–382. https://doi.org/10.1145/69586.358140.

    Article  Google Scholar 

  25. Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253. https://doi.org/10.3102/0013189X20912798.

    Article  Google Scholar 

  26. Lepp, A., Barkley, J. E., & Karpinski, A. C. (2015). The relationship between cell phone use and academic performance in a sample of U.S. college students. SAGE Open, 5(1). https://doi.org/10.1177/2158244015573169.

  27. Lindberg, S. M., Hyde, J. S., Petersen, J. L., & Linn, M. C. (2010). New trends in gender and mathematics performance: A meta-analysis. Psychological Bulletin, 136(6), 1123–1135. https://doi.org/10.1037/a0021276.

    Article  Google Scholar 

  28. Liu, A. S., & Schunn, C. D. (2020). Predicting pathways to optional summer science experiences by socioeconomic status and the impact on science attitudes and skills. International Journal of STEM Education, 7(1), 49. https://doi.org/10.1186/s40594-020-00247-y.

    Article  Google Scholar 

  29. Loid, K., Täht, K., & Rozgonjuk, D. (2020). Do pop-up notifications regarding smartphone use decrease screen time, phone checking behavior, and self-reported problematic smartphone use? Evidence from a two-month experimental study. Computers in Human Behavior, 102, 22–30. https://doi.org/10.1016/j.chb.2019.08.007.

    Article  Google Scholar 

  30. Luu, K., & Freeman, J. G. (2011). An analysis of the relationship between information and communication technology (ICT) and scientific literacy in Canada and Australia. Computers & Education, 56(4), 1072–1082. https://doi.org/10.1016/j.compedu.2010.11.008.

    Article  Google Scholar 

  31. Mägi, M.-L., Adov, L., Täht, K., & Must, O. (2013). Who is willing to take low-stakes assignments? Trames. Journal of the Humanities and Social Sciences, 17(4), 417. https://doi.org/10.3176/tr.2013.4.07.

    Article  Google Scholar 

  32. Miller, T. (2018). Developing numeracy skills using interactive technology in a play-based learning environment. International Journal of STEM Education, 5(1), 39. https://doi.org/10.1186/s40594-018-0135-2.

    Article  Google Scholar 

  33. Navarro, D. (2015). Learning statistics with R: A tutorial for psychology students and other beginners http://health.adelaide.edu.au/psychology/ccs/teaching/lsr/.

    Google Scholar 

  34. Neuman, S. B. (1988). The displacement effect: Assessing the relation between television viewing and reading performance. Reading Research Quarterly, 23(4). https://doi.org/10.2307/747641.

  35. Nissen, J. M., Jariwala, M., Close, E. W., & Dusen, B. V. (2018). Participation and performance on paper- and computer-based low-stakes assessments. International Journal of STEM Education, 5(1), 21. https://doi.org/10.1186/s40594-018-0117-4.

    Article  Google Scholar 

  36. OECD (2016). PISA 2015 results (volume I): Excellence and equity in education. OECD. https://doi.org/10.1787/9789264266490-en.

  37. OECD. (2017a). PISA data. http://www.oecd.org/pisa/data/2015database/

  38. OECD (2017b). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving. OECD. https://doi.org/10.1787/9789264281820-en.

  39. OECD (2018). How to prepare and analyse the PISA database Data - PISA. http://www.oecd.org/pisa/data/httpoecdorgpisadatabase-instructions.htm.

    Google Scholar 

  40. Orben, A. (2020). The Sisyphean cycle of technology panics. Perspectives on Psychological Science, 174569162091937. https://doi.org/10.1177/1745691620919372.

  41. Orben, A., & Przybylski, A. K. (2019). The association between adolescent well-being and digital technology use. Nature Human Behavior, 3(2), 173–182. https://doi.org/10.1038/s41562-018-0506-1.

    Article  Google Scholar 

  42. Petko, D., Cantieni, A., & Prasse, D. (2017). Perceived quality of educational technology matters: A secondary analysis of students’ ICT use, ICT-related attitudes, and PISA 2012 test scores. Journal of Educational Computing Research, 54(8), 1070–1091. https://doi.org/10.1177/0735633116649373.

    Article  Google Scholar 

  43. Przybylski, A. K., & Weinstein, N. (2017). A large-scale test of the goldilocks hypothesis. Psychological Science, 28(2), 204–215. https://doi.org/10.1177/0956797616678438.

    Article  Google Scholar 

  44. R Core Team (2020). R: A language and environment for statistical computing (3.6.3) [computer software]. R Core Team.

  45. Ravizza, S. M., Hambrick, D. Z., & Fenn, K. M. (2014). Non-academic internet use in the classroom is negatively related to classroom learning regardless of intellectual ability. Computers & Education, 78, 109–114. https://doi.org/10.1016/j.compedu.2014.05.007.

  46. Richardson, J. T. E. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6(2), 135–147. https://doi.org/10.1016/j.edurev.2010.12.001.

  47. Rozgonjuk, D. (2019). Problematic smartphone use: Behavioral, psychopathological, dispositional, and educational correlates. University of Tartu Press.

  48. Rozgonjuk, D., Elhai, J. D., & Hall, B. J. (2019). Studying psychopathology in relation to smartphone use. In H. Baumeister, & C. Montag (Eds.), Digital Phenotyping and Mobile Sensing (pp. 185–199). Springer International Publishing. https://doi.org/10.1007/978-3-030-31620-4_11.

  49. Rozgonjuk, D., Elhai, J. D., Ryan, T., & Scott, G. G. (2019). Fear of missing out is associated with disrupted activities from receiving smartphone notifications and surface learning in college students. Computers & Education, 140. https://doi.org/10.1016/j.compedu.2019.05.016.

  50. Rozgonjuk, D., Kattago, M., & Täht, K. (2018). Social media use in lectures mediates the relationship between procrastination and problematic smartphone use. Computers in Human Behavior, 89, 191–198. https://doi.org/10.1016/j.chb.2018.08.003.

    Article  Google Scholar 

  51. Rozgonjuk, D., Kraav, T., Mikkor, K., Orav-Puurand, K., & Täht, K. (2020). Mathematics anxiety among STEM and social sciences students: The roles of mathematics self-efficacy, and deep and surface approach to learning. International Journal of STEM Education, 7(1), 46. https://doi.org/10.1186/s40594-020-00246-z.

    Article  Google Scholar 

  52. Rozgonjuk, D., Levine, J. C., Hall, B. J., & Elhai, J. D. (2018). The association between problematic smartphone use, depression and anxiety symptom severity, and objectively measured smartphone use over one week. Computers in Human Behavior, 87, 10–17. https://doi.org/10.1016/j.chb.2018.05.019.

    Article  Google Scholar 

  53. Rozgonjuk, D., Saal, K., & Täht, K. (2018). Problematic smartphone use, deep and surface approaches to learning, and social media use in lectures. International Journal of Environmental Research and Public Health, 15(1). https://doi.org/10.3390/ijerph15010092.

  54. Rozgonjuk, D., & Täht, K. (2017). To what extent does internet use affect academic performance? Using evidence from the large-scale PISA study. Annual Review of Cybertherapy and Telemedicine, 15, 39–44.

    Google Scholar 

  55. Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24–31. https://doi.org/10.1016/j.compedu.2012.10.003.

    Article  Google Scholar 

  56. Sawilowsky, S. S. (2009). New effect size rules of thumb. Journal of Modern Applied Statistical Methods, 8(2), 597–599. https://doi.org/10.22237/jmasm/1257035100.

    Article  Google Scholar 

  57. Schneider, B., Reilly, J., & Radu, I. (2020). Lowering barriers for accessing sensor data in education: Lessons learned from teaching multimodal learning analytics to educators. Journal for STEM Education Research, 3(1), 91–124. https://doi.org/10.1007/s41979-020-00027-x.

    Article  Google Scholar 

  58. Segool, N. K., Goforth, A. N., Bowman, N., & Pham, A. (2016). Social networking practices in school psychology: Have moral panic concerns been overstated? Journal of Applied School Psychology, 32(1), 66–81. https://doi.org/10.1080/15377903.2015.1121194.

    Article  Google Scholar 

  59. Selker, R., Love, J., & Dropmann, D. (2020). jmv: The “jamovi” analyses (1.2.23) [Computer software]. https://CRAN.R-project.org/package=jmv

  60. Silm, G., Must, O., & Täht, K. (2013). Test-taking effort as a predictor of performance in low-stakes tests. Trames. Journal of the Humanities and Social Sciences, 17(4), 433. https://doi.org/10.3176/tr.2013.4.08.

    Article  Google Scholar 

  61. Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453. https://doi.org/10.3102/00346543075003417.

    Article  Google Scholar 

  62. Skryabin, M., Zhang, J., Liu, L., & Zhang, D. (2015). How the ICT development level and usage influence student achievement in reading, mathematics, and science. Computers & Education, 85, 49–58. https://doi.org/10.1016/j.compedu.2015.02.004.

    Article  Google Scholar 

  63. Solvak, M., Unt, T., Rozgonjuk, D., Võrk, A., Veskimäe, M., & Vassil, K. (2019). E-governance diffusion: Population level e-service adoption rates and usage patterns. Telematics and Informatics, 36, 39–54. https://doi.org/10.1016/j.tele.2018.11.005.

    Article  Google Scholar 

  64. Star, J. R., Chen, J. A., Taylor, M. W., Durkin, K., Dede, C., & Chao, T. (2014). Studying technology-based strategies for enhancing motivation in mathematics. International Journal of STEM Education, 1(1), 7. https://doi.org/10.1186/2196-7822-1-7.

    Article  Google Scholar 

  65. Twenge, J. M., & Campbell, W. K. (2018). Associations between screen time and lower psychological well-being among children and adolescents: Evidence from a population-based study. Preventive Medical Reports, 12, 271–283. https://doi.org/10.1016/j.pmedr.2018.10.003.

    Article  Google Scholar 

  66. Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin, 140(4), 1174–1204. https://doi.org/10.1037/a0036620.

    Article  Google Scholar 

  67. White, K. R. (1982). The relation between socioeconomic status and academic achievement. Psychological Bulletin, 91(3), 461–481. https://doi.org/10.1037/0033-2909.91.3.461.

    Article  Google Scholar 

  68. World Health Organization. (2019). Guidelines on physical activity, sedentary behaviour, and sleep for children under 5 years of age. http://www.ncbi.nlm.nih.gov/books/NBK541170/

  69. Wu, M. (2005). The role of plausible values in large-scale surveys. Studies in Educational Evaluation, 31(2–3), 114–128. https://doi.org/10.1016/j.stueduc.2005.05.005.

    Article  Google Scholar 

  70. Yang, D. (2017). Instructional strategies and course design for teaching statistics online: Perspectives from online students. International Journal of STEM Education, 4(1), 34. https://doi.org/10.1186/s40594-017-0096-x.

    Article  Google Scholar 

  71. Zakaria, N. A., & Khalid, F. (2016). The benefits and constraints of the use of information and communication technology (ICT) in teaching mathematics. Creative Education, 07(11), 1537–1544. https://doi.org/10.4236/ce.2016.711158.

    Article  Google Scholar 

  72. Zhang, D., & Liu, L. (2016). How does ICT use influence students’ achievements in math and science over time? Evidence from PISA 2000 to 2012. EURASIA Journal of Mathematics, Science and Technology Education, 12(9). https://doi.org/10.12973/eurasia.2016.1297a.

  73. Zuo, H., Ferris, K. A., & LaForce, M. (2020). Reducing racial and gender gaps in mathematics attitudes: Investigating the use of instructional strategies in inclusive STEM high schools. Journal for STEM Education Research, 3(1), 125–146. https://doi.org/10.1007/s41979-019-00021-y.

    Article  Google Scholar 

Download references

Acknowledgements

The authors are thankful for the Foundation Innove Estonia for their collaboration.

Funding

This work did not receive funding. Open Access funding enabled and organized by Projekt DEAL.

Author information

Affiliations

Authors

Contributions

DR designed the study, analyzed the data, and wrote the first draft; KT designed the study and provided a critical review for the manuscript; KV provided a critical review for the manuscript. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Dmitri Rozgonjuk.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary Table 1.

Breakdown of responses to Internet use variables by gender and average socio-economic status

Additional file 2: Supplementary Table 2.

The results of Holm's post hoc comparisons for ANCOVAs

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rozgonjuk, D., Täht, K. & Vassil, K. Internet use at and outside of school in relation to low- and high-stakes mathematics test scores across 3 years. IJ STEM Ed 8, 27 (2021). https://doi.org/10.1186/s40594-021-00287-y

Download citation

Keywords

  • Internet use at school
  • Adolescence
  • Mathematics
  • Longitudinal study
  • High-stakes testing
  • PISA