Finding time for computer science in the elementary school day: a quasi-experimental study of a transdisciplinary problem-based learning approach

Abstract

Background

As the number of computer science (CS) jobs become increasingly available in this country and computing skills become essential tools for managing all aspects of our personal lives, CS is quickly becoming an essential element of K-12 education and recently, there has been increased attention to bringing computer science to the elementary grades. However, with a schedule that emphasizes literacy and mathematics, and other subjects competing for instructional time, creating opportunities for CS in the elementary school day is challenging. This study aimed to address this problem by investigating the use of problem-based transdisciplinary modules (i.e., “Time4CS” modules) that combined English language arts (ELA), science, and social studies lessons with the Code.org “Fundamentals” CS education program.

Results

Results indicated that teachers who taught Time4CS modules completed more CS lessons than teachers who did not teach the modules. Further, across all classrooms, completing a higher percentage of non-grade level assigned Code.org Fundamentals lessons (i.e., Code.org lessons above or below grade level that were available to teachers, but not required for their particular grade level) was positively associated with students’ achievement outcomes on state ELA and mathematics tests. Additionally, higher amounts of interdisciplinary teaching practices were associated with higher student achievement, specifically students’ state assessment ELA scores.

Conclusions

This study demonstrated that transdisciplinary problem-based modules that integrate the teaching of CS with other subject areas are a feasible way to bring more CS opportunities to younger learners. Moreover, it showed that implementing such modules is linked to more positive student academic achievement outcomes. With attentive revision, the modules featured in this study may be useful tools for elementary schools. These findings have implications for researchers, school district administrators, and those individuals who are in-charge of public policy initiatives seeking ways to bring CS to all elementary school students. Specifically, they highlight that it is possible to make time in the elementary school day for CS, and that there are no negative consequences for core subjects (e.g., ELA and mathematics).

Introduction

Computer science (CS) education has become a critical element of the US’ efforts to keep pace with the growing number of CS jobs available in this country (National Science Foundation, 2012). Computing skills are not only limited to CS jobs; computing now plays a role in every sector, and processes employed in CS such as problem-solving and algorithmic thinking, are crucial for success in many occupations (Stephenson & Dovi, 2013). Further, computing skills are becoming essential tools for managing all aspects of our personal lives including finance, communication, and health, and for simply navigating the world we live in (Wing, 2006). And yet, public schools do not provide equitable access to quality CS education (Wilson & Moffat, 2010; Google Inc., & Gallup Inc., 2016).

Challenges extend to higher education as well. Programming courses have some of the highest university dropout rates across all courses (Yadin, 2011). Accordingly, researchers have proposed that learning CS at the university level might be easier if it were introduced to students at an earlier age (Zaharija, Mladenovic, & Bolijat, 2013). As a result, introducing CS to elementary school students has become a growing area of global interest (Bargury et al., 2012; Grgurina, Barendsen, Zwaneveld, van Veen, & Stoker, 2014; Grout & Houlden, 2014; Tucker, 2003).

Elementary schools are a natural entrée to CS education as the prevalence of technology has resulted in young learners having a familiarity with computers long before they enter the classroom (Palfrey & Gasser, 2008). Research has suggested that students as young as five years old have demonstrated the capability of learning programming and computational thinking concepts (Bers. Flannery, Kazakoff, & Sullivan, 2014; Fessakis, Gouli, & Mavroudi, 2013). Further, interactive CS platforms (e.g., Scratch and Code.org) have shown success in teaching students these skills (Ouahbi et al., 2015; Saez-Lopez, Roman-Gonzalez, & Vazquez-Cano; 2016).

The elementary years are also critical for forming positive attitudes toward CS and STEM subjects. Elementary school students exposed to even short, weekly doses of CS have demonstrated a higher interest in CS (Lambert & Guiffre, 2009). Less research has explored factors that influence interest in CS careers; however, there are findings about STEM more broadly that show that students who express early interest and confidence in STEM subjects are more likely to pursue STEM careers (George, 2000). The majority of high school students interested in STEM careers, and scientists and graduate students in the sciences, report that their interest in STEM was initiated by experiences prior to or during middle school (Aschbacher, Li, & Roth, 2010). When STEM interest is not established early, there is a marked decline in positive attitudes toward STEM as students move through middle and high school (George, 2000). Conversely, students who report interest in science careers in eighth grade are three times more likely to obtain a degree in a science field than those who do not (Maltese & Tai, 2009). Despite this evidence, most current efforts to engage students in CS often focus on high school students with far fewer approaches designed to bring CS into the elementary grades.

As expectations related to standards and standardized testing heighten, finding time in the school day for anything new is challenging at all grade levels. The elementary school day is particularly full (Repenning et al., 2015; Webb et al., 2015) with literacy and mathematics instruction taking priority over other subjects, leaving science, social studies, and the arts competing for remaining instructional time. Thus, the challenge of bringing CS to the elementary day is especially difficult. This study sought to address this challenge.

Project overview

The authors collaborated with practitioners from Broward County Public Schools (BCPS) in Florida on this study, which was funded by the National Science Foundation (NSF) (#1542842). The study focused on the district’s strategy for bringing CS to elementary students, which was to embed CS content within the mandated 180-min literacy block. The district developed transdisciplinary modules (i.e., Time4CS modules) that included problem-based ELA, science, and social studies lessons associated with CS lessons from the Code.org “Fundamentals” program and Scratch activities. A group of teachers, in collaboration with district staff, created two modules for each, 3rd, 4th, and 5th grades¹. The modules provided teachers with the resources they needed to introduce lessons from CS into the literacy block.

To investigate the use of Time4CS modules, especially the CS component, the study asked “What are the effects of implementing CS lessons within an integrated curriculum on grade 3-5 students’ attitudes toward CS and their academic achievement?” To answer this question, we examined two sub-questions:

RQ 1: How is the implementation of Time4CS modules associated with grade 3–5 students’ academic achievement outcomes?

RQ 2: How is the implementation of Time4CS modules associated with grade 3–5 students’ attitudes toward school and CS?

Time4CS modules and context description

Time4CS modules were designed to be embedded within the district’s 180-min literacy block. The first 90 min of each block was dedicated to literacy and included the study of ELA texts and student engagement in classroom-based ELA centers. The second half of the block was dedicated to Time4CS module activities. Each module was focused on a social-studies standard-based theme with an associated problem. Students were provided opportunities to address this problem through their experiences with literacy, science, and social studies lessons that were connected to the Code.org CS Fundamentals program. The six modules consisted of daily, 90-min lesson plans and each module was taught over five to seven weeks. The lessons were organized into week-long lesson “collections” aligned to the Florida State Standards in ELA, science, social studies, and CS. Teachers who taught Time4CS modules were expected to teach all lessons in the Code.org Fundamentals course assigned to their grade level, and were also invited to teach additional CS lessons from Code.org, Scratch, or other sources (e.g., Barefoot coding, Kahn Academy).

Module example

One of the 4th grade modules, titled Invasive Species, asks students to generate solutions to the problem of invasive species in the local ecosystem. In the Invasive Species module, students are asked to take the role of an ecologist and investigate the real-world problem of the invasive Burmese python in the Everglades (e.g., how it was introduced to the ecosystem, how it is affecting native populations, and what is currently being done to tackle the problem) and develop a project to illustrate the problem and possible solutions to local citizens and government officials at a town hall meeting.

This module includes lessons to address content area standards in science (e.g., basic needs of living things, energy flow in an ecosystem, and interdependence), social studies (e.g., local and state governments and civic engagement), and CS (e.g., crowd-sourcing, sequencing, conditionals, events, functions, programming, debugging, and models/simulations). Students apply English Language Arts (ELA) standards as they research, read, write, and present orally about the topic of study.

The module is composed of five lesson “collections” (i.e., Everglades and its ecosystems, food chains in the Everglades, living in the Everglades, invasive impacts on the Everglades, and citizens affect change in the Everglades). Each collection contains four, approximately 60-min lessons for a total of 20 lessons. In the course of the lessons, students are asked to work in Scratch or to complete Code.org lessons associated with the module activities and to create a project (e.g., simulation, quiz, and game) in Scratch.

Each lesson collection follows the same structure: lesson 1 addresses science and social studies content standards needed to understand the problem; lesson 2 is an “unplugged” lesson that introduces the computational thinking (CT) and CS concepts without the use of a computer; lesson 3 asks students to revisit the problem and apply ELA and writing standards as they research, read, and write to delve further into concepts introduced in lesson 1; and lesson 4 asks students to apply the CS and CT concepts learned (in the previous “unplugged” lesson).

The module’s lesson design was informed by the Biological Sciences Curriculum Study (BSCS) 5E Instructional Model, which has been implemented in elementary, middle, and high school integrated science classrooms since the 1980s (Bybee et al., 2006). The 5E model includes five phases: engage, explore, explain, elaborate, and evaluate. Please see Appendices 1, 2, and 3 for a more detailed outline of lessons in the Invasive Species module and brief descriptions of the other modules. See Appendix 4 for a more in-depth description of the 5E model.

Teacher preparation

Treatment school teachers as well as support staff members (e.g., literacy coaches, media specialists, and specials teachers) from their schools participated in a 3-day professional development institute in summer 2016 prior to implementation of their first module. The institute was led by veteran teachers who were serving in a BCPS role as CS instructional facilitators. Teachers also participated in a 2-day institute during the school year (e.g., Winter 2016–2017) prior to implementation of the second module. The professional development institutes included CS Fundamentals content and pedagogy from Code.org and pedagogy in interdisciplinary instruction from the Science IDEAS model. Participating teachers were provided time during the professional development session to collaboratively plan for successful implementation. Prior to beginning the institute for module 2, teachers completed two reflections on implementation barriers and successes experienced when teaching module 1, one through the study’s implementation survey instrument and one as a whole group reflection. In order to support and provide on-going professional development, the two CS instructional facilitators provided each treatment school personalized support at the school sites. This support included assisting with implementation planning, coaching, and lesson modeling. Throughout the process, the teachers and the CS instructional facilitators maintained journals with reflections on the barriers and support structures needed to implement the modules.

Research design and methods

Sample

Broward County Public Schools is one of the largest school districts in the USA, serving approximately 271,000 students from diverse racial, ethnic, and socioeconomic backgrounds. Sixteen elementary schools were enrolled in the study with 321 teachers and 5791 students in grades 3–5 participating in data collection.

We used a randomized block design to create treatment and comparison conditions (Dunlap, Cortina, Vaslow, & Burke 1996; Kirk, 2007). BCPS staff recruited schools to participate, and then schools were matched using a set of criteria specified a priori based on the project goals. Specifically, schools were matched based on (1) a school-level ranking derived from students’ Florida Standards Assessment (FSA) scores; (2) the percentage of students receiving free and reduced-priced lunches (proxy for socioeconomic status [SES]); (3) the ethnic/racial diversity of the student body; (4) the percentage of students who were classified as English language learners (ELL); (5) the percentage of students with physical, cognitive (i.e., learning), social, or emotional disabilities; and (6) the percentage of students enrolled in the school’s gifted/high-achieving program. Once schools were matched, they were randomly assigned to either the treatment or comparison group. Schools with large and small student body populations were equally distributed across both study conditions.

Nearly all (98%) participating teachers reported having a degree in a subject other than CS. Their teaching experience ranged from 1 to 30+ years, and their years of teaching CS ranged from 0 to 12 years (M = 1.63 years). The student samples were diverse and distributed across grade levels (Appendix 5).

Theoretical framework and variables

This study utilized an implementation measurement approach to understand associations between Time4CS module implementation and student outcomes. The study design was grounded in an innovation implementation theoretical framework that calls for clearly defining and organizing the innovation (i.e., Time4CS modules) by its components (Century et al., 2012). A “component approach” is now widely accepted in implementation science (Century & Cassata, 2016), and it enables researchers to identify particular innovation components that actually occur and the ways the enactments of those components are related to desired outcomes.

Components are organized in two general categories: (1) structural components, including procedural components such as lesson order or omission as well as educative components, such as background content information; and (2) interactional components that include pedagogical components entailing teacher behaviors and interactions, and participant components entailing student behaviors and interactions (Century & Cassata, 2014). In this study, we measured structural, procedural implementation (e.g., how many lessons were omitted and which CS lessons did the teachers use) and interactional, pedagogical implementation (e.g., teacher facilitation of group work or cognitively demanding work). A summary of all components and other variables measured is presented in Table 1.

Table 1 Conceptual overview of the components under investigation

Implementation measurement variables

We used Hierarchical Linear Modeling (HLM) to examine potential differences in outcomes of interest for students in treatment (i.e., Time4CS modules) and comparison (i.e., “business as usual” instruction; no Time4CS modules) classrooms. Many implementation studies only compare treatment and comparison groups on these differences, treating interventions as a single “black box” with no explanation for why differences emerged. In this study, we used a component approach for measuring implementation to ascertain not only if there were differences between conditions but also to determine which parts of the intervention were associated with particular outcomes. Additionally, because this study’s over-arching research question focused on CS, we honed in on the structural components of Time4CS modules related to CS: the grade-level assigned Code.org CS Fundamentals lessons.

In BCPS, Code.org, Scratch, and other CS lessons are available to all teachers, not only those teachers using Time4CS modules. Rather than ignore their use in the comparison group, it was essential that we measure CS engagement in treatment and comparison groups. Doing so enabled us to more clearly determine the extent to which these groups were truly different in associations between engagement with CS and student outcomes.

Specifically, each Time4CS module had an associated grade-level assigned Code.org Fundamentals course to complete. All teachers had the opportunity to incorporate additional non-grade-level assigned Code.org lessons; therefore, we also measured students’ exposure to these lessons. Each teacher was assigned a completion percentage score for both grade-level and non-grade-level assigned CS lessons by dividing the number of lessons completed by the total number of possible lessons. We also measured teachers’ use of additional CS activities (e.g., Barefoot Computing) using a yes/no scale (Table 2).

Table 2 Structural component implementation measures

We also measured interactional pedagogical components of Time4CS modules. These components entailed the instructional strategies embedded in Time4CS modules that teachers were expected to enact during module implementation. These instructional strategies included teachers’ self-report of their use of interdisciplinary teaching practices as well as their facilitation of group work, cognitively demanding work, and intellectual risk-taking (Table 3).

Table 3 Interactional component implementation measures

The implementation framework used in the study has a companion framework that organizes the contexts and conditions (“factors”) that affect innovation implementation into several levels (i.e., individual, organizational, and environmental) (Century & Cassata, 2014). This study explored direct effects of three individual (teacher) factors—innovativeness, resourcefulness and coping, and years of teaching CS (Table 4).

Table 4 Teacher factors measured

Outcome variables

Outcome variables included student attitudes toward school in general and toward CS, and student academic achievement outcomes (Table 5). For attitude measures, students completed questionnaires at pre- (i.e., September 2016) and post- (i.e., May 2017) intervention. All items were completed on a 5-point Likert rating scale with values ranging from 1 (Disagree a lot) to 5 (Agree a lot), and each scale anchor had a corresponding smiley face representing its value. Across all scales, higher scores represented more positive attitudes.

Table 5 Student outcome measures

The academic achievement outcomes included Achieve3000 literacy scores and the FSA ELA, mathematics, and science scores. Achieve3000 LevelSet© is an online assessment tool that measures students’ reading ability and text difficulty to match them to appropriate informational texts. LevelSet is administered up to three times a year, beginning with a baseline assessment at the start of the year, followed by an interim assessment at mid-year, and a post-assessment at the end of the year. Students are assigned a Lexile score, which serves as a proxy for literacy ability level.

In the spring of each academic year, students in grades three, four, and five complete FSAs in ELA and mathematics. The FSA in science is administered only to 5th grade students. Students respond to items in multiple ways, including creating graphs, writing extended responses, and using other interactive features. Different types of questions are designed to assess students’ higher-order critical thinking skills and to provide students with a range of options to demonstrate their learning in each subject area. Students are assigned an achievement level scale score with higher scores representing higher academic achievement in a given subject area.

Data collection

Teachers’ structural and interactional implementation was measured with a post-implementation questionnaire taken after they completed Time4CS modules. Attitudes were measured using a pre-post design to capture changes in teachers’ and students’ attitudes over the 2016–2017 academic year. Teachers’ post-Time4CS module implementation questionnaire and post-attitude questionnaire were combined for ease of administration. All questionnaires were administered online using Qualtrics©, and took approximately 20 min to complete. Teachers facilitated student completion of questionnaires during regularly scheduled class time and were present to answer questions. Teachers completed questionnaires at their convenience and were provided a $15 store card for their participation. The district provided de-identified student academic achievement data matched to individual student study IDs, which were associated with individual teachers’ classrooms. The district also provided socio-demographic data for students, which complemented their self-reported gender, race, and ethnicity data shared through the post-questionnaires.

Analysis samples

Due to issues related to matching and missing data, the samples differ for the analyses investigating the attitudinal and academic achievement outcomes (Table 6). The sample for the student attitude analyses was reduced as a result of teacher and student error when entering study ID numbers. The samples available for the academic achievement outcome analyses varied slightly due to matching and missing data issues. Before conducting analyses, we examined missing data in each dataset and used listwise deletion to handle students and teachers who were missing data on key study variables.

Table 6 Analysis samples

Analytic approach

HLM (i.e., HLM 7 software; Raudenbush et al., 2011) was used to examine associations between implementation (i.e., teacher implementation of Time4CS modules, amount of CS instruction, and presence of teacher instructional practices) and student outcomes. HLM was selected because it is adept at dealing with hierarchically structured data (Raudenbush & Bryk, 2002), such as students nested within classrooms. In this situation, student outcomes may be explained by predictors at varying hierarchical levels (Raudenbush, 1988). Therefore, two-level random intercept HLM with individual student (e.g., gender, SES, and grade level) variables entered at the first level, and teacher (e.g., instructional practices) variables entered at the second level were conducted to examine students’ post-intervention academic achievement scores and school and CS attitudes.

For both research questions, two-level unconditional HLM including only the outcome variable of interest was conducted as the baseline model. The intra-class correlation (ICC) was calculated to assess how much variance in the dependent variable could be explained by between-teacher variance. An ICC value greater than 0.10 was specified to distinguish which outcome variables should be examined using the conditional model (Kreft & de Leeuw, 1998). For all conditional models, separate models were conducted for each outcome variable.

Primary Analytic Model

RQ1 examined associations between implementation of Time4CS modules and students’ academic achievement outcomes. Implementation was measured in three ways: (1) Time4CS module use or not; (2) amount of CS lessons taught (i.e., structural implementation); and (3) extent of particular teacher instructional practices present (i.e., interactional implementation).

The student-level model (i.e., level 1) included the following variables (all categorical variables were dummy coded with reference group listed as follows): gender (reference group: male students), race (reference group: White students), ethnicity (reference group: Non-Hispanic students), grade level (reference group: 3rd grade students), English language proficiency (reference group: native English speakers), and free and reduced-price lunch status (reference group: students not receiving free and reduced-price lunch). Students’ previous experience completing Code.org activities (reference group: no previous experience completing Code.org activities) was also included at level 1. All categorical variables were entered as uncentered terms. For RQ2 (i.e., attitudinal outcomes), student attitudes toward school in general and CS measured through the pre-intervention questionnaire were also included at level 1 as covariates, and they were centered around the group mean.

The teacher-level model (i.e., level 2) included the following variables: Time4CS module completion, teacher facilitation of group work, teacher facilitation of cognitively demanding work, teacher facilitation of students taking intellectual risks, teacher use of interdisciplinary teaching practices, and the completion percentage of grade-level assigned CS lessons. The percentage of non-grade-level assigned Code.org CS lessons, additional (non-Code.org) CS activities, and teachers’ previous experience with CS were also included. Variables representing teachers’ resourcefulness and coping and innovativeness were also included at level 2. All level 2 continuous variables were centered around the grand mean, whereas categorical variables were entered as uncentered terms (Appendix 6).

Results

In this paper, we only reported findings about the teacher-level variables related to the research questions under investigation. Findings from additional analyses related to the student-level variables (i.e., socio-demographic characteristics) will be presented in a separate paper.

Research question 1

How is the implementation of Time4CS modules associated with grade 3–5 students’ academic achievement outcomes?

Unconditional HLM

A two-level, unconditional HLM was used as the baseline model. The ICC coefficient was calculated for each academic achievement outcome variable to generate the amount of variation (i.e., Achieve3000 literacy, FSA and ELA, mathematics, and science scores) accounted for by between-teacher differences (Table 7).

Table 7 Academic achievement outcome ICC values

Conditional HLM

We conducted four separate analytical models to answer this question. Each analysis used a different student academic achievement outcome and associated teacher and student samples (Table 6).

Achieve3000 Lexile score

No significant differences in Achieve3000 Lexile scores emerged between treatment (Time4CS modules) and comparison (no Time4CS modules) groups. Interactional implementation components (i.e., teacher instructional practices listed in Table 3) were not significantly associated with students’ Lexile scores. However, completing a higher percentage of grade-level assigned CS lessons was significantly associated with lower Lexile scores (β = − 155.02, p < 0.05), whereas completing a higher percentage of non-grade-level assigned CS lessons was significantly associated with higher Lexile scores (β = 232.16, p < 0.01).

Florida Standards Assessment—ELA

No significant differences in FSA ELA scores resulted between treatment and comparison groups. Interdisciplinary teaching practices were positively associated with FSA ELA scores (β = 2.73, p < 0.05). No significant associations emerged for grade-level assigned CS lessons and FSA ELA scores. However, completing a higher percentage of non-grade-level assigned CS lessons was positively associated with FSA ELA scores (β = 16.43, p < 0.05).

Florida Standards Assessment—mathematics

No significant differences in FSA mathematics scores resulted between treatment and comparison groups. Interactional implementation was not associated with students’ FSA mathematics scores. We found no significant associations between grade-assigned CS lessons or additional CS activities. However, completing a higher percentage of non-grade-level assigned CS lessons was positively associated with FSA mathematics scores (β = 19.88, p < 0.05).

Florida Standards Assessment—science

Teachers’ self-reported resourcefulness and coping levels were negatively associated with FSA science scores (β = − 6.10, p < 0.05). All other associations were non-significant.

Research question 2

Is the implementation of Time4CS modules associated with increases in grade 3–5 students’ attitudes toward school and CS?

Unconditional HLM

A two-level, unconditional HLM was used as the baseline model, and the ICC coefficients were calculated in order to illustrate the amount of total variation in each student attitudinal outcome accounted for by between-teacher differences (Table 8). Conditional HLM was only run on outcome variables that had ICC values higher than 0.10 (i.e., school affinity, CS affinity, CS identity, and CS utility).

Table 8 Attitude outcome ICC values

Conditional HLM

We conducted four separate analytical models to answer this question. Each analysis examined a different student attitudinal outcome and utilized customized teacher and student samples (Table 6).

The results of all four models examining associations between implementation and attitudinal outcomes were the same. Students’ attitudes did not differ by study condition, and there were no significant associations between structural implementation (CS lessons and activities) or interactional implementation (teacher instructional strategies) and student attitudes. However, teachers’ self-reported innovativeness was significantly associated with students’ CS identity.

Discussion

Findings summary

There were no significant differences in attitudinal or academic achievement outcomes between students taught by teachers in treatment (Time4CS modules) and comparison (no Time4CS modules) conditions; however, teachers in the treatment group who taught Time4CS modules completed more CS lessons than teachers who did not teach Time4CS modules. Although no effect of study condition was observed, our component approach for measuring implementation enabled us to investigate how specific components of the Time4CS modules contributed to student outcomes. We consistently found positive associations between some of the Time4CS module components and several key academic outcomes. Specifically, we found that teaching a higher percentage of non-grade-level assigned CS lessons and use of interdisciplinary teaching practices were associated with higher academic achievement outcomes in ELA (FSA and Achieve3000) and FSA mathematics. Teachers’ innovativeness was positively associated with student CS identity attitudes. Alternatively, we found negative associations between the percentage of grade-level assigned CS lessons completed and Achieve3000 scores, and teachers’ resourcefulness and coping, and students’ FSA science scores.

Considerations for finding time for CS in the elementary school day

This study sought to address the challenge of finding time in the elementary school day for CS, and although no differences in academic achievement or attitudinal outcomes were found across study conditions, we were encouraged because our findings suggest no negative consequences for students’ FSA ELA or mathematics scores when CS is integrated into the curriculum through Time4CS modules. While many elementary practitioners designate the majority of the school day for teaching ELA and mathematics, these subjects alone will not serve our children well in the future (English, 2016; Regan & DeWitt, 2015). It is critical for young learners to have opportunities to engage in STEM experiences, including early exposure to CS (Kelley & Knowles, 2016; Wang, Hong, Ravitz, & Ivory, 2015). The current study demonstrated that using transdisciplinary Time4CS modules in the literacy block did not detract from success in ELA, mathematics, or science. Rather, the findings generated from this study suggest that concerns about lower test scores due to time taken away from ELA may be unfounded and require further investigation.

The role of CS

Perhaps the most important finding of the study was that it is possible to infuse CS into the literacy block during the elementary school day and that participation in CS, particularly the non-grade-level assigned (above or below grade level)Code.org CS lesson component of the modules was positively associated with students’ FSA ELA and mathematics scores. These effects may have emerged for several reasons. First, teachers may have hand-picked the non-grade-level assigned CS lessons to complement curricula they were already teaching. Or, they may have selected the lessons due to personal interest, which in turn, positively influenced their instructional practices and students’ subsequent academic achievement outcomes. Alternately, teachers may have prioritized completing CS lessons from higher grade levels because their students were already high academic achievers and seemingly capable of completing these more challenging CS lessons. Another possibility is that classrooms with more high-achieving students may have completed all grade-level assigned lessons more quickly and had additional time for non-grade-level assigned CS lessons. This explanation would stand to reason as high-achieving students often need a faster pace of instruction so that they do not become bored or distracted (Lüftenegger et al., 2015). In contrast, teachers who need to spend additional time on other areas of the curriculum (e.g., literacy) would not consider doing more than the grade-level assigned lessons.

It is also important to highlight that even though both treatment and comparison teachers had the same access to all Code.org Fundamentals lessons through district resources, teachers in the treatment group (i.e., Time4CS modules) implemented a significantly higher percentage of both, non-grade-level assigned CS and grade-level assigned CS lessons. This finding is particularly important given that non-grade-level assigned CS lessons were positively associated with students’ academic achievement outcomes, regardless of study condition. This finding is noteworthy in that it suggests that the Time4CS modules may highlight and direct teachers to opportunities to engage their students with CS in ways that they would not have seen had they not been using the Time4CS modules.

In contrast, greater completion of grade-level assigned CS lessons had a negative effect on students’ Achieve3000 literacy scores. Achieve3000 differs from mandatory, state-wide FSA subject examinations as it is a level-set test designed to help direct students to a particular level of literacy activities. After students are grouped by literacy abilities, they engage in computer-based Achieve3000 activities until the next level-set test is administered. Previous research suggests that computer-based instruction programs like Achieve3000 can add time burdens to the elementary school day because it takes time for teachers to facilitate students’ interactions with the online activities (Kunze & Rutherford, 2018). Given that success on the Achieve3000 test depends largely on students’ exposure to and engagement with the Achieve3000 program, it is possible that carrying out Time4CS modules during the literacy block took time away from those Achieve3000 activities, resulting in lower Achieve3000 scores. Future revisions of Time4CS modules should account for this finding and balance emphasis on CS lessons with other aspects of literacy instruction.

The role of interdisciplinary strategies

Time4CS modules’ foundational instructional strategy also emphasized the use of interdisciplinary teaching practices, and our findings suggest that carrying out these specific teaching practices within the context of transdisciplinary Time4CS modules shows promise for students’ literacy achievement. These results are consistent with other studies that demonstrate the benefits of problem-based learning, a commonly utilized interdisciplinary approach linked to student academic achievement (Han, Capraro & Capraro, 2015; Tandogan & Orhan, 2007). Elementary teachers often teach all core subjects (i.e., literacy, math, science, and social studies), and as a result, they are well positioned to use interdisciplinary teaching strategies in their daily instruction (Avargil, Herscovitz, & Dori 2012; Wood, 1997). As such, even though no differences in outcomes were observed based on study condition, Time4CS modules still have the potential to facilitate more opportunities for teachers to use interdisciplinary teaching strategies with their students. Further development of the Time4CS modules with a focus on increasing teachers’ use of interdisciplinary practices should be a priority for future Time4CS module revisions.

It should also be noted that teachers’ resourcefulness and coping were negatively associated with students’ FSA science scores. Although resourcefulness and coping are hypothesized to be associated with the use of interdisciplinary strategies and ultimately positive student achievement, in this study, they seemed to have a negative effect. Additional research is needed to better understand associations between teacher factors related to CS and other subject-area outcomes.

Attitudinal outcomes

We found no significant associations between Time4CS module implementation and students’ attitudes toward CS. Despite some research that indicates the malleability of students’ (particularly young learners) attitudes toward STEM subjects (Cheng & Hau, 2003), these findings are consistent with other research that found students’ attitudes and perceptions towards CS to be unchanged after exposure to a 7-week introductory CS course (Grover & Pea, 2016). Attitudes are a complex psychological construct, and much research suggests that they remain stable even in changing situations (Bohner & Dickel, 2011; Fazio, 2007; Petty, Briñol, & DeMarree, 2007; Schwarz, 2007). As such, students in the current study may already hold well-developed attitudes toward CS by the time they reach third grade. Therefore, experiences with CS over the course of one academic school year through two, 6–8 week Time4CS modules may not be sufficient to make measurable attitudinal shifts.

Another explanation may come from other research that suggests that attitudes toward STEM and CS subjects have been shown to decline as students progress through their academic careers (Tytler & Osborne, 2012). These were young students who started the experience with relatively high attitudes toward CS. It is possible that some did not fully understand what CS was before this experience and found it to be less appealing once they learned it did not align with their preconceptions. It is also consistent with the research and reasonable to consider that the students’ experiences with CS during the study were more challenging than previous ones, which would naturally affect some students’ ability beliefs and affinity.

Revisions to Time4CS modules must closely consider these findings given the connection between students’ attitudes toward STEM and CS subjects, completion of STEM and CS courses, and pursuit of STEM and CS careers established in the literature (Regan & DeWitt, 2015; Wang, 2013). More specifically, incorporating instructional practices specifically focused on changing students’ attitudes may help Time4CS modules make a positive contribution to students’ CS attitudinal development, especially students’ CS identity and CS self-efficacy attitudes. Further, we believe that increasing attention to the needs of students in particular socio-demographic groups in future revisions may also address the findings about attitudes. Future module revisions should do so by incorporating what is known about students with learning differences and culturally responsive pedagogy.

Limitations

The results of the current study must be interpreted in light of several limitations. First, this study is exploratory, and the Time4CS modules were in the early stages of development³ during this study. As a result, Time4CS modules may have been difficult for teachers to implement. The external evaluator for the project confirmed this may be the case; her findings revealed that the teachers felt the modules were too long, particularly because they concurrently felt pressure to directly focus on improving standardized test scores.

Additionally, this study employed a pre- and post-research design where data were collected at the beginning and end of one school year. The research team was unable to control for the school’s academic calendar and standardized testing schedule, which affected the cadence of instruction and timing of data collection. For example, some teachers may have taught one or two lessons a week, whereas others may have taught the lessons every day. Other teachers may not have finished, or even started, implementing Time4CS modules until after standardized tests were administered in the spring due to pressures from administration to have students adequately prepared. Therefore, exploring differences in duration and scheduling of Time4CS module instruction is an important direction for future research. Finally, the current study collected school and CS attitudinal data through questionnaires administered electronically on Qualtrics. After conducting factor analyses, we determined that our scales were psychometrically valid and reliable (see Cronbach’s α values reported in Tables 3, 4, and 5); however, focus groups or one-on-one interviews may provide more suitable data collection methods with young learners (i.e., third–fifth graders) who may have difficulties with reading comprehension, language skills, concentration, or use of online questionnaire-completion platforms (Borgers et al., 2000).

It is also worth noting that this study used teacher questionnaires to collect measurements of teachers’ use of interdisciplinary teaching practices, and facilitation of group work, cognitively-demanding work, and intellectual risk-taking. Given that the data were self-report, we acknowledge that teachers may have inaccurately (more specifically overestimated) reported their use of these pedagogical approaches. Therefore, future research should capitalize on mixed-methods research designs, including the collection of quantitative questionnaire data, classroom observation data, and qualitative interview data from teachers to address this limitation.

Conclusions, implications and future directions

This study is an important step toward solving the problem of finding time for CS in the “crowded” elementary school day. It has demonstrated that the strategy employed here—transdisciplinary problem-based modules, which integrate the teaching of CS principles with other subject areas—is one possible way to bring more CS opportunities to younger learners. Moreover, implementing such modules is linked to more positive student academic achievement outcomes. However, this study, and specifically the Time4CS modules in their current form (i.e., content, structure, delivery, and educative resources), is merely a starting point for inquiry into the integration of CS in elementary classrooms. Results also illustrate the need to improve the quality of the Time4CS modules to bolster the presence of interdisciplinary practices in elementary school classrooms, and to emphasize teaching practices designed to positively influence student attitudes toward school and CS. A more targeted focus on teachers’ feedback⁴ will help to improve the Time4CS modules and approaches aimed at integrating CS with literacy and other subjects in elementary classrooms for future use with elementary-aged students (Estapa & Tank, 2017; Margot & Kettler, 2019).

Attitudinal outcomes are an urgent area for further investigation because students’ positive attitudes toward CS are related to their future choices about CS and STEM majors and careers (Wang et al., 2015). It is particularly important to consider this with attention to students who are underrepresented in STEM fields. It is essential that strategies to increase attitudes in interdisciplinary teaching contexts and studies of these strategies account for the widely varied elementary school population (i.e., racially/ethnically diverse samples, male and female students, and students from different SES backgrounds). Moreover, future strategy developments must also account for students in different settings (i.e., rural, urban, and suburban school districts) and grade levels (i.e., elementary, middle, and high school grades).

Attention must also be paid to the role that teacher attitudes play in implementation of interdisciplinary problem-based module learning experiences like Time4CS modules. Teachers’ attitudes toward CS and interdisciplinary teaching practices are likely to impact their implementation of interdisciplinary problem-based modules and need to be considered in both module improvements and study designs. Additionally, teachers’ attitudes toward accepting new instructional approaches—their readiness for change—and how they affect instruction and student outcomes also command attention in future work. Measuring attitudes and other teacher characteristics will inform not only how to develop problem-based integrated modules, but also how to support their implementation.

Finally, in addition to building on the promise of integrated problem-based units with the strategies outlined above, it is also essential to generate new ideas for the future. Developers and researchers can work together to imagine and create new ways to generate interdisciplinary learning opportunities (that include CS) that are more easily accessed, embraced, and implemented. We need to ask ourselves how we can create instructional resources that teachers can more easily use, that capitalize on and leverage technology in ways that mesh with classroom conditions, and that tap into problems that are meaningful for students. Doing so will end the pattern of giving our youngest learners short shrift and provide them with the opportunities they need to thrive in the future.

Appendices

Appendix 1. Invasive Species Overview

Table 9 Collection descriptions

Appendix 2. Collection 2 lesson descriptions

Table 10 Lesson descriptions

Appendix 3. Module descriptions

Grade 3 modules

Module 1

In this module, “Lighting Up North America,” students were presented with a problem-based learning experience that applied the grade level standards in science and social studies. Students were given the following scenario: “You are part of a team that has just been hired to develop a travel guide and tourist map for North America. In order to learn more about the continent, your co-worker, The Traveler, has been visiting all of the different countries and regions in North America, including, the United States, Canada, Mexico, and the Caribbean. She has been sending you postcards, letters, photographs, and objects from the different locations that she has been visiting so that you can compile (or record and organize) this information into a travel guide and map.” This module included the use of Code.org CS Fundamentals course 2, which focused on algorithms, sequence, debugging, loops, and conditionals.

Module 2

This module focused on plants and how plant structure responds to its environment. Students were posed with the problem to design a genetically engineered plant according to specifications from a customer in a particular region of North America so that it will be able to thrive, enhance the lives of the customer and other inhabitants of the region, and help boost the economy. This module included the use of Code.org CS Fundamentals course 2, which focused on algorithms, sequence, debugging, loops, and conditionals.

Grade 4 modules

Module 1

This module, “Florida history with science applications to renewable energy,” provided the following problem-based learning scenario: “The owner of a popular Florida theme park knows that many foreign tourists visit her theme park each year. One family visiting from Latin America enjoys the theme park, but told the owner that they would like to learn more about Florida and its history while visiting. The owner has an idea to create a ride that will teach visitors about Florida. Her challenge to you is to design and build an attraction for a popular Florida theme park that will teach riders about Florida's significant people, events, and industries. Additionally, she would like the ride to utilize an alternative energy source in order to be more environmentally friendly.” This module included the use of Code.org CS Fundamentals course 3 and Scratch animations.

Module 2

This module, “Invasive Species,” is described in Appendices 1 and 2.

Grade 5 modules

Module 1

This module, “How did we get here?” focused on navigation, including effects on navigation and navigation tools. Students were provided with the following problem-based goal: "You work for a blended-learning educational materials publisher. They created games and kits that use hands-on building and computer technologies to teach about a concept. Your team must design a kit that can be sold to families with children in 4th – 6th grade who want to learn more about early explorers and how technological advances in navigational tools, ship building, weather prediction, and climate affected exploration over time." This module used Code.org CS Fundamentals course 4 and a culminating project using Scratch.

Module 2

In this module, “Westward Expansion,” students are posed with the job of developing a storyboard and animation (or game) in Scratch to help engage others in the events, scientific contributions, and characters in westward expansion. This module used Code.org CS Fundamentals course 4 and a culminating project in Scratch.

Appendix 4. Overview of the 5E model

Biological sciences curriculum study 5E instructional model (Bybee et al., 2006) components

• 1. Engage. Teachers assess students’ prior knowledge of the topic by discussing previous learning experiences and orienting students toward intended learning outcomes.

• 2. Explore. Teachers provide students with exploration activities designed to provide them with a common base of experiences for building new information. Students use prior knowledge to generate questions and new ideas to build an understanding of the current concepts.

• 3. Explain. Teachers directly introduce new concepts or skills and provide the opportunity for students to demonstrate their newfound conceptual understanding.

• 4. Elaborate. Teachers challenge and extend students’ understanding as they apply skills, concepts, and vocabulary to new situations.

• 5. Evaluate. Teachers assess students as they apply new concepts and skills and encourage them to assess their own understanding.

Appendix 5. Students’ socio-demographic characteristics in FSA ELA/mathematics sample

Table 11 Students’ socio-demographic characteristics in FSA ELA/mathematics sample

Appendix 6. Two-level HLM for examining associations between teacher implementation measures and student outcomes

Level 1 (students)

$$ {Y}_{ij}={\beta}_{0j}+\sum \limits_{p=1}^p{\beta}_{pj}{(X)}_{pij}+{r}_{ij} $$

Level 2 (teachers)

$$ {\beta}_{0j}={\gamma}_{00}+\sum \limits_{s=1}^s{\gamma}_{0s}{(W)}_{sj}+{u}_{0j} $$

$$ {\beta}_{pj}={\gamma}_{p0} $$

Y_ij represents student outcome (i.e., each academic achievement score and students’ attitudes towards school and CS) for student i taught by teacher j, and X is a vector of student demographic characteristics, including race/ethnicity, gender, free and non-reduced lunch status, English language proficiency, previous experiences on code.org, and grade level. β_0j represents the mean outcome in teacher j’s classroom. r_ij is a random student effect, that is, the deviation of student ij’s outcome score from the classroom mean. These effects are assumed to be normally distributed with a mean of 0 and variance σ². γ₀₀ represents the average of the classroom means on each student outcomes across teachers. W represents a vector of teachers’ implementation measures. u_0j represents the unique effect of teacher j on mean student outcome holding W_sj constant. At the level 2, β_0j is allowed to vary randomly across teachers, and the coefficients for students’ demographic characteristics are fixed across teachers.