Conceptual connections between science and engineering in elementary teachers’ unit plans

Abstract

Background

Around the world, efforts are underway to include engineering design as part of elementary science instruction. A common rationale for those efforts is that Engineering Design-based Science Teaching (EDST) is a productive pedagogical approach for developing students’ understanding of core science concepts. Effectively utilizing EDST requires that teachers develop design activities that are highly connected to science content so that students can apply and expand their understanding of relevant concepts. In this study, we examine how a group of elementary (grades 3–5) pre-service and in-service teachers incorporated EDST into their planned science instruction. Those teachers were participants in a professional development project aimed at supporting EDST. We examine the ways that participants used EDST, the extent to which engineering design activities were connected to science concepts, and factors associated with those connections.

Results

Most of the participants in the study developed science units in which an engineering design activity was placed at the end of the unit. Approximately half of those design activities lacked connections to the science concepts in the unit; they were typically related to the topic of the science unit, but did not require the use or development of key science ideas. Eleven percent of participants developed engineering activities with deep connections to science concepts, and 35% developed activities with shallow connections. No differences were found between life science, physical science, and earth/space science units in terms of the extent of conceptual connections. However, we did find that participants who utilized and adapted published engineering curriculum materials rather than make them from scratch were more likely to have unit plans with higher levels of conceptual connections.

Conclusions

Our findings suggest that elementary teachers need additional support in order to effectively utilize EDST in their classrooms. Even within the context of a supportive professional development project, most of the engineering activities developed by our participants lacked substantial connections to the science concepts in their unit plans. Our findings highlight the value of high-quality curriculum materials to support EDST as well as the need to further expand the curriculum resources that are available to elementary teachers.

Introduction

Many recent standards and policy documents call for engineering to be addressed in K-12 science classrooms (Moore, Tank, Glancy, & Kersten, 2015; National Research Council, 2012; National Research Council, 2014; NGSS Lead States, 2013). Although many reasons exist to include engineering in pre-college education, a primary rationale to include engineering in science instruction is its potential to enhance students’ science learning (Apedoe, Reynolds, Ellefson, & Schunn, 2008; National Academy of Engineering, 2009; National Research Council, 2012). Typically, engineering is incorporated into science classrooms by engaging students in design activities, in which students develop technological solutions to context-specific problems (Brophy, Klein, Portsmore, & Rogers, 2008; National Academy of Engineering, 2009). Engineering design activities are often viewed as productive settings for science learning because they provide authentic contexts in which students can develop and apply scientific knowledge (Atman et al., 2007; Johri & Olds, 2011; Mehalik, Doppelt, & Schuun, 2008; Puntambekar & Kolodner, 2005; Riskowski, Todd, Wee, Dark, & Harbor, 2009; Roth, 2001). Engineering in the science classroom can therefore be viewed as a pedagogical approach, which we will refer to as Engineering Design-Based Science Teaching (EDST).

A growing body of empirical work is emerging around the use of EDST, and multiple studies have found evidence that engineering can be used to support students’ interest in and learning of science (e.g., Capobianco, DeLisi, & Radloff, 2018; Cunningham & Carlsen, 2014; Kolodner et al., 2003; Mehalik et al., 2008; Riskowski et al., 2009; Schnittka & Bell, 2011; Wendell & Rogers, 2013). Not all engineering design activities, however, will necessarily advance students’ scientific knowledge. Students can easily overlook the relevant scientific ideas during an engineering design activity and fail to make the connections intended by the teacher (Berland, Steingut, & Ko, 2014; Kanter, 2010; Puntambekar & Kolodner, 2005; Schnittka & Bell, 2011; Silk, Schunn, & Cary, 2009). Teachers must play a critical role in developing design activities that emphasize the relevant science ideas while also helping students to recognize and make those connections during instruction. That is not an easy task for teachers and requires that they be knowledgeable in science and engineering as well as in their respective pedagogies (Dare, Ellis, & Roehrig, 2014, 2018; Walkington, Nathan, Wolfgram, Alibali, & Srisurichan, 2014). Those demands are particularly substantial for elementary teachers, who typically have limited preparation in science and engineering (Banilower et al., 2018).

Engineering design-based science teaching

In this study, we focus on using engineering design as an instructional approach to promote students’ learning of science, a perspective Purzer and Quintana-Cifuentes (2019) refer to as “engineering as pedagogy.” As a pedagogical approach, we define EDST as the engagement of students in the constellation of engineering practices associated with technological design and development in order to advance students’ science learning. We take a broad view of what constitutes engineering design the classroom in that EDST need not necessarily engage students in a complete process of defining an engineering problem all the way through creating a finished product. Instead, consistent with the nature of authentic engineering practice (Pleasants & Olson, 2019), essential features of classroom engineering instruction include students’ meaningful engagement in core engineering practices (cf. Cunningham & Kelly, 2017) while addressing a problem of technological design or development.

Although the current emphasis on engineering design in science standards is relatively recent, the use of technological design as a pedagogical approach is not novel (Lewis, 2006). In a summary of studies from the late 1990s, Roth (2001) presented evidence of how middle school students’ learning of concepts such as force and energy were deepened by engaging in the design of simple machines. Fortus, Dershimer, Krajcik, Marx, and Mamlok-Naaman (2004) developed the Design-Based Science approach to “help students construct scientific understanding and real-world problem-solving skills by engaging them in the design of artifacts” (p. 1082). In addition, the Learning By Design approach developed by Kolodner et al. (2003) similarly promoted students’ science learning via engagement in engineering design. In the intervening years, additional studies have provided evidence that engineering design activities can support students’ learning of science concepts (e.g., Apedoe et al., 2008; Goldstein et al. 2018; Kanter, 2010; Mehalik et al., 2008; Penner, Lehrer, & Schauble, 1998; Schnittka & Bell, 2011; Sidawi, 2009).

Like any pedagogical approach, EDST has various strengths and challenges. Design activities can create opportunities for students to reason with science concepts, but not all engineering design experiences necessarily promote science learning. Most crucially, an engineering design activity must be strongly connected to the science concepts students are meant to learn. That is, the design task should necessitate the application of scientific ideas such that students must either use their existing knowledge or develop new knowledge. Engineering activities will not advance students’ learning of science if they “do not create a need for students to explore and apply the underlying… science concepts” (Berland et al., 2014, p. 718). Even when an apparent need exists for students to utilize certain science ideas, students might fail to recognize those ideas as relevant or useful (Chao et al., 2017). To promote students’ science learning through EDST, teachers therefore have a critical role to play beyond simply providing students with an engaging design activity. They must help students recognize how science concepts can be used to make informed design decisions, analyze the performance of different designs, and reflect on design choices (Goldstein et al. 2018; King & English, 2016; Levy, 2013; Purzer, Goldstein, Adams, Xie, & Nourian, 2015; Roth, 2001; Tytler, Prain, & Hobbs, 2019).

Developing and implementing engineering design lessons in the ways described above is challenging for teachers (Dare et al., 2014, 2018; Walkington et al., 2014), and they often express concerns about their abilities to accomplish the task (Radloff & Capobianco, 2019). EDST not only requires that teachers deeply understand both science and engineering (Frykholm & Glasson, 2005) but also possess the pedagogical skills necessary to draw students’ attention to how the science ideas are relevant to a particular engineering context (Chao et al., 2017). These demands are likely to be particularly challenging for elementary teachers. As content generalists, elementary teachers typically have limited preparation in science and engineering and have generally lower confidence teaching those subjects (Banilower et al., 2018). Professional development (PD) efforts have therefore been deployed to provide elementary teachers with adequate support for EDST (e.g., Capobianco & Rupp, 2014; Crotty et al., 2017).

Characterizing teachers’ use of EDST in PD contexts

In research examining how teachers utilize EDST after receiving PD, much of the focus has thus far been placed on how teachers engage their students in certain engineering design practices or the extent to which certain components of the engineering design process are present during design activities (e.g., Capobianco et al., 2018; Capobianco & Rupp, 2014; Wheeler, Navy, Maeng, & Whitworth, 2019). Generally, studies of PD have documented success in terms of helping teachers incorporate engineering design and certain engineering practices into their science instruction (Capobianco et al., 2018; Dare et al., 2018; Guzey, Tank, Wang, Roehrig, & Moore, 2014; Maeng, Whitworth, Gonczi, Navy, & Wheeler, 2017). Even though it is rarely the main research focus, the degree to which the engineering design activities developed by PD participants are connected to science concepts has also been described in several studies. As described below, however, the levels of success in that area have been mixed.

Capobianco and Rupp (2014) investigated integrated science-engineering units developed by grades 5–6 teachers who participated in a PD aimed at EDST. They analyzed the unit plans on a wide range of dimensions, characterizing the extent to which the planned instruction reflected the approaches presented by the PD. Only one dimension attended specifically to the question of how engineering was used to advance the science goals of the unit: “Integration of the design task” (p. 264). They found that teachers had difficulty with that dimension: “Missing from their plans was attention to key science concepts and their placement, use, and application within a design task” (p. 265).

Also in a PD context, Guzey et al. (2014) examined unit plans and artifacts created by grades 3–6 teachers as they incorporated engineering design into their science instruction. Many of the teachers in their study implemented design activities from published materials such as Engineering is Elementary (Museum of Science Boston, 2007), but many also developed their own engineering design lessons or substantially modified previously published lessons. The analysis focused primarily on the nature of the engineering design challenges implemented by the teachers, but the researchers also noted that many design activities (particularly those adhering to a minimal “build and test” model) “did not allow students to apply appropriate and/or adequate science knowledge” (p. 145). They also found that most of the engineering design lessons targeted physical science rather than life science content and speculated that engineering design might be more readily connected to physical science concepts.

Dare et al. (2014) studied how highschool physics teachers utilized engineering design in their classrooms after participating in a PD around EDST. While the teachers stressed the value of engineering as an application of science ideas, the researchers found that the engineering activities rarely made any explicit reference or use of science concepts. During interviews, the teachers often admitted that the science content was emphasized less than other objectives such as problem-solving. In a later study of grades 4–8 teachers, Dare et al. (2018) found that teachers often perceived tension when incorporating engineering into science instruction in that they felt a need to either focus on science or engineering, due in part to time pressures. The teachers in their study also expressed concerns about the extent to which students were making the desired connections to science content during engineering design activities.

Maeng et al. (2017) studied how grades 4–6 teachers implemented engineering design in their science instruction after participating in a PD that included EDST. Similar to Guzey et al. (2014), they found that most teachers utilized EDST when addressing physical science content and, to a lesser extent, earth/space science content; few teachers used EDST in life science units. Engineering design was typically situated as a way for students to apply science concepts at the end of a unit, and teachers often espoused science content learning goals when discussing the engineering instruction. However, the researchers did not systematically investigate the extent to which the science content goals were met by the engineering activities, although they provided some example design activities that appeared to be connected to science content.

In sum, studies of teachers’ use of EDST points to the challenging nature of developing design activities that promote students’ science learning. Most teachers utilize engineering design activities after science instruction as a way for students to apply their science ideas (Capobianco & Rupp, 2014; Dare et al., 2014, 2018; Guzey et al., 2014; Maeng et al., 2017). However, many engineering activities do not achieve that intended purpose because the science concepts are either not essential for the design problems or because the teachers do not help students recognize the relevance of the science ideas. Diefes-Dux (2014) has suggested that elementary teachers need multiple years of experience implementing EDST before they can effectively connect design activities to science content. In early years of implementation, Diefes-Dux found that elementary teachers often implemented engineering design lessons as standalone activities that were disconnected from science; over time, however, those teachers were more successful in tying engineering design activities to their targeted science concepts. An additional finding that has emerged from the research relates to the science content being addressed via EDST. Studies suggest that EDST is more readily utilized when teaching physical science or earth/space science concepts rather than life science concepts (Guzey et al., 2014; Maeng et al., 2017; Roehrig, Dare, Ring-Whalen, & Wieselmann, 2021; Roehrig, Moore, Wang, & Park, 2012).

Yet, while the studies summarized above address how teachers connected engineering design activities to science concepts, none made those connections the focus of study. If EDST is conceptualized as a pedagogical approach for advancing students’ science learning, this is an area in need of more detailed investigation. In particular need of investigation is the role that curriculum resources play as teachers plan for EDST, and whether those resources facilitate conceptual connections between science and engineering. Prior studies illustrate the breadth of ways in which teachers develop instructional units that utilize EDST. In some cases, teachers were provided published curriculum materials (e.g., Kelly & Cunningham, 2019) or with materials developed by researchers (e.g., Fortus et al., 2004; Puntambekar & Kolodner, 2005). In other cases, teachers and researchers collaboratively developed instructional units during PD (e.g., Crotty et al., 2017; Lehman, Kim, & Harris, 2014). In relatively few cases, teachers were mostly independent in creating novel instructional units after participating in PD (e.g., Dare et al., 2014; Maeng et al., 2017). In only a handful of those cases (e.g., Capobianco & Rupp, 2014; Maeng et al., 2017) were teachers’ lesson plans examined for the extent of the science and engineering connections present. Given the important and varied role that curriculum resources play in elementary teachers’ science instruction (Banilower et al., 2018), this is an area in need of further investigation.

Purpose of study

The present study seeks to provide insight into the issues described above. In the context of a PD project that introduced elementary teachers to EDST, we examine how participants planned engineering activities that were part of broader science units, with a focus on the extent to which those engineering activities were conceptually connected to science concepts. We aim to not only characterize the extent of those connections but also examine several factors that emerged as potentially relevant from our review of the literature above. We investigate the extent to which the science content domain is related to the conceptual connections made between engineering activities and science content. In particular, we investigate whether physical science content is indeed more conducive to EDST as suggested by prior studies (Guzey et al., 2014; Maeng et al., 2017; Roehrig et al., 2012). We also investigate whether the use of published engineering curricula leads to stronger connections to science content. Finally, because some individuals participated in our PD more than once, we had the opportunity to investigate whether repeated participation leads to stronger conceptual ties between engineering and science, as suggested by Diefes-Dux (2014).

Conceptual framework: teachers as curriculum makers

Unlike prior studies that have either provided teachers with curriculum (e.g., Fortus et al., 2004; Kanter, 2010; Riskowski et al., 2009) or co-developed curriculum with teachers development during a PD experience (e.g., Capobianco et al., 2018; Crotty et al., 2017; Guzey, Moore, & Harwell, 2016), we sought to understand how participants in our PD utilized the resources available to them to plan EDST. We view teachers not simply as implementers or conduits of curriculum, but as makers of the curriculum specific to their classrooms, even as they utilize various pre-existing curriculum resources at their disposal (Clandinin & Connelly, 1992). Teachers might choose to implement instructional activities from published curriculum materials more or less as-is, extensively modify those activities, or develop their own lessons when they find published materials wanting. Whatever the case may be, we view teachers as decision-making agents who are ultimately responsible for determining both the planned and enacted curriculum in their classrooms (Barab & Luehmann, 2003; Davis, 2006; Remillard, 2005). Of course, when teachers modify existing resources, the changes they make may or may not be consistent with the goals of the curriculum designers (Davis, 2006; Schneider & Krajcik, 2002). When the published materials are not of high quality, such changes are desirable and necessary; alternatively, teachers might eliminate valuable aspects of high-quality materials during their modifications. The ability to skillfully modify published curriculum resources is therefore an essential skill for teachers (Davis, 2006; Schwarz et al., 2008).

For the case of EDST at the elementary level, various curriculum resources exist, most notably Engineering is Elementary (Museum of Science Boston, 2007) but also a wealth of resources that can be searched on the web (e.g., from sites such as teachengineering.org). Since the introduction of the NGSS in the USA, many science curricula now also include engineering design activities intended to serve as EDST resources (e.g., Full Option Science System: Next Generation, Lawrence Hall of Science, 2015). Not all of those resources, however, have been recognized as necessarily being of high quality (Maeng et al., 2017; National Academy of Engineering, 2009), highlighting the important role of the teacher in skillfully adapting and modifying curriculum materials.

Study context and research questions

The present study examines how teachers planned EDST within the context of a PD project that was designed to introduce EDST to teachers. Given our perspective on teachers as curriculum makers, our intent within the PD was not to impose any specific curriculum or set of instructional activities in a top-down manner. Rather, we viewed the PD as an interactive process of supporting teachers in their efforts to adapt EDST to their own contexts (Craig, 2012). That process included presenting exemplars of EDST and making certain curriculum resources available, but also outlining goals that might be pursued by EDST and providing support for participants to pursue their learning objectives. For the purposes of this study, we focus on teachers’ planned rather than enacted curriculum. Although what is enacted does not necessarily flow unproblematically from what teachers plan (Remillard, 2005), teachers’ plans are nevertheless important and provide valuable information regarding how they think about instruction as well as what students ultimately experience. Our work was guided by the following research questions:

• 1. How did participants in the PD project incorporate engineering into their planned science instruction?

• 2. To what extent were the engineering lessons in those unit plans conceptually linked to the science content in the unit?

• 3. To what extent, if any, were the following factors related to the extent of the conceptual connections between the engineering lessons in the unit plans and the science content: science content area (physical science, earth/space science, life science), use of published engineering curricula, and repeat participation in the PD?

Methods

Participants and PD context

This work was part of a larger research project aimed at supporting elementary science and engineering instruction in grades 3–5. The project modified the student teaching experience of an undergraduate elementary education program at a university located in the Midwestern USA. In its typical form, student teaching occurs during the final semester of a pre-service teacher’s undergraduate degree program. The student teacher is placed with a mentoring classroom teacher (“cooperating teacher” hereafter) and gradually takes on all of the teaching responsibilities in the classroom. We modified that typical experience by teaming the student teacher/cooperating teacher pair with an engineering graduate student (“engineer” hereafter) to form a triad. The engineers in the study were completing a graduate degree across a range of engineering disciplines at the same university as the student teachers. The triads participated in PD experiences together and were jointly responsible for applying the ideas presented in the PD to their classroom contexts.

The engineers spent one full day per week in the classroom with their triads and also supported planning for science and engineering outside of that time. The intended role that we conveyed to the engineers was for them to help their teachers better understand the science and engineering concepts they were addressing with students as well as seek out engineering activities that would connect to the science standards being taught. We expected that the engineers would be well-suited to the task because of their extensive content preparation in both science and engineering; engineering graduate students have necessarily completed a large amount of coursework in both science and engineering. In an earlier study examining the interactions of the triads (Spinler, 2018), we found that the engineers perceived themselves primarily as resource providers and therefore brought a variety of activity ideas and resources to planning sessions.

All members of the triads experienced a 2-day PD workshop at the beginning of the semester, and the student teachers and cooperating teachers completed an additional full-day workshop midway through the semester. During the workshops, the facilitators modeled science lessons that emphasized how to help students make sense of hands-on experiences with phenomena in order to progress their conceptual understanding of key science ideas. The facilitators then modeled engineering instruction with an emphasis on engaging students in core engineering practices and advancing students’ learning of science concepts. During the engineering activities that were modeled, the workshop facilitators explicitly drew participants’ attention to instances where science concepts needed to be applied to the design tasks at hand. The workshops also addressed what engineering is, how engineering is related to yet distinct from science, different ways to structure engineering activities, how to work productively as a team, and overall expectations for the project.

The initial PD workshop was intended to introduce participants to pedagogical principles related to EDST. We then provided participants with sustained support for implementing the ideas presented in the workshop (Loucks-Horsley, Stiles, Mundry, Love, & Hewson, 2010). Most central to that effort were the engineers, who were intended to help their triads plan and implement EDST. The engineers attended a weekly, 1-h seminar that met on campus and provided them with fundamental principles of effective science teaching in general and EDST in particular, practical suggestions for how to contribute to classroom instruction, and assistance in navigating interpersonal conflicts that arose in their triads. Triads were also visited every other week by a project researcher who observed classroom instruction and provided feedback and guidance to the triads. During those interactions and across other aspects of the PD, a consistent message communicated to participants was the goal that engineering activities be used to support students’ science learning. We also consistently emphasized participants’ agency in planning their science and engineering instruction. When the researcher gave feedback or made suggestions, care was taken not to impose specific lessons or activities but rather to provide participants with resources to pursue their identified goals and objectives. The researcher helped triads procure curriculum resources when asked to do so, but acted primarily as a facilitator of the participants’ decision.

The research project took place over 8 semesters and worked with 10 triads per semester. 46 of the 80 total triads that participated in the project met the inclusion criteria for this study by submitting the plans for a science unit that included engineering. Several reasons exist for why only 46 triads were included in the study. First, the requirement to submit a unit plan was only placed on participants starting in the second semester of project implementation. Second, some of the triads chose to upload unit plans that incorporated engineering into a non-science content area such as math. Finally, some triads simply failed to submit a unit plan even though they taught a science unit that included engineering.

Participants were situated in 21 elementary schools within the same diverse urban school district located in the Midwestern USA. The schools in this district served students with a range of demographic characteristics, with socioeconomic status ranging from 24.6 to 97.9% of students qualifying for free/reduced lunch (mean = 67.7%) and student populations in the schools ranged from 4.3% White to 78.4% White (mean = 48.9%). Of the 46 participating triads, 19 were situated in 3rd grade classrooms, 15 in 4th grade, and 12 in 5th grade. Cooperating teachers in the study ranged in teaching experience from 4 to 28 years (mean = 16.6 years), and 61% of those teachers held Masters degrees. 14 of the cooperating teachers had participated in the project in a previous year of implementation, though no cooperating teachers were present in more than one of the triads included in the study. Each unit plan used in the study, therefore, represents the contribution of a unique cooperating teacher. Importantly, given the triad structure of the project, the unit plans should be considered joint products of the three triad members rather than the product of any single individual.

Teachers in the participating school district were provided with Full Option Science System (FOSS; Lawrence Hall of Science, 2015) curriculum materials and a district calendar that specified when each science unit was to be taught. The district purchased FOSS Next Generation units in the fifth semester of the project to replace the older units. Most participants’ science instruction drew heavily from FOSS, but some chose to create their own curriculum materials. Participants were also not required to use a specific engineering curriculum; the research project made materials from Engineering is Elementary (Museum of Science Boston, 2007) available as a resource, but many teachers chose to utilize materials they found elsewhere or developed their own.

Methodology

This study employs a convergent parallel mixed methods design, in which the quantitative and qualitative data were collected concurrently and analyzed separately before mixing results for the interpretation (Creswell & Plano Clark, 2011). As applied to our study, some of our research questions are more quantitatively oriented than others, and by using a convergent mixed methods design, we leveraged the insights gained by the qualitative analyses to interpret the quantitative ones. This type of convergent design provides a more complete understanding of the research topic as the qualitative and quantitative approaches provide different perspectives and data on the same topic. This can help to answer questions that cannot be answered adequately by one of the approaches alone (Creswell & Plano Clark, 2011). Additionally, the mixing of approaches also allows for a more complete picture and deeper explanation of a complex phenomenon, crucial for studies like the present one which attend to the complexities inherent in educational settings (Teddlie, & Tashakkori, A. (Eds.)., 2009).

Data collection

Qualitative data sources provided insight into how participants used engineering in their planned science instruction (Research Questions 1 + 2) and included (1) unit plans submitted by each of the participating triads for a unit in which they incorporated engineering into science instruction and (2) artifacts related to the submitted plans, such as published curriculum materials used by the triads, supplemental materials, and student sheets. Triads uploaded their unit plans and supporting materials, in whatever format was typical for their schools, to a project website and they were asked to clearly label which part(s) of the unit plan represented engineering. During the workshops, we presented participants with specific perspectives on the nature of engineering and what engineering lessons might look like, but we did not necessarily expect the engineering components of the unit plans to adhere to what was presented. Consistent with our view of teachers as curriculum makers, we aimed to characterize how our participants made sense of EDST, and therefore took participants’ labels of “engineering” at face value.

We raise two important points regarding our choice to use triads’ submitted unit plans to investigate how they implemented EDST. First, we do not claim that the unit plans necessarily represent participants’ typical EDST. From other sources of project data, we know that most triads engaged students in engineering activities multiple times over the semester and that the unit plans generally corresponded to instruction that occurred later in the semester. That, along with triads’ knowledge that the unit plans would be made available to other project participants on the project website, means that the unit plans likely represent what triads regarded as their best instruction rather than what was typical. Second, triads’ plans do not necessarily coincide with how those plans were implemented in the classroom. From classroom observations, we confirmed that the triads did teach the lessons they described in their unit plans, but the way the lessons were enacted inevitably differed in varying degrees from how they were planned. Although the unit plans have certain limitations as a data source, overall we have good reason to believe that they reflect triads’ thinking about EDST and that they represent lessons that were actually taught in the classroom.

To explore factors related to the conceptual connections between science and engineering in the unit plans (Research Question 3), we transformed the qualitative data (as described below) and used several additional sources of quantitative data. As described in the “Data analysis” section that follows, we wished to control for some of the variability that existed in our participants in terms of their content knowledge and confidence in teaching science and engineering. To do that, we administered a set of surveys to all participants. One was a measure of science content knowledge based on set of items from the MOSART instrument (Sadler, Coyle, Cook-Smith, & Miller, 2006). We used 24 items from the middle-school MOSART that aligned with the science topics taught in grades 3–5 in the partnering school district. We measured participants’ self-efficacy beliefs for teaching engineering via the TESS (Yoon, Evans, & Strobel, 2012) and for teaching science via the STEBI-B (Enochs & Riggs, 1990). All surveys were administered as both a pretest, prior to participating in the PD, and as a posttest.

Data analysis

In the following sections, we describe how the qualitative and quantitative data collected were analyzed separately before merging the results to provide an overall interpretation that could be used to address the research questions (Creswell & Plano Clark, 2011). Importantly, the purpose of the present study is not to evaluate the efficacy of project experiences on participants. Rather, the PD project formed the context in which to investigate our research questions.

RQ1: How did participants in the PD project incorporate engineering into their planned science instruction?

To describe the way that engineering was included in the submitted unit plans, we drew from prior studies of how teachers utilize EDST (e.g. Capobianco & Rupp, 2014; Crotty et al., 2017; Guzey et al., 2016; Maeng et al., 2017) to identify several salient dimensions: the science content addressed by the unit, whether published curriculum materials were being utilized, when in the unit that engineering was incorporated, and the type of engineering instruction (e.g. as a design challenge or something else). For each of those dimensions, we qualitatively analyzed the unit plans using an inductive approach to identify major patterns across the unit plans (Miles, Huberman, & Saldaña, 2014). The first two authors jointly developed codes for each of the dimensions and reached consensus on how to code each of the submitted plans. The third author independently reviewed and confirmed the assigned codes as well as the relevance and clarity of the coding frame, which is summarized in Table 1.

Table 1 Descriptive codes assigned to unit plans

RQ2: To what extent were the engineering lessons in those unit plans conceptually linked to the science content in the unit?

To investigate this research question, our objective was to develop an ordinal coding frame that could be used to assess the degree to which the engineering lessons within a unit plan were linked to the science concepts addressed in that unit. Our choice to focus only on the connections to the science content targeted by the unit was informed by our conceptualization of EDST. Like any pedagogical approach, EDST should be used within a science unit as part of coherent instruction around a set of targeted science concepts. When used within a unit of study, an engineering design activity serves little purpose if students develop or apply ideas that are unrelated to the rest of the unit. As an example, many engineering design activities involve students using different materials to design and build prototypes, which means that students must often know something about the properties of those materials as well as some basic ideas about structural stability. However, a scientific understanding of those ideas is rarely (though not never) the focus of the science units in which those activities are placed. Although students might learn something of material properties or stability, those activities would not be consistent with our conceptualization of EDST.

We began the process of creating coding categories by identifying conceptually derived endpoints. We defined, a priori, the lowest category as corresponding to a unit in which the engineering had no conceptual relationship to the science content, and the highest category corresponding to a unit in which the engineering had extensive connections to the science content in the unit. Beginning with descriptions of the highest and lowest categories, we developed intermediate categories using a data-driven, inductive approach (Miles et al., 2014). Using data from semesters 2–5 (26 unit plans), we identified clusters of units that were similar in terms of the extent to which the science concepts were connected to engineering. Based on the clusters that we identified in the unit plans, we constructed two intermediate categories: one to account for plans in which the science and engineering were related by topic only and another for plans in which only basic science knowledge was either used or developed during engineering. After identifying those categories and developing an initial coding frame, the three authors engaged in iterative cycles of independent coding of the full data set followed by discussion of disagreements and coding frame revision. We concluded the iterative process of coding frame revision when our intercoder agreement, as measured by Krippendorff’s α, was found to exceed a satisfactory value of .66 (Krippendorff, 2004, 2012). Using the final coding frame, given in Table 2, the three authors came to consensus on the ratings of science-engineering connection for all submitted unit plans.

Table 2 Coding frame for ratings of connections to science content

To illustrate the coding frame and how it was used within this work, coding examples from each of the four established categories are presented in Table 3. Each example is taken from a unit plan targeting the same science concepts of weathering, soil composition, erosion, and landforms. The difference between each example lies in the engineering activities in the unit and their connection to the targeted science idea of the unit.

Table 3 Coding examples for ratings of connections between engineering and science

RQ3: Factors associated with ratings of science-engineering connection

To explore possible factors related to the levels of conceptual connections between science and engineering in the unit plans, we identified three variables of interest: the science content domain, the use of published curriculum materials, and whether the cooperating teacher was repeating the PD program. Table 4 provides the variable levels for each variable.

Table 4 Summary of variables used for research question 3

We used multiple ordinal logistic regression (Cohen, Cohen, West, & Aiken, 2003) to test which of the variables in Table 4 were significant predictors of ratings of science-engineering connection. We constructed a model using each of the variables in Table 4 as dummy-coded categorical predictor variables. A limitation of our use of regression modeling is that our sample size is relatively small, resulting in limited statistical power (Cohen et al., 2003). Given the exploratory nature of the study, we deemed this an acceptable limitation, but we note that limited power means that certain predictive variables might not have been detectable using our methods.

Because participants in the project, especially the cooperating teachers, differed in terms of their preparation for and experience teaching science and engineering, we sought to account for those differences in the statistical model by including one or more relevant covariates. As described above, we collected pre/post survey data regarding participants’ science content knowledge and self-efficacy beliefs for teaching science and engineering, as well as data on cooperating teachers’ years of experience and highest degree earned. Those data sources produced a large number of variables that could give insight into participants’ preparation for teaching science and engineering, but which were likely to be collinear with one another. We chose to cull that set of variables using a stepwise regression procedure to identify a small number of variables that we could use as reasonable proxies for participants’ preparation. We acknowledge that stepwise regression has multiple well-documented shortcomings (Harrell, 2015), which is why we emphasize that this procedure was not used in order to make any specific claims about the variables it was used to identify. Rather, we used it only to winnow the set of variables so that we could use a relatively small number as covariates in our statistical model. The stepwise regression began with the full set of possible covariates¹ as predictors of science-engineering connection and sequentially removed those predictors in order to minimize the Akaike’s Information Criterion (AIC) value of the model (Long & Freese, 2006). This procedure yielded four variables: cooperating teachers’ years of experience, student teachers’ posttest TESS scores, cooperating teachers’ STEBI-B scores, and engineers’ STEBI-B scores. Those four variables were added to the ordinal logistic regression model alongside the three main variables of interest (see Table 4).

Results

RQ1: How did participants in the PD project incorporate engineering into their planned science instruction?

Table 5 summarizes the descriptive codes assigned to the unit plans. The unit plans represented the three science content domains (life science, physical science, and earth/space science) with relatively equal frequency. Most of the time (71%), engineering instruction was placed at the end of the science unit and was typically intended to serve as an application of science ideas. Even when engineering was placed in the middle of the unit, it was also intended to serve as an application of the science ideas that preceded the engineering. Nearly all triads incorporated engineering into their units as engineering design activities (95%); very few alternative approaches to engineering instruction were evident in the plans. This is not surprising because the PD workshops modeled engineering design challenges as examples of engineering instruction.

Table 5 Descriptive codes assigned to unit plans

Slightly less than half (43%) of the unit plans utilized published engineering curriculum, most typically Engineering is Elementary, but also engineering lessons included in FOSS Next Generation and activities published on the web. In the remaining units, the triads developed their own engineering lessons from scratch. We emphasize that even when participants utilized published curricula, they still acted as curriculum makers in the sense that adapted those materials to their classroom contexts. The degree to which curriculum materials were adapted by participants naturally varied, which is illustrated by the following two examples. Triad 28 used the design challenge from the Engineering is Elementary: Thinking Inside the Box unit primarily as inspiration for their design activity. In the published design activity, students design a package for a plant that will be sold at a marketplace. In the triad’s version, students instead designed a package to ship live crayfish that they had been studying in the unit. The triad’s unit plan made use of some of the original handouts from the published curriculum, but modified them for the new context. Triad 42 used the design challenge from Engineering is Elementary: The Attraction is Obvious in which students design a maglev train system. In contrast to Triad 28, however, Triad 42 made far fewer modifications to the activity; they used the context, handouts, and design parameters specified in the curriculum materials. They did change the pacing of the activity in that they chose to omit many of the introductory lessons, and they also modified the testing procedures of the activity.

We were wary that project participants might simply replicate the engineering activities modeled during the workshops regardless of how well they aligned with their grade-level science content. However, we did not find this to be the case. Only eight (17%) of the unit plans included an engineering activity that was similar to one modeled during the PD workshops, and in all of those cases the teachers made substantial adaptations to the modeled activity (e.g., changed the context, provided different design parameters, and used different materials).

RQ2: To what extent were the engineering lessons in those unit plans conceptually linked to the science content in the unit?

Table 6 presents a summary of the connections between engineering and science ratings that were assigned to each of the unit plans. Of the 46 unit plans we analyzed, only 2 used engineering as stand-alone activity that was not connected to the science content in their unit. In one of those units, after a series of lessons on health and nutrition, the students were engaged in the Engineering is Elementary Water, Water Everywhere: Designing Water Filters activity where they designed a water filter to clean water for a pet turtle. The other triad submitted a unit on force and motion combined with an engineering challenge in which students designed a package for a plant (an activity modeled during a PD workshop).

Table 6 Ratings of science-engineering connection assigned to unit plans

Half of the lessons connected the engineering and science through a topical connection where the link between the science learned in the unit and the engineering design challenge was related only to the same overall topic. For example, Triad 16 completed a food and nutrition unit where they learned what food is made of and how the different nutrient groups contribute to healthy nutrition. At the end of the unit, students created a package for food that could be used by a nonprofit to deliver meals to hungry people. However, within the design activity, there were no criteria-related types of food, food storage, food groups, or balanced nutrition. The activity was only connected to the science unit via the overall topic of “food.” Similarly, Triad 66 developed an engineering activity in which students designed pencil holders for the classroom out of recycled materials. This activity was placed within a unit addressing environmental science content, which included attention to pollution and recycling. While the topic of recycling linked the engineering activity and the science in the unit, no specific science ideas were relevant for the design activity. In these cases, the participants clearly attempted to develop engineering activities that related to the science unit. However, the lack of connections between the engineering activities and the science content under study caused those units to fall short of the science learning goals of EDST.

The next most common type of connection seen in the submitted units was a shallow conceptual connection where unit science concepts were relevant to the engineering design challenge, but not necessary. For instance, Triads 22 and 76 each planned a science unit on plant structure and plants’ needs before asking students to create a package for the plant that protects the plant and keeps the plant alive. This engineering design challenge is related to science concepts around plants’ external structures and needs that were covered in the unit. Students could potentially apply those concepts in the design challenge, but doing so would not necessarily lead to more successful designs. Another instance of shallow connections is illustrated by the unit plan developed by Triad 53, in which students were learning science concepts related to climates around the world. The plan included an engineering design activity in which students had to design houses that would be appropriate for a given climate region. Students needed to know whether a particular reason would likely be hot or cold, or whether snow would be likely. Beyond those basic facts, however, the design activity focused on creating a structure that was sturdy and stable—ideas that were unrelated to the overarching unit. Thus, while units at this level of connection did include some connections between the science concepts in the unit and the engineering lessons, those connections existed at the periphery of the design activities.

Five unit plans reached a deeper level of conceptual connection between the engineering and science and included an application of the science concepts that was integral to the task they were solving in the engineering design challenge. For example, the plan developed by Triad 69 addressed electricity, insulators, conductors, and types of circuits before introducing an engineering design challenge requiring students to create an alarm for their classroom pet rabbit’s cage that would trigger a light and buzzer when the rabbit ran out of food. Another Triad, 35, had students design a prosthetic hand after learning about the structure and function of bones, joints, muscles, and ligaments in a human body unit. In both cases, students could not have successfully completed the design challenge without utilizing the concepts they learned during science instruction. As such, units at this level of conceptual connection were the only ones that exemplified the goals of EDST.

RQ3: Factors associated with ratings of science-engineering connection

As described in the methods section, we constructed an ordinal logistic regression model to test which of the possible factors listed in Table 4 were associated with unit plans’ ratings of science-engineering connection. Four additional variables were included in the model to account for the variability of participants across triads. Because only two unit plans received the lowest rating of science-engineering connection, that lowest rating level was excluded from the analysis in order to meet the assumptions of the model (Cohen et al., 2003).

The results of the ordinal logistic regression model are summarized in Table 7. The pseudo-R² value of the model, using McFadden’s R² (Long & Freese, 2006), is .282. Note that because each of the main variables of interest (content domain, engineering curriculum, and repeat cooperating teacher) were categorical, each was dummy-coded. The comparison group for content domain was earth/space science, for engineering curriculum was made from scratch, and for repeat cooperating teacher was non-repeat participant. The results indicate that only one variable of interest was statistically significant in its association with the dependent variable: whether the unit plan utilized published engineering curriculum materials. Table 7 also includes significance tests for the covariates, but we emphasize that those variables were not the focus of the research and so served a narrow role. We include significance tests not to advance any claims about those variables but to simply indicate their relevance to the overall model.

Table 7 Results from ordinal logistic regression model

For triads who used published engineering curriculum materials, the odds of submitting a unit plan with a higher rating of science-engineering connection were 6.62 times higher than for triads who did not, holding all other variables constant. As noted previously, triads who used published curricula in their unit plans modified and adapted those materials to various degrees. Their unit plans therefore do not represent exact copies of published curricula, but rather reflect participants’ approaches to EDST when utilizing published materials as planning resources. Of course, using published curricula does not always lead to success. Triad 62, for instance, adapted the Water, Water Everywhere materials from Engineering is Elementary to fit into a unit on human health. The engineering design challenge in that unit involved developing water filters, and the triad modified the context to connect to ideas of human health—namely, the importance of clean water. However, beyond that topical connection, the design challenge had no relationship to any of the science concepts related to human health addressed in the unit. In this case, the curriculum resource was not well matched to the science unit at hand.

The non-significance of the content domain variable is interesting given prior research indicating that the physical sciences may be more conducive to EDST (Guzey et al., 2014; Maeng et al., 2017; Roehrig et al., 2012). Our findings indicate that what might matter more than the content domain are the specific science concepts being addressed within that domain. Within physical science, for instance, five unit plans targeted electricity concepts, three of which scored a “4” on science-engineering connection and two of which scored a “2.” Four units addressed physical science concepts related to forces and motion, two of which scored a “3” on science-engineering connection with the remaining two scoring a “2.” Although we lack a large enough sample to advance any statistical claims about those distributions, our interactions with triads who were planning units around force and motion often revealed how difficult it was to develop engineering activities that would relate to those concepts. Because force and motion are addressed only qualitatively at the elementary level, students may not yet have a detailed enough understanding of those ideas to apply them in a practical setting.

Discussion

Calls to include engineering in pre-college science education are often rooted in the argument that engineering can be used to enhance students’ learning of science concepts (e.g., Apedoe et al., 2008; Capobianco et al., 2018; Schnittka & Bell, 2011). Engineering design tasks are often viewed as valuable contexts for students to develop and deploy scientific ideas (Atman et al., 2007; Johri & Olds, 2011; Puntambekar & Kolodner, 2005; Roth, 2001). However, many studies have documented challenges that teachers encounter when trying to implement EDST in their classrooms (Berland et al., 2014; Capobianco & Rupp, 2014; Crotty et al., 2017; Silk et al., 2009; Walkington et al., 2014). In the present study, we explored the specific challenge of planning engineering lessons that allow students to apply and develop scientific concepts.

Nearly all of the participants in our study incorporated engineering into their science units via design challenges placed after the introduction of science concepts. The extent to which those design challenges were related to the science concepts varied considerably. More than half of the unit plans submitted by participants scored a “1” or “2” in terms of science-engineering connection, scores that represent no discernable conceptual connection to the science. Of the remaining unit plans, the majority showed only basic conceptual connections. Overall, very few unit plans included the kinds of deep conceptual connections that are necessary to advance students’ science learning within EDST. Given that the participants were in a supportive PD context, the rarity of deep conceptual connections in the unit plans indicates just how difficult it is to plan for EDST.

Conceptual connections between engineering lessons and science ideas are necessary in order for the engineering to support students’ learning of science, but those connections are not sufficient to ensure students’ science learning. Studies have found that even when science ideas are highly relevant to an engineering situation, students often fail to recognize or utilize them in the context of the engineering lesson (Bamberger et al. 2010; Chao et al., 2017; Crismond, 2001; Levy, 2013; Puntambekar & Kolodner, 2005; Roth, 2001; Silk et al., 2009). Therefore, even when participants submitted unit plans with deep conceptual connections, the extent to which the engineering in those plans actually progressed students’ science depended on how the lessons were enacted. A limitation of our study is that we were unable to systematically observe participants’ enactments of their planned instruction, so we cannot know whether unit plans with deep conceptual connections were successful in advancing students’ learning of science. On the other hand, unit plans that received low scores for conceptual connection were unlikely to deepen students’ understanding of science ideas, regardless of how they were taught.

A crucial question is how to help teachers plan (and ultimately implement) engineering lessons that are conceptually tied to science ideas. The factors that we explored in this study provide some insight into that question. We found no statistically significant differences between unit plans across the three content area domains of earth and space science, life science, or physical science. Unlike prior research indicating greater potential for physical science content (Guzey et al., 2014; Maeng et al., 2017; Roehrig et al., 2021; Roehrig et al., 2012), our findings indicate that EDST is equally possible (or, alternatively, challenging) across the different content domains. Although we are unable to provide a definitive answer based on our limited data, we suspect that substantial variability exists within each domain in terms of how conducive certain concepts are to EDST. Within the physical sciences for instance, certain ideas such as electricity appear to be more readily applied within engineering design contexts than ideas such as forces and motion. This is an area that is worthy of further study.

We also found no statistically significant differences between the lesson plans created by triads that included a repeat-participant cooperating teacher and by triads that did not. The fact that participating in the project more than once did not necessarily result in units that had more substantial conceptual connections may indicate the nature of the challenges inherent to the task, given that additional time and support was insufficient to overcome them. The non-significant findings reported here should not be taken as definitive answers to the questions, because as noted previously, the statistical power of this study might have been insufficient to detect differences that did exist for content area domain or repeat participants. An additional complication is that the unit plans were the joint products of multiple triad members, diminishing the influence of a cooperating-teacher-specific variable. More longitudinal work in this area would be valuable for the field.

We did find that participants who used and adapted published engineering curricula in their unit plans were more likely to develop plans with greater conceptual connections between science and engineering. Using published curricula certainly did not guarantee a high level of conceptual connection; 35% of units utilizing them scored only a “1” or “2” for science-engineering connections, and only 20% earned the highest rating. Nevertheless, our findings indicate that published curricula are important resources for EDST, even if they are far from perfect. Generating engineering design activities from scratch that have strong connections to a targeted set of science concepts is a difficult task (Kolodner et al., 2003), especially for time-constrained teachers. Among triads who created their own engineering lessons from scratch rather than use or modify existing materials, many did so because they were unable to locate any materials that adequately related to the science content at hand. That indicates both the value of and need for high-quality engineering curriculum materials that are accessible to teachers and connected to a wide range of science content. Those that currently exist often do not always connect to science ideas as well as intended, and teachers might also find themselves teaching science content for which no relevant engineering curricula have yet been developed.

The ways in which teachers utilize curriculum materials in the context of EDST is a phenomenon worthy of further study. In the present study, we differentiated participants who created engineering lessons from scratch versus those who used and adapted published ones. A limitation of the present work, however, is that we did not assess the degree to which participants modified published materials. A larger study would likely be needed to investigate how the extent of adaptation influences the quality of the resulting instruction. Future work might also investigate how using and adapting different curriculum resources might affect the relative emphasis within a unit on science versus engineering. Examining these issues would add to our general understanding of teachers as curriculum makers (e.g.,Barab & Luehmann, 2003 ; Davis, 2006) in the important context of EDST.

Conclusions

Engineering has the potential to enhance science instruction, but preparing teachers to effectively utilize EDST poses many difficulties, especially given elementary teachers’ limited background in science and engineering (Banilower et al., 2018). Providing teachers with curriculum materials is a reasonable way to support teachers (e.g., Chao et al., 2017), but EDST resources do not yet exist for a wide variety of science concepts, nor are extant engineering resources of sufficiently high quality (Maeng et al., 2017; National Academy of Engineering, 2009). Furthermore, as curriculum makers, teachers assemble their science instruction from a variety of sources (Banilower et al., 2018), modifying and adapting those resources to their classroom contexts (Barab & Luehmann, 2003; Davis, 2006). As resources for EDST proliferate, a critical skill for teachers to develop is the capacity to determine which resources are high quality and which are not. An additionally important skill is the ability to effectively adapt high-quality resources to suit specific classroom needs while preserving effective elements of the resources (Schneider & Krajcik, 2002; Schwarz et al., 2008). In the present study, we observed how the same curriculum resources were utilized to different degrees of effectiveness by different triads. If we had given those participants more preparation in critiquing and skillfully adapting curricula (cf. Davis, 2006), they might have had more success; this is a crucial consideration for future PD efforts.

Although we found that the use of published engineering curricula was beneficial for participants, we emphasize that our participants were part of a PD project that specifically prepared them for EDST. Unfortunately, most elementary teachers are likely to receive little PD in science or engineering. Given the varied quality of the curriculum materials that exist, we are left with many questions about the state of elementary science instruction and how to support teachers as more schools begin to include engineering in science. We are worried that most elementary teachers will not receive sufficient support to achieve the goals associated with the inclusion of engineering in science. If teachers are not sufficiently prepared and provided appropriate resources, including high-quality curriculum materials and sufficient instructional time, incorporating engineering into science might diminish rather than enhance student science learning. In order for this current reform effort to be successful, substantial resources need to be devoted to in-service and pre-service teacher education and curriculum development at a level that far exceeds what is currently being allocated (Banilower, 2019). Our work indicates several important aspects of teacher knowledge and practice that could be enhanced by additional teacher education. What remains to be seen is how best to marshal the resources at scale that are needed to do this important work.