Designing, facilitating, and scaling-up video-based professional development: supporting complex forms of teaching in science and mathematics

Designing for teacher learning

The projects described herein encompass innovative approaches to PD, approaches that built on contemporary ideas about the nature of cognition, learning, and teaching (Greeno et al., 1996). They also represent a shift from using workshops to teach techniques to using different PD strategies to build teachers’ understanding of subject matter, pedagogy, and student thinking (Stein et al., 1999). Following situative learning theory (Greeno, 2006), the designers have located teachers’ learning opportunities in their classroom activities (Ball & Cohen, 1999; Borko & Koellner, 2008; Putnam & Borko, 2000). As such, their work builds on a growing consensus that teachers’ learning should be situated within the artifacts of instructional practice (Ball & Cohen, 1999; Borko et al., 2008). For example, Reiser (2013) suggested for the PD to support NGSS vision to include rich images of classroom enactment to facilitate teachers’ sense making.

Prior research has identified features associated with PD that can support teachers as they transition to adopting ambitious instructional practices (Garet et al., 2001; Hawley and Valli, 1999; Loucks-Horsley et al., 2010; Penuel et al., 2007). Many features identified as successful are “practice-based” meaning that they are close to teachers’ day-to-day practice and involve the use of classroom artifacts and/or different representations of instruction (Ball & Cohen, 1999; Putnam & Borko, 2000). One feature that shows exceptional promise is the use of videos of teachers’ own and others’ instructional practice (Borko et al., 2008; Brophy, 2004; Sherin 2004).

As a particular manifestation of practice-based professional development, videos focus participants’ thinking and talk on artifacts from real classrooms and capture teaching in all of its complexity while, at the same time, affording space and time for guided reflection (Miller & Zhou, 2007; Sherin, 2004). Based on Gaudin and Chaliès’ (2015) recent review of international literature about the use of video in teacher education and PD, video viewing has become a prominent part of PD of teachers in all subject areas, at all grade levels, and on nearly every continent. There has been growing evidence about the positive effects of video-based PD programs on teachers’ learning and instructional practices e.g., (Tekkumru-Kisa & Stein, 2015; Borko et al., 2011; Koellner & Jacobs, 2015; Roth et al., 2011; Sherin & Han, 2004; van Es & Sherin, 2010) leading to a growing number of high-quality video-based PD programs for science and math teachers.

Although successful video-based PD programs have been developed, the next—even larger—challenge is to devise strategies for taking effective video-based PD programs beyond pockets of excellence to a larger scale. Developing sustainable and scalable PD programs is important to meet an increasing demand for teacher learning opportunities (Borko et al., 2014; Marrongelle et al., 2013; Wilson, 2013). This special issue brings together four effective video-based PD programs that use classroom videos as the centerpiece of their efforts and have “scaling up” as their goal. In each paper, the authors surface their design considerations for creating scalable and sustainable video-based PD interventions and the challenges and successes they experienced as they tried to scale-up their PD interventions.

A missing piece of the puzzle: PD facilitators

A deep understanding of what PD should look like is only part of the equation (Borko et al., 2014). It is now widely recognized that the well-prepared facilitator is an essential component of the equation for ensuring PD’s effectiveness (Borko et al., 2014; Elliott et al., 2009; Koellner et al., 2011). Discussing high-quality PD without focusing on the facilitators and their role is like discussing high-quality instruction without mentioning teachers and their role. The evaluations of National Science Foundation’s Local Systemic Change Initiative Projects found that a high-quality leader makes a difference in the effectiveness of PD programs to support teachers’ learning (Banilower et al., 2006). Similarly, Clark et al. (2008) attributed the emergence of mathematically rich discourse within a teacher professional learning community in the PD project they examined to the active role taken by the designated facilitator in modeling productive discourse among the teachers. They went on to suggest that highly skilled facilitation requires specific training and coaching of the facilitator. In fact, some researchers (e.g., Borko et al., 2011; Borko et al., 2014; Coles, 2013; Elliott et al., 2009) have begun to focus on the development of PD facilitators in order to move their programs to different contexts and ensure the sustainability of these programs (Gaudin and Chaliès, 2015).

A closer look into the papers: each representing a particular phase of PD research

Borko (2004) organized programs of research on PD into three phases, each building on the previous one: Phase 1 research activities focus on an individual PD program at a single site; researchers provide initial evidence that the program can have a positive impact on teacher learning. Phase 2 research activities focus on a single PD program enacted by more than one facilitator at more than one site; in this phase, researchers are interested in the fidelity and thus examine whether the program can be enacted with integrity in different contexts and by different facilitators. Phase 3 research activities compare multiple PD programs, each enacted at multiple sites, and investigate their impact on teacher and student learning.

At the time of Borko’s publication, the vast majority of the PD research could be categorized into phase 1; research on phase 3 was scarce. Since that time, several phase 2 (e.g., Bell et al., 2010) and phase 3 (e.g., Heller et al., 2012; Penuel et al., 2011) types of research studies have been conducted (Borko et al., 2014).

For this special issue, we identified four PD projects, each of which represents a particular phase of PD research as described by Borko (2004). Tekkumru-Kisa and Stein focus on an individual video-based PD program at a single site (phase 1); Borko and colleagues as well as Jacobs, Seago, and Koellner focus on a single video-based PD program enacted by different facilitators in different sites (phase 2). Finally, Roth and colleagues focus on a video-based PD program enacted at multiple sites (phase 3). We purposefully sampled in this way in order to surface the growing demands placed on PD designers as they progress through these phases. For example, unlike phase 1 designers, phase 2 designers must take on the preparation and use of knowledgeable facilitators. They also have to design for issues of fidelity and adaptation that will inevitably arise in new sites. How much flexibility should be designed into the program? What parts are adaptable; what parts are not? Who should make adaptations? When?

Phase 3 designers need to deal with all of the above issues plus do so at multiple sites and assemble a research infrastructure that is up to the task of comparing their program with other PD programs. By surveying PD programs at each of these phases, we are positioned to learn how designers’ foci, tools, and resources evolve as they transition from a single program to one that is scalable beyond the initial developers’ vision. By limiting our sample to video-based PD projects, we are positioned to learn about the particular challenges and affordances of video at each of these phases.

Phase 2 research studies aim to determine whether the PD program can be enacted with fidelity in different settings and by different PD facilitators (Borko, 2004). Jacobs and colleagues focused on the fidelity of implementation of the LTG-PD by analyzing the preparation process of a PD facilitator, her use of PD materials and her rationale for the adaptations she made. Borko and colleagues report on the implementation of PSC model of PD and Mathematics Leadership Preparation model of PD leader preparation (Borko et al., 2015) within an urban school district through a Design-Based Implementation Research project. Even though context remains unstudied in phase 2 research, given the collaborative design effort with the district leaders within a DBIR research project, context of the school district becomes an important factor in their paper. The authors emphasize for the PD researchers the importance of being responsive to the context of the school partners if they expect their work to be meaningful.

Finally, phase 3 research studies are about comparing multiple PD programs, each enacted at multiple sites. In phase 3 research, the relationship among the facilitator, PD program, teachers, and context is studied (Borko, 2004). Roth and colleagues focus on the STeLLA video-based PD program. As such, they do not technically compare multiple PD programs. They are, however, one of a very few PD programs in science that has scaled to multiple contexts and has been implemented by different facilitators. Additionally, they have tested variations of the STeLLA PD program including one that was of equal duration but focused on content deepening and did not include any video analysis work.

Opening the black box: how were the PD programs designed and facilitated

In an NCTM-commissioned study of the field’s readiness for the CCSS, Marrongelle et al. (2013) identified the need for more studies that “open the black box of professional development” (p. 209). Given the increasing interest in and demand for development of video-based PD programs, we believe that clearly articulating the design features of effective PD programs and revealing the intentions and rationales of PD designers through a rich description of their programs will help the field to think more carefully about the contents of the black box. Importantly, the need for carefully articulated design principles does not end with specification of the program itself but extends to principles that guide how to set up and orchestrate the interactive discussions that occur as the program is enacted. Borko et al. (2014) argued for articulating the practice of leading high-quality PD to be able to define how these programs contribute to changes in teachers’ learning, instructional practices, and students’ learning.

Across these four projects, we expose the thinking and decision-making that occurred behind the scenes in facilitators’/designers’ minds as they planned and led video-based PD. In the course of describing their most recent research and design efforts, the paper authors (who are also the designers of the PD) make transparent the design decisions involved in setting up and orchestrating the programs, including the role of different frameworks and tools in their behind-the-scenes work.

For example, the essential design consideration underlying Jacobs and colleagues’ LTG-PD program was its high specification and expectation of fidelity and/or reasonable adaptations defined in terms of when and how to adapt. A second design consideration was the concept of “video in the middle,” an underlying structure of pre- and post-activities wrapped around a video as a way to get the most out of each video viewing event. In contrast to Jacobs and colleagues, Borko et al. characterized PSC-PD as a highly adaptable model, which—from the beginning—was designed to allow for tailoring it to the goals, interests, and needs of participating districts. Each of these decisions will have far-reaching impact on the shape and form of future iterations of these two PD programs; exposing the decisions around fidelity/adaptation marks the decision point for novice designers who otherwise might be surprised with “lethal mutations” (Brown, 1992) as they try to expand their programs to new sites.

These and other design decisions are commonly hidden from view: They are shared assumptions among the “founding” designers but left unarticulated to others. These assumptions undergird a wide range of decision-making-in-the-moment for this initial group. Without exposing these tacit beliefs/assumptions, however, others new to PD facilitation/design will flounder. One of the critical design considerations in the Tekkumru-Kisa and Stein program was “an approach to learning that first set up learner-led exploratory experiences, followed by experiences that channeled that exploration a bit, to a culminating experience in which the learning is directly addressed by the teacher/facilitator and connected to learners’ initial less-structured experiences” (p. 38). Knowing and understanding this structure could be immensely helpful to a novice facilitator. In the Roth et al. paper, the most mature of the PD efforts examined, the majority of the material focuses on elucidating the design principles associated with STeLLA, its implementation and its scaling up. The authors welcomed the opportunity to write about these undergirding—but mostly unspoken—aspects of their work. The STeLLA design principles are made even more vivid by their contrast with traditional research findings on what constitutes effective PD.

Exposing design decisions goes beyond decisions about the PD and teachers. This is especially evident in the case of video-based PD. In several of the studies, video was used not only for supporting teachers’ learning but also for supporting facilitators’/teacher leaders’ learning to orchestrate PD discussions. Therefore, while the papers discuss how they capitalized on the power of video in their projects, several of them also elaborated on how they supported facilitators’ learning to use video to effectively support teachers’ learning through videos of others leading similar PD discussions.

Preparing PD leaders for scalable and sustainable PD programs

As noted earlier, there is a growing demand for widespread and high-quality professional development across the nation. However, there is a lack of systematic research on how best to train facilitators to both deliver well-designed PD with fidelity when appropriate and to learn when and how to best adapt the program (Borko et al., 2011, 2014; Elliott et al., 2009; Lesseig et al. 2016). Even’s international review of the literature on the facilitation of PD programs focused on this missing piece in the literature (Even, 2008). She points out the ill-defined nature of the field and lack of systematic investigations of PD facilitators, which may present challenges for designers of tools and programs to support PD facilitators.

Over the past decade, however, some research has begun to emerge that focuses on developing PD leaders’ capacity to design and facilitate high-quality PD to support high-quality instructional practices and improve student gains (e.g., Borko et al. 2014; Elliott et al., 2009; Jackson et al., 2015). Borko et al. (2014) stated “A central component of a sustainable, scalable PD model is the ability to prepare PD leaders who can adapt the model to a variety of local contexts and advocate for school and district support of the PD while maintaining integrity to its goals and design” (p. 151). Koellner et al. (2011), for instance, worked with novice teacher leaders to help them incorporate important features of PD programs including (1) fostering a professional learning community, (2) developing teachers’ mathematical knowledge for teaching, and (3) adapting PD to support local needs and interests, as they learn to facilitate Problem-Solving Cycle in their own schools. In a related study, Lesseig et al. (2016) studied how their use of video cases supported leaders’ noticing of the work required to facilitate a mathematics PD.

Some recent studies have also begun to articulate specific knowledge and skills leaders need to facilitate teachers’ learning including deep subject matter knowledge, familiarity with the special affordances and challenges of video, and appreciation of the beliefs and understandings of session participants (Lesseig et al., 2016; Coles, 2013; Schifter & Lester, 2005). Considering all these potential challenges and the needs of PD facilitators, the question turns to how novice facilitators’ skills and knowledge can be developed and by whom.

The PD programs represented in this issue build on this small but growing literature to take on this question. All of the papers address how to support the work of PD leaders and emphasize the role they play for the scalability and sustainability of the programs. In the paper by Jacobs and colleagues, we learn about the kinds of support provided to their facilitator-in-training (Hannah) during conference calls scheduled before and immediately after PD sessions as well as providing the opportunity for Hannah to rehearse some of the sessions. The rehearsal, in particular, is important for marking the idea that the PD design cannot be captured solely in materials but that it also consists of interactions between participants and the leader that support a particular kind of learning environment. An important way in which Roth et al. support facilitators-in-training is through detailed PD leader guides that include specific teacher learning goals for each PD session (building off of their knowledge that teachers implement more coherent lessons when they are guided by explicit student learning goals (see more below for building off of classroom research)).

In the paper by Borko and colleagues, authors provide insights into the adaptations they needed to make in their PD program as they worked with an urban district to develop their capacity to support mathematics reform, the most notable of which pertained to teacher leaders’ support. Because the novice leaders were less steeped in the curriculum than expected, the designers strengthened their modeling and debriefing activities with them as well as provided extended support to help them learn to select video clips that would be discussed in the PD sessions. Finally, Tekkumru-Kisa and Stein (in anticipation of training future facilitators in their PD program) propose a framework that can be used as a tool to support future facilitators’ preparation and facilitation of video-based programs.

Drawing on classroom-based research and teacher learning research

As a field, we have experience in designing and studying case studies of PD aimed at teacher learning and improved student achievement in one particular setting. We have less experience designing and studying PD that “goes to scale.” Figure 1 depicts the layers of research and development on professional development that comprise efforts aimed at instructional improvement at scale.

The individuals on the right side of the ovals (regardless of which layer they are in) can be seen as learners; the individuals on the left side of the ovals can be seen as the teachers.¹ In each layer, the “substance” of the teaching and learning activities is represented by the downward pointing vertex of the triangle. The innermost oval represents the classroom in which teachers work with students to learn the science or mathematics content (some refer to this as the instructional triangle; Cohen & Ball, 1999; 2000). In the middle layer, a PD facilitator helps a teacher to learn ways of helping students learn the content in the classroom. In the third layer, a researcher/program designer works with the PD facilitators to help them to learn how to help teachers to help students to learn. Because there is little systematic research focusing on the teaching and learning of PD facilitators, researchers and program designers often reach into “lower” levels of the diagram to appropriate ideas, tools, and resources to guide their work.

All of the studies in this issue draw on classroom-based research (inner layer) as well as teacher learning research (middle layer) to support their work in training facilitators. They draw on classroom-based research for two reasons. First, the target of PD is improved instructional practice and student learning. Consequently, designers and facilitators need to know and draw upon research on effective instructional practices in mathematics and science as they design their programs. Second, research conducted in classrooms that reveals key practices that support student learning can be useful when designing PD that will support teacher learning. Because research on teaching students is abundant, all four of the papers—at some point and to some degree of specification—reach out to frameworks, findings, and tools from classroom research. They also draw on teacher learning research to support the design of learning environments for novice facilitators who are learning how to support teacher development in PD.

Jacobs and colleagues reached into research on teaching to justify the assumptions that undergird their design principles and to build tools for supporting the PD facilitator. As they introduce their phase 2 work, they justify their focus on fidelity by reminding the reader of decades of curriculum research which established that teachers’ enactment of curriculum rarely completely matches the curriculum itself (with some changes being labeled as “lethal mutations”). Just like curriculum materials, PD programs, they argue, are susceptible to a wide range of adaptations. They also “borrowed” the tool of rehearsal from research on teacher preparation when designing tools for training of facilitators.

One might say that the entire edifice of STeLLA (Roth et al.) is built on careful attention to student thinking and teacher actions in the science classroom. A specific example of their building on classroom knowledge was their recognition of the need to incorporate teacher learning goals into their PD session agendas. Without a clear goal of what teachers were to “get out of” the session, discussions, they found, could drift in unproductive directions, similar to what they had seen when studying science lessons not guided by student learning goals.

The remaining papers also borrow from classroom research by focusing on the importance of supporting teachers’ learning toward carefully defined goals. For example, Tekkumru-Kisa and Stein stated, “when used by skilled facilitators, they [goals] can become the ‘north star’ by which to steer participants’ contributions toward a deeper understanding” (p. 8). Similarly, Borko and colleagues emphasize the role of goals in helping teacher leaders to recognize how much explicit attention they needed to devote to the planning of video-based discussions. The authors also emphasized that videos should be selected with a clear goal in mind. In addition, this group drew on Grossman and colleagues’ (2009) framework for the teaching of complex professional practice to pre-service teachers to guide their thinking about the design and facilitation of experiences for the professional development facilitators. Finally, both Tekkumru-Kisa and Stein, and Borko and colleagues adapted a tool based on classroom practice (Five Practices for Orchestrating Productive Discussions— Smith & Stein, 2011) for their professional development work with teachers.

Figure 1 also makes transparent the complexity of taking effective PD programs to scale. Often discussions about “going to scale” are framed by numbers, as in the number of teachers that will be “touched” and the number of personnel that are needed to be trained so that they, in turn, can train others (e.g., the train-the-trainers approach). Such discussions are often silent with respect to the substance of what must be learned at different layers of the system. Cast within a teaching and learning approach, Fig. 1 shows the complexity of the content to be learned, the amount of which rapidly increases as you move up the layers. For example, in the top layer, researchers/designers support facilitators to learn the substance in layer 1 (what effective math and science teaching and learning looks like in the classroom) and in layer 2 (how to work with teachers-as-learners to improve what they do in the classroom). This represents a steep learning curve for most researchers/designers as well as the facilitators. All of this suggests the importance of keeping a learning perspective on issues related to scale because it requires us to think carefully and conceptually about the requirements—many of which may be hidden from view—of rapidly expanding an effective PD program.