Design principles for effective video-based professional development

Design principle 1: conceptual framework

The STeLLA VbPD program was designed to improve elementary science teaching at a time when science teaching and science PD activities were low on teachers’ priority lists (Dorph, Shields, Tiffany-Morales, Hartry, and McCaffrey 2011). At the same time, researchers and PD providers were advocating that effective PD engages teachers in sustained, collaborative analysis of practice over time using artifacts of practice such as videos and student work (Ball and Cohen 1999; Garet et al. 2001). How could we interest elementary teachers in participating in such sustained science PD? In this context, the STeLLA research and design team decided to explore what would be possible if elementary teachers engaged in video-based, analysis-of-practice PD activities intensively for 1 year. This limited time frame raised important questions about the substance of the PD program: What is possible within a 1-year period? Is it possible to change science teaching practice enough in just 1 year to impact student learning? To achieve such a goal within a 1-year period, it was assumed that the substance of the program would need to be tightly focused on a few ideas and teaching practices addressed in depth and over time.

Based on a thorough review of the research on effective PD and the much larger research base about effective science teaching, the STeLLA-I design team made difficult choices about what to focus on and what to leave out. The result was a conceptual framework for the entire PD program (see Fig. 2). The framework centers on two types of pedagogical content knowledge (PCK) that were hypothesized to have the most impact on teacher learning, science teaching practice, and student learning within a 1-year time frame: (1) revealing, supporting, and challenging students’ ways of thinking about specific science content, and (2) supporting students in constructing coherent science content storylines. Note that we did not attempt to address all kinds of pedagogical content knowledge. We selected these two dimensions for three reasons. First, they represent an important subset of the topic-specific professional knowledge defined in expert models of PCK (Gess-Newsome 2015). Second, we hypothesized that these two dimensions would be impacted by the video analysis intervention. And finally, these are perspectives that teachers do not typically use to frame and guide their planning and teaching (Sherin and Van Es 2002; Zannoni and Santagata 2002). To focus on these aspects of PCK, teachers in the program use two lenses to guide their video-based analysis work and their classroom science teaching practice: the Student Thinking Lens and the Science Content Storyline Lens.

Throughout all of this, the teacher probes student thinking to find out how students are making sense of new data or ideas, challenges students to stretch their thinking and to make new connections, and teaches students to communicate in scientific ways (such as making claims, providing evidence and reasoning to support claims, and listening and responding to others’ ideas).

The Science Content Storyline Lens focuses attention on how the science ideas in a science lesson or unit are sequenced and linked to one another and to lesson activities to help students construct a coherent “story” that makes sense to them. The Trends in International Mathematics and Science Study’s (TIMSS) video analysis of science teaching and Horizon’s Looking Inside the Classroom observational study of science teaching in the USA both identified that US science lessons frequently fail to support students in making connections between science classroom activities and the development of science ideas and explanations (Roth et al., 2006; Weiss et al. 2003). The Science Content Storyline Lens addresses this gap by supporting teachers in using nine planning and teaching strategies that help students build coherent content storylines (see definitions in Appendix 2)

Grounded in research, the conceptual framework serves as the foundation and the centerpiece of the PD program, with all summer and school-year sessions organized around the STeLLA lenses and strategies. It anchors the program and assures program coherence by keeping the substance and activities in the program tightly focused on two analytical lenses and a few key ideas and teaching strategies that can be explored in depth and developed into teachers’ practice.

Design principle 2: specified teacher and student learning goals

The STeLLA VbPD program is guided by clearly specified teacher and student learning goals that are closely tied to the conceptual framework. Identifying and articulating these learning goals from the beginning is essential for designing and implementing both science lessons and PD sessions that are coherent, focused, and supportive of student/teacher learning. In our work, learning goals for students focus on understanding and using core science ideas. Learning goals for teachers include science content knowledge, pedagogical content knowledge related to classroom science teaching, and abilities to use both these types of knowledge to analyze and teach science.

Additional learning goals for teachers

In addition to science content learning goals, STeLLA teachers are also expected to develop pedagogical content knowledge related to the Student Thinking and Science Content Storyline Lenses and teaching strategies. Furthermore, it is a goal for teachers to be able to use their content knowledge and their pedagogical content knowledge to analyze science teaching and learning. A final broad learning goal is for teachers to use their content and pedagogical content knowledge about the STeLLA lenses and strategies in their science teaching practice. The conceptual framework outlines the specific teaching strategies that teachers learn to understand and use (Fig. 2).

These broad teacher learning goals related to PCK, analysis abilities, and teaching practice were useful in creating the overall design of the STeLLA program and the development of research assessments. However, these overall program goals were insufficient for mapping out the daily PD sessions during the summer institutes and school year study group sessions. Over time, especially as we began to develop new PD leaders and to write detailed PD leader guides, we recognized the need to state more specifically the teacher learning goals for each professional development session and for each segment of a PD session. We recognized that just as students’ science lessons are more coherent and powerful if they remain clearly focused on one main learning goal throughout a lesson (STeLLA strategy A, see Fig. 2), so our PD sessions are more coherent and powerful if we remain focused on what we expect teacher participants to learn from a particular session and from particular segments of a PD session. Figure 3 shows how the STeLLA PD leader guide provides fifth grade PD leaders with specific statements of purpose and intended learning goals for a 70-min time segment that occurs during a full-day summer institute session.

In STeLLA VbPD, such specificity of the learning goals is critical in assuring coherence among all the components of the program. It plays a key role in developing the science lesson plans and videocases, in selecting video clips to use for analysis, and in deciding how to focus the video analysis discussions. Video analysis work is time intensive, so planning for a productive session is important. Video clips for STeLLA analysis are selected not just because they are interesting but because they will help teachers wrestle productively with intended learning goals—both specific science content learning goals and pedagogical learning goals.

Design principle 3: program substance prioritizes depth versus breadth

A key design feature of all STeLLA-related programs is depth over breadth—maintaining a relatively small set of teacher learning goals that are introduced early in the program and then revisited and practiced throughout the program. The substance of the program is limited to a few core science ideas in two topic areas, the STeLLA teaching strategies, and the STeLLA video analysis process that uses the Student Thinking and Science Content Storyline Lenses as analytical tools. In parallel with what we know about students’ needs in developing understandings of science ideas, teachers also need multiple opportunities to use and analyze the science content ideas and the STeLLA teaching strategies that they are learning. Teachers initially encounter almost all of the program’s science content ideas and STeLLA teaching strategies during the summer institute. Across the school year, only a few new learning goals are introduced. Instead, teachers have multiple opportunities to use and deepen their understandings of previously introduced ideas in their teaching and in their collaborative analyses of videos and student work.

Design principle 4: theory of teacher learning

Many PD programs are not guided by an articulated theory of teacher learning (Ball and Cohen 1999; Borko 2004; Putnam and Borko 2000). In the STeLLA program, in contrast, decisions about program design are explicitly guided by a situated cognition theory of teacher learning and a cognitive apprenticeship model of instruction. Situated cognition posits that learning is naturally tied to authentic activity, context, and culture (Collins 2006; Lave and Wenger 1991). In line with this theoretical stance, the STeLLA program is organized to support grade-level specific, classroom-based (“situated”) learning over time, with more direct scaffolding and guidance by PD tools and PD leaders at the beginning of the program that moves toward more teacher-directed work at the end of the 1-year program. The cognitive apprenticeship phases of modeling, coaching and scaffolding, and fading guide how the PD leaders work with participants across time.

Design principle 5: video-based analysis of practice

What is being analyzed in STeLLA video analysis? Before describing what teachers and PD leaders do in STeLLA video analysis, we start by linking back to the foundational design principles 1–3 and the importance of clearly defining the substance of the PD program; that is, what are teachers going to learn from this video analysis work? What are they analyzing and why?

While most descriptions of PD programs in research reports provide only a broad, general view of the program substance, we attempted to be more specific in defining what teachers would learn from video analysis while at the same time resisting the temptation to “cover” everything that we know is important in achieving the best practice in science teaching. For the science content, we identified science ideas in the teachers’ curriculum that we knew to be challenging for students and teachers, to be useful in explaining a variety of phenomena in teachers’ and students’ experiences, and to be linked to important crosscutting concepts (e.g., matter and energy).

To identify the STeLLA teaching strategies, we reviewed the research on science teaching and learning to identify what others now refer to as “high-leverage teaching practices” (Ball and Forzani 2011; Windschitl, Thompson, Braaten, and Stroupe 2012: Grossman, Hammerness, and McDonald 2009). And it was important to us that the lenses and strategies in the conceptual framework were not just a random collection of teaching strategies but that they hung together to help teachers develop a vision of effective science teaching that they could internalize over time. Thus, before we decided how to organize and structure video analysis, we worked hard to clarify why teachers would be analyzing videos and to sharpen our understanding of what they would be learning from this work.

What is the form of STeLLA video analysis? Descriptions of PD programs often focus more on the form of the program than on the substance—how teachers are organized to work together, how often they meet, when and where they meet, for how long, who leads the work, the kind of activities they engage in, and so forth (Kennedy 1999). The following features of program form are included in the version of the consensus model presented earlier: active participation of teachers in analysis of practice, sufficient duration, and collective participation of teachers (teachers at a given school or grade level). The STeLLA design principles include attention to each of these consensus model features of program form. Principle 5 defines analysis of practice as a core active learning activity for teachers. Teachers are also engaged actively in content deepening activities (principle 6) and teaching activities (principle 7). Principle 8 identifies program duration and intensity as a key feature, with STeLLA teachers meeting together for 2 weeks in the summer followed by monthly ½ day meetings during the school year. In between meetings, they teach STeLLA lesson plans and collect and analyze student work. Principle 9 addresses the collective participation of teachers at the same grade level. STeLLA teachers meet face-to-face in small grade-level specific study groups (5–8) that are led by experienced PD providers.

But there is an important way in which the STeLLA program goes beyond the consensus model in defining its program form. Decisions about STeLLA program form are driven by the situated cognition of teacher learning and the cognitive apprenticeship model of instruction (foundational design principle 4). This is especially evident in the way different kinds of activities are sequenced over time.

In the summer institute, teachers begin the analysis process using videocases from more experienced teachers. In addition to the modeling provided by these teachers in the videos, PD leaders model how to productively analyze these videos and they coach teachers in their analytical efforts. Thus, in line with our theoretical stance, teacher participants have access at the beginning of the program to models of expertise.

During the fall of the school year, they transition to analyzing video their own and their peers’ efforts to teach STeLLA model lesson plans, receiving support, feedback, and guidance in this work through their collaborative analyses of videos and student work in study group interactions. This is consistent with the coaching phase of the cognitive apprenticeship model.

In the second half of the school year, scaffolding supports such as model lesson plans are removed (fading) and teachers are challenged to use STeLLA strategies and tools to plan and teach lessons in a new content area. This intentional use of a theory of teacher learning required us to consider what types of activities would be most beneficial for teacher learning across time, rather than settling on one type of video analysis and using it throughout the program.

What features contribute to productive video analysis? While the foundational design principles drive the substance and form of STeLLA video analysis, there are a number of other features that work together to make important contributions to the quality of video analysis and to teacher learning:

Design principle 15: PD leadership

In the STeLLA design, leadership of analysis-of-practice work plays a critical role in deepening teacher learning. Our choice of the title, “PD leaders,” reflects our recognition that leadership is necessary for the kinds of transformative change we seek. Some might view this as too “top down” and argue for an approach where teachers and PD leaders are co-constructing knowledge together (this is what some refer to as “facilitating” rather than “leading”). But our experience early on in the development of the STeLLA program convinced us that elementary teachers—who are typically reluctant to teach science—initially need both access to expertise from the science education community and support in learning to incorporate this knowledge into their science teaching practice. With fading supports from PD leaders over time, study groups gradually begin to function in a more bottom up way. In fact, we can imagine experienced STeLLA study groups functioning productively without a PD leader as they continue beyond the 1-year STeLLA program because they know how to analyze science teaching and learning productively to improve their science teaching practice, they know when and where to find outside expertise, they understand how to function as a true learning community, and they understand the kinds of science knowledge they need to develop when they approach new science content areas—knowledge that enables them to explain a variety of phenomena using key science concepts and practices.

Design principle 16: PD leader knowledge and decision-making abilities

It is not enough to say that “PD leadership is important.” In thinking toward the scale up of the STeLLA approach, we recognized the need to understand what it takes to develop new PD leaders: What kinds of knowledge and abilities are needed? Analysis of the work of effective STeLLA leaders enabled us to answer this question with the knowledge and decision-making model of STeLLA PD leadership shown in Fig. 4 (Landes & Roth, 2013).

The model highlights that effective leadership of STeLLA lesson video analysis depends on PD providers who are able to draw from a rich knowledge base to make decisions and take actions—both in planning and while leading video analysis sessions—that support teacher learning through deep and meaningful analyses. The model specifies that well-prepared STeLLA PD leaders make decisions and take actions by drawing from three types of knowledge: (a) knowledge of the STeLLA PD curriculum (conceptual framework, learning goals, STeLLA strategies, STeLLA resources, lesson plans, etc.), (b) science content knowledge (including how science practices can be used to explain phenomena and to understand disciplinary core ideas and cross cutting concepts), and (c) research-based knowledge about how teachers learn in communities of learners. In addition, PD leaders draw from knowledge of the particular videos that are being analyzed. PD leaders know how to use all of this knowledge to make planning and in-the-moment, program-aligned decisions and adaptations.

A key planning decision made by PD leaders is the selection and sequencing of video clips so that they best support targeted teacher learning goals for a given study group session. PD leaders watch the full lesson videos from the participating teachers to select a 3–8-min video clip from each teacher whose video will be analyzed during a given study group session. In making these selections, the PD leaders consider ways in which the video clips can be used (a) to deepen teachers’ understanding of the STeLLA strategies, (b) to clarify and deepen teachers’ science content understandings, and (c) to analyze interesting examples of student thinking. PD leaders also consider how the set of 3–4 video clips for a given study group session can hang together to focus on specific teacher learning goals for the overall meeting. From watching the full lesson videos, the PD leader learns a great deal about the strengths and weaknesses of the lessons and of teacher implementation of the lessons and uses this knowledge to identify key issues that could be productively examined through video analysis. Last but not least, PD leaders consider how a video clip might support and encourage the teacher featured in the clip while also challenging that teacher to reflect on her/his own teaching in ways that lead to growth; this requires assessing that teacher’s readiness to be challenged. All of this is considered as PD leaders select the video clips, identify specific teacher learning goals for the session, and create specific identify and analysis questions to focus the discussion and analysis for each clip.

A key decision PD leaders make is about when and how to address gaps in teachers’ science understandings that arise during video analysis. Taking the time in video analysis sessions to wrestle with science content confusions is a critical aspect of the STeLLA program. Our research results show that teachers’ science understandings were better developed in the context of lesson video analysis (in the STeLLA program) than in sessions that focused specifically on deepening teachers’ understandings of the science content (in the content deepening comparison program). This is consistent with our situated theory of teacher learning—understanding the science content is more meaningful to teachers when it occurs in relationship to their teaching context.

Without the PD leaders’ active role in scaffolding the video analysis process, especially at the beginning of the program, teachers tend toward a surface level discussion of video clips, favoring positive comments about the teaching or teacher featured in the clip. With PD leader guidance and use of the Lesson Analysis Protocol, teachers’ attention is focused on student thinking, the science content, the storyline, and the use of STeLLA strategies, minimizing the focus on an individual teacher while maximizing the analysis of more consequential issues that impact student learning.

Design principle 17: PD leader development

The development of STeLLA VbPD leaders who have the knowledge and abilities described in Fig. 4 is critical to enabling more teachers and students to have access to this powerful program. To address this need, a leadership development program was designed, developed, and enacted in the RESPeCT project. The leadership development program attends to the full scope of the STeLLA knowledge and decision-making model (Fig. 4).

When the audience is experienced science PD leaders with little experience with the STeLLA VbPD program, the balance of leadership development tips toward the content of the STeLLA VbPD curriculum (lower left hand corner in Fig. 4). However, when the audience has experienced the STeLLA VbPD program and implemented the STeLLA classroom curriculum, the balance of leadership development tips toward developing understandings about and abilities to lead adult learning (lower right hand corner in Fig. 4). For science and mathematics professors unfamiliar with either the STeLLA approach or working with teachers, leadership development focuses on the bottom two corners of the triangle in Fig. 4, developing their understanding of the STeLLA VbPD program goals and components and their knowledge about teacher learning in learning communities.

Regardless of audience, one element of VbPD leadership development remains constant—a focus on making decisions and taking actions consistent with the STeLLA approach. Effective PD leaders know the purpose for each component of the program. They understand what they are doing as PD leaders and why they are doing it. Major goals and activities in the leadership program are described in the following paragraphs.

Taking on a leadership identity

To help classroom teachers who have not previously led PD see themselves as leaders, we are explicit about how the leadership development program helps prepare them to lead the learning of their teacher colleagues. For each PD experience, we point to a model similar to the figure presented in the introduction to this special issue (Tekkumru-Kisa and Stein 2017, Fig. 5). This model helps RESPeCT teacher participants see where they fit in terms of their experiences as a learner in the STeLLA program (teacher as learner, inner layer), their experiences in the RESPeCT leadership development program (facilitator as learner, outer layer), and their eventual role as PD facilitators (middle layer).

Design principle 18: partnership development and design principle 19: scalability and sustainability

While the STeLLA VbPD approach has demonstrated strong results in terms of teacher and student learning, it is an ambitious program that faces challenges in expanding the reach of the program. One challenge is the development of new, grade-level specific PD leaders who have the knowledge and abilities described in the previous section. Other requirements that might be impediments to long-term sustainability are the need for funding and other benefits to motivate teacher PD leaders in continuing to offer the VbPD to peer teachers; funding and other rewards to encourage teachers to participate in the VbPD; support and strategies for filming classrooms and making, transcribing, and disseminating video clips on a tight timeline; time for teacher PD leaders to review lesson videos and plan study group sessions; funding for and distribution of PD and science lesson plan materials; stable student science learning goals at each grade level so that the videocases remain relevant; stable grade-level teaching assignments so that teachers are teaching the content addressed in the VbPD program; supports to motivate and sustain university faculty commitment to providing ongoing science content advice to teacher PD leaders beyond the life of the grant; refilling the pipeline of teacher leaders and university support faculty; and strong district and university leaders who can lead successful responses to changes in staffing, curriculum, and institutional priorities that impact the VbPD program.

To preserve fidelity to the STeLLA design features and sustain the VbPD program, we must do more than prepare new teacher PD leaders. Educational systems must also be readied. In the RESPeCT project, the systems involved in sustaining the work are the Pomona Unified School District and the California State Polytechnic University at Pomona.

Together, but not individually, CPP and PUSD have the needed expertise identified in design principle 18: science education experts and researchers, scientists and mathematicians, teachers, and school personnel at multiple levels. This combined expertise is necessary to modify the VbPD program in ways that preserve the STeLLA design features while incorporating features that will contribute to the program’s scalability and sustainability. Thus, neither organization can scale and sustain the program independently of each other. However, the components of readiness required by both partner systems include (a) a shared priority for improving student science learning, (b) shared values and commitments to VbPD; (c) ongoing communication and shared decision making to support, sustain, and study the collaboration; (d) investments in building capacity for VbPD; and (e) inclusion of multiple stakeholders in shaping and implementing the program.

Developing systems readiness

Both CPP and PUSD partners have worked on developing these components of system readiness, as described in the following paragraphs.

Importance of research on student learning

Improving student learning outcomes is essential to sustain the current high level of support for the RESPeCT project. Both partners initially bought into the program largely based on the multiple and strong lines of evidence of effectiveness of this VbPD approach not only for teacher learning and teaching practice but also more importantly for student learning. Thus, high-quality research is essential for sustaining this work.

Strong preliminary analyses of teacher and student science learning and the enthusiasm of participating teachers are fueling efforts in the district to find ways to fund this work in the future. For example, PUSD is already committed to supporting ongoing RESPeCT PD work by using the district’s summer PD time for the RESPeCT program, taking on the tasks of purchasing and supplying science kits, scheduling the videotaping of science lessons, creating video clips and transcripts of those clips, and copying and organizing binders for future teacher participants. In support of these district commitments, the RESPeCT leadership team and CPP support staff are experimenting with ways to make the videotaping, uploading and sharing of video, clip making, and transcription processes easier for teacher leaders and Pomona support staff to do. In addition, some of the CPP science faculty are interested in continuing to provide district RESPeCT teacher leaders with “phone a friend” science content support even after the end of the grant.

Impact on student learning is also important at the university. In addition to formal research goals, the project also hopes to improve science and mathematics teaching and learning at the university through participating professors’ implementation of STeLLA strategies in their university classrooms. In addition, we anticipated from the beginning that some CPP science and math faculty will become ready and motivated to start up and support RESPeCT PD efforts in new school districts, especially those in areas surrounding the university so that future CPP students receive a strong foundation in their scientific understandings. Toward this goal, the project currently supports CPP science and math faculty with mentoring, modest reassigned time, experiences that help them learn about the K-6 teaching context in needy Title I schools, and involvement in education research and publication. Continued university support is needed to encourage faculty to engage in VbPD work beyond the life of the grant.

The power of the set of STeLLA design principles

We argue that this set of design principles is powerful and useful beyond this one project for four reasons. First and foremost, the use of these design principles produced a video-based, analysis-of-practice PD program that had a significant impact on teacher learning, on teaching practice, and, most importantly, on student learning. In addition, teaching practice was demonstrated to be highly correlated with student achievement. Second, the design principles were built on existing theoretical and empirical research literatures. Third, none of the design principles were dropped or significantly revised as the program moved over a 10-year time span to different geographic locations, school district contexts, and grade levels. In fact, the design principles proved to be critical guides in determining how changes would be made to adapt to new contexts without losing the power of the program’s impact on teaching and learning. For example, as the program moved in 2009 from STeLLA-I in California to STeLLA-II in Colorado, the design principles played a key role in guiding our work with new state science teaching standards and in supporting us in resisting the temptation to tailor the program to the specific wishes of each school district. Adaptations were made only when they were consistent with the design principles. Similarly, we responded to the release of the NGSS in 2013 by reviewing our conceptual framework in light of this reform document. We were able to make minor changes to the STeLLA teaching strategies and the language describing these in the STeLLA strategies booklet without needing to change our overall conceptual framework. Thus, the design principles remained largely stable over time, another indicator of their power.

Finally, the design principles have power because they are closely interwoven and supportive of each other. For this reason, it was difficult to write about them separately in this paper. For example, productive learning through video-based, analysis-of-practice (principle 5) is dependent on working in a context where teachers feel safe to work with their colleagues as they to dig deeply and challenge themselves and each other (principle 9). Such challenges and deeper analyses are more likely when tackling shared content- and grade-level specific curriculum (principles 2, 3, 10) and using the same curriculum materials (principle 11). Feeling safe in a learning community is not productive if you are not challenged to stretch and change your thinking. In the STeLLA line of research, we have found that such challenges can come from knowledgeable and skilled PD leaders (principles 15, 16), as well as from models by more experienced STeLLA teachers (principle 12). But the ultimate goal is to create learning communities where such challenge can come from teacher participants themselves (principle 9) as they learn from videos from their own classrooms (principle 12). And all of this takes time (principle 8).

Challenging the consensus model of effective PD

At the beginning of this paper, we made the case that the long-accepted and widely used consensus model of effective PD is of limited value in guiding the development, implementation, leadership, and scaling up of VbPD programs that positively impact teaching and student learning. We argued that this model lacks research support showing that it is effective in improving teaching and student learning. In addition, it provides only broad, surface level recommendations about how to design and implement PD experiences: Effective PD is content-focused, engages teachers in active learning, is matched to local, state, national policies, is of sufficient duration, and involves collective participation of teachers at a school or grade level.

Of course, a refined consensus cannot be built based on findings from this one study. Additional research is needed before this list of nominated features can change the field’s consensus about effective PD (and effective VbPD, in particular). There is some existing research that examines the impact of science PD on both teacher and student learning and supports some of these STeLLA nominations. For example, results from the Heller et al. study (2012) support the intertwining of science content learning with analysis-of-practice activities. The importance of scaffolding by knowledgeable PD leaders emerged in studies by Greenleaf et al. (2011), Heller et al. (2012), and Penuel et al. (2011). But none of these studies looked specifically at video-based PD. Clearly more research is needed.

We hope that this analysis of lessons learned from the STeLLA design principles can be useful for the design, implementation, and study of other professional development experiences for teachers, especially those that focus on video-based analysis of practice. We believe that an evidence-based articulation of VbPD design principles for the field would represent a major step forward in our knowledge about effective PD. This could help us move from the current surface level recommendations about “effective” PD that are today so widely used to guide the development of science teacher professional development programs to a set of well-specified, coherent design principles that are linked to evidence of student learning.