STEM education institutional change projects: examining enacted approaches through the lens of the Four Categories of Change Strategies Model

Abstract

Background

Enacting STEM education reform is a complex task and there are a variety of approaches that might be selected by change agents. When working on an institutional change project to impact multiple parts of the STEM education system, teams of change agents may select multiple strategies and tactics to enact at one time and over multiple years of a project. However, the literature lacks studies which document and analyze strategies and tactics used by change project teams in a way that can be useful for other change agents. The current study seeks to fill this gap by investigating National Science Foundation-funded change initiatives at three public research universities focused on encouraging the adoption of evidenced-based instructional practices by STEM faculty in order to understand the strategies used within and across projects.

Results

Qualitative framework analysis using the lens of the Henderson et al. (Journal of Research in Science Teaching 48(8): 952–984, 2011. https://doi.org/10.1002/tea.20439) Four Categories of Change Strategies Model showed that institutional projects enact a wide range of tactics that span the four strategies represented in the four categories of the model both across institutions and within each institution. The analysis documents a number of change tactics not previously described by the model and offers expanded definitions of the change processes that operate within each category in the context of institutional change projects.

Conclusion

This descriptive work advances our understanding of the breadth and depth of actions taken by institutional change initiatives and provides insights into types of variations that might be observed based on different institutional contexts. The current analysis both affirms the value of the original model and identifies expanded ways to think about the four categories within the context of institutional change projects.

Introduction

There are broad calls going back decades to change Science, Technology, Engineering, and Mathematics (STEM) pedagogy in the service of improving student outcomes related to equity, retention of students, and student learning (Honey et al., 2020; Urbina-Blanco et al., 2020). In response to these calls, many different strategies to facilitate change in STEM teaching practices have been enacted and studied, such as communities of practice (Cross et al., 2021; Gehrke & Kezar, 2017; Johnson et al., 2021; Kandakatla & Palla, 2021; Ma et al., 2019), faculty learning communities (Cox, 2001; Shulman et al., 2004), faculty development workshops on evidence-based instructional practices (EBIPs) (Biswas et al., 2022; Derting et al., 2016; Haviland et al., 2010; Phuong et al., 2020), and efforts to build shared vision (Doten-Snitker et al., 2021), among others.

In support of these calls for change, national organizations and funding agencies, such as the Association of American Universities, National Science Foundation (NSF) and Howard Hughes Medical Institute, have dedicated significant resources toward efforts to support transformational STEM education change at the institutional level or even across institutions. These efforts are important because they necessarily invite change agents to think at the institutional level, beyond disciplinary boundaries and beyond a single activity or tactic. Studies that emerge from these efforts help to illuminate the conditions and effort needed for successful institutional change. For example, a number of studies have documented the efforts and their impact across institutions (AAU, 2017; Chasteen et al., 2015; Foote & Knaub, 2018; Hill et al., 2019; Kezar & Holcombe, 2021; Lord et al., 2017; Peteroy-Kelly et al., 2019). The findings from studies of this type are generally focused on extracting broad ideas (e.g., the importance of administrative support or the finding that a supportive institutional culture is needed for successful departmental change). They are particularly important because they illuminate lessons learned that are likely to be impactful across institutional contexts. However, because of the level of analysis (i.e., across institutions), these approaches to understanding institutional change are less likely to illuminate the particularities of a change process within a given institution. Publications associated with institutional change projects funded by the NSF and aimed at STEM education transformation can provide some insight about institutional-level change processes. Here we find examples of publications that describe particular programmatic approaches (Karlin & James, 2014; Vanasupa et al., 2014), reforms at the course level (Herman et al., 2018; Zhao et al., 2015), change project deliverables (Madsen et al., 2017), research examining teaching practices or faculty perceptions (Lund & Stains, 2015; Nguyen et al., 2017; Sansom et al., 2023; Shadle et al., 2017; Stains et al., 2018; Williams et al., 2015, 2022), and research examining student outcomes (Barthelemy et al., 2015; Mooring et al., 2016; Pond & Chini, 2017). Generally, these studies reflect a relatively narrow scope, which suggests either that the funded efforts focused on a particular aspect of the complex STEM education system (e.g., a particular course or programmatic approach) or that what was shared for publication represents a select portion of a larger, more complex change initiative. In either case, there remains a need to understand how multiple categories of change strategies might work together to facilitate change across an institution. By design, institutional change projects working to impact multiple levels of the STEM educational system are complex and involve a wide variety of activities being enacted at one time and over multiple years of a project. By characterizing the range of activities that occurred in three grant-funded projects, the current study seeks to further our understanding of institutional change and to contribute to the growing repository of knowledge about how institutional change might be supported.

Theoretical framing

In 2011, Henderson, Beach and Finkelstein published a framework designed to categorize the strategies that were extant in the literature for supporting STEM education change. This analytic review sorted strategies along two axes: the focus axis and the outcomes axis. The focus axis is a continuum from a focus on supporting change at the individual level to a focus on changing institutional structures and environments. The outcomes axis is also a continuum; it represents whether the outcomes of the action taken are prescribed by the change agent or emergent. The two axes allow for the categorization of institutional change initiative actions into four categories. In the resulting Four Categories of Change Strategies Model, each of the four categories has a descriptive label. Further, for each category, the authors described a “change process”, which is a general mechanism by which the actions taken within a given category appear to effect change. Each category represents an overarching strategy to fostering change; the activities that might be undertaken under the umbrella of that strategy can be described as tactics (Henderson & White, 2019). An adaptation of the model is shown in Fig. 1.

Fig. 1

Representation of the Four Categories of Change Model from Henderson et al. (2011)

Category I represents the strategy by which a change project seeks to impact individuals to achieve a prescribed outcome. The strategy represented by this quadrant is labeled “disseminating: curriculum and pedagogy” and is associated with the change process “Tell/Teach individuals about new teaching conceptions and/or practices and encourage their use”. Category II represents the strategy by which a change project is focused on emergent outcomes and is targeting the desired impact on individual faculty. Its label is “developing: reflective teachers” and the associated change process is that change agents “engage/support individuals to develop new teaching conceptions and/or practices.” The strategy focus in Category III has prescribed outcomes and is focused on impacting the environment and structures of the institution. The strategy in this quadrant is labeled as “enacting: policy” with the associated change process “prescribe new environmental features that require/encourage new teaching conceptions and/or practices”. Finally, Category IV represents the strategy focused on emergent outcomes that are designed to impact the environment and structures of the institution. Its label is “developing: shared vision” and the associated change process is that change agents “Empower/Support stakeholders to collectively develop new environmental features that encourage new teaching conceptions and/or practices”. Within each category, a given strategy and change process might be supported by a number of specific activities, or tactics, that help the project move toward the desired change (Henderson & White, 2019). As part of their original analysis (Henderson et al., 2011), the researchers observed that the strategies for making change tended to emerge and be centered within different disciplinary traditions; they noted that discipline-based education researchers, educational developers, and higher education scholars tended to take different approaches to facilitating change that fell in different categories. In acknowledging the complexity of the STEM education system, the authors suggested that “change agents would be wise to learn about strategies outside their typical practice and to work with other change agents across disciplinary boundaries” (Henderson et al., 2011, p. 979).

Follow-on work conducted by Borrego and Henderson (2014) used the four categories to help change agents think about the underlying logic for change activities that might fall in each category, to prompt thinking about their distinctions, and how they might be interconnected. Additionally, Borrego and Henderson (2014) suggested that rather than focusing on one category for change it is important to take a systems view of how change initiatives operate; they asserted the need for a “diverse set of goals” focusing at different levels of the STEM educational system (Borrego & Henderson, 2014, p. 244). Additional work by Henderson and White (2019) has noted that any project seeking to have a sustained impact on the institution will need to use a strategy focused on “Environments and Structures” (Categories III and IV), even if there are activities, or tactics, that might be aimed at individuals.

An important gap in the literature about institutional change is the lack of a detailed description of the range of strategies and tactics that change agents in such projects select when working to enact change. We also lack an understanding of how a set of activities might align with the previously identified categories of strategies for change in a complex project. Finally, we lack descriptions of how a set of strategies and tactics fit together within a given project in response to an institution's specific context. To address these gaps, we chose to leverage information in the gray literature, specifically, annual reports to the NSF. Analyzing NSF reports provides a number of affordances, including insight into strategies and activities generally not documented outside of change teams, including granular level logistics for the change initiative and facilitation of institutional-level discussions. This descriptive study examines in detail the full range of activities enacted by three independent NSF-funded institutional transformation projects, each of which was focused on increasing the adoption of Evidence-based Instructional Practices (EBIPs) by STEM faculty, along with other student success-oriented goals. The focus of the study is on actions taken by change projects independent of the outcomes. The projects that are the subject of this study sought to impact multiple aspects of the STEM education system and were enacted by institutional teams, with representation from multiple disciplines. As part of our analysis, we use the Four Categories of Change Strategies Model to gain insight into the extent to which multiple strategies might be operating and then to use that information to understand the change processes operating within the projects. Specifically, we seek to address two research questions:

• 1) To what extent do the activities undertaken by large institutional change initiatives reflect the use of multiple change strategies aimed at impacting the STEM education system?

• 2) How might the strategies and tactics used by these projects inform our understanding of the change processes that operate in the context of institutional change initiatives?

Methods

Data corpus

We used annual and final reports submitted to the NSF by three institutional change projects as the source of information about the actions of each change project. The projects selected were a convenience sample. Some of the researchers involved in this study were involved in project implementation; others were not involved at all or only peripherally. The research questions in this study emerged out of collaborative interest in better understanding the change processes operating across our institutions. The annual reports were provided directly to us by the project leaders for our analysis. Annual reports are an example of gray literature, which has been defined as “information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing, where publishing is not the primary activity of the producing body” (Grey Literature Guides, 2023). Annual reports were used because they serve as the official documentation of what happened in the project. Further, they provide a broad overview of the actions undertaken by each project in each year of funding, rather than showcasing individual areas of focus that may be highlighted in published works about projects. We analyzed information provided by the projects to questions in the NSF annual reports under the following sections: accomplishments, products, participants/organizations, impacts, and changes/problems. Analysis did not include any attached documentation past these sections. The choice to limit analysis to these sections ensured consistency across projects; not all projects chose to include an optional supplement. Finally, because the framework for NSF reporting was the same for all projects, using annual reports allowed us to have similar depth and breadth of information about each project. We analyzed a total of 15 reports across the three institutions.

The change projects which were the focus of this study were conducted at three different institutions, each of which had received funding from the NSF, Division of Undergraduate Education for “institutional transformation” focused on STEM education reform. All three institutions are large, research focused state institutions in the United States.

Institutional contexts

Institution A is a large, public doctoral university with very high research activity. The change team consisted mostly of discipline-based education research (DBER) faculty in STEM at the institution and included an upper-level administrator. The project engaged chairs and faculty members across fourteen STEM departments. The project had three main areas of activity: (a) conducting educational research and using results to provide empirical evidence to support faculty members’ engagement in instructional change; (b) engaging stakeholders in changing the expectations for evaluation of teaching on campus; and (c) providing pedagogical professional development programs to STEM faculty members.

Institution B is classified as a large, public doctoral university with very high research activity. The change leadership team consisted of faculty and department chairs in STEM, a member of the Provost’s office staff, and professional advising staff. The project also engaged DBER faculty and staff from the institution's Center for Teaching and Learning. The project intentionally engaged with a local community college to support students transferring to the university. This project engaged nine STEM departments from two colleges (College of Arts and Sciences and College of Engineering) and included involvement of STEM faculty from a local feeder community college. The project’s main activities provided (a) faculty support to design their courses, (b) graduate student laboratory training on evidence-based teaching practices, and (c) peer support to undergraduates transferring to the university from a local community college.

Institution C is a large, doctoral university with high research activity. The change team consisted of academic leaders (deans from each college with STEM departments), the director of the institution’s Center for Teaching and Learning, and four STEM faculty. The project engaged all 12 STEM departments from two colleges (College of Arts and Sciences and College of Engineering). The project’s main activities involved support to faculty and departmental teams for course redesign and efforts to catalyze dialogue and activity within the departments by supporting a faculty liaison from each department.

Analytical framework

This research study utilized Framework Analysis as the analytical framework (Ritchie & Spencer, 2002). Consistent with the steps involved in Framework Analysis, our process involved data familiarization, including an initial round of categorization. Study data were then coded, followed by charting/visualizing within a thematic framework to summarize the coded data and identify patterns (Goldsmith, 2021; Kiernan & Hill, 2018; Parkinson et al., 2016; Ritchie & Spencer, 2002). Rather than develop a new framework, we used framework analysis as a way to root our data in the existing Four Categories of Change Strategies Model (Fig. 1; Henderson et al., 2011), which served as a scaffold to sort and interpret actions taken by the initiatives and as a way to generate the overarching change processes operating in each category.

Our analysis began with data familiarization and identification of broad classification labels for all actions taken by each project. The actions we examined and discussed in this paper are analogous to the “strategies” used by Henderson et al (2011) and Borrego and Henderson (2014). We analyzed only actions that were reported to have been completed; we did not include planned actions. This was followed by an inductive coding process for all actions. Our focus was on cataloging all actions; we did not count how many times a particular code appeared in the data. Further, because our data source did not allow us to determine the relative importance of a given action, we chose to treat each action as equally weighted. The coding process was conducted using the qualitative analysis software, Dedoose. The two researchers involved in the coding, S.F. and S.S., employed a consensus coding approach (Saldaña, 2013). S.F. and S.S. collaboratively coded actions from one report (from Institution A) in order to clarify the meaning of each code. As more coding was complete, they identified nuances in the codes, and refined or identified new codes as needed (Saldaña, 2016). One coder was very familiar with one of the projects and the other was unfamiliar with details of any of the projects; this approach allowed the coding to be both informed by deep knowledge of how projects of this type proceed and by a more distant perspective from outside the projects. Coding then proceeded independently for the remaining reports with the researchers engaged in ongoing, reflexive dialogue throughout the coding process to ensure that codes were being used consistently (Braun & Clarke, 2006; Saldaña, 2013). The researchers discussed any instances of disagreement until consensus was reached (Saldaña, 2013). A complete list of the codes, code definitions, and example excerpts for each code are included in Additional file 1: Appendix Table A1.

The next step involved mapping the identified codes onto the Four Categories of Change Strategies Model. For each collection of actions represented by a code, we asked whether the approach was focused on primarily impacting individual faculty or the broader institutional environment and on whether a particular strategy had prescribed or emergent outcomes. We utilized the questions developed by Henderson et al. (2011) for their original analysis to help guide our mapping process. Namely, “What is the primary aspect of the system the activity seeks to directly impact?” (individuals or environments) and “To what extent is the intended outcome for the individual or environment known in advance?” (prescribed or emergent). Each code was selected to fit within one of the four categories of the model. As has been acknowledged, the categories have interconnections and overlaps. However, there is value in choosing a primary location for a code to help illustrate distinctions that assist in “relating […] various change strategies to one another” (Borrego & Henderson, 2014, p. 224). When we encountered a code that was difficult to map to a category because of this interrelatedness, we returned to the raw data and made a category choice based on particular examples in relationship to the questions that form the two axes of the model. The results of the mapping process are included in Additional file 1: Appendix Table A1. After the mapping was complete, we looked at the actions holistically in each separate category to elucidate themes (Additional file 1: Appendix Fig. A1) and to identify an overall apparent change process for the actions in each category. Then, the mapping step was repeated for data from each separate institution to generate a view of the actions for each project in each category separately. The flow of analysis is represented in Fig. 2. Throughout the remainder of the manuscript, we will refer to these coded actions as tactics, to remain consistent with the language introduced by Henderson and White (2019).

Fig. 2

The iterative process of data analysis used in this study involved numerous passes through the data to generate the framework results presented used in this study

Trustworthiness

This study was undertaken within an interpretivist naturalistic research paradigm. In attending to trustworthiness (Lincoln & Guba, 1985), this study utilized a thick description of the institutional change initiatives, their context, analysis process, and the interpretations of the data. Peer debriefing was conducted with other researchers who were not involved directly with analysis to look at the analysis and reporting of the data. These researchers included individuals with specialties in biology education, chemistry education, anthropology, and sociology. Another set of researchers who were involved in peer debriefing were members of the broader project’s advisory board which included experts in higher education and institutional change. Both teams included experts in both qualitative and quantitative methodologies. Member checking was done with individuals who were part of the institutional change teams through reporting of analysis results in all steps of the process. Member checking consisted of eliciting feedback on interpretation and characterization of the data for both their respective change project and in the aggregate.

Results

Our analysis of strategies for change enacted by three institutional change projects shows that project leaders adopted a variety of strategies and tactics to support their efforts toward STEM education reform. We identified 46 codes for actions, or tactics, which fell within the following broad classifications (alpha order): Assessing Institutional Progress/Status, Communicating to External Audiences, Engagement with Leadership, Fostering Collaboration, Infrastructure, Motivating Stakeholders, Opportunities for Discussion, Policies/Practices/Structures, Professional Development, and Roles/Positions. A full description of each code and de-identified example excerpts from the raw data are provided in Additional file 1: Appendix Table A1. In an effort to make further sense of the variety of strategies we observed, we mapped each tactic code onto the Four Categories of Change Strategies Model (Fig. 1; Henderson et al., 2011). Consistent with the approach of the original construction of the model, our analysis categorized each tactic according to whether it was primarily focused on changing the institutional environment or on impacting individual faculty (vertical axis) and whether the strategy had prescribed or emergent outcomes (horizontal axis). A visualization of the mapping of codes onto the model is provided in Additional file 1: Appendix Fig. A1.

Our results demonstrate that the projects implemented tactics across the full range of the Four Categories of Change Strategies Model. For example, all projects reported tactics that fall into Category I, with prescribed outcomes and aimed at individual faculty; these included educational development workshops and workshops designed for instructors with specific roles. Tactics that fall into Category II, with emergent outcomes and aimed at individual faculty, included events and programs which created opportunities for collaboration and/or discussions about teaching, such as faculty learning communities, retreats, and collaborative course redesign projects. Tactics that fall into Category III, with prescribed outcomes and aimed at the institutional level, included redesign of classrooms and changes to practices or policies, such as those related to the course evaluation process. And tactics that fall into Category IV, with emergent outcomes and aimed at the institutional level, included building alliances with other change makers and fostering opportunities for shared understanding of the value of evidence-based instructional practice.

Based on the mapped tactics and themes in each category, we identified an apparent change process for each category that reflects the intentional work for change that institutional change agents were enacting through their projects. The thematic label and the change process, or general mechanism for change, for each category are presented in Fig. 3.

Fig. 3

Adapted Four Categories of Change Strategies Model to reflect the change process operating in each category within institutional change projects

While the change processes presented in Fig. 3 were derived from the analysis of aggregate strategies across all three projects, we also found these processes were operative within each project. In other words, each project used tactics that span all four categories and are representative of the four change processes presented in Fig. 3. That said, the exact collection of tactics enacted by a given project were dependent on context. In order to see this comparison more clearly, in Table 1 we provide a summary of the tactics shared across more than one institution as well as those unique to each project.

Table 1 Summary of institutional change tactics across institutions

Category I—encouraging: adoption and support. The tactics enacted by projects in this category reflected change agents’ efforts to provide individual faculty and academic leaders opportunities to learn about and explore new ways of teaching. Each project offered structured professional development events and intentionally designed resources for faculty. In addition, projects engaged institutional academic leaders (e.g., provost, deans, department chairs) to provide them with the tools and information needed to support “buy-in” from individual faculty. More detailed descriptions of these tactics across the three projects are described here.

All three institutions used educational development opportunities designed to introduce evidence-based pedagogies. Examples of this tactic included a program that engaged faculty over six 90-min sessions focused on backwards course design, departmental sessions which provided an overview of evidence-based pedagogy, and short workshops offered during department meetings designed to introduce easy-to-adopt EBIPs. These events and programs were generally offered by faculty, project leadership, or staff in a professional development center (e.g., a Center for Teaching and Learning). In some cases, these events or programs were advertised and open to any STEM faculty member; in others, faculty were required to apply, or be nominated, and selected for a specific opportunity. The projects also provided documents or web-based resources designed to support instructors’ adoption of EBIPs. For example, Institution A provided web-based simulations in a particular discipline, Institution B created a resource guide for teaching assistants, and Institution C compiled and shared summaries of EBIPs with literature references grounded in particular disciplines. Institutions B and C supported faculty to travel to off-campus professional development conferences and workshops related to the adoption of EBIPs.

Some of the ways the institutions supported individual faculty to learn about and explore adoption of EBIPs were unique to each institution. As part of Institution A’s project, local DBER researchers engaged in scholarship to understand the most effective use of particular pedagogies (e.g., the use of formative assessment or Peer Instruction) on student engagement and success (Crouch & Mazur, 2001). They then used their local research results as the focus of professional development events at which others could learn about the pedagogy and its impact. Institution B provided specific professional development opportunities based on one’s role. For example, specific professional development was offered for teaching assistants engaged in laboratory instruction. In addition, this institution included workshops for advisors supporting STEM transfer students. Institution C provided data to faculty to improve student success outcomes. For example, the project helped to leverage a report that showed how students did in future courses, which helped faculty explore the impact of pedagogical changes in a prerequisite course. This institution also provided specific professional development to a group of STEM faculty serving as liaisons between the project and their home department. The professional development included support for approaches participating faculty could use to engage their department colleagues with pedagogical information and ideas.

While the direct engagement of faculty constituted most of the activity of the projects in this category, all three institutions also used specific strategies to engage individuals in institutional leadership roles (e.g., deans, chairs, faculty senators, and/or provost) with the specific aim of gaining support for project efforts. This included efforts to inform institutional leaders of ongoing progress, direct engagement of institutional leaders as participants in project activities, and invitations for institutional leaders to provide input and/or nominate faculty for involvement in project activities. These tactics were aimed at expanding the reach of the change efforts by engaging individuals outside the immediate project leadership to help provide specific support for the change process.

Category II—fostering: connections and community. The tactics that fall into this category reflect the creation of opportunities for instructors to explore and enact EBIPs with their colleagues. The tactics in this category intentionally leverage the fact that faculty have the capacity to influence one another’s teaching choices and that peers can effectively learn from one other. Consistent with project goals, activities are aimed at the adoption of EBIPs, and the choices faculty make for exploration/adoption are emergent and faculty-driven. Tactics in this category include opportunities for faculty sharing of ideas, as well as collaborative projects to move teaching changes forward.

All three institutions intentionally supported programming that allowed faculty to apply or opt-in to opportunities to learn from each other in groups over a semester or academic year. These programs provided opportunities for sustained dialogue and support for faculty exploration of EBIPs. Both institutions A and B engaged faculty in faculty learning communities (FLCs). One FLC focused on faculty exploring how instructional choices might better support the student transition from high school to college. Another example involved faculty learning about how to better support students underrepresented in STEM. Institution C supported several faculty communities of practice organized around particular pedagogies (e.g., Team Based Learning (Michaelsen & Sweet, 2011) or Process Oriented Guided Inquiry Learning (Simonson, 2019).

All three institutions offered peer observation programs. At Institutions A and B, these programs included opportunities for faculty to provide constructive feedback to improve their teaching. Institution B created partnerships in which two faculty were connected to another faculty member who functioned as an experienced guide to help them adopt evidence-based teaching practices. At Institution C, a “teaching visit” program allowed faculty to observe a colleague using evidence-based teaching and then to meet to discuss what they learned from the observation. All three institutions also engaged in activities designed to have faculty learn from one another’s examples. Institutions A and C used faculty presentations and interactions at campus-wide teaching symposia as opportunities to increase awareness of EBIPs, engage faculty in discussions about evidence-based teaching, and build community among faculty interested in evidence-based teaching.

Institutions A and B held faculty retreats. Institution A hosted a day-long retreat for faculty who had engaged in the intentional use of formative assessment in their teaching, allowing participants to reflect on their work and learn from each other. Institution B held disciplinary and interdisciplinary retreats in partnership with a local community college during which faculty discussed effective teaching strategies.

Another common tactic was the support of collaborative course redesign projects. This was a prominent focus of Institution C’s project, which supported collaborative teaching projects from every STEM department in which groups of faculty from a department came together to redesign a course or to engage in departmental activities focused on the exploration of evidence-based teaching practice. In the case of Institution B, the above-mentioned retreats were specific opportunities for faculty to create teams that then proposed and enacted course redesigns supported by the project.

In addition to the “in-common” strategies described above, Institution C engaged in some additional tactics unique to their context. Leaders at Institution C facilitated departmental discussions in each STEM department and then followed this by engaging departmental liaisons (one from each STEM department) to foster regular discussions and activities in support of changes in pedagogy. This project also supported the exchange of ideas between these individuals in regular meetings of the liaisons.

Category III—institutionalizing: policies and structures. The tactics adopted by projects in this category reflect change agents’ efforts to provide an environment that supports adoption of evidence-based teaching and, ideally, sustains changes into the future. This includes changes to standard practice and policies, the addition of people and positions to support effective teaching, changes to physical classrooms to remove physical barriers to the use of EBIPs, and faculty recognition. We also included in this category systematic activities undertaken by the projects to assess the institutional context in order to inform future changes.

All three institutions engaged in advocating for and improving classroom spaces to facilitate the use of active learning, especially in large “lecture” halls. Institution B undertook significant efforts to catalog the institutional classroom environment. Institution C engaged a group of faculty to review the strengths and challenges of different classroom environments on campus and added a Center for Teaching Learning staff member involved in the project to the university classroom planning committee. All three institutions engaged in the renovation or upgrading of classroom spaces to help make the built environment more supportive of active learning pedagogies.

All three institutions engaged in changes to policies or standard practice. Institutions A and B worked to inform changes to the end-of-semester course evaluation form. Institution A also advanced discussions with department chairs about effective evaluation of teaching. Institution B implemented a new grading and advising policy that would allow students to stay engaged in a course in which they were struggling. Institution C’s project leaders were involved in changes to the tenure and promotion policy to make an expectation for evidence-based teaching more explicit. Institution C also changed the language in faculty hiring offer letters to make this expectation for teaching explicit.

Institutions B and C intentionally recognized and highlighted faculty practice and exploration of evidence-based teaching, designed to contribute to an environment that values the use of these pedagogical practices. At institution B, this was an award for which each faculty applied or was nominated. At Institution C, a traveling poster-presentation, which was presented in each of the STEM buildings, highlighted work happening in a variety of faculty member’s courses.

Institutions A and C engaged in the systematic collection of data from faculty and/or students through surveys, focus groups, or observations. The results of these efforts informed the projects as they advocated for the project’s goals. Institution A engaged in several studies to understand teaching choices in their local context. For example, the results from a study based on its use of classroom observation protocols were used to inform a more intentional evaluation of teaching for faculty. Institution C implemented an annual teaching climate and teaching practices survey and used the results to assess impact and to inform change leaders’ choices. In both cases, while data collection does not by itself influence change, these efforts were an intentional component of the change efforts over the years of implementation; the results influenced and/or were integrated into tactics undertaken as the projects unfolded over time. Tactics involving the systematic collection of data were placed in Category III because data were collected in a way to inform project leaders’ thinking about the larger institutional landscape, rather than about individual faculty.

As with other categories, each institution also engaged in unique efforts. Institution A engaged in restructuring support for teaching on campus which led to new hires and a new professional development center in support of faculty teaching. Institution B focused efforts on institutionalizing the use of undergraduate peer advisors to support STEM students transferring from community colleges and seeking to major in STEM. This involved creating space for advising discussions to happen and expanding the number of peer advisors engaged in this work across disciplines. Institution C created a “data team” to both provide departments and faculty with data and to signal the value of using data to inform institutional student success efforts. Institution C also extended some of the support opportunities (e.g., for course redesign) beyond STEM disciplines; they identified university funds and used the programs built for the project to extend opportunities for engagement in disciplines beyond STEM.

Category IV—facilitating: meaning-making. The tactics enacted by projects in this category reflect change agents’ efforts to facilitate meaning-making regarding expectations related to the use of EBIPs. The purpose of these efforts was to shift perspectives and norms and build a new, shared understanding of how to most effectively support student learning and success, as well as how to make sustainable institutional change.

Compared with the other categories, there were fewer discrete tactics placed into this category (e.g., unique, independent strategies with emergent outcomes that were specifically aimed at an institutional level). In order to engage in the change process of Category IV, most projects leveraged tactics that were also found in other categories, meaning that a given activity can serve dual purposes. For example, the sharing of results of an assessment effort that might be found in Category III, aimed at understanding the institutional context, also contributed to a shared understanding of what changes are needed to support effective teaching and was therefore also placed in Category IV.

There were also no examples of tactics that were in-common across institutions within this category. Internal efforts focused on meaning-making were more contextually bound, reflecting differences in how each project supported this important work in their institutional environment. For example, Institution A used opportunities for shared processing of the results of their assessment efforts to support the change process of this category; this work helped STEM faculty collectively examine assumptions and fostered openings for change to occur at the institutional level. One example of this scholarly approach involved interviews and dialogue with STEM department chairs about how teaching was evaluated. This effort provided an opportunity for reflection on the part of chairs and provided the project team with insights they used to build shared interest in changes to teaching evaluations. Institution A also used the project to bring DBER faculty together and to build relationships with staff in the newly structured teaching and learning center; their collaborative and complementary work created momentum for the shared goal of changing teaching practice.

At institution B, the use of departmental and interdisciplinary retreats (already described as being in Category II) provided opportunities to build a shared understanding of the value of EBIP adoption. The intentional engagement of academic leaders, noted for Institution B in Category I, allowed the project to foster a sense of department chair ownership for supporting EBIP adoption in the department. This engagement supported chairs to make meaning about the work and contributed to a shared understanding of the project goals. Finally, shared dialogue built around visiting STEM education seminar speaker events, which also functioned to support individual educational development in Category I, contributed to an emergent understanding of new ways to teach.

At institution C, the project leadership facilitated departmental discussions about teaching as the project was launching. These discussions introduced the project’s vision for teaching. Insights from those discussions were used to build shared understanding and to inform strategies of the project. These discussions were carried forward by project leaders and the liaisons embedded in each department as the project unfolded. Each opportunity gave colleagues a chance to share ideas and build a collective understanding and ownership of the work of reforming teaching. Also, at institution C, the faculty involved in collaborative course design projects (Category II) were asked to identify ways the results of their work could be shared with other members of the department.

Overall approaches to change. Each of the three projects studied reported tactics in each of the four Change Strategy categories. While there were similarities across the different contexts, as was discussed in the preceding section, each institution enacted a unique combination of strategies. Change agents in each project appear to have drawn on strengths and responded to needs within their institutional context. The sum of the actions enacted within each institutional change initiative across all categories allows for a general description of the overall approach to change at each institution.

Institution A leveraged DBER scholarship aligned with evolving institutional resources for faculty development. Institution A’s project was led by an interdisciplinary team of DBER faculty who regularly leveraged their scholarly expertise to move the various elements of their project forward. In their annual reports, the project focused heavily on the collection and analysis of data to study what was happening in classrooms at their institution. In Category I, our analysis identified tactics in which the change team leveraged their knowledge of the education research literature to develop and offer professional development workshops on specific topics (e.g., Just-in-time Teaching). Additionally, the change team worked with departments to coordinate resources, training, and the development of materials to disseminate to faculty and administration. In Category II, the change team offered programming to support faculty to adopt evidence-based teaching through collaborative interactions (e.g,. a peer observation program). In Category III, the change team used the findings from the DBER research in which they had studied classroom outcomes for faculty engaged in project activities. They also coordinated/collaborated with two administrative units to gather and disseminate data. By the end of the project, these two units merged into a newly established Center for Teaching and Learning. In Category IV, the change team organized their project around assessment and research, seeking to understand and then influence the institutional landscape of STEM teaching and learning. The team supported the meaning-making of data collected through classroom observations.

Institution B sought to build relationships between and within STEM Departments. Institution B’s change project focused on work with selected departments and units willing to engage in the larger change process (e.g., laboratory courses with willing coordinators, departments wanting to engage in a curriculum redesign, willing community college partners). In Category I, they implemented tactics that supported individuals with specific role-based professional development (e.g., workshops for advisors and TAs in particular departments). In Category II, the change team provided programming to support collaboration within and between departments. This is seen through the interdisciplinary, interdepartmental, and departmental retreats, the faculty focused FLCs, and discussions about department-related grading policies. Further, the project fostered collaboration between Institution A’s engineering college and the local feeder community college with a focus in supporting student transitions into the College of Engineering. This project also leveraged department heads and program coordinators to collaborate. In Category III, the change team further supported the connection between the College of Engineering and local feeder community college through the hiring of Peer Advisors for incoming students from the community college. The change team also worked with Institution B’s faculty senate to influence and change the end of course evaluations for faculty to be more teaching practice focused. In Category IV, we see strategies that engage academic leaders, such as department heads, to foster a shared understanding of project goals. One way this was done was through a Provost-office sponsored speaker series focused on evidence-based teaching that emphasized engagement and dialogue between attendees and the speaker. Faculty were also explicitly asked to engage in meaning-making about evidence-based practice at Institution B as part of departmental and interdepartmental retreats.

Institution C supported departmentally focused change using a Center for Teaching and Learning as a hub. The Institution C change team included institution leaders (the Director of the Center for Teaching and Learning (CTL), Deans of the College of Arts and Sciences and the College of Engineering) and several faculty/staff leaders from STEM departments, including one DBER faculty member. Institution C’s project team organized the tactics of their program using the CTL as a hub. In Category I the project leveraged the Institution C’s CTL to provide topic-specific professional development and to disseminate student outcomes data to faculty to improve outcomes. Professional development was also targeted at departmental liaisons who worked to engage a broader group of faculty than those who would normally engage in CTL programming. In Category II, the CTL provided support for collaborative faculty course redesign projects within departments. The departmental liaisons were also guided to support discussions and strategic planning with their colleagues to increase the use of evidence-based practices. In Category III, the project team worked to influence changes to the tenure and promotion policy revision. In Category IV, the project team, in concert with the CTL, leveraged faculty and departmental relationships to drive the project’s vision, with every STEM department across two colleges participating in the project.

Discussion

To what extent do the activities undertaken by large institutional change initiatives reflect the use of multiple change strategies aimed at impacting the STEM education system? Our results show that institutional projects enact a wide range of tactics spanning the full space of the Four Categories of Change Strategies Model. The categories of the model reflect different approaches, or strategies, to facilitating change in a complex STEM education system (Borrego & Henderson, 2014; Henderson & White, 2019). While there was nothing in the annual reports that suggested any of the projects intentionally considered these categories when choosing their activities, that each project included tactics which spanned all categories suggests that change agents within each project were thinking across categories when seeking to enact change. The current work allows us to build on the Four Categories of Change Strategies Model developed by Henderson et al. (2011) in two important ways.

First, our results provide an expanded view of the types of tactics that can be employed for change beyond those originally used to create the model. The original Four Categories of Change Strategies Model was derived from change efforts found in published journal articles. Studies or programmatic descriptions that find their way into the literature are generally going to be those that are of an appropriate scale and/or have robust findings to make a coherent story for a manuscript. Our analysis of activity found in the gray literature allowed us to expand the change tactics possible in the four categories. The tactics reported by the projects in this study included formal, programmatic approaches that had been described in the literature. For example, communities of practice (Gerhke & Kezar, 2017) and faculty learning communities (Cox, 2001, 2004) are approaches that are well-described in the literature and were employed in these projects. Similarly, all three projects created opportunities for faculty to engage in professional development and documented work to alter teaching spaces; both of these approaches have been documented in the literature. In addition to these well-documented tactics, we observed many activities that were not identified in Henderson et al’s (2011) initial literature search for the model. In Category I, in addition to strategies used to generate the model such as professional development workshops and teaching resources, we observed support for travel to off-campus professional development opportunities, sharing of local research and student outcomes data with faculty for teaching, leadership development for departmental liaisons, and engagement of academic leaders to support buy-in. In Category II, we observed faculty retreats, a traveling poster session, opportunities for faculty to share their teaching with others, collaborative teaching activities like course redesigns, and specific groups of faculty meeting to work in concert with the change initiative. These were in addition to faculty learning communities, communities of practice, and peer observation opportunities that were described in the original model. In Category III, we observed tactics similar to those that contributed to the original model such as institutional policy changes, alterations of physical space, and intentional use of institutional data. In addition, we observed the intentional collection and analysis of data to inform future activities of the project, new positions and hires related to teaching support, implementation of changes to advising, and engagement of department chairs in thinking about effective evaluation of teaching. In Category IV, we observed support for sharing and meaning-making about teaching practices at all levels of the institution, similar to the kinds of work originally described for this category. In addition, in this study, we also observed active building of collaborative relationships to support change, and invitation of speakers from other institutions to engage in discussions about teaching and learning. The analysis in the present study provides a more expansive sampling of tactics that can be situated in the different categories of the Four Categories of Change Strategies Model (Henderson, et al., 2011). Our results do not replace the ideas in the original model, but provide an expanded set of tactics that might be used by institutional change agents.

A second important way the current work builds on the Four Categories of Change Strategies Model is to illustrate the model’s utility beyond disciplinary boundaries. The teams of change agents working on these projects were drawing from ideas that had been documented by higher education researchers, educational developers, and DBER faculty. Many of the additional tactics identified by this work are narrower in scope relative to well-documented approaches and they may never appear in the literature on their own; it likely does not make sense to study these more narrowly focused approaches independently. However, they may play an important role in the overall approach to institutional change in a particular context. An important question for future research is to understand how a collection of strategies and tactics work together to enact change, including which tactics are essential and how more narrowly drawn tactics, which may be very context-specific, interact with bigger strategies to move change forward. Further, this work highlights that people fostering change do so within their local ecosystem and that the local context shapes the tactics selected for a given project. Connecting and triangulating the Four Categories of Change Strategies Model with theoretical framings of how change occurs, such as the Social Ecological Model (Bronfenbrenner, 1979; Leonard, 2011), could help further our understanding of the relationship between chosen activities and the institutional context (Eblen-Zayas et al., 2023).

In addition to an expanded view of the tactics that might be employed by change agents, our analysis also illustrates how the Four Categories of Change Model can serve as a framework representing a collection of approaches that can be undertaken within a single change project. This stands in contrast to the findings upon which the original model was derived in which change activities in different categories tended to be represented by scholars from a particular disciplinary focus. Each of the projects in our study used tactics across all categories, even though the leadership teams had different disciplinary strengths. For example, Institution A’s team had multiple DBER faculty and Institution C’s team was led by an educational developer. We did not see evidence in the reports that any of the projects were thinking about the different strategy categories from the Four Categories of Change Model as they selected tactics. However, it may be the case that the idea that change agents seeking systemic change should draw from multiple categories, as was proposed in the original work, was operating implicitly in the environment of these projects. The Four Categories of Change Strategies Model was available in the broader literature ecosystem that may have informed a systems-level approach to institutional change within each project. More recently, Eblen-Zayas et al. (2023) indicate the Four Categories of Change Strategies Model is helpful for practitioners in choosing change initiative strategies; they also advocate for connecting strategies explicitly to change theory.

How might the strategies and tactics used by these projects inform our understanding of the change processes that operate in the context of institutional change initiatives? In order to understand the change processes operating across these change projects, we began with the same questions that were derived in the original construction of the Four Categories of Change Model, namely “What is the primary aspect of the system the activity seeks to directly impact?” (individuals or environments) and “To what extent is the intended outcome for the individual or environment known in advance” (prescribed or emergent) (Henderson, et al., 2011). Mapping the observed actions onto the four categories and identifying themes in the resulting collections of tactics within each category allowed us to identify an apparent change process operating within each category, reflective of strategies enacted across the three projects in our study. The change processes we identified have important similarities and differences relative to the original model. The change processes identified in our study are presented alongside those from the original work in Table 2.

Table 2 Comparison of the chance processes identified in this work compared to those identified by Henderson et al. (2011)

In Category I, the change process we identified is very similar to the one identified in the original model, both include a focus on providing faculty with the information they need to know in order to adopt a new-to-them teaching practice. What the current work adds is that each project in our study intentionally used strategies to impact individuals in leadership positions (chairs, deans, etc.) and to enlist them in helping to move the project forward with faculty. This addition intentionally works to expand the group of change agents supporting the goals of the project. This reflects the importance of recognizing power relationships (Bolman & Deal, 1991) and the need to engage opinion leaders (Rogers, 2010) in making change. The label for Category I “Encouraging” that emerged from this work reflects that the tactics used were aimed at the colleagues of those acting as change agents and may be particularly useful for change agents working within their own institution, as was the case for these projects.

In the original model, Category II was focused on the development of reflective teachers, with the underlying idea that instructors should “use their own knowledge/experience/skill to improve their instructional practices” (Henderson, et al., 2011 p. 961). In the current study, the strategies in Category II were focused on leveraging peers to influence other faculty in learning about and adopting new teaching practice. Most of the tactics we observed placed faculty in communities in an effort to prompt the sharing of ideas and the creation of mutual support around change. This is an expansion of the original model, as it is the case that within the context of these tactics, the idea of faculty making changes based on reflection can still be present. As for Category II, the label that captured this category, “Fostering: Connections and Community”, reflects a focus of activity within an institution in which faculty relationships are important for creating change. The idea of supporting change in this way is consistent with insights that emerge from social network theory (Lane et al., 2019, 2020; Skvoretz et al., 2023) and social cognition theories for change (Kezar, 2018).

The change process we identified for Category III, to institutionalize policies, structures, and practices consistent with project priorities, is very similar to the ideas in the original model. In this change strategy, the change agent is focused on shifting the context in which instructors make teaching choices in order to support “new behaviors and attitudes that will lead to changes in instruction” (Henderson, et al., 2011, p. 962). The label of this category in the current study, “Institutionalizing: Policies and Structures”, reflects that these projects were grant funded and change agents were working to make the activity of the grant have a lasting impact.

Our identified change process for Category IV, which focuses on meaning-making around changed expectations for teaching, is also similar to the idea expressed in the original model to create opportunities for stakeholders to build a shared vision and work collectively toward the change. One nuance is that in the case of the projects examined in this study, the change projects implicitly or explicitly had a vision and the engagement of others in meaning-making was focused on creating opportunities for people to make sense of that vision and to think about what it might look like in their context and how different stakeholders could move toward it. This is what led to the label for this category: “Facilitating: Meaning Making”. This is in contrast to a scenario in which change agents might create the opportunities for others to co-create a shared vision. While there are tactics that can be adopted that fit squarely in Category IV (Borrego & Henderson, 2014), in the projects we studied, most tactics operating in Category IV had significant overlap with another category. For example, a project might have used a retreat focused on professional development for faculty attendees to learn about evidence-based practice to also do some meaning-making together about those practices and their potential impacts. This is similar to findings from a recent study that specifically focused on understanding how a project can support the development of shared vision; Doten‑Snitker et al. (2021) examined NSF-funded change initiatives focused on department-level change in engineering. They found change teams focused on shared vision through co-orientation, formational communication, and collaboration in concert with moving other strategies forward.

It is important to note that in our analysis of activities in the annual reports, we identified actions utilized by the change teams that did not cleanly map onto the model because they are not actions that intend to influence faculty or the institution directly. Generally, these actions involved communication beyond the boundary of the institution in the form of conference presentations and peer reviewed journal articles. While there is an expectation for publications and/or presentations as part of NSF-funded projects, they also serve as the currency for legitimacy within and beyond one’s institution. By publishing and presenting, the change teams were able to signal that the projects should be perceived as contributing something valuable to the appropriate scholarly community. This likely provided legitimacy for the change team and contributed to colleagues taking the change project seriously. Institution A took a strong research approach; many of their publications provided insights for institutional change with the broader research community. Institution B focused on presentations at conferences and invited talks. Institution C engaged in a mix of presentations at conferences and peer reviewed publications about their project. They also held a professional development summit for other institutions planning change projects.

The expansion of the change processes explicated by this study that can operate in the different categories of the Four Categories of Change Model provides validation of the value of the original model and simultaneously provides additional ideas for how change agents might consider approaches to change. This model is helpful in supporting change agents to think about the distinctions between strategies, and this work underscores the value of thinking about multiple change processes working in concert within a larger project. Our approach affirms the framing of the original model by identifying the relevance of the categories for enacted change initiatives. For change agents working across the full institutional system and using multiple approaches, our work delineates multiple change processes, operating across all quadrants of the Model that can be integrated and then could operate simultaneously. These results also offer an example of how a diverse set of goals within a project, focusing on multiple aspects of an institution, may be helpful in moving change forward (Borrego & Henderson, 2014).

Conclusion

The work presented here sought to provide a descriptive analysis of approaches used to enact institutional change. The research has focused on capturing the multiplicity of tactics and strategies, taken by change teams over time and across institutions. This work fills an important gap in the literature by providing a description of the specific tactics taken in institutional change projects. Understanding what this looks like across multiple contexts helps characterize the multifaceted landscape of large institutional change projects. Further, the extension of the Four Categories of Change Model to describe institutional change projects adds to the utility of the model. Researchers and change leaders/practitioners can use the change processes that emerged from this work to inform and frame the different types of strategies they choose as they design a change project. Further, the work demonstrates that each change process can be engaged in unique ways based on local expertise and context.

An important limitation of this work is that we focused on annual reports for information about project activities. We chose to focus our efforts on this data source because it offered both a rich source of information and allowed us to have the same kind of information across institutions in the study. Future work might expand on this study by adding additional sources of information, such as interviews with change leaders, focus groups with faculty and staff engaged by the project, or evaluator reports. Further, this study focused on three relatively similar institutions; all are public, with a relatively similar student body size, and all have relatively high research expectations for faculty. Future research can expand this approach to other institution types to explore the extent to which the types of tactics and change processes can be generalized to other institutional contexts. Further, all of these projects were focused on the adoption of EBIPs. Projects with a different focus might reveal different strategies and/or different change processes. For example, STEM education reform is appropriately shifting to more explicitly center the needs of students historically excluded from STEM (Asai, 2020). Future work should explore how change processes shift when centered around dismantling institutional barriers for those who have been marginalized and underserved in STEM. Finally, we focused on a rich description and analysis of tactics taken to enact change. An important question not addressed by our study is the relationship of these inputs to the outcomes of the change work. Future work should seek to explore the extent to which working within and across the four categories of change successfully enacts change, especially change that is sustained over time.