Skip to main content

Accelerating STEM education reform: linked communities of practice promote creation of open educational resources and sustainable professional development

Abstract

The preparation of future scientists, the technical workforce, and informed citizens will require continued transformation to the ways we approach STEM teaching and learning. Undergraduate STEM education is rapidly emerging as a focus of faculty scholarship, but new models for reform need to be developed and tested to accelerate changes in teaching practices. This paper describes a flexible, participant-driven, multi-phase, collaborative approach to developing open educational resources (OERs) that leverages linked communities of practice (CoPs). Equally valuable, our framework for development, adaptation, dissemination, and validation of OERs provides a platform for faculty professional development and sustained support through cooperative mentoring. The three linked CoPs in the framework include incubators for the creation of initial OERs, Faculty Mentoring Networks (FMNs) for the implementation and adaptation of OERs for classroom use, and Education Research Communities to assess the effectiveness of the OERs. The CoPs create numerous benefits for participating faculty, including the ability to collaborate in the Scholarship of Teaching and Learning (SoTL) through scholarly publication of OERs and their assessment; ongoing mentorship in implementation of OERs in the classroom; and development of educational leadership skills and experience. Thus, the three CoPs synergize with one another to build and sustain capacity through providing vetted, up-to-date educational resources, as well as ongoing training and support for faculty. While we developed this approach for the rapidly changing field of bioinformatics, the linked CoP framework will have utility for STEM education reform more broadly and disciplines beyond STEM.

Introduction

Long-term reform of undergraduate science, technology, engineering, and mathematics (STEM) education requires building the capacity to support the continued development of both STEM faculty and appropriate learning resources. Here, we argue that this capacity building can be fostered by the development of a community of faculty who collaborate in the creation, implementation, adaptation, and assessment of flexible, high-quality, open educational resources (OERs). Since its inception, the Network for Integration of Bioinformatics into Life Sciences Education (NIBLSE) (Dinsdale et al., 2015; Toby et al., 2022) has promoted the incorporation of bioinformatics into life sciences education by empowering collaboration among peer faculty. As we describe here, our framework of linked communities of practice (CoPs) provides stable infrastructure to promote STEM reform by building high-quality, adaptable learning resources, training faculty to implement them in the classroom, and collaboratively assessing their effectiveness. Although our examples draw from bioinformatics education, these ideas have broad utility across STEM disciplinary domains and potentially into disciplines beyond STEM. Other emerging, interdisciplinary fields, like data science, global health, and equity and justice could benefit from focused work such as the framework provides. While we believe that emerging fields are a natural place for grass-roots development of materials like OERs, established fields could benefit from similar attention, given the emphasis on active learning modalities (Freeman et al., 2014) and interest in increasing students’ quantitative skills (Bialek & Botstein, 2004) that has occurred over the last 20 years. In addition, faculty need opportunities to build new skills and explore new teaching strategies, no matter the discipline.

Communities of practice promote supportive professional growth environments

A community of practice (CoP) is a peer group with shared goals and commitments that interacts and learns together over multiple meetings, collectively documenting their ideas and products (Wenger, 1999, 2011). By uniting professionals with diverse expertise and perspectives in a common endeavor, CoPs promote the synergistic use of complementary skill sets, knowledge, and classroom experiences, while fostering participant accountability.

The use of CoPs in the context of faculty development has several advantages. Importantly, the design of faculty-centric CoPs emphasizes peer interactions and community support, engagement in pedagogical reform, and accountability (Landau & Broz, 2020; Webster-Wright, 2009). In contrast to the single short workshop or more intensive “boot camp” approaches (Feldon et al., 2017; Zhan et al., 2019), CoPs can provide a framework for long-term support and collaborative scholarship (Stewart, 2014). As we will describe, CoPs can support learning resource implementation in the classroom and help generate adaptations for diverse teaching settings and student audiences that are useful to the wider community. CoPs can also be used to bridge the divide between research experts and educational practitioners by purposefully bringing them together into a collaborative community (Robertson et al., 2021).

The linked CoP framework arose from targeting an emerging interdisciplinary field (Bioinformatics) that lacked a rich collection of vetted educational resources and had a need for faculty training (Fig. 1). In more established fields there is also demand for quality, vetted materials to be in the open sphere to allow for faculty to more easily customize student resources. Additionally, many established fields lack a wide range of resources designed with pedagogical best-practices in mind as disciplinary efforts to generate community-reviewed resources based on active learning are in their infancy (Lombardi et al., 2021). Likewise, there is a void in STEM for resources that apply concepts and competencies such as integrating quantitative skills, analysis of primary scientific literature, working with authentic research data, and overt attention to the integration of social justice, equity, diversity, and inclusion within STEM curricula (Diaz Eaton et al., 2022). Using this model to galvanize a community to generate these types of resources can drive education reform. Therefore, our strategy of linking carefully designed CoPs (see the CoP Guiding Principles subsection) provides a mechanism for long-term human capacity building for nurturing and sustaining essential educational resources and developing community intellectual capital through professionalization of faculty teaching practices.

Fig. 1
figure 1

The linked CoP framework for OER development and professional capacity building. Linked CoPs function to create OERs, train and support faculty in their use, and evaluate their educational effectiveness. Synergistic outcomes from the combined CoPs include professional capacity building yielding interconnected faculty communities, OER and adaptations, and validated assessment instruments and datasets. Icon credits: Ideation Cycle by Weltenraser from NounProject, Bundle by Siipkan Creative from NounProject, and Assessment by Justin Blake from NounProject

Open educational resources (OERs) promote accessibility and instructional flexibility

Freely accessible, adaptable open educational resources (OERs) have become increasingly popular in the academic community over the past decade (Bozkurt et al., 2019; Spilovoy et al., 2020), particularly as initiatives at colleges and universities have raised awareness of OERs, created incentives for their production, and bolstered their adoption (Spilovoy et al., 2020). Defined as “teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use and re-purposing by others, [OERs can] include full courses, course materials, modules, textbooks, streaming videos, tests [or other assessments], software, and any other tools, materials or techniques used to support access to knowledge” (Spilovoy et al., 2020).

While addressing barriers of affordability and accessibility, OERs also provide an exceptional opportunity for keeping learning resources current and adapting them to varied classroom contexts (Atkins et al., 2007; Nipa & Kermanshachi, 2020). The customizable nature of OERs is especially advantageous for emerging disciplines that change rapidly as they become better defined and integrated into diverse educational contexts with distinct learning goals and student audiences. Freed from copyright restrictions, OERs can be dynamically updated in a public manner in response to new findings, processes, and tools. In addition, OERs are customizable and allow instructors to tailor OERs to specific classroom learning objectives or local projects that promote student engagement.

However, OER production and utilization comes with a unique set of challenges (Hodgkinson-Williams, 2010). In particular, quality assurance mechanisms often vary, in some cases relying on pride-of-authorship or author expertise rather than peer review or other external vetting (Hodgkinson-Williams, 2010). In addition, while OER repositories provide a powerful mechanism for dissemination, OERs are difficult to keep current, particularly in rapidly changing fields. A related challenge is that many OERs lack evidence of effectiveness for student learning, especially across varied classroom contexts. Furthermore, the integration of professional development with OER requires continued attention, as instructors often have limited experience developing or adapting OERs and/or may require additional training and support for OER implementation, particularly in emerging disciplines. Our linked CoP approach can alleviate these issues associated with OERs.

Leveraging CoPs to foster OER development

To address these common OER challenges, CoPs can function as structured communities to collaboratively create, reuse, revise, and remix OERs and to share implementation and pedagogical strategies (Tosato & Bodi, 2011). CoPs can assist faculty in transforming a private draft resource into a publishable OER aligned to learning outcomes and established disciplinary competencies, while providing community vetting of the product (Ryder et al., 2020). In addition, CoPs provide the means for faculty to learn about contemporary breakthroughs within their discipline and pedagogical best practices, while adapting a resource so that it can be implemented in a specific context (Addis et al., 2013; Sirum & Madigan, 2010). By providing expert support during implementation, CoPs can also function as meaningful professional development models for faculty and answer calls for instructor training (Emery et al., 2021; Holmberg et al., 2021). CoPs can also structure the development and validation of assessment instruments, the collection of cross-institutional data, and the dissemination of assessment findings (Kleinschmit et al., 2021). These more rigorous and generalizable scholarship products that typically fall within the realm of discipline-based education research (DBER) (National Research Council, 2012) facilitate the development of vetted resources as publishable products of scholarship that enhance not only the individual faculty member’s reputation, but foster engagement by faculty, curricular administrative units, and institutions in the work of STEM education reform (Austin, 2011; Fairweather, 2008).

Leveraging CoPs to foster professional capacity building

As we describe below, our framework of linked CoPs promotes capacity building for a professional community that aims to develop faculty self-efficacy for participating in reform practices (Eade, 1997). Our CoP framework is distinctive because its human capacity building elements strengthen intellectual capital through expanded knowledge and leadership. Not only do faculty create and adapt OER, but immerse themselves in collaboration, professional development, OER implementation, the application of pedagogical strategies, and scholarly teaching. Whether the focus is on generating curricular innovations, classroom implementation, systematic assessment, or the use of evidence to drive classroom practices, intentionally designed CoPs (see below) can build capacity for faculty members to incorporate elements of the Scholarship of Teaching and Learning (SoTL) (Felten, 2013) into their regular practices, while also formally sharing their knowledge and discoveries with colleagues. Collectively, this linked CoP framework provides a space for faculty growth and collaborative innovation.

CoP guiding principles

An effective CoP requires explicit attention to the structure and shared practices within the group. This is particularly important when the group is working online, as the remote interactions may cause individuals to drop out or feel marginalized. In creating and field testing the CoPs we describe below, it became apparent that the following guiding principles (Table 1) were integral for fostering a productive and enriching experience. Derived from Conole's (2012) and Hegarty's (2015) eight attributes of open pedagogy, as well as from individual and team competencies described by the Science of Team Science (SciTS) (Lotrecchiano et al., 2021; Stokols et al., 2008), these principles prioritize coordinating participation and contributions around the shared goals of the group. They emphasize the importance of considering multiple perspectives and empowering all participants to share their ideas and experiences within the community. Collectively, these principles support CoP participants in collaboratively exploring features of teaching and learning that draw on the varied experiences of educators and can be applied to multiple learning settings and student audiences.

Table 1 CoP guiding principles that promote bottom-up, faculty-driven STEM education reform*

To foster a faculty-driven approach to transform STEM teaching practices, new modes of collaboration are necessary (Lieberman & Pointer Mace, 2010). In particular, a cyberinfrastructure that facilitates the social infrastructure promoted by these principles is needed to sustain an online professional workspace for faculty at different institutions. Existence of this dual infrastructure promotes collaboration, trust, accountability, and evolution of OERs over time. OER repositories need to be more than a platform for static OER storage; they should provide a support structure for searching, sharing, reusing, and collaborating (Atenas & Havemann, 2014). Faculty that are part of a community are better positioned to fully achieve the benefits of OERs (Booth & Kellogg, 2015; Kezar et al., 2018). Furthermore, the availability of social support for intellectual efforts promotes faculty success and builds positive experiences that can support future work. By seeing a project through implementation and sharing back to the community, faculty participants are more likely to sustain the use of the curriculum. Cyber infrastructure can also provide a public landing page describing existing CoPs. This provides an avenue for educators outside of the CoP to view public portions of active work and provides a conduit for an influx of faculty to the CoP or through future iterations.

Thus, the CoP guiding principles and associated cyberinfrastructure set the stage for sustainable STEM education reform, by fostering a community that can contribute to future changes in and outside of the classroom. This model for STEM education reform leverages emerging cyberinfrastructure to support faculty participation in SoTL. We built on, and extended, the QUBES project’s Faculty Mentoring Network (FMN) model (Donovan et al., 2015). The primary contribution of the FMN model is to foster creative ways to leverage online collaboration and OER publishing to accelerate STEM education reform. FMNs have been implemented over 50 times during the last five years, engaging more than 500 faculty (https://qubeshub.org/community/fmns/browse). FMN outcomes have included providing tools and support for faculty interested in developing data-driven curriculum, as well as generating publically available OERs that have been implemented and tested in the classroom (Akman et al., 2020; Bonner, 2017; Naithani et al., 2022). FMNs were also used to help faculty adapt to online teaching (Stack Whitney et al., 2022). Building on the FMN model, the linked CoP framework (Fig. 1, described in detail below) can be reiterated indefinitely with previous participants taking a leadership role and mentoring additional faculty members, resulting in a sustainable, yet dynamic, community.

A framework for linked CoPs to augment capacity building

Members of NIBLSE pioneered the linked CoP framework we describe here. The framework is a series of intentionally designed CoPs that function to address barriers to ongoing educational reform. The framework addresses barriers (1) to the transformation of private resources into vetted OER, (2) to instructor adoption, adaptation, and classroom implementation of OERs, and (3) to formally assessing learning resources to determine OER effectiveness (Fig. 1, left). We collaborated with QUBES (Quantitative Undergraduate Biology Education in STEM) (Donovan et al., 2015), which provided both cyber and social infrastructure essential for CoP functioning, as well as a flexible OER repository (Fig. 1, right). The linked CoPs provide a ‘pathway to publication’, by providing a way of turning rough classroom materials to OERs vetted and improved by peers and posted on the QUBES site with a DOI (Incubators), and implemented by others (FMNs), making subsequent formal peer-reviewed publication of the OER much more likely. Furthermore, the focus of the Education Research CoP is to publish a validated assessment instrument and associated dataset. Thus, a major focus of the framework is to provide support for SoTL and education research with tangible scholarly output for academics. No monetary stipend was provided for participation in any CoP, rather the value to participants came from the development of a social network of like-minded individuals with common goals. Here we describe the general features of the linked CoPs, as well as a specific example of a bioinformatics OER that was developed, adapted for diverse teaching settings and student audiences, and assessed using the framework.

CoP #1: resource incubator—OER enhancement, refinement, and quality assurance

Resource incubator (CoP #1) general features

Resource incubators serve a dual purpose: they provide a structure for development of current, relevant, and high-quality OERs, as well as support for the scholarship required to create and implement these resources (Ryder et al., 2020). Each Resource incubator is a short-term (4–8 weeks) online CoP that refines private draft teaching materials into a publishable OER that is ready for adoption or adaptation by other instructors. Incubator participants are recruited from the associated professional society membership (in this case, NIBLSE), as well as from interested colleagues and students. Incubators are intentionally designed to include a variety of participants (e.g., the resource author(s), novice instructors, content experts, assessment experts, and a managing editor), from academic institutions that vary widely in size and location, to ensure that the developed resource is accurate, engaging, and accessible, and that the final product is adaptable to many classroom contexts. The OER is mapped to standards established by disciplinary leaders, such as curriculum guidelines, core concepts, or core competencies. Incubator participants agree on areas where improvement is needed in the draft OER. The managing editor sets the agenda for meetings, ensures that tasks are equitably distributed and completed, and keeps the incubator moving forward. The incubator serves not only to develop the OER, but to encourage its implementation and initial assessment in the classrooms of the participants. Upon completion, the incubator publishes the OER in an OER repository for broader dissemination.

Joining an incubator can serve to engage participants more fully with the greater professional community. Participants are rewarded through the creation of a tangible, publicly available OER that can be referenced as part of their scholarly output, as well as through creation of a supportive network of peers who help to further their scholarship and teaching goals.

Resource incubator (CoP #1) exemplar

The Sequence Similarity Resource Incubator refined a learning resource that had been used by the resource authors in their own classrooms for several years, but had not been widely disseminated. Foundational to this bioinformatics exemplar is a set of bioinformatics core competencies for life scientists (Wilson Sayres et al., 2018). Following submission to the NIBLSE Resource Review Committee (RRC), the RRC managing editor and a second NIBLSE community member external to the RRC reviewed the resource. This initial screen was to ensure that its content was germane to bioinformatics education and to provide initial recommendations for avenues of resource improvement during the incubator (e.g., assessment, alignment to learning outcomes, content, and structure). Following an initial meeting between the RRC managing editor and the resource authors to discuss the reviewer recommendations and establish incubator goals, a content expert, two experienced instructors new to bioinformatics, an assessment expert, and a QUBES support liaison joined the incubator team. The incubator participants met four times over eight weeks to map the resource to the NIBLSE Core Competencies (Wilson Sayres et al., 2018) and to refine the resource based on suggestions made by the external NIBLSE reviewer and incubator participants. Following completion of the incubator process, the revised OER was published in the NIBLSE Resource Collection housed on QUBES (https://qubeshub.org/community/groups/niblse/resourcecollection) and ultimately published in a peer-reviewed journal (Kleinschmit et al., 2019b). The incubator also provided an opportunity for colleagues seeking to increase their bioinformatics knowledge to then use the resource in their own classrooms. In addition, the incubator laid the groundwork for formal evaluation of OER efficacy by providing an opportunity to draft an assessment instrument and to identify a cohort of interested faculty from among those who implemented the OER and could further refine the assessment instrument in a subsequent CoP.

CoP #2: faculty mentoring networks—enabling supportive adoption and adaptation of OER

FMN (CoP #2) general features

To further adapt and disseminate a resource, an incubator can be followed by a Faculty Mentoring Network (FMN) (Bonner, 2017; Donovan et al., 2015), a community of educators with a shared interest in adapting and implementing a defined curricular resource. While the FMN framework is adaptable to a wide range of use cases (https://qubeshub.org/community/fmns), here we describe their use as a means to support faculty members seeking to implement and adapt a previously incubated OER. Inclusion of FMNs into the CoP framework addresses some of the major barriers to implementing contemporary learning activities in the classroom, including lack of faculty training, lack of access to vetted curricular materials, and lack of time to restructure the curriculum (Børte et al., 2020; Kim et al., 2019; Miller & Metz, 2014; Williams et al., 2019).

Each FMN is initiated and run by the resource author and one or more incubator (CoP #1) participants transitioning to a leadership role (Kleinschmit et al., 2021). FMN participants are typically non-experts who are recruited via open solicitations in professional society newsletters, as well as other STEM education community venues. In a typical structure, FMNs meet on a biweekly basis over the course of an academic semester with topical asynchronous forum discussions between meetings. Knowing the constraints that teaching faculty face, the meeting schedules are tailored to participant schedules, with the asynchronous forums providing flexibility and continued support from the FMN mentors when attendance at the synchronous meetings is not possible. Each meeting has a topic and assigned tasks that are organized by the FMN facilitators, to help participants incrementally achieve their end goals of resource adaptation, implementation, and assessment. The participants engage in discussion focused on adaptation, implementation, and pedagogy throughout the CoP. After implementation, participants are encouraged to share their classroom experience, pedagogical practices, and their assessment results. This structure provides the just-in-time training (Novak, 2011) needed to support an instructor’s adaptation and implementation of a given resource in their classroom. The adaptations are authored by the participants after aligning components of the resource to their class-specific learning goals. The resulting adaptations are then made widely available by publication through an OER repository platform. By providing an experience that addresses a complete OER product cycle of finding, adopting, adapting, implementing, and sharing (Beaven, 2018; Santally, 2011), participating in an FMN orients faculty to effective use of and professional investment in OERs.

FMNs also function as a ready-made community for mentoring faculty in how to monitor and evaluate their own classroom teaching and learning. Mentoring can encourage faculty to plan the means by which they will collect student learning data (Drew et al., 2021), perform its analysis, and formally report lessons learned to the CoP. Mentors support data-driven classroom teaching decisions by providing an assessment instrument, drafted in the Resource Incubator (CoP #1), as a tool to measure student learning gains. The resulting assessment data further support the long-term iterative development and evaluation of the assessment tool.

Participants join FMNs to enhance student learning, to gain experience in applying tools/concepts in an unfamiliar field, to network with like-minded educators, and to further their professional development. The added value resulting from FMN participation includes the publication of adapted OER materials and growth opportunities where, as aforementioned, FMN participants can themselves become leaders in future FMNs, a “train the trainer” model (Pfund et al., 2015; Via et al., 2019). This intentional scaffolding thus creates a path for building leadership capacity. Repeating an FMN also widens community participation and generates additional adaptation products.

FMN (CoP #2) exemplar

In establishing a NIBLSE-sponsored FMN we identified 9 target objectives (Additional file 1: Table S1), some of which are widely used across QUBES FMNs and others tied to our specific goals. The NIBLSE FMNs were created to provide faculty with immediate and continued support for the effective incorporation of bioinformatics into their varied classrooms. These objectives were designed to address barriers that frequently deter faculty from incorporating bioinformatics into their curriculum, including a lack of bioinformatics training/knowledge, lack of a support network, inadequate time for resource development, and lack of vetted curriculum (Williams et al., 2019). FMN objectives included providing vetted OERs that were classroom-ready, support for the implementation and assessment of the resources, and mentorship needed to adapt the supplied resources to create new resources targeted to each participants’ course and student learning objectives.

NIBLSE has twice offered semester-long FMNs for the implementation and adaptation of the Sequence Similarity OER; the second offering included participants from the first offering taking on leadership roles as the Faculty Mentors. In both FMNs, participants implemented a collection of bioinformatics learning modules in a diverse array of courses, including AP Biology, Introductory Biology, Introductory Genetics, Conservation Genetics, Developmental Biology, Disease Ecology, Plant and Fungal Biology, Virology, and Bioinformatics. FMN participants were encouraged to publish their newly created scholarly resource adaptations as an OER on the QUBES platform, after an informal community peer-review process. Published products included eight adaptations with a variety of alterations, including changing the examples to fit the content-focus of the course, streamlining the modules for a more introductory audience, and use of an alternative computational algorithm to demonstrate the alignment concepts (Additional file 1: Table S2).

To assess the value of the FMN experience to the participants, a survey was conducted after each FMN. While the number of participants who completed the survey was relatively small (n = 12 of the 21 participants), the data showed that both FMNs were successful in establishing a supportive and collaborative community. All 12 FMN respondents viewed their participation as a positive professional development experience, resulting in increased confidence in teaching bioinformatics and creating module adaptations. Further, the community formed by the peer mentors leading the FMN created a comfortable learning environment and fostered beneficial feedback from both the leading peer mentors and fellow peer FMN participants. Indeed, all of the participants planned to use their module adaptations in the future, indicating the significant benefit to the time spent as a participant of the FMN. Given the positive feedback, it is not surprising that all respondents would recommend a NIBLSE FMN to colleagues who wish to incorporate bioinformatics into their own courses. Thus, an FMN is an effective method to bring together a group of educators with different levels of proficiency in bioinformatics, diverse expertise and interests in biology (e.g., cellular, organismal), and a range of experience (early career to established faculty). Given that half of the respondents believed that the pedagogical scholarship promoted by FMNs would be valued by their institution, these CoPs play an important role in fostering a supportive community of like-minded colleagues. The resulting community facilitates removal of barriers inhibiting the integration of bioinformatics in the classroom and generates novel learning resource adaptations to make resources that are more approachable to educators.

In summary, participation in an FMN provides a supportive venue for faculty training, for publication of scholarly adaptations, and for open access to these materials for other educators. The impact of the NIBLSE learning resource adaptations can be inferred from the over 8,300 downloads (Additional file 1: Table S2).

CoP #3: education research community—evaluation of OER effectiveness

Education research community (CoP #3) general features

The Education Research Community (CoP #3) brings together a cohort of faculty with diverse expertise to collaboratively study learning gains from OER interventions and to refine instrument validity and reliability (Kleinschmit et al., 2021). The participants of this CoP can be drawn from alumni of the previous Resource Incubator (CoP #1), FMNs (CoP #2), and external assessment experts. At least some of the participating faculty should have experience implementing the resource in their classrooms. To make broad generalizations on classroom effectiveness, it is helpful if the group consists of faculty from varied institutions who collectively have experience in implementing the OER in different types of courses. Recruitment is initiated by reaching out to previous CoP participants and encouraging faculty implementing the OER to join the CoP and collect data at the same time. Leadership of this CoP may be the author of the OER in combination with a previous participant taking on a leadership role, through use of the train-the-trainer model (Pfund et al., 2015). The CoP leadership establishes an agenda for meetings, including goals and action items for the group, throughout the duration of the CoP. This CoP analyzes assessment data from the incubator (CoP #1) and the FMN (CoP #2), while simultaneously improving the quality of both the OER and the assessment instrument through iterative discussion and revision. The disseminated assessment data also increase the value of the learning resource for other educators, to help them to decide upon the utility of adopting or adapting the resource for their classroom. The Education Research Community (CoP #3) also has the capacity to produce further CoP iterations for groups that decide to pursue their own education research questions involving the OER, such as in which classes or instructional contexts the OER is most useful or for investigating how the resource can facilitate learning gains. Such collective engagement can build a supportive group of colleagues, with a shared vision to strengthen both instruction and assessment (Anderson & Krathwohl, 2001).

Education research community (CoP #3) exemplar

NIBLSE utilized the Education Research Community (CoP #3) across two academic semesters to refine an assessment instrument and scale up data collection (Kleinschmit et al., 2021). The CoP began by analyzing assessment data from the Resource Incubator (CoP #1) and the FMN (CoP #2). This analysis allowed for iterative data-driven refinements of the instrument. Next, another round of data was collected and analyzed. Dataset analysis included validity and reliability measures (Campbell & Nehm, 2013; Carmines & Zeller, 1979) along with more qualitative implementation and debrief discussions to enrich the interpretation of both the validity and utility of the assessment (Adcock & Collier, 2001). The group used the evolving analysis to determine whether the instrument needed to be further refined or if data collection could perhaps be expanded. The iterative nature of the Education Research Community further provided networking among the CoP members, resulting in subsequent scaled-up data collection across multiple semesters and multiple institutions. This in turn increased statistical power and generalizability of the analysis, which was able to show learning gains for students using the OER in a variety of course subjects and institutions (Kleinschmit et al., 2021).

Participation in and timeline for linked CoPs

These linked CoPs contributed to an expanded OER collection comprising the original OER resource, adaptations, a valid and reliable assessment instrument, recommended usage instructions, and publicly available assessment data (Kleinschmit et al., 2019a, b). Participants can move from a contributing role in one incubator, to a leadership role as author or managing editor of another incubator, and continue on to lead an FMN or Education Research Community based on a developed OER. Thus, the linked CoPs further the overall goals of both developing and disseminating OERs and nurturing the professional careers of the people who create them. For our bioinformatics example, six faculty participated in two or more CoPs, with one participant from the Resource incubator becoming a mentor in the first FMN CoP offering and two participants in the initial FMN becoming mentors in a subsequent round. The framework described here for development of the linked set of resources (the original OER, adaptations, and assessment tools) takes 1–2 years, where 2–4 months is devoted to the peer-review process within the incubator, 4–6 months to the FMN for faculty to learn the OER and then develop adaptations, and 12–15 months to develop and test the assessment instruments after implementation of the resource within classrooms.

Linked CoPs produce synergistic outcomes

While each CoP applies the guiding principles to prioritize a specific objective, the multiple outcomes of the framework benefit from the synergies achieved by linking the CoPs (Table 2). Below we describe how the linked CoP framework enhanced the final outcomes of OERs and adaptations, validated assessments, and interconnected and sustained faculty communities.

Table 2 The synergistic outcomes of linked CoPs exceed individual outcomes

CoP synergistic outcome #1: interconnected faculty communities

Professional development experiences that remove barriers to pedagogical change can impact learning in the classroom by promoting a change in faculty instructional practices. Evidence suggests that effective strategies for lasting impacts on student learning and the effectiveness of various teaching approaches in the classroom involve long-term interactions (a semester or longer) and buy-in from the bottom-up through collaborative learning communities that discuss pedagogical best practices, implementation concerns, assessment, and the classroom experience (Henderson et al., 2011; Sunal et al., 2001; Turpen et al., 2016). By creating safe and collegial professional spaces where diverse viewpoints are valued, CoPs encourage faculty to engage with colleagues, to use their expertise and experience, and to potentially reach outside of their own expertise and experiences, thereby promoting strategies proven effective for long-term community engagement (Aster et al., 2021).

Each individual CoP in the framework has a deliberate and purposeful structure to create a culture of community and collaboration. Biweekly meetings, established group goals, and upfront CoP agreements (e.g., expectations, commitment details, proposed products) all contribute to this structure. Formal completion certifications can also provide an incentive to follow through with the agreement. While composed of faculty with shared interests and goals, each CoP is typically diverse in teaching experiences and levels of expertise in the discipline, which enriches the depth of topic discussions and enhances the assistance with OER implementation (Booth & Kellogg, 2015). Additionally, the development of interpersonal relationships yields group social capital, which is essential for establishing a reliable and resourceful community and reducing professional isolation (Lieberman & Pointer Mace, 2010; Tseng & Kuo, 2014).

CoP synergistic outcome #2: OER and adaptations

One of the overarching objectives of the framework is the generation of high-quality, accessible OERs. To address challenges in OER quality and accessibility (Clements & Pawlowski, 2012; Jimes et al., 2021), the framework integrates multiple OER quality assurance mechanisms (Hodgkinson-Williams, 2010). First, each CoP has internal peer-review mechanisms to validate OERs along with mechanisms for external review by users and the educator community. In addition, Resource Incubators (CoP #1) incorporate vetting of OERs by subject area authorities, including alignment of learning outcomes to an established learning framework. Furthermore, the novice perspective within the incubator is invaluable for identifying hidden assumptions and places for additional clarity within the resource. Finally, resource adaptations created during FMNs (CoP #2) help to ensure that OERs are updated and remain relevant and accurate.

Each CoP also integrates scholarship in the form of curricular innovations and/or education research in a well-documented fashion to motivate participating faculty. The collaborative scholarship products from the incubator (CoP #1) and the FMNs (CoP #2) focus on curricular innovations, their implementation, systematic collection of student data, and analytical discourse within the respective CoP. The Education Research Community (CoP #3) then builds on these scholarship products by focusing on formal evaluation within the contexts of assessment validity and reliability. In particular, the Education Research Community (CoP #3) studies the efficacy of the OER, working with the community to perform education research on how the OER promotes learning and to support other learning-related questions. Collectively, the linked CoPs concentrate intellectual capital to catalyze collaborative generation of OER products by completing the OER product cycle.

CoP synergistic outcome #3: validated assessment instruments and datasets

Assessment instrument development is an iterative process that requires (1) generating assessment items aligned to learning outcomes, (2) gathering pilot data for each assessment item, and (3) analyzing and using the pilot data to improve individual items and the collective set. This process to establish a valid and reliable instrument is typically time intensive and requires a longitudinal commitment to iterative instrument development (Anderson & Krathwohl, 2001). We found that the framework catalyzes the development of robust, valid, and reliable assessment instruments and a collection of large, robust datasets across diverse implementation contexts (Kleinschmit et al., 2021).

Incubators (CoP #1) establish clear learning objectives and map assessment items to instructional content and overall learning objectives. This directed process guides participants in establishing a draft instrument that can be used to evaluate the degree to which students have satisfied learning objectives at the completion of the activity. The format of this draft is typically open response with directions to provide rationale to support responses. These student responses provide a rich dataset of student misconceptions that can then be used for generating distractors for closed-response style questions to create a version of the instrument useful for scaled-up data collection. Carefully scoring student open-response data is also helpful for establishing overall face validity for the instrument items (Taherdoost, 2016). FMNs (CoP #2) then provide a platform for recruiting interested faculty in piloting the transformed closed-response assessment items as a pre/post learning gains instrument, as well as providing content validation. Education Research Communities (CoP #3) uses the resulting data for item analysis and instrument refinement. Formal assessment data collected across a variety of implementation contexts provides efficacy evidence for learning gains associated with the implementation of the OER and its adaptations. The development of a formal instrument to assess learning gains and scaled-up assessment data collection encourages sustainable, long-term updating and implementation of the OER, as well as community interactions and exchanges; these outcomes would not be as feasible in a single CoP.

The integration of assessment instrument development in each CoP within the framework functions at two levels: in the immediate CoP, use of the assessment instrument by individual faculty provides a data stream to assist faculty with systematic inquiry of student learning. Over the long term, leadership across the linked CoPs can use aggregated data for instrument development to provide a valid and reliable instrument for more formal education research questions. Furthermore, assessment data across institutions are useful for evaluating overall OER effectiveness in diverse classroom contexts and can be linked back to the OER for formative improvements.

The linked CoP framework as a faculty-driven intervention to promote scholarly teaching practices

Linked CoPs as a tool for reform of teaching practices in undergraduate STEM education

Scientists and educators recognize that changes are needed for the future of STEM instruction and training to best prepare the next generation of scientists (Alberts, 2022; Asai et al., 2022). In particular, the adoption of active learning practices in STEM classrooms has been demonstrated to improve student learning in general (Freeman et al., 2014) and the outcomes for underrepresented minorities in particular (Austin, 2011; Fairweather, 2008; Theobald et al., 2020). Although faculty awareness of the impact active learning has within the classroom has broadened, there is not widespread adoption of these classroom best practices (Laursen et al., 2019; Stains et al., 2018), which leaves much room for reform efforts.

By empowering faculty to change classroom practices through access to structured professional development (i.e., training, knowledge, skills, leadership), the linked CoP framework can assist with building capacity for reform of teaching practices in undergraduate STEM education. By promoting long-term discussions and evaluation of individual classroom practices in a supportive community, CoPs have been shown to influence faculty attitudes and behaviors and ultimately transform the learning environment (Kezar et al., 2018). The ability of CoPs to reform and transform STEM education practices is perhaps most strikingly demonstrated in “Communities of Transformation (CoTs)”, long-lasting STEM education CoPs that “create and foster innovative spaces that envision and embody a new paradigm of practice” (Kezar et al., 2018), such as BioQUEST, Project Kaleidoscope (PKAL), the Process-Oriented Guided Inquiry Learning (POGIL) Project, and Science Education for New Civic Engagements and Responsibilities (SENCER).

The linked CoP structure promotes educational scholarship

Ernest Boyer’s landmark model of an expanded view of scholarship beyond discovery research (Boyer, 1990) ignited contemporary interest in education research, and multiple studies (reviewed by O’Meara, 2010) indicate that this model has been increasingly institutionalized as part of faculty evaluation and reward systems across institutions of higher education. The Scholarship of Teaching and Learning (SoTL) is rooted in the scholarly inquiry of student learning with the use of sound methodology and public discourse, often within an individual course and specific context (Felten, 2013). For example, SoTL is most frequently associated with descriptive individual classroom study, such as with a unit-level curriculum innovation or intervention specific to a course. In contrast, discipline-based education research (DBER) (National Research Council, 2012) studies are often grounded by a theoretical framework, associated with a particular discipline, and when not purely evaluative, typically move beyond a single classroom to yield more generalizable knowledge (Molinaro et al., 2020). Scholarship expectations vary depending on faculty appointment, but both SoTL curriculum products and efficacy-based education research studies are increasingly embraced by departments centered on specific disciplines (Dolan et al., 2018).

By fostering valued and impactful forms of scholarship (Marcketti & Freeman, 2016; Marcketti et al., 2015), the infusion of educational scholarship and research practices into instruction further enhances the value of CoPs. In particular, the CoP framework provides a structured opportunity to engage faculty participants in the pursuit of educational scholarship in collaboration with experienced peer faculty. By collecting assessment data to determine the efficacy of an OER and its variations, CoPs incorporate SoTL and DBER practices. Furthermore, carefully articulated approaches to student learning using strategies supporting SoTL also provides a more focused, researchable intervention for STEM programs seeking to enhance student performance and success (Freeman et al., 2007). Collectively, we found that documented scholarship opportunities naturally build across the linked CoP framework, provide a collaborative goal to work toward, and are increasingly valued by faculty as the shared work progresses.

Interconnected CoPs build capacity for sustained communities

Long-term professional communities can foster sustained collaboration, enhanced networking, and robust leadership (Lieberman & Miller, 2008; Stoll et al., 2006). The linked CoP framework structure encourages faculty to become engaged community members, yet the modular nature of each CoP with distinct compositions and objectives gives faculty flexibility to join and leave based on their interests and availability. Overall, we found that the linked framework of CoPs promotes engagement of faculty across multiple CoPs for long-term collaboration.

In addition, the linked CoPs provide a path for building leadership capacity. In this integrated model, CoP mentors model the use of structural elements, virtual tools, and approaches for engaging participants and developing community to prepare participant mentees for future CoP leadership roles. In our experience, it is useful for one co-mentor to have previous experience leading a CoP, with the second co-mentor eventually transitioning to the leadership role. The transition of leadership between CoPs may be associated with iterations of FMNs (CoP #2) or Education Research Communities (CoP #3) (Fig. 1), or with a framework of CoPs #1–3 focused on a different OER. By scaffolding leadership, where participants become leaders of future CoPs, a stable framework for linked CoPs and continued OER development is facilitated.

Concluding remarks

Here we describe a three-part framework for developing OER from peer-reviewed inception, an incubator (CoP #1; Ryder et al., 2020); to training of participant faculty on the OER through an FMN as well as coaching the participants in development of adaptations of the original OER (CoP #2; https://qubeshub.org/community/fmns); and finally to developing collaboratively vetted assessment instruments and datasets to evaluate the utility of the OER in an Education Research Community (CoP #3; Kleinschmit et al., 2021). This CoP framework is unique for capacity building beyond creation of OER, namely by enriching and developing human capital through knowledge acquisition, leadership, and the promotion of faculty-driven STEM education reform.

The linked CoP framework provides a structured process to facilitate the integration and assessment of lessons and learning activities into curricula and addresses major barriers to implementing new concepts and/or competencies in the classroom (Børte et al., 2020; Kim et al., 2019; Miller & Metz, 2014; Williams et al., 2019), including access to well vetted and adaptable materials as well as faculty training in a supportive community, while harnessing community intellectual capital for capacity building. More broadly, this model could be easily adapted to other academic disciplines and subdisciplines within or beyond STEM. This is especially relevant to emerging or rapidly evolving fields or disciplines looking to encourage instructors to redesign curricula around an established set of key concepts, competencies, and associated learning outcomes using appropriately targeted resources (Brownell et al., 2014; Clemmons et al., 2020).

Availability of data and materials

The dataset used and analyzed during the current study is available from the corresponding author.

Abbreviations

CoP:

Community of practice

DBER:

Discipline-based education research

FMN:

Faculty Mentoring Network

NIBLSE:

Network for Integrating Bioinformatics into Life Sciences Education

OER:

Open educational resource

QUBES:

Quantitative Undergraduate Biology Education and Synthesis Project

SoTL:

Scholarship of Teaching and Learning

STEM:

Science Technology Engineering and Mathematics

References

Download references

Acknowledgements

We would like to thank Hayley Orndorf (University of Pittsburgh) and Deb Rook (QUBES) as they were instrumental in coordinating the logistics and streamlining the QUBES FMN.

Funding

Research support from National Science Foundation Grant #1539900 awarded to William Tapprich (University of Nebraska at Omaha).

Author information

Authors and Affiliations

Authors

Contributions

All authors collaborated on the writing of this commentary to the extent that it represents a synthesis of each author’s ideas. All authors contributed to refining the model and editing the manuscript. Each author read and approved the final manuscript.

Corresponding author

Correspondence to Adam J. Kleinschmit.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Table S1. Objectives of the CoP #2—Faculty Mentoring Network (FMN). Table S2. OER adaptations of Kleinschmit et al., (2019a, 2019b) published on QUBES OER Repository.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kleinschmit, A.J., Rosenwald, A., Ryder, E.F. et al. Accelerating STEM education reform: linked communities of practice promote creation of open educational resources and sustainable professional development. IJ STEM Ed 10, 16 (2023). https://doi.org/10.1186/s40594-023-00405-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40594-023-00405-y

Keywords