Results from the analysis are reported in four sections: (1) the repositories accessed for teaching-related information during planning; (2) the specific retrieval functions that faculty use to access these repositories; (3) patterns in how these memory and retrieval functions operate in practice (i.e., decision chains and implicated contextual factors); and, (4) the degree to which the USE project influenced these memory and/or retrieval functions.
The information repositories faculty drew upon when planning
First, we report the types of repositories where information pertaining to curricular and instructional issues were stored. The locations and the frequency with which they were referenced are depicted in Figure 1, below.
These data demonstrate the diversified forms in which teaching-related information are stored and that these repositories contain information about what to teach, how to teach, what works in the classroom, and what resources or materials to use.
Individual memory/perspectives
Respondents reported individual-level modes of storing information that included past experiences teaching a course as well as different objectives for teaching. Fifteen respondents recalled drawing upon their personal experiences with a given course when planning the next one. These experiences pertained to successful (or unsuccessful) lectures or activities, difficult topics for students, and so on. For 14 respondents, memory of what worked in past classes helped form their teaching objectives. For example, one respondent stated that she consistently tried to engage students during the class period, and that recalling her use of this technique guided much of her planning for her next courses.
The respondents also reported that their goals and objectives for teaching were criteria they used when designing their courses and class plans. These included objectives for engaging students in the classroom (13 respondents), deepening students’ conceptual understanding of the material (9 respondents), establishing a foundation in the discipline (7 respondents), making the material accessible (5 respondents), and striving to meet previously established learning goals (5 respondents). While these objectives may not appear at first glance to constitute information repositories, they were included in our analysis given that respondents described them as a type of consideration that was recalled (in much the same way past experiences in the course were recalled when planning for a particular class session).
Cultural norms and practices
Decision information can also be ‘stored’ in cultural forms, particularly the shared expectations and beliefs among members of a group that constitute high-status knowledge (Weick [1979]). Sixteen respondents reported that their colleagues held general expectations regarding teaching and learning, such that these norms represented a clearly identified body of information within their departments. In addition, six respondents reported that norms pertained to colleagues’ expectations regarding the canon of their discipline and the topics that should be taught to undergraduate students. These expectations were perceived as so strong as to be beyond debate, such that determining the curriculum for certain courses was as simple as selecting a canonical text.
Social networks/human resources
The social networks and human resources repository of decision information refers to the expertise and knowledge of colleagues and co-workers that are available to faculty. For nine respondents, information about teaching and learning was available through general teaching-related programs at their institutions such as brown bags and workshops available through Centers for Teaching and Learning. In addition, 11 respondents specifically mentioned the USE project as an information resource that they drew upon as part of their teaching practice. Six respondents also reported that a community of like-minded peers within their departments or institutions served as a resource for curricular information. Additionally, these communities provided a support system for faculty engaged in instructional improvement.
Curricular artifacts
Decision information about the curriculum is also stored in physical artifacts. For 11 respondents, their own curricular materials (broadly described as collections of notes, slides, and other materials) for a particular course were important repositories of information. Another seven respondents noted that they inherited these materials from other instructors. Regarding specific types of curricular artifacts, respondents reported PowerPoint slides (5 respondents) and lecture notes (9 respondents) as important sources of information.
External archives
Finally, curricular information could also be stored outside of a given department or institution. Nine respondents reported that the educational research literature helped them glean insights into effective teaching strategies. Another six respondents reported that textbooks helped shape the sequence and content of their courses. Finally, four respondents reported that they regularly drew upon online resources (e.g., other faculty’s course websites and online libraries related to teaching) when planning courses.
The processes whereby faculty actually prepare for their classes
Next, we report the different processes whereby faculty actually prepare for undergraduate classes, which centered on whether respondents ‘fine-tuned’ existing materials or prepared new ones (see Figure 2, below).
It is important to note that in reporting these data, we focus on the two major actions that the faculty reported in regard to their planning and not the discrete steps taken to fulfill these tasks or the factors that influenced them. In the next section where we report faculty decision chains, we delve into these nuances of faculty practice.
Fine-tuning existing materials
The most widely reported retrieval process did not involve developing or creating new class plans, especially in the case of courses that the respondent had taught before. For 13 respondents, planning generally involved fine-tuning or ‘tweaking’ materials that already existed in some form. These materials were retrieved either from respondents’ existing stockpile of slides, notes, and other media, or from colleagues who had previously taught the course.
In this course, when I first agreed to teach it, I went to who had taught the course before and talked to them about what should be in the course and what text was used. What they did was give me the word files with their course notes and I went through them and fine-tuned them.
This finding is consistent with Stark’s findings from 14 years ago (Stark [2000]), which suggests that the routine maintenance of curricular artifacts is a deeply entrenched practice among postsecondary faculty.
Respondents also offered insights into those factors that ‘triggered’ this fine-tuning process, which included being required to design a new course (8 respondents), student reactions to prior classes (6 respondents), and poor student performance (6 respondents). In addition, we highlight two instances where faculty reported that their fine-tuning process was instigated by notes that they had made the previous year on their lecture notes or PowerPoint slides about how particular activities or the structure of the lesson worked (or did not work) in class. Thus, these comments can be seen as a type of ‘note to self’ that the faculty member would then notice the next time she taught the course.
Making new plans
Another reason why respondents retrieved curricular information was to develop entirely new syllabi and/or individual lesson plans. Six respondents described retrieving information from a variety of sources when developing new plans that included lecture notes, PowerPoint slides, and so on. In each of these cases, the nature of the course (i.e., brand new) or the topic being covered in class necessitated the creation of new instructional materials. In the latter case, this was sometimes due to a major discovery in the field that led to the previous year’s materials being outdated.
Thus, what differs in terms of how faculty are retrieving information to prepare for their courses is their use of different repositories of information, the processes undertaken to prepare a plan (i.e., fine-tuning materials or creating new materials), and the purposes for which the planning activity is taking place (i.e., updating materials for a previously taught course or creating a new one). The final product, however, does not vary considerably in that faculty reported going into the classroom with lecture notes (in paper or electronic form), PowerPoint slides, or a combination of each.
Specific decision chains that faculty use when preparing for their courses
While it is important to document the nature of the memory retention structure and related retrieval processes, it is by tracing the actual flow of decision-making and information retrieval that we can arrive at an accurate portrayal of organizational memory functions in action. In this section, we present results from two different decision chain analyses of the steps that respondents went through while planning and if/how aspects of organizational memory were engaged. The first graphic reflects an attempt to model decision chains across multiple individuals (n = 7) at a more coarsely grained level, and the second graphic is a more detailed analysis of three individuals where aspects of the sociocultural and organizational context are included.
Decision chains in the aggregate
Eighteen respondents provided clear indications of the temporal process of their planning, and the most common ‘chain’ of planning steps involved two themes: looking through old lecture notes and updating these notes. These themes were reported by a total of seven respondents, either as a direct link between the two themes (4 respondents), an indirect link between the two themes (2 respondents), or only as a reference to a single theme (1 respondent). Examining these seven respondents’ planning procedures, it is possible to see the aggregate patterns in planning and the degree to which different repositories and/or retrieval processes were involved (see Figure 3).
The figure depicts the processes whereby individuals retrieved curricular information from the retention structure while planning their classes. Each decision chain (i.e., sequence of boxes) represents this process for an individual instructor as depicted by his or her code in the study sample. When the decision chain component could be clearly linked to one of the information repositories previously identified, it is outlined in thick, black lines. Within each box is one of the 75 decision steps identified in the analysis of the entire sample.
For example, respondent C01 (a biologist teaching a genetics course) reported the following steps in her planning: ensuring materials were in place, looking through old notes, and then making sure class activities were ready. Respondent C11 (a chemist teaching a general chemistry course) reported a more complex sequence of steps but shared the common theme of looking through old notes. Indeed, each of the seven respondents depicted retrieved curricular information that was stored in the curricular artifacts repository—lecture notes. These artifacts were either the result of prior experience teaching the course or were inherited from prior instructors. As a result, this process implicates two other aspects of the retention structure: individual memory (personal experience of the course) and social networks (experienced colleagues)f.
These results highlight the importance of preexisting curricular artifacts in shaping how some faculty in the study sample plan their courses, but that this process of artifact adoption is often not one of simply ‘plugging and playing’ old notes with no adaptations. As one biology faculty described this process:
“Since this is my first semester teaching I took many colleagues’ advice, which is to start with somebody else’s notes. So I start with those, and then I take a look at the homework assignment, look at the book and see what the students will be asked to do, and compare that to my notes. At that point I revise, maybe introduce new examples or replace examples. And then it becomes a question of how am I going to present that content? And to personalize the teaching for myself, hoping that the way I understand it offers an alternative way for them to see it as well.”
This quote illustrates how a single instructor fine-tunes existing notes by retrieving information from existing notes to update them while also drawing upon other resources, such as the textbook and beliefs about student learning. Importantly, as discussed above, all seven respondents depicted in Figure 3 referenced the core process of looking through notes to fine-tune them as a sort of habituated or automatic practice. This behavior was described by many, especially those who had taught for several years, as part of their regular routine for preparing a course or individual classes. Finally, it is worth noting that for two respondents (C16 and C18), the final stage of their planning process included altering their lecture notes immediately after the class and adding new observations about what worked or did not work, which reflects an effort to update these artifacts based on reflection about the efficacy of their own teaching practices.
Individual-level decision chains that include contextual factors
Next, we provide a more in-depth analysis of three individual’s decision chains (see Figure 4).
In this graphic an assortment of personal, sociocultural, and contextual factors are included to provide some background to each decision chain. For example, the relatively simple decision-making process of respondent C01 becomes more nuanced when her other considerations are included (e.g., time constraints and availability of local professional development). While the three individuals depicted here are certainly not representative of their departments or institution, it is notable that each reported past experiences teaching the course, local professional development programs, and cultural norms of peers as salient influences. When referring to cultural norms, each of the respondents identified two groups: one group of faculty supportive of pedagogical reform and one group that were either unsupportive or uninvolved in such activities. Each respondent noted that the presence of a supportive group was an important factor in their decision-making by acting as a source of information as well as a support system.
It is important to note that each individual underscored the fact that no single factor influenced or determined their planning behaviors; rather, a complex web of factors influenced their behaviors. This fact highlights the nonlinear and complex nature of decision-making processes, as well as challenges inherent in discerning meaningful implications from such data. Finally, it is also notable that no respondent described organizational procedures or policies that governed the existence of continuous improvement mechanisms that would require faculty to reflect upon their own teaching. The existence of such systems is a key facet of facilitating organizational change and learning (Walsh and Ungson [1991], Kezar [2005a]). The only evidence of such mechanisms in place were at the micro-level, where individual faculty took it upon themselves to engage in reflective practice at the conclusion of each class period, making notes on their curricular artifacts about what went wrong and what worked.
Effects of the USE project on faculty engagement with organizational memory
Finally, we consider the influences of the USE project on the elements of the retention structure and subsequent information retrieval processes. By examining organizational memory functions in places where reforms are being attempted, we can illuminate local practices and procedures that influence the success of these initiatives, and we can track changes in the organizational memory as a way to assess the depth and sustainability of reform efforts. Of course, definitive claims about the effects of the initiative on faculty behaviors cannot be made here, given the nature of the data. That said, we suggest that effects based on respondent self-reports are useful in highlighting whether and how STEM education reforms are targeting organizational memory functions (see Figure 5 below).
Individual memory/perspectives
Seven respondents reported that they developed new learning goals for their courses as a result of the USE project. Interestingly, six individuals had participated in the project while the other one was influenced second-hand by a colleague. Once learned, these goals are stored in the memories of the instructors as they plan and teach their courses. Similarly, six respondents reported that they altered their personal objectives and rationale for teaching in general (not for a specific course) as a result of their involvement in the USE project. This change came about through increased knowledge and appreciation of the learning theory and the educational literature.
Cultural norms and practices
Respondents claimed that the USE project influenced two features of cultural life related to curricular decision-making. First, 12 respondents reported that the expectations and norms related to teaching and learning evolved from an almost exclusive focus on research to one that also acknowledged the value of teaching. These respondents attributed this shift in cultural norms in part to the USE project, as well as other campus-wide initiatives focused on teaching and learning. Second, one respondent noted that the USE project’s efforts led directly to a change in how some of her colleagues viewed the canon of their discipline regarding introductory courses. These colleagues altered their previous focus on covering a broad range of topics to accommodate an emphasis on what could be taught well during a semester-long course. Importantly, this change was not uniformly observed across the department, as some in the department pushed back and argued that covering the entirety of the canon was of primary importance.
Social network/human resources
One of the most notable effects of the USE project on organizational memory was the increase in teaching and learning resources available to the respondents. These included human resources (11 respondents) in the form of postdoctoral researchers who were widely viewed as representing a substantial increase in the knowledge base of teaching and learning within departments. Thus, through hiring and ‘planting’ these individuals within departments, the USE project immediately enhanced the local human resources for curricular information. Another impact of the project included the facilitation of a community of like-minded instructors focused on education within science and math departments (7 respondents). This community was viewed as an important resource that provided a local support system and knowledge base.
Curricular artifacts
The USE project impacted the artifactual forms in which plans were made and stored, including new course syllabi (5 respondents), packets of materials (4 respondents), assessment materials (4 respondents), and clicker questions (4 respondents). Two departments participating in the project are making a concerted effort to archive these materials for future use. As one physics faculty noted:
“That’s why I’m trying very hard to create materials that are easy for another faculty member to make sense of…. It should be obvious to them why this helps. And it should be easy for them to find and use, so we’re trying not to write 10-page instructor guides. We’re making it freely available, and we’re trying not to go overboard.”
For this instructor, the storage of the new curricular materials in physical form was a critical part of the pedagogical reform process, as she recognized that instructors coming to a new course often seek out materials from previous instructors. Yet in this department, no formal mechanism existed for archiving curricular artifacts. However, the instructor also observed that by simply storing these materials, one could not assume that a new instructor would adopt them. Instead, the person would need to be motivated to adopt these admittedly complex and demanding materials. This underscores the importance of the retrieval process and how the presence of high-quality curricular information in the retention structure may be a necessary antecedent to adoption, but not a guarantee.
External archives
The USE project influenced how five faculty drew upon an external source of curricular information—that of educational research. Prior to their involvement with the project, these individuals had little exposure to either formal learning theory or educational research in their disciplines, both of which provided a new source of knowledge upon which they drew when planning their courses.
Changes to the retrieval of curricular information
Of the seven respondents whose information retrieval practices are depicted in Figure 1, three had been engaged with the USE project (respondents C01, C16, and C18). Each drew upon preexisting materials, including lecture notes, PowerPoint slides, and clicker questions when planning their classes, and each reported that the USE project influenced the development of these materials. This included the articulation of student learning goals that became the groundwork for course syllabi and all subsequent materials such that lecture notes, slides, and clicker questions were explicitly linked to these goals. Thus, through leading faculty to articulate learning goals and embed them within their course materials, the USE project directly shaped the artifactual repositories of curricular information. When respondents began to plan future classes, they drew upon these resources. That is, the USE project did not directly influence the nature of the retrieval process itself, but instead it led to alterations in the information sources that were habitually tapped as part of these processes. It is important to note that two instructors observed that the traditional ways in which curricular information are retrieved (i.e., in an unstructured fashion by instructors who rotate in and out of courses) were significant barriers to the realization of the USE project’s goals. This was due to the lack of guarantees that the new instructors would adopt the newly created materials or have the pedagogical training to use them proficiently in the classroom.