Exploring the dynamics of organizational learning: identifying the decision chains science and math faculty use to plan and teach undergraduate courses

Background

The field of STEM education is increasingly focusing on processes of individual, cultural, and organization-level change in postsecondary institutions, yet current approaches tend to focus on individual leverage points isolated from other factors and the broader institutional context. Research on reform implementation highlights how individual decision-making is shaped by a variety of inter-connected factors - or what we call 'decision chains’.

Organizational learning theory offers a way to conceptualize how these decision chains are implicated in the change process. Organizational learning refers to the processes whereby organizations store information in what is known as the ‘organizational memory’, how this information is retrieved, and how alterations to these processes can affect organizational.

In this paper, we report findings from a qualitative case study of how 24 science and math faculty at a large, public research university in the United States engaged with their organization’s memory while planning courses. We also explore how a reform initiative—the Undergraduate Science Education (USE) project—influenced these memory functions. We analyzed semi-structured interviews using a structured approach to grounded theory as well as techniques for graphically depicting verbal data.

Results

Results indicate that faculty accessed five repositories of curricular information within the organizational memory: individual memory, cultural norms, social networks and human resources, curricular artifacts, and external archives. When retrieving information from these repositories, faculty primarily ‘fine-tuned’ existing curricular artifacts (i.e., lecture notes and PowerPoint slides). Analyses of decision chains used by faculty highlight the idiosyncratic manner in which planning unfolds in practice, the centrality of existing artifacts, the role of contextual factors, and the absence of continuous improvement systems. Analyses of the USE project’s effects indicate changes to features of the organizational memory.

Conclusions

Besides contributing new insights into the nature of organizational learning in higher education, the decision chain method described in this paper can be used to complement existing metrics for program evaluation and to diagnose leverage points for new STEM education change efforts. A potentially useful approach may target curricular artifacts for regular updating and the imposition of continuous improvement systems, while allowing faculty local control over this process.

Understanding the processes whereby colleges and universities change and evolve over time is a long-standing problem explored by researchers of higher education (Clark [1986]; Kezar [2001]). Increasingly, researchers are focusing on change processes in the science, technology, engineering, and mathematics (STEM) disciplines, given the considerable attention paid to improving the way that faculty plan and teach their courses (e.g., Henderson et al. [2011]). A promising line of inquiry is focusing on how individual faculty think about, and react to, instructional innovations and interventions (Henderson and Dancy [2008]), as well as the factors in the organizational context that appear to support and/or inhibit pedagogical reform (Walczyk et al. [2007]). Since no single factor represents a ‘silver bullet’ that can magically transform practice, it is more likely that a variety of factors (e.g., disciplinary cultures, incentive systems, institutional missions) will need to be altered to affect long-lasting change (Birnbaum [1988]; Lattuca and Stark [2009]). As a result, research is needed that accounts for the ways in which faculty decisions are influenced by the multiple and often interconnected factors or ‘levers’ within postsecondary institutions (Henderson and Dancy [2007]; Austin [2011]).

However, there exists little empirical research on how these factors interact with one another in real-world settings to shape faculty work. Instead, a more common approach is to either list a variety of influential factors without articulating specific relationships among them (Stark [2000]) or to posit linear relationships that oversimplify the complexity of real-world practice between certain variables and faculty work (Umbach [2007]). Further limiting the field of STEM education in identifying these interconnected chains of individual and contextual factors that shape faculty decision-making is the reliance on individual-level theoretical frameworks and survey methodologies for studying change. In particular, a commonly used theoretical framework is that of diffusion of innovation theory (Rogers [1995]), which in its application in STEM education research tends to focus on perceptions about reform and subsequent instructional behaviors at the individual level (e.g., Henderson et al. [2012]). Interestingly, while this approach is predicated on the notion that adoption is strongly influenced by the situation surrounding the potential change, less attention has been paid to the nuances of institutional contexts and how they affect faculty adoption (or rejection) of instructional reforms. Understanding faculty decision-making at a fine-grained level is difficult, if not impossible, when relying exclusively on survey data. As Stark noted in relation to survey-based research on curriculum design (2000, p. 435), ‘Our work fell short of exploring in depth the actual decisions teachers make about course plans and curriculum’.

Why is it important for the STEM education community to better understand the subtle interactions among individual attributes, contextual factors, and instructional practices? Evidence from research on reform implementation demonstrates that reforms aligned with existing cultural norms and behaviors are more successful than exclusively top-down efforts, leading scholars to argue that change initiatives should be based upon accounts of teachers’ actual practice (Kezar and Eckel [2002]; Spillane et al. [2002]; Henderson et al. [2011]). In addition, some have argued that ‘culture of teaching’ is one of the primary elements within colleges and universities that is thwarting reform, and consequently it must be changed before instructional reforms will be adopted and perpetuated (e.g., Wieman et al. [2010]; Anderson et al. [2011]). However, culture is a notoriously slippery concept that in practice is hard to operationally define, and this state of affairs becomes particularly problematic for instructional designers who need to identify such local teaching cultures prior to creating programs that build on existing practice while also attempting to change it. Some argue that one indicator of culture in educational settings is practice itself and patterns of local decision-making behaviors that reflect the complex and nonlinear realities of educational work more accurately than singular measures such as institution or disciplinary types (Trowler and Cooper [2002]). A promising line of inquiry that examines the social networks implicated in STEM education reform highlights the importance of capturing the broader socio-cultural milieu in which individual faculty practice takes place (Neumeyer et al. [2014]).

Once these individual-level practices and related organizational functions are identified, educational leaders can pinpoint specific policies, procedures, or artifacts to be altered or otherwise addressed in order to improve educational practice (Spillane et al. [2001]; Petrides [2002]; Cobb et al. [2009])^a. But in order to design interventions in accordance with (or to be responsive to) local practice, practitioners need a way to diagnose local decision-making processes that capture the nonlinear nature of faculty work at the local level.

Organizational learning theory is an approach that holds promise as an analytic lens through which faculty practice can be studied in such a systemic and fine-grained manner. Organizational learning theory applies concepts from human learning to that of organizations and focuses on how organizations change and evolve (or not) in light of new information and experiences. A key aspect of organizational learning relates to the organizational memory, or how ‘organizations encode, store, and retrieve the lessons of history despite the turnover of personnel and the passage of time’ (Levitt and March [1988], p. 319). In organizations, information can be stored in various forms or ‘repositories;’ and over time, the retrieval of information from these repositories can become an entrenched feature of local practice (Walsh and Ungson [1991]).

Through illuminating such deeply habituated practices, organizational learning analyses speak to the nature of organizational change or maintenance of the status quo, as sustainable change is often possible only when institutionalized and habituated practices such as memory repositories and retrieval are changed in what Argyris and Schön ([1974]) called ‘double-loop learning’. Such change takes place when new information, either from the external environment or from internal systems of self-assessment, is received by organizational members who then update the relevant procedures and attendant beliefs (Cyert and March [1963]). Of course, this process is contingent upon the existence of organizational systems that provide regular feedback (i.e., continuous improvement systems) and/or individuals who are willing and ready to update their own behaviors and belief systems (Senge [1990]).

Despite the promise of the organizational learning framework for elucidating critical features of faculty work (Boyce [2003]; Kezar [2005a]), little empirical work exists that applies organizational learning theory to teaching-related practices at the postsecondary level. In this paper, we examine the nature of organizational memory functions at a single public research university in the United States by identifying the constituent parts of faculty decision-making for planning undergraduate STEM courses. In doing so, we focus on the ‘decision chains’ that comprise faculty decision-making while planning for a course, which include the types of organizational memory repositories accessed (and related contextual determinants) and specific considerations and actions subsequently taken^b. In addition, we examine the degree to which a single STEM education reform initiative, the Undergraduate Science Education (USE) project, influenced these functions. We chose to highlight a situation in which a reform was being attempted since it is in these collisions between new ideas and the status quo that organizational functions and traditions are often brought into bold relief. In this study, we analyze interview data from 24 STEM faculty using a structured approach to grounded theory and verbal analysis techniques to answer the following questions: (1) In what information repositories do science and math faculty draw upon when planning for an undergraduate class? (2) What are the processes whereby science and math faculty actually prepare for an undergraduate class? (3) What are the specific decision chains that faculty use as they access information repositories and prepare for their classes? (4) To what degree, if any, did the USE project influence faculty engagement with the organizational memory? We then consider the implications of the results for STEM education reform initiatives in postsecondary institutions.

Exploring factors influencing faculty decision-making

Higher education researchers have long been interested in delineating the specific factors that shape faculty work, particularly in regard to curriculum design and instructional practice. In the 1970s and 1980s, researchers sought to develop explanatory models of course planning to help designers create and improve degree programs (e.g., Mayhew and Ford [1971]), though these early efforts were critiqued for positing an overly linear and simplistic account of faculty practice that likely did not exist in real-world settings (Conrad and Pratt [1983]). Later research attempted to account for a more complex array of factors including environmental inputs (e.g., professions and community), the nature of the material, instructional traditions, and resource allocations, to name but a few (Conrad and Pratt [1983]; Lattuca and Stark [2009]).

A shift to more multifaceted and systemic account of faculty practice was also the hallmark of the research program of Joan Stark (Stark [2000]). Given the autonomy that many faculty enjoy in their work, Stark hypothesized that personal characteristics (e.g., beliefs about teaching and learning) would largely drive their decisions about the structure and content of a course, but these individual-level factors were influenced or ‘filtered’ by features of the organizational context that ultimately determined how a course was designed and taught (Stark [2000]). A key finding from this line of inquiry was that much of postsecondary instructors’ planning involves minor alterations to existing materials on account of the fact that instructors often teach the same course(s) year to year—what Stark ([2000]), p. 420 called ‘routine maintenance’. However, as this work was primarily survey based, it did not capture or describe the specific features of faculty practice as it unfolded in real-world situations.

More recently, research on STEM education has begun exploring similar territory, largely based on the widespread recognition that the pace of pedagogical change has not been what policymakers or educational leaders anticipated or desired. As previously noted, a growing body of research is exploring which individual, cultural, and organizational factors appear to be most influential in supporting or inhibiting faculty adoption of interactive teaching modalities. For example, Henderson and Dancy ([2007]) found that factors including student attitudes towards school, departmental expectations of content coverage, lack of time, and class size each acted as barriers to the adoption of new teaching methods. Building on the promise represented by such analyses, researchers have subsequently called for additional work to identify ‘the factors at work within the multiple contexts that affect faculty work’ , so that educational leaders can target these ‘multiple change levers’ when designing and implementing reform initiatives (Austin [2011], p. 3)^c. However, a theoretical framework that is able to model decision-making within complex systems, as well as research methods to empirically identify them, is needed to help the field of STEM education meet these goals.

Organizational learning in postsecondary institutions

Organizational learning has a long history in organization and administration science, but it is important to distinguish between the empirical research program of organizational learning on the one hand and the managerial theory of the learning organization on the other hand (Kezar [2005a]). The empirical research program is generally focused on the investigation of the processes of organizational functions and change. In contrast, the learning organization approach is a more of a proscriptive approach to management that describes or advocates an idealized state of affairs. While the latter is grounded in the former, some consider the learning organization approach to be a popular yet fleeting management theory similar to total quality management (TQM) (Birnbaum [2000]; Kezar [2005a]). While the concepts are certainly related, the long-standing research program of organizational learning is the main concern of this paper.

At its core, research on organizational learning focuses on the underlying processes of how organizations learn, change, and adapt (Levitt and March [1988]). Much of the later research in this area was strongly influenced by advances in cognitive psychology. Concepts such as cognitive ‘scripts’ and ‘schema’ , which refer to mental representations related to particular information stored as neural pathways in the brain, became widely used in the organizational learning literature (e.g., Gioia and Poole [1984]). For example, information processing theory focuses on the development of organizational systems to process information as a key function of organizational success in uncertain environments (Galbraith [1977]). This approach emphasizes the role of information pertaining to organizational experience (e.g., established routines and effective strategies) in organizational functioning and improvement. This stored information is important for understanding organizational behavior because, once retrieved, it may act as ‘decisional stimuli’ that elicit particular responses and actions (Walsh and Ungson [1991] p. 61).

In any case, one of the most appealing features of organizational learning theory for education researchers is the prospect that an organization can in fact grow, evolve, and change in a positive manner, and that these changes can be empirically studied to provide evidence for organizational improvement. When considering this application, it is useful to understand the long-standing debate in the field regarding what constitutes evidence of learning and change. For some theorists, evidence of organizational learning is only apparent by increased performance or observable changes in behavior, whereas others argue that changes resulting from learning may not be visible or immediately evident. For example, Huber ([1991]) argues that ‘an entity learns if, through its processing of information, the range of potential behaviors is changed’ (p. 89). This focus on alterations to the mechanisms that underlay the potentialities of human behavior is similar to Argyris and Schön ([1974]) well-known distinction between single- and double-loop learning. In this view, single-loop learning occurs when an organization changes superficial behaviors or protocols but leaves internal norms and practices intact. In contrast, double-loop learning focuses on the internal mechanisms of the group or organization and involves enacting changes to deeply held traditions, assumptions, and behaviors (Argyris and Schön [1974]). Interestingly, just as an organization may learn by altering entrenched behaviors, an ‘unlearning’ process can be useful when organizations discard old and unproductive information or practices (Nystrom and Starbuck [1984]).

In delving more deeply into the specific processes of organizational learning, Huber ([1991]) argued that researchers should think in terms of information acquisition, distribution, processes of interpretation, and memory functions. In particular, organizational memory, or what Levitt and March ([1988]) call the recording and conservation of experience, is perhaps the most critical part of the learning process. This is due in part to the fact that most organizations experience personnel turnover and regular upheavals (e.g., course rotation procedures) that make the storage of information critical because newcomers will draw upon these storehouses of information to guide their behavior.

Importantly, information can be stored in a variety of locations or repositories (Cyert and March [1963]). One way to conceptualize the different locations where organizational information can be stored was advanced by Walsh and Ungson ([1991]), who posit six different locations where organizational information is frequently stored: individual memory, cultural norms and practices, procedures, roles, artifacts, and external archives^d. In this study, we build on this approach while removing certain repositories (e.g., procedures and roles) and adding others such as social networks and human resources, which refers to the skills and knowledge represented by individuals that can be accessed by other organizational members (Gamoran et al. [2003]). Overall, these different places where curricular information is stored can be thought of as the retention structure for organizational memory (see Table 1).

From these different repositories, information is retrieved during preparation for a task or during the task itself. This retrieval process can be thought of as either a deliberate, controlled process or an automatic, habituated process where decision makers are not consciously aware of how they retrieve information or from where. Once retrieved, the information enters into an active decision-making process and is thus subject to a variety of influences including sociocultural and physical characteristics of the immediate task environment and the preexisting cognitive attributes of the decision maker. That is, the prior experiences, knowledge, and beliefs of individuals will shape how the new information is interpreted and then used (Daft and Weick [1984]; Hora [2014]). This process of interpretation highlights the fact that the organizational information, once retrieved, is not simply enacted into practice but instead undergoes a translation between the retention structure and task performance.

Educational applications

What constitutes learning is particularly salient in the field of postsecondary education, where the concept of organizational learning is widely viewed as a productive way to think about improving education. As noted, a distinction must be made between the empirical research program on organizational learning and the managerial theory of the learning organization. This distinction is particularly significant in higher education, given that many scholars use the terms interchangeably and/or to refer to a collective of managerial strategies (e.g., total quality management and management by objectives) (Kezar [2005a]). While there are empirical studies about the nature of organizational learning in postsecondary institutions, most researchers advocate for colleges and universities to become learning organizations (e.g., Boyce [2003]; Petrides [2002]; Kezar [2005b]). A promising line of inquiry that uses the organizational learning framework focuses on issues of equity and student learning. Bensimon ([2005]) analyzes individual faculty and their ‘cognitive frames’ , which are seen as instrumental in shaping the ways in which they teach and think about student learning, all of which can ‘create the problem of unequal outcomes’ (p. 101). A core idea motivating this work is that organizational learning is not about adopting new programs or best practices per se but instead is dependent upon institutional actors changing how they think about students and learning.

In this study, we adopt an organizational learning framework to examine an activity that is common to all postsecondary institutions: course planning. The focus on course planning is important because of the centrality of curricular design to the instructional process (Lattuca and Stark [2009]; Hora and Ferrare [2013]). Course planning practices, such as creating course syllabi and selecting instructional materials and teaching activities, constitute the primary means whereby faculty design learning environments for their students. This activity is particularly amenable to analyses using the organizational learning framework in general and the construct of organizational memory in particular. This is because, during the planning process, faculty retrieve information about the curriculum, course, and topic from various sources (i.e., the organizational memory).

Besides documenting the nature of these processes, we investigate whether or not a particular project influenced the organizational memory functions among the faculty participants in our study. As noted, in situations where new information or policies are introduced, the processes of routinized practice (i.e., memory functions) can be brought into bold relief. Further, since the organizational learning framework can be used as a way to detect deep-seated changes to an organization, studying these processes in light of the USE project represents an opportunity to field test this framework as one approach to capturing project effects on a specific set of departmental functions. Thus, detecting whether or not instructional reforms such as the USE project altered these memory functions of storage and retrieval would indicate the degree to which the organization has ‘learned’ or changed in response to the initiative. Finally, insights into these issues can shed light on the processes of organizational change in general and how faculty respond to innovations and new policy initiatives in particular.

Our design is that of a qualitative case study, where the case is course planning and classroom instruction in math, biology, chemistry, geology, and physics at a large, public research university^e. These disciplines were selected because the larger study from which this analysis is drawn was focused on STEM disciplines and funded by the National Science Foundation. The course component of interest is the classroom lecture, instead of discussion, laboratory, or tutorial sessions. Focusing on a single portion of a course was essential to reduce the complexity underlying planning processes for each course component. The case study method involves an intensive analysis of a single-bounded unit that draws upon a variety of data and perspectives in order to provide a rich and detailed account of that unit (Yin [2008]).

Within the context of a case study design, we adapt theory and method from decision-making research and the learning sciences to examine in depth the underlying decision-making procedures that inform course planning. Given that many real-world tasks (e.g., course planning) are dynamic and unbounded and that interviewer intervention is often required to keep respondents on topic, researchers of naturalistic decision-making developed a technique called the critical-decision method that is a retrospective think-aloud procedure (Crandall et al. [2006]; Feldon [2010]). Researchers then analyze these data to identify themes or patterns in the structure and temporal nature of respondent cognition. For this study, we drew on the critical-decision method to craft the data collection protocols and a structured approach to grounded theory (Corbin and Strauss [2007]) and the verbal analysis method (Chi [1997]) to analyze the resulting discourse data. To ensure the reliability and validity of the research findings, we regularly met to discuss coding procedures and preliminary findings, and we tested inter-rater reliability at regular intervals throughout the analytic process (Miles and Huberman [1994]).

Finally, it is important to note that the analysis reported in this paper is based on a dataset collected for a larger, multi-institution study of which the first author was the lead researcher. For the fieldwork that comprised the bulk of the data analyzed in this paper, the first author collected data with two additional researchers that did not include the second author. Instead, the second author conducted four interviews using the interview protocol developed for the larger study and then participated in subsequent data analysis activities.

The Undergraduate Science Education (USE) project

The USE project was a department-driven change effort undertaken at the study site. The 5-year project was funded by a large private donation and led by science faculty members in collaboration with several departments. The project’s goals were to improve student learning outcomes by improving faculty members’ understanding of student learning and to encourage the use of inquiry-based teaching practices and high-quality formative and summative assessments of learning. In addition, an overarching goal of the project was to ‘change the culture’ of how research-oriented faculty were thought about and engaged in their teaching practice. To achieve these goals, the USE project collaborated directly with departments, which had committed to participate in the change effort through successful submissions as part of a competitive proposal process. The main charge to participating departments was to identify a sequence of courses to be revised, generally starting with high-enrollment, lower-division undergraduate courses, and then proceeding to upper-division courses. The revision process involved identifying and agreeing upon quantifiable learning goals for each course based on what the department wished students to be able to do at the course’s end. Central to this process was the hiring of several postdoctoral researchers who were located within each participating department. The researchers worked with faculty for two or three semesters to guide and support efforts to revise the selected courses, providing both intellectual and practical assistance. It is important to note that neither authors were directly involved in this initiative.

Sampling procedures

The sampling frame for this study included 170 individuals listed in the spring 2012 timetable as the instructors of record for undergraduate courses in math, physics, chemistry, biology, and geology departments at the study site. We contacted individuals up to two times via email, inviting them to participate. Ultimately, 20 faculty self-selected into the study. Of these, six were considered active participants in the USE project based on their participation in course revision activities, which we first identified from USE project records and then corroborated during interviews. Given the centrality of the USE project to the study, we contacted additional instructors from a list of participants provided by USE project staff, and four agreed to participate, which resulted in a total of 24 participants. The final sample included 11 participants who had been active in the USE project and 13 participants who had not. This split sample was desirable in order to obtain a portrayal of organizational memory functions that were both potentially directly influenced by the USE project and those that had not been directly influenced. Information about the study sample is provided in Table 2.

Data collection

All data were collected in spring 2012. The first phase involved three researchers, led by the first author. The second phase, conducted by the second author, included interviews with the four additional participants in the USE project who were identified at a later date. All researchers underwent the same training procedures, which entailed in-depth discussions about research instruments and their use in the field. For this study, we followed the critical-decision method approach by asking a focused question about a recently performed task and then asking follow-up questions to delve more deeply into the decision-making process of each respondent (see Feldon [2010]). We used a semi-structured interview protocol that elicited an extensive account of the respondent’s most recent planning activities. Following this question were probes that examined the role of the course syllabus, course type, prior experience, and knowledge of educational research in shaping the respondent’s planning process (see Appendix A). These probes were based on evidence from the literature regarding the primary factors that influence course planning (e.g., Stark [2000]). While we did not explicitly elicit information about organizational memory functions, the question resulted in rich data about respondents’ retrieval processes and the sources from which they drew while planning their classes. Then, to explore if and how teaching-related professional development (e.g., the USE project) influenced respondents’ planning procedures, we asked: ‘How, if at all, has your participation in professional development (or the USE project) influenced how you plan and teach?’ Not all respondents had participated in such projects and thus not all answered this question.

Data analysis

The research team transcribed and entered all interviews into NVivo® qualitative analysis software. Our analysis drew on a structured approach to grounded theory, which involves a combination of inductive analyses of textual data and consideration of theory that is external to the data (Corbin and Strauss [2007]). We began by segmenting the complex transcripts into more manageable units. To do this, we used an inductive, open-coding process in which new codes were created from terms, phrases, or ideas in the text. To develop the initial codes, the analysts reviewed five randomly selected transcripts and independently created a list of codes that described higher-order categories that aligned with the interview questions (e.g., steps and processes in planning and effects of professional development). The first author applied the coding scheme to all 24 transcripts; then, the analysts focused on the data coded as ‘steps and processes in planning’ and data coded as ‘effects of professional development’. This involved an open-coding process whereby the analysts independently developed new codes from approximately 10% of the coded text. Each analyst then compared each successive instance of the code to previous instances in order to confirm or alter the code and its definition (i.e., the constant comparative method) (Glaser and Strauss [1967]). After coding the initial selection of data, the analysts met to discuss their new codes and collaboratively developed a new code list. Then, the analysts coded another 10% of the data and repeated this process of code revision until they developed a final coding scheme. Examples of codes included ‘personal experiences of the course’ , ‘fine-tuning lecture notes,’ and ‘inheriting course materials’. The results of an inter-rater reliability test showed that the proportion of instances where both analysts applied the same code was 89%. The first author then applied the coding scheme to all transcripts, which resulted in an extensive NVivo® library of coded text.

Identifying characteristics of organizational memory and retrieval

At this stage of the analysis, we began to consider how organizational learning theory could inform the interpretation of the data and at which point constructs such as organizational memory and retrieval entered into the analytic process. That is, the codes (and thus, the organizational memory and retrieval functions as reported in this paper) were not originally derived in light of this theoretical framework but were identified through an inductive process. We later used the theoretical framework to interpret these findings. Using this theory as a guiding framework, we re-analyzed the themes identified as core components of planning in order to identify the primary components of the retention structure according to the framework set forth by Walsh and Ungson ([1991]). Thus, we mapped the codes onto these bins: individual, cultural, procedural, structural, artifactual, and archival. For example, we mapped the theme of ‘personal experiences of the course’ as a key factor shaping course planning onto the memory bin of ‘individual memory’ and so on, until the results indicating the nature of the organizational memory structure were identified. Then, to identify the nature of information retrieval functions, we examined specific references to the individual retrieving information from these sources in order to prepare for the course. It is important to note that we report themes that were reported by a total of four or more respondents only. Thus, with the exception of the detailed decision chain models of two instructors, more idiosyncratic findings limited to an individual or small groups are not reported in this manuscript due to space limitations.

Identifying temporal steps in planning routines

Next, we sought to identify how organizational memory and retrieval functions operated in practice. To do this, we used the verbal analysis technique of Chi ([1997]), which is a variation of protocol analysis that uses discourse-based data to identify structural features of human cognition. This is similar to causal network analysis, which is an approach for identifying relationships between concepts in a graphic and time-ordered fashion (Miles and Huberman [1994]). For this analysis, we revisited the data coded as ‘steps and processes’ and identified clearly demarcated steps in how respondents planned their courses. Not all respondents provided sufficiently detailed data, which resulted in a more limited dataset (n = 18). These data represented decision chains for each individual, comprising specific components of his or her planning activities. Overall, we identified 75 unique decision components (e.g., use lecture notes) across the entire set of 18 transcripts.

With these data in hand, we created two graphics. The first represents aggregated decision chains across seven individuals who exhibited similarities in their use of the organizational memory. In developing this graphic, we identified ‘use lecture notes’ and ‘revise lecture notes’ as the most common components reported by multiple respondents. The decision chains that included one or both of these codes were visually depicted in order to examine the structure and temporal nature of course decision-making. This approach involves graphically depicting the decision steps as a network map (see Chi and Koeske [1983] for a seminal example of this method). Importantly, the resulting displays represent the accounts of a relatively small number of respondents from the study and thus should not be extrapolated to entire departments or institutions within the study sample or viewed as definitive accounts of action and behavior within these administrative units. In addition, we included in this graphic only those behaviors clearly described by the respondent. Thus, no analyst inferences were made regarding contextual influences or other factors that may have impinged upon their behaviors.

Next, we selected three individuals from this analysis in order to delve more deeply into the sociocultural and contextual factors that influenced their planning. To do this, we focused on the three participants in the USE project who were included in the previous analysis in order to begin exploring the effects of the project on their behaviors. For this analysis, we returned to the full interview transcript and in places where the respondent discussed their planning, which often took place throughout the interview, we identified other factors that influenced their planning. These sociocultural and contextual factors were then graphically overlaid on the decision chains identified in the previous analysis.

Identifying the effects of the USE project

Finally, we analyzed the sections of the interviews coded as ‘effects of professional development’ to identify whether or not the USE project or other similar initiatives had any demonstrable impact on organizational memory or retrieval functions at this institution. Using the inductive analytic techniques described earlier (e.g., open-coding and constant comparative method), the analysts first identified instances where respondents clearly attributed a change in their course planning activities to the USE project. Thus, the analysts did not infer effects but instead relied on respondent accounts. These statements of attribution were mapped onto the bins of the retention structure used in previous analyses (e.g., individual memory and curricular artifacts).

Limitations

Limitations to the study include a small, self-selected sample and the reliance on self-reported data to analyze course planning procedures and the effects of the USE project. The issue of self-selection is difficult to avoid in field-based research such as this, and we attempted to address the possibility of collecting data only from faculty engaged in teaching-related activities by creating a split sample, though we recognize the limitations of this sampling approach. In addition, ideally planning procedures could be observed ‘live’ , but practical considerations precluded this possibility for faculty participating in the study. In addition, the sample represents only a small portion of the disciplines represented in the ‘STEM’ acronym, and thus results should not be generalized to broader populations of STEM faculty. Similarly, because this study was conducted in a large, research institution, the study population reflects a unique organizational setting. Finally, the reliance on respondent self-reports of the effects of the USE project was unavoidable, given the lack of a pre- and post-project evaluation established at the inception of the project.

Results from the analysis are reported in four sections: (1) the repositories accessed for teaching-related information during planning; (2) the specific retrieval functions that faculty use to access these repositories; (3) patterns in how these memory and retrieval functions operate in practice (i.e., decision chains and implicated contextual factors); and, (4) the degree to which the USE project influenced these memory and/or retrieval functions.

The information repositories faculty drew upon when planning

First, we report the types of repositories where information pertaining to curricular and instructional issues were stored. The locations and the frequency with which they were referenced are depicted in Figure 1, below.

Types of information repositories accessed during course planning.

Full size image

These data demonstrate the diversified forms in which teaching-related information are stored and that these repositories contain information about what to teach, how to teach, what works in the classroom, and what resources or materials to use.

Individual memory/perspectives

Respondents reported individual-level modes of storing information that included past experiences teaching a course as well as different objectives for teaching. Fifteen respondents recalled drawing upon their personal experiences with a given course when planning the next one. These experiences pertained to successful (or unsuccessful) lectures or activities, difficult topics for students, and so on. For 14 respondents, memory of what worked in past classes helped form their teaching objectives. For example, one respondent stated that she consistently tried to engage students during the class period, and that recalling her use of this technique guided much of her planning for her next courses.

The respondents also reported that their goals and objectives for teaching were criteria they used when designing their courses and class plans. These included objectives for engaging students in the classroom (13 respondents), deepening students’ conceptual understanding of the material (9 respondents), establishing a foundation in the discipline (7 respondents), making the material accessible (5 respondents), and striving to meet previously established learning goals (5 respondents). While these objectives may not appear at first glance to constitute information repositories, they were included in our analysis given that respondents described them as a type of consideration that was recalled (in much the same way past experiences in the course were recalled when planning for a particular class session).

Cultural norms and practices

Decision information can also be ‘stored’ in cultural forms, particularly the shared expectations and beliefs among members of a group that constitute high-status knowledge (Weick [1979]). Sixteen respondents reported that their colleagues held general expectations regarding teaching and learning, such that these norms represented a clearly identified body of information within their departments. In addition, six respondents reported that norms pertained to colleagues’ expectations regarding the canon of their discipline and the topics that should be taught to undergraduate students. These expectations were perceived as so strong as to be beyond debate, such that determining the curriculum for certain courses was as simple as selecting a canonical text.

Social networks/human resources

The social networks and human resources repository of decision information refers to the expertise and knowledge of colleagues and co-workers that are available to faculty. For nine respondents, information about teaching and learning was available through general teaching-related programs at their institutions such as brown bags and workshops available through Centers for Teaching and Learning. In addition, 11 respondents specifically mentioned the USE project as an information resource that they drew upon as part of their teaching practice. Six respondents also reported that a community of like-minded peers within their departments or institutions served as a resource for curricular information. Additionally, these communities provided a support system for faculty engaged in instructional improvement.

Curricular artifacts

Decision information about the curriculum is also stored in physical artifacts. For 11 respondents, their own curricular materials (broadly described as collections of notes, slides, and other materials) for a particular course were important repositories of information. Another seven respondents noted that they inherited these materials from other instructors. Regarding specific types of curricular artifacts, respondents reported PowerPoint slides (5 respondents) and lecture notes (9 respondents) as important sources of information.

External archives

Finally, curricular information could also be stored outside of a given department or institution. Nine respondents reported that the educational research literature helped them glean insights into effective teaching strategies. Another six respondents reported that textbooks helped shape the sequence and content of their courses. Finally, four respondents reported that they regularly drew upon online resources (e.g., other faculty’s course websites and online libraries related to teaching) when planning courses.

The processes whereby faculty actually prepare for their classes

Next, we report the different processes whereby faculty actually prepare for undergraduate classes, which centered on whether respondents ‘fine-tuned’ existing materials or prepared new ones (see Figure 2, below).

Processes whereby faculty retrieve and use information while planning.

Full size image

It is important to note that in reporting these data, we focus on the two major actions that the faculty reported in regard to their planning and not the discrete steps taken to fulfill these tasks or the factors that influenced them. In the next section where we report faculty decision chains, we delve into these nuances of faculty practice.

Fine-tuning existing materials

The most widely reported retrieval process did not involve developing or creating new class plans, especially in the case of courses that the respondent had taught before. For 13 respondents, planning generally involved fine-tuning or ‘tweaking’ materials that already existed in some form. These materials were retrieved either from respondents’ existing stockpile of slides, notes, and other media, or from colleagues who had previously taught the course.

In this course, when I first agreed to teach it, I went to who had taught the course before and talked to them about what should be in the course and what text was used. What they did was give me the word files with their course notes and I went through them and fine-tuned them.

This finding is consistent with Stark’s findings from 14 years ago (Stark [2000]), which suggests that the routine maintenance of curricular artifacts is a deeply entrenched practice among postsecondary faculty.

Respondents also offered insights into those factors that ‘triggered’ this fine-tuning process, which included being required to design a new course (8 respondents), student reactions to prior classes (6 respondents), and poor student performance (6 respondents). In addition, we highlight two instances where faculty reported that their fine-tuning process was instigated by notes that they had made the previous year on their lecture notes or PowerPoint slides about how particular activities or the structure of the lesson worked (or did not work) in class. Thus, these comments can be seen as a type of ‘note to self’ that the faculty member would then notice the next time she taught the course.

Making new plans

Another reason why respondents retrieved curricular information was to develop entirely new syllabi and/or individual lesson plans. Six respondents described retrieving information from a variety of sources when developing new plans that included lecture notes, PowerPoint slides, and so on. In each of these cases, the nature of the course (i.e., brand new) or the topic being covered in class necessitated the creation of new instructional materials. In the latter case, this was sometimes due to a major discovery in the field that led to the previous year’s materials being outdated.

Thus, what differs in terms of how faculty are retrieving information to prepare for their courses is their use of different repositories of information, the processes undertaken to prepare a plan (i.e., fine-tuning materials or creating new materials), and the purposes for which the planning activity is taking place (i.e., updating materials for a previously taught course or creating a new one). The final product, however, does not vary considerably in that faculty reported going into the classroom with lecture notes (in paper or electronic form), PowerPoint slides, or a combination of each.

Specific decision chains that faculty use when preparing for their courses

While it is important to document the nature of the memory retention structure and related retrieval processes, it is by tracing the actual flow of decision-making and information retrieval that we can arrive at an accurate portrayal of organizational memory functions in action. In this section, we present results from two different decision chain analyses of the steps that respondents went through while planning and if/how aspects of organizational memory were engaged. The first graphic reflects an attempt to model decision chains across multiple individuals (n = 7) at a more coarsely grained level, and the second graphic is a more detailed analysis of three individuals where aspects of the sociocultural and organizational context are included.

Decision chains in the aggregate

Eighteen respondents provided clear indications of the temporal process of their planning, and the most common ‘chain’ of planning steps involved two themes: looking through old lecture notes and updating these notes. These themes were reported by a total of seven respondents, either as a direct link between the two themes (4 respondents), an indirect link between the two themes (2 respondents), or only as a reference to a single theme (1 respondent). Examining these seven respondents’ planning procedures, it is possible to see the aggregate patterns in planning and the degree to which different repositories and/or retrieval processes were involved (see Figure 3).

Decision chains of course planning processes for seven faculty.

Full size image

The figure depicts the processes whereby individuals retrieved curricular information from the retention structure while planning their classes. Each decision chain (i.e., sequence of boxes) represents this process for an individual instructor as depicted by his or her code in the study sample. When the decision chain component could be clearly linked to one of the information repositories previously identified, it is outlined in thick, black lines. Within each box is one of the 75 decision steps identified in the analysis of the entire sample.

For example, respondent C01 (a biologist teaching a genetics course) reported the following steps in her planning: ensuring materials were in place, looking through old notes, and then making sure class activities were ready. Respondent C11 (a chemist teaching a general chemistry course) reported a more complex sequence of steps but shared the common theme of looking through old notes. Indeed, each of the seven respondents depicted retrieved curricular information that was stored in the curricular artifacts repository—lecture notes. These artifacts were either the result of prior experience teaching the course or were inherited from prior instructors. As a result, this process implicates two other aspects of the retention structure: individual memory (personal experience of the course) and social networks (experienced colleagues)^f.

These results highlight the importance of preexisting curricular artifacts in shaping how some faculty in the study sample plan their courses, but that this process of artifact adoption is often not one of simply ‘plugging and playing’ old notes with no adaptations. As one biology faculty described this process:

“Since this is my first semester teaching I took many colleagues’ advice, which is to start with somebody else’s notes. So I start with those, and then I take a look at the homework assignment, look at the book and see what the students will be asked to do, and compare that to my notes. At that point I revise, maybe introduce new examples or replace examples. And then it becomes a question of how am I going to present that content? And to personalize the teaching for myself, hoping that the way I understand it offers an alternative way for them to see it as well.”

This quote illustrates how a single instructor fine-tunes existing notes by retrieving information from existing notes to update them while also drawing upon other resources, such as the textbook and beliefs about student learning. Importantly, as discussed above, all seven respondents depicted in Figure 3 referenced the core process of looking through notes to fine-tune them as a sort of habituated or automatic practice. This behavior was described by many, especially those who had taught for several years, as part of their regular routine for preparing a course or individual classes. Finally, it is worth noting that for two respondents (C16 and C18), the final stage of their planning process included altering their lecture notes immediately after the class and adding new observations about what worked or did not work, which reflects an effort to update these artifacts based on reflection about the efficacy of their own teaching practices.

Individual-level decision chains that include contextual factors

Next, we provide a more in-depth analysis of three individual’s decision chains (see Figure 4).

Decision chains and salient contextual influences for three faculty.

Full size image

In this graphic an assortment of personal, sociocultural, and contextual factors are included to provide some background to each decision chain. For example, the relatively simple decision-making process of respondent C01 becomes more nuanced when her other considerations are included (e.g., time constraints and availability of local professional development). While the three individuals depicted here are certainly not representative of their departments or institution, it is notable that each reported past experiences teaching the course, local professional development programs, and cultural norms of peers as salient influences. When referring to cultural norms, each of the respondents identified two groups: one group of faculty supportive of pedagogical reform and one group that were either unsupportive or uninvolved in such activities. Each respondent noted that the presence of a supportive group was an important factor in their decision-making by acting as a source of information as well as a support system.

It is important to note that each individual underscored the fact that no single factor influenced or determined their planning behaviors; rather, a complex web of factors influenced their behaviors. This fact highlights the nonlinear and complex nature of decision-making processes, as well as challenges inherent in discerning meaningful implications from such data. Finally, it is also notable that no respondent described organizational procedures or policies that governed the existence of continuous improvement mechanisms that would require faculty to reflect upon their own teaching. The existence of such systems is a key facet of facilitating organizational change and learning (Walsh and Ungson [1991], Kezar [2005a]). The only evidence of such mechanisms in place were at the micro-level, where individual faculty took it upon themselves to engage in reflective practice at the conclusion of each class period, making notes on their curricular artifacts about what went wrong and what worked.

Effects of the USE project on faculty engagement with organizational memory

Finally, we consider the influences of the USE project on the elements of the retention structure and subsequent information retrieval processes. By examining organizational memory functions in places where reforms are being attempted, we can illuminate local practices and procedures that influence the success of these initiatives, and we can track changes in the organizational memory as a way to assess the depth and sustainability of reform efforts. Of course, definitive claims about the effects of the initiative on faculty behaviors cannot be made here, given the nature of the data. That said, we suggest that effects based on respondent self-reports are useful in highlighting whether and how STEM education reforms are targeting organizational memory functions (see Figure 5 below).

Effects of the USE project on information repositories.

Full size image

Individual memory/perspectives

Seven respondents reported that they developed new learning goals for their courses as a result of the USE project. Interestingly, six individuals had participated in the project while the other one was influenced second-hand by a colleague. Once learned, these goals are stored in the memories of the instructors as they plan and teach their courses. Similarly, six respondents reported that they altered their personal objectives and rationale for teaching in general (not for a specific course) as a result of their involvement in the USE project. This change came about through increased knowledge and appreciation of the learning theory and the educational literature.

Cultural norms and practices

Respondents claimed that the USE project influenced two features of cultural life related to curricular decision-making. First, 12 respondents reported that the expectations and norms related to teaching and learning evolved from an almost exclusive focus on research to one that also acknowledged the value of teaching. These respondents attributed this shift in cultural norms in part to the USE project, as well as other campus-wide initiatives focused on teaching and learning. Second, one respondent noted that the USE project’s efforts led directly to a change in how some of her colleagues viewed the canon of their discipline regarding introductory courses. These colleagues altered their previous focus on covering a broad range of topics to accommodate an emphasis on what could be taught well during a semester-long course. Importantly, this change was not uniformly observed across the department, as some in the department pushed back and argued that covering the entirety of the canon was of primary importance.

Social network/human resources

One of the most notable effects of the USE project on organizational memory was the increase in teaching and learning resources available to the respondents. These included human resources (11 respondents) in the form of postdoctoral researchers who were widely viewed as representing a substantial increase in the knowledge base of teaching and learning within departments. Thus, through hiring and ‘planting’ these individuals within departments, the USE project immediately enhanced the local human resources for curricular information. Another impact of the project included the facilitation of a community of like-minded instructors focused on education within science and math departments (7 respondents). This community was viewed as an important resource that provided a local support system and knowledge base.

Curricular artifacts

The USE project impacted the artifactual forms in which plans were made and stored, including new course syllabi (5 respondents), packets of materials (4 respondents), assessment materials (4 respondents), and clicker questions (4 respondents). Two departments participating in the project are making a concerted effort to archive these materials for future use. As one physics faculty noted:

“That’s why I’m trying very hard to create materials that are easy for another faculty member to make sense of…. It should be obvious to them why this helps. And it should be easy for them to find and use, so we’re trying not to write 10-page instructor guides. We’re making it freely available, and we’re trying not to go overboard.”

For this instructor, the storage of the new curricular materials in physical form was a critical part of the pedagogical reform process, as she recognized that instructors coming to a new course often seek out materials from previous instructors. Yet in this department, no formal mechanism existed for archiving curricular artifacts. However, the instructor also observed that by simply storing these materials, one could not assume that a new instructor would adopt them. Instead, the person would need to be motivated to adopt these admittedly complex and demanding materials. This underscores the importance of the retrieval process and how the presence of high-quality curricular information in the retention structure may be a necessary antecedent to adoption, but not a guarantee.

External archives

The USE project influenced how five faculty drew upon an external source of curricular information—that of educational research. Prior to their involvement with the project, these individuals had little exposure to either formal learning theory or educational research in their disciplines, both of which provided a new source of knowledge upon which they drew when planning their courses.

Changes to the retrieval of curricular information

Of the seven respondents whose information retrieval practices are depicted in Figure 1, three had been engaged with the USE project (respondents C01, C16, and C18). Each drew upon preexisting materials, including lecture notes, PowerPoint slides, and clicker questions when planning their classes, and each reported that the USE project influenced the development of these materials. This included the articulation of student learning goals that became the groundwork for course syllabi and all subsequent materials such that lecture notes, slides, and clicker questions were explicitly linked to these goals. Thus, through leading faculty to articulate learning goals and embed them within their course materials, the USE project directly shaped the artifactual repositories of curricular information. When respondents began to plan future classes, they drew upon these resources. That is, the USE project did not directly influence the nature of the retrieval process itself, but instead it led to alterations in the information sources that were habitually tapped as part of these processes. It is important to note that two instructors observed that the traditional ways in which curricular information are retrieved (i.e., in an unstructured fashion by instructors who rotate in and out of courses) were significant barriers to the realization of the USE project’s goals. This was due to the lack of guarantees that the new instructors would adopt the newly created materials or have the pedagogical training to use them proficiently in the classroom.

In this paper, we presented a new approach to studying organizational change and learning at the postsecondary level that emphasizes the importance of how curricular decision information is stored and retrieved. This approach results in an account of faculty practice that offers analytic possibilities and applications for STEM education researchers as well as policymakers engaged in educational improvement.

An approach for understanding instructional decision-making in postsecondary settings

The approach described in this paper for studying faculty practice moves beyond a singular focus on individual decision-making and the role of isolated factors that shape practice to a more comprehensive accounting of organizational functions and the role of information storage and retrieval. A particular contribution of this study is the method of identifying interconnected factors that comprise decision-making in real-world settings, or what we call ‘decision chains’. It is through these chains that it becomes possible to account for the influence of multiple levels of organizational functions (i.e., micro, meso, and macro) without privileging any single level in describing how organizations ‘learn’ and change. That is, by focusing on individual-level decision-making as it occurs within the sociocultural and organizational contexts of postsecondary institutions, a more realistic and nuanced account of faculty work that accounts for the complex intersection of disciplinary, professional, and organizational influences becomes possible (Austin [1996]; Umbach [2007]). Future research in this area should explore the intersections between this perspective on organizational learning and integrative theories of practice that are widely recognized in fields such as the learning sciences, such as activity theory (Leontiev [1978]), situated and embodied cognition theory (Lave [1988]; Wilson [2002]), and distributed approaches to educational practice (Spillane et al. [2001]).

Insights into organizational learning and change processes among science and math faculty

One of the most important findings from this study is that curricular information is not stored in a single location or format. Instead, information about what and how to teach is stored in formalized systems and artifactual or physical forms (i.e., the ‘hard’ memory) as well as in informal systems and individual and/or sociocultural forms (i.e., the ‘soft’ memory). While planning for a course or a class, individuals will likely draw upon both sources of information. For example, a chemist in our study (respondent C11) drew upon both soft (i.e., memories of previous class topics and successes) and hard (i.e., old notes and topics in the syllabus) repositories of information when planning a class.

As a result, an exclusive focus on hard memory sources, such as lecture notes or course syllabi, which is possible given the ease with which hard memory sources can be identified, will overlook a significant portion of the information available to faculty as they plan their courses. That is, less tangible soft memory sources, such as cultural norms, also play a considerable role in faculty decision-making. Even entrenched beliefs and cultural norms must interact with the technical and structural features of departments and institutions, such that the hard and the soft features of organizational life and information co-exist in an interactive and even recursive dynamic.

While a variety of information repositories are accessed by faculty in practice, it is important to highlight the finding regarding the centrality of curricular artifacts in the information retrieval process. That is, the regular pattern of retrieving old lecture notes and subjecting them to fine-tuning or ‘tweaking’ is a dominant mode of planning practice. This highlights two critical features of organizational learning: the importance of artifacts and the habituated nature of many organizational routines.

First, curricular artifacts play a considerable role in dictating how faculty approach course planning (Hora and Ferrare [2013]). This suggests that once an instructor creates lecture notes, PowerPoint slides, and clicker questions, he or she is highly likely to return to them time and again. Retrieval of preexisting artifacts is understandable given the limited time that the faculty members have to prepare courses. While it is beneficial for well-designed materials to have an enduring impact on an individual’s teaching and subsequent experiences for students, poorly created materials will have a similarly long-lasting, though this time negative, impact. The role of artifacts in mediating action within complex social environments is also an area that future researchers should examine (Wertsch [1991]).

Second, the tacit and habituated nature in which curricular information is retrieved is an important factor to consider in relation to organizational learning. While the data do not provide evidence that the retrieval of information from lecture notes was automatic, the respondents did describe it as recurring and habitual. One respondent characterized the retrieval and refinement of lecture notes as ‘just something that I’ve always done’ , which shows that this common practice is a deeply ingrained and non-problematic behavior for some individuals. This finding highlights the role of habit, which can supersede or override controlled or conscious decision-making processes (Bargh and Williams [2006]). Over time, these can become internalized as unconscious decision rules or heuristics, such that, when particular situations are encountered (e.g., being unexpectedly assigned a new course), habit may dictate individual behavior (Greeno [1998]). This may be particularly true in academic settings, where time constraints on faculty press individuals to constantly seek ways to make their work more efficient. In research universities, this is particularly the case with teaching, which generally takes a backseat to research. As one respondent stated, having course materials ‘in the can’ that could be easily retrieved each semester made her teaching much easier.

Finally, we suggest that the results hint at how culture operates in postsecondary settings. In doing so, we take a broad view of culture that encompasses a variety of phenomena including physical artifacts, recurrent practices and behaviors, and tacit beliefs or understandings. Importantly, these manifestations of culture do not operate in a homogenous fashion, influencing all faculty within a department in the same manner. Instead, different configurations of these elements can have different results for different faculty. That said, the centrality of curricular artifacts in cultural practice in general, and among the faculty in this study in particular, underscores their importance and influence. In any case, we suggest that by tracing the decision chains of faculty as they interact with organizational memory functions, it is possible to illuminate certain aspects of local cultural practice.

Implications for educational improvement at the undergraduate level

Besides contributing to the theory of organizational learning in higher education, the approach described in this paper also has implications for educational improvement efforts. Specifically, we suggest that this approach can be used to complement existing metrics for program evaluation and to diagnose leverage points for new change efforts.

New metrics for studying change

So is there evidence that organizational learning occurred as a result of the USE project? Educational reform efforts traditionally gauge project efficacy by measuring particular formative and summative indicators. For an initiative such as the USE project, these could include the number of faculty involved in course transformation activities or project effects on faculty teaching practices and/or student learning outcomes. While we do not suggest that these measures are insufficient for evaluating the ultimate effects of an initiative, metrics for capturing if and how an intervention influenced the underlying mechanisms that support organizational behavior are rarely integrated into educational evaluations. That is, measures that could demonstrate that an organization has ‘learned’ or has demonstrated some degree of double-loop learning (Argyris and Schön [1974]) are rarely seen. Given the centrality of organizational memory functions in supporting faculty practice, measuring changes to how curricular information is stored and retrieved would provide insights into these matters. Thus, we consider that one of the hallmarks of the effectiveness of initiatives such as the USE project in influencing teaching and learning is its influence on the core organizational functions of memory storage and retrieval as they pertained to course planning (e.g., new course syllabi).

However, in considering the effects of specific projects on these memory functions, it is important to remember that these functions are the result of a complex array of factors, including departmental history and tradition, institutional policy, disciplinary expectations, and so on. While a single initiative such as the USE project could be viewed as influencing or affecting organizational memory functions, it will be extremely difficult to attribute changes in these processes to any single factor. For example, the USE project was not the only pedagogy-related initiative underway at this particular institution. Other factors that may have contributed to changes in memory functions included supportive departmental leadership that predated the initiative. That said, through the development of new curricular artifacts via course-specific ‘packets’ that included lecture notes, PowerPoint slides, clicker questions, and anticipated student misconceptions, the project targeted a particularly influential leverage point that has the potential for strongly influencing faculty planning and teaching practices. With additional attention paid to making these materials user-friendly, given the likelihood of new instructors coming into the course on a rotational basis, the project also paid attention to the critical support systems that promise to assist faculty as they go about their daily work.

Diagnostics for identifying leverage points

An organizational learning framework sheds light on specific organizational processes and elements that strongly shape how and why faculty do what they do. Such descriptions can act as a diagnostic account of curricular decision-making that organizational leaders can use to support or alter particular leverage points (e.g., policies governing course rotations) that can lead to educational improvement (Spillane et al. [2001]). In other words, prior to planning and implementing a reform initiative, educators would be well served to first obtain a diagnostic snapshot of local practice or what Umbach ([2007]), p. 264 calls a ‘roadmap’ that leaders should use to guide their efforts.

We suggest that because organizational memory plays a critical role in instructional practice, efforts to improve teaching and learning will need to engage or otherwise alter these organizational processes in order to meet their goals. For example, if preliminary scans of a department targeted for change indicate that faculty within that department habitually use catalogued PowerPoint slides when preparing their courses, then, efforts to improve teaching would be well served by focusing on improving these materials. Efforts that ignore these habituated practices by introducing innovations that are at odds with these practices are likely to be met with outright resistance or poor fidelity to the goals of the initiative (Rogers [1995]).

In identifying leverage points, however, we do not suggest that altering a single factor (e.g., lecture notes) will result in wholesale transformations to faculty practice. Indeed, Lattuca and Stark ([2009]), p. 319 argue that:

A considerable body of evidence from studies of curricular and organizational change in higher education suggests that change efforts that draw from several models, anticipating and addressing as many factors as possible, stand the best chance of success.

Along these lines, we argue that a theory of change that integrates planned change efforts (i.e., top-down) while allowing for local adaptation (i.e., bottom-up) may be useful in light of the results (Austin [2011]). Specifically, we suggest that a potentially useful approach may target oft-used curricular artifacts for regular updating, as well as the introduction of systems for facilitating regular faculty reflection on their own teaching (i.e., continuous improvement systems). Indeed, the absence of any institutional mechanism for encouraging faculty to engage in reflective practice relegates the prospects of organizational learning or deep-seated change regarding course planning to happenstance. In the two cases where such systems were observed among the study samples (C16 and C18), their practice of taking post-class notes about what did or did not work was entirely self-motivated. However, the fact that these practices were generated ‘from the ground up’ highlights the importance of allowing faculty substantial local control over the change process.

This idea is based on the notion that successful interventions are those that are well aligned with and responsive to features of existing practice, rather than efforts that completely ignore the daily realities of faculty work (e.g., Fishman [2005]). Recent evidence shows that some STEM faculty are resistant to the efforts of teaching-related reforms because of a top-down approach to change, which highlights the importance of paying close attention to the existing features of faculty work (Henderson and Dancy [2008]). That said, planned change models that allow for some local adaptation yet are based on some external set of ideas or best practices have been shown to be effective (Lindquist [1978]; Kezar [2001]).

As such, specific features of organizational memory should not be viewed as determinants of faculty behavior but as factors that individuals negotiate while planning. In other words, memory functions do not ‘cause’ teaching behaviors in the sense of a unidirectional notion of causality, but they are instead an integral feature of behavior that includes individuals’ navigation of their sociocultural and organizational milieu (Greeno [1998]; Martin [2011]). This distinction is important because educators and policymakers would be well served by appreciating the complexity of organizational functioning while also being attentive to those factors that seem to play an outsized role in how faculty go about planning and teaching courses.

^aWhile qualitative research using diffusion of innovations theory could yield similarly rich insights into faculty practice and the contextual factors that support or inhibit particular behaviors, an organizational learning framework has the advantage of illuminating certain organizational systems (i.e., memory functions) that are known to play a critical role in both individual-level practice and organizational change over time.

^bThis focus on decision chains builds on prior work exploring faculty decision-making in regard to instructional technology use (Hora and Holden [2013]), faculty use of course syllabi while planning (Hora and Ferrare [2013]), and curriculum design processes (Hora [2014]).

^cSee also Schoenfeld ([2000]) for an example of tracing instructional decision-making at a fine-grained level.

^dIt is important to note that individual memory encompasses a wide range of topics, and we interpret this category to include beliefs about teaching and other individual attributes. We also recognize that while some may object to an individual’s memory being included as a repository of organizational information due to its intangible and ambiguous nature, we do so in accordance with research on organizational learning and memory that emphasizes the important role that experiences, rules, traditions, and beliefs (all forms of information potentially salient to a given task) held by individuals plays in decision-making.

^eGiven that these five disciplines do not represent each of the disciplines encompassed within the ‘STEM’ acronym, when referring to the dataset reported in this paper, we use the term ‘science and math’ disciplines. When referring to the broader set of disciplines that are the focus of this journal, we use the term ‘STEM.’

^fWhile these repositories do implicate other information sources (i.e., individual memory and social networks), they were not included in this particular analysis for two reasons: (1) the other sources were not explicitly mentioned by respondents during the interview but were only inferred by the analysts, and (2) including these additional nuances would render the graphic unreadable. To capture these more nuanced aspects of decision chains, see Figure 2.

Intro script

Thank you very much for taking the time to talk with me today and for letting me observe your class. As the email mentioned, we are studying course planning and classroom teaching at the undergraduate level in your field, and the interview will focus on how you went about planning the class that I’ll be observing this week.

Before we start I have some consent forms for you to sign—one for the interview and the other for the observation. Thanks. And is it OK if I record this conversation?

Grand tour/warm-Up

Q1. Can you please briefly tell me about your background and current position?

Planning freelist

Q2. Please list all of the things that come to mind that influence instructors as they plan a specific lecture or class at the undergraduate level, using only single words or phrases. For example, the things that influence how a tomato grows could be sunlight, good soil, fertile seeds, and so on.

Planning for the observed class—critical decision method

Q3. Now I have a few questions about your planning process for the class I’ll be observing on [fill in day]. I’d like you to think about how you went about preparing for this class, starting as far back as you can in the planning process, and walk me through the exact steps that you went through, paying particular attention to specific points at which you made important decisions about things like course content and/or teaching strategies-activities.

(If not volunteered, probe for when exactly the planning took place).

[Take detailed notes of the process being laid out, and identify 1–2 decision point that addresses the selection of a particular teaching method or strategy. These points will be the foundation for subsequent probes. If respondent answers probes below for the 1st teaching strategy-activity rapidly, then ask the same probes for the 2nd teaching strategy-activity].

Now I’d like to ask a few follow-up questions about specific aspects of your planning process.

Q4. Did considerations of departmental or colleagues expectations for your work factor into this decision, and if so, how?

Q5. Did considerations of the type of students in the course factor into the decision?

Q6. Did considerations of the course syllabi factor into the decision?

Q7. When you decided to [name teaching strategy-activity] were you drawing on any prior experiences when making this decision?

Q8. When you decided to [name teaching strategy-activity] were you drawing on any particular knowledge base about teaching or learning when making this decision?

Q9. When you decided to [name teaching strategy-activity] did you have any specific goals for students at the time, and if so, what were they?

How do you know if you’ve achieved these goals?

[Repeat for 1 more teaching strategy-activity if time; e.g., if only 13–15 minutes have elapsed].

Focus on cultural forces

Q10. How, if at all, does the culture of your department and/or institution influence how you plan for this class—especially what you teach and how in this class?

[IF interview is moving quickly, probe for any interesting responses here].

Q11. How, if at all, does your particular position (as lecturer, tenure-track faculty, etc.) influence how you plan for this class?

Plan format

Q12. What is the physical format of your plan? (e.g., paper, PowerPoints, etc.). [at this point, if plan is in hard-copy format, ask if you can take a picture or otherwise get a visual/copy of the artifact; if slides, then ask for copy of slides to be emailed to you].

Professional development

Q13. Have you participated in any teacher training or professional development in the past 2 years? If so, why? If not, why not?

Q14. How, if at all, has your participation in professional development (or the USE project) influenced how you plan and teach?

Q15. What, if any, aspects of your department or institution have supported (or discouraged) changes to your teaching?

Q16. What, if any, aspects of your department or institution have supported (or discouraged) changes to teaching in general in your department?

Do you have any additional points you’d like to make about any of the questions I’ve asked?

Thank you very much for your time.

The authors thank anonymous reviewers for their critiques and insights, as well as Joseph Ferrare and Amanda Oleson for helping to collect and analyze the data reported in this paper as part of the Culture, Cognition, and Evaluation of STEM Higher Education Reform (CCHER) study. The CCHER study was supported by the National Science Foundation Award # DRL-0814724. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Affiliations

1. Wisconsin Center for Education Research, University of Wisconsin-Madison, Madison, 53706, WI, USA
   • Matthew T Hora
2. Ethnography and Evaluation Research, University of Colorado-Boulder, 580 UCB, Boulder, 80309-0580, CO, USA
   • Anne-Barrie Hunter

Corresponding author

Correspondence to Matthew T Hora.

Competing interests

The authors declare that they have no competing interests.

Authors’ contribution

Both MH and AB conducted interviews with faculty at the study site. MH collected these and other data as part of a larger research team based at UW-Madison. MH conceptualized the theoretical framing of the analysis and conducted the literature review for this paper. MH and AB collaborated on data analysis and identifying implications from the results. All authors read and approved the final manuscript.

Reprints and Permissions