Engineering pedagogical content knowledge: examining correlations with formal and informal preparation experiences

Abstract

Background

Developing pre-service educators’ content and pedagogical knowledge is critical for providing high-quality instruction in science, technology, engineering, and mathematics (STEM) disciplines. Specifically, pedagogical content knowledge (PCK) has been identified as one of the most critically needed research areas within engineering education. However, limited research exists on PCK in engineering education contexts. Therefore, this study investigated whether specific teacher preparation coursework and informal educational experiences influenced high school instructors’ teaching of engineering content and practices.

Results

Using methods similar to a previous study examining technology and engineering educators’ teaching of science content and practices (Love & Wells in International Journal of Technology and Design Education 28:395–416, 2018), this study utilized a random sample of 55 Foundations of Technology and Engineering (FoTE) educators from 12 county school systems in the United States. The participants completed the TEES-PCK survey (Love in The Journal of Technology Studies 41: 58–71, 2015), which collected data about their formal and informal preparation experiences. Based on participant responses, eight educators were purposefully selected to be observed while teaching the same FoTE lesson. The observed teaching of engineering content and practices for these eight educators were assigned a rating using the reliable and validated RTOP instrument modified by Love et al. (Journal of Technology Education 29: 45–66, 2017). The TEES-PCK survey data and teaching observation ratings for the eight educators were analyzed using an exploratory correlational design. Spearman’s rho tests were used to examine the strength of the relationship between specific formal or informal preparation experiences and their teaching of engineering content and practices. The data were validated through corroboration with FoTE curriculum content analyses, classroom audio recordings and notes, and interviews. The analyses found several formal and informal preparation experiences significantly correlated with participants’ teaching of engineering content and practices.

Conclusions

This study presents recommendations for informing the preparation of educators to teach engineering content and practices in greater depth. The findings provide implications for educational researchers, teacher preparation programs, and in-service professional development efforts. This study contributes to the limited yet essential research area of engineering PCK.

Introduction and background

This study investigated to what extent pre-service and in-service preparation experiences influenced secondary level educators’ pedagogical practices and teaching of engineering content. It specifically examined educators teaching the International Technology and Engineering Educators Association’s (ITEEA) Engineering byDesign Foundations of Technology and Engineering (FoTE) curriculum. The findings, implications, and recommendations from this study focus on the preparation of educators and teaching of engineering concepts at the secondary level. The definition of content knowledge is an educator's detailed knowledge of facts, concepts, theories, and principles corresponding with their subject domain. Pedagogical practices can be defined as an educator’s thorough knowledge and application of teaching and learning methods, theories, and philosophies to guide individual student learning (Berisha & Vula, 2021). The literature indicates that the synthesis of content knowledge and pedagogical practices is the foundation for developing pedagogical content knowledge (PCK) (Ball et al., 2008; Loughran et al., 2001). Shulman (1987) defined PCK as “that special amalgam of content and pedagogy that is uniquely the province of educators, their own special form of professional understanding” (p. 8) developed over the educator’s career. PCK is “the blending of content and pedagogy into an understanding of how particular topics, problems, or issues are organized, represented, and adapted to the diverse interests and abilities of learners, and presented for instruction” (Shulman, 1987, p. 8). The concept of PCK has become synonymous with high-quality teaching and improved student learning (Darling-Hammond, 2000; Love, 2013). Therefore, quality teaching involves more than content knowledge; it consists of the educator having both in-depth content knowledge and the ability to transfer that knowledge to students in practical ways that are meaningful and accessible for each student.

The literature has well documented the desire, need, and ambition to improve science, technology, engineering, and mathematics (STEM) education (Kelley & Knowles, 2016; Kelley et al., 2021; Parker et al., 2016). The context of knowledge and practices inherent in each of the individual STEM disciplines are complex. Thoroughly integrated STEM is even more complex and dynamic due to the interconnectedness of STEM content knowledge and practices (Chai et al., 2019; Kelley & Knowles, 2016; Kelley et al., 2021; Wells, 2016). The complexity of this interconnectedness has been witnessed in the United States where current P-12 standards documents call for the integration of engineering concepts in science and mathematics courses. The Next Generation Science Standards (NGSS) elevated engineering content and practices as critical components of science instruction, placing increased expectations on science educators. Moreover, engineering has been viewed as a context for students to apply mathematical ideas and practices to address major societal challenges as called for in the Common Core State Standards: Mathematics Standards (Lau & Multani, 2018). Other countries have relied on science and mathematics educators to teach engineering concepts due to the shortage of engineering educators (Love & Love, 2022). One study in Australia reported that 84% of educators teaching technologies courses (encompassing engineering topics) were from other content areas (DATA Australia, 2019), while a study in New Zealand found that 68% of schools had hired out of content area educators to teach technologies courses (Reinsfield & Lee, 2021). Expectations placed on secondary science and mathematics educators to integrate engineering content and practices with limited to no preparation in teaching engineering (Lau & Multani, 2018; Love & Wells, 2018), assumes these educators know how to meld engineering content and practices with authentic science or mathematics contexts. Furthermore, providing in-depth integration of engineering content and practices within authentic science or mathematics contexts would require greater PCK development in science and mathematics educators’ core content area as well as engineering (Lau & Multani, 2018).

As alluded to in the previous paragraph, the transformational process for educator preparation programs to attempt to adequately prepare pre-service educators with the content and pedagogical knowledge to seamlessly integrate a myriad of STEM concepts poses significant challenges (Hughes & Partida, 2020). STEM integration involves a broad set of skills and abilities, including inquiry, problem-solving, design, systems thinking, modeling, cognitive, metacognitive, and more (ITEEA, 2020; NGSS Lead States, 2013). The plethora of knowledge and practices innate to integrated STEM is profound and would require educator preparation programs to deliver ambitious instruction focused on developing a vast range of educator content and pedagogical knowledge as a precursor to PCK development (Hughes & Partida, 2020). Although the teaching of STEM concepts provides ample interdisciplinary opportunities, the notion of developing an educator’s PCK in great depth across all STEM disciplines and topic-specific areas they will encounter in their teaching is likely not feasible (Rose et al., 2015). Wells (2008) proposed the collaboration of educators that have content and pedagogical expertise directly related to the lesson being delivered is a more feasible method for teaching integrative STEM concepts in greater depth.

Engineering educator preparation and experiences

As highlighted in the Standards for Technological and Engineering Literacy (STEL), engineering educators’ unique blend of content and pedagogy is crucial for helping students become more technological and engineering literate (ITEEA, 2020). The STEL specifically discuss the need for engineering educators to develop unique pedagogical skills to integrate core technological and engineering content and practices meaningfully within authentic contexts, “The technology and engineering contexts and practices provide comprehensive details about the unique pedagogies used in technology and engineering learning environments” (ITEEA, 2020, p. 6). PCK literature concludes that when an educator’s content and pedagogical knowledge lacks depth, so does their PCK (Kind, 2009). Correspondingly, increasing educator content and pedagogical knowledge are critical to developing PCK and developing quality engineering educators (Jones et al., 2021). However, pre-service and in-service engineering educators in various countries have experienced a limited focus on content and pedagogical knowledge development, highlighting the importance of having a high level of content and pedagogical knowledge before initial entry into the teaching profession (de Vries et al., 2016; Jones et al., 2021; Litowitz, 2014).

Throughout the integrated STEM movement, P-12 educator preparation programs that were once solely focused on science, technology, engineering, or mathematics started rebranding themselves as proclaimed STEM preparation programs without changing much of their required course content or foci (Fantz & Katsioloudis, 2011). The conflation of once separate disciplines with integrated STEM might support the rhetorical task of programs and educators convincing others that the skills developed in these programs are valuable for all students. Wells (2008) indicated these newly rebranded STEM programs highlighted the need for a profoundly broad-spectrum change in P-12 education and preparation programs of educators in STEM content areas. He cautioned that if programs attempted to prepare what they deemed to be STEM educators, significant changes were needed to focus on enhancing educators’ PCK in a myriad of topic areas that could contribute to integrative STEM instruction.

The value placed on P-12 engineering education has increased with considerations that it serves as pragmatic model to provide authentically integrated STEM teaching and learning experiences (Chai et al., 2019; Kelley & Knowles, 2016). Rose et al. (2015) indicated that there are increasing expectations for engineering educators to “charge the engineering and science pipeline … [to] maintain a competitive edge in the world marketplace” (p. 4). These expectations include engineering educators applying science and mathematics concepts, principles, and processes to enhance an engineering pathway (Rose et al., 2015). Based on these increased expectations, current science, mathematics, and technology educators face challenges in offering students high-quality engineering experiences (Denson & Lammi, 2014; Hughes & Denson, 2021). Even with the challenges associated with providing high-quality engineering education experiences, engineering has been proposed as an ideal way to deliver integrated STEM for all students (Kelley & Knowles, 2016; Kelley et al., 2021; National Academy of Engineering & National Research Council, 2009). Developing engineering-specific PCK is critical for preparing high-quality engineering educators who can deliver rigorous engineering instruction (De Miranda, 2018; Gumbo & Williams, 2014; Phillips et al., 2009).

Litowitz (2014) indicated that technology education programs were making efforts to transition to an engineering focus (design is also frequently included). Even prior, but especially with the release of the Standards for Technological Literacy in 2000, technology education programs addressed engineering concepts (Rose et al., 2015). The later release of the STEL (ITEEA, 2020) more thoroughly addressed technological and engineering knowledge, practices, and habits of mind to be authentically developed in students, which will aid in developing technological and engineering literacy in more students (ITEEA, 2020). The STEL promoted engineering to have a more standards-based focus for engineering educators and related educator preparation programs.

Nevertheless, pre-service educator preparation programs often do not focus primarily on content knowledge, pedagogical knowledge, and PCK, despite the indicated importance (Jones et al., 2021; Litowitz, 2014; Rose et al., 2015). For example, corresponding with Jones et al. (2021), Litowitz (2014) indicated that technology and engineering educator preparation programs focused primarily on pedagogical knowledge and lacked focus on developing content knowledge. Additionally, Rose et al. (2015) suggested that technology and engineering educator preparation programs might not require a sufficient number of science and mathematics courses to adequately develop pre-service educators’ content knowledge for integrating STEM concepts.

Evidence has been presented in the literature suggesting that educator’s depth of content knowledge influences their depth of PCK (Ball et al., 2008; Graber, 1995; Kind, 2009; Loughran et al., 2012). PCK literature concludes that when an educator’s content knowledge lacks depth, so does their PCK (Kind, 2009). Love and Wells (2018) discovered that the extent to which technology and engineering educators integrated science concepts in their instruction correlated with previous science content learning experiences. Furthermore, pre-service engineering educators experience a limited focus on developing content knowledge in their educator preparation programs (Jones et al., 2021; Litowitz, 2014). This lack of content knowledge development indicates the need for greater content knowledge before initial entry into educator training or the need for educator preparation programs to focus more heavily on developing this content knowledge. Correspondingly, increasing educator content knowledge is critical to developing PCK and developing quality educators (Jones et al., 2021). Developing PCK is essential for preparing high-quality educators who can thoroughly deliver rigorous and meaningful integrated STEM instruction (Love, 2013; Love & Wells, 2018).

Engineering educators’ ability to thoroughly engage in engineering contexts and practices during educator preparation, in turn, helps them translate these into student experiences. Engineering educator preparation programs can better develop PCK by bridging the gap between pre-service educators’ learning of content knowledge and pedagogical practices (Rose et al., 2015; Vossen et al., 2020). Addressing pre-service educators’ content knowledge gaps will require effectively designed and implemented professional development and pre-service educator preparation (Hughes, 2017; Hughes & Partida, 2020). As Litowitz (2014) discovered, technology and engineering educator preparation programs are not standardized. This non-standardization of technology and engineering educator preparation programs does not make the practice of connecting content and pedagogical knowledge self-evident (Litowitz, 2014).

Furthermore, the continuous evolutionary nature of technology and the dynamic nature of the technology and engineering education discipline makes it challenging to continually identify the content knowledge competence that engineering educators will require to develop PCK adequately (Rose et al., 2015). For example, suppose the field of engineering education is expecting educators to integrate STEM thoroughly. In that case, educator preparation programs will need to focus on modeling rigorous integrative experiences for educators to develop adequate content knowledge and practices related explicitly to the purposeful blending of science, technology, engineering, and mathematics during unique learning experiences that happen in engineering learning environments (ITEEA, 2020; Kelley et al., 2021; Love & Wells, 2018).

Pedagogical content knowledge research

Previous PCK studies have utilized many methods and instruments to better understand and capture what has been described as an often complex area to assess (Love, 2013; Settlage, 2013). The difficulty in assessing PCK relates to its complex nature. PCK is more complex than Shulman originally implied (Kind, 2009). It can fluctuate based on several related items, including the individual, the content, and the circumstances or context. As the educational environment constantly changes, an educators' PCK should also adapt based on various factors. One of those factors is the content topic within a discipline, as PCK studies have proposed that PCK is topic-specific. Educators need content knowledge specific to specialized areas within engineering and pedagogical knowledge to deliver effective classroom instruction (De Miranda, 2018; Phillips et al., 2009). Specialized PCK in each engineering topic area results from the educator blending unique content and pedagogical knowledge. Similar to Phillips et al. (2009) and De Miranda (2018), Gumbo and Williams's (2014) research concluded that PCK is “individual, unique, varies from class to class and changes over time” (p. 487). Reflective of these findings, the STEL acknowledges the need for specialized pedagogical knowledge to integrate core technology and engineering concepts meaningfully within emerging authentic engineering practices and contexts (ITEEA, 2020, p. 6).

Recently, more PCK research efforts are shifting to investigate PCK through an integrative lens. With current standards documents (ITEEA, 2020; NGSS Lead States, 2013) calling for crosscutting teaching and learning, studies have started to examine the influence of preparation experiences and teaching methods on educators’ science and engineering PCK. Love and Wells (2018) used a mixed-methods approach to triangulate data from a survey about preparation experiences, teaching observation ratings, interviews, and a curriculum content analysis in examining technology and engineering educators’ teaching of science content and practices. Love and Wells (2018) found that technology and engineering educators rated much lower in their teaching of science content and practices than technology and engineering content and practices. Contrary to previous PCK studies, Love and Wells’s research did not find a significant correlation between teaching experience and educators’ observed teaching ratings. However, they did find that numerous formal and informal preparation experiences were significantly associated with educators' teaching of science content and practices embedded within a technology and engineering lesson. Experiences such as the number of high school science courses completed (especially physics), completing undergraduate robotics or physics courses, mentoring a technology and engineering educator, serving on a science education committee, collaborating with other technology and engineering educators, delivering discipline-specific in-service sessions, and not helping with after school technology and engineering clubs were all found to increase educators' teaching of science content or practices significantly. This research suggested that utilizing meaningful and influential preparation experiences can enhance educator preparation programs and school systems' efforts to improve the interdisciplinary quality of engineering instruction.

Hughes and Partida (2020) presented research on educator professional development designed to improve metacognitive awareness, specifically pre-service STEM educators’ knowledge and regulation of cognition. The three dimensions of the NGSS, including crosscutting concepts and science and engineering practices within the disciplinary core ideas, formed the foundation of the 5-week-long educator professional development (Hughes & Partida, 2020). NGSS requires educators to shift their practices from teacher-centered to student-centered to improve student learning. Implementing NGSS requires educators to have domain-specific science and engineering PCK, which is necessary to adapt their pedagogical practices “to allow students to practice and apply the range of skills that scientists and engineers use when engaged in inquiry and problem solving” (Hughes & Partida, 2020, p. 6). Educator preparation programs and professional development providers will need to focus on effective methods for developing science and engineering PCK, namely developing educator content knowledge and pedagogical practices (Hughes, 2017; Hughes & Partida, 2020).

Most research on technology and engineering PCK has been conducted outside of the United States (Love, 2013). For example, Jones and Moreland (2004) found that case study reflections, workshops, educator agreement meetings, and student portfolios increased the PCK levels of technology and engineering educators in New Zealand. Niiranen et al. (2020) indicated the importance of developing reflective practitioners in technology, engineering, and design education to develop an educators’ PCK. Additionally, Jones and Moreland (2004) discovered a connection between increased PCK and gains in technology and engineering knowledge, pedagogy, and student learning. Another study out of New Zealand found that technology and engineering PCK positively impacted student learning, specifically design and procedural knowledge (Fox-Turnbull, 2006).

Furthermore, Williams and Lockley (2012) investigated the utility of the Content Representation (CoRe) instrument borne out of science education to examine the PCK of early-career science and technology (encompassing engineering) educators in New Zealand. They determined that the CoRe helped develop educators’ procedural and conceptual knowledge related to their PCK. Later research by Hynes (2012) examined the PCK of Project Lead the Way (PLTW) educators. Hynes (2012) revealed increased teaching experience was associated with educators’ ability to connect mathematics and science concepts within a PLTW engineering unit, implementing a more student-centered approach, and greater student engagement.

In an attempt to address concerns about commonly used time-consuming PCK data collection methods, Rohaan et al. (2011) explored the feasibility of using a multiple-choice instrument to examine the science, technology, and engineering PCK of Dutch primary school technology and engineering educators. Their Teaching of Technology Test (TTT) presented promising results for analyzing educators’ ability to convert subject matter into practical activities, predict teaching behavior and PCK reasoning, and demonstrated the potential to include larger samples in PCK research to provide more generalizable conclusions. However, while the TTT was found to have strong reliability measures and addressed previous researchers' concerns about the time constraints and small sample sizes associated with PCK research (Abel, 2008), its depth of investigation into observed teaching practices (a core component of PCK) has been questioned (Love, 2013; Love & Wells, 2018).

Although the literature on PCK in engineering education is limited compared to other content areas, especially in the U.S., the literature indicates that PCK is topic-specific and a critical component of teaching. The literature also demonstrates a strong connection between preparation experiences and educators' PCK, further emphasizing the importance of researching this area to continually improve engineering pre- and in-service preparation efforts, especially with the recent release of new standards (ITEEA, 2020). Teaching experience was also commonly associated with increased PCK levels in the literature. What remains unclear from the literature is a consensus about the best methods and instruments to analyze educators' PCK. While there have been a number of promising PCK studies conducted, this is an area in need of additional research to mirror the greater emphasis placed on this topic within science and mathematics education research (Love, 2013; Love & Wells, 2018; Love et al., 2017). Examining the preparation factors that improve PCK and the teaching of engineering content and practices is essential given current educational initiatives spurred by standards documents calling for the teaching of engineering content and practices in science (NGSS Lead States, 2013), technology, and engineering education (ITEEA, 2020) (Kelley et al., 2021; Lau & Multani, 2018; Love & Wells, 2018).

A major limitation of the empirical evidence in this line of experimental studies is that most have been conducted in science and mathematics education settings and not engineering education settings. Additionally, PCK was identified as one of the most critically needed research areas within engineering education (Martin & Ritz, 2014). However, limited research exists on engineering PCK (De Miranda, 2018; de Vries, 2015; Doyle et al., 2019; Gumbo & Williams, 2014). Conversely, there has been an extensive amount of notable research investigating PCK within science education (Abel, 2008; Gess-Newsome & Lederman, 2002; Hume & Berry, 2011; Kind & Chan, 2019; Loughran et al., 2012; Magnusson et al., 1999) and mathematics education (Alamri et al., 2018; Hill et al., 2008; Krauss et al., 2008; Manizade & Mason, 2011).

Research questions

The following research questions (RQ) were derived from the literature and used to guide this fully integrated mixed-methods study (Teddlie & Tashakkori, 2006) to examine the teaching of engineering content and practices within a FoTE lesson:

RQ1: To what extent did teaching experience inform educators' teaching of engineering content and practices?

RQ2: To what extent did formal preparation experiences inform educators' teaching of engineering content?

RQ3: To what extent did informal preparation experiences inform educators' teaching of engineering content?

RQ4: To what extent do formal preparation experiences inform educators' teaching of engineering practices?

RQ5: To what extent do informal preparation experiences inform educators' teaching of engineering practices?

Methodology and procedures

This study utilized a fully integrated mixed-methods design (Teddlie & Tashakkori, 2006). The quantitative and qualitative data were mixed at all levels, from the conceptualization to the inferential stages. From their experience with PCK research, Williams and Lockley (2012) recommended that future PCK studies focus on common themes taught in engineering education related research. For this reason, the teaching of a FoTE lesson was selected to ensure all educators were observed teaching the same engineering lesson, which they were expected to deliver with fidelity as members of a FoTE consortium state in the U.S.

The first analysis conducted was a qualitative content analysis (Vaismoradi et al., 2013) of FoTE lesson plans provided by ITEEA. The content analysis was deemed an acceptable method to identify emerging characteristics of the lesson content (Bloor & Wood, 2006). Due to the time of the year that the observations occurred, the FoTE units that consortium schools were covering at that point in time, and the amount of engineering content found from the content analysis, a lesson from the Design: Energy and Power unit was selected for the teaching observations. In this unit, students learned about various forms of energy and converting energy into forms utilized by humans. The students designed a plan for utilizing various forms of energy to power their community. They also learned about electrical energy and electronic components and participated in a series of labs solving electronic circuit design challenges while measuring and calculating data related to the circuits. Each researcher independently identified the core engineering and science concepts that the instructor was expected to explain and demonstrate. This was done by conducting a content analysis of the FoTE Energy and Power instructor’s guide, PowerPoint presentations, worksheets, design challenges, and assessments. The researchers then compared their lists to ensure they captured all key concepts and reached consensus on what were classified as engineering concepts or science concepts (Table 1). From this arbitration process, it became apparent that some concepts could be classified as physical science concepts interwoven within the energy and power engineering content, pending how the instructor discussed or presented them (e.g., nuclear power). This was noted and taken into account for the observations to (a) identify and verify the critical power, energy, and physical science concepts that the researcher could expect to observe during their classroom visit; and (b) help the researcher delineate between the teaching of engineering and science concepts. Additional details about the observation instrument and accompanying rubric that were used are provided in the instrumentation section.

Table 1 Engineering and science concepts identified in the lesson content analysis

Following the content analysis of the energy and power lesson plan, the researchers visited the engineering education division website from a state department of education that was an ITEEA consortium state. The website provided a list and contact information for the secondary engineering education curriculum coordinator from every county school system in that state. Next, the researchers contacted each county level supervisor requesting they send the link for the online Technology and Engineering Educators’ Science PCK (TEES-PCK) survey instrument (Love, 2015) to all FoTE educators in their school system. Twelve county supervisors sent the link to their FoTE educators, resulting in 233 teachers being invited to voluntarily participate in the survey. Fifty-five educators responded, resulting in a 24% response rate which was deemed acceptable for an online survey (Nulty, 2008). The 55 respondents were categorized according to their demographic responses into three teaching category levels (novice, intermediate, and veteran) based on the median and quartiles of their years of teaching experience (e.g., educators whose years of teaching experience fell below the median were labeled as novice). This participant categorization resulted in an almost equal distribution of educators across teaching experience categories.

Using descriptive statistics, the modes for every survey question were calculated according to each teaching experience category. These modes from each category were used to rate participants as low (below the mode), average (at the mode), or high (above the mode) for comparison within their respective experience category. After this, each low rating was converted to a score of − 1, average equaled 0, and high was + 1. This was done for every TEES-PCK survey question, and the sum of these quantitative ratings was calculated. This process allowed the researchers to identify educators with unique preparation characteristics based on the median and quartiles of the rating sums (low, intermediate, or high). Of the 55 survey respondents, four participants were identified as having low preparation factors (the lowest sum) within each experience category, and four had high factors (the highest sum) within each experience category. These eight individuals were purposefully selected for the next phase of the study, which included classroom observations of their teaching of the FoTE energy and power lesson. This purposeful selection process was utilized to examine various educators with broad levels of preparation experiences, which the literature indicated was a pivotal contributor to PCK levels (Love & Wells, 2018; Shulman & Hutchings, 2004; Williams & Lockley, 2012). A sample of eight was found to be sufficient for data collection and analysis of observations (Collins et al., 2007; Creswell, 2002; Onwuegbuzie & Leech, 2005), and that sample size was larger than those examined in some previous technology and engineering PCK studies (Hynes, 2012; Hynes et al., 2010; Jones & Moreland, 2004; Williams & Lockley, 2012).

Prior to the observation, each participant was provided with the first two questions from the interview instrument (Appendix A, questions 1–2) to examine their planning and preparation for the lesson. To analyze the classroom observations, the researchers used Love et al.’s (2017) modified Reformed Teaching Observation Protocol (RTOP), which was previously established as a reliable instrument. The researchers also utilized Love et al.’s (2017) modified RTOP rubric to rate educators’ teaching of engineering content and practices. Content ratings reflected, “the concepts, principles, relationships, processes, and applications a student should know within a given academic subject, appropriate for his/her and organization of the knowledge” (Özden, 2008, p. 634), and examined the accuracy and depth of engineering topics discussed by the teachers. It was not expected that a teacher would cover all topics identified in the lesson content analysis due to the breadth of topics within the lesson and class time constraints; rather, the observer rated each teacher on their depth of knowledge about the engineering topics presented. The ratings for practices reflected: “A set of teaching strategies and methods of instruction employed in the classroom. The interaction between the teacher and his students in order to expand their cognitive and skillful perceptions….” (Cotton, 1995, as cited in Khader, 2012, p. 77). The accuracy and depth of these practices to enhance the teaching of engineering concepts (e.g., demonstrations, simulations, explanations, etc.) were examined over breadth of practices implemented. Love et al.’s (2017) accompanying RTOP rubric helped to distinguish the depth of content and practices observed. One of the researchers, who had previously been trained and demonstrated acceptable interrater reliability using the modified RTOP, observed all eight purposefully selected FoTE educators for one class period in which they taught the FoTE energy and power lesson. This protocol was found to be valid by Lomas and Nicholas (2009), who similarly used the RTOP with one observer during a single class period. Love et al.’s (2017) modified RTOP was utilized to convert the qualitative teaching observations of engineering content and engineering practices into a summative rating for each of the modified RTOP subscales.

These numerical ratings allowed the researchers to examine relationships between instructors’ preparation experiences and their teaching by conducting Spearman’s rho analyses as described later in this section. During their teaching, the educators wore a lapel mic to increase accuracy and verify the RTOP ratings. The audio was linked to the observation notes on the researcher’s laptop to review the audio and transcript for accuracy. Directly after the classroom observations, the observer interviewed the educators to ask follow-up questions about the lesson (Appendix A, questions 3–12). Participants were later provided with the typed transcripts from their lesson audio and interview responses to verify and validate the results through member checking (Doyle, 2007). The observer then used the observation notes, audio recordings, and interviews to corroborate their RTOP ratings based on what they observed and expected to observe from the FoTE lesson content analysis. Lastly, the sums of the ratings for each modified RTOP subscale and the TEES-PCK survey data were entered into the SPSS software. The final stage of this study utilized an exploratory correlational design to examine the direction and strength of association between participants’ preparation experiences and their ratings for teaching of engineering content and practices. Due to the ordinal and nonparametric nature of the data, Spearman’s rho analyses were deemed most appropriate for examining these relationships (Sheskin, 2011) directly related to the research questions and sample of this study.

Instrumentation

As part of the fully integrated mixed methods design, the TEES-PCK survey, modified RTOP, and interview instrument were used to collect quantitative and qualitative data to address the research questions. This section describes the instruments used in this study.

TEES-PCK survey instrument

The TEES-PCK survey was created by Love (2015) in collaboration with a panel of four university faculty members who had expertise in P-12 science education, engineering education, and educational assessment. The items were modified from several demographic and teaching preparation survey items used to collect data in previous science and mathematics PCK studies (Ball & Hill, 2008; Cwik, 2012; Perez, 2013). Face validity of the instrument, which consisted of the demographic questions described below, was established by the panel of experts reaching consensus on the items. The TEES-PCK was administered online via Qualtrics and involved a series of multiple selection questions, which took participants approximately 30 min to complete. Love (2015) and Love and Wells (2018) presented a detailed description of the development of the TEES-PCK instrument and the items included in the instrument. This survey instrument comprised questions about the educator's demographics, teaching preparation program experiences, higher education coursework, and informal STEM-related experiences. It included a series of multiple selection items that also allowed for open responses, such as “How many of the following engineering content and engineering education courses did you complete in your undergraduate experience” and “How many hours per week do you participate in the following professional development activities related to engineering education.”

RTOP observation instrument

The Reformed Teaching Observation Protocol (RTOP) (Sawada et al., 2000, 2002) is a valid and reliable instrument that has been widely used to examine the reformed teaching practices of science educators. Sawada et al. (2002) found the propositional (r² = 0.769, p < 0.01) and procedural pedagogical knowledge subscales (r² = 0.971, p < 0.01) of the RTOP to have acceptable and strong construct validity measures, respectively. They also conducted Cronbach’s alpha tests and found the propositional knowledge (α = 0.80) and procedural knowledge subscales (α = 0.93) to have acceptable and strong reliability measures, respectively. Taylor et al. (2015) later demonstrated that the RTOP could serve as a reliable measure of inquiry-based teacher practice in alignment with the NGSS.

The RTOP instrument used in this study was slightly modified by Love et al. (2017) to measure four distinct areas based on the NGSS's inclusion of science and engineering content and practices. Subscale four of the RTOP, which examined the teaching of content (propositional and procedural knowledge), was duplicated to create two similar yet separate subscales to rate teaching of both engineering and science content and practices. Additionally, the term “subject matter” from the original RTOP was replaced with “content” to better align with the NGSS. This also allowed observers to better distinguish between teacher’s content knowledge and general pedagogical knowledge during observations. Love et al. (2017) defined content as essential aspects of disciplinary content knowledge, and practices were defined as behaviors that scientists or engineers engage in as they apply core disciplinary content to solve problems. Face validity of the modified items and their alignment with the NGSS were established among a panel of four university faculty members who had expertise in P-12 science education, engineering education and educational assessment.

Love et al.’s (2017) instrument was determined to be the best-suited instrument for this study, given its focus specifically on examining observed teaching of engineering content and practices. Guimarães and Lima’s (2021) analysis of the use and adequacy of 68 engineering education classroom observation protocols found in the literature between 2000 and 2020 ranked Love et al.’s modified RTOP in the top 20% of protocols. Additionally, researchers have cited the alignment of the RTOP with standards in STEM education fields as a strength (Love et al., 2017, Taylor et al., 2015). Although this instrument was created before the release of the STEL, the researchers determined that the modified RTOP also aligned closely with the focus of the STEL on core content standards and engineering practices, and could be applied within various engineering contexts (ITEEA, 2020, p. 6). Therefore, the researchers determined that no additional modifications were needed to Love et al.’s (2017) instrument.

Validity, reliability, and interrater reliability tests

In this study, the researchers only analyzed ratings from the engineering content and engineering practices subscales from section 4 of Love et al.’s (2017) modified RTOP. Using the same methods as Sawada et al. (2002), the authors conducted Cronbach’s alpha tests to examine reliability, and best fit linear regression correlational analyses to examine construct validity. The Cronbach’s alpha tests revealed strong reliability among the items in the engineering content (α = 0.906) and engineering practices (α = 0.902) subscales. The two subscales were each used in the linear regressions to predict the total score from all RTOP subscales. The engineering content (r² = 0.882) and engineering practices (r² = 0.933) subscales demonstrated strong construct validity measures. This indicated that the instrument subscales analyzed in this study were reliable and measured the criteria intended to be measured.

An example from subscale four of the modified RTOP is item 9b which helped the researchers examine the teaching of engineering content “Elements of technology and engineering abstraction (e.g., symbolic representations, theory building) were encouraged when it was important to do so” (Love et al., 2017, p. 61). An example that helped the researchers examine an educator’s engineering practices included item 11b, “Students used a variety of means (models, prototypes, drawings, graphs, concrete materials, manipulatives, etc.) to represent T&E phenomena” (Love et al., 2017, p. 61). The accompanying RTOP rubric created by Love et al. (2017) was used during the observations to help differentiate ratings for each item and maintain consistency.

Before using the instrument, interrater reliability was established through a panel of two science and engineering education specialists and one of the authors. The panelists used the instrument to independently rate a recorded FoTE lesson. During each of the first two rounds the panelists provided ratings for 10% of the modified RTOP items across all subscales. After each round they engaged in arbitration to discuss the rationale for their ratings and work toward building consensus about the rating criteria. During the third round, the panelists individually rated the lesson using the remaining 80% of the RTOP items. There was agreement on 83% of the items among the panelists, which was deemed to be an acceptable level of interrater agreement (Howell, 2007). To further examine the level of interrater agreement, an intraclass correlation coefficient (ICC) was calculated from the panelists’ third round ratings. An ICC of 0.729 indicated moderate interrater reliability (Landis & Koch, 1977). This allowed the researcher to independently use the instrument with the expectation that ratings would be consistent with those expected from the panel of specialists. Similarly, Love et al. (2017) found it appropriate to use an independent observer’s ratings from the modified RTOP after establishing an acceptable level of interrater reliability.

Interview instrument

The interview instrument by Love and Wells (2018), which was adapted from Park et al.’s (2011) instrument, was deemed most appropriate for this study (Appendix A). The original instrument by Park et al. was developed to investigate connections between PCK levels and ratings from the RTOP. Love and Wells (2018) adapted the instrument items from Park et al. (2011) to help gain further insight into the observed strategies used to teach science and engineering content and practices, and the preparation experiences that participants believed informed their teaching of science and engineering concepts. This instrument also helped the researchers delineate between observed teaching of engineering content and practices, science content and practices, and participants’ preparation experiences that informed each. Since Park et al.’s interview questions were developed in alignment with the original RTOP, Love and Wells (2018) carefully modified the items through triangulation with the TEES-PCK survey and an engineering lesson content analysis to provide more details about participants’ preparation factors and their PCK. Face validity of the interview items was established by one of the researchers and a panel of four university faculty members who had expertise in P-12 science education, engineering education, and educational assessment. Similar to Park et al. (2011), Love and Wells (2018) found this interview instrument helped to corroborate the observer’s RTOP ratings.

Findings

TEES-PCK survey

Among the 55 participants who completed the online survey, most were White (93%) males (73%) with a mean age of 43. The mean years of overall teaching experience and FoTE teaching experience were 13 and five, respectively. Approximately 84% had state certification to teach engineering education. The majority had earned either a bachelor’s degree (68%) or master’s degree (28%) in technology education (which later transitioned to technology and engineering education), and a few had earned a bachelor’s degree (11%) or master’s degree (6%) in industrial arts (IA). Slightly more than half (51%) had attended an ITEEA-led training to learn about teaching the FoTE curriculum. Among that 51%, the most common FoTE training educators attended was a whole week event (33%) instead of a shorter training.

Related to content coursework participants completed, 64% reported taking at least one physics course in high school (the least completed course among all science areas). In comparison, 77% reported taking at least one IA or technology education course. Only 27% had completed a physics course during their undergraduate studies, and 7% completed a physics course in graduate school. Regarding engineering coursework, the majority of participants had completed an electronics (53%), power, energy, and transportation (PET) (49%), or robotics (31%) course during their undergraduate studies. Approximately 80% had completed a teaching methods course in either technology education or IA. In their graduate content coursework, fewer students had completed a course in electronics (15%), PET (15%), or robotics (7%). Compared to their undergraduate experience, fewer educators reported taking a master’s level teaching methods course in either technology education or IA (67%). Additionally, 73% of the educators reported completing at least one higher education course that taught strategies for integrating science and math concepts within engineering education.

In addition to formal coursework, information about participation in informal collaborative and non-collaborative experiences was collected. For example, it was found that 58% of educators did not engage in any clubs or after-school activities. However, among those that did, 25% helped with a robotics club, and 13% helped facilitate a Technology Student Association (TSA) club. In addition, only 27% of the participants had attended a state or national engineering education conference within the past 3 years. In terms of collaboration with other educators, 36% of participants reported collaborating with other engineering educators daily. Further collaborative efforts included 42% of the participants delivered in-service engineering workshops for their school district, 45% served on an engineering education committee and 18% on a science education committee, and 73% reported being involved in an engineering professional learning community (PLC).

RTOP teaching observations

From the 55 TEES-PCK survey participants, eight educators were identified as having unique preparation experiences based on their responses being above or below the mode according to others in their teaching experience category. These eight educators were purposefully selected to participate in the classroom observation and interview phases. The classroom observation sample comprised primarily white (88%) males (88%) with a mean age of 48. The mean years of total teaching experience among the group was 18, with an average of 5 years of experience teaching FoTE. Approximately 63% had attended a 1-week training on how to teach the FoTE curriculum, while 37% reported attending a half-day training. In addition, half of the group reported completing a higher education integrated STEM methods course.

Approximately 63% completed a physics course in high school. In terms of undergraduate courses, 25% took a robotics course, 38% completed a physics course, and 88% had a methods course in teaching IA or technology education. Regarding after-school clubs, 25% helped with robotics, and 25% facilitated the Technology Student Association (TSA) at their school. Within 3 years prior to the survey, 88% of the educators had attended a state engineering education conference, and 25% had attended a national engineering education conference. For six or more hours a year, 75% of the educators reported participating in an online engineering education PLC, and 75% said they collaborate with another engineering educator at least two to three times per week.

Additionally, some participants reported delivering in-service engineering (50%) or science (25%) workshops in their school district. One of the participants was a past president of a national engineering education association and served as a writer and pilot site for the Standards for Technological Literacy. Another educator had served as a writer for the FoTE assessment items, and one other participant was a teacher effectiveness coach (TEC) to train educators about delivering the FoTE curriculum. One other unique characteristic to note is the previous work experiences of the educators. Prior to teaching, one participant was an industrial engineer, one used to own a home remodeling business and worked at an engineering firm during the summers, one previously worked in the HVAC industry, one was previously a business education educator, and another was previously a physical education educator but was drawn to the hands-on nature of engineering education. As exemplified by the descriptions above, the educators purposefully selected for the classroom observation possessed various preparation and background experiences (Table 2).

Table 2 Characteristics of observed participants

Table 3 displays each educator's ratings on the RTOP for their observed teaching of engineering content and practices during the FoTE energy and power lesson. The mean ratings indicate that participants were more proficient in teaching engineering content (13.6 out of 20) than engineering practices (7.6 out of 20). Examining these ratings more closely further highlights the divide between ratings for content and practices. For example, only three educators scored a nine or lower for engineering content, but three scored a two or lower for engineering practices.

Table 3 Observed RTOP ratings for teaching engineering content and practice

Survey and observation correlational data

Teaching experience and teaching of engineering content and practices (RQ1)

The review of literature found that many PCK studies have correlated teaching experience with higher levels of PCK. Furthermore, studies have suggested that PCK is content-specific, and the context of educators’ teaching experience is another factor that can contribute to their PCK (De Miranda, 2018; Gumbo & Williams, 2014; Phillips et al., 2009). Given the STEL call for educators to teach core engineering content and practices within various engineering contexts (ITEEA, 2020), a series of correlational analyses were performed to examine the relationship between participants’ years of teaching experience and the ratings they received for their teaching of engineering content and practices. Teaching experience was examined according to total years of teaching experience and years of experience teaching the FoTE curriculum. The correlational analyses revealed that although the relationship between FoTE teaching experience and teaching of engineering content and practices was stronger than total years of teaching experience, there was no significant relationship between years of teaching experience (total or FoTE specific) and the teaching of engineering content or practices (Table 4).

Table 4 Spearman’s rho correlation table of years of teaching experience and teaching of engineering content or practices

Findings from these analyses do not reflect what one would expect to find based on the literature. The findings indicate that more years of teaching experience (especially specific to the FoTE curriculum) was not significantly associated with greater proficiency in teaching engineering content and practices. These findings prompted further analyses to investigate if specific preparation experiences significantly influenced FoTE instructors’ teaching of engineering content and practices. Similar to previous research that utilized the TEES-PCK (Love & Wells, 2018), due to the extensive nature of the instrument and the amount of data collected, only those preparation experiences which were found to be significantly associated with the teaching of either engineering content or practices are presented to address the research questions. Additionally, the researchers also reported results from this study that reflected similar preparation experiences presented in Love and Wells’s (2018) study examining correlations between the teaching of science content and practices within an engineering lesson.

Formal experiences and teaching of engineering content (RQ2)

The Spearman’s rho tests revealed that the number of undergraduate robotics, high school physics, and undergraduate physics courses completed by educators had a strong positive correlation and were significantly correlated with the teaching of engineering content ratings from the observed lesson. This correlation indicates that the more of these aforementioned courses that one completed, the better prepared they were to teach engineering content embedded within the lesson (Table 5). Spearman’s rho analyses were also conducted on a number of other formal experiences reported on the TEES-PCK. The Spearman’s rho analyses found no significant association between the teaching of engineering content and undergraduate electronics, PET, or IA/technology education teaching methods courses. Furthermore, there was no significant association with graduate-level electronics, PET, or robotics courses.

Table 5 Spearman’s rho correlations between formal preparation experiences and teaching of engineering content ratings

Informal experiences and teaching of engineering content (RQ3)

The same analyses were conducted for RQ3 and found a number of informal experiences significantly associated with teaching engineering content. The amount of time spent helping with TSA activities after school was found to have a strong negative correlation with engineering content. Similarly, educators who reported not helping with any after-school STEM clubs had a strong positive correlation with engineering content ratings. This correlation indicates that educators who did not help with extracurricular clubs would be expected to score higher on their teaching of engineering content. Additionally, the amount of engineering or science in-service delivered for their district, the amount of time spent collaborating with other engineering educators, and the amount of science committees one served on were also strongly associated with greater proficiency in teaching engineering content. These correlations suggest that as participants spent more time engaging in the aforementioned informal experiences, it could also be expected that their teaching of engineering content would increase (Table 6).

Table 6 Spearman’s rho correlations between informal preparation experiences and teaching of engineering content ratings

Formal experiences and teaching of engineering practices (RQ4)

Spearman’s rho analyses were also utilized to examine the association between formal and informal preparation experiences and the teaching of engineering practices. The analyses found that undergraduate courses in robotics, IA or technology education teaching methods, and physics all demonstrated strong positive associations with increased ratings in engineering practices. Additionally, high school physics and graduate physics courses were found to have a strong positive and a weak positive correlation, respectively. These courses were both significantly associated with the engineering practices ratings and revealed that educators who completed more courses in these areas could be expected to demonstrate greater proficiency in teaching engineering practices (Table 7).

Table 7 Spearman’s rho correlations between formal preparation experiences and teaching of engineering practices ratings

Informal experiences and teaching of engineering practices (RQ5)

Similar to RQ3, a Spearman’s rho analysis identified a strong negative correlation between helping with TSA activities after school and teaching engineering practices. This correlation indicates that educators who spent more time helping with TSA activities could be expected to demonstrate significantly lower ratings toward their teaching of engineering practices. The analyses also found the following informal preparation experiences had a strong positive correlation with higher engineering practice ratings: the amount of engineering or science in-service that educators delivered for their district, the amount of time educators spent collaborating with science educators, the amount of hours educators spent each week in an engineering PLC, educators’ attendance at a recent national or state engineering education conference, and the amount of science committees educators served on. These findings suggest that participants who spent more time engaging in these informal experiences could be expected to be more proficient in teaching engineering practices (Table 8).

Table 8 Spearman’s rho correlations between informal preparation experiences and teaching of engineering practices ratings

Interview data

Participant interviews helped to corroborate the findings and provide additional insight about the classroom observations. During the post-observation interviews, when asked what experiences helped participants better understand and teach the engineering concepts in the FoTE unit that were closely interwoven with science concepts, they cited their experiences with high school and higher education physics courses. Participants also indicated that the opportunity to deliver engineering and science in-service sessions in their district encouraged them to be prepared for explaining the content in more detail and modeling engineering practices for other educators (Table 9).

Table 9 Responses to interview questions about preparation experiences to teach engineering and interwoven science concepts

Additionally, the educators highlighted engineering education conferences, time to collaborate and plan with other engineering educators and their school’s physics educator, and completing a technology education teaching methods course as beneficial to their teaching of engineering concepts. For example, Teacher 5 said “I feel real comfortable integrating science concepts in engineering, but I would still check in with my colleagues because I want to make sure that I introduce the vocabulary, I introduce the concepts so the kids can make the connection, and that I’m doing it at an age appropriate level. Teaming and collaboration are the biggest pieces. If we’re spot on some of my science and math colleagues come in and watch, sometimes I go in and watch them around a particular topic because kids shouldn’t see them in two different ways. The closer they can see the relationship and the terminology and know the dynamics of that theory or how the principle works, the stronger we all are.” Additionally, Teacher 7 said “I get some professional development from the state engineering education association conference. Last conference I visited a couple of science demonstrations about windmills and their apparatus for teaching the science of wind energy and things like that.” These responses reaffirmed the importance of the significant preparation experiences that emerged from the Spearman’s rho analyses. However, educators also believed a few preparation experiences were influential, but those experiences were insignificant in the statistical analyses. These experiences included: prior experience teaching electronics courses, engineering work experiences (e.g., construction, designer, Engineer), high school and higher education engineering content courses (e.g., electronics, PET), collaborating with family members who were engineers, and tinkering with materials and tools growing up. As Teacher 7 indicated, “Working in air conditioning and refrigeration gives me a good basis in hydraulics and thermodynamics because of the nature of working with fluids and change of state and understanding the temperature of the refrigerant is based upon the pressure so I have a background knowledge from doing that, so I have that application. My science background has mostly been experience, again mostly working with refrigerants and electricity in the workplace.”

One interesting finding from the Spearman's rho tests was related to the amount of time educators spent helping with after-school clubs, including TSA. These analyses found that educators who spent more time helping with after-school clubs demonstrated lower RTOP ratings. During the interviews, educators explained that while they valued after-school clubs like TSA, they found other after-school experiences more valuable for their teaching. For example, the educators indicated that time spent after school participating in online PLCs for engineering educators, collaborating with other engineering and science educators, and serving on school district committees related to engineering and science instruction were more valuable, in their opinion. Specifically, participants expressed that participating in PLCs and collaborative activities with other FoTE instructors (e.g., ITEEA’s Idea Garden) helped them understand the content of the FoTE curriculum better and provide students with new ideas or examples for modeling engineering concepts. As Teacher 5 indicated, “I don’t participate in a lot of after school clubs not because I’m against them, they’re just really not pushing me or pulling me forward. My professional development that helps me the best is my personal PLC, which is made up of some really good engineering teachers, and really good people from a large engineering company.” Teachers valued ITEEA’s FoTE support, “I love the summer FoTE workshops. They also offer help on Mondays and Wednesdays live online with FoTE people. I didn’t hesitate to call it” (Teacher 2). Teacher 8 also indicated, “I definitely think all of the FoTE in-service and professional development stuff hosted by the ITEEA was good because they were teaching what they give us in FoTE. When you get a bunch of FoTE teachers together we can collaborate and talk about the things that work for us and what do we do, and get ideas from other people. On our in-service days I'm with all FoTE teachers so we get to talk about what we’re all teaching, which are the same lessons. So I think those are very beneficial.” Teacher 6 requested additional resources in this area to help with their teaching, “I would like to see some professional development and working with other engineering teachers to come up with some different ideas of how to meld the hands-on engineering and the science content. I’d probably look up how to demonstrate science concepts online and consult with the physics teacher. I do consult with the physics teacher on occasion on the different things we're doing.” As Teacher 3 indicated, there are challenges to attending PLCs and collaborating with colleagues “FoTE training where they're actually going through the science concepts and showing us how to teach it in the engineering course would be helpful. Giving us guidance on how to teach those concepts and make it more relatable to what we’re doing. I do not usually collaborate with my science teachers, just due to not having enough time. We all have different planning times.”

These interview responses helped corroborate what was observed and provided further insight into the correlational findings, allowing the researchers to draw valuable conclusions about preparation factors that significantly influenced participants’ teaching of engineering content and practices.

Discussion

There are a number of limitations that must be considered within the context of this study. First, participation in the study was limited to educators from 12 county school systems from one consortium state in the U.S. who voluntarily self-reported their preparation experiences. Thus, the study had a homogenous sample for both the survey and classroom observations. However, the demographics from this study were similar to those reported in national technology and engineering educator studies (Ernst & Williams, 2015; Love & Roy, 2022). Additionally, this study provided a snapshot of each participant’s yearly teaching through one classroom observation. While acceptable interrater reliability and intraclass correlation coefficient measures were established for the modified RTOP before data collection and interviews helped corroborate the observations, the data reflect what was observed by a single researcher during one lesson. Furthermore, participants’ emphasis on factors such as physics coursework and professional development opportunities present concerns about outside variables influencing their responses (ex. the FoTE curricular materials may emphasize physics more than other areas). These factors may also be unique due to the self-selection nature of this study involving a small sample of observed and interviewed participants. For these reasons, the results from this study may not be generalizable beyond the sample. Additional data collection efforts among a larger or more diverse sample may be warranted to help examine different aspects of PCK which were not captured through the instrumentation and methodology of this study.

Contrary to what would be expected from the majority of the literature on PCK, the first set of Spearman’s rho analyses in RQ1 indicated that more years of teaching experience were not significantly associated with higher ratings for teaching engineering content and practices. Correspondingly, Hughes (2019) reported that years of experience did not impact educator knowledge or regulation of their cognition. The literature also suggests that PCK can be topic-specific, requiring unique content and pedagogical training experiences (De Miranda, 2018; Gumbo & Williams, 2014; Phillips et al., 2009). When further investigating the impact of teaching experience, the analyses in RQ1 found that years of teaching the FoTE curriculum did not significantly influence participants' teaching of engineering content and practices. These correlational analyses emphasize the importance and quality of preparation experiences related to content development over mere classroom teaching experience. Preparation experiences are critical given the growing shortage of engineering educators and the number of instructors without an engineering education related degree, without technology and engineering teaching certification, or who are from a non-engineering related content area and are being tasked with engineering teaching responsibilities (DATTA Australia, 2019; Love, 2015; Love & Love, 2022; Love & Roy, 2022; Love et al., in press; Reinsfield & Lee, 2021). These findings align with Hughes and Denson’s (2021) research, which suggested that educators untrained in engineering content and practices would be challenged to implement engineering concepts presented in the NGSS and STEL. This study also reaffirms findings from Love and Wells (2018) regarding high-quality engineering instruction being strongly associated with meaningful preparation experiences, and teaching experience is not always a strong predictor of higher PCK levels.

Higher proficiencies in teaching engineering content and practices were associated with a number of significant preparation experiences. One interesting finding from this study was the significant influence of numerous science courses, whereas fewer types of engineering courses had a similar impact. Physics courses were found to be the most significant and had the strongest correlations. From the literature, one would expect that due to the topic-specific nature of PCK, a greater amount of content courses in the area of electronics and PET would have been significantly associated with educators' proficiency in teaching the content within the FoTE energy and power lesson. However, the only engineering course found to be significantly associated was undergraduate robotics. The post-observation interviews proved beneficial in interpreting findings like these from the correlational analyses. During the interviews, educators expressed that robotics courses helped them better understand the applications of electronics content. The educators also attributed physics courses to enhancing their understanding of electrical energy theories. Specifically, the educators believed high school physics classes were more beneficial than higher education physics classes because of the smaller class sizes and an integrated theory and practical application format in high school. The educators did not believe the separate lecture and lab coursework format they experienced at the higher education level was as beneficial. These interview responses corroborated the greater impact of high school physics classes found in Spearman’s rho tests.

Courses similar to those previously discussed from the engineering content correlational analyses (RQ2) were also beneficial for teaching engineering practices (RQ4). This included the aforementioned science courses and the addition of IA or technology education teaching methods courses during participants’ undergraduate studies. The significant influence of these courses was corroborated by the interview responses in which educators expressed they valued seeing teaching practices modeled by instructors. Hence, they had greater knowledge about how to teach abstract concepts to students. The significance of IA or technology education teaching methods courses aligns with the literature, which suggests that modeling appropriate teaching practices can develop educator knowledge and regulation of cognition (Schraw, 1998). Educators indicated that robotics and physics classes also enhanced their practices by allowing for the application of electronics and PET content in memorable ways that helped them recall those examples when explaining concepts to students. Love and Wells's (2018) study found similar formal experiences to be significant, but in informing educators' teaching of science content and practices within an engineering lesson. In the current study, the TEES-PCK instrument revealed that 75% of the observed educators had completed at least one electronics or PET course during their undergraduate studies. However, only 38% of the observed educators took a physics course during their undergraduate coursework. Given that fewer educators completed physics courses as it is not a core requirement in all undergraduate technology and engineering educator preparation programs like electronics or PET courses were (Litowitz, 2014), this may have been a unique preparation experience that the Spearman’s rho found significant. Engineering preparation programs should reflect on the significant formal preparation experiences highlighted in this study. Engineering educator preparation programs should focus on content and pedagogical knowledge development during the pre-service educator preparation experience.

Beyond formal coursework, several informal experiences were strongly correlated with the teaching of engineering content and practices. Educators' increased involvement with TSA and other after-school clubs were associated with lower teaching ratings. As discussed in the interview results, educators found these after-school activities took away from the valuable time they needed to participate in positively correlated experiences such as PLCs, collaborating with other engineering and science educators, and serving on district science committees. Similar to their responses about coursework, educators felt these collaborative experiences in addition to attending engineering education conferences and delivering engineering or science in-service sessions, helped them learn better ways to teach abstract concepts. In particular, participants expressed that they valued asking questions of other educators from both engineering and science to understand the core and crosscutting content better and see demonstrations of new ways to engage students in the content through engineering practices. Lastly, those that had to deliver in-service sessions within their school district described how this encouraged them to study the curriculum content in greater detail and think more in-depth about the practices they would demonstrate for colleagues or how they would engage colleagues in practices like they were students. These findings suggested that even though the sample had an average of 5 years of experience with teaching FoTE, having additional informal learning experiences significantly enhanced their teaching. Participants described how collaborative experiences helped them gain ideas for providing current and relevant examples to engage their students while continually improving their content knowledge and pedagogical strategies. The literature supports the notion that collaborative activities can be highly beneficial for engineering educators. For example, collaborative science and engineering education conferences have been shown to increase educators' perceptions regarding their ability to teach both engineering and science concepts (Love & Loveland, 2014). These findings highlight the importance of educators continuously engaging in collaborative learning opportunities to enhance their teaching beyond their educator preparation coursework.

Conclusions

If engineering educators are expected to authentically integrate content and practices from all STEM disciplines as called for in national science, and technology and engineering standards documents (ITEEA, 2020; NGSS Lead States, 2013), then appropriate educator preparation to do so at a high level is warranted. This study, along with previous research from Love and Wells (2018), suggests that engineering educators can benefit from interdisciplinary preparation experiences to teach both science and engineering content and practices more proficiently. The findings from this study specifically highlight the broad range of experiences that should be considered when preparing educators for teaching engineering concepts embedded within crosscutting STEM lessons. In addition, the formal and informal experiences found to be significantly associated with higher levels of teaching engineering content and practices provide practical implications for researchers, teacher educators, school districts, administrators, and educators.

Implications and recommendations

Given the association found between teaching engineering content and numerous formal and informal science preparation experiences, additional research is needed to examine if these findings are similar for science educators tasked with teaching engineering content and practices as called for by the NGSS. Since science educators would be expected to have more extensive science preparation experiences, this would provide an opportunity to focus on the influence of formal and informal engineering experiences, which are often not as common among science educators (Love & Wells, 2018). Future research on science and engineering PCK should consider using the TEES-PCK (Love, 2015), modified RTOP (Love et al., 2017), and interview instruments (Appendix A) utilized in this study and by Love and Wells (2018). The instruments and methods used in these studies have demonstrated valid and reliable measures for examining preparation experiences associated with teaching science and engineering content and practices as called for by both the NGSS and STEL. Replicating these efforts within specific science and engineering education contexts involving a larger sample could further inform educator preparation efforts given the limited research on engineering PCK.

This study found that educator preparation related to formal and informal science experiences (e.g., physics classes, delivering science in-service, serving on science committees, collaborating with science educators) had the strongest correlation with teaching engineering concepts in the observed FoTE energy and power lesson. These findings may indicate the need for more science preparation experiences due to the increasingly interdisciplinary nature of engineering education as expressed in the NGSS and STEL. While technology and engineering educator preparation programs have traditionally had a limited focus on core engineering content knowledge, findings from this study indicate that more thorough interdisciplinary preparation experiences can enhance the teaching of engineering content and practices when interwoven with science concepts. For in-service educators, the findings indicate that involvement in TSA and other after school clubs do not enhance secondary educators’ teaching of engineering. While TSA and other after school clubs have often been viewed as beneficial for secondary engineering education programs and students, and have the potential to provide opportunities for educators to collaborate, it is recommended that the engineering focus of TSA events and other clubs be examined. Furthermore, this study suggests that school districts should provide additional time for engineering educators to collaborate among STEM colleagues. Such collaborative experiences could include attending STEM conferences, participating in interdisciplinary online PLCs, and delivering or co-delivering in-service instructional sessions to enhance educators’ STEM content knowledge and pedagogical knowledge.

Appendix A

Interview instrument

Completed prior to the observation

1. When you were planning today’s lesson, were there specific things you had to take into consideration? Were there any concepts or ideas you were unsure of? How did you prepare to teach them? 2. Do you think there are any concepts students will struggle with today?

Completed after the observation

Lesson reflection

3. How do you feel the lesson went today and why? 4. Did you teach this class different than other class periods? Why?

Pedagogical practices and content taught 5. What key concepts did you want your students to take away at the end of this lesson? Why? a. What about [insert concept(s) from lesson content analysis that were very rarely mentioned in the lesson]? Do you believe that is an important take away? Why or why not? 6. Tell me a little bit about why you needed to take more time to cover [insert concept(s) from observation of the lesson that were frequently mentioned during the lesson] a. Do you believe your preparation experiences influenced your teaching of these concepts? What specific experiences? Why? 7. Tell me a little bit about why you spent less time covering [insert concept(s) from the lesson content analysis that were rarely mentioned during the lesson] a. Do you believe your preparation experiences influenced your teaching of these concepts? What specific experiences? Why?

Teaching preparation experience questions 8. Tell me a little bit about how you got into teaching engineering education? a. What about your teacher preparation experiences? How do you believe your engineering education courses and labs inform your teaching of the FoTE curriculum? b. How about any science courses and labs you had, how do you believe those inform your teaching of the FoTE curriculum? c. What about any work or home (informal) related experiences? Can you tell me a little about any of those informal engineering related experiences which you believe inform your teaching of FoTE. Any informal science related experiences? 9. Describe for me how you feel about your preparation to teach the engineering content in this lesson? How prepared do you feel to model this content through demonstrations and labs? 10. What about your preparation to teach the science content in this lesson? How prepared do you feel to model this content through demonstrations and labs? 11. To enhance your preparation, what pre-service resources or experiences (content courses, labs, methods courses) would help you teach the engineering concepts embedded within this lesson? How about in-service resources or experiences? 12. To enhance your preparation, what pre-service resources or experiences (content courses, labs, methods courses) would help you teach the science concepts embedded within this lesson? How about in-service resources or experiences?

1. Modified from Park et al. (2011) by Love and Wells (2018)