Intercultural competence outcomes of a STEM living–learning community

Abstract

Background

Living–learning communities and global or diversity learning experiences have been identified as educational practices which often have a “high impact” on student success, as well as providing interpersonal competencies that are greatly valued by employers. Even without a specific intercultural or diversity component, living–learning communities would seem to offer rich settings for the development of the ability to work effectively across cultural difference. Yet intercultural learning outcomes are rarely assessed outside the domain of study abroad or diversity training programs. The purpose of this study was to determine whether participation in a “global science” living–learning community can increase the intercultural competence of first year international and domestic students, as measured by a well-known quantitative instrument, the Intercultural Development Inventory (IDI).

Results

In the first 2 years of the study, the intercultural learning content focused primarily on ‘dealing effectively with difference’ and produced minimal mean gains in intercultural competence. Examination of qualitative data from these experiences (using a well-known rubric to frame the analysis) as well as a review of the literature around intercultural learning (principally in study abroad contexts) suggested that focusing on similarity and self-awareness, coupled with individualized feedback, was likely to be a more appropriate pedagogy for students’ competency development. Following the curriculum revision, 2 years worth of participants exhibited much higher mean gains in IDI scores, as well as higher rates of shifting to a new stage of effectiveness by semester’s end.

Conclusions

This study contributes to the STEM education literature by attempting to apply several years of research findings about effective intercultural competence development, principally from study abroad programs, to STEM education in on-campus contexts. In so doing, it has implications for how STEM educators can more effectively work towards cultivating global-ready STEM graduates, and towards reaching STEM students who, for whatever reason, do not typically participate in study abroad.

Introduction

Increasingly, one of the goals of higher education is exposure to cross-cultural perspectives and the development of the intercultural competence necessary for career success in an increasingly globalized world economy. Students in the fields of science, technology, engineering, and mathematics (STEM) are expected to be prepared to work in collaboration with people from diverse backgrounds (Akdere et al., 2019). STEM practitioners have recognized that intercultural skills, often more than technical skills, are of tremendous value. Ghasemi Mighani et al. (2019) reported that intercultural skills were listed as very important or important by 90% of surveyed STEM employers. Intercultural competence (ICC) is defined as the ability to interact effectively and appropriately with people from culturally diverse backgrounds (Hammer et al., 2003). It requires the development of attitudes, skills and acquisition of knowledge (Deardorff, 2009) and has been identified as increasingly important for professional and scientific practice in the twenty-first century (Beneroso & Alosaimi, 2020; Davis & Knight, 2018; Demetry & Vaz, 2017; Mazzurco et al., 2020). The development of ICC is regarded as both an imperative for and a potential outcome of the internationalization of STEM education. It is crucial for preparing students for success and for empowering them both personally and professionally. Fifty percent of global STEM practitioners surveyed, however, identified STEM graduates to be deficient in soft skills (e.g., intercultural communication skills, teamwork, and interpersonal skills; Hurrell, 2016). Moreover, policymakers and education practitioners have called for higher education to systematically support the development of intercultural skills and knowledge so as to enable students to succeed in an increasingly diverse campus and world (Association of American College and Universities [AAC&U], 2007; Beneroso & Alosaimi, 2020; Deardorff & Arasaratnam-Smith, 2017; Qadir et al., 2020). Disciplines in fields as diverse as engineering, technology, agriculture, and the health sciences have recognized the increasing need for supporting the development of interculturally competent graduates (Lucietto & Russell, 2020; MacCleoud, 2018; Ortiz-Marcos et al., 2020).

Current practices that lead to the development of ICC in higher education are of two types: co-curricular (e.g., research experiences, study abroad, community engagement), and curricular (Ji, 2020). Few on-campus co-curricular activities intentionally focus on the development of ICC, and relatively few students study abroad (Krishnan et al. 2021), particularly in STEM disciplines (Klawe, 2019). Curricular integration of intercultural outcomes outside the study abroad realm is, despite some high-profile examples, also rare (Brajkovic & Helms, 2018; de Wit & Altbach, 2021). Plough (2016) reported that fewer than ten percent of undergraduates take a course focused on intercultural or related topics, and fewer than twenty percent of colleges require more than 2 years of world language study.

When implementing structured learning interventions with students from diverse backgrounds, co-curricular programs have been identified as effective at enhancing student engagement, sense of belonging, and intercultural maturity (Perez & Shim, 2020; Smith, 2018; Zhao & Kuh, 2004). However, these outcomes are not by any means a foregone conclusion. On many campuses, in the US and elsewhere, large numbers of students live in residential proximity to students of other nationalities and/or ethnicities. Research suggests that the perception of large cultural or value differences between groups or individuals can be a barrier to friendship development, even when there is ample opportunity for social interaction (Gaston, 2017; Tsang, 2020). Yet few campuses attempt to use intercultural development theory along with quantitative assessment instruments and formative feedback in residential life contexts. This current study aims to examine the use of a structured, theory-based curriculum to develop intercultural competence among first-year STEM students in a living–learning community (LLC).

Learning community

In the U.S. higher education context, the LLC has its roots in the 1920s, with efforts to marry a formal instructional environment with a residential component, where informal shared pursuit of learning could occur (Brower & Inkelas, 2010; Caviglia-Harris, 2021). Early research on outcomes of LLCs was primarily practitioner-oriented, examining their impact on student satisfaction and student retention, while more recently scholars have focused on how LLC participation correlates with higher order skills, such as critical thinking or civic engagement (Gansemer-Topf & Tietjen, 2015; Inkelas et al., 2018). Taken together, the literature suggests that a well-resourced living–learning community can increase participants’ sense of belonging, their social ease with other students, and their willingness to seek out interactions with others of diverse background (Hurtado et al., 2020; Sriram et al., 2020). Rarely, however, have researchers used quantitative instruments to attempt to assess the possible interpersonal or intercultural outcomes of a living–learning community (Blondin, 2015).

In the United States, the globally focused or cross-cultural residential community traces its history back to at least 1924, with the opening in New York City of the International House (Davis, 2006), an educational non-profit whose purpose is to build a supportive community of individuals from both host and visitor nations (International House New York, n.d.). Notwithstanding the research suggesting that such communities increase social interaction skills, student retention or satisfaction (Inkelas et al., 2018; Sriram et al., 2020), there has been very little research on intercultural competence outcomes of such global living–learning centers (Gansemer‐Topf & Tietjen, 2015). Even among scholars who examine the role of residence halls in nurturing belongingness in the student body, few have attempted to investigate relationship development between domestic and international students who live in residential proximity to one another (Gaston, 2017; Sriram et al., 2020).

Learning outcomes of internationalized campuses

Many institutions look to study abroad programs to help develop intercultural competence in their students. However, only a relatively small proportion of students from US universities study abroad, and even fewer study in the type of program that intentionally boosts intercultural competence. Study abroad often includes mixed-nationality living–learning situations, such as homestays or mixed nationality dorms. In the popular imagination, it is widely assumed that this type of frequent or prolonged social contact with persons of another nationality during university studies will almost magically yield intercultural competence (Cubillos & Ilvento, 2018; Kulturel-Konak, 2020; Woolf, 2007). Repeatedly, however, research has indicated that these living situations do not reliably yield mutual understanding. For example, homestay relationships fizzle or friendships fail to crystalize, leading students to re-adjust their goals of becoming linguistically adept (Isabelli-García et al., 2018; Marijuan & Sanz, 2018). Research suggests, also, that students of the host nation may not see the value of engaging in high-anxiety social interactions with ‘foreigners’ (Tsang & Yuan, 2021) or of pursuing cross-national roommate pairings (Yao, 2016). On US campuses, international and domestic students have been found to engage (separately) in different types of leisure activities, thus reducing opportunities to create bonds around shared interests (Heng, 2017; Lehto et al., 2014;). Finally, studies have found that neither living with nor taking classes with locals had a significant impact on intercultural competence development (Marijuan et al., 2018; Vande Berg et al., 2009; Whatley et al., 2021). More recently, research has also challenged the notion that a long-duration program of a semester or year is a necessity for intercultural competency development. Study abroad programs as short as 2 weeks have been found to yield measurable growth in intercultural effectiveness (Nguyen, 2017), intercultural competence (Yngve, 2018), and/or global awareness (Schellhase et al., 2021; Whatley, 2019). A recent review of research about effective intercultural development (in study abroad contexts), suggests strongly that, rather than duration or other programmatic structures, guided reflection is the key to students’ development; without such support, instrumentalism, colonialism and cultural backlash are just as likely to occur as more positive outcomes (Johnstone et al., 2020).

These findings bear highlighting: contrary to received wisdom on acquiring intercultural competence, study after study has found no evidence that measurable intercultural competence growth reliably results from simply spending a semester or longer in residential or classroom proximity to individuals from another nation. What if, however, a STEM living–learning community was designed intentionally to develop intercultural competence in its participants?

Case study: a globalized STEM learning community

In this paper, we present an iterative case study of assessing and improving a globally themed residential learning community for STEM students. The ‘Transnational Learning Community’ (TLC) came about as a response to the high proportion of international students in the College of Science at Purdue University. At the time of the launching of TLC, international students comprised 17% of the undergraduate student body at this institution and 28% of undergraduates within its College of Science (Purdue University Institutional Data Analytics + Assessment, 2020). Coincidentally, the university had also inaugurated a new undergraduate core curriculum in the year just prior to the TLC launch. The new core curriculum includes a requirement that intercultural competence be an embedded learning outcome of the undergraduate degree program for all students. The TLC, as a result, focuses on neither simple friendship-building nor on academic success (such as many LLCs and other student engagement opportunities), but intentionally incorporates a number of practices intended to help STEM students to become better at working effectively across cultural difference. These practices have shifted over the course of the LLC’s existence (as described further below), but have always included a one-credit leadership seminar which focuses on intercultural competence as a vital tool of scientific collaboration, including an introduction to Hofstede’s research on culture-general values and patterns of behavior Hofstede (2010), and classroom assignments as well as service-learning activities that intentionally require students to work in multinational teams.

Research question

In summary, although previous studies have demonstrated that learning communities are helpful to improve students’ interpersonal interaction skills (Sriram et al., 2020), limited research has investigated intercultural competence outcomes of global living–learning communities. In addition, little is known about the intercultural interaction between international and domestic students who live in the learning communities (Gaston, 2017). Therefore, the purpose of the study is to examine growth in intercultural competence of four cohorts of first-year international and domestic students (N = 112) in a “global science” learning community.

Our research question is, therefore, practice-based and student-centered, as follows:

Does participation in a globally focused learning community for a semester increase the intercultural competence of participating international and domestic students?

Theoretical frameworks

The conceptual framework for this study is shaped by three bodies of theoretical work: the Developmental Model of Intercultural Sensitivity or DMIS (Bennett, 1986), which models how an individual advances beyond ethnocentrism; Intergroup Contact Theory (Allport, 1954), which describes the conditions under which contact does or does not yield a reduction in intergroup conflict; and the theory of Challenge and Support (Sanford, 1962), which suggests how to better scaffold learning, whether in formal or informal educational contexts.

Developmental intercultural competence

In 1984, Milton J. Bennett introduced a developmental model for understanding the process of becoming effective and appropriate across cultural differences (Bennett, 1986). The model codifies a number of common reactions to cultural difference, which represent stages along a journey from ethnocentrism to “ethno-relativism”—a word that was coined to describe the state of being able to empathize with and behave in a way that honors a different cultural worldview. An important distinction of this model is the recognition that movement between intercultural stages can (and often does) occur in either direction between the two endpoints (Acheson & Schneider-Bean, 2019; Bennett, 1986; Krishnan et al., 2017). In other words, in some situations, if the cultural challenge is too threatening or too emotionally charged, the individual who had previously been functioning at, for example, a stage of seeking common ground (Minimization) may revert to a prior stage of response to difference, such as one of judging difference (Polarization).

Intergroup contact theory

Intergroup Contact Theory (Allport, 1954; Sherif & Sherif, 1965) informs us that it is not enough to bring two groups together to promote learning, positive attitudes and the willingness to work together. The required conditions are for the groups to have equal status, common goals, cooperative structure, the support of authorities, laws or customs, and informal personal interaction.

Challenge and support

To this, Sanford (1962) adds that in educational contexts, learning and attitudinal changes fail to occur when contact situations create boredom, anxiety or aversion for the participants. His educational theory of Challenge and Support (1962) can help us understand why students who have just entered college as well as students on study abroad may not end up with increased intercultural competence. Sanford identifies three interlocking factors that predict successful learning: readiness, challenge and support. With “readiness” defined as a function both of internal maturation processes and beneficial external influences, the individual cannot exhibit certain behaviors until they are developmentally ready to do so (a concept which is also integral to M. Bennett’s developmental model). Then, when an individual resides within that zone of readiness, two things must be in balance: the amount of learning challenge and the amount of support. If the challenge exceeds the amount of support given, the learner will withdraw physically or mentally, exhibit stress and lowered cognitive ability, and often challenge the authority of the instructor or the validity of the learning exercise. If, on the other hand, the amount of support is too high in relation to the degree of learning challenge, the learner may feel appreciative of the learning opportunity, but is seldom motivated to progress cognitively or to change behaviorally.

Methods

Study participants

This study examines four cohorts (2013, 2014, 2017, and 2018) of participants in the TLC. Students self-select into the learning community, choosing from a menu of housing alternatives. In addition to in-state students and those from the rest of the United States, this learning community has welcomed students from China, Japan, South Korea, Mongolia, India, Pakistan, Indonesia, Malaysia, Singapore, Brunei Darussalam, Turkey, Saudi Arabia, United Arab Emirates, Costa Rica and Canada (see Table 1 for demographics of TLC cohorts with annual number of domestic and non-domestic participants). The study protocol was approved by the university Institutional Review Board (IRB).

Table 1 Demographics of the transnational learning community cohorts

The learning intervention

The TLC is comprised of three major dimensions: shared campus housing, a course, and a program of co-curricular and social activities. Students live together in a residence hall. The leadership seminar introduces students to the concepts of culture, cultural differences, intercultural conflict, and intercultural leadership (see the abbreviated syllabus of the seminar in Additional file 1: Appendix S1). Students hear from faculty as to the collaborative and cross-cultural nature of science. They complete classroom exercises and homework assignments that encourage them to pay attention to culture and reflect on its meaning. For example, in one widely available exercise, often used by science educators (California Academy of Sciences, n.d.; Starr, n.d.), students are asked to make a sketch of a scientist. Pictures are compared for their similarities and differences across cultural lines. The idea that perceptions of science differ among cultures is then explored. From that, other differences in value dimensions are examined for their impact upon both personal interactions and science as a profession. For purposes of the LLC, this activity was included to help learners recognize how culture can shape our ideas of professional practice. In terms of theories of Intercultural Development, it is an activity which is supportive of learners in the stage of Denial (who need to learn to ‘spot’ how cultural differences matter) and learners in the stage of Minimization (who need to refine their understandings of cultural similarity and difference).

For co-curricular and social activities, these first-year learning community students are partnered with upperclassmen—usually in the same major—from the College of Science. Initially, these partnerships aid in the transition to college, and eventually informal mentoring relationships develop. Together, students in the learning community take part in activities that aim at increasing their cross-cultural effectiveness. They also participate in group service-learning events. For instance, for Halloween they coordinate a trunk-or-treat program for local preschoolers and schoolchildren. This activity not only introduces some of the international students to American traditions, but also brings domestic and international students together in striving to reach an authentic, non-academic common goal. In other words, this and other informal group activities were designed to offer the support systems which Allport (1954) and Sherif and Sherif (1965) found to be necessary to positive attitude development across cultural difference: e.g., working in support of common goals and equal status of group members.

Instruments

There are many definitions of intercultural competence (Gregersen-Hermans, 2017; Sinicrope et al., 2007) and even more ways of assessing it (Deardorff, 2017; Fantini, 2009; Paige, 2004). We chose two highly regarded instruments to evaluate intercultural learning outcomes of TLC participants: a survey intended to measure the developmental stages of intercultural competence and a rubric designed to measure direct evidence of intercultural learning. A description of each is below.

The intercultural development inventory (IDI)

The IDI is a cross-culturally validated survey instrument comprised of a 50-item questionnaire, which has been found to have little to no social desirability bias (Paige et al., 2003; Wiley, 2016). The instrument identifies a test-taker’s developmental stage along a five-stage continuum ranging from a mono-cultural (ethnocentric) to an intercultural (ethno-relative) mindset, as elucidated in Table 2.

Table 2 Intercultural development stages (IDI, LLC; 2017)

For each individual who takes the survey, the Inventory identifies among other things, a Developmental Orientation (DO), which captures the mindset from which an individual functions in situations, where cultural differences need bridging. When used with a group or team, the IDI also provides a group profile, which indicates the mean developmental orientation (DO) for the cohort as well as its distribution. Knowing the IDI stage of a group or individual can help the instructor predict the amount of support and challenge needed to effect growth. For example, when learners are in Polarization or on the cusp of Minimization (as has tended to be the case with most TLC cohorts), the stage-appropriate pedagogy is to work on anxiety reduction and self-awareness, thereby facilitating the ability to find common ground (Bennett, 2004b).

The AAC&U intercultural knowledge and competence rubric

In November of 2012, the university’s Faculty Senate approved a university-wide undergraduate core curriculum, which, among other requirements, stated that a number of higher order thinking skills (e.g., Quantitative Literacy, Ethical Reasoning, Intercultural Competence) were to be embedded within the 4-year degree program for all students. The Senate further identified a set of rubrics, adapted from the AAC&U VALUE project (Association of American Colleges & Universities, 2009), as suggested assessment tools. These rubrics, each created through an iterative process by a team of university-based subject-matter experts, have been found to facilitate high inter-rater reliability when used to assess authentic artifacts of learning (Rhodes, 2011), and also, at least at one research university, to offer more cost-effective and actionable data than using a proprietary survey instrument (Pusecker et al., 2011). The VALUE rubric for Intercultural Knowledge and Competence sets forth a suite of knowledge, skills and attitudes which, taken together, define the personal capacity to work effectively across cultural difference. These are: (1) Cultural self-awareness, (2) Knowledge of cultural worldview frameworks, (3) Empathy, (4) Verbal and nonverbal communication, (5) Curiosity, and (6) Openness. Each of the six elements of this rubric has four possible developmental levels, as in the example excerpted in Table 3. This rubric is particularly apt for use with the IDI, because its creators calibrated it to the minimization stage, e.g., the midpoint of the Intercultural Development Continuum (Cartwright, 2021).

Table 3 VALUE rubric excerpt (AAC&U, 2009)

Data collection

Each entering cohort of participants in the Transnational Learning Community took the Intercultural Development Inventory (IDI) at the start of their first semester and again as the semester was concluding. In addition, for the later cohorts (2014, 2017 and 2018), two required short reflective writing exercises which focused on the challenges and successes of learning to work across cultural differences, were collected from each student as part of the instructional process. Reflection prompts for these two assignments were as follows:

• Assignment One: “Discuss one significant intercultural challenge or conflict you have successfully faced since arriving at Purdue University”, and

• Assignment Two: “Discuss one significant intercultural challenge or conflict you have NOT successfully faced since arriving at Purdue University”.

Data analysis

This study was structured as a three-phase action-research project. In phase 1, we analyzed the early (2013 and 2014) cohorts’ IDI DO scores only. In phase two, we looked at written reflective assignments from the 2014 year, using a rubric as our framework for analysis, to try to understand the IDI results and students’ learning struggles more thoroughly. Based on the lessons learned during the first 2 years, we used the interim years (2015 and 2016) to refine the curriculum and try to achieve a better balance of challenge and support. In phase three, we used the IDI as well as framework analysis (e.g., rubric-based analysis) of the reflective essays, to evaluate the possible effectiveness of instructional changes to the Leadership Seminar.

Survey data (phase I and phase III)

Each student’s initial and terminal IDI scores were compared to compute degree and direction of point change in Developmental Orientation (DO), if any. Initial group mean DO was also compared to terminal group mean DO for each annual cohort. Statistical analyses on the changes in DO scores were then conducted via ANOVA, which was performed using JMP statistical software. The validity of the assumptions for the analyses were assessed via standard residual diagnostics. Specifically, the residuals were examined to assess whether they were approximately normally distributed, centered at zero, and did not exhibit any patterns or outliers. Differences among the four cohorts were tested using ANOVA. Afterwards, differences between the early group (2013 and 2014 combined) and later group (2017 and 2018 combined) were examined.

Qualitative data (phase II and phase III)

The aforementioned AAC&U rubric, in its original four-level version as published on the AAC&U website, served as the researchers’ framework for making sense of the students’ written data. Framework analysis is a deductive qualitative methodology often used in applied and interdisciplinary research contexts (Gale et al, 2013). It has been cited as being a better fit than grounded theory methodology (for making sense of qualitative data) when the inquiry features context-specific questions, a limited time frame, and a pre-existing (e.g., non-random) sample (Srivastava & Thomson, 2009), as is common in educational assessment contexts. To put this methodology into practice, first, each student essay was independently read by an individual researcher and coded as to which mastery level(s) of which aspect(s) of intercultural competence (e.g., self-awareness, empathy, openness, etc.) had been demonstrated, as defined by the rubric. Some examples of coded data are shown in Additional file 1: Appendix S2, Table S1.

Next, the researchers met and discussed specific cases to ensure inter-rater reliability across the samples. Following this, each student reflection paper was given a six-part “grade” indicating the highest level of proficiency exhibited for each of the six elements of the rubric. Finally, to make it easier to compare effectiveness of the learning interventions across the annual cohorts, these tabulations were used to determine what percentage of a given year’s cohort had demonstrated level three proficiency or above. Level 3 proficiency was chosen as the standard, because the expert design team for the AAC&U rubric calibrated the instrument such that Level Three descriptions align to the intercultural development stage of Minimization (Cartwright, 2021); the stage at which learners begin to have the ability to connect effectively and appropriately across cultural difference.

Results

No correlation was found, in either the quantitative or the qualitative data, between a student’s country of nationality and that same student’s demonstrated level of intercultural competence. In other words, as a group, these international students were not, as is often assumed, more competent at bridging cultural difference as entering students than were the domestic students. There were also no significant differences found between gender groupings.

Survey findings

All four cohorts entered the university at an equivalent stage of intercultural development (as shown by the mean DO scores). Each cohort’s group mean for behavioral competence (Developmental Orientation or DO) was at approximately the demarcation score of 85 points on the IDI continuum, which is in the borderland between Polarization and Minimization. In Polarization, the individual’s response to cultural difference is an “us vs. them” approach, which values a given culture’s beliefs and worldview highly, while denigrating, deploring or aggressively ignoring those of the out-group culture. At Minimization, individuals expect to find similarities and even universal values across cultures, and are eager to find common ground (Bennett, 2004a). However, failure to find common ground may lead to a regression into Polarization (Acheson & Schneider-Bean, 2019; Schellhase et al., 2021).

Despite beginning university studies and the learning community at the same apparent level of learning readiness for intercultural development, the four cohorts differ in how they developed over the course of their first semester, as shown in Table 4. Whereas the mean DO scores for the 2013 cohort showed slight regression over the semester and the 2014 cohort demonstrated minimal development, the group DO means for the 2017 and 2018 cohorts approached the midpoint of minimization (score of 100).

Table 4 Group developmental orientation (DO) at entry and semester's end

There was no significant difference in the amount of change in DO scores for the 2013 and 2014 cohorts; there was no significant difference between the 2017 and 2018 cohorts as well. Therefore, we grouped the early (2013 and 2014) cohorts and the later ones (2017 and 2018). The mean change in DO scores for the early combined 2013 and 2014 cohort group was 0.21 with a standard error of 2.31, while the mean change for the latter group (2017 and 2018) was 9.68 with a standard error of 2.39. The distribution of the change between the pre- and the post-tests for the two groups (generated in JMP) are displayed in Fig. 1. The gains in DO scores between pre- and post- for the later (2017 and 2018) group are significantly higher than for the early (2013 and 2014) group (p = 0.0053; Table 5).

Fig. 1

Change in DO scores between pre- and post for early vs. later groups

Table 5 ANOVA for difference between the early (2013 and 2014) and later (2017 and 2018) groups

The 2017 and 2018 cohorts’ gains of 10.6 and 9.0 IDI points, respectively, can be compared to other intercultural learning research. The IDI has frequently been used to measure development of intercultural competence resulting from participation in study abroad (Hammer, 2015), and seldom been used to investigate the intercultural competence impact of having an internationalized student body. From the “internationalization at home” research literature, we found two pertinent recent studies that used the IDI to assess learning among first-year students of mixed nationality. At an Australian university, a pre–post administration of the IDI to 122 domestic and international first-year students found a very little difference in their initial group mean Minimization score of 93 points, after 7 months of inhabiting the same campus (Lantz-Deaton, 2017). On the other end of the outcomes spectrum was a 2017 pilot project involving 16 first-year engineering students at a US university (approximately half domestic and half international), which incorporated many of the best practices suggested for high-impact learning. Students in this project attained a group mean gain of over 22 IDI points over the course of the 10-month intervention, having begun their college career, such as our TLC groups, at the cusp between Polarization and Minimization (Render et al., 2018).

As regards the research on US students studying abroad, summarized in a thorough literature review by Paige and Vande Berg (2012), the record suggests that group mean growth may vary from as little as 1.32 IDI points for a semester abroad (pp. 34–35) to 17.5 IDI points for a multi-year, comprehensively internationalized degree program (pp. 49–51). The only semester-long program that is cited as having demonstrated better IDI results than our 2017 and 2018 TLC cohorts is the American University Center of Provence (AUCP), which attained a group mean gain of 13.43 IDI points with a sample size of 411 students (Engle & Engle, 2012, p. 303). The AUCP program has a high-impact design very similar to our 2017 and 2018 learning communities; e.g., the challenge of instructor-led intercultural course and numerous immersion activities combined with the supportive features of stage-appropriate assignments focusing on similarities as well as differences and individualized cultural mentoring. In the company of such best-practice study abroad programs, the gains made by these STEM learning community students in an on-campus semester-long program seem quite impressive.

Another way of evaluating a program’s effectiveness is to examine the proportion of students who made gains or regressed in their IDI score or who remained in stasis (e.g., in the same stage). In this program, our instructional goal was to move the TLC group from an unstable position at the cusp of an “us vs. them” orientation (borderline between Polarization and Minimization) to a stage of being able to successfully find common ground (fully into Minimization or beyond).

In Fig. 2, we have indicated for each annual cohort the percentage of students who moved forward at least one IDI stage (such as from Polarization to Minimization), remained in the same stage (in stasis), or moved backward at least one IDI stage. Note that over the years, there has been a tendency for fewer students to regress a stage and for more to advance. A seven-point change in developmental orientation (DO), half a standard deviation, is considered a meaningful change with this instrument. Looking at the proportion of the cohorts gaining seven points or more on the IDI, we find that 23% gained at least 7 points in 2013, while 63% did so in 2018 (Fig. 3).

Fig. 2

Patterns of IDI stage change by cohort

Fig. 3

Percent of cohort gaining at least 7 points on DO score

In sum, more students in 2017 and 2018 moved ahead a stage and fewer regressed a stage than in the first two cohorts. In addition, almost three times as many students moved ahead at least seven points on the IDI in 2018 than in 2013. According to Sanford’s theory, inconsistent outcomes in a learning cohort can be an indication that many of the learners are experiencing an imbalance in the ratio of challenge to support, as evidenced in study abroad contexts (Vande Berg et al., 2012). Thus, these different outcome patterns suggest that, in the earlier cohorts there was too much challenge, while in 2017 and 2018, the intercultural challenge and support balance was better attuned to the groups’ developmental needs, as fewer students regressed and more made significant gains in their IDI scores, than in previous cohorts. More details of findings are shown below.

Framework analysis findings and curriculum revision

Despite the attention to creating a curriculum and a set of shared co-curricular experiences centering on science as a global profession, the initial 2 years of IDI results suggested that relatively small percentages of students were, in fact, increasing their intercultural competence (as has been shown to also often happen in study abroad contexts). Two reflective assignments, added to the TLC curriculum in 2014, seemed ideal for extending our understanding into these learners’ intercultural development journey (and how to improve it).

Preliminary analysis of these essays showed that the reflection prompts only rarely evoked student responses that aligned with two of the rubric elements: Knowledge of Worldview Frameworks and Curiosity. Consequently, we compared only the other four aspects of intercultural competence across the annual cohorts; e.g., Openness, Empathy, Self-Awareness and Communication. Again, Additional file 1: Appendix S2 shows coded examples of the data and how they align with the levels of the rubric.

In the initial (2014) cohort of TLC learners, a majority of the students manifested only level two competencies, at best, even when asked to identify and discuss a success in overcoming cultural differences. For example, the comment “I have not faced any significant intercultural challenges or conflicts and succeeded in solving them since arriving at Purdue. There is one huge issue with my roommate and his culture that has ideals that conflict with my ideals, but I have not succeeded in resolving that.” This example shows minimal cultural self-awareness (rubric level one). In another example, “One of the hardest challenges I’ve faced since coming here is being able to understand and form new relationships with international students. Typically, I am very good with people and I make new friends easily. However, I find it very hard to understand international students’ accents and I get frustrated and feel awful when I ask them to repeat themselves several times throughout a conversation.” In this instance, we see that the student knows that a posture of openness is expected, but is feeling unable to find an effective solution to overcoming communication differences (we coded this as Level Two for both Openness and Communication).

Reading (and collaboratively coding) these comments led the instructors to realize that a strategy of focusing primarily on patterns of cultural difference (in the leadership seminar) may have been too challenging for some students. Accordingly, several changes were made to help the group move more solidly into minimization across the duration of the semester. Many of the changes made in the 1-credit Leadership Seminar over the years are depicted in Table 6. In the latter years, there has been less emphasis on cognitive skills—thinking about culture, and more on experiences and reflection about one’s interactions with culture. More opportunities are provided—in-class and out—for students who hail from various cultures to interact among themselves and discover their similarities and differences. In addition, all students are provided with individualized feedback on their initial IDI results by a qualified IDI administrator (the instructor or her LLC support personnel), along with homework assignments that guide each of them through a stage-appropriate process of goal-setting for improvement.

Table 6 Changes made in the leadership seminar from 2013 and 2014 to 2017 and 2018

Following the shifts in curriculum, the research team again evaluated the effectiveness of the TLC experience in advancing students’ intercultural competence, using both the IDI and review of the same reflective essay assignments. In the essays from the 2017 and 2018 cohorts, a much higher proportion of the students showed Level Three mastery of empathy, communication, self-awareness and openness, as measured by the AAC&U intercultural rubric. For example, the comment which follows shows a Level Three mastery of the attitude of Openness. “This experience taught me that not everyone follows the same social norms that I grew up with simply, because we did not grow up in the same environments. Keeping this in mind makes it easy to convey my message in a calm manner, without passing judgments and being rude to them, because I know that they are unaware as to why they should not be using those words.” Fig. 4 indicates which percentage of the cohort reached mastery level three or above in these four aspects of the construct.

Fig. 4

Cohort comparison of building block attributes of intercultural competence

Discussion

The consistency of the initial IDI mean developmental orientation score for each annual cohort of TLC suggests that each year’s group began the semester with a roughly similar capacity to be open to difference. The disparity in concluding DO scores suggests that students in the latter cohorts benefited more than those in the initial TLC cohorts. The qualitative data give us a peek “under the hood” of the intercultural developmental stage to determine specific competency skills gained. Metaphorically speaking, intercultural competence stage, as quantified by the IDI, equals one's position in the climb to the top of Everest, while interpersonal skills, such as empathy, are the mountain climbing tools (akin to ropes and oxygen tanks) which allow you to succeed (Calahan, 2021). The AAC&U rubric analysis suggests that, during the semester, the later cohorts became more open to difference, while practicing empathy and being aware of how their own cultural upbringing was affecting their perceptions.

Regarding our research question, this study appears to uphold Goodman and Salisbury’s findings (2009) about the impact of pedagogy; e.g., indicating that participation in a months-long on-campus learning experience with immersive residential components (similar to a semester abroad with homestay) does not necessarily create a measurable increase in intercultural competence. Rather, as has been found in previous IDI-measured studies, proximity to students from other cultures, even with residential cross-cultural immersion, creates a wide variety of intercultural learning outcomes, some positive and some negative, among individual participants.

Being attentive to the developmental stage of the learners, both as a group and individually is important for higher proportions of student development. (Indeed, we hypothesize that attention to group developmental stage may be particularly important in learning communities.) For students in minimization, it is appropriate to primarily work on self-awareness and the finding of common ground (Bennett, 2004b); ideally the instructor should provide opportunities to supportively address anxieties about difference, but should also limit time spent focusing solely on dissimilarity. The assignments for the TLC leadership seminar are designed now more to support than to challenge: rather than being asked almost exclusively to look for and navigate differences, the TLC participants investigate cultural commonalities and develop cultural self-awareness as well.

This exploratory study suggests that implementation of individualized feedback, structured reflection, and open discussions about student assumptions—a “challenge and support” methodology that is increasingly recognized as important to the intercultural competence goals of study abroad (Johnstone et al., 2020; Paige & Vande Berg, 2012)—may also be helpful for guiding more STEM students towards the acquisition of the global professional skills demanded by employers.

Limitations of the study

While small size may benefit learning community participants, it does not bring statistical power to the findings of any exploratory study. However, we are encouraged that our mixed methods research results appear to align with the large recent body of work suggesting that high-impact pedagogical practices, such as learning communities and global experiences, done well, can foster significant intercultural competence development. For this particular program, we are also conscious that it will take multiple iterations of TLC to determine whether similar IDI growth patterns obtain for additional cohorts.

Conclusions

This study contributes to the STEM education literature by investigating the effect of a globalized STEM learning community for first-year students and providing a model of intercultural learning to cultivate global-ready STEM graduates. The findings provide practical implications to STEM educators in the USA and throughout the world for designing, implementing and assessing intercultural programs on campus. In this current study, we have presented data that suggest that when it uses the high-impact intercultural mentoring methods of cutting-edge study abroad programs, a semester-long learning community for first-year students hailing from around the world, can foster significant intercultural competence growth for domestic and international students alike. Not only does participation in such a learning community foster the skills needed for a successful career, but by investing in these skills early in students’ academic careers, such a program might further students’ integration into and sense of belonging to the full university community—an inclusive win–win situation for all involved. As telegraphed by our deliberate shift from global competence language to inclusivity language in these final sentences, this study also has implications for improving an on-campus climate for diversity and inclusion. As demonstrated by this TLC case study, focusing on developing self-awareness and finding common ground, rather than teaching students about navigating difference, appears to be the key to attaining the program’s learning goals (e.g., reducing students’ ethnocentricity). Turning the temperature up too high or too quickly (e.g., focusing too much on cultural difference) is very likely, as expressed by the memorable metaphor of Janet Bennett (2004a), to cause one’s intercultural frogs to jump out of the hot water to seek safety.