Table 5 Our Revealed Causal Mapping (RCM) method

1. Choose participants^1,4 and collect data^1,3,4

Select participants who are experts in the area of interest for the study. Collect data in the form of interviews, focus groups, and/or artifacts.

We interviewed instructors with experience teaching SCALE-UP physics.

2. Purposively sample data⁴

Depending on the amount of data collected, sample data to include data from a broad group of participants.

We sampled data based on the instructors’ institutional factors and SCALE-UP and overall teaching experience.

3. Identify causal statements^1,2,3,4

Identify causal statements by searching for key linking words.

Searching for: if, then, because/‘cause, so, since think/thought, know/knew, use, believe, feel/felt.

Same

4. Separate causes and effects^1,2,3,4

Separate causal statement into causes and effects by looking at directionality of linking words.

Same

5. Inter-rater reliability on identification of causal statements^1,4

Investigate reliability of coding for causal statements.

First two authors independently coded each interview for causal statements and these responses were compared.

6. Identify relevant concepts^1,3,4

Group frequently mentioned words from both causes and effects into categories (names are based on the participants’ own words).

Same

7. Inter-rater agreement on concept identification¹

Investigate reliability of concepts identified in causal statements.

Same

8. Construct raw causal map for each participant^1,2,3,4

Use concepts and causal statements to construct causal map for each participant.

This was the end of our sample analysis as we only investigated four instructors. Future studies should continue the process.

9. Aggregate raw causal maps based on relevant sampling variables^1,2,3,4

Combine causal maps for related participants to create an aggregate causal map. Participants can be grouped by variables of interest.

In future studies, we will group instructors by their institution and prior SCALE-UP and overall teaching experience.

10. Member checking²

Discuss causal map and aggregate causal maps with participants to check for appropriateness of interpretations and correctness of findings (Lincoln & Guba, 1985).

In future studies, we will follow-up with instructors to check our interpretation of their interviews and causal statements.

11. Analyze aggregated causal maps^1,2,3,4

Use metrics such as the point of redundancy^1,2,3,4, adjacency matrix^2,3,4, reachability matrix^1,2,3,4, centrality^1,3, and density^1,4 to analyze and validate the aggregated causal maps.

In future studies, we will investigate the aggregated causal maps.