Table 1 Codebook used for content analysis

Instructor relates concepts directly to scientific work.

How often did you relate course material to scientific research?

Course concepts are related to one another and to outside bodies of knowledge.

Did you use concept maps in your most recent introductory course?

Students analyze or manipulate data.

Students use technology tools to analyze data.

Conversation or debate about course concepts among students or between students and the instructor.

In my interactions with students in this subject, I try to develop a conversation with them about the topics we are studying.

Students extend and apply knowledge to new or challenging situations or to relevant contexts.

What percentage of time on average did your students spend working on problems or projects that required students to seek out new information not previously covered in class?

Students’ are provided with opportunities to demonstrate their conceptual understanding, process skills, or behaviors.

In your selected course section, how much do you encourage students to explain course material to other students?

Forms of instructional practice described broadly or vaguely and not described by another code.

Activities are used to help students learn statistics.

Group work described broadly and without specific product or context.

How often do you put students into pairs or small groups for most of a class period to answer questions or solve problems?

Group work involving solving problems.

Students engaged in structured collaborations to solve problems.

Students complete in-class tests or quizzes in groups or complete out-of-class assignments in groups with the goal of creating a formal product.

Did you use group and team projects that produce a joint product?

Students are doing lab or experiment-based tasks.

How frequently did you use fieldwork in your most recent introductory course?

Students solve problems individually in class. Problems could be conceptual or computational.

How frequently did students solve/discuss quantitative/mathematical problems in the lecture portion of course?

Students engage in a scientific investigation with varying degrees of guidance from the instructor.

How frequently did students design experiments/activities in the lecture portion of course?

Students read primary literature that may or may not be provided by the instructor.

In your interactions with undergraduates, how often do you encourage them to look up scientific research articles and resources?

Strategies to encourage students to think about what they know and why they know it.

In how many of the courses that you teach do you use reflective writing/journaling?

Students have some level of control over what they learn and/or the order in which they learn.

Students posed and solved their own problem.

The instructor speaks and students passively listen.

How often did you use extensive lecturing (more than 15 min per session without breaks for questions or active engagement of students)?

The instructor displays a phenomenon or how to do a task.

How frequently did you use lecture with demonstration in your most recent introductory course?

The instructor posing questions to the class as a whole and receiving individual student responses.

How often do you address questions to the class as a whole?

Students work within a real-world context OR the instructor relates content to real-world examples.

Did you use service learning, co-op experiences, or assignments requiring interactions with the community or business/industry?

Assessment designed to evaluate students’ understanding of course concepts or ability to perform a given task.

One use of my assessments is to reveal whether students are using statistical language properly.

Practices that elicit, build upon, or evaluate students’ knowledge and ideas prior to a summative assessment.

Have you used a pre-test of prior knowledge?

Forms of assessment described broadly or vaguely and not described by another assessment code.

Did you use exams in your most recent introductory course?

Instructors’ set of rules about how students are assessed and how those policies are communicated to students.

In how many of the courses that you teach do you use grading on a curve?

Students complete in-class tests or quizzes in groups.

My assessments include a component in which students are required to collaborate (e.g., group project, group quiz)

Details of how (e.g., frequency, immediacy, and level of detail) instructor provides feedback on student work.

In how many of the courses that you teach do you use electronic quizzes with immediate feedback in class?

Type or format of question and/or the use of a variety of question types on summative assessments.

Have you used essay questions?

Students provide feedback on each other’s work.

Have you used student evaluations of each other’s work?

Students present their ideas or findings to the class in a formal presentation.

Did you use oral presentations in your most recent introductory course?

Students produce a formal writing product.

How often did you use writing assignments (reflective writing, journals, essays, reports)?

Instructors gather and use information to inform their future instruction.

Have you used mid-semester course evaluations regarding your teaching?

Learning goals are provided to students.

In your undergraduate courses, to what extent do you clearly explain course goals and requirements?

Instructor recommendations that encourage behavior that will lead to student success in the course.

In your interactions with undergraduates, how often do you encourage them to ask questions in class?

The instructors’ rationale behind particular instructional, curriculum, or assessment strategies.

I feel it is important to present a lot of facts to students so that they know what they have to learn for this subject.