Raising attainment for low-scoring students through quectures: an analysis of achievement and engagement with personalised learning in lectures

Abstract

Background

Quectures are flipped lectures with embedded ‘quecture questions’, which employ metacognitive reflection as an active learning intervention. Quecture questions are students’ own questions, identified and constructed by students around learning objectives during lectures. The quecture question intervention aims to support each student to engage better with their learning and to work at an individually appropriate level, and is accessible to all students irrespective of learning background. This research explores engagement and performance of students using the intervention, with a focus on those who traditionally receive low scores, to measure the effectiveness of quecture questions.

Results

Accumulated data demonstrate that student engagement with quecture questions is associated with improved learning on wider topics than those on which the intervention was used, as evidenced by improved overall course scores. Students who participate minimally within other elements of the course but who do submit their own quecture questions gain disproportionate benefit. The improved performance benefits of engagement with quecture questions are most marked for students with low prior scores. Students originating from the same country as our university and those whose parents did not attend university represent typically low-scoring demographic groups with low participation levels who might potentially benefit from the quecture strategy. However, lack of engagement with the intervention is also prevalent in these student groups precluding unengaged students from benefitting.

Conclusions

Quecture questions represent a simple yet effective intervention for improving learning in lectures. Broad learning benefits suggest improved learning habits, and support the notion that the intervention leads to students assuming increased responsibility for their own learning. The use of quecture questions generated most benefit to low-scoring students, validating the intervention as a useful tool with which to address learning inequities. The quecture question intervention readily adapts to suit online learning and represents an ideal first step for busy instructors wishing to adapt their lectures towards a more student-centred approach to learning.

Introduction

Calls for science education reform are loud, clear and long-standing (Bradforth et al., 2015; New thinking’, 1965; Volpe, 1984; Waldrop, 2015). The central message is that undergraduate courses should be student-centred, active and enquiry-driven (AAAS, 2009). Despite this, the 50-min lecture has persisted as the central backbone for most university science courses, and most often constitutes an unbroken oratory delivered by an expert teacher to passive students. Active learning, on the other hand, aims to have students both working and learning during the lecture period, practising higher level skills and developing their scientific skills-set towards becoming scientists (McMillan et al., 2018). Active learning normally engages students in reflection, discussion and/or hands-on activities that rehearse or apply the newly learned material, and leads to a deeper understanding of the material compared to didactic lecturing where direct transfer of information encourages a more surface approach to learning. Abundant evidence now unequivocally supports the assertion that active learning and student-centred pedagogy improve student learning (Armbruster et al., 2009; Freeman et al., 2007, 2014; Michael, 2006).

The high level of structure, support and challenge required for high quality active learning offers additional potential for addressing the pressing problem of inequity in universities. It is clear that specific demographic groups, such as under-represented minority or low socio-economic groups and first-to-university students, receive poorer outcomes even when data are corrected for entry qualifications (Haak et al., 2011; Office for students UK, 2018). These educational disadvantages are poorly understood, but are thought to be connected to cultural differences, lack of inclusion, and impoverished learning histories (Bourdieu, 1986). Studies show that active learning, with its increased support and improved student engagement, has a marked positive effect on learning for specific disadvantaged student sub-groups, thus helping to close this gap (Cottone & Yoon, 2020; Eddy & Hogan, 2014; Gavassa et al., 2019; Haak et al., 2011). Indeed, a meta-analysis of such studies has revealed that active learning provides a disproportionate advantage to traditionally low-scoring student groups across STEM disciplines (Theobald et al., 2020).

Reforming lectures to incorporate active learning is challenging for the busy academic (Auerbach & Andrews, 2018), and is less effective without care and belief in the strategy (Andrews et al., 2011; Canning et al., 2018; Theobald et al., 2020). Allen and Tanner (2005) provide a list of suggested activities to enable active learning within lectures, placing strong emphasis on the benefits of discussion and metacognitive reflection. Indeed, Avargil et al., (2018) argue that explicit teaching of metacognitive skills should be an integral part of science education. Structured reflection on one’s own learning might be expected to particularly benefit disadvantaged learners lacking prior experience of such techniques. To that end, Canning et al. (2018) asked new biology undergraduate students to write reflective essays that explored the personal relevance of learning material at the start of their first university semester, and found that engagement and persistence with learning were both improved, particularly for students with a history of low performance. Identifying and constructing their own questions is another metacognitive activity that leads to improved awareness of a students’ own understanding, and improved engagement with learning material (Rosenshine et al., 1996). Teaching students to generate questions can be achieved by scaffolding question-asking skills using procedural prompts which facilitate completion of the task, and results in transferable gains in comprehension (Rosenshine et al., 1996). Studies across multiple STEM disciplines demonstrate significant learning benefits of student-posed questions (Hardy et al., 2014; McQueen et al., 2014; Yu et al., 2003). In these studies classes of students build and review their own online repositories of multiple-choice questions. Aflalo (2021) supported students to construct and classify their own questions as transformational or simply confirmational, using dedicated sessions within a cell biology class. The authors found that this structured activity which gave value to the process of question generation had positive measurable benefits on the students’ abilities to handle assessment questions that required higher-order thinking.

Quecture questions are student-generated questions, scaffolded by response to a minimal generic question stem, within the lecture period (McQueen & McMillan, 2020). This heuristic active learning intervention combines reflective questioning with peer discussion and is similar to the think–pair–share active-learning technique widely used in school classrooms, small group teaching and more recently in biotechnology lectures (Prahl, 2017). Traditionally think–pair–share involves reflection and small group discussion, followed by wider sharing, on an instructor-posed question. The quecture strategy encourages students to capture their own personal confusion, misconception, curiosity or idea associated with the lecture learning objective by constructing their own question for discussion and sharing. The technique is also reminiscent of the ‘muddiest point’ technique (Angelo & Cross, 1993), using questions to construct understanding that exploits Vygotsky’s zone of proximal development (Doolittle, 1997). In this way all students, irrespective of experience, confidence or competency, are encouraged to reflect and build upon their own individual learning (McQueen & McMillan, 2020). Quecture questions form an integral part of a flipped lecture (‘quecture’) whereby content (organised around lecture learning objectives) is provided for study prior to class (Bergman & Sams, 2012). Students are asked to consider their own questions that focus on the learning objectives, as part of their individual preparation, then to refine, and discuss them during live teaching under instruction to ‘think, type, talk’, using three minutes of lecture time for each quecture question. The ‘type’ instruction makes use of a digital personal response system for student-generated questions to be shared during the class with other students and with the instructor. Students are encouraged to further investigate their own questions after the lecture, with the intention of building habits of taking responsibility for their own learning. Quecture questions are also revisited for general comment by the instructor at the start of a specified future lecture.

The improved learning habits, and improved personalisation of learning that students report as a result of the quecture question intervention (McQueen & McMillan, 2020) might be expected to particularly benefit low-scoring students and to improve learning that is not limited to the lecture block employing the intervention. Using accumulated voluntary engagement and performance data, over repeated iterations of the course, this manuscript explores the following questions:

• 1) How well do students engage with the quecture question intervention?

• 2) Does engagement with the quecture question intervention have any effect on overall student learning on this course?

• 3) What is the relationship of quecture learning benefits to general student participation on the course, and to prior performance?

• 4) Does quecture engagement have the potential to benefit traditionally low-scoring demographic groups of students?

Method

Context and ethics of study

This study took place over five annual iterations of an introductory genetics course that runs in the second semester (January–May) each academic year. The standard undergraduate degree at this Scottish university has a 4-year programme and most biology students take this course in their second year. The course consists of lectures, practicals and tutorials and is taught by a team of around six academic staff, with the author being responsible for seven or eight of the 30 lectures each year. The remaining lectures are interactive, but not flipped lectures. The course content was not significantly changed between years although some lecture objectives were refined or combined to better focus student learning. Student course scores were generated from a single-sitting summative exam that contained three components (worth 20% each); a multi-part problem, an essay and a multiple-choice test based mostly on application of concepts, as well as an in-course data handling test (also worth 20%) and three smaller items of coursework. The content of all four large components varied at each sitting.

Student contextual data were self-declared by a voluntary tick-box questionnaire completed by 248 and 265 students in only the first two years of the study (2016 and 2017, respectively). Class demographics with respect to the four contextual questions: gender, geographical origin, first in family to university (or parents attended university) and previous education, were similar in both years in which they were collected. Across both years there were 68% female students, 29% males and 3% gender other students. 33% of students were Scottish, 24% were from the rest of UK, 20% from the rest of Europe and 23% were from other international locations. 30% of students were first in family to attend university while 70% had parents that had attended university. 57% had attended a state school, 29% private school and 14% had studied at a further education college or other place. Private school is paid for by the family while state school is free. Around 40% of our Scottish students were ‘widening participation’ students who are often admitted to this university with relaxed academic entry requirements. 53% of the students on this course in 2017 (which was a typical year) were studying for degrees in biology, with 42% studying for biomedical or other science degrees and 5% doing non-science degrees. The study received ethical approval (reference hmcqueen-0001) from the School ethics committee which adopts the UK research integrity office code of practice for research. Where contextual data were collected, students were advised about the method and purpose of the study before voluntarily providing any personal information and were free to withdraw from the study at any point. All data were anonymised after collation and stored digitally on a password-protected computer in accordance with the EU General Data Protection Regulation (GDPR), and the UK Data Protection Act.

Quectures

The quectures formed an integral part of the lecture course in all 5 years of the study (three quectures in 2016, eight in 2017, seven in 2018, seven in 2019 and seven in 2020). The quecture question purpose and method was explained to students using a short video, an entry in the course book and ‘in-lecture’ guidance. The purpose was summarised as helping each student to ‘retain more, apply better’ and further explanation of the intended benefits included that it would encourage students to (i) think rather than just remember; (ii) personalise their learning; (iii) identify and deal with their own gaps or misconceptions, and (iv) take responsibility for their own learning. Explanation of the method included clarification of two rules: (1) that student questions should be relevant to the learning objective under discussion, and (2) that the task of answering quecture questions falls to the students during independent study. The range of possible question types was explained clearly both by video and during lectures. A preparation list of learning material was provided to students one week in advance that included a short quiz (contributing a minimal proportion of course assessment points) with instruction to consider their own questions about the learning objectives prior to lectures. Lectures were organised into one, two or three blocks (according to the number of learning objectives) each comprising a mixture of peer instruction questions, demonstrations and lecturer explanations before pausing for the quecture question. At this point 3 min were set aside for students to ‘think, type and talk’ in response to a prompt that reiterated the learning objective and added the instruction to ‘Discuss and refine your own question to advance your understanding or reflection’. Peer instruction and quecture questions were shared and answered using students’ own mobile devices and the ‘Top hat’ personal response system. Students could read, ‘like’ and comment upon other students’ questions during the session. Submitted questions were then revisited by the instructor at a later lecture, providing guidance on how students could address their own questions. The number of likes could be used as a guide for the instructor to gauge the importance that students attached to each question.

Data collection

Quecture questions submitted by students during lectures over five iterations of the course were collected from the ‘top hat’ records. All questions were checked for relevance and 48 non-valid questions (mostly incomplete duplicates) were removed before noting the total number. In the three middle years of the study (2017, 2018 and 2019) the number of quecture engagements was recorded for each student. A student’s quecture engagement was classified at one of three levels: high engagers (QH) submitted three or more quecture questions; low engagers (QL) submitted one or two questions, and non-engagers (Q0) did not submit any. Three quecture submissions was chosen as the cut-off for high engagers as this generally involved submitting questions in more than one lecture.

General participation in ‘Top hat’ personal response questions during all lectures (apart from quecture questions), was recorded for each student for four iterations of the course (although ‘Top hat’ was used in the fifth year, in-person attendance was disrupted due to the global pandemic). Participation was measured by counting the number of ‘Top hat’ (non-quecture) questions each student responded to across all 30 lectures. The total number of such questions was 112, 89, 95 and 102 in 2016, 2017, 2018 and 2019, respectively. All but three students who submitted quecture questions were within the subset of students that participated on the course via ‘Top hat’.

Course results spreadsheets were collected for the course under study and for the prior learning course, which was also a second semester course focussed on molecules, genes and cells, but taken in the first (previous) year of the degree. Students within each year were classified into one of four equal cohorts (quartile A for lowest scoring and quartile D for highest scoring) based on course scores from prior learning. An additional cohort (E) included students who had not taken the feeder course the previous year. We categorised prior scores into cohorts in this way, rather than simply using the prior score directly, to allow us to include the students without prior learning in our analysis, since it was possible that quecture effects would differ in this group given their different prior experience. Seven students with course scores of zero in the course under study were removed on the assumption that they had withdrawn from the course, but otherwise datasets were not pruned. Some of the data collected in the earlier years of the study were limited due to GDPR.

Data analysis

• 1) How well do students engage with the quecture question intervention?

  We first plotted the number of students submitting quecture questions across all 5 years of the study (2016–2020). The hypothesis that this would include students achieving both low and high course scores was further investigated by a scatter plot of quecture submissions against course scores in both the trial and prior courses, for students from 3 years of the study (2017, 2018 and 2019). These 3 years were used in all analyses of the effects of quecture question submissions on course scores because 2016 was a smaller scale pilot trial involving only three quectures and 2020 scores were not comparable due to the global pandemic.

• 2) Does engagement with the quecture question intervention have any effect on overall student learning on this course?

  We next tested for an association between quecture engagement and course mark by fitting a General Linear model (GLM) to our 2017–2019 data using the lm function in R. We used course mark as the response variable and fitted quecture engagement as a three-level fixed factor. We also included year as an additional three-level factor to account for any year-to-year variation in average course mark, and an interaction between quecture engagement and year to account for any difference in quecture effects among years. In this and subsequent analysis year was treated as a fixed factor due to the fact that we only have 3 years data, and these years are not a random sample of possible years. Formally, this means our statistical conclusions are limited to these three cohorts, but we do not see anything to suggest that any conclusions drawn from these three cohorts should not apply more generally.

• 3) What is the relationship of quecture learning benefits to general student participation on the course, and to prior performance?

  We hypothesised that quecture engagement effects might differ between students with differing levels of prior score or general participation. To examine the interaction with prior scores we fitted the same GLM as above, but with prior learning cohort included as an additional fixed factor with 5 levels, a cohort-by-quecture interaction to examine whether any quecture effect varied among cohorts and also a cohort-by-year interaction. We then looked at interactions with general participation level of students using the percentage of ‘Top hat’ (non-quecture) questions each student answered in lectures across the course as a measure of general participation. We started with the same basic GLM as we used for the first model but with the participation score of a student included as a continuous linear covariate. We also fitted a quecture-by-participation interaction to test whether any effect of quecture engagement was consistent across general participation levels and a participation by year interaction to ensure that any relationship between participation and mark was consistent across years.

• 4) Does quecture engagement have the potential to benefit traditionally low-scoring demographic groups of students?

Finally, we hypothesised that quecture engagement had the potential to improve traditionally low course scores for groups of students belonging to specific demographic groups. We only had demographic information for the years 2016 and 2017 with which to investigate course score patterns but, since quectures were not fully used in 2016, our subsequent analysis of quecture effects was limited to 2017 only. To first look for effects of the four different demographic variables on overall performance across both years we fitted a GLM with course score as the response variable and each of our four demographic variables (gender, origin, whether parents were at university and last place of education) as well as year fitted as fixed factors. We also fitted pairwise interactions between each of the factors to examine higher-order effects. To then examine whether effects of quectures differed for different demographic groups we fitted another GLM to the 2017 data which included the same four demographic variables and quecture engagement as fixed effects, as well as pairwise interactions between each demographic variable and quecture engagement to examine whether effects of quectures were consistent across different levels of each demographic variable.

All statistical analysis was carried out using R version 4.1.2. (R Core Team, 2021) and using an Alpha level of 0.05 for all significance tests. For all GLMs, diagnostic plots were examined for deviations from model assumptions of normality and homogeneity of variance and, where relevant, points with high influence. In some cases these suggested deviations from normality were caused by students that did not achieve a pass mark for the course generating an extended tail to the distribution, but their numbers were low, and linear models are robust to small deviations from normality so we do not believe this affects the conclusions of the models. All other assumptions appeared to be met. We used the following process of model simplification to obtain minimal models. Starting with the maximal model, non-significant terms were excluded sequentially starting with the highest order interactions. Where a model contained several non-significant terms at the same hierarchical level, the term with the largest non-significant p value was removed first. In situations where an interaction was significant, main effects involved in that interaction were retained irrespective of significance to maintain model hierarchy (Grafen & Hails, 2002). P values for significant terms are reported from the minimal model, while P values for non-significant terms are reported from the point of their removal. To examine differences between specific levels of a factor after a significant omnibus test, we used the TukeyHSD function to form pairwise comparisons. To assist with interpretation of significant interactions, least squares means and standard errors were extracted from models using the lsmeans function from the lsmeans package (Lenth, 2016).

Results

Student engagement with the quecture question intervention

Quectures, or flipped lectures employing the quecture question, were deployed over five iterations of the same second year genetics course. Over the five consecutive years students submitted a total of 2266 of their own quecture questions in response to 71 instances of the intervention within 29 quectures (Fig. 1). The proportion of enrolled members of the class that engaged by submitting quecture questions ranged from 27 to 37% in individual years, averaging 32% over the 5 years (Table 1).

Fig. 1

Quecture question submissions over 5 years of study. The total number of student submissions (y-axis) to each of 71 quecture question interventions (x-axis) over 5 years is shown. The number of lectures (L1–L8) varies between the years, and the number of interventions within each lecture ranges from one to three, dependent on the number of learning objectives. Three quectures given by another instructor in 2020 are not included

Table 1 Numbers of students taking part in voluntary interventions over 5 annual course iterations

Figure 1 shows that the number of quecture question submissions decreased across multiple learning objectives within most lectures, and that engagement with the intervention was highest at the start of each year and tailed off throughout the year (Fig. 1). Closer analysis of engagement patterns showed that different students were active in different quectures within one year, and that new students were engaging, even within the last session.

The number of quecture questions submitted by individual students was plotted against their course scores, for both the course under study and for a related prior course, to observe whether the low proportion of students that engaged was biased towards high-performing students (Fig. 2). On the course under study, there is a bias towards better scores for good engagers (Fig. 2a) that is consistent with an association between performance and quecture engagement. This bias is less marked on the prior course where a scatter of students submitting multiple quecture questions are found on the left-hand, low-scoring end of the plot (Fig. 2b), indicating that students with a variety of performance histories do engage with quectures.

Fig. 2

The relationship of quecture question submissions to students’ course scores. Scatterplot showing scores for a the course under study, and b a prior course, for students submitting one or more quecture questions across the 2017, 2018 and 2019 iterations of the course combined

Effects of quecture engagement on course scores

Course outcomes for students that engaged by submitting quecture questions were investigated over 3 years (2017–2019). Whilst average course scores differed among years (year effect: F_2,1041 = 3.41, P = 0.033), there was no significant interaction of quecture engagement with year (quecture-by-year interaction, F_4,1037 = 2.07, P = 0.083) and, once year variations were controlled for, there was a clear effect of quecture engagement on course score (Fig. 3, quecture effect F_2,1041 = 39.17, P < 2 × 10^–16), with both high and low engagers doing significantly better than non-engagers (Tukey comparisons P < 0.05). Average course scores for students that engaged well with quectures (QH) were 2.9%, 8.7% and 8.1% higher than the class averages for these 3 years, and 5.0%, 11.9% and 10.3% higher, respectively, than the scores for those that did not engage with quectures (Fig. 3).

Fig. 3

The effect of quecture engagement on course scores over 3 years. Average course results for students that did or did not submit quecture questions are shown for each of three consecutive yearly course iterations. For each year the class course average is indicated by a broken line (64.2% in 2017, 62.4% in 2018 and 59.5% in 2019). Standard error of the mean is shown for each sub-group. QH  high quecture engagers, QL  low quecture engagers, Q0  students that did not engage with quectures. Both high and low quecture engagers have significantly higher scores than those who did not submit a quecture question (Tukey P < 0.05 for QH vs Q0 and QL vs Q0)

Relationship of quecture learning benefits to general student participation, and prior performance

Submitting a quecture question might itself be taken as an indication of diligence, a quality that could be expected to produce a positive outcome for a student irrespective of engagement with this particular intervention. Similarly, we expected that prior learning might influence engagement, and thus benefit, from the quecture intervention. To try to understand the relationship between these interconnected factors, we included measures of prior performance and online participation with questions during lectures (other than quecture questions) in our analysis of quecture benefits.

Whilst high quecture engagement, corrected for prior learning, was associated with elevated performance overall (Fig. 4, quecture main effect, F_2,1029 = 15.60, P = 2.11 × 10^–7), the exact effect differed in different cohorts (quecture-by-cohort interaction, F_8,1029 = 2.29, P = 0.020). Overall, the effect of engaging with quectures appears to be reduced in mid-scoring quartiles, while larger effects are attributed to the lowest and highest scoring cohorts.

Fig. 4

The effect of quecture engagement on course score for different prior performance cohorts. Least squares means and standard errors (corrected for between-year and sample size differences) are plotted for each cohort quecture combination from the 2017, 2018 and 2019 iterations of the course combined. Quartiles A–D were of equal size within each year (50 students in 2017, 83 or 84 students in 2018 and 79 or 80 students in 2019) while group E contained 65 students from 2017 and 2019, and 67 students from 2018. QH  high quecture engagers, QL low quecture engagers, Q0 students that did not engage with quectures. Whilst overall quecture engagers (QH and QL) have higher scores than Q0 (P < 0.0005), the benefits of quecture engagement varied among cohorts (P = 0.0198) with most benefit for cohorts A, D and E. * p < 0.05 **, p < 0.01, *** p < 0.005, n.s. = not significant

Next, we investigated the effects of quectures when corrected for participation in other digital lecture interactions across the course. As expected, students that engaged with quectures more in general performed better on the course (Fig. 5, quecture effect F_2,1038 = 3.89, P = 0.021). However, the effect of quectures also depended on the general level of participation (quecture-by-participation interaction F_2,1038 = 3.47, P = 0.032) with larger benefits at low levels of participation, and no real effect at higher levels of participation (Fig. 5).

Fig. 5

The effect of quecture engagement on course score at different levels of participation. Least squares means and SE are shown at three levels of participation corresponding to the 25th percentile (2% participation), median (16.2% participation) and 75% percentile (39.3% participation) participation levels of students. Scores are combined from 3 years of study (2017–2019). QH  high quecture engagers, QL  low quecture engagers, Q0 students who did not engage with quectures. The increased performance of high and low quecture engagers depends on the level of participation (P = 0.032) being greatest in the lowest participation group, and becoming smaller as participation levels increase

Analysis of differential course outcomes and quecture engagement for demographic groups of students

Given the disproportionate benefits of quectures for low-performing and low-participating students, we wished to investigate the relevance of these results to demographic groups who often score lower at university. Academic outcomes within specific demographic groups were first investigated via students in the 2016 and 2017 cohorts who had been invited to self-report their identity with respect to gender, geographical origin, and educational background (prior place of learning and if they were first in family to attend university). To allow visual comparison across the two different years of study, average course scores for each of the demographic sub-groups were expressed as differences (positive or negative) from the course average for that year (Fig. 6).

Fig. 6

Variations in course score within demographic groups. Relative course scores within four demographic categories for each of 2 years (2016 and 2017) are shown as differences between the average score for each group and for the whole class. RUK  rest of UK (not Scottish), Eur  rest of Europe and Int  international (all other parts of the world). Parents  parents attended university, First  student is first in family to attend university. State  state school, private  private school. In 2016 there were only two ‘gender other’ students giving a high average score of 78%, indicated by a thin blue line. *Significant difference (P < 0.05) between course scores for this sub-group and all other sub-groups within the same category

The patterns of course outcomes for the four demographic categories were consistent across both years of the study with one sub-group from two demographic sub-categories outperforming the other groups within that category (Fig. 6, Tukey comparisons P < 0.05). Specifically, non-UK European students out-performed non-Scottish rest of UK and other international students, who in turn both out-performed Scottish students (F_4,502 = 14.16, P < 0.0005); students whose parents had been to university out-performed students who were first in their family at university (F_2,502 = 3.68, P = 0.026); while there was no significant difference in average course score based on gender (F_2,490 = 0.54, P = 0.584) or school (F_3,502 = 2.29, P = 0.078). It was reassuring to note that groups that scored below the class average in 2016 did better in 2017 when more quectures were incorporated into the course (Fig. 6).

Next, 2017 students within demographic sub-groups were further divided with respect to their engagement with the quecture intervention (Table 2). Groups who engaged with quectures and had an average course score increase that was larger than the average course benefit are shown in bold in Table 2. Group sizes for sub-groups that did or did not engage with the quecture intervention were small and no statistically significant differences in course scores for high and low quecture engagers were detected when demographic effects were controlled for (F_2,248 = 2.20, P = 0.113).

Table 2 Average course scores (2017) for quecture engagers within demographic categories

Table 2 also shows the proportion of students that engaged with the quecture intervention to vary across demographic sub-groups (n engaged vs n not engaged with quecture). Scottish and first-to-university students perform least well within their category (Fig. 6) and show the lowest engagement with quecture questions (Table 2). Table 2 also shows that the lowest general participation with ‘Top hat’ questions (across the course) is similarly attributed to Scottish, rest of UK and first-to-university students. One further striking observation was the low general participation for students that attended private fee-paying school prior to university, who score well on the course (Table 2, Fig. 6).

Discussion

The quecture intervention leads students to engage with their own information-seeking questions, misunderstandings, and information-checking questions or more sophisticated questions that probe or synthesise knowledge, or raise wider interests. Previously students reported that quectures encouraged personalised learning and improved their enquiry skills and learning habits, leading them to take increased responsibility for their own learning (McQueen & McMillan, 2020). This preparation for future learning along with the effortful nature of the quecture intervention from a student perspective might be expected to lead to transfer of learning (Bransford & Schwartz, 1999). This research shows that constructing and submitting quecture questions is associated with improved overall course scores, suggesting benefits that do indeed extend beyond improved understanding of the lecture block within which the quecture intervention was employed.

Engagement with quecture questions is associated with improved course scores above that predicted by general engagement on the course, specifically for low-participating students, which suggests that the quecture benefits are not merely due to engagement by diligent students that would perform better in any case. Importantly, we have also shown that engagement with quecture questions is of particular benefit to students with low prior performance that is typical of some demographic groups with perceived educational disadvantages.

When student scores were separated with respect to prior learning, the most consistent significant quecture benefits were demonstrated for the previously lowest performing students. Benefits were also clear for highest performing and direct entry students (who had not been part of the cohort in the previous year), but the benefits were less marked within mid-performing sub-groups of students engaging with quecture questions. This finding has interesting parallels with a similar prior learning analysis of the benefits of the unrelated ‘Peerwise’ intervention (McQueen et al., 2014), where mid-performing students were also concluded to derive the least benefit. This variable penetrance of learning interventions has been seen elsewhere (Bates et al., 2012; Denny et al., 2008; Hardy et al., 2014) and depressed benefits for mid-performing students has been postulated to result from habitual strategic learning aimed at gaining sufficient marks via minimal effort.

Low engagement appears to be the main obstacle to extracting the full benefit of the quecture strategy across the student population. On average, around one-third of students engaged overtly with the strategy, and this group was unequally spread across prior performance and demographic sub-groups. Lack of engagement was most marked in student sub-groups with the lowest scores or highest educational disadvantage, and the greatest need for (and potential benefit from) the intervention. This confirms the urgent need for further research towards unravelling and reducing barriers to engagement. Engagement is understood to result from a complex and dynamic interplay of social, academic and behavioural factors involving the student, the task, the teacher and the learning environment (Kahu, 2013). It is impossible without further research, therefore, to explain the lack of uptake by some students, but changing the design of some aspect of the intervention could help. Although communication on the purpose and benefits of this intervention were considered clear and iterative, the support around question construction was necessarily limited owing to its implementation by one instructor within large live classes. One obvious improvement would be to allocate more time specifically to teaching students to generate questions alongside the quecture intervention.

Temporal variations in engagement were also noted in this study, with a general trend towards more quecture question submissions at the start of term and fewer as term progressed. Higher engagement was also frequently observed for the first learning objective within any one lecture compared to learning objectives two and three (where used). This might indicate that enthusiasm for the technique waned both throughout a lecture and throughout term. Alternatively, students may have found the seemingly simple task of asking a question cognitively demanding in the context of a university lecture. One can envisage this as a particular issue for a struggling student. Perhaps students tried to engage with the strategy, but found the accumulating cognitive demands within a lecture (or term) to progressively outweigh their ability to handle the deceptively complex task of identifying and then articulating the gaps, misconceptions and connections in their own learning. This being the case, the extra embedded support for learning to generate questions suggested above could help. Partial engagement, such as students identifying comprehension gaps or considering their own question, that fell short of our measured activity of submitting questions, would still be expected to be valuable to the student. Indeed, the reported levels of engagement in this study are likely underestimates owing to significant vicarious engagement with the quecture strategy that was not recorded. Arguably many students formed and perhaps discussed questions without submitting them (for a variety of reasons), and would have received associated learning benefits.

While the benefits of the quecture strategy are encouraging, it is important to note that this data pertains to one biology course at one institution and one instructor. The strategy has been employed by other instructors with students on different courses and at different levels of study with anecdotal positive feedback but no data with respect to educational benefits is available to date. There is no reason, however, to suggest that the strategy’s success should be limited to biology, or even STEM teaching.

Subsequent to the collection of data discussed here, the global pandemic effected seismic change to university learning and teaching practice. Notably, there has been a widespread move to present lectures online as shorter pre-recorded videos with live interactive follow-up sessions with the instructor (effectively a move to flipped lectures), rather than as 50-min live lectures. The quecture strategy, originally designed as an interactive intervention that would break up the 50-min lecture, can also be well used alongside these shorter videos. Indeed, pairing the watching of each video with the task of constructing one’s own question is an ideal strategy to tackle the danger of passivity, exactly as in the live lecture, but with the added possibility of a reduced cognitive load since students can carry this task out at their own pace rather than in a time-constrained portion of the lecture.

Conclusion

The quecture question is a simple active learning intervention that provides time and structure to support each student to engage with their own questions about learned material during the lecture period and beyond. Constructing and submitting quecture questions is associated with significant learning benefits that support students’ perceptions that repeated use of quecture questions encultures habits of pausing and reflecting upon current learning, discussing with peers, and of students assuming responsibility for their own learning. If real, such improved learning habits might explain the finding that students with low prior performance or low general participation, possibly due to educational disadvantages, enjoy the greatest learning benefits from the quecture question intervention.

Whether we see a post-pandemic return to previous teaching habits, or whether we use what we have learned to advance the reform of HE learning, the pressing need to attend to social and academic inequities within our student populations persists, as does the requirement for a better understanding of the complexities around student engagement. The quecture question represents a simple yet powerful tool that can be employed alongside any defined learning objective to encourage habits of personalised construction of knowledge, and to improve course scores for students, irrespective of unseen educational disadvantages.