Table 4 Examination of the 7 implementation factors of the sustained STEM intervention
Factor | Consideration of factor in this study |
|---|---|
Adherence | a)Â The intervention was delivered to a loose enabling framework, and therefore, measuring adherence is challenging b) The full programme offer was not delivered in any of the evaluation schools or participating schools c) Activities taken up were delivered according to the intended treatment model |
Quality | a) Feedback from schools and reflection on quality of intervention were good b) Data used within the process evaluation did not allow for this factor to be examined in much depth |
Responsiveness | a)Â There was lower than expected take up of number of programme activities in all schools b) Levels of engagement waxed and waned over the course of the intervention according to changes in school circumstances, particularly related to the school accountability regime |
Dosage | a) Schools generally took up low-impact activities, such as whole-school assemblies and class workshops b) Some of the higher impact elements of the programme offer were not taken up readily (CPD and parental engagement) c) Young people within the measured cohort study group received low levels of intervention at an individual level |
Reach | a) Lower than expected reach among some year groups of the schools b) There was greatest reach among low-impact activities, such as whole-year group assemblies c) The take-up of intervention activity among the measured intervention cohort was low |
Programme differentiation | a) The intervention was only one of many things happening in young peoples’ lives during the intervention period b) New curriculum and qualifications were introduced during this time (2015) |
Monitoring of Control/ Comparison groups | a) Comparator groups have been employed, accounting for natural changes as young people age b) Design of comparator group cannot take into account the counter-factual of education policy and other changes occurring during the programme |