Instrument | Validity/reliability | Used validation/reliability method |
---|---|---|
Pre- and post-questionnaire | Content validity | Expert opinion method: an agreement by 12 independent experts in the field of science education as to whether the content of a measure covers the full domain of the content |
Construct validity | Analysis of the Estonian secondary school science curriculum and syllabus to ensure items are valid in terms of expected learning outcomes. For data analysis CFA was used | |
Reliability | Cronbach alpha = 0.82 over the sample (with each factor over 0.74). CFA was used to test whether measures of the construct are consistent with a researcher’s understanding of the nature of that construct (factors) | |
Interviews | Content validity | Inductive thematic analysis was used to analyze the transcripts of the interviewers´ answers |
Construct validity | Themes identification and labeling | |
Inter-coder reliability | The percentage agreement between two coders (science educators) was, with students’ interviews, 86% and teachers interviews, 78%. For resolving disagreements, the coders negotiated toward consensus. Coders made estimations and negotiated their response before reporting the final outcome (Epley & Gilovich, 2006) |