NE STEM 4U: an out-of-school time academic program to improve achievement of socioeconomically disadvantaged youth in STEM areas
© Cutucache et al. 2016
Received: 24 July 2015
Accepted: 10 March 2016
Published: 24 March 2016
The Nebraska Science, Technology, Engineering, and Mathematics 4U (NE STEM 4U) program was initiated at the University of Nebraska at Omaha (UNO) in 2013. NE STEM 4U is a student-run, faculty-led program facilitating problem-based learning (PBL) sessions in science, technology, engineering, and mathematics (STEM) for socioeconomically disadvantaged kindergarten through grade 8 (K-8) students. PBL sessions are provided throughout the academic year in a twice-weekly, after-school, informal education program. The instructional material provided after school builds upon the curricula of the school day. Importantly, this program is a partnership between faculty members and administrators in higher education at UNO with community partners of Omaha including Collective for Youth, Beyond School Bells, and Omaha Public Schools. We focus on engaging K-8 youth in after-school immersion experiences in STEM fields using undergraduate students as mentors and facilitators using a model of problem-based learning.
This program fosters an educational pipeline for students with hands-on experience in problem-solving and critical thinking. The partnerships among the community provide the foundation for success for students across the K-16 pipeline.
Herein, we describe the model of this program as documented by demonstrated successes to date in an effort to guide others in developing such a model in their city or region. We also provide models for implementation of assessment instruments.
KeywordsSTEM Outreach After-school Out-of-school time Academic programs Underprivileged Metropolitan youth
There exists a disparity between skills of college graduates and those that employers expect from employees (Employer Survey from NACE, 2015). Employers routinely cite the need to “teach” critical thinking, problem solving, and teamwork to their potential employees. Moreover, other countries routinely outscore the USA in the areas of science, technology, engineering, and mathematics (STEM), and students from low-income families and minority status often have exacerbated deficits in these areas (Sacco et al. 2015; Knopf et al., 2015; NDE, 2015).
A similar call to action comes from the American College Testing group, who report that 78% of high school graduates are not ready for entry-level college classes. We are projected to need several million new college graduates in STEM by 2018 (Carnevale et al. 2010). Nebraska certainly reflects this national trend with a projected 48,000 new STEM positions expected to be left unfilled in 2018 (U.S. Department of Commerce, Economics, and Statistics Administration, 2011). Clearly, a need exists for improved STEM education and training across kindergarten through undergraduate degree completion (K-16).
Given the short timeframe for addressing the STEM deficit, programs that engage youth in structured, high-quality after-school programming that excite youth about STEM areas through consistent, mentored activities while also training STEM undergraduates in career readiness skills have the best chance to make immediate impact as well as establish and maintain a competitive pipeline in STEM. President Obama’s 2010 executive action Reform for the Future supports this in the USA (https://www.whitehouse.gov/issues/education/reform) (Reform for the Future 2012). To address these needs, the University of Nebraska at Omaha (UNO) has implemented a program called NE STEM 4U. This program connects undergraduate STEM majors with students from low-income schools in kindergarten through grade 8 (K-8), to provide after-school, dynamic STEM learning activities that have the potential to benefit both student populations.
NE STEM 4U structure
NE STEM 4U is a Nebraska-based out-of-school time program that takes place in the hours immediately after the school day throughout the academic year. The program targets underprivileged, minority, and underperforming students in schools in Omaha wherein the standardized assessment rates in science and mathematics are the lowest in the state. One hundred percent of participants are on free or reduced lunch. The model of the NE STEM 4U program is outreach through teaching, research, and mentorship by undergraduate students from the University of Nebraska at Omaha (UNO). UNO students are selected through a careful application and interview process. Once accepted, they are trained and led by a team of faculty representing the STEM disciplines as well as education.
The educational experience for NE STEM 4U students is further enhanced through participation in discipline-based education research (DBER). A large body of evidence supports the benefits of research in undergraduate education (Gregg-Jolly et al., 2011; Lopatto, 2007; Lopatto, 2005). Similarly, teaching experiences greatly enhance preparation for STEM careers (Gregg-Jolly et al., 2011). NE STEM 4U students add value to their own educational program through a synergy of teaching and research while inspiring a future generation of STEM students with their outreach. This provides immediate as well as long-term impacts on STEM preparation (DuBois et al., 2002).
Out-of-school time infrastructure in Nebraska
In the state of Nebraska, a state-wide agency called Beyond School Bells and a city-wide, non-profit agency called Collective for Youth manage all such programs. Communication among NE STEM 4U faculty leaders, community groups, and state groups on assessment and impact is frequent and ongoing, thereby making it relatively easy to change direction or to make change happen in a rapid manner.
Objectives of NE STEM 4U
Objectives for K-8 student participants
We aim for all K-8 students who are performing these experiments and participating in these activities to behave like scientists and feel like scientists! In our program, they ask questions, make observations, collect data, analyze results, and draw conclusions just as professional scientists do. Taken together our overarching research question related to K-8 student participants, with hands-on explorative learning through NE STEM 4U, do the K-8 student participants show an increase in content knowledge? This overarching question is broken down into specific questions and objectives for K-8 student participants below.
K-8 students participate in activities from all STEM areas. For example, our STEM lessons include the following: biology—strawberry DNA extraction where students extract DNA from strawberries and are taught about DNA, who/what shares DNA, the process of isolating the DNA, etc.; chemistry—dry ice lesson where students participate in a series of activities to learn about dry ice, gas properties, phase changes, sublimation, etc.; engineering—making a fruit battery to teach the concept of generating an electrical current using the natural chemicals found in a fruit or making a solar-powered smore out of an old pizza box; mathematics—calculating heart rate, timing races across the room with different conditions, or quantifying and comparing values by taking the mass of single-stuffed verses double-stuffed Oreos to see if they are in-fact double-stuffed. Moreover, we ask all undergraduate student mentors to spend the last few minutes of each activity discussing a famous STEM professional or a specific career in STEM that complements the activities that day.
Demonstrate a greater proficiency in STEM fields;
Improve critical thinking and problem solving skills;
Decrease behavioral problems that develop when students are not involved in structured activities after school;
Participate in activities of which they can easily replicate at home with siblings and parents thereby learning through iteration and gaining confidence as they become the “teachers” (extra materials from each experiment are given to students in order to do just this); and
Build upon lessons from the regular school day using hands-on experiments, to provide additional perspective.
Each of these objectives has been linked to assessment instruments.
Objectives for undergraduate student mentor participants
Gain proficiency in STEM fields and enhance the education experience;
Engage in community service;
Expand confidence and self-esteem by improving organizational skills and STEM content knowledge through mentoring;
Prepare for careers in STEM fields by developing communication skills and integrity/dependability; and
Increase proficiency in research.
Each of these more specific objectives map to directive research questions and have been linked to assessment instruments.
Metrics collected on NE STEM 4U
Summary of undergraduate mentor and K-8 student participant assessments. Included in the table is the list of the assessment itself by participant level, as well as the intended goal and how the assessment was administered (therefore informing data collection)
Undergraduate mentor assessments
K-8 student assessments
For mentors to reflect upon the session to process gains and areas for improvement
After every mentoring session
Pre/post content-based assessment (questions are written using Bloom’s taxonomy)
Determine gains in content knowledge after activity
Before and after every lesson (i.e., daily)
Evaluate critical thinking, content knowledge, and personal gains
12 and 24 weeks after mentoring has begun of each year
Long-term post-test assessments
Evaluate long-term content knowledge gains
After a minimum of 30 days after content delivery
Gauge retention in STEM fields and career preparedness
Upon graduation or exiting the program
Assess student interest in STEM fields
At beginning and end of an 8-week session
Dimensions of Success (DoS)
Assess program in 12 dimensions, including organization, materials, space utilization, participation, purposeful activities, engagement with STEM, inquiry, reflection, relationships, and youth voice
By internal and external DoS-certified evaluators a minimum of 8 times per school per academic year
Dimensions of Success (DoS)
Assess program in 12 dimensions, including organization, materials, space utilization, participation, purposeful activities, engagement in STEM, inquiry, reflection, relationships, and youth voice
By internal and external DoS-certified evaluators a minimum of 8 times per school per academic year
Preliminary results of pilot program
While this report serves as a programmatic introduction, our subsequent reports will describe the results representing specific gains by both K-8 student participants and undergraduate students in the NE STEM 4U program. Importantly, this is a short report summary of a program and, therefore, not a comprehensive report of all data collected per the metrics described in Table 1.
To address our first research question of the impact of this intervention on content knowledge by K-8 participants, our preliminary results (2013–2014; n = 297 K-8) indicated an average of twofold gains (range of 1.7–2.5) in STEM content knowledge for youth that participate at least weekly in the after-school program, NE STEM 4U. Moreover, K-8 student participants in NE STEM 4U perform statistically significantly better on long-term retention assessments of the content (control group earned an average of 30 % on the long-term retention test, NE STEM 4U participants earned an average of 45 %; n = 727; p = 0.0026). While content knowledge in STEM areas is measured through pre- and post-tests in this program, we are also interested in understanding how youth approach STEM inquiry and analysis and these data are obtained using the Dimension of Success tool.
Similar to Knopf et al. (2015), we found that a focused program (i.e., on problem solving and critical thinking in STEM) is effective at improving student gains in STEM. We also, unexpectedly, saw a significant increase in the mentor-mentee relationship with the undergraduates serving as role models to the youth and answering questions about what college is like, how to pick a career, and how to pay for college. Moreover, the youth gain confidence in their abilities to share information in STEM areas, teach one another, and lead experiments.
Importantly, the daily exercises are all hands-on, minds-on experiments encouraging inquiry-based learning where there likely is not a “right” answer and students must discuss and justify their observations. For example, mentor—“Do plants and animals share the same DNA?” Student—“well, they both have DNA, but probably different.” Mentor—“What about a mother and child?” Student—“I would think different, I am a boy and she’s a girl.” Mentor—“What about a skin cell and heart cell?” Student—“they do different things, but they are all my DNA?” (said with puzzled look). All great teaching/learning opportunities are found with these interactions between the mentors and students. In this way, K-8 students are behaving like scientists, practicing investigative iteration, and reflecting upon their interpretations. As a result, these students are well on their way to the skill set that will make them effective in their careers.
The kids really loved the CSI theme and I think [Student mentor] and I were able to use that theme to keep the kids interested and engaged the whole period. They really seemed to be having a good time and most of them did really well on the post-test even though the questions were tough!
I like hearing from the after-school administrators and aides that the students are learning a lot and are having a great time. The fact that the students are talking about our experiments after they leave, and are excited about them, makes me feel like I am doing what I am supposed to be.
I liked that I was able to help students one-on-one how to do Punnett Squares. They understood how to do them better with individual attention.
Furthermore, using the 12- and 24-week interviews, which assessed self-perceived organization skills, content knowledge, preparedness, dependability, and engagement using a ranking from 1 to 10 (10 being the best), we uncovered some notable self-reported growth among the mentors. Specifically, after 12 weeks of mentoring, the average self-rank in all of these categories was a 6.5 or greater, while after 24 weeks of mentoring, the average self-rank was an 8 or greater. As reported in detail in Nelson et al. (under review), the cumulative average percent change from student self-reporting after participating in the NE STEM 4U program indicated the greatest gains in organization, preparedness, and engagement skills. However, students reported gains in all categories over the 24-week time period, with significant gains (p ≤ 0.05 for each category) reported for organization, STEM content knowledge, preparedness, and engagement.
Lastly, after completing the program, 55.5 % of UNO student mentors indicate that they will include mentoring in a future career and 18.5 % reported that this experience caused them to change their career trajectory to teaching science. NE STEM 4U aims to build a pipeline of students able to think critically and address questions in STEM fields.
The NE STEM 4U program deploys insight from the literature on how to best provide learning opportunities in STEM disciplines for youth during the out-of-school time. Specifically, the program creates an educational pipeline for students through hands-on experiences in problem solving and critical thinking through daily lessons in STEM areas. The most critical part of this program is the strong tie between the university and community partners. Through these relationships and ongoing discussions, we are able to determine what educational interventions are necessary, which are working, and how we can improve the program in real time.
In this programmatic overview, we describe the structure, objectives, and assessments used in the NE STEM 4U program. We hope that this information will be of benefit to those in other metropolitan cities to replicate such a model. The NE STEM 4U program is meant to provide pre-professional training to college students and to engage K-8 youth in genuine STEM experiences through hands-on, active learning in the after-school setting.
Herein, we described a novel out-of-school time program implemented in Nebraska called NE STEM 4U. As a result of a pilot of this program, we observed gains in content knowledge by youth participants. Additionally, as the programmatic model deploys undergraduates as mentors/teachers of the program, we also observed gains by the undergraduate mentors through self-reported gains in content knowledge, preparedness, organizational skills, and engagement. Importantly, this program has a profound impact on the mentors with more than half reporting that they intend to serve as a mentor in some capacity again in the future. Therefore, after analyzing data from the first year, there is ample evidence to support the hypothesis that we are achieving our core objectives for both K-8 student participants and undergraduate student mentors.
discipline-based education research
dimensions of success
kindergarten through undergraduate degree completion
kindergarten through grade 8
- NE STEM 4U:
Nebraska Science, Technology, Engineering, and Mathematics 4U
science, technology, engineering, and mathematics
University of Nebraska at Omaha
The authors acknowledge the support from the Sherwood Foundation, the Nebraska University Foundation, and the UNO Office of Research and Creative Activity for the funding of this project. The funders had no role in the study design, the data collection and analysis, the decision to publish, or the preparation of the manuscript. Additionally, thanks to the UNO’s Colleges of Arts and Sciences and Education for supporting such collaborations. Thanks to Collective for Youth, including Gwyn Williams and Megan Addison, Dr. George Haddix, and the Sophie and Feodora Kahn family. Thanks to Lauren Dahlquist and Tyler Herek for their continued support of the program. Thanks to all undergraduate mentors that have devoted their time to teaching youth about STEM topics in the after-school setting.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Carnevale, A. P., Smith, N., & Stoll, J. (2010). Help wanted: projections of jobs and education requirements through 2018. Georgetown University Center on Education and the Workforce. Available: https://cew.georgetown.edu/wp-content/uploads/2014/12/fullreport.pdf. Accessed July, 2015.
- DuBois, D. L., Hollaway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: a meta-analytical review. American Journal of Community Psychology, 30, 157–197.View ArticleGoogle Scholar
- Gregg-Jolly, L. A., Kington, R., Lopatto, D., & Swartz, J. E. (2011). Benefits of intertwining teaching and research. Science, 331, 532.View ArticleGoogle Scholar
- Knopf, J.A., Hahn, R.A., Proia, .K.K., Truman, B.I., Johnson, R.L., Muntaner, C. et al (2015). Out-of-school-time academic programs to improve school achievement: a community guide health equity systematic review. J Public Health Manag Pract, Epub ahead of print
- Linstone, H.A., Turoff, M. (1975). Delphi method: techniques and applications. Addison-Wesley Publishing
- Lopatto, D. E. (2005). Undergraduate research experiences and the epigenesis of a science career. Developmental Biology, 283, 586–587.Google Scholar
- Lopatto, D. (2007). Undergraduate research experiences support science career decisions and active learning. CBE Life Sciences Education, 6, 297–306.View ArticleGoogle Scholar
- [NACE] National Association of Colleges and Employers. (2014). Job outlook 2015. Available: https://www.umuc.edu/upload/NACE-Job-Outlook-2015.pdf. Accessed 12 Mar 2016.
- [NDE] Nebraska Department of Education. (2013). 2012–2013 state of the schools report. Available: http://reportcard.education.ne.gov. Accessed 12 Mar 2016.
- Reform for the Future, 2012. Available: (https://www.whitehouse.gov/issues/education/reform). Accessed 12 Mar 2016.
- Sacco, K., Falk, J. H., & Bell, J. (2015). Informal science education: lifelong, life-wide, life-deep. PLoS Biology, 12(11), e1001986.View ArticleGoogle Scholar
- U.S. Department of Commerce, Economics, and Statistics Administration. (2011). U.S. innovations report, Department of Commerce, United States of America. STEM: good jobs now and for the future. Available: http://www.esa.doc.gov/sites/default/files/stemfinalyjuly14_1.pdf. Accessed 12 Mar 2016.