EXECUTIVE SUMMARY OF EVALUATION FINDINGS

Transcription

1 B R A I N P R O F E S S I O N A L D E V E L O P M E N T EXECUTIVE SUMMARY OF EVALUATION FINDINGS R E P O R T B Y : J E A N I N E A N C E L E T N O V E M B E R 1 5, Jeanine E. Ancelet - Audience Focus Kings Retreat Ct Davidsonville, MD Tel

2 P r o j e c t D e s c r i p t i o n Since July 2014, The Franklin Institute ( TFI) has offered professional learning experiences for teachers, principals, superintendents, and out- of- school time educators that focus on topics related to the brain and learning. TFI has partnered with several school districts and educational organizations in the tri- state area ( PA, NJ, & DE) to reach a broad range of educators and administrators to communicate that understanding the brain is critical in order to effectively educate children across ages and settings. In 2016 and 2017, Audience Focus ( AF) conducted a summative evaluation to measure the impact of the program on participants. The summative evaluation investigated 3 key questions: 1) How are classroom teachers and school/ district administrators impacted by their participation in Brain PD?, 2) What do classroom teachers and school/ district administrators perceive are the impacts of Brain PD for educators on student learning?, and 3) What components of the Brain PD model do target audiences perceive contribute most to the program s impacts? G o a l s a n d O b j e c t i v e s O f B r a i n P D The primary outcomes articulated by TFI for program participants are to: 1) increase understanding about how the brain works related to learning and memory, 2 ) increase understanding of teaching strategies supported by brain research, 3) change beliefs about how students learn to be more aligned with brain research, 4) change teaching practice in the classroom to embody strategies that are supported by brain research, 5) change administrative decisions in their schools/ districts, and 6) increase interest in using learning strategies supported by brain research in their professional work. 0 1

3 E v a l u a t i o n P l a n I N A U D I E N C E F O C U S C O N D U C T E D T H R E E S E P A R A T E, B U T R E L A T E D, I N V E S T I G A T I O N S T O M E A S U R E T H E I M P A C T O F B R A I N P D O N S C H O O L A D M I N I S T R A T O R S & T E A C H E R S. The first evaluation was a quasiexperimental, retrospective pre/post study that measured program impacts on teachers and administrators who participated in Brain PD before May of The second study was launched the following year using a quasiexperimental pre/post design to measure program impacts on teachers who participated in the program during the program year. Finally, a third, qualitative study was conducted as a way to document teachers' and school/district administrators' perceptions of the program's impacts in greater depth. A TOTAL OF 403 TEACHERS AND ADMINISTRATORS PARTICIPATED IN THE EVALUATION STUDY: 253 TEACHERS AND ADMINISTRATORS PARTICIPATED IN THE RETROSPECTIVE PRE/POST STUDY; 121 PARTICIPATED IN THE PRE/POST STUDY; AND, 29 PARTICIPATED IN THE QUALITATIVE STUDY. 0 2

4 Impacts on Teachers FINDINGS SHOW THAT BRAIN PD HAS SIGNIFICANT, POSITIVE IMPACT ON TEACHERS AND ADMINISTRATORS KNOWLEDGE, AWARENESS, PRACTICES, BELIEFS, AND INTERESTS RELATED TO THE BRAIN AND LEARNING In both the retrospective pre/post and the pre/post studies, teachers rated a series of scaled items designed to measure program impacts. In each case, teachers rated their: 1) understanding of how the brain learns, 2) interest in learning more about how the brain learns, 3) awareness and use teaching strategies supported by brain research and 4) beliefs about how students learn to be more aligned with how the brain learns significantly higher after participating in Brain PD than before. (See Table 1 below) Through qualitative lines of inquiry, participants spoke in depth about the many ways participating in TFI s Brain PD positively impacted them, reinforcing findings from the quantitative studies. Results from the qualitative study indicate participants believe their participation in Brain PD increases their awareness and understanding of how the brain works related to learning, challenges and expands their beliefs about how children learn, and changes the way they approach learning in their classrooms. Table 1: Teachers' pre post ratings of program impacts. Interest and understanding used a 7 point scale, where 1=low end and 7=high end. Awareness of teaching strategies, use of teaching strategies, and beliefs about how students learn based on a 5-point scale, where 1=low end and 5=high end. Data is from pre / post study. *Individual items for scales available upon request. 03

5 I m p a c t s o n A d m i n i s t r a t o r s In the retrospective pre/post study, administrators also rated a series of items designed to measure program impacts using a 5 -point scale, with 1=strongly disagree and 5=strongly agree. For each item, administrators rated their: 1) understanding of how the brain learns, 2) interest in learning more about how the brain learns, 3) awareness and use teaching strategies supported by neuroscience research and 4) beliefs about how students learn to be more aligned with brain science significantly higher after participating in Brain PD than before. Administrators also rated scale items designed to measure their use of learning strategies supported by brain research to inform key school/decision making significantly higher after the program than before. (See Table 2) Table 2: Administrators' ratings of ways Brain PD influenced their administrative decision making (data is from retrospective pre/post study) 0 4

6 I m p a c t s o n T e a c h e r s & A d m i n i s t r a t o r s Increased Awareness & Understanding Teachers and school/district administrators clearly indicate that Brain PD increases or reinforces their understanding of how the brain works related to learning, as well as their understanding of teaching strategies aligned with how the brain learns, like providing active and sensory learning experiences, activating prior knowledge and interests, and demonstrating growth mindset. School/district administrators, in particular, say having an increased understanding of the how the brain works and sound evidence to back brain-based strategies and learning theories allows them to communicate more clearly with classroom educators and to push harder for the adoption of strategies supported by neuroscience research in the classroom. " I T H I N K T H I S I S G R O U N D B R E A K I N G. I T S U N I Q U E T O T H E C L A S S R O O M A N D I A M S E E I N G T R E M E N D O U S R E S U L T S I N T H E C L A S S R O O M A S A R E S U L T O F T H I S. T H E K I D S A R E P E R F O R M I N G A T L E V E L S T H A T I H A V E N E V E R S E E N B E F O R E. " Changes in Attitudes & Beliefs B E F O R E I W A S R E A L L Y C O N C E R N E D W I T H T H E L E S S O N P L A N, V E R Y C O N C E R N E D W I T H M A K I N G S U R E I M E T D I S T R I C T G O A L S. N O W A F T E R T H E B R A I N P D I T H I N K H O W C A N I M A K E T H I S C O N T E N T F I T F O R M Y S T U D E N T S? I A M P U T T I N G M Y S T U D E N T S F I R S T M O R E T H A N I H A V E E V E R D O N E B E F O R E, B A S E D O N T H I S R E S E A R C H. Teachers and administrators eloquently describe ways that participation in Brain PD leads to greater open-mindedness and empathy towards students and their learning needs. Participants describe beliefs that are more clearly aligned with growth mindset, especially the notion that all people learn differently, that intelligence can grow and change over time, and that young people should be treated according to their brain development. Some say participating in Brain PD changes how they prioritize students needs over externally mandated goals and testing. strategies. Changes in Professional Practice Teachers say they are implementing teaching and learning strategies that are aligned with how the brain learns into their classrooms. Strategies that educators say they are incorporating with more frequency include: activating students prior knowledge, curiosity, and interests; providing more opportunities for students to engage in active and sensory learning; giving students brain breaks; emphasizing growth mindset by teaching students how their brains work and referencing growth mindset in language; and, providing more opportunities for students to go deeper into one topic and/or stimulate higher-order thinking skills. Most administrators say they now offer additional PD and foster ongoing conversations related to the brain and learning with their staff. Some say they now incorporate evidence of brain research teaching strategies into classroom evaluation tools, have modified the curriculum, and/or have new expectations for assessment 0 3 and pacing. B E F O R E I W A S M O R E T E A C H E R - D I R E C T E D, T E A C H E R F O C U S E D, A N D N O W I A M M O R E S T U D E N T - F O C U S E D. I H A V E M Y S T U D E N T S B E C O M I N G M O R E O F A N E X P L O R E R A N D T R Y I N G T O C R E A T E A N D C O N S T R U C T T H E I R L E A R N I N G, A N D B E T H E G U I D E O N T H E S I D E I N S T E A D O F T H E P E R S O N U P F R O N T T E A C H I N G E V E R Y T H I N G. 0 5

7 I m p a c t s o n S c h o o l C u l t u r e Through the pre/ post study, teachers reported positive significant changes in the degree to which teachers, administrators, and students talk about research on the brain and growth mindset in their schools. (See Table 3) In addition, many teachers and school/district administrators who participated in the qualitative study describe the ways participating in Brain PD influences their beliefs about student learning and the decisions being made at a district-wide level. Administrators describe increased communication and dialogue about how the brain works and growth mindset among faculty at all levels, as well as more systemic changes to lesson-planning and curriculum development that will now incorporate more brain-based strategies to improve student learning. Table 3: Teachers' ratings of ways Brain PD influenced school culture using a 5 point scale where 1=low end and 5=high end. Data come from pre/post study; however, retrospective pre/post scale was used for this question as many of the statements were not asked on the baseline. Statements were revised to be more in line with TFI s indicators for this impact following the results from the qualitative study 0 6

8 I m p a c t s o n S t u d e n t L e a r n i n g F I N D I N G S R E L A T E D T O T H E I M P A C T O F B R A I N P D O N S T U D E N T L E A R N I N G A R E L E S S C O N S I S T E N T B E T W E E N T H E T H R E E S T U D I E S. While teachers and administrators who participated in the retrospective pre/post and true pre/post studies say they believe students benefit indirectly as teachers improve their teaching practice and/or become more empathetic towards students, most struggle to describe direct, concrete benefits for students. Many administrators, in particular, say they are unsure how students benefit and that only time will tell. Classroom educators who participated in the qualitative study, on the other hand, strongly believe their participation in Brain PD impacts students and were able to articulate those benefits in the following ways: Increased Interest & Engagement Teachers say students are demonstrating more interest and engagement in learning since they have incorporated teaching strategies supported by neuroscience in their classroom. Educators say when they tap into students prior experiences and interests, provide more active and sensory learning experiences, and differentiate learning based on individual student needs, students noticeably become more engaged in their learning. " I F S T U D E N T S A R E I N T R I G U E D A N D T H E B R A I N I S C U R I O U S I T I S G O I N G T O W A N T T O L E A R N, I T I S G O I N G T O W A N T M O R E I N F O R M A T I O N. I A M S E E I N G G R E A T R E S U L T S. " " I T S P R E T T Y I M P R E S S I V E T H A T A F T E R 2 Y E A R S O F B E I N G I N V O L V E D W I T H T H I S Y O U C O U L D W A L K I N T O M Y R O O M A N D A N Y O N E O F M Y K I D S F R O M T H E L O W E S T T O H I G H E S T C O U L D T E L L Y O U W H A T G R O W T H M I N D S E T I S. W E M A K E I T J U S T A S I M P O R T A N T A S R E A D I N G, M A T H, W R I T I N G A N D T H E Y A R E O N L Y 6 " Increased Confidence & Perseverance Increased Awareness & Understanding Teachers and a few school/district administrators report increased confidence in students to persevere in the face of learning challenges as a result of the growth mindset approach adopted in classrooms following the Brain PD. Educators say they have heard students using more positive language when talking about their own or their peers learning abilities and have also seen students improve on assessment scores, which they attribute to the introduction of growth mindset in their classrooms. Teachers say that many students are more aware of and interested in the ways their brains work and what growth mindset is as a result of Brain PD. Educators are more likely to share information about the brain with students after participating in the program, which translates into greater awareness and understanding of, and interest in the brain among students. " I H A V E H A D, B E C A U S E O F P A R T I C I P A T I O N I N T H E B R I A N P D, 3 S T U D E N T S W H O H A V E N E V E R R E C E I V E D S T R A I G H T A S I N T H E I R E N T I R E E L E M E N T A R Y C A R E E R, N E V E R, T H E Y G O T S T R A I G H T A S T H I S M A R K I N G P E R I O D. A L L 3 S I G H T E D I T I S B E C A U S E O F H O W M Y B R A I N W O R K S. I F O U N D O U T H O W M Y B R A I N W O R K S A N D I A M U S I N G I T I N C L A S S E S A N D I A M D O I N G V E R Y W E L L. 0 7

9 S t r e n g t h s o f P r o g r a m m o d e l T H R O U G H T H E Q U A L I T A T I V E S T U D Y, C L A S S R O O M E D U C A T O R S A N D S C H O O L / D I S T R I C T A D M I N I S T R A T O R S I D E N T I F I E D S I X C O R E C O M P O N E N T S T H A T T H E Y F E E L C O N T R I B U T E M O S T T O T H E B R A I N P D P R O G R A M S I M P A C T S. Component C O M P O N E N T S T A R T D A T E E N D D A T E Research-based Immediately Applicable Teachers and administrators deeply value hearing from experts and learning new information about the brain that is grounded in research. Participants also value having previously held beliefs questioned and common misconceptions dispelled. Teachers and administrators agree that learning how to translate brain research into practical classroom strategies and activities to improve student learning is one of the most valuable aspects of the Brain PD model. Both audiences praise the Brain PD for providing concrete and immediately applicable ways for brain theory to be translated into action. Museum-based Learning Environment Teachers and administrators strongly value the museum-based learning context. Participants say being at the museum is a welcome change of pace and scenery and helps stimulate interest and curiosity, as well as increase concentration and focus. Participants say that learning about the brain first and exploring the Your Brain exhibit second makes the learning more relevant and applicable. 0 8

10 S t r e n g t h s o f P r o g r a m m o d e l Component Learnercentered Teachers and school/district administrators clearly value the learnercentered facilitation approach used by TFI educators during the Brain PD. Participants appreciate having opportunities to participate in hands-on activities, explore the Your Brain exhibit, and take on the role of a learner. Collaborative Participants say they appreciate having opportunities to learn with school and district colleagues. Benefits of the collaborative model include having opportunities to share ideas and implementation strategies, as well as leaving the program with a shared language and vision. Individualized Experience C O M P O N E N T S T A R T D A T E Teachers and administrators say they appreciate E Nthe D Dway A T ETFI creates learning experiences that are tailored to the individual interests and needs of each partner. In the future, classroom educators and school/administrators suggest TFI provide follow-up workshops and deeper dives for Brain PD participants. In addition to offering subsequent PD sessions, participants also say providing regular updates on brain research and offering periodic check-ins with PD alumni through on-site visits, , or other online communication would be welcomed additions to the program. Several educators and administrators say the program should be offered to educators and administrators in their schools or districts who have not yet participated. Some classroom educators and school/district administrators would like to see TFI incorporate even more opportunities to learn specific, brain-based teaching strategies that can be implemented in the classroom. Some suggest having customizable experiences based on the needs of the school or district. Finally, a couple school/district administrators suggest offering more opportunities for experienced teachers to take leadership roles in future Brain PD sessions

11 T h a n k y o u T o o u r P a r t n e r s The Franklin Institute is deeply grateful for the participation and collaboration of teachers and administrators who have helped to shape this initiative since 2014, and contributed to the evaluation study. Thank you to our incredible partners from Centennial School District, Unionville-Chadds Ford School District, Bensalem Township School District, Ridley School District, and Mount Laurel School District for their ongoing commitment to using research on the brain to improve education. A special thanks to Dr. David Baugh for co-designing our first district-wide partnership committed to understanding the brain and learning for K-12 educators and administrators. 1 0