• 
• 
• 

VideoReView Project Overview 2017-2018

Empowering Teachers Through VideoReView is a NSF funded research and development project (#1415898) to enhance elementary science teachers’ attention to students’ science thinking. The VideoReView program involves teams of teachers in school-based, video-supported professional learning, and a technology to capture and study classroom videos.

In the VideoReView program, teachers participate in a four-part sequence: PLAN, ENACT, STUDY, and MEET. First, teachers plan science discussions to lead with their students. Then, they enact the discussions in their classrooms and use a classroom-friendly technology—a camera and software tool—to capture the discussions. Next, teachers use the software tool to study and analyze videos of their science discussions. Based on the pilot implementation of the program in 2016-2017, a new element of coaching was introduced during the 2017-2018 implementation to facilitate teachers’ video study. Teachers meet individually with a program coach to study their videos, and they use a set of written guidelines to scrutinize students’ thinking and to prepare video cases for deeper reflection. Finally, they meet with colleagues in teacher-led Video Clubs to discuss the video cases, and to plan next steps in instruction.

VideoReView Research Overview

The VideoReView research investigates how school-based, video-supported professional learning helps teachers to notice students’ science thinking. The research is based on a theory of change in which teachers videotape science discussions, and then study the video on their own (with a program coach) and with colleagues, with the explicit intent of understanding their students’ ideas and reasoning.

This video reflection is hypothesized to increase teachers’ reflection on students’ learning, and teachers are likely to develop new ways of attending, interpreting and deciding how to respond to students’ thinking. Additionally, increased teacher reflection is hypothesized to improve classroom interactions. Changes in teachers’ facilitation and students’ participation in science discussions are expected. Improved teacher reflection and classroom interactions, ultimately, are hypothesized to increase students’ scientific conceptual understandings.

The diagram below depicts this theory of change, showing relationships between the VideoReView program, the intermediate outcomes of improved teacher reflection and classroom interactions, and the final outcome of students’ scientific conceptual understandings. The research draws on prior work in using video to support teachers’ noticing of students’ thinking (e.g., Sherin, Jacobs, & Philipp, 2011; van Es & Sherin, 2010). This work shows that classroom video is a powerful means to help teachers attend to and make sense of their students’ ideas, and to respond in ways that can foster deeper student thinking.

The VideoReView research investigates changes in four areas of teacher and student learning: (i) teachers’ noticing of students’ thinking while studying classroom videos; (ii) teachers’ facilitation of classroom science discussions; (iii) students’ participation in science discussions; and, (iv) students’ scientific conceptual understanding.

Specifically, the following questions are at the heart of this research:

• Does participation in the VideoReView program improve teachers’ attention to and interpretation of students’ thinking, and pedagogical decisions based on students’ thinking?
• Does participation in the VideoReView program improve teachers’ facilitation of discussions to promote students’ learning?
• Does students’ participation in science discussions improve as teachers participate in the VideoReView program?
• Does students’ conceptual understanding improve as teachers participate in the VideoReView program?

To answer these questions, the VideoReView team partnered with two cohorts of teachers: the first cohort participated in a research study during 2016-2017, and the second cohort participated in a study during 2017-2018.

The data collected to answer these questions, and the procedures used in collecting and analyzing them during the 2017-2018 study are elaborated in the Methods section. The procedures were refined based on the pilot implementation of the program in 2016-2017. Specifically, in the second cohort, teachers videotaped and studied a practice science discussion as a pre-study to provide a true baseline measure of teachers’ skills in noticing and responding to students’ thinking, and of students’ participation in science discussions.

The research findings from the 2017-2018 study are presented in the Impact section to show how the VideoReView professional learning program helps promote changes in teachers’ noticing of students’ science thinking during reflection on video, changes in classroom discussions, and changes in students’ conceptual understanding.

References

Sherin, M. G., Jacobs, V. R., & Philipp, R. A. (Eds.). (2011). Mathematics teacher
     noticing: Seeing through teachers’ eyes. Routledge.

Van Es, E. A., & Sherin, M. G. (2010). The influence of video clubs on teachers’
     thinking and practice. Journal of Mathematics Teacher Education, 13, 155-176.

Methods

Participants in the second cohort – 2017-2018

14 elementary grade science teachers across four schools formed the second cohort participating in the VideoReView professional learning program in 2017-2018. The schools were a mix of urban, suburban and rural schools in Massachusetts and Vermont. The teachers taught upper elementary grades (grades 3-5). Of the 14 teachers, 11 teachers were participating in the program for the first time, and three teachers were returning to the program a second time. The VideoReView program was implemented separately with five teacher teams over approximately 10 weeks. Each team comprised grade-level teachers who taught the same science curriculum and partnered with a program coach. The teachers led Video Club meetings with their respective team members.

Procedure for Data Collection and Data Analyses

Video Study with Program Coach (Think Alouds): The teachers videotaped a “practice” (pre) science discussion before the start of the program, and at least four whole class discussions during the program that they enacted as part of their school’s regular science curriculum. The teachers met with a program coach to study videos of those discussions and used a set of written guidelines to conduct the video studies. Additionally, the teachers revisited the pre-discussion video at the end of the program to take stock of their learning. Of these study meetings, the researchers focused on the pre-video study and the fourth (final) video study that was conducted during the program. During these study meetings, for research purposes, the teachers were asked to ‘think aloud’, i.e., to comment on what they noticed as they reviewed their classroom videos. The study meetings were audiotaped and transcribed. The researchers then analyzed the transcripts to identify what teachers noticed in the videos, and how they made sense of their observations. The coding scheme used to analyze the Think Aloud transcripts was similar to that used in analyzing transcripts of teachers’ Video Club meetings as described below.

Video Club Meetings: Teachers participated in four Video Club meetings; in each meeting, two teachers shared and discussed video cases, and planned next steps in their teaching. These meetings were audiotaped and transcribed, and the transcripts were analyzed by the researchers to identify changes in what teachers noticed in the video cases, and how teachers made sense of their observations. The researchers focused on the first and last Video Club presentation for each teacher. As mentioned before, the coding scheme used to analyze the Video Club transcripts was similar to that used in analyzing transcripts of teachers’ Think Aloud meetings.

Video of Science Discussions: The classroom discussions recorded by the teachers served as another source of research data. The teachers videotaped a minimum of four discussions (as stated earlier) – a pre-discussion prior to the program and at least three discussions during the program. The VideoReView researchers analyzed transcripts of these recordings to identify changes in two aspects of classroom instruction: (i) changes in teachers’ facilitation of science discussions; and, (ii) changes in students’ participation during discussions. Specifically, the researchers examined how teachers respond to and take up students’ thinking, and how students engage with scientific concepts and practices during discussions. The first two discussions taped during the program were considered the “early” discussions and the last two discussions (or the last one for three teachers) were considered the “late” discussions.

Exit Interview: The teachers also met with their program coach for an exit interview at the end of the VideoReView professional learning program. The purpose of the interviews was to learn about teachers’ experiences during the program, and any changes that teachers perceived in their practice and in their students’ science learning as a result of the program. The interviews were audiotaped and transcribed. The transcripts of these interviews were analyzed to understand teachers’ self-reported shifts in planning and enacting discussions; in attending to and facilitating students’ science thinking; in reflecting on their own teaching; and shifts in their students’ science learning. The returning teachers were also asked to compare their experiences over the two years of their participation in the program.

Student Assessment: Finally, a total of 21 students across two Grade 4 classrooms took pre-and-post assessments before and after the science unit that was implemented during the VideoReViewprogram. The science unit was part of the Inquiry Project curriculum, an elementary science curriculum covering foundational concepts related to matter. The pre - and post- assessments focused on core matter concepts from the curriculum. The pre-and post-assessments were identical in form, each with the same 7 assessment tasks measuring a range of concepts from the 4th grade Inquiry Project curriculum. Each task addressed one core matter concept, one measurement skill, and required students to provide an argument based on evidence. Responses on the assessments were scored to determine changes in students’ conceptual understanding during the VideoReView program. (See attached assessments.) Student scores were computed by summing their scores on all items. The maximum possible score was 35. Sub-scores were also calculated for three subtests: Matter (9 of 35 score points), Measurement (12 of 35 score points), and Argumentation (14 of 35 score points).

Impact

This report summarizes key findings from research with the second cohort of teachers participating in the Empowering Teachers Through VideoReView project during 2017-2018. This project is a four-year research and development initiative funded by the National Science Foundation (#1415898) to develop teachers’ capacities in noticing and responding to their students’ thinking in science. The VideoReView program involves school-based, video-supported professional learning, and an easy-to-use technology for teachers to capture and study their own classroom discussion videos.

In this program, teachers participate in a four-part professional learning sequence – PLAN, ENACT, STUDY, and MEET (PESM). First, teachers use a discussion planning tool to PLAN science discussions in the classrooms. The tool asks them to identify learning goals and discussion questions, and to anticipate students’ ideas and their possible teaching moves in response. Then, teachers ENACT the discussions in their classrooms, and use technology—a camera and software—to record the discussions. Next, teachers use the software to STUDY videos of their science discussions. Teachers meet with a program coach to study the videos. The teacher and coach follow a set of written guidelines to study specific events in the video. In so doing, teachers attend to and analyze students’ ideas and reasoning related to the learning goals. Teachers also use the software to prepare video cases to discuss with colleagues. Finally, they MEET with school-based colleagues in primarily teacher-led, school-based “Video Clubs” to discuss the cases, and to consider implications for their teaching.



The purpose of the VideoReView research was to study how school-based, video-supported professional learning helped teachers to notice and respond to students’ thinking in science. As teachers participated in the professional development program, the research examined changes in four areas of teachers’ and students’ learning: (i) teachers’ noticing of students’ thinking as they study classroom videos; (ii) teachers’ facilitation of classroom science discussions; (iii) students’ participation in science discussions; and (iv) students’ conceptual understanding in science Specifically, the following questions guided the research:

• Does participation in the VideoReView program improve teachers’ attention to and interpretation of students’ thinking, and pedagogical decisions based on students’ thinking?
• Does participation in the VideoReView program improve teachers’ facilitation of discussions to promote students’ learning?
• Does students’ participation in science discussions improve as teachers participate in the VideoReView program?
• Does students’ conceptual understanding in science improve as teachers participate in the VideoReView program?

14 elementary grade science teachers across four schools formed the second cohort participating in the VideoReView professional learning program in 2017-2018. The schools were a mix of urban, surburban and rural settings across Massachusetts & Vermont. The teachers taught upper elementary grades (grades 3-5). All except three teachers were new to the program. The VideoReViewprogram was implemented separately in each school site over approximately 10 weeks. There was a total of five teacher teams; each team had two-three teachers who studied their own videos with a program coach and led Video Club meetings with their team members in their respective schools. Data for research purposes were gathered from the teachers as they participated in the VideoReView program. Multiple sources of data were used to address the research questions: (i) transcripts of audio recordings of ‘Think Aloud’ Video Study meetings in which teachers met one-on-one with a program coach, and commented on what they noticed while reviewing their own videos; (ii) transcripts of audio recordings of video club meetings in which teachers discussed video cases with their colleagues; (iii) transcripts of video recordings of classroom science discussions enacted by the teachers; (iv) transcripts of audio recordings of exit interviews with each teacher; and (iv) students’ responses on written pre-post science assessments


This section summarizes data analyses and presents key findings from the research. The transcripts of two Video Study meetings - pre-video study and fourth (final) video study - from each of 13 teachers were analyzed to identify what teachers noticed in their own videos, and how they made sense of their observations. The transcripts were analyzed using a formal coding scheme developed deductively from literature on teacher noticing, and inductively from data.

Overall, comparing the pre-video study to the fourth (final) study during the program, the teachers showed sophisticated capacities in noticing their classroom interactions. They began with a high focus on attending to students’ science thinking, and this focus increased in the fourth study. Additionally, teachers made more frequent attempts to attend to specific ideas of students in the fourth video study, and to interpret students’ thinking. Finally, they shifted away from commenting on matters of pedagogy, making fewer comments of this kind in the final study. These shifts were in accordance with the theory of change underlying the professional learning program (see Overview section for Theory of Change).

Each of the 14 teachers presented a video case at least twice during the VideoReView program. The transcripts of the first and last video case presentation for each teacher were analyzed to study changes in what teachers noticed in the presented video cases, and how teachers made sense of their observations. The formal coding scheme used to analyze the Video Club transcripts was similar to the one used to analyze transcripts of teachers’ Think Aloud meetings.

The data analysis revealed mixed findings, some more desirable than others. Across the five school teams, teachers showed a high focus on discussing students’ science thinking, and this focus also increased in the last video case presentation. Similarly, while discussing pedagogy, teachers’ attempts to connect science pedagogical matters to students’ thinking also increased over time and their comments about generic pedagogical matters decreased. This finding indicates that when teachers discussed pedagogy, they were more likely to incorporate students’ thinking in their reflections during the last Video Club presentation compared to the first. Furthermore, teachers showed a high level of attention to individual ideas of students, and this level of specificity in teachers’ attention to students’ thinking increased in the final presentation. These shifts were in accordance with the underlying theory of change.

On the other hand, although teachers interpreted students’ thinking frequently during both the first and last video case presentations, their interpretive stance became less frequent over time. This finding was opposite to what was hypothesized in the theory of change. Teachers’ interpretive stance increased during video study with a program coach but decreased during the Video Club meetings. Two possible reasons may account for this unexpected finding. First, informal observations of the data suggest that the presenting teacher spent time explaining the context of the video to her colleagues during the Video Club meetings, which was part of the protocol for the Video Clubs. But it was not necessary to do so with a coach during the Video Study meetings. Second, there was a fair amount of interpretation of students’ thinking during Video Study meetings with a coach, shaped by the written guidelines provided for the video study. These interpretive attempts helped teachers ponder questions about pedagogical implications to discuss with colleagues during Video Club meetings. Therefore, teachers may have devoted less time to interpreting students’ thinking with colleagues and instead utilized the Video Club environment more to explore pedagogical matters. Transcripts of at least four classroom discussions – a pre-discussion videotaped prior to the program and at least three discussions taped during the program - for each of 13 teachers were analyzed to study changes in teachers’ responsiveness to students’ thinking as they facilitated science discussions. The transcripts were analyzed based on a formal coding scheme developed deductively from literature. The transcripts of the first two discussions during the program were considered the “early” discussions, and transcripts of the last two discussions (or the last one for three teachers) were considered the “late” discussions. For each teacher, the pre-discussion transcript was compared to the early and late discussion transcripts, and the early and late transcripts were compared to one another. These comparisons were made to better understand when changes in teachers’ facilitation occurred during the VideoReView program.

The data analysis indicated that teachers showed a high level of responsiveness towards their students’ thinking, engaging in uptake of students’ ideas, probing those ideas, or asking students to respond to their peers’ ideas. Their responsive teaching moves increased from pre-discussion to early discussions. This shift was consistent with the underlying theory of change. Further, between the pre and the late discussions teachers probed students’ thinking more frequently (e.g., asking clarifying questions) and asked students to comment on their peers’ ideas, thus essentially staying with the students’ ideas. On the other hand, teachers less frequently took up student ideas with moves like re-voicing or extending the ideas in the late discussions than in the pre-discussion. Comparing the early and late discussions, similar shifts were observed. Teachers more frequently probed students’ thinking but less frequently took up students’ thinking with moves like revoicing or extending students’ ideas. Students’ participation in the pre-discussion (practice discussion videotaped prior to the program) was compared to their participation in the “early” discussions (the first two discussions videotaped during the program), and in the “late” discussions (the last two discussions videotaped during the program, or the last one for three teachers). Their participation in the early and late discussions was also compared to each other. At least four science discussions for 13 teachers were analyzed with a formal coding scheme based on the ‘Science Lens’ tags in the VideoReView software tool, and on prior research from the NSF-funded Talk Science project (#0918435).

First, comparing the pre-discussion to early discussions, the data analysis revealed a high frequency with which students contributed science ideas related to the science learning goals during both discussions. This finding suggests that the discussions tended to stay focused on the underlying goals of students’ learning. Students also reasoned frequently based on evidence and experiences from classroom investigations during both pre- and early discussions. Additionally, the frequency with which students merged their ideas with peers’ thinking; supported their ideas with scientific concepts and principles; and reasoning based on everyday experiences increased in the early discussions as compared to the pre-discussion.

Further, comparing the pre-discussion to late discussions, students again more frequently merged their ideas with peers’ thinking and supported ideas with scientific concepts in the late discussions. On the other hand, they less frequently supported their ideas with evidence from their classroom investigations in the late discussions. One possible explanation for this finding is that as students gain increased familiarity with the formal scientific ideas, they may draw more on their conceptual understanding when explicating their reasoning, and less on specific evidence and details of the classroom investigations.

Finally, similar findings were noted while comparing the early discussions to late discussions. There was an increase in the frequency with which students merged their ideas with peers’ thinking and supported their ideas with scientific concepts during the late discussions. On the other hand, they drew less frequently on everyday experiences in the late discussions, and on evidence and experiences from classroom investigations.



In the exit interviews, teachers reported several areas of growth in their classroom practice as a result of their participation in the VideoReView program. Teachers recognized the value of planning science discussions. Some teachers had not explicitly planned discussions before participating in the program. Planning discussions helped teachers to focus on the learning goals and to identify organizing questions and strategies like follow-up questions. Teachers also considered anticipating student thinking as key element of planning discussions. They reported shifting from anticipating their own moves/actions to what students might think in relation to the learning goals, and how students’ ideas could be used as building blocks.

Teachers also reported shifts in using more open-ended questions to extend students’ thinking, and in opening up the discussion floor to invite greater student participation. Furthermore, they pointed to changes in how they perceived their students’ learning in science. Specifically, teachers began to focus more on students’ progress towards the learning goals instead of limitations in their understanding, and they reported greater awareness of the complexity of students’ thinking. Finally, teachers described greater participation in classroom discussions from students with diverse needs and inclinations.



Students across two Grade 4 classrooms (N = 21) from one school participated in a written pre-and post-assessment measuring their understanding of concepts addressed in the Inquiry Project curriculum, which was the curricular unit taught in these classrooms during the VideoReView program. The maximum possible score on the overall test was 35. Sub-scores were also calculated for three sub-tests: Matter (9 of 35 score points); Measurement (Scale, Proportion and Quantity – 12 of 35 score points); and Argumentation (14 of 35 score points). There were statistically significant differences in students’ scores between the pre- and post-assessment, with respect to both the overall score and the three sub-scores: Measurement; Matter; and Argumentation. Post-assessment scores were all significantly higher than their pre-assessment counterparts, both for the overall score and for the separate sub-scores. The average pre-assessment score was 13.95 which increased to 21.52 on the post-assessment, thus showing an average learning gain of 7.57 points. Among the sub-scores, the largest growth was for the sub-test on Measurement. The average pre-assessment score for this sub-test was 4.33 and increased to 8.00 points on the post-assessment, thus showing an average learning gain of 3.67 points.



The implementation of the VideoReView professional learning program with the second cohort of teachers revealed key insights into the nature of teachers’ skills in noticing and responding to students’ thinking, and into the nature of supports that are important to foster these skills. The findings reveal that with supports targeted explicitly at noticing students’ ideas and reasoning about the subject matter, teachers can develop sophisticated skills in attending to and making sense of students’ thinking. Additionally, teachers develop increased inclinations to focus on specific student ideas, thus shifting away from perceiving their students’ understandings as undifferentiated events to identifying specific ideas that lend themselves to further scrutiny. Sharpening their focus on students’ thinking is also accompanied by a greater consideration of science pedagogy as it relates to students’ thinking (during Video Club meetings), and a shift away from thinking about generic pedagogical matters. Finally, teachers adopt an interpretive stance more frequently as they engage in sense-making of students’ ideas (during Video Study meetings with a program coach).

These developing skills may enable teachers to hone their attempts at responsive teaching by considering students’ progress towards the learning goals. The heightened awareness of students’ thinking translates into changes in the moment of teaching, as teachers begin to respond more frequently to students’ ideas, probing them for clarifications or explanations, and fostering co-construction of meaning by encouraging students to build on their peers’ thinking. Additionally, teachers also report shifts in their reflection on teaching and on their students’ science learning, revealing a greater attunement to students’ ideas. Thus, during both reflection on students’ thinking and actual facilitation of it, teachers begin to “stay with students’ ideas” more frequently.

These findings may be better understood in light of the design of the professional learning program. The design was refined based on insights from the pilot implementation with a smaller cohort of teachers. Specifically, different supports were added to foster teachers’ noticing during their independent Video Study and during the Video Club meetings. First, the Video Study was enhanced with an element of coaching; teachers met with a program coach to study their videos. These meetings were structured with the help of written guidelines containing explicit language on identifying and analyzing a small number of student ideas that seemed intriguing, promising or problematic to the teachers. The emphasis on studying one or two student ideas was intended to engage teachers more deeply with students’ thinking, thus limiting cognitive overload that likely arises when analyzing a large number of ideas. The guidelines also drew teachers’ attention to two features of the VideoReView software tool that were designed to facilitate deep engagement with students’ ideas. First, teachers were prompted to use the ‘Event Marker’ feature to highlight specific events in the video for deeper analysis. Second, for the highlighted events, teachers were prompted to use the “Science Lens” tags to scrutinize the reasoning behind students’ ideas, and to discuss with the coach what students meant by their statements, why students might think in particular ways, and how different student ideas compare.

Thus, this Video Study environment was designed to help teachers externalize their observations and analyses to a program coach, and to engage deeply in sense-making of students’ thinking, leading them ultimately to identify questions about student thinking and to consider pedagogical implications with their colleagues. As a result, the social nature of the video study introduced the teachers to new ways of noticing and examining their classroom events, and these skills were further reinforced by collegial discourse during Video Club meetings.

Regarding Video Club meetings, although the basic structure and protocols continued from the pilot implementation, adaptive supports were added in the present implementation. The program provided teachers with additional video resources –exemplar videos from other teachers’ classrooms (sourced from the NSF-funded Talk Science project) to highlight strategies for productive science discussions. These supports responded to needs expressed by a subset of the teacher cohorts to elicit students’ ideas, to deepen their reasoning, and to help students build on their peers’ thinking. It is conceivable that the additional supports influenced the greater frequency with which teachers probed students’ ideas and fostered co-construction during classroom discussions. More in-depth analyses of classroom facilitation by different teacher cohorts is needed to confirm this point.

The design of the research associated with the professional learning program was also refined following the previous pilot implementation. The present research incorporated a pre-discussion and its video study as a true baseline measure of teachers’ skills at noticing and responding to students’ thinking. Comparing the baseline data to the data gathered on teachers’ video studies and classroom discussions during the program generated rigorous, summative information on the impact of the program and revealed when changes occurred in the course of the program.