Project Profile

One way K-12 teachers engage in lifelong professional learning is by reflecting on their own practices, for example by reviewing video recordings from their class sessions. Past research shows that reflection has powerful effects on teachers’ classroom practice. A separate line of past research shows that students learn very well with AI-based intelligent tutoring software (ITS), for example, in middle-school and high-school mathematics, and that teachers' support contributes to this effect. It is likely that teachers could help students more effectively if they were to periodically reflect on their classroom practices with regard to this type of software. Yet effective reflection is hampered by the fact that it is nearly impossible for teachers to notice and remember all that happens during any given classroom period. A well-designed tool that presents analytics extracted from class sessions could be a great help. The current research therefore investigates how to design an analytics tool that can support K-12 teachers in reflecting collaboratively with trusted colleagues on their classroom practices around students’ work with intelligent tutoring systems (ITS). The project will build an innovative tool that allows computers to automatically collect and analyze multiple sources of data (i.e., multimodal analytics). The tool will prompt reflection and inform discussion among teachers, for example about their students’ learning progress or challenges, and about their own, possibly implicit, biases towards certain students or groups of students. Through reflection with data, teachers can identify ways to improve their teaching practices, student learning, and equity in their classrooms. The main goal of the project is to design and create such a tool, test whether it supports effective collaborative reflection among teachers, and whether it leads to improved classroom practices. The project will contribute to the broader goal of making analytics useful for teachers. It has the potential to improve a highly effective form of K-12 classroom instruction, namely, learning with ITS, an increasingly common learning environment.

The project will design, create, and pilot-test a new analytics tool for supporting teachers in jointly reflecting on their classroom practices around ITS. The tool will leverage multimodal analytics; it will extract and show trends in students’ learning and teachers’ practices and illustrate them with strategically selected examples from its own data store, recorded during ITS sessions. The tool will take advantage of (1) location data, collected with sensors placed in classrooms, to reveal patterns in teachers’ movement in the classroom, (2) physiological data, collected with physiological wristbands worn by teachers, to reveal ways in which their stress level affects students, (3) log data detailing students’ interactions with the tutoring system, analyzed to detect progress, struggles, knowledge growth, and learning behaviors, and (4) video data that capture important detail about classroom interactions. The reflection tool will be designed with middle-school teachers. Teachers will be involved in the research start-to-finish to make sure the tool matches their preferences and needs. The design will be guided by a proven model of teacher reflection, adapted for multimodal analytics from sessions with an ITS. The tool will be implemented within CTAT+Tutorshop, a widely-used platform for ITS research and development. Many of the needed analytics already exist within CTAT+Tutorshop, but new ones will be developed as well.  Over time, the tool will be continuously improved and pilot tested. At the end, teachers will test a classroom-ready prototype with their classes.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.