Project Profile

The proliferation of artificial intelligence (AI) and powerful Large Language Models (LLMs) has evoked excitement and confusion among K-12 teachers regarding AI's impact on teaching, assessments, and student work. It is vital for researchers with expertise in human-centered teaching and learning to share empirically grounded proofs-of-concept of teacher-AI teaming to enhance teacher capacities for better learning outcomes for all students. This RAPID project examines how the use of LLM-powered tools in high school math classes empowers teachers to create complex technology-enhanced assessments (TEAs) that formatively measure higher-order thinking skills and facilitate deeper learning. The task of authoring such TEAs has necessitated programming expertise and extensive technical skills and thus excluded most K-12 math teachers from participating. By showcasing an exemplary model of teacher-AI teaming, this project addresses a crucial, timely need to establish an early, positive narrative that places teachers at the center of the AI revolution in K-12 education. This proposal was received in response to the Dear Colleague Letter (DCL): Rapidly Accelerating Research on Artificial Intelligence in K-12 Education in Formal and Informal Settings (NSF 23-097) and funded by the Innovative Technology Experiences for Students and Teachers (ITEST) program, which supports projects that build understandings of practices, program elements, contexts and processes contributing to increasing students' knowledge and interest in science, technology, engineering, and mathematics (STEM) and information and communication technology (ICT) careers.

This time-sensitive project will advance research on the use of AI and LLMs for teaching in high school mathematics classrooms through the use of Edfinity software that uses the open-source WeBWorK format to generate interactive, auto-gradable, technology-enhanced formative assessments to support student learning. Within this context, teachers will describe a math problem with an LLM tool (ALICE) that is trained to use natural language inputs, generating source code for TEAs along with hints and student feedback. The project brings together a multidisciplinary team of math educators, STEM education and learning sciences researchers, K-12 teacher educators, AI tool developers, and AI experts to examine the integration of ALICE into high school Finite Mathematics courses across 34 rural, urban, and suburban schools in Indiana and Illinois. The project will train high school teachers and gather data to advance and shape our understanding of teacher-AI teaming and domain-specific LLM prompt engineering. The research involves gathering log data from the platform on teachers' ALICE usage and prompt engineering, as well as teacher feedback through surveys and interviews. These data will be analyzed to understand teachers' experiences in using an LLM tool, the impacts on teacher attitudes toward and confidence in AI, and the success of ALICE from teachers' perspectives for the generation of quality, interactive, formative assessments.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.