How Adding Metacognitive Requirements in Support of AI Feedback in Practice Exams Transforms Student Learning Behaviors

Providing personalized, detailed feedback at scale in large undergraduate STEM courses remains a persistent challenge. We present an empirically evaluated practice exam system that integrates AI generated feedback with targeted textbook references, deployed in a large introductory biology course. Our system encourages metacognitive behavior by asking students to explain their answers and declare their confidence. It uses OpenAI's GPT-4o to generate personalized feedback based on this information, while directing them to relevant textbook sections. Through interaction logs from consenting participants across three midterms (541, 342, and 413 students respectively), totaling 28,313 question-student interactions across 146 learning objectives, along with 279 surveys and 23 interviews, we examined the system's impact on learning outcomes and engagement. Across all midterms, feedback types showed no statistically significant performance differences, though some trends suggested potential benefits. The most substantial impact came from the required confidence ratings and explanations, which students reported transferring to their actual exam strategies. About 40 percent of students engaged with textbook references when prompted by feedback -- far higher than traditional reading rates. Survey data revealed high satisfaction (mean rating 4.1 of 5), with 82.1 percent reporting increased confidence on practiced midterm topics, and 73.4 percent indicating they could recall and apply specific concepts. Our findings suggest that embedding structured reflection requirements may be more impactful than sophisticated feedback mechanisms.

Understanding Teacher Perspectives and Experiences after Deployment of AI Literacy Curriculum in Middle-school Classrooms

Artificial Intelligence (AI) and its associated applications are ubiquitous in today's world, making it imperative that students and their teachers understand how it works and the ramifications arising from its usage. In this study, we investigate the experiences of seven teachers following their implementation of modules from the MIT RAICA (Responsible AI for Computational Action) curriculum. Through semi-structured interviews, we investigated their instructional strategies as they engaged with the AI curriculum in their classroom, how their teaching and learning beliefs about AI evolved with the curriculum as well as how those beliefs impacted their implementation of the curriculum. Our analysis suggests that the AI modules not only expanded our teachers' knowledge in the field, but also prompted them to recognize its daily applications and their ethical and societal implications, so that they could better engage with the content they deliver to students. Teachers were able to leverage their own interdisciplinary backgrounds to creatively introduce foundational AI topics to students to maximize engagement and playful learning. Our teachers advocated their need for better external support when navigating technological resources, additional time for preparation given the novelty of the curriculum, more flexibility within curriculum timelines, and additional accommodations for students of determination. Our findings provide valuable insights for enhancing future iterations of AI literacy curricula and teacher professional development (PD) resources.

Constructing Dreams using Generative AI

Generative AI tools introduce new and accessible forms of media creation for youth. They also raise ethical concerns about the generation of fake media, data protection, privacy and ownership of AI-generated art. Since generative AI is already being used in products used by youth, it is critical that they understand how these tools work and how they can be used or misused. In this work, we facilitated students' generative AI learning through expression of their imagined future identities. We designed a learning workshop - Dreaming with AI - where students learned about the inner workings of generative AI tools, used text-to-image generation algorithms to create their imaged future dreams, reflected on the potential benefits and harms of generative AI tools and voiced their opinions about policies for the use of these tools in classrooms. In this paper, we present the learning activities and experiences of 34 high school students who engaged in our workshops. Students reached creative learning objectives by using prompt engineering to create their future dreams, gained technical knowledge by learning the abilities, limitations, text-visual mappings and applications of generative AI, and identified most potential societal benefits and harms of generative AI.