AI Friends: A Design Framework for AI-Powered Creative Programming for Youth

What role can AI play in supporting and constraining creative coding by families? To investigate these questions, we built a Wizard of Oz platform to help families engage in creative coding in partnership with a researcher-operated AI Friend. We designed a 3 week series of programming activities with ten children, 7 to 12 years old, and nine parents. Using a creative self efficacy lens, we observe that families found it easier to generate game ideas when prompted with questions by AI Friend; parents played a unique role in guiding children in more complex programming tasks when the AI Friend failed to help, and children were more encouraged to write code for novel ideas using the AI friend help. These findings suggest that AI supported platforms should highlight unique family AI interactions focused on children's agency and creative self-efficacy.

Scratch Copilot Evaluation: Assessing AI-Assisted Creative Coding for Families

How can AI enhance creative coding experiences for families? This study explores the potential of large language models (LLMs) in helping families with creative coding using Scratch. Based on our previous user study involving a prototype AI assistant, we devised three evaluation scenarios to determine if LLMs could help families comprehend game code, debug programs, and generate new ideas for future projects. We utilized 22 Scratch projects for each scenario and generated responses from LLMs with and without practice tasks, resulting in 120 creative coding support scenario datasets. In addition, the authors independently evaluated their precision, pedagogical value, and age-appropriate language. Our findings show that LLMs achieved an overall success rate of more than 80\% on the different tasks and evaluation criteria. This research offers valuable information on using LLMs for creative family coding and presents design guidelines for future AI-supported coding applications. Our evaluation framework, together with our labeled evaluation data, is publicly available.