论文

研究:五六年级学生用语音+屏幕共享与 Gemini 调试 Scratch 程序

Share your Screen and Talk to Gemini: How Young Learners Interact with Multimodal GenAI While Solving Scratch Tasks

精选理由

让孩子对着屏幕直接跟 Gemini 说话改 Scratch 代码,听着很酷,但研究测出它 61.5% 的回合里根本没发现孩子的错误,还有近四成小孩盲目相信它。做儿童编程教育或教育 AI 产品的人应该看看这份数据。

一项课堂研究让 23 名五六年级学生在完成六节 Scratch 课程后,通过语音加屏幕共享的方式与多模态生成式 AI 导师互动,完成 8 个调试任务。

正文

结果显示,导师的回答可理解、可执行,且单次回复从不直接给出完整答案;但问题在于错误检测:603 个与错误相关的课堂回合中,导师漏判已有错误的比例达 61.5%,而未进入课堂的 8 任务试点研究中这一比例仅为 15.7%。

另一个值得注意的数据是,尽管事先看过说明视频,仍有 39.1% 的孩子认为导师的回答总是正确的。研究者据此指出,语音交互的自然流畅和屏幕共享本身,并不能保证 AI 看到真实的程序状态或给出正确支持,这对教育者和开发者都有直接参考意义。

原文 · arXiv cs.AI

Share your Screen and Talk to Gemini: How Young Learners Interact with Multimodal GenAI While Solving Scratch Tasks

Children using Scratch to learn programming can struggle with text-heavy, lengthy explanations and hints during debugging. Verbal interactions based on screen sharing or camera data may be an interesting alternative for supporting learners. We therefore explore the quality, use, and perceptions of fifth- and sixth-graders' verbal interactions with a GenAI tutor. The goal is to investigate the quality and use of the multimodal GenAI tutor in the given context. First of all, a pilot study of eight tasks conducted without students helped inform the process of the tutor's classroom evaluation. The classroom study was realized after fifth- and sixth-graders had participated in six Scratch classroom lessons. Then 23 students were introduced to the tutor, worked on the eight tasks with the GenAI tutor, and completed a questionnaire. The analysis of the pilot and classroom study shows that the tutor's responses were intelligible and actionable, and no single turn revealed a complete solution. However, the tutor missed existing errors in 61.5% of 603 error-related classroom turns, compared to 15.7% in the pilot study. Children's voice prompts were brief and mostly responded to the tutor. Although a video had explained that responses could be incorrect, 39.1% of the children trusted the tutor's answers to always be correct. These findings have implications for educators and learners using multimodal GenAI tutors, as they need to be aware that an intelligible and human-like voice does not ensure correct support. For developers, our results suggest that screen sharing alone does not ensure responses that match the program state.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Gemini、Scratch、多模态、AI教育,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Sven Jacobs, Lukas Berning, Natalie Kiesler),经 AITOP 多维评分管线结构化提炼。

研究:五六年级学生用语音+屏幕共享与 Gemini 调试 Scratch 程序|AI热报