AGENTIC-SYSTEMSJuly 8, 20261 min read

CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming

Researchers introduce CSTutorBench, a benchmark evaluating small language models (SLMs) as tutors for block-based programming. The study reveals that while SLMs show promise, they still struggle with deep pedagogical tasks like avoiding answer leakage.

Computer Science > Artificial Intelligence

Title: CSTutorBench: Benchmarking Small Language Models as Tutors for Block-Based Programming

Abstract

Large language models are increasingly explored as AI tutors, yet deploying them in K-12 settings raises concerns around privacy, cost, and reliance on proprietary models. Small language models (SLMs) offer a promising alternative, but selecting the right model for a specific educational context remains difficult, particularly when the target domain, such as block-based programming, is largely absent from model training data. We introduce CSTutorBench, a benchmark for evaluating language models as CS tutors in VEX VR, a block-based robotics environment. The benchmark comprises 17 scenario-based questions scored against a pedagogical rubric grounded in established tutoring and feedback research, with a human-in-the-loop LLM-as-judge pipeline for evaluation. Preliminary findings across 11 models (4B-120B parameters) reveal that models perform well on surface-level criteria such as vocabulary and tone but struggle with deeper pedagogical behaviors, particularly avoiding answer leakage and engaging with student debugging histories. In our sample, model family and instruction-tuning approach appear to be better predictors of tutoring quality than parameter count alone, though the small number of models limits the strength of this conclusion. A targeted prompt revision grounded in recent educational prompt engineering research improved scores for 10 of 11 models. These results underscore the value of context-specific, pedagogically grounded benchmarks for SLM selection in educational deployment.

Source: arXiv cs.AI Recent

Stay in the loop

Get the best AI & tech insights in your inbox — no spam, unsubscribe anytime.

More in this category

agentic-systems

Prompt-to-Paper: Agentic AI System for Bioinformatics

Researchers have introduced Prompt-to-Paper, a multi-agent AI framework capable of executing real computational biology experiments and writing complete scientific papers, addressing critical issues like data fabrication and fake citations for just $0.31 per paper.

agentic-systems

Narrative World Model: Narratology-Grounded Writer Memory for Long-Form Fiction

Researchers have introduced the Narrative World Model (NWM), a specialized memory system designed for AI writing long-form fiction. By utilizing a narratology-grounded temporal-state graph, NWM helps AI maintain consistency in complex plotlines, such as character knowledge and evolving relationships.

agentic-systems

Foundation Models for Automatic CAD Generation

Recent advances in LLMs and VLMs enable the automatic generation of parametric 3D designs from natural-language specifications. This study introduces LLMForge, a framework evaluating seven leading foundation models on mechanical design tasks, showcasing the potential of AI in industrial workflows.

agentic-systems

From Graphs to Gradients: Physics-Inspired Structural Attribution for Cyber-Physical IoT Systems and Beyond

Researchers have proposed a novel physics-inspired framework that leverages statistical mechanics and energy-based models to provide robust, dependency-aware explanations for complex cyber-physical IoT systems without the need for explicit directed causal graphs.

agentic-systems

Intelligence is Free, Now What? <br> Data Systems for, of, and by Agents

As the cost of AI inference plummets, we are entering an era of virtually free intelligence. This shift demands a fundamental redesign of data systems to support, manage, and be built by autonomous AI agents.

agentic-systems

Automated Data Readiness for Scientific AI

Researchers have introduced REDI, an open-source framework that automates the transformation of raw scientific datasets into AI-ready data. Tested on the Frontier supercomputer, REDI addresses the data preparation bottleneck in scientific AI domains.