Iterated creation, grading, and revision of data science projects by language models
arXiv — cs.AI ·
What to know
A study evaluated agentic AI models on end-to-end data science projects with iterative feedback. Across 147 initial submissions from seven models, average scores rose from 59% to 76% after three rounds, with gains varying by model (3 to 46 percentage points). Self-grading inflated scores by about 9 percentage points, and half of round-to-round differences were within grader noise.
Source and evidence
arXiv — cs.AIOriginal language · und ·Original author: Rohan Alexander · Source: arXiv and the named paper authors. Descriptive metadata: CC0 1.0.Source licence
Why it matters by industry
- Artificial intelligence
Relevant to AI models and agents: quantifies iterative feedback effects and self-grading bias in agentic workflows.
Reviewed summary. The complete article remains at its original source.