XStack: A Benchmark for ML Repo Migration
arXiv — cs.AI ·
What to know
The paper introduces XStack, a benchmark for evaluating AI agents on migrating machine learning repositories between PyTorch on NVIDIA GPUs and JAX on Google TPUs. Agents receive a PyTorch repository and a clean TPU machine, and must deliver a JAX repository that meets grading criteria based on the source repository's published result. Results show that migration remains unreliable, with Claude Opus 5 passing 75%, GPT-5.6-sol 45%, and all other agents at most 40%.
Source and evidence
arXiv — cs.AIOriginal language · und ·Original author: Linrong Cai, Taiming Lu, Haozhe Chen, Zhuang Liu · Source: arXiv and the named paper authors. Descriptive metadata: CC0 1.0.Source licence
Why it matters by industry
- Artificial intelligence
This benchmark directly addresses cross-framework and cross-hardware migration, a key challenge for AI model deployment and portability, relevant to model and agent development.
Reviewed summary. The complete article remains at its original source.