Calibrated Decision Rewards: Training Language Models with a Judge That Knows How Sure It Is
arXiv — cs.AI · Rudransh Agnihotri, Manasvi Kapoor · Published on Readzer
Summary
The paper introduces RLCDR, a reinforcement learning method that uses a calibrated decision model as a reward signal, converting an LLM into a probability distribution over typed questions about a response. Training Qwen3-1.7B with GRPO on MATH, this reference-free reward improves MATH500 accuracy from 72.2% to 76.8% (+4.65 points, p < 1e-4), matching training on the answer key (76.9%), while a random reward gains only +0.65. The reward remains calibrated on the policy's own samples, with expected calibration error half that of a softmax judge on a 27B backbone (0.074 vs. 0.141), and shows no sign of reward hacking.
This is a reviewed summary. Read the complete article at the original source.
Why it matters by industry
- Artificial intelligence
This research introduces a novel reward modeling approach for reinforcement learning in language models, directly relevant to the 'Reasoning' and 'Models' subtopics, with potential implications for improving model training efficiency and reliability.