Skip to content
AI editorial reviewArtificial intelligence

Calibrated Decision Rewards: Training Language Models with a Judge That Knows How Sure It Is

arXiv — cs.AI · Rudransh Agnihotri, Manasvi Kapoor · Published on Readzer

Summary

The paper introduces RLCDR, a reinforcement learning method that uses a calibrated decision model as a reward signal, converting an LLM into a probability distribution over typed questions about a response. Training Qwen3-1.7B with GRPO on MATH, this reference-free reward improves MATH500 accuracy from 72.2% to 76.8% (+4.65 points, p < 1e-4), matching training on the answer key (76.9%), while a random reward gains only +0.65. The reward remains calibrated on the policy's own samples, with expected calibration error half that of a softmax judge on a 27B backbone (0.074 vs. 0.141), and shows no sign of reward hacking.

This is a reviewed summary. Read the complete article at the original source.

Loading reading state…

Why it matters by industry

  • Artificial intelligence

    This research introduces a novel reward modeling approach for reinforcement learning in language models, directly relevant to the 'Reasoning' and 'Models' subtopics, with potential implications for improving model training efficiency and reliability.