Skip to content

READZER / INDUSTRY INTELLIGENCE

Industry desk

Follow important changes. Read the original evidence.

Nov 22 · UTC

Your reading desk
Newest first

Summary preview

Back to updates

Study Finds LLM API Relays Show Behavioral Differences, Tool Permission Violations, and Billed Token Discrepancies

arXiv — cs.AI ·

What to know

A study tested 20 API relays offering GPT-6 Astra and Claude Opus 5, comparing answers to official responses. It flagged 7/10 low-price and 2/10 same- or higher-price relays for behavioral differences, with some flags changing on retesting. No injection was confirmed in 3,332 screened responses, but four relays returned tool calls despite instructions disabling them. Five relays reported 4,200 to 60,000 more input tokens than the official API for identical requests, and charges followed reported usage where balances were available.

Loading reading state…

Source and evidence

arXiv — cs.AIOriginal language · und ·

Original author: Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li, Jianbing Ni · Source: arXiv and the named paper authors. Descriptive metadata: CC0 1.0.Source licence

Why it matters by industry
  • Artificial intelligence

    For AI model deployment and API procurement, this study highlights that advertised models and prices are insufficient to judge relay reliability, emphasizing the need for repeated testing and scrutiny of tool permissions and billing accuracy.

Reviewed summary. The complete article remains at its original source.