Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation
A model learned to choose its own reasoning budget, cutting average output tokens 41% with a small MATH500 accuracy drop. Agent builders could replace fixed effort settings with task-level routing.
A **1.5B distilled model** learned to begin each response in NoThink, Short, or Long mode without a separate router. On MATH500, mean output fell from **4,796 to 2,811 tokens** while accuracy moved from 0.796 to 0.782.
Builders running coding agents should reconsider one fixed reasoning budget for every task. A learned or explicit difficulty gate could reserve longer runs for harder work and shorten routine calls, reducing latency and token use.
A **1.5B distilled model** learned to begin each response in NoThink, Short, or Long mode without a separate router. On MATH500, mean output fell from **4,796 to 2,811 tokens** while accuracy moved from 0.796 to 0.782. Builders running coding agents should reconsider one fixed reasoning budget for every task. A learned or explicit difficulty gate could reserve longer runs for harder work and shorten routine calls, reducing latency and token use. The model was trained on MATH, not coding workloads, and the result averages only **three seeds**. Transfer was strongest on easier benchmarks, including a **76% token reduction on GSM8K**, so coding-agent gains still need direct evaluation.
This adds learned per-request compute selection to the prior menu of test-time reasoning techniques, challenging fixed reasoning budgets as the default. The modest MATH500 accuracy tradeoff and stronger savings on easier tasks suggest gating is most credible for routine work; it does not yet establish that coding-task difficulty can be predicted or that shorter coding runs preserve reliability.