Context Caching
Current Answer
No editorial synthesis yet — the evidence below is collected automatically from source labels. A current answer lands here once an editor approves one.
Evidence
TokTier targets a hidden agent-serving bottleneck: repeatedly tokenizing long transcripts after small tool results. Its stateful repair cut median vLLM time to first token by 16–34% in reported tests.
ReToken adds one learned retrieval embedding to select relevant visual tokens from a cached context, improving long-image and video benchmarks while fitting training and inference on one H100.
Two API settings—reasoning retention and compaction—reportedly tripled GPT-5.6’s ARC-AGI-3 score. Agent evals should treat runtime configuration as part of the tested system.
A delayed KV-eviction policy helps when memory reuse is sharp and observable, but loses its edge on natural-text benchmarks—a useful warning for agent-memory claims.
OmniRoute puts many model providers behind one OpenAI-compatible endpoint, with routing, quota failover, cost telemetry, compression, and coding-agent setup. Its breadth raises operational and trust questions.
Reasonix is a configurable terminal coding agent built to preserve DeepSeek prefix-cache reuse across long sessions, with optional planner/executor models and subprocess-based plugins.