Why AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax
MiniMax argues that long-running, multimodal agents need enough context for repeated tool responses, while sparse attention makes that context more tractable.
MiniMax M3 has roughly **400B total parameters**, activates **20B**, and supports text, images, and video. Its **1M-token context** uses MiniMax Sparse Attention: an index branch selects relevant blocks, then a sparse branch performs attention over them.
For agent builders, the practical claim is that multi-round tool responses and long, unstructured inputs can exceed short contexts. M3’s native multimodal training also points toward agents that inspect reports, presentations, tutorials, images, and video before acting with tools.
MiniMax M3 has roughly **400B total parameters**, activates **20B**, and supports text, images, and video. Its **1M-token context** uses MiniMax Sparse Attention: an index branch selects relevant blocks, then a sparse branch performs attention over them. For agent builders, the practical claim is that multi-round tool responses and long, unstructured inputs can exceed short contexts. M3’s native multimodal training also points toward agents that inspect reports, presentations, tutorials, images, and video before acting with tools. A large context window does not establish useful recall across its full length, and the discussion supplies no retrieval or agent-task measurements. Trillion-token attention is presented only as a research direction requiring architecture and hardware work.
This signal supplies an architectural rationale for million-token, multimodal agents: accumulated tool output and unstructured media may require sparse attention rather than simply larger dense windows. It does not validate full-window recall or agent performance, so it reinforces—not resolves—the candidates’ recurring requirement for matched workload tests before treating context capacity as useful capability.