Selective Agent Guidance via Entropy: Learning Autonomous Policies from Imperfect VLM Teachers
SAGE uses an expensive VLM only when a learner is uncertain, tests its advice against environment rewards, and distills useful guidance into a policy that needs no VLM at runtime.
**SAGE** queries a VLM teacher only when the learner is uncertain, executes the proposed action during training, and distills it into a lightweight RL policy. Environment-derived advantages can downweight bad teacher advice.
For visual or computer-use agents, consider the strong model a temporary exploration coach instead of the permanent policy. The learned agents needed **zero VLM calls at deployment** and sometimes exceeded their teachers on sparse-reward reasoning and navigation tasks.
**SAGE** queries a VLM teacher only when the learner is uncertain, executes the proposed action during training, and distills it into a lightweight RL policy. Environment-derived advantages can downweight bad teacher advice. For visual or computer-use agents, consider the strong model a temporary exploration coach instead of the permanent policy. The learned agents needed **zero VLM calls at deployment** and sometimes exceeded their teachers on sparse-reward reasoning and navigation tasks. Selective guidance helped when the teacher revealed high-reward trajectories. It added less when unguided exploration already worked or teacher actions produced little useful experience; the material does not quantify the call reduction or deployment savings.
This narrows strong-model guidance to a training-time exploration aid rather than a permanent computer-use dependency. Environment rewards can discount poor advice, and successful guidance can be distilled into a lightweight policy with no deployment-time VLM calls. The benefit is conditional, however: it depends on the teacher exposing rewarding trajectories that unguided exploration would not readily find.