Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute
Training on a production harness could adapt an agent to real workflows, but non-replayable interactions break common RL assumptions and make feedback harder to turn into updates.
Replayable synthetic environments support **GRPO** by running several trajectories from the same initial state, grading them, and adjusting model weights toward the better paths.
For a custom agent, preserve traces and feedback at the model boundary. A **bring-your-own-harness** setup can train on the actual workflow instead of a brittle simulation, even when the training team cannot inspect its source.
Replayable synthetic environments support **GRPO** by running several trajectories from the same initial state, grading them, and adjusting model weights toward the better paths. For a custom agent, preserve traces and feedback at the model boundary. A **bring-your-own-harness** setup can train on the actual workflow instead of a brittle simulation, even when the training team cannot inspect its source. Production interactions may be non-replayable and off-policy, so conventional comparative rollouts no longer fit. **Self-distillation**, automated trace curation, and qualitative-feedback ingestion remain open research directions rather than general solutions.
This shifts post-training from generic datasets toward learning at the real model–harness boundary, where preserved traces and feedback become core infrastructure. It confirms comparative GRPO is practical in replayable environments but sharply narrows that promise for non-replayable production work, leaving self-distillation and trace curation as unresolved bridges rather than established substitutes.