Learning on the Job: The Future of Post-Training — Raymond Feng, Applied Compute
Training on a production harness could adapt an agent to real workflows, but non-replayable interactions break common RL assumptions and make feedback harder to turn into updates.
Replayable synthetic environments support **GRPO** by running several trajectories from the same initial state, grading them, and adjusting model weights toward the better paths.
For a custom agent, preserve traces and feedback at the model boundary. A **bring-your-own-harness** setup can train on the actual workflow instead of a brittle simulation, even when the training team cannot inspect its source.
Replayable synthetic environments support **GRPO** by running several trajectories from the same initial state, grading them, and adjusting model weights toward the better paths. For a custom agent, preserve traces and feedback at the model boundary. A **bring-your-own-harness** setup can train on the actual workflow instead of a brittle simulation, even when the training team cannot inspect its source. Production interactions may be non-replayable and off-policy, so conventional comparative rollouts no longer fit. **Self-distillation**, automated trace curation, and qualitative-feedback ingestion remain open research directions rather than general solutions.
This turns production traces and harness feedback into potential post-training material, extending self-improvement from issue discovery toward weight updates on the real workflow. It also sharply limits that promise: GRPO depends on replayable comparable trajectories, while live interactions are often off-policy and non-replayable, leaving trace selection, qualitative feedback, and self-distillation unresolved.