Emulated: The Data for Fully Autonomous Software Engineers and Companies — Joseph Wang
Repo-only tasks omit deployments, incidents, traffic, and distributed failures. Training infrastructure agents may require multi-node environments with real resources, despite much higher setup cost.
Current coding tasks often keep agents inside one repository for **50–100 turns**. Emulated instead packages organizational context, deployments, live-service constraints, network failures, corruption, and clock skew into training environments.
If an agent will operate production infrastructure, evaluate the entire operational loop rather than only its patch. The proposed **multi-node sandbox** provisions real cloud resources so agents can practice deployment, rollback, observability, and blast-radius decisions.
Current coding tasks often keep agents inside one repository for **50–100 turns**. Emulated instead packages organizational context, deployments, live-service constraints, network failures, corruption, and clock skew into training environments. If an agent will operate production infrastructure, evaluate the entire operational loop rather than only its patch. The proposed **multi-node sandbox** provisions real cloud resources so agents can practice deployment, rollback, observability, and blast-radius decisions. This remains an emulation: customer traffic and failures that appear only at scale are still missing. Some stacks take **hours** to provision, making rollout duration, cost, reproducibility, and the remaining simulation-to-reality gap unresolved constraints.
This broadens coding-agent evaluation from repository completion to the full operational lifecycle, including deployment, rollback, observability, failures, and blast radius. Multi-node sandboxes could train and test those behaviors, but the signal narrows autonomy claims by exposing provisioning cost, reproducibility limits, and a simulation-to-reality gap where scaled traffic and production-only failures remain absent.