S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?
S3Gym tests whether agents can explore, judge outcomes, and reuse experience. Raw history, compressed memory, and training each help in some tasks, but none is a reliable default.
S3Gym evaluates self-testing, self-judging, and self-improvement in **seven text-based games** with executable verifiers. It compares **History ICL**, score-conditioned Summary Memory, and parameter Training.
Choose memory by task structure. Summaries fit reusable strategic rules; raw history can preserve precise state details; parameter updates may help when context-only adaptation is insufficient.
S3Gym evaluates self-testing, self-judging, and self-improvement in **seven text-based games** with executable verifiers. It compares **History ICL**, score-conditioned Summary Memory, and parameter Training. Choose memory by task structure. Summaries fit reusable strategic rules; raw history can preserve precise state details; parameter updates may help when context-only adaptation is insufficient. Improvement was inconsistent across model-game pairs. Training sometimes produced large gains but also instability and severe negative transfer, showing that recognizing good actions does not guarantee a transferable policy.
S3Gym extends the candidates’ trajectory-aware reliability tests from observing sequential behavior to comparing three ways agents adapt from their own verified outcomes. It narrows the self-improvement claim sharply: executable feedback and correct self-judgment are not sufficient for consistent learning, while the best memory or update mechanism depends on whether a task rewards reusable strategy, exact state retention, or parameter change.