Research framework lets an agent revise its own harness across tasks

A new arXiv paper describes a framework in which a language-model agent modifies the software harness it uses across multiple tasks. A harness organizes prompts, tool calls, context and execution, making it part of how an agent carries out work rather than the model itself. The paper’s central change is to use the same frozen model in both roles: first as a task solver, then as a proposer that examines complete run records and edits its own harness. The paper is titled “Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer” and lists Qiankai Xu as its author.

The approach differs from methods described in the paper that use a separate proposer operating on a human-designed harness, and that evolve a different harness for each benchmark. Here, evolution batches draw tasks from five benchmarks across different domains. Training tasks and held-out tasks are kept separate to test generalization, and the evaluation also includes five out-of-distribution benchmarks that were not used during evolution. The authors frame the work in two stages: multi-task pretraining followed by continual training.

The first stage began with a seed harness of 49 lines. At its end, the resulting harness raised the average score by 4.48 points on in-distribution benchmarks and by 12.64 points on out-of-distribution benchmarks. The paper reports that this result surpassed Codex on the in-distribution set and matched it on the out-of-distribution set. In the second stage, continued evolution on Claw-Eval lifted the score there from 66.17 to 68.06, which the paper says exceeded Codex. These figures describe the paper’s reported benchmark results, rather than a general guarantee for other tasks or agent systems.

The analysis identifies output truncation, history compaction and independent review as mechanisms that emerged during evolution. For builders, the work treats the harness as something that can be improved through task experience, not only as fixed scaffolding designed in advance. Its evaluation design also highlights the importance of checking changes on held-out and out-of-distribution tasks, rather than relying only on tasks used during evolution. The abstract does not establish how the approach performs in production settings, but it presents a concrete multi-task benchmark evaluation of an agent directly editing its own harness.