ServiceNow introduces AutoSynthData for enterprise-agent training data

ServiceNow CoreAI has introduced AutoSynthData, a pipeline intended to turn enterprise-agent weaknesses into training examples suited to the environments where those agents operate. The project was described in a post published October 2, 2026. The authors say that general model capability does not guarantee success with an enterprise’s particular systems, data, workflows or rules. They present the challenge as creating many varied tasks that target weak capabilities while remaining feasible, realistic and possible to evaluate reliably.

AutoSynthData represents each task through three parts: a system specification, a user request and a verifier. The specification lays out applicable instructions and environment rules, and may include initial conditions such as a prepared database state. The request describes the outcome the agent should achieve, along with user-level constraints. The verifier checks whether the agent’s actions produced a valid result, and the authors say it should reject failures without requiring one prescribed route to success.

The pipeline first evaluates a target model in its environment to identify recurring failure patterns, then uses a stronger teacher to help determine which tasks are solvable and what success entails. The team says it extracts the tested capability, relevant tools and workflow, the target model’s failure points, and the properties a correct result must meet. Those findings become capability cards, which guide the creation of fresh tasks rather than passing along original evaluation prompts, entities, action histories or verifier details. Each generated task is checked in the environment, and accepted examples can be used for post-training.

The authors illustrate the approach with EnterpriseOps Gym, an environment they use to demonstrate the workflow. They describe a cycle in which the updated model is evaluated again, allowing remaining weaknesses to shape later rounds of task generation. They emphasize that tasks should reflect plausible work and be achievable with the environment’s available tools and permitted actions. For AI builders adapting agents to internal workflows, the approach ties training-data generation to observed failures and checked tasks in the target environment, rather than treating generic examples as sufficient. The post does not provide performance results or a measure of how much AutoSynthData improves a model.