High-fidelity RL environments of real business contexts, grounded in public and proprietary data.
See evalsWe uncover the principles and structures that shape how complex systems behave, and turn them into RL environments.
We write down the system’s rules and how whoever is on the other side behaves.
We fit the model to public and proprietary data, and check it on data it never saw.
The agent gets a goal and works through the same tools it would use for real.
The grade is what actually happened in the world. Same seed, same world, so agents are compared fairly.
Eval 01X Ads
Each agent ran every brief end to end through the same X Ads tools. Its score on a brief is the conversions it got as a share of the best known result: the most any run reached on that brief. Shown is the geometric mean over the held-out briefs.
Best known result: on each brief, the most conversions any of the 1,101 runs reached. Baseline: one ad group set up by the brief’s rules and never touched; it gets 60% of the best known result. Faint ticks are single briefs. Train/held-out split: DeepSeek V4.1 Flash was trained on 181 briefs and is scored here on 45 held-out briefs it never saw.
We built a high-fidelity simulation of X, with its users and advertisers, to post-train models on launching and managing ad campaigns.
Every setup ran three times in the simulator, each with the brief’s budget, one ad group and X’s automatic bidding. The map replays the flight from those runs: each dot is a share of the brief’s audience on X, drawn in its country.
Book a 30-minute call to learn more, or to tell us which system you’d like simulated.