ShelfLife
A digital twin of a real e-commerce company.
- Tasks
- 200
- Last Measured
- Aug 13, 2026
We created a digital twin, ShelfLife, of a live e-commerce company. From ShelfLife, we present a collection of 200 environments drawn from the company's operational data over the past decade to answer the question: can you deploy an agent in a leadership role within this company, and trust it to execute with appropriate judgment and precision?
Every environment is constructed in direct collaboration with the company's leaders and the operators who performed the work in production. We worked iteratively with them to develop a deep understanding of their workflows and domain expertise.
We report benchmark results for the following models on this collection: Claude Opus 5, Claude Fable 5, GPT-5.6 Sol, Grok 4.6, Kimi K3, and Gemini 3.6 Flash.
Overall Leaderboard
Share of tasks each model passed on a single attempt.
Skill Profile
Each model's pass rate on the tasks tagged with a given skill.
Skill Leaderboard
The same figures as a table, with the best score in each row marked.
| Skill | Tasks | Opus 5 | GPT‑5.6 Sol | Fable 5 | Grok 4.6 | Kimi K3 | Gemini 3.6 Flash |
|---|---|---|---|---|---|---|---|
| Long-horizon planningTasks in this collection tagged “Long-horizon planning” — 87 of them. | 87 | 78.8% | 60.8% | 65.1% | 28.7% | 22.6% | 1.7% |
| Deriving unstated methodsTasks in this collection tagged “Deriving unstated methods” — 106 of them. | 106 | 40.3% | 38.0% | 43.7% | 21.9% | 15.6% | 7.1% |
| Numerical finance precisionTasks in this collection tagged “Numerical finance precision” — 141 of them. | 141 | 67.2% | 50.5% | 47.7% | 43.5% | 30.0% | 13.4% |
| Optimization under constraintTasks in this collection tagged “Optimization under constraint” — 46 of them. | 46 | 63.1% | 24.9% | 34.3% | 35.3% | 19.6% | 12.2% |
| Long-horizon state managementTasks in this collection tagged “Long-horizon state management” — 31 of them. | 31 | 68.5% | 69.4% | 62.5% | 29.3% | 48.3% | 60.2% |
| Safety and disciplineTasks in this collection tagged “Safety and discipline” — 59 of them. | 59 | 85.8% | 55.8% | 60.4% | 9.9% | 25.1% | 14.7% |
| Identifying quality of dataTasks in this collection tagged “Identifying quality of data” — 33 of them. | 33 | 54.3% | 39.2% | 77.7% | 17.8% | 35.0% | 26.3% |
| Complex tool-useTasks in this collection tagged “Complex tool-use” — 13 of them. | 13 | 98.1% | 88.5% | 100.0% | 94.9% | 75.0% | 0.0% |
| Avoiding hallucination in artifactsTasks in this collection tagged “Avoiding hallucination in artifacts” — 7 of them. | 7 | 26.8% | 25.8% | 26.8% | 13.3% | 25.4% | 13.1% |
| Disclosure controlTasks in this collection tagged “Disclosure control” — 27 of them. | 27 | 53.8% | 54.5% | 49.5% | 25.2% | 47.5% | 35.5% |
| Overall | 200 | 69.0% | 55.7% | 54.7% | 39.4% | 30.6% | 16.1% |
Cost / Performance
Score against dollars per rollout. The dotted line is the Pareto frontier.