Collections
Built on real production data, measured against frontier models.
NewShelfLifeA digital twin of a real e-commerce company.200 tasks · measured Aug 13, 2026Top models · Pass@1Claude Opus 569.0%GPT-5.6 Sol55.7%Claude Fable 554.7%AccountingMarketingFinanceDue Diligence & Operational DisciplineEnterprise-Scale Context AwarenessEthical ConductNewShelfLife E-SimA stateful simulator of retail operations.12 tasks · measured Aug 26, 2026Top models · Mean Reward (pts)Grok 4.665.0GPT-5.6 Sol54.1Gemini 3.7 Flash51.7Pricing & InventoryRetail OperationsLong-Horizon Turn-Based Business SimulationMulti-Strategy Resource ManagementAccess Upon RequestCorpLawA legal benchmark built from a real law firm's anonymized data.50 tasks · measured Aug 18, 2026Top models · Pass@1Claude Opus 542.8%Claude Fable 539.5%GPT-5.6 Sol36.0%Legal Writing & ReviewClient CounselNegotiationProfessional Discipline & JudgementLegal Issue Severity AssessmentAccess Upon RequestLong-Horizon SWERepo tasks in Go, Python, Rust, C++, TS, and Java. Each task has 100+ steps and a pass rate below 50% for frontier models.Access Upon RequestCybersecurityCVE finding, patching, and exploitation across public and private codebases.Access Upon RequestRecursive Self-ImprovementModel training, data curation, inference optimization and harness engineering.Access Upon RequestTerminal AgentsExecution environment setup, enterprise API and tool use, data cleaning and anonymization.Request a New DatasetWe make bespoke evaluations for your model. Tell us what to measure next.