Agentsia Labs · Research surface
Frontier leaderboards measure SWE-bench, MMLU-Pro, GPQA Diamond, HLE. The active Agentsia evidence is adtech: auction decisions, bid shading, and MFA classification. Credit reasoning, clinical documentation, automotive agents, and other narrow workflows remain future Assay candidates until their evidence is frozen.
Agentsia Labs publishes Assay, an open benchmark programme for domain-specific model evaluation. The current active surface is Assay-Adtech v1 on a released 344-scenario corpus; other verticals are roadmap candidates, not active proof.
Current benchmark
Brand-safety, bid shading, and MFA classification under a 100ms auction envelope. Released corpus v1.8.0-rc.4 with current-hash production frontier baselines for Claude Opus 4.7, Gemini 3.1 Pro Preview, and GPT-5.5. The released leaderboard and release artefacts remain the citable record.
Specification
Methodology draft
Released corpus v1.8.0-rc.4
Current-hash frontier proof package
Technical corpus inspection
The programme
Assays are designed to ship with scenario datasets, rubric prose, scoring code, and redacted non-sensitive run metadata. The current public surface shows the programme status before the first frozen dataset is published.
Each release candidate will be tied to a dataset tag, harness version, model version, date, temperature, and run configuration. Retest policy is part of the release contract, not a vague promise.
Rubrics are written in prose first, then translated into scoring functions. Practitioner reviewers are named on release pages when a release is frozen, so review status stays auditable.
Methodology at a glance
The current methodology is a draft protocol. It defines how an Assay moves from scenario authoring to reviewer validation, frozen dataset, scored run, release note, and retest.
The roadmap
Future labels are planning posture, not released results. A non-adtech vertical enters the refresh cadence only after its first dataset, rubric, and result table are frozen.
Active evidence: released 344-scenario RTB reasoning, bid shading, pre-bid MFA filtering, supply-path optimisation, blocklist decisioning, and latency-constrained reasoning.
Future candidate: fraud typology, credit decisioning, transaction-pattern reasoning, and compliance-aligned reasoning after adtech evidence.
Statutory interpretation, contract analysis, regulatory reasoning under jurisdiction-specific constraints. In design.
Future candidate: clinical triage, decision support, and documentation quality after clinical reviewer gates are established.
ADAS perception edge cases, in-vehicle agent reasoning, driving-telemetry interpretation. In design.
Colophon
Labs is the public research surface for Agentsia’s vertical evaluation work. The public site shows release specifications, methodology, roadmap, and links to open harness artefacts. Customer data, private prompts, raw logs, adapter paths, and internal run artefacts stay outside Labs.
If you want the platform behind the programme, visit agentsia.uk/platform. For release feedback, use the contact form.