Status
No benchmark run has been published yet. We would rather release nothing than release scores we cannot back up, so the first run will appear only when every result is supported by saved model outputs and a written rubric.
What we measure
- Accuracy: does the output get the facts and reasoning right?
- Completeness: does it address every part of the prompt?
- Tone: does it match the professional register the role needs?
Cost and response time are recorded separately, because they depend on plan, region and load.
Why publish the method first
AI tool capabilities change quickly, and a ranking is only as trustworthy as the process behind it. Publishing the method up front lets readers challenge it before any numbers exist. The full document is at /benchmarks/methodology/ and every change will be versioned.
Open dataset
Published runs will be released as CSV files under CC-BY-4.0. Researchers, journalists and other publishers may cite and reuse them with attribution.
Frequently asked questions
Do you publish benchmark scores yet?
Not yet. We publish the methodology first and will release scores only when every result is backed by saved outputs and a written rubric. Until then our recommendations rest on editorial review, vendor documentation and hands-on use.
What will the benchmarks measure?
Accuracy, completeness and tone on a 0 to 10 scale for profession-specific tasks, averaged over three runs. Cost and response time are recorded separately.
Will the data be open?
Yes. Published runs will be released as CSV files under the CC-BY-4.0 licence so researchers and journalists can cite and reuse them with attribution.