Home / Tests

Tool tests

AI tool tests

Every test runs the same prompt on each tool, publishes the raw outputs, and scores them on a fixed 5-part rubric. No published tests yet. The first batch is in progress, and we won't show results we didn't actually run.

No tests published yet

Below is the card format each test will use. It's only an example of the layout, not a result.

Test card format

Example structure · not a real test

[Task name]: e.g. "Write a SQL query from a schema"

Run on [date] · [tool] [plan] · [model/version] · prompt hash [#]

Prompt (verbatim)
[The exact prompt, identical for every tool]
Output (raw, unedited)
[Full output or screenshot + link to the log]
RubricWhat it meansScore
AccuracyFacts and code correct when we check them[0–5]
Instruction followingDid it do exactly what was asked[0–5]
UsefulnessCould a professional ship this with light edits[0–5]
SpeedTime to complete answer[0–5]
Cost per runFrom tracker prices or API rates[0–5]
[x.x]/5overall · verdict: [one-line verdict]

Method: [n] runs, median scored. Limitations: [what this test doesn't measure].

The price-change email

The price changes that matter, and nothing else.

Signups are not live yet. Free, unsubscribe anytime, never sold.