The LLM proving ground.

Why not a leaderboard, a playground, or an eval platform?

Leaderboards give you an aggregate score you can't verify. Playgrounds give you a chat you can't cite. Eval platforms make you build the pipeline yourself. Nobody publishes a specific, dated, fully reproducible test of one model on one realistic task — exact prompt, exact parameters, raw output, at a permanent URL.

What a proving-ground result looks like

A probe is one realistic task; a run is that probe against one model on one date. Every box below shows a piece of what a published run page contains — with mock content, clearly marked, since Phase 1 hasn't run yet.

{
  "vendor": "Acme Supply Co.",
  "invoice_number": "INV-20431",
  "line_items": [
    { "description": "Widget A", "qty": 12, "unit_price": 4.50 },
    { "description": "Widget B", "qty": 3,  "unit_price": 19.99 }
  ],
  "total": 113.97
}

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id>",
    "temperature": 0,
    "seed": 42,
    "messages": [{ "role": "user", "content": "<probe prompt>" }]
  }'

Why should you believe a number on this site?

Because you don't have to take it on faith. Every claim here traces back to a raw output, a published rubric, or a versioned probe definition — the same discipline that the fabricated, unmethodologied rankings previously on this domain didn't have.

01

Raw output, never summarized away

Every run publishes the exact prompt, the exact parameters, and the model's unedited response. The score is never shown without the output that produced it.

02

Judge prompts are published too

Where a rubric needs a judge model to grade it, that judge's own prompt is public. If you don't trust the grade, you can re-grade the output yourself.

03

No rating without a method

A score with no stated process behind it is a rumor, not data. Nothing is published here that can't be traced back to a documented scoring method.

04

Runs are immutable

A published run is never edited or deleted. If a probe's definition changes, that's a new, separately versioned probe — the old runs stay exactly as they were.

Why this gets more valuable over time

Every run is permanent, so the dataset compounds monthly instead of resetting. A competitor starting next year can't manufacture a baseline they didn't record — the history is the moat.

1. Phase 1In progress

   Curated probe library

   20–30 hand-built probes across structured extraction, code, instruction-following, and consistency, run against 8–12 current models.

2. Phase 2Planned

   Regression tracking

   The full probe suite re-runs monthly. This is the moat: a longitudinal record of model behaviour that can't be backfilled by a competitor starting later.

3. Phase 3Planned

   Bring your own probe

   Authors add their own probes, run them with their own key, and optionally publish to the public library after review.

4. Phase 4Planned

   Paid tier

   Private probe suites and scheduled regression runs for teams who depend on a specific model behaving consistently.

The public probe library, every historical run, and the full methodology are free — always.

Frequently asked

These are close to the exact questions people already type into ChatGPT, Claude, or Perplexity — and today those assistants have no good source to cite, because leaderboards don't answer task-specific questions and playgrounds don't publish anything. Here's the direct answer to each one.