Track eval results over time
Every score, for every model and checkpoint, on every dataset and harness version. The eval job writes with add-only access, so old scores can’t be quietly changed.
An example
@ADoor Eval job finished for r-0412 at 40k: GSM8K 61.3, MATH 24.8, HumanEval 38.4. Eval harness v2.3.
ADoor Added 3 entries to Eval results
What you can ask later
- “Which checkpoint has the best GSM8K on harness v2.3?”
- “Did anything regress more than two points since last week?”
- “Show scores for every model on MATH, newest first.”
- “Which results were run on the old harness and need rerunning?”
- “Chart GSM8K and MATH scores for every checkpoint of r-0412.”
Who can see what
- The eval job writes results with add-only access, so it can’t change old scores.
- An agent like Cursor or Claude can read results and can’t write them.
- Researchers read everything and add notes.
- A partner lab can read results for one shared model only.
How it compares with a spreadsheet or an eval platform
Eval platforms like Braintrust or LangSmith are better at running the evals themselves, per-example traces and side-by-side output review.
As the long-term scoreboard, ADoor holds up: add-only results, plain SQL for regressions, and read-only access for the agents that decide what to train next.
How to start
- Open an assistant like ChatGPT, Claude, Grok or Cursor and connect ADoor through MCP.
- Tell it: “Set up eval results: Eval results (model, checkpoint, dataset, dataset version, score, harness version, run at).”
- Give the eval job add-only access, and share one model’s results with a partner lab if you need to.
Questions people ask
Can results be changed after the fact?
Give writers add-only access, and the history records each change and who made it.
Can I store per-example outputs?
Yes, as entries in a linked list. For large volumes, store them elsewhere and link them.
Is ADoor free?
Yes. The free plan holds 3 tables and 100 rows. Pro is $10 a month per workspace, with 1 GB of storage and no per-GB charges.
What if someone leaves?
Remove their access and it stops working on their next request. The history still shows what they added and changed.
Related
- Keep a model training log
- Log benchmarks and latency measurements
- Start a dataset registry
- ADoor vs Supabase
- ADoor vs Google Sheets
- All use cases
Updated October 20, 2026.