Accuracy
The answer matches a known-correct reference on a factual or procedural ask. An invented citation or a quietly wrong step fails the item.
ELIMINATOR · THE CARD
One ask, three runs, four cuts. Bring your own reference and keep the prompt on the card.
The answer matches a known-correct reference on a factual or procedural ask. An invented citation or a quietly wrong step fails the item.
Same prompt, three runs. Large swings with no temperature excuse count against the model. A stable wrong answer still fails. It is only stably wrong.
Does it refuse what it should, and allow what it should? Over-refusal and under-refusal both go on the card with the exact prompt. Refusal is a temperament. The cut is whether the refusal matches the ask.
Where it fabricates a tool, an API, a paper, or a person. Saying a tool was called is not a tool call. Smooth prose does not save the round.
A reader can reuse the four cuts on any model they already have. Publish the ask and the outcome. Vibes are not a bout. A new model that only wins the demo is a cut.
Which model is best usually means best at which job, on whose dollar, with whose tools.
A fast cheap model and a slow expensive one are not the same weight class. One ladder is how a poster lies.
A long window that forgets the middle is not memory. Agreement with the prompt mistake is not accuracy.
Almost as good, at a much lower cost, can take the cheap seat. Almost has to be shown, not announced.
You can read one family's weights and not another's. A closed card is not an open weigh-in.