What benchmarks miss about LLMs

Benchmark scores for new models keep going up, but that does not always match my experience when I use them for real work.

Benchmarks measure something useful, but they are not the same as real usage. A model can score well on coding tasks because it memorized the test set, not because it can reason through new problems, and it can do great on a math benchmark while it still fails to follow a simple instruction in my editor.

A big part of the difference comes from everything around the model: the system prompt, the agent you use, the MCPs and skills you load, and most of all, the way you prompt. Two people can use the same model and have completely different experiences with it.

GPT-5.5 is a good example. I have been testing it recently and it fits my workflow well, but it is quite sticky. Once you ask it to do something, it has a hard time letting go, so if you tell it to commit a change, it will keep committing every change after that. Some people will love this behavior and others will find it frustrating, because it depends on how you prompt and what you expect.

In my real work, Kimi K2.6 performs about the same, but I have not found any benchmark or leaderboard that shows this, or that explains why one of them feels better than the other on a given day.

Opus 4.7 is one of the models I enjoy working with the most, and a big reason is how it feels in a long session, which is something no score really captures. The thing I value most is honesty about uncertainty. I would 100% prefer a model that says it does not know over one that invents a wrong answer and sounds sure about it, and I have not seen a benchmark that measures that well.

There is another problem: sometimes models are trained on the benchmark data itself. LMSYS showed that a 13B model trained on rephrased test questions could match GPT-4 on those benchmarks, and the usual decontamination checks did not detect it. OpenAI also mentioned in the GPT-4 technical report that parts of BIG-bench were accidentally included in GPT-4's training data.

The benchmark I trust is my own work. Run the model on your own tasks with your own setup, and pay attention to what happens. The numbers are a good starting point, but for now that is all they are.