Benchmarks Keep Climbing. So Why Do Researchers Say LLMs Can't Jump?

Every model launch this year has come with the same chart: benchmark bars going up and to the right. Math, coding, reasoning, all of it. Meanwhile a slice of the research community keeps arguing the exact opposite — that LLMs fundamentally cannot jump. Scores climb, capability doesn’t. That claim sounds absurd until you look at how it’s constructed.

Worth flagging up front: this isn’t a debate raging on Hacker News or r/MachineLearning. Scan the last month of discussion and you’ll find almost nothing on it. That’s telling in itself — the argument lives in conference tracks, not comment threads. So this piece is about the structure of the disagreement, not the crowd reaction to it.

Position papers are a different genre

ICLR and ICML both run a position paper track. These aren’t papers announcing a new model or a new experimental result. They’re papers arguing that the field is pointed in the wrong direction. The evidence is usually a reinterpretation of existing work, and the core contribution is an argument, not a dataset.

That changes how you should read them. The question isn’t “did it hit SOTA.” The question is “is this claim falsifiable.” That’s also why LLM-skeptic position papers reliably light up every conference cycle. They make claims that no single experiment can knock down.

The whole fight comes down to two words

Interpolation and extrapolation.

Interpolation is filling in between points you’ve already seen. Your training data has A and C, so you estimate B. Extrapolation is going outside — predicting past the edge of the distribution entirely.

The skeptic position is clean: everything an LLM does is interpolation. The training corpus is so vast that the interpolation space now covers roughly the full span of human common sense, but in principle the model never steps one foot outside it. What reads as creativity is just novelty to us, because we haven’t read all the data ourselves.

The counterargument has real teeth. Does the interpolation/extrapolation distinction even mean anything in high dimensions? Once you’re at thousands or tens of thousands of dimensions, essentially every new data point lands outside the convex hull of the training set. Apply the definition literally and every deep learning model is always extrapolating. At which point the distinction has stopped doing any work.

This isn’t really a fight about whether LLMs are good. It’s a fight about how you define the yardstick.

Why rising benchmarks don’t kill the skeptics

Here’s the crux. Benchmark gains are not a refutation, for a few reasons.

The first is contamination. Text resembling benchmark items may have leaked into training data, and proving it didn’t is close to impossible. A score goes up and you cannot cleanly separate capability from recall.

The second is that benchmarks may be interpolation tests by construction. The moment a human writes an exam question, that question sits inside the space of problem types humans already know. Measuring genuine extrapolation would mean asking questions humanity hasn’t answered yet — which nobody can grade.

The obvious rebuttal is goalpost moving, and it lands. Ten years ago Go was the line. Five years ago it was olympiad math. Both fell. The skeptics have an answer, though: the goalposts didn’t move, we just discovered that clearing the earlier ones didn’t mean what we thought it meant.

The case that scaling won’t get you there

“A wall scaling can’t climb” is a strong claim. The reasoning runs roughly like this.

Scaling laws are an empirical regularity: add data and parameters, and loss falls as a power law. But a power law doesn’t converge to zero. It just decays, gently, forever. And what that falling loss measures is next-token prediction accuracy. That means a tighter approximation of the training distribution — not a guarantee of anything beyond it.

This is where the two camps split. Skeptics say filling in a distribution more densely is still just filling it in. Optimists say sufficient density triggers a qualitative phase change. That’s where the emergence debate attaches. But a lot of claimed emergent jumps flatten into smooth curves once you swap a discrete metric for a continuous one — work that has made the emergence case notably less sturdy than it looked a few years ago.

One side says impossible in principle. The other says maybe, if you try hard enough. They’re arguing at different levels, which is exactly why nobody wins.

What this actually means if you ship things

It looks like philosophy. It has a very concrete operational edge.

The line between what a model handles well and what it fumbles may not be difficulty — it may be data density. You’ve had the experience: the model crushes a genuinely hard math problem, then falls apart in front of some cursed legacy system that exists only inside your company. That may not be a coincidence. The first has tens of millions of near-analogues on the internet. The second is one of one.

Accept that framing and your tooling priorities shift. Pulling your problem into the training distribution matters more than reaching for a bigger model. Retrieval augmentation, densely worked examples in the prompt, domain fine-tuning — mechanically different, strategically identical. You aren’t pushing the model outward. You’re pulling your problem inward.

The question that’s left

Which side wins matters less than which question you end up asking. “Is this model smart” has no answer. “Is this task inside the model’s training distribution” does.

So look at what you currently hand off to a model and find the tasks with almost no analogue anywhere on the internet. That list may be exactly the work still waiting for you after the next model ships.