It Was Never Hard, It Was Just Expensive: Kimi K3 and the Myth of the Frontier Lab

I have wanted to write this post for a long time, and Moonshot’s Kimi K3 finally gave me the excuse. The short version is that large language models are not a hard thing to build, and the things that gate this work have little to do with talent. The architectures are public, the techniques are public, and the majority of them are already implemented in open-source models, often first. What actually separates a frontier lab from everyone else is access, to infrastructure, to GPUs, and to data, the kind of access that only enormous amounts of capital can buy. And because the gate is built from money rather than from insight, the people inside it have every incentive to tell the story differently.

Let me start with the data, because the data is what makes this hard to argue with anymore.

Everyone is landing on the same curve

Epoch Capabilities Index scores versus release date for 212 models, colored by open versus closed weights, with a rolling median and 10-90% band across all labs, showing a flattening curve and shrinking variance after the January 2024 transition from the pre-training era to the post-training era, with Kimi K3 circled on the frontier band
The Epoch Capabilities Index versus release date for 212 models, colored by weight accessibility, with the rolling median and the 10–90% band across all labs. I built this chart from Epoch AI's publicly downloadable ECI data (CC BY); a similar one recently ran in Paul Kedrosky's newsletter. Notice how the open-weight points sit inside the same band as the closed ones, all the way up to the frontier.

Kimi K3 has people excited, as if some trend has been broken, but the more interesting trend broke a while ago, quietly, and in the other direction: model performance began converging once we exited the pre-training era around early 2024 and the field became reliant on post-training. Paul Kedrosky recently made this same point in his newsletter, and the numbers hold up however you slice them. Using Epoch’s Capabilities Index, the center of the distribution rose from roughly 117 in early 2024 to about 150 by mid-2026, but the slope is declining and the spread between labs is collapsing, with successive releases delivering smaller gains, fewer durable outliers, and less defensible technical differentiation. Nor is this one chart’s opinion. Stanford’s AI Index measured the score difference between the top-ranked and the 10th-ranked model falling from 11.9% to 5.4% in a single year, with the top two separated by 0.7%; the gap between the best US model and the best Chinese model went from 9.3% to 1.7% over roughly the same period; and Epoch’s own analysis finds that open-weight models, the ones you can download, inspect, and run yourself, lag the closed state-of-the-art by an average of about four months, around 8 ECI points. Four months. That is the entire technical moat of the frontier labs.

So when a Chinese lab, operating under US compute export controls, releases a 2.8-trillion-parameter open-weight model that ranks at or near the top of blind evaluations, it is tempting to read it as a shock, and I would invite you to read it the other way, as a confirmation of a curve that has been sitting in plain sight for two years. None of this takes inside knowledge; every number in this section is public, and anyone who spends an afternoon with Epoch’s data can watch the same convergence unfold. Kimi K3 did not break the trend, it is the trend, arriving on schedule.

Everyone is building the same machine

Here is the part that people outside the field are never told, and people inside the field rarely say out loud: architecturally, all of these models are the same machine.

Sebastian Raschka wrote the definitive comparison of modern LLM architectures back in mid-2025, and although a year of releases has passed since, his observation has only grown more true: from GPT-2 in 2019 to DeepSeek V3, Llama 4, and everything that has shipped after them, the structure is remarkably unchanged. Positional embeddings went from absolute to rotational, multi-head attention gave way to grouped-query and latent variants, GELU became SwiGLU, LayerNorm became RMSNorm, and the feed-forward block became a mixture-of-experts, and that list, refinements layered on the same foundation, is roughly the sum of seven years of frontier “innovation,” every item of it published, documented, and implemented in open models you can download today. The models of this past year have kept to the same pattern, and if anything the flow of ideas runs opposite to what the mystique suggests: Kimi K3’s headline feature, Kimi Delta Attention, builds directly on Gated DeltaNet, an academic line of work from late 2024 that also shows up in AI2’s Olmo models and NVIDIA’s Nemotron, and DeepSeek’s V4-Pro, 1.6 trillion parameters with 49 billion active and a million tokens of context, ships under an MIT license with its sparse-attention scheme described in the open, cutting per-token inference FLOPs to 27% and KV cache to 10% of its predecessor. The most interesting efficiency work in the field is being published, by the labs that supposedly lag, while the closed labs’ proprietary variations on these same techniques are, as far as anyone can reverse-engineer or benchmark, poor man’s versions of published ideas, tuned with more data and more GPUs. A genuinely good idea travels from an academic paper to a frontier-scale model within months, in the open, and nothing along that path passes through a locked room; the only queue it waits in is the one for GPUs.

I want to be precise about what I am claiming, because it is easy to strawman. I am not saying training a frontier model is trivial, or that execution does not matter. Scaling a training run across tens of thousands of GPUs without it falling over is real engineering, and I respect it, the way I respect any large infrastructure project. What I am saying is that there is no scarce individual capability involved. The knowledge is distributed, published, and reproduced on every continent; the scarce ingredient is the capital to act on it, not the ability to understand it.

The gate is capital, not capability

Look at the actual numbers. DeepSeek V3 trained for 2.788 million H800 GPU-hours, about $5.6M at rental prices, and Kimi K2’s base model came in at roughly the same figure, while Stanford estimated GPT-4’s training compute at $78M and Anthropic’s CEO has put current frontier runs at $100 million to $1 billion. If the difference between a $5.6M model and a $500M model were technique, or insight, or some rare human capability, the capability gap would look like the cost gap; instead it looks like four months and 8 ECI points, held in place by exactly the things money buys, more GPUs, more licensed data, more post-training human feedback, more inference infrastructure to serve it all. A well-trained person, or a small team, is not kept out of this work by any limit of talent, they are kept out by not owning a datacenter. And none of this should be news, because a Google researcher wrote it in an internal memo back in May 2023, “We have no moat, and neither does OpenAI”, arguing that the open-source community was faster, more customizable, and pound-for-pound more capable, and that the closed labs’ quality edge was closing astonishingly quickly. That memo was treated as a provocation at the time; three years later, it reads as a measurement.

Who keeps the myth alive

So why does the myth persist, that frontier labs possess something special, that “because X joined Y, Y will now win”? Because the people best positioned to correct it are the people paid by it. The experts do not accept this framing, and I understand why: they are the ones enforcing it. Trillion-dollar valuations, nine-figure acquihires, and the entire talent-poaching news cycle depend on the premise that model capability lives inside particular skulls rather than inside published papers and GPU clusters. If capability is fungible and the moat is capital, then the star-researcher premium collapses, and with it a lot of paper wealth. It is not a conspiracy; it is just incentives, all pointing the same direction, at every level from the individual engineer’s equity package to the lab’s next funding round.

And to be clear, I am not denying the usefulness of these tools. I use them daily. The models are genuinely good, and the harnesses and products around them are often excellent. What I am denying is the pricing of the mystique. What we are sold as a scientific miracle is, structurally, an expensive luxury item: the underlying good is nearly a commodity, converging in quality across a dozen suppliers, and the premium is brand. It should not cost what it costs, and the convergence data says it eventually won’t.

The part where I paid for my own opinion

Now the personal part, because I promised myself I would write this honestly or not at all.

Two years ago, Anthropic reached out to me. I declined. Even then, less verbally formed than it is now, I held the same understanding I have laid out above: that the work inside frontier labs was not a different kind of work than the field already knew, that the capability lived in the infrastructure and not in some anointed set of individuals, and that I did not want to join a horde whose market price rested on a story I did not believe. I will not pretend this was a purely rational financial calculation, it obviously was not. I paid the price, in my daily earnings, and eventually in what a good outcome on those options would have been worth. Believing a thing early, and refusing to profit from the opposite of it, is expensive. I am writing this post partly so that the belief is at least on the record.

I do not regret the reasoning. Watching the curves flatten, watching a lab under export controls ship an open frontier model for a fraction of the spend, I think the reasoning was right, and the market is slowly agreeing. What I would tell my 2024 self is only this: being right about a myth does not exempt you from living in the economy the myth built.

Who actually brings the value

There is one honest counterargument, and it is the one I find most interesting. You can say: fine, if everyone has the same architecture, then the winner is whoever finds the efficient technique, the one that bypasses the infrastructure, data, and compute limits everyone else is buying their way through. I agree completely. But notice what that concedes, because once efficiency is the frontier, the game stops belonging to a handful of model architects and spreads across entire fields, hardware, memory, interconnect, compilers, deployment, datacenters, power, cooling, inference economics, and I think that is a genuinely good thing. It is also, I would argue, the honest picture of where the value came from all along. The systems these models train and run on are built by people from dozens of disciplines, and the capability we casually attribute to a lab is really the accumulated work of all of them, sitting on top of a research literature that anyone can read. This is what I meant when I wrote, in my last post, that no single person or company ever really invents a technology; it emerges from a pool of people, and assigning the credit to one name or one logo is almost always the wrong model of how it happened. Some of the researchers at the frontier labs are, without question, brilliant, and I am not disputing any individual’s capability. What I am disputing is the narrative that a small set of “talented” people, who happened, partly by the luck of their path through life, to be standing near a gold mine when it opened, are the ones who bring the value, while the far larger crowd that built the mine’s every support beam is treated as interchangeable. We should change our accounting of who contributes what to these systems, and the convergence data is giving us every reason to do it.

That is also why Kimi K3 is the right closing note. Its self-reported benchmarks and blind-arena results put it in the same band as the best closed models, and the weights land on Hugging Face days after this post. Whatever you think of the specific numbers, the gesture is the point: here is a frontier-class model, built from published techniques, by people who feel no need to be secretive or ceremonious about it, because there is nothing to be secretive about. The basic things are basic, they always were, and the only thing that was ever proprietary was the bill.

Sources

Benchmark comparisons above are as reported by the respective labs or third-party leaderboards, not my own evaluations; self-reported numbers should be read with the usual skepticism, which is, conveniently, part of my point.