---
title: "AI’s path to “intelligence”"
description: "From the model that predicts the next word to the agent that acts in an environment. What each leap solved, and what it left unsolved."
url: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/
date: 2026-09-26
lang: en
translations:
  es: https://latent-universe.pages.dev/el-camino-de-las-ias-hacia-la-inteligencia/
  eu: https://latent-universe.pages.dev/eu/el-camino-de-las-ias-hacia-la-inteligencia/
---

# AI’s path to “intelligence”

*From the model that predicts the next word to the agent that acts in an environment. What each leap solved, and what it left unsolved.*

## TLDR

In this article I propose a way of reading how large language models (LLMs) have advanced in recent years, built around three questions.

-   **Why did the first LLMs, the ones before the reasoning models of 2024, seem so “stupid”?** Because they learned from an internet full of results and almost empty of processes, and they had to give every answer in one go, without working step by step.
-   **What changed with reasoning?** Chain of thought let them solve problems piece by piece, and reinforcement learning taught them to do it on their own and to think longer when necessary.
-   **Why are today’s reasoning models unable to come up with new methods or ideas?** My hypothesis is that reinforcement only refines what the model already manages at least occasionally, and that almost no new idea comes with an automatic way of checking it.

For the first two I draw on papers from recent years. The third has no established answer yet, so there I move into hypothetical territory.

## The first LLMs predicted the next word

A language model does just one thing. Given a text, it spreads probability over all the possible ways of continuing it, picks the next *token* (a piece of a word), appends it and starts again. This is next-token prediction (NTP), and training consists of exactly that, repeated over hundreds of billions of tokens from the internet, books and code. GPT-2 and, above all, GPT-3 showed that, with enough data and parameters, this simple task goes a long way. They could translate, summarise or answer questions without anyone having trained them for any of those tasks (Radford et al., 2019 · Brown et al., 2020).

The versions of ChatGPT from 2022 and 2023 were given a little extra training so that their answers would please people, based on examples of which answers people preferred (Ouyang et al., 2022). That tuning made them far more useful, but I leave it aside here, because I don’t think it is a key piece of the path towards intelligence. Deep down, they were still machines for continuing text.

Two problems came up again and again. The first was hallucinations, false answers stated with complete confidence, such as a citation that doesn’t exist or a made-up date. They are not a rare glitch but the natural consequence of how these models are trained and evaluated. As in a multiple-choice exam that doesn’t penalise wrong answers, taking a guess pays off more than admitting you don’t know (Kalai et al., 2025). The second was multi-step problems, such as a long multiplication, a logic puzzle or a plan. The models got the easy cases right and collapsed as soon as many steps had to be chained together (Dziri et al., 2023). Figure 1 shows this with three examples.[^1]

> **Figure 1.** Next-token prediction in three cases. Press “Next token” or pick one of the bars yourself. In “A fact”, the model cannot know the answer, but saying “I don’t know” is almost impossible. In “A calculation”, it has to write the result digit by digit without having done the multiplication. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-1)*

There is a deeper reason for the second problem. The internet is full of results and almost empty of processes. People publish the solution to a problem and not the twenty drafts, the finished paper and not the weeks of dead ends, “37 × 48 = 1776” and not the long multiplication worked out in columns. Something similar happens to us. Once we have mastered a process, we internalise it and stop writing it down, because we no longer need to. A child, on the other hand, needs to see everything broken down in order to learn to multiply. An LLM that learns from what adults write comes across the results, but hardly ever the breakdown.

Faced with “37 × 48”, there are two ways of getting it right. The first is to have seen that calculation so many times that you know the result by heart. The second is to have learned how to multiply and to apply it to any pair of numbers. We want the second, because it is the only one that generalises, and the second requires chaining together smaller skills, such as multiplying one digit by another, carrying a one or adding in columns. It is like building with Lego pieces, which go on one at a time. The problem is that, to write each token, the model makes a single pass through the network, with a fixed number of layers. If the chain of pieces is longer than what fits in that pass, it cannot assemble it in one go, and all it can do is guess or fall back on memory.

> Specifically, Feng et al. (2023) prove that, under standard assumptions from complexity theory, a fixed-depth transformer cannot directly give the answer to basic arithmetic problems unless its size grows out of all proportion with the length of the problem. And there are simple planning tasks in which NTP training learns a shortcut that imitates the data well but does not learn to plan (Bachmann & Nagarajan, 2024).

This is not just an intuition. There are theoretical results that prove it (Feng et al., 2023), and in practice that is exactly what happened. GPT-3 added two-digit numbers without a single error, but with five digits it was right less than 10% of the time (Brown et al., 2020). And a model getting many examples right does not mean it has learned the procedure. Transformers tend to solve these problems by recognising fragments of calculations they have already seen, not with a systematic method, which is why accuracy plummets as the problem grows (Dziri et al., 2023).

Imagine learning maths from nothing but the answer key, hundreds of problems, each with its result and not a single worked solution. Over time you would recognise patterns (“if the problem mentions percentages, the answer tends to be a round number”) and you would get quite a few exercises right that resemble ones you have already seen. But faced with a new one you could only go on intuition and, when your intuition failed, you would have no way of noticing, because you never learned to check anything. That is what an LLM trained only with NTP is like, a brilliant student who has memorised the answer key. It has excellent intuition, but it is only intuition.

> The LIMA authors call this the **superficial alignment hypothesis**. A model’s knowledge and capabilities are learned almost entirely during pretraining, and the later tuning only teaches it which format to use when answering.

That said, the answer key teaches a lot. Almost everything a model knows it learns in pretraining, and the later stages mostly serve to bring it to the surface. I like to think of it in two words, reach and control. Pretraining provides the reach, that is, what the model can eventually solve. Post-training provides the control, which is knowing when to keep working on a problem and when to stop. LIMA showed this with a very clean experiment. A thousand well-chosen examples were enough to fine-tune a pretrained model, with no reinforcement learning and no human preferences, and its answers turned out to be as good as or better than GPT-4’s in 43% of cases (Zhou et al., 2023). And in the other direction, when you try to teach it new facts during fine-tuning, the model learns them much more slowly than the ones it already knew and, once it finally learns them, it hallucinates more (Gekhman et al., 2024).

The knowledge, then, is there from pretraining. What is missing is a way of using it on problems that need many steps. And if many steps are needed, the solution seems obvious. Let the model take them one at a time, in writing.

## Chain of thought and post-training

> There was a precedent. In 2021, a model trained to write its intermediate calculations on a “scratchpad” was already learning to do long addition and to execute programs (Nye et al., 2021).

Then came “reasoning”. In 2022, a Google Brain team, with Jason Wei as first author and Denny Zhou leading it, published the paper that gave chain of thought (CoT) its name (Wei et al., 2022). The idea is very simple and very powerful. If, instead of asking the model for the answer directly, you show it a few examples solved step by step, the model also solves the problem step by step, and gets it right far more often. Shortly afterwards it turned out that even the examples were unnecessary. Simply adding “Let’s think step by step” to the question was enough for an OpenAI model to go from 17.7% to 78.7% accuracy on a set of arithmetic problems (Kojima et al., 2022). From then on we began teaching models to tackle problems bit by bit (Figure 2).

> **Figure 2.** The example from Figure 1 of Wei et al. (2022). Each token the model writes is one pass through the network, with the same layers. Step by step, the model makes many more passes before answering, and each intermediate result stays written down for the passes that follow. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-2)*

> Two more papers along the same lines. When a model is taught two made-up facts separately, it fails to combine them internally, without writing out the intermediate step (Balesni et al., 2024). And reasoning step by step helps precisely because training data is local. The model has seen neighbouring pieces together, but hardly ever the two ends of the chain, and chaining together what it has seen is more reliable than guessing the result directly (Prystawski et al., 2023).

Why it works has to do with what came before. Each token written is one more pass through the network, and the intermediate results stay in the text, where later passes can read them. This is the other half of that theoretical result, because with a chain of thought a fixed-size transformer can indeed solve the problems it could not solve in one go (Feng et al., 2023). But I think there is something more. Writing out the steps forces the model to build the solution piece by piece, from the ground up, instead of jumping to a remembered answer, and there is work pointing in that direction. Models answer each sub-question well on its own and still fail the question that combines them, a gap that does not close as the model gets bigger but does close when you ask it to pose the sub-questions to itself out loud (Press et al., 2022). That said, CoT did not work with just any model. In the original paper it only improved results from around 100 billion parameters upwards, and small models wrote fluent chains with no logic to them (Wei et al., 2022). It is a hint that the chain organises knowledge that already has to be there.

CoT has many advantages and few drawbacks. It is cheap, it does not require changing the model and it lets you see where the reasoning is going, although it does not always reflect what the model computes internally (Chen et al., 2025). Today it is present, in one form or another, in every frontier model. With this tool in the box we could dream of solving more complex problems, but one piece was missing. How could we get models to learn *on their own* to solve problems with CoT?

> In 2022, STaR was already generating rationales, keeping the ones that reached the right answer and training the model on them again (Zelikman et al., 2022). Today this kind of training is called RL with verifiable rewards, or RLVR (Lambert et al., 2024).

The answer was reinforcement learning (RL). Given a verifiable problem, the model proposes several solutions, we check which ones reach the correct answer and the model is adjusted so that the ones that worked become more likely. It is an elegant idea, and not a new one, but from 2024 onwards it reached an enormous scale. OpenAI unveiled it with o1 (OpenAI, 2024) and DeepSeek published the details with R1 (DeepSeek-AI, 2025), trained with GRPO, an algorithm that compares each attempt with the average of its group (Shao et al., 2024). Figure 3 reproduces it in miniature.

> **Figure 3.** RL with verifiable rewards, in miniature. At each step, the model attempts the problem eight times, each attempt is checked, and the strategies of the attempts that beat the group average are reinforced. Train for several steps, and start from scratch more than once too. Accuracy goes up, diversity collapses and, sometimes, the model settles on strategy B even though A is better. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-3)*

RL did not just teach models to work more autonomously. It also opened up a new axis for scaling. Until then, improving a model meant making it bigger or giving it more data, and now you can also let it think longer before answering. CoT gave us the tool for scaling in time, and RL the method for making the most of it. During the training of R1-Zero, the responses grow longer on their own. Nobody asks it to think more, but thinking more earns more reward (DeepSeek-AI, 2025). This is known as test-time compute, and it works. For the same total compute, a small model that thinks longer can outperform one fourteen times larger, provided the small one already gets the problem right now and then (Snell et al., 2025).

> On the credit for Navier–Stokes, Tristan Buckmaster, a mathematician at New York University, maintains that OpenAI followed a line of attack he himself was developing, and OpenAI denies it (Kahn, 2026 · Stan, 2026).

So we began devoting enormous amounts of compute to very hard problems, and in July 2025 a version of Gemini officially reached gold-medal level at the International Mathematical Olympiad (Luong & Lockhart, 2025). In September 2026, OpenAI went much further. A system of around 10,000 agents, costing millions of dollars in compute, produced a proof that the three-dimensional Navier–Stokes equations can develop a finite-time singularity when a smooth external force acts on the fluid (OpenAI, 2026a · Kahn, 2026). This is one of the alternatives allowed by the official statement of the Millennium Prize Problem, so, if it is confirmed, it would be the first of those problems to be solved by a machine. The proof runs to 166 pages and comes with a formalisation in Lean (OpenAI, 2026b), but the community is still reviewing it and there is an open dispute over the credit. The more time (and therefore money) we give the machine, the *better*.

The story, however, is not as rosy as it seems. RL has side effects.

-   **It makes the model repeat what has already worked for it.** Rewarded solutions become ever more likely and the rest disappear, so the diversity of answers collapses and the model explores less and less (Cui et al., 2025). This happens because each reinforcement makes what already came up often even more likely, while what hardly ever comes up never gets reinforced. It is like someone who finds a restaurant they like and never tries another one. They will never know whether there was a better one two streets away.
-   **It depends on the model already having the knowledge needed** (the *features*) to solve the problem. With bare right-or-wrong rewards, we are not going to solve a problem for which the model is not at least minimally prepared. This is not a limit of RL as such, but of how we use it. With the same training, a model that comes with habits such as checking its calculations or backtracking already built in improves a lot, and one that lacks them barely moves (Gandhi et al., 2025). In controlled experiments, RL only genuinely expands what the model can solve when it is trained at the edge of its competence (Zhang et al., 2025). These are problems that are hard but not out of reach, which the model still solves every now and then. In that study, for example, they are problems somewhat longer than those in pretraining, which the base model still gets right in at least one of 128 attempts.
-   **Solutions can add little and be extremely long.** Even when the model solves very hard problems, its reasoning can be so long as to be unintelligible. R1-Zero mixed languages and was hard to read (DeepSeek-AI, 2025), and optimising for accuracy alone makes solutions harder for a person to check (Kirchner et al., 2024). What is more, these models overthink even trivial problems, such as adding 2 + 3 (Chen et al., 2024). It is the consequence of pushing test-time compute to the limit.

To understand where these limits come from, it helps me to go back to the Lego. Pretraining fills the box with pieces, which are facts, operations and ways of reasoning. Chain of thought lets you assemble them one at a time, and RL is the practice that makes the assembly succeed more often. But if a piece is missing from the box, no amount of practice will help (Figure 4).

> **Figure 4.** Building with pieces. Each piece placed is a step in the chain of thought, and it can fall off. Without chain of thought, everything has to be placed in one go. Practice with RL makes each piece fall off less often, but no adjustment puts into the box the piece that “Something new” is missing. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-4)*

The first two effects show up clearly in another toy (Figures 5 and 6). Imagine two hundred problems and, for each one, the probability that the model gets it right in a single attempt. GRPO attempts each problem $G$ times, gives each attempt $i$ a reward $r_i$ (1 if it is right and 0 if it is wrong) and compares it with the average of its group. That comparison is the attempt’s advantage,

$$
A_i = \frac{r_i - \operatorname{mean}(r_1, \dots, r_G)}{\operatorname{std}(r_1, \dots, r_G)},
$$

and the model raises the probability of the attempts with a positive advantage and lowers that of the ones with a negative advantage (Shao et al., 2024). The problem lies in the numerator. If all the attempts are right, or all of them are wrong, every $r_i$ equals the mean, so all the advantages are zero and there is nothing to reinforce (the denominator is zero too, and in practice a very small number is added to it to avoid dividing by zero). Learning only happens in a band of intermediate difficulty. In fact, DAPO simply discards the problems that are always solved or always failed, because they contribute nothing (Yu et al., 2025).

> **Figure 5.** Each dot is a problem, placed according to the probability that the model gets it right in one attempt. The curve is the learning signal, the probability that the attempts in a step include both successes and failures. As you move the RL steps, whatever lies inside the band rises towards 100%, and whatever is almost never solved gives no signal and sinks towards 0%. With more attempts per problem, the band widens. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-5)*

The result looks a lot like a vote. If you ask a model for sixteen answers and keep the most frequent one (majority voting), you get it right considerably more often than with a single answer (Wang et al., 2023). RL achieves something similar without voting. It sharpens the distribution towards the correct answers that already came up often and lets the ones that hardly ever came up fall away. DeepSeekMath measured this. After RL, majority-vote accuracy improved, but the probability of getting it right at least once in many attempts did not. According to its authors, RL makes the distribution of answers more robust but does not improve the underlying capabilities (Shao et al., 2024). It redistributes the answers that were already there, but does not create new ones. So much so that, without any training at all, sampling the base model in a sharper way matches RL on many tasks, and without losing diversity (Karan & Du, 2025).

> There is no complete consensus. With much longer RL training and more varied tasks, ProRL finds problems that the base model did not solve even with 256 attempts. Even so, those gains depend on how competent the base model already was at each task (Liu et al., 2025).

The second effect shows up in *pass@k*, the probability of getting it right at least once in *k* attempts. At *k* = 1, the RL-trained model wins clearly. But if both are allowed hundreds of attempts, the base model ends up solving more problems (Yue et al., 2025). RL does not expand what the model can solve. It concentrates probability on what it could already solve and, along the way, loses some of the rest. The same authors find that distilling from a better model, by contrast, can bring in new reasoning patterns, something RL, as we use it today, does not achieve.

> **Figure 6.** pass@k for the base model and for the RL-trained one, with the same two hundred problems as in Figure 5. Move the pointer over the chart to read the values. The shape of the curves is the one Yue et al. (2025) measure in real models, but the numbers are not. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-6)*

> A word of caution about the toy. It assumes that what RL does with a problem depends only on its initial probability of success, and that is not the case. Moreover, many of the published pass@k crossovers are not statistically sound (Yang et al., 2026).

> There is work pointing both ways on this hypothesis. In favour, today’s RL stays within what the base model could already generate (Wu et al., 2025), and almost all of its effect is concentrated in a few decisions where the model was hesitating between options it already knew (Akgül et al., 2026). Against, with long training runs, hints from a teacher or partial rewards, RL solves problems the base model did not solve (Liu et al., 2025 · Cai et al., 2026 · Sun et al., 2025). And on ideas themselves, more than a hundred researchers judged an LLM’s research ideas more novel than those of human experts (Si et al., 2024), but when put into practice they lost much more value than the human ones (Si et al., 2025).

This is, to my mind, the best answer we have to the third question from the start. A reasoning model is very good at combining what it already knows, and that sometimes looks like creativity. RL does learn to chain more pieces than it saw in training. With compositions of two functions, a model goes from solving 5% to 30% of the chains of three that it has never seen, something supervised fine-tuning on the same data does not achieve (Yuan et al., 2025). But this is the simplest form of composition, repeating the same step on the previous result, and a pretrained model has seen a huge amount of code with nested calls. Even with a model trained from scratch, what RL extends is the depth of the chain, not the kind of composition (Zhang et al., 2025). Proposing a new method often requires a piece that was not in the data, or a way of combining pieces that is not just repetition. This last point is no longer a result but my hypothesis.

Even so, in my view, these models brought the first truly big results. At times they seem intelligent. But they were still far from working well in real situations, even in the areas where they shine most, such as programming or maths. Solving a closed exercise is not the same as working for hours on a real project.

The next axis for scaling was still RL, but this time giving the model real tools it could use inside a controlled environment. Yue et al. (2025) themselves point in this direction. To go beyond the base model, they propose continuing to scale RL and having the model interact with an environment over multiple turns.

## RL environments and agents, towards *world models*?

An RL environment is a small world with rules in which the model can act. At each step, the model, which we now call an *agent*, sees the state (a code repository, a terminal, a web page), chooses an action (read a file, run a command, edit a line) and the environment responds with what happened. Reasoning and acting alternate. The agent thinks, acts, looks at the result and thinks again (Yao et al., 2023). At the end of the episode, a verifier decides whether the task has been solved, for example by running some tests. If they pass, the reward is 1, and if not, 0. Training is the same RL as before, but on a grand scale, with thousands of episodes, each with tens or hundreds of actions, in which the trajectories that end well are reinforced. For programming there are, for example, environments built from real GitHub issues. The agent has to fix the bug and the project’s own tests act as the verifier (Jimenez et al., 2024 · Pan et al., 2025).

> **Figure 7.** An episode in a programming environment, step by step. Compare the three trajectories, all three of which can end with a reward of 1. Turn on the hidden tests to see which one stops collecting it. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-7)*

> From video, Genie learns which actions can be taken without anyone labelling them and generates worlds that can be controlled frame by frame (Bruce et al., 2024). V-JEPA 2 is pretrained on more than a million hours of video and, with under 62 hours of video of a robot, learns to predict what will happen if it acts, well enough to plan with it (Assran et al., 2025). And Dreamer 4 obtains diamonds in Minecraft by training only inside a learned world model, without touching the game (Hafner et al., 2025).

This is where the idea of *world models* comes in, an internal model of how the world works that lets you foresee what will happen if you do something without having to do it (Ha & Schmidhuber, 2018). My intuition is that giving a model actions pushes it to build one, and there is work pointing in that direction. To predict what a command will return, the agent needs to know what state the system is in, and that state changes with everything it does. Meta is testing this at scale. It trained a 32-billion-parameter model on observation-action trajectories from a Python interpreter and Docker environments, precisely so that it would learn a world model of code (FAIR CodeGen team, 2025). Something similar happens with images and video (Bruce et al., 2024 · Assran et al., 2025 · Hafner et al., 2025). And something like this also happens inside language models. A transformer trained only to predict legal Othello moves ends up representing the state of the board internally, without anyone having taught it to (Li et al., 2023). But there are also reasons for caution. Models that give almost perfect directions for getting around New York have an incoherent map inside, and fail as soon as the task changes slightly, for example with a detour (Vafa et al., 2024). Something similar happens in driving. DriveBench tests twelve vision-language models with more than 20,000 questions about real traffic scenes, and finds that they often answer convincingly even when the image is degraded or not there at all, drawing on general knowledge and cues from the text instead of looking at the scene (Xie et al., 2025). Getting the next action right does not guarantee understanding the world.

And scaling is happening in every direction, with coding agents that work for hours on real repositories, agents that browse the web, models that generate interactive 3D worlds in real time in which to train other agents (Parker-Holder & Fruchter, 2025), or world models for driving, which generate realistic traffic scenes with which to train and test self-driving cars (Hu et al., 2023). The underlying bet is summed up by Silver and Sutton (2025). After the era of human data would come the “era of experience”, in which agents would learn mainly from what they do, and not from what we have written. Even so, learning from experience does not free agents from what came before. You can only reinforce what the agent already manages at least occasionally, and that depends largely on the skills it brings from pretraining.

### The problems

> Baker et al. (2025) found something unsettling. If the model is penalised when its chain of thought reveals that it is about to cheat, it does not stop cheating, but learns to hide it instead.

The problems are those of RL, plus a few more. The first is *reward hacking*, which is when the agent finds a way to collect the reward without doing the task (Skalse et al., 2022). In programming environments this is not a theoretical risk. It has been observed in frontier reasoning models such as o3-mini, which, instead of fixing the code, looked for ways to get the checks to pass (Baker et al., 2025). It is the cheating trajectory in Figure 7.

The second is *credit assignment*. The reward arrives at the end, after dozens of actions, and does not say which of them contributed to the success and which were superfluous. Rewarding each step would help, because supervising the process works better than supervising only the outcome (Lightman et al., 2024), but it requires knowing what a good step is. This is so hard that the DeepSeek team tried it and dropped it for R1. There was no clear way to define a step or to tell whether it was correct, and the model ended up learning to fool the evaluator (DeepSeek-AI, 2025). There is a third way, which is to try things out. From each intermediate step, several continuations are rolled out and we measure how many of them end well, and the credit for a step is how much it raises that proportion (Kazemnejad et al., 2024). No step judge is needed, only the final verifier and a great deal more compute. Figure 8 tells the story with a football move.

> **Figure 8.** A move that ends in a goal. With only the final result, every action receives the same credit. A commentator who rates each action needs to know what a good move is, and can get it wrong. Replaying the rest of the move many times from each moment, and counting how many replays end in a goal, finds the actions that mattered, but with few replays it is noisy and with many it is expensive. *(interactive figure, on the web: https://latent-universe.pages.dev/en/el-camino-de-las-ias-hacia-la-inteligencia/#el-camino-de-las-ias-hacia-la-inteligencia-fig-8)*

> There are attempts to train without an automatic verifier, with rubrics written by people, with the model’s own confidence as the reward or with votes among its answers, but none of them has won out yet (Gunjal et al., 2025 · Zhao et al., 2025 · Zuo et al., 2025).

The third is that we depend on verifiable environments. We can only train this way on what can be checked automatically, such as a test that passes, a numerical answer or a game won. It sounds like a technical detail, but it decides the direction in which AI advances. Models improve very quickly precisely where there are cheap checks, such as maths, programming or games, and much more slowly where there are none, such as writing well, doing research or negotiating. The result is a very uneven intelligence, brilliant at some tasks and clumsy at others that seem just as hard to us. Moreover, when the check does not fully capture what we want, the model learns to satisfy the check rather than the goal, which is another form of reward hacking. That is why building good verifiers and good environments has become one of the most important jobs in the field.

In my view, these last two problems are the same problem seen from two sides. If we knew how to verify every intermediate step, we would have partial rewards. Since we only know how to verify the final result, and only on some tasks, we are left with sparse rewards, or with paying in compute for what we cannot verify, and with tasks whose outcome can be checked. Exactly the opposite of what is needed to propose new ideas, which do not come with a test to check them.

## What remains open

If I had to sum up the path in one sentence, I would say that first we taught models to know (pretraining), then to chain together what they know (CoT and RL), and now we are trying to get them to learn by acting (environments). Each leap solved the problem left by the one before and exposed a new one. The one in front of us, how to reward what cannot be verified, is to my mind the one that separates a model that solves problems from one that proposes ideas.

I will develop this in the next post, *What post-training doesn’t teach*, because I think everything above fits into a single inequality. If the model attempts each problem $G$ times and only learns when at least one of those attempts is right, it can only train on what it already gets right roughly once in every $G$ attempts, that is, $p \gtrsim 1/G$. Below that there is nothing to reinforce (this is the left edge of the learning zone in Figure 5), and the problem, instead of being learned, is lost (Zhou, 2026). Almost everything that manages to go beyond the base model today is, at heart, a way of getting around that inequality. Either you raise $G$, which costs money (Hu et al., 2025), or you raise $p$ without paying for it in attempts, for example with a teacher who gives hints at the edge of what the model can do (Cai et al., 2026), with a partial reward that pays for passing some tests before demanding all of them (Sun et al., 2025) or with mid-training that prepares the model before RL (Zhang et al., 2025). And what is learned inside the band can reach a little further. A model pretrained from scratch on problems of up to 10 operations, then given RL on problems of 11 to 14, also improves on those of 15 to 20, which it almost never got right before (Zhang et al., 2025). Further, yes. Different, not yet.

Seen this way, the Navier–Stokes proof does not contradict anything I have said. It is the same arithmetic pushed to the extreme. AlphaEvolve had already shown this on a smaller scale. An LLM proposing changes to a program, automatic evaluators and evolutionary search found how to multiply two 4 × 4 complex-valued matrices with 48 multiplications, the first improvement in 56 years over Strassen’s algorithm in that setting (Novikov et al., 2025). In Navier–Stokes there were ten thousand agents in parallel, a result that can be checked mechanically and, if Buckmaster is right, a line of attack that already existed in human work. If I am right, machine creativity will not be decided in the RL algorithm but in two far less glamorous places, how much we can pay for each attempt and what we know how to verify. And almost nothing of what we call a good idea comes with a verifier.

This article was originally written in Spanish, with the assistance of AI models (Claude Opus 5.5, by Anthropic). This English version was translated automatically by Claude Opus 5.5.

[^1]: The figures in this post are toy models. Their numbers serve to show the mechanism and do not reproduce any experiment.

## References

1.  Akgül, Ö. F., Kannan, R., Neiswanger, W. & Prasanna, V. (2026). Rethinking RL for LLM Reasoning: It’s Sparse Policy Selection, Not Capability Learning. arXiv:2605.06241. [https://arxiv.org/abs/2605.06241](https://arxiv.org/abs/2605.06241)
2.  Assran, M., Bardes, A., Fan, D., Garrido, Q., Howes, R. et al. (2025). V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv:2506.09985. [https://arxiv.org/abs/2506.09985](https://arxiv.org/abs/2506.09985)
3.  Bachmann, G. & Nagarajan, V. (2024). The Pitfalls of Next-Token Prediction. In *International Conference on Machine Learning*. [https://arxiv.org/abs/2403.06963](https://arxiv.org/abs/2403.06963)
4.  Baker, B., Huizinga, J., Gao, L., Dou, Z., Guan, M. Y., Madry, A. et al. (2025). Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation. arXiv:2503.11926. [https://arxiv.org/abs/2503.11926](https://arxiv.org/abs/2503.11926)
5.  Balesni, M., Korbak, T. & Evans, O. (2024). Lessons from Studying Two-Hop Latent Reasoning. arXiv:2411.16353. [https://arxiv.org/abs/2411.16353](https://arxiv.org/abs/2411.16353)
6.  Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P. et al. (2020). Language Models are Few-Shot Learners. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2005.14165](https://arxiv.org/abs/2005.14165)
7.  Bruce, J., Dennis, M., Edwards, A., Parker-Holder, J., Shi, Y., Hughes, E. et al. (2024). Genie: Generative Interactive Environments. arXiv:2402.15391. [https://arxiv.org/abs/2402.15391](https://arxiv.org/abs/2402.15391)
8.  Cai, P., Fang, T., Li, X., Zeng, Q., Li, G. & Chen, J. (2026). Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model. arXiv:2606.22317. [https://arxiv.org/abs/2606.22317](https://arxiv.org/abs/2606.22317)
9.  Chen, X., Xu, J., Liang, T., He, Z., Pang, J., Yu, D. et al. (2024). Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs. arXiv:2412.21187. [https://arxiv.org/abs/2412.21187](https://arxiv.org/abs/2412.21187)
10.  Chen, Y., Benton, J., Radhakrishnan, A., Uesato, J., Denison, C., Schulman, J. et al. (2025). Reasoning Models Don’t Always Say What They Think. arXiv:2505.05410. [https://arxiv.org/abs/2505.05410](https://arxiv.org/abs/2505.05410)
11.  Cui, G., Zhang, Y., Chen, J., Yuan, L., Wang, Z., Zuo, Y. et al. (2025). The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models. arXiv:2505.22617. [https://arxiv.org/abs/2505.22617](https://arxiv.org/abs/2505.22617)
12.  DeepSeek-AI (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. *Nature*, *645*, 633–638. [https://arxiv.org/abs/2501.12948](https://arxiv.org/abs/2501.12948)
13.  Dziri, N., Lu, X., Sclar, M., Li, X. L., Jiang, L., Lin, B. Y. et al. (2023). Faith and Fate: Limits of Transformers on Compositionality. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2305.18654](https://arxiv.org/abs/2305.18654)
14.  FAIR CodeGen team (2025). CWM: An Open-Weights LLM for Research on Code Generation with World Models. arXiv:2510.02387. [https://arxiv.org/abs/2510.02387](https://arxiv.org/abs/2510.02387)
15.  Feng, G., Zhang, B., Gu, Y., Ye, H., He, D. & Wang, L. (2023). Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2305.15408](https://arxiv.org/abs/2305.15408)
16.  Gandhi, K., Chakravarthy, A., Singh, A., Lile, N. & Goodman, N. D. (2025). Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs. arXiv:2503.01307. [https://arxiv.org/abs/2503.01307](https://arxiv.org/abs/2503.01307)
17.  Gekhman, Z., Yona, G., Aharoni, R., Eyal, M., Feder, A., Reichart, R. & Herzig, J. (2024). Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? In *Proceedings of the Conference on Empirical Methods in Natural Language Processing*. [https://arxiv.org/abs/2405.05904](https://arxiv.org/abs/2405.05904)
18.  Gunjal, A., Wang, A., Lau, E., Nath, V., He, Y., Liu, B. et al. (2025). Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains. arXiv:2507.17746. [https://arxiv.org/abs/2507.17746](https://arxiv.org/abs/2507.17746)
19.  Ha, D. & Schmidhuber, J. (2018). World Models. arXiv:1803.10122. [https://arxiv.org/abs/1803.10122](https://arxiv.org/abs/1803.10122)
20.  Hafner, D., Yan, W. & Lillicrap, T. (2025). Training Agents Inside of Scalable World Models. arXiv:2509.24527. [https://arxiv.org/abs/2509.24527](https://arxiv.org/abs/2509.24527)
21.  Hu, A., Russell, L., Yeo, H., Murez, Z., Fedoseev, G., Kendall, A. et al. (2023). GAIA-1: A Generative World Model for Autonomous Driving. arXiv:2309.17080. [https://arxiv.org/abs/2309.17080](https://arxiv.org/abs/2309.17080)
22.  Hu, J., Liu, M., Lu, X., Wu, F., Harchaoui, Z. & Diao, S. (2025). BroRL: Scaling Reinforcement Learning via Broadened Exploration. arXiv:2510.01180. [https://arxiv.org/abs/2510.01180](https://arxiv.org/abs/2510.01180)
23.  Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O. & Narasimhan, K. (2024). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? In *International Conference on Learning Representations*. [https://arxiv.org/abs/2310.06770](https://arxiv.org/abs/2310.06770)
24.  Kahn, J. (2026). OpenAI says it cracked one of math’s grand challenges. But there are troubling questions about how they did it—and what it means for us all. *Fortune*. [https://fortune.com/2026/09/08/openai-says-it-cracked-navier-stokes-math-grand-challenge-buckmaster-accusation-cheating-intimidation-tao-lament/](https://fortune.com/2026/09/08/openai-says-it-cracked-navier-stokes-math-grand-challenge-buckmaster-accusation-cheating-intimidation-tao-lament/)
25.  Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664. [https://arxiv.org/abs/2509.04664](https://arxiv.org/abs/2509.04664)
26.  Karan, A. & Du, Y. (2025). Reasoning with Sampling: Your Base Model is Smarter Than You Think. arXiv:2510.14901. [https://arxiv.org/abs/2510.14901](https://arxiv.org/abs/2510.14901)
27.  Kazemnejad, A., Aghajohari, M., Portelance, E., Sordoni, A., Reddy, S., Courville, A. et al. (2024). VinePPO: Refining Credit Assignment in RL Training of LLMs. arXiv:2410.01679. [https://arxiv.org/abs/2410.01679](https://arxiv.org/abs/2410.01679)
28.  Kirchner, J. H., Chen, Y., Edwards, H., Leike, J., McAleese, N. & Burda, Y. (2024). Prover-Verifier Games improve legibility of LLM outputs. arXiv:2407.13692. [https://arxiv.org/abs/2407.13692](https://arxiv.org/abs/2407.13692)
29.  Kojima, T., Gu, S. S., Reid, M., Matsuo, Y. & Iwasawa, Y. (2022). Large Language Models are Zero-Shot Reasoners. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2205.11916](https://arxiv.org/abs/2205.11916)
30.  Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F. et al. (2024). Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv:2411.15124. [https://arxiv.org/abs/2411.15124](https://arxiv.org/abs/2411.15124)
31.  Li, K., Hopkins, A. K., Bau, D., Viégas, F., Pfister, H. & Wattenberg, M. (2023). Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. In *International Conference on Learning Representations*. [https://arxiv.org/abs/2210.13382](https://arxiv.org/abs/2210.13382)
32.  Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T. et al. (2024). Let’s Verify Step by Step. In *International Conference on Learning Representations*. [https://arxiv.org/abs/2305.20050](https://arxiv.org/abs/2305.20050)
33.  Liu, M., Diao, S., Lu, X., Hu, J., Dong, X., Choi, Y. et al. (2025). ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models. arXiv:2505.24864. [https://arxiv.org/abs/2505.24864](https://arxiv.org/abs/2505.24864)
34.  Luong, T. & Lockhart, E. (2025). Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad. *Google DeepMind (blog)*. [https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/](https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/)
35.  Novikov, A., Vũ, N., Eisenberger, M., Dupont, E., Huang, P.-S., Wagner, A. Z. et al. (2025). AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv:2506.13131. [https://arxiv.org/abs/2506.13131](https://arxiv.org/abs/2506.13131)
36.  Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D. et al. (2021). Show Your Work: Scratchpads for Intermediate Computation with Language Models. arXiv:2112.00114. [https://arxiv.org/abs/2112.00114](https://arxiv.org/abs/2112.00114)
37.  OpenAI (2024). Learning to Reason with LLMs. *OpenAI (blog)*. [https://openai.com/index/learning-to-reason-with-llms/](https://openai.com/index/learning-to-reason-with-llms/)
38.  OpenAI (2026a). Finite Time Blowup for Navier–Stokes. *OpenAI*. [https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8a9a/navier-stokes.pdf](https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8a9a/navier-stokes.pdf)
39.  OpenAI (2026b). NavierStokesAndEuler: Lean certificates accompanying Navier–Stokes and Euler results. *GitHub*. [https://github.com/openai/NavierStokesAndEuler](https://github.com/openai/NavierStokesAndEuler)
40.  Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P. et al. (2022). Training language models to follow instructions with human feedback. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2203.02155](https://arxiv.org/abs/2203.02155)
41.  Pan, J., Wang, X., Neubig, G., Jaitly, N., Ji, H., Suhr, A. & Zhang, Y. (2025). Training Software Engineering Agents and Verifiers with SWE-Gym. In *International Conference on Machine Learning*. [https://arxiv.org/abs/2412.21139](https://arxiv.org/abs/2412.21139)
42.  Parker-Holder, J. & Fruchter, S. (2025). Genie 3: A new frontier for world models. *Google DeepMind (blog)*. [https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/](https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/)
43.  Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A. & Lewis, M. (2022). Measuring and Narrowing the Compositionality Gap in Language Models. arXiv:2210.03350. [https://arxiv.org/abs/2210.03350](https://arxiv.org/abs/2210.03350)
44.  Prystawski, B., Li, M. Y. & Goodman, N. D. (2023). Why think step by step? Reasoning emerges from the locality of experience. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2304.03843](https://arxiv.org/abs/2304.03843)
45.  Radford, A., Wu, J., Child, R., Luan, D., Amodei, D. & Sutskever, I. (2019). Language Models are Unsupervised Multitask Learners. *OpenAI*. [https://cdn.openai.com/better-language-models/language\_models\_are\_unsupervised\_multitask\_learners.pdf](https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf)
46.  Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X. et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300. [https://arxiv.org/abs/2402.03300](https://arxiv.org/abs/2402.03300)
47.  Si, C., Yang, D. & Hashimoto, T. (2024). Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers. arXiv:2409.04109. [https://arxiv.org/abs/2409.04109](https://arxiv.org/abs/2409.04109)
48.  Si, C., Hashimoto, T. & Yang, D. (2025). The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas. arXiv:2506.20803. [https://arxiv.org/abs/2506.20803](https://arxiv.org/abs/2506.20803)
49.  Silver, D. & Sutton, R. S. (2025). Welcome to the Era of Experience. *Preprint, capítulo de «Designing an Intelligence» (MIT Press)*. [https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf](https://storage.googleapis.com/deepmind-media/Era-of-Experience%20/The%20Era%20of%20Experience%20Paper.pdf)
50.  Skalse, J., Howe, N. H. R., Krasheninnikov, D. & Krueger, D. (2022). Defining and Characterizing Reward Hacking. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2209.13085](https://arxiv.org/abs/2209.13085)
51.  Snell, C., Lee, J., Xu, K. & Kumar, A. (2025). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. In *International Conference on Learning Representations*. [https://arxiv.org/abs/2408.03314](https://arxiv.org/abs/2408.03314)
52.  Stan, A. M. (2026). The mathematicians published machine-checkable proofs. OpenAI announced its result on a call with reporters. *The Next Web*. [https://thenextweb.com/news/openai-navier-stokes-claim-verification-credit](https://thenextweb.com/news/openai-navier-stokes-claim-verification-credit)
53.  Sun, Y., Cao, Y., Huang, P., Bai, H., Hajishirzi, H., Dziri, N. & Song, D. (2025). RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs? arXiv:2509.21016. [https://arxiv.org/abs/2509.21016](https://arxiv.org/abs/2509.21016)
54.  Vafa, K., Chen, J. Y., Rambachan, A., Kleinberg, J. & Mullainathan, S. (2024). Evaluating the World Model Implicit in a Generative Model. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2406.03689](https://arxiv.org/abs/2406.03689)
55.  Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S. et al. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. In *International Conference on Learning Representations*. [https://arxiv.org/abs/2203.11171](https://arxiv.org/abs/2203.11171)
56.  Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2201.11903](https://arxiv.org/abs/2201.11903)
57.  Wu, F., Xuan, W., Lu, X., Liu, M., Dong, Y., Harchaoui, Z. & Choi, Y. (2025). The Invisible Leash: Why RLVR May or May Not Escape Its Origin. arXiv:2507.14843. [https://arxiv.org/abs/2507.14843](https://arxiv.org/abs/2507.14843)
58.  Xie, S., Kong, L., Dong, Y., Sima, C., Zhang, W., Chen, Q. A. et al. (2025). Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives. In *International Conference on Computer Vision*. [https://arxiv.org/abs/2501.04003](https://arxiv.org/abs/2501.04003)
59.  Yang, C., Zhang, X. & Chen, J. (2026). RLVR is a Kernel, Not a Function: Statistical Inference for pass@$k$ Crossovers. arXiv:2609.22547. [https://arxiv.org/abs/2609.22547](https://arxiv.org/abs/2609.22547)
60.  Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. In *International Conference on Learning Representations*. [https://arxiv.org/abs/2210.03629](https://arxiv.org/abs/2210.03629)
61.  Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y. et al. (2025). DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476. [https://arxiv.org/abs/2503.14476](https://arxiv.org/abs/2503.14476)
62.  Yuan, L., Chen, W., Zhang, Y., Cui, G., Wang, H., You, Z. et al. (2025). From $f(x)$ and $g(x)$ to $f(g(x))$: LLMs Learn New Skills in RL by Composing Old Ones. arXiv:2509.25123. [https://arxiv.org/abs/2509.25123](https://arxiv.org/abs/2509.25123)
63.  Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y. et al. (2025). Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2504.13837](https://arxiv.org/abs/2504.13837)
64.  Zelikman, E., Wu, Y., Mu, J. & Goodman, N. D. (2022). STaR: Bootstrapping Reasoning With Reasoning. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2203.14465](https://arxiv.org/abs/2203.14465)
65.  Zhang, C., Neubig, G. & Yue, X. (2025). On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models. arXiv:2512.07783. [https://arxiv.org/abs/2512.07783](https://arxiv.org/abs/2512.07783)
66.  Zhao, X., Kang, Z., Feng, A., Levine, S. & Song, D. (2025). Learning to Reason without External Rewards. arXiv:2505.19590. [https://arxiv.org/abs/2505.19590](https://arxiv.org/abs/2505.19590)
67.  Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y. et al. (2023). LIMA: Less Is More for Alignment. In *Advances in Neural Information Processing Systems*. [https://arxiv.org/abs/2305.11206](https://arxiv.org/abs/2305.11206)
68.  Zhou, T. (2026). When RLVR Shrinks the Reasoning Boundary: Diagnosing Pass@k Inversion. arXiv:2607.20543. [https://arxiv.org/abs/2607.20543](https://arxiv.org/abs/2607.20543)
69.  Zuo, Y., Zhang, K., Sheng, L., Qu, S., Cui, G., Zhu, X. et al. (2025). TTRL: Test-Time Reinforcement Learning. arXiv:2504.16084. [https://arxiv.org/abs/2504.16084](https://arxiv.org/abs/2504.16084)