Absolute Zero: Learning Without Any Data at All

Reinforcement learning with verifiable rewards — RLVR — was the engine behind the reasoning-model boom: train on outcome rewards from checkable answers, no human-written chain-of-thought needed. But even the "zero" variants, which skip distillation from stronger models, still depend on humans for the one thing nobody talks about: the questions. Tens of thousands of curated problem-answer pairs, hand-assembled per domain. The Absolute Zero paper asks what happens when you delete that too. One model, zero external data, learning to propose its own tasks and solve them — self-play, but the game is reasoning itself, and the opponent is the model's own ignorance.

The proposer-solver loop

A single language model plays two roles. As proposer π^propose, it samples a task τ conditioned on a seed variable z — in practice, a small sample of past task-answer pairs drawn from a continually updated task memory, so the curriculum evolves with the model's history. As solver π^solve, it attempts the task. A code executor serves as the environment: it validates proposed tasks into well-formed problems (x, y*) and verifies answers, providing the grounded feedback that keeps open-ended self-play from drifting into fantasy.

Two rewards drive learning. The learnability reward r^propose scores a proposed task by the expected improvement the model would get from training on it — tasks that are too easy or impossible score poorly; tasks at the frontier of capability score well. The solution reward r^solve is the familiar verifiable-correctness signal. A coefficient λ balances exploring learnable new tasks against getting better at solving. Both roles are trained jointly, end-to-end, with a new advantage estimator designed for the multitask structure — and the loop can in principle run forever, with the task distribution perpetually chasing the model's growing competence.

The concrete instantiation is elegant: tasks are built from program/input/output triplets, giving three complementary reasoning modes — deduction (given program and input, predict output), abduction (given program and output, infer input), and induction (given input-output pairs, infer the program). The code executor can validate all three, so one environment grounds a whole curriculum of reasoning types.

The proposer conditioning is a quiet but important detail: z is sampled from a continually updated memory of past task-answer pairs, so the curriculum is stateful — the proposer sees what the solver has already mastered and is pressured by the learnability reward to aim just beyond it. It's curriculum learning where the designer and the student share weights, and the "zone of proximal development" is operationalized as expected learning signal rather than intuition.

Zero data, state of the art

The headline: trained with no human-curated data whatsoever, the Absolute Zero Reasoner (AZR) achieved overall state-of-the-art among zero-setting models on coding and math reasoning — beating models trained on tens of thousands of in-domain human examples by 1.8 absolute points on average. The ablations are full of surprises:

• Cross-domain transfer is stronger for AZR than for RLVR. Expert code models trained with RLVR gain only ~0.65 points on math; AZR trained purely on self-proposed code tasks gained 10.9 (base) and 15.2 (coder variant) points on math. General reasoning, not domain skill, is what's being learned.
• Code priors amplify reasoning. The Qwen-Coder-7B base started 3.6 points behind Qwen-7B on math; after AZR training it finished 0.7 points ahead.
• Gains scale with size: +5.7, +10.2, +13.2 points at 3B, 7B, 14B.
• Behaviors emerge unprompted: step-by-step planning interleaved as code comments (ReAct-style), enumeration, trial-and-error — with abduction producing the longest traces, since the model brute-forces inputs until outputs match.

Absolute Zero's real claim isn't the benchmark number — it's that the curriculum can be self-generated. The proposer chasing learnability is the reasoning-model analog of AlphaZero's self-play: the training distribution perpetually re-targets the frontier of what the model can't yet do.

Limitations worth naming

The paper includes its own warning label, and it's worth quoting the spirit of it: the authors observed an "uh-oh moment" — the Llama-3.1-8B-based AZR occasionally producing concerning chains of thought — and flag the need for safety-aware training. A model optimizing its own curriculum with no human in the loop is exactly the setup where you want oversight, and this paper has none. More technically: the environment is code execution, which is beautifully verifiable but narrow — it's unclear how far "propose anything a Python interpreter can check" stretches toward genuinely open-ended reasoning. The learnability reward is a proxy for "useful to learn from," and proxies get gamed; a sufficiently clever proposer could learn to propose tasks that maximize measured learnability without building real capability. And "zero data" is doing some quiet work — the base models were pretrained on the internet, so this is zero curated data, not zero human knowledge.

Where it sits in the RSI arc

Absolute Zero is the paper that finally ports AlphaZero's structure to language models: self-play, no human supervision, grounded by an environment instead of game rules. Where AlphaZero's verifier was the rules of Go, AZR's is the code executor — and the proposer/solver split is the curriculum-learning machinery that replaces "play the latest version of yourself." It also answers, partially, the question STaR left open: STaR needed gold answers as its filter; AZR generates the questions and the filter (execution), needing neither. The remaining gap — and it's the one the safety section gestures at — is that AlphaZero's loop converged to superhuman play, while AZR's loop converges to better reasoning, and we have far weaker intuitions about what the fixed point of that recursion looks like, or whether we'd recognize it going wrong.

Paper: Absolute Zero: Reinforced Self-play Reasoning with Zero Data — Andrew Zhao et al., 2025.