Essays & notes
Writing
Research notes first — then everything else I’m thinking through.
Research
The benchmarkMy research area is continual learning — how models retain and adapt capabilities over time. Right now that means one question: which memories will an agent need in the future, and can we predict it? I’m working through it in the open.
Counterfactual credit assignment for memory admission
Forthcoming
Can we predict which memories an agent will need before it needs them? Notes from the agent-memory-bench build.
What the benchmark is teaching us
Forthcoming
Early results on which memory strategies win where — and how badly they break under poisoning.
Paper notes
Deep dives on the papers shaping how I think — recursive self-improvement, reinforcement learning, and continual learning. Each one is a few minutes on what the paper actually shows, and what it leaves open.
Recursive self-improvement
On-Policy Distillation: When the Teacher Has to Keep Up With the Student
Meta's fix for the frozen-teacher paradox: let the privileged teacher co-evolve with the student (DCE) while keeping revisions concise (SRCL) — +35.6 points over frozen self-distillation.
AIDE²: An AI Research Agent That Rewrites Its Own Code for 8 Days
An AI research agent rewriting its own harness code for 8 unsupervised days — 7 accepted improvements, matching two years of human R&D, with reward hacking falling as a side effect.
Absolute Zero: Learning Without Any Data at All
Zero curated data: a single model proposes its own reasoning tasks (scored by learnability) and solves them against a code executor — reaching SOTA among zero-setting reasoners.
Self-Developing: Inventing the Algorithm That Improves You
Recursion one level up: an LLM 'algorithm factory' invents model-merging algorithms as code, gets trained by DPO on their results — and beats human-designed merging by 4.3% on GSM8K.
Gödel Agent: Monkey-Patching Your Way to Recursive Self-Improvement
The pragmatic Gödel machine: an LLM agent that monkey-patches its own runtime code and recurses — updating both its policy and its learning algorithm.
Self-Rewarding Language Models: The Judge That Learns to Judge Better
Meta's iterative DPO loop where one model both follows instructions and judges them — and the judge gets better every round, beating GPT-4 0613 on AlpacaEval 2.0.
RLAIF: What Happens When the Labeler Is You
The controlled head-to-head: AI feedback matches human feedback (~50/50), wins on harmlessness — and works even when the labeler is the same checkpoint as the policy.
STaR: Teaching Yourself to Reason, Then Reasoning to Teach Yourself
The first LLM-era self-improvement loop: generate rationales, keep the ones that reach correct answers, retrain, repeat — matching a 30× larger model on CommonsenseQA.
AlphaZero: The Self-Play Proof That Tabula Rasa Scales
The existence proof: tabula rasa self-play to superhuman chess, shogi, and Go within 24 hours, searching 1,000× less than Stockfish — and winning.
The Gödel Machine: Self-Improvement With a Proof Attached
The 2003 blueprint for self-improvement with mathematical guarantees — a machine that rewrites any part of its own code only after proving the rewrite is useful.
Reinforcement learning
The Open Recipe for Reasoning RL
Reproducing R1-style results openly: four unglamorous fixes — decoupled clipping, dynamic sampling, token-level loss, length shaping — that made the difference.
What SFT Can't Learn
The controlled experiment the field needed: SFT clones the training distribution, RL optimizes toward outcomes — and only one of them generalizes.
A New Axis of Scaling
The independent confirmation of R1's paradigm: 128k-context RL, no MCTS or value functions, matching o1 — plus long2short distillation that actually works.
Reasoning Without Teachers
Pure RL, no supervised fine-tuning, no human reasoning traces — and reflection, verification, and the 'aha moment' emerged on their own.
Deleting the Critic
GRPO replaced PPO's value network with group-relative advantages — halving RL's memory cost and becoming the engine behind R1.
Skipping the Middleman in Preference Learning
DPO collapses the whole RLHF pipeline into a single classification loss — no reward model, no PPO — and asks what the RL step was ever doing.
Teaching AI to Criticize Itself
Replace human labelers with a written constitution and the model's own critiques: the birth of RLAIF and self-supervised alignment.
The Three-Step Recipe Behind ChatGPT
Demonstrate, rank, optimize: how a 1.3B model with human feedback beat a 175B GPT-3 — and turned RLHF into an industry.
The Thirty-Line Trick That Tamed Policy Gradients
How a clipped surrogate objective became the substrate under all of RLHF — and why every later method is a reaction to it.
Before ChatGPT, There Were Atari Preferences
The 2017 paper that invented RLHF: learn the reward from pairwise human preferences, then optimize it — with under 1% of interactions labeled.
Continual learning
Grow the Brain, Freeze the Past
Width upscaling with frozen pretrained weights and provable preservation: SCALE argues continual-learning progress now depends on scaling the right structure, not just more parameters.
Forget Less by Training Less
Full fine-tuning drops old knowledge 89%, LoRA 71% — but updating only TF-IDF-targeted memory slots drops just 11%: forgetting is a targeting problem, and sparsity in the right substrate beats low-rankness in the wrong one.
Merge Models by What They Represent
Instead of averaging weights and hoping, RECALL merges models by layer-wise representational similarity over clustered typical samples — preserving shallow generality, negotiating deep specialization.
Teaching Agents Without Breaking Them
With 103B tokens of agent trajectories and API docs mixed at a scaling-law 1:1:1 ratio, Hephaestus adds whole capability families — tool use, planning, feedback adaptation — without losing generality.
Forgetting Gets Worse With Scale
Across 1B–7B models, continual fine-tuning forgets more as models get bigger; decoder-only architectures and broad instruction tuning help — and sometimes forgetting debiases.
Measuring What Matters
GEM's algorithm projects gradients to protect past tasks, but its lasting gift is the measurement framework — accuracy, backward transfer, forward transfer — that the field still argues in.
Credit Along the Whole Trajectory
Instead of estimating importance at the finish line like EWC, Synaptic Intelligence keeps a per-synapse ledger during training — a path integral of gradient times movement, computed for free.
The Spring That Remembers
Inspired by dendritic spines that persist for months in mouse brains, EWC anchors each weight to its old value with a Fisher-information spring — stiff where it mattered, loose where it didn't.
Immunity by Construction
Freeze every old column, grow a new one per task, wire old features in laterally: the most honest solution to forgetting achieves immunity by construction — at linear cost.
The Paper That Named the Problem
In 1989, two psychologists showed that sequential training doesn't degrade old knowledge in neural networks — it annihilates it, and no hyperparameter tweak fixes the structural flaw.
Elsewhere
Philosophy, geopolitics, and everything else the technology keeps raising.
Who Gets the GPUs?
How access to scarce compute is becoming a political question.
Who Is Accountable When the Agent Acts?
As agents move from suggesting to acting, our frameworks for consent and liability strain. A taxonomy of agentic accountability grounded in real interface decisions.
Evaluation Harnesses Are an Ethics Problem
Safety isn't a post-training checkbox — it's a measurement problem. What belongs in an eval suite before you ship, and why the harness itself encodes values.
Latency Budgets for Real-Time ML Serving
Inference isn't just model quality — it's queues, batching, and tail latency. Notes on designing serving paths where milliseconds actually matter.
Why ML Pipelines Belong on the Stream
Batch training and online inference often meet at a Kafka topic. A field guide to language boundaries, backpressure, and keeping the model off the critical path until it needs to run.