Teaching Agents Without Breaking Them

Most continual learning research asks: how do we teach the model something new without breaking what it knows? Hephaestus asks a harder, more practical question: how do we teach a model an entirely new kind of capability — being an agent — when the pretraining corpus barely contains the relevant data? You can't instruction-tune your way to robust tool use if the base model has never seen 76,000 APIs. The answer the authors give is continual pre-training at serious scale: 103 billion tokens of agent-specific data, mixed carefully enough that general capabilities survive.

The problem: agents are data-starved at pretraining time

LLM-based agents typically get their agentic abilities from prompting tricks or fine-tuning. Both are brittle: prompting doesn't add capability, and fine-tuning on small agent datasets tends to produce narrow specialists that lose generality. The root cause is distributional — standard pretraining corpora contain some API docs and code, but almost nothing resembling the actual trajectories of an agent reasoning, calling tools, observing results, and replanning. Hephaestus-Forge, the corpus introduced here, is the first large-scale attempt to fix the data rather than the algorithm: 103B tokens spanning tool documentation (knowledge of what APIs exist and do) and function-calling trajectories (the reasoning patterns of using them), covering 76,537 APIs.

The corpus design reflects a theory of what "agent capability" decomposes into: API function calling, intrinsic reasoning and planning, and adapting to environmental feedback. Documentation teaches the first; trajectories teach the second and third. It's a curriculum intuition implemented as data mixture rather than training stages.

The method: continual pre-training with a scaling-law recipe

Hephaestus takes existing open models — LLaMA-3-8B, Mistral-7B, Qwen2-7B — and continually pre-trains them on Hephaestus-Forge. Two methodological choices carry the weight:

• Data mixing from scaling laws. Rather than guessing the ratio of agent data to general replay data, the authors fit scaling laws to identify the optimal mixture, landing on a 1:1:1 ratio across their data categories. This is the anti-forgetting mechanism: enough general data in the mix that the model's existing capabilities aren't washed out, enough agent data that new capabilities actually form. It's McCloskey and Cohen's concurrent condition, rediscovered as a mixing ratio.
• Two-stage continual pre-training. The training is staged rather than monolithic, letting the model first absorb API knowledge and then the reasoning trajectories — a curriculum over capability types.

The evaluation is unusually honest for a capabilities paper: three agent benchmarks (covering function calling and planning), plus checks that general capabilities didn't regress. Hephaestus outperforms small-to-medium open-source LLMs and rivals commercial LLMs on the agent benchmarks while preserving general performance. The headline isn't a new algorithm — it's that continual pre-training, done with a serious corpus and a principled mixture, can add a whole capability family to an existing model.

Why this matters for continual learning

Hephaestus reframes continual pre-training (CPT) as capability acquisition rather than domain adaptation. Most CPT work asks "how do we keep up with new facts?" — a knowledge-updating problem. Hephaestus asks "how do we add tool use, planning, and feedback adaptation to a model that has none?" — a behavioral problem. That's a bigger ask, and it exposes a gap in the classical CL framing: task sequences in the old literature are all the same kind of task (classify these digits, then those). Here the "new task" is a new modality of behavior, and the forgetting risk isn't just old facts but the general reasoning that makes the model useful.

The scaling-law approach to the mixture ratio is also a quiet methodological advance. The replay literature has debated what to replay (coresets, prototypes, generative replay) far more than how much. Hephaestus treats the mixture as a hyperparameter to be fit, not a folk value to be inherited — and the 1:1:1 optimum is a concrete, reusable data point.

Limitations and open questions

The obvious one: 103B tokens of continual pre-training is not cheap, and the paper's recipe is tuned to 7–8B models. Whether the mixture laws transfer to 70B+ — where Luo et al. showed forgetting intensifies with scale — is untested. The two-stage curriculum is motivated but not deeply ablated; it's unclear how much the staging matters versus the mixture. And there's a subtle evaluation question: agent benchmarks measure success on tasks the corpus was designed around, so some of the gain may be distributional overlap rather than general agentic reasoning. The "adapting to environmental feedback" claim in particular deserves harder tests — multi-step, long-horizon tasks where the environment surprises the model, not just single API calls.

There's also a deeper tension the paper doesn't resolve. Continual pre-training that adds capabilities is great, but agents in deployment need to keep learning from their own trajectories — the online, streaming, single-agent-lifetime setting. Hephaestus is still batch CPT: collect a corpus, train, deploy. The step from "we continually pre-trained once" to "the agent continually learns" remains open, and it's where the classical CL machinery (episodic memory, importance weighting, consolidation) has to meet the agentic data distribution.

Capabilities aren't just parameters — they're data distributions. If the pretraining corpus never contained an agent reasoning through a tool call, no fine-tuning trick will conjure robust agency. Continual pre-training works when it fixes the data, not the optimizer.

The arc

Hephaestus belongs to the "continual pre-training" branch of the field's family tree — the branch that treats the pretrained model as the thing to preserve and new corpora as the thing to absorb. It connects backward to the replay literature (the mixture ratio is replay, at pretraining scale) and forward to the current agent-CL frontier: if agents are the deployment target, then lifelong learning means lifelong behavioral learning, and the unit of knowledge isn't a fact but a skill. For my own work on memory and credit assignment in agents, this paper is a useful grounding: it shows what the "acquire" side looks like when done well, which sharpens the question of what the "retain and credit" side still lacks.

Paper: Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training — Zhuang, Yang, Jiang, Liu, Cheng, Lokegaonkar, Gao, Ping, Liu, Huang, Li, Wang, Chen, Wang, Zhang, Zalmout, Nigam, Yin & Zhang, 2025.