Post-training · 2025–26 · ten reports on one schema

LLM report diff

Ten model reports — seven from the onboarding reading list plus DeepSeek V4, Gemma 4 and GLM-5 — each read against the same skeleton: SFT → preference → verifiable-reward RL. OLMo 3 is the reference because it publishes every stage. Each tab opens with the thesis and the pipeline, then a stage-by-stage ledger: what the data was, who it was distilled from, which pipelines synthesized it, how it was filtered, the algorithm, the rewards and environments, and the schedule — with the numbers the reports actually give. The self-play papers from the same list have their own note: self-play for reasoning.

pretrain / baseSFT · distillationpreference · RLHFverifiable-reward RLother

Ai2 · Nov 2025 · report Dec 2025 ·source · arXiv · alphaXiv

OLMo 3 — the reference skeleton reference#

ThesisThe model flow is the product: every stage, dataset, checkpoint and the RL code are released so the recipe itself can be studied, forked, and benchmarked against.

Architecture

7B and 32B, dense
Dolma 3: 5.93T tokens → Dolmino 100B mid-training → Longmino long-context
8k → 65k context

Openness

weights · yespretrain data · yespost-train data · yescode · yescheckpoints · yesrecipe · yes

Stages

PretrainDolma 3 · 5.93T
SFTDolci Think SFT · 2.27M
DPODelta Learning · 200k pairs
RLVROlmoRL · Dolci-Think-RL ~105k
RL-Zero pathRLVR from base · no SFT

Pipeline

Stage by stage

1PretrainDolma 3 · 5.93T

Data: Dolma 3 (5.93T tokens), then a 100B-token Dolmino mid-training mix, then Longmino for context extension to 65k.
All released.

2SFTDolci Think SFT · 2.27M

Data: 2,268,468 prompts (7B).
Math: OpenThoughts3 math 753k (kept at 16× repetition) + STEM 99k + SYNTHETIC-2 verified 105k.
Code: Dolci Python Algorithms 467k + OpenThoughts3 code 89k + Nemotron post-training code 114k.
Precise IF: Persona IF 223k + Precise IF 136k.
Chat: WildChat 83k, OpenAssistant 7k.
Safety: CoCoNot 10k, WildGuardMix 38k, WildJailbreak 41k.
Aya 99k, TableGPT 5k, identity 290.
Distilled from: QwQ-32B regenerates incomplete OpenThoughts3 math traces at 32k (vs the original 16k), writes up to 16 code responses per prompt, and answers precise-IF prompts.
DeepSeek R1 supplies chat, safety, science and other-domain traces.
GPT-4.1 writes the unit tests that filter the code responses.
Synthetic: Persona-IF prompts regenerated with Nemotron-Personas; code prompts from AceCoder, The Algorithms, Llama-Nemotron post-training and OpenCodeReasoning get fresh QwQ solutions verified against GPT-4.1 test cases; IF responses kept only if every constraint verifier passes.
Filtering: Heuristics drop non-commercial licenses, incomplete chains, failed constraint/test checks, mentions of other model developers, repetition, and traces with excessive Chinese characters or political content.
Topic filter uses the OpenAI query taxonomy.
Decontamination: 8-gram matching, 50% overlap threshold, with math-symbol exceptions.
Schedule: Mixing ablated on a 100k OpenThoughts3 base mix + \(\le 100\text{k}\) per category.
Two epochs, learning-rate sweep, then a linear soup of the two best-LR checkpoints via mergekit.
Switching from Open Instruct to OLMo-core gave 8× throughput.

3DPODelta Learning · 200k pairs

Data: 200,000 prompts: WildChat 40.7k, UltraFeedback 32.8k, FLAN 19.7k, Precise IF 19.4k, OpenThoughts3 science 19k, Python Algorithms 13.2k, Persona MATH 10.7k, Evol CodeAlpaca 7.6k, WildJailbreak 5.6k, WildGuardMix 5.3k, Aya 4.1k, and smaller sets.
Synthetic: Every pair is machine-made with one rule: chosen = Qwen3 32B in thinking mode, rejected = Qwen3 0.6B in thinking mode.
The observation that motivated it: further SFT on Qwen3-32B traces outright hurt the SFT model, so the traces are turned into a contrastive signal instead.
Algorithm: Plain DPO, one epoch, sweeping learning rate and dataset size — early stopping is important.
Rejected responses are left unfiltered on the theory that a bad rejection is a useful contrast.
Note: The Instruct path adds GPT-4.1-judged pairs with a delta-aware twist (always include a weak model, pick the worst as rejected), synthetic multi-turn self-talk and synthetic-context conversations, and a length-bias cap of 100 tokens between chosen and rejected.
Modernizing the old UltraFeedback pipeline with a better judge alone made things worse.

4RLVROlmoRL · Dolci-Think-RL ~105k

Data: 104,869 prompts for Think (171,950 for Instruct).
Precise IF: IF-RLVR 30.2k.
Math: OMEGA 15k, AceReason 6.6k, Open-Reasoner-Zero 3k, Klear-Reasoner 3k, DAPO-Math 2.6k.
Code: AceCoder 9.8k, Klear code 8k, SYNTHETIC-2 3k, Nemotron code 2.3k.
General chat: Tülu 3 SFT 7.1k, WildChat-4.8M 7.1k, Multi-Subject RLVR 7.1k.
Rewards / verifiers: Math: rule-based SymPy equivalence, 0/1.
Code: test cases executed on AWS Lambda, either pass-rate or all-pass.
IF: constraint functions, 1 only if every constraint holds.
Chat with reference: Qwen3-32B (thinking off) judge scores 0–1 against the reference.
Open chat: same judge, no reference.
Puzzle and length-control verifiers were tried and dropped.
Synthetic: Code RL data built by problem rewriting → solution generation → test generation, keeping items whose solution passes >80% of the generated tests.
Chat references rewritten with GPT-4.1; prompts kept only if a Qwen2.5-7B-OpenThoughts2 sampler scores mean F1 between 0.1 and 0.8 over eight tries — removes both noise and impossibles.
Filtering: Offline difficulty filter: 8 rollouts from the DPO checkpoint at temperature 1, drop prompts with pass rate >62.5% (7B).
The 32B skipped this and relied on active sampling over the 7B-filtered set.
Named OMEGA subtasks the model struggled with were downsampled 50%.
Algorithm: GRPO with: zero-gradient group filtering; active sampling to keep batch size constant; token-level loss; no KL; clip-higher; truncated importance sampling between vLLM and trainer logprobs; no standard-deviation normalization in the advantage (Dr GRPO) to remove the difficulty bias.
Scale: Fully asynchronous: DeepSpeed learner + a pool of vLLM actors, continuous batching, in-flight weight updates.
At 32k max / 14.6k mean length, static batching would waste up to 54% of compute.
32B run: 8 training nodes vs 20 inference nodes; 7B: 2 vs 7, roughly 14× the compute on inference.
A step averaged 1000 s of which 125 s was training.
The 7B run took 15 days on the old stack and 6 on the new one; the 32B Think went 750 → 2300 steps for 3.1 and had not saturated.

5RL-Zero pathRLVR from base · no SFT

Data: Math: DAPO deduplicated and English-only, plus one representative per semantic cluster from Klear-Reasoner, Open-Reasoner-Zero and OMEGA, decontaminated against pretraining and mid-training data, then offline-filtered to drop prompts solved 8/8 by the base model → 13.3k prompts.
Code, IF and chat subsampled from Dolci-Think-RL.
Finding: Simple per-domain prompt templates greatly outperform post-training templates like <think></think> when starting from a base that never saw special formatting.
Training at 16k response length, evaluating at 32k and temperature 1: AIME pass@1 jumps in the first few hundred steps, then climbs slowly; pass@32 also rises, so diversity is preserved.
Note: Positioned as a clean benchmark for RL algorithms — fully open pretraining data means spurious rewards from contaminated mid-training can be ruled out.

What makes it the reference

This is the baseline every other tab is diffed against.
Distinctive: Delta-Learning DPO (rejected = a much weaker version of the same family), verifiers for chat via an LM judge inside RLVR, and an RL-Zero path that skips SFT entirely.
Everything is released, including the RL infrastructure — which is the thesis, not a side effect.

Microsoft · Apr 2025 ·source · arXiv · alphaXiv

Phi-4-reasoning #

ThesisCuration beats scale: a 14B model approaches DeepSeek-R1 by SFT-ing on “teachable” prompts with o3-mini traces; a 90-step GRPO pass on ~6k math problems adds a further increment.

Architecture

14B dense (Phi-4 base; chosen over the mid-train checkpoint for its safety post-training)
RoPE base frequency doubled → 32k context
two unused placeholder tokens repurposed as <think> / </think>

Openness

weights · yespretrain data · nopost-train data · nocode · nocheckpoints · norecipe · partial

Stages

BasePhi-4 · 14B
SFT1.4M pairs · 8.3B unique tokens
GRPO-plus only · 90 steps · ~6.4k problems

Pipeline

Stage by stage

1BasePhi-4 · 14B

Note: Phi-4 was already pretrained on reasoning-oriented synthetic data; the reasoning models start from the post-trained Phi-4, not the mid-train checkpoint.

2SFT1.4M pairs · 8.3B unique tokens

Data: A single seed database shared with RL: questions from web sources, existing datasets and licensed collections, plus synthetic questions grounded in filtered web content.
STEM and coding dominate; general QA and alignment/safety prompts are included.
Final mixture ~1.4M prompt–response pairs, 8.3B unique tokens, 16B tokens trained (2+ passes).
Filtering: “Teachable” = at the edge of Phi-4's ability and reasoning-heavy rather than recall.
Where no ground truth exists, the plurality answer of a strong reference model is the proxy, and difficulty is the agreement rate of weaker models (Phi-4, GPT-4o) with that proxy; seeds with a meaningful gap are kept.
Rubric-based LLM evaluators score the number and complexity of reasoning steps.
Decontaminated against AIME-2024, MATH, GPQA, LiveCodeBench, OmniMATH, SWE-Bench Verified and ~15 more.
Synthetic: Seeds are rewritten into verifiable forms — a proof problem becomes a numeric-answer problem, coding problems become word problems.
Synthetic math seeds with short final answers gave a consistent +3–10% on AIME 2022–24 and made the data usable for RL verification later.
Distilled from: o3-mini.
Medium reasoning effort matched DeepSeek-R1 as a teacher but was more token-efficient; high effort was stronger on every task and longer, which is why context was extended to 32k.
Safety responses came from the same teacher with Responsible-AI guidelines in the prompt, removed at training time.
Schedule: ~16k steps, global batch 32, 32k context, AdamW lr 1e-5 (grid 1e-6 to 2e-5), 450-step warmup, weight decay 1e-4.
A fixed reasoning system message is required — removing or randomizing it hurt.
Mixture weights are epoch counts per source cluster, tuned per domain and simply concatenated (“additive” property).
Response length slightly decreases over SFT, unlike RL.

3GRPO-plus only · 90 steps · ~6.4k problems

Data: 72,401 math prompts from the seed database; 64 sampled per iteration; the selected checkpoint (best AIME-24) is step 90, i.e.
~6.4k problems × 8 rollouts.
No coding data — visible in the LiveCodeBench numbers.
Rewards / verifiers: Rule-based.
Answer extracted from \boxed{}, checked by equivalence rules with an LLM verifier as fallback.
Length-aware accuracy: correct answers score 1.0 down to 0.5 via a cosine schedule once they exceed 25,600 tokens; incorrect answers score −1.0 rising to −0.5 as length approaches 3,702 tokens — i.e. be concise when right, think longer when wrong.
Missing end token → −0.5; broken <think> block → −1.0.
Repetition penalty on 5-gram frequency.
Final reward = 8/13·accuracy + 1/13·repetition.
Algorithm: GRPO in verl: batch 64 on 32 H100, lr 5e-8 with 10-step cosine warmup, group size 8, KL \(\beta = 0.001\), entropy \(\gamma = 0.001\), 32k max length (tested to 64k).
Finding: Traces get longer with accuracy held or improved; a single small math-only RL pass moved non-math benchmarks (planning, calendar) as well.

Diff vs. OLMo 3

No preference stage of any kind — SFT straight into a tiny GRPO pass.
One teacher, proprietary: not reproducible outside Microsoft, which weakens ‘curation beats scale’ as a general claim.
RL data is a 6k subset of the same seeds used for SFT; the pass@k comparison between -reasoning and -plus is never shown.
Same difficulty-filtering instinct as Nemotron and OLMo's RL curricula, applied to SFT prompts instead.

NVIDIA · Apr 2026 ·source

Nemotron 3 Super #

ThesisThroughput at parity: match GPT-OSS-120B and Qwen3.5-122B on benchmarks while serving 2–7× more tokens per GPU — with an industrial synthetic-data operation feeding a four-stage RL pipeline.

Architecture

120B total · 12B active · LatentMoE
hybrid Mamba-Attention, MTP heads for speculative decoding
pre-trained in NVFP4 · 25T tokens (20T diversity + 5T quality) · 1M context

Openness

weights · yespretrain data · partialpost-train data · yescode · partialcheckpoints · yesrecipe · yes

Stages

Pretrain25T · NVFP4
SFT stage 17M samples · 80B tok
SFT stage 285% blend + long-context
RLVR21 envs · 37 datasets · 3 rounds
SWE-RL20B tok · end-to-end
RLHF18B tok · principle GenRM
MTP healingheads only

Pipeline

Stage by stage

1Pretrain25T · NVFP4

Data: 25T tokens in two phases: 20T for breadth, 5T of high-quality data for benchmark accuracy.
First model pre-trained in NVFP4.

2SFT stage 17M samples · 80B tok

Data: Over 7M samples.
Reused from Nemotron 3 Nano: Chat, Infinibyte, Formal Proofs.
Regenerated with new teachers (DeepSeek v3.2, Kimi K2): Competition Math, Competition Code, Conversational Tool Use, Multilingual, Science.
The rest are new pipelines listed below; agentic data's share of the blend was raised sharply.
Synthetic: Agentic CLI: NeMo Data Designer writes 20k queries from a 24-action taxonomy, GPT-OSS-120B judges them down to 15k tasks (each with an AGENTS.md-style spec), plus 3k SWE tasks with a real repo/commit and 10k Node.js web-dev tasks from a 100-task taxonomy; Qwen3-Coder-480B and MiniMax M2.5 are run inside Codex, OpenCode, Qwen Code CLI and Stirrup and their interactions recorded, then normalized to OpenAI tool format.
Software engineering: SWE-Gym, R2E-Gym (problem statements rewritten by Qwen3-Coder-480B), SWE-rebench, OpenHands trajectories distilled from Qwen3-Coder-480B.
Long context: cluster pretraining docs by topic, concatenate to 128k/256k/512k, LLM-write multi-hop QA needing 4–7 steps, sample 8 traces, keep the shortest majority-vote answer; plus rule-chained sequential tasks over Qwen3-235B-Thinking snippets and SQL-verified questions over Nemotron-Personas records.
Financial: 565 SecQue seeds × S&P 500 × FY2019–24, GPT-OSS-120B paraphrases, GenSelect (5 candidates, Qwen3-235B picks, Qwen3-30B checks answerability) → 366,243 QA pairs with traces.
CUDA: 100k samples from DeepSeek-R1 + GPT-OSS-120B, seeds from OSS libraries, NVIDIA APIs and BackendBench, kernels executed, validated and ranked by speed; repair/optimize tuples from an internal CUDA agent using error messages and Nsight logs.
Safety: existing prompt libraries plus synthetic over-refusal, bias, copyright and prompt-injection prompts; a response-policy classifier picks the mode, then deliberative-alignment two-stage generation (safety reasoning trace, then answer), moderation-classifier filter.
Search: Wikidata SPARQL hub entities, 4–8-hop random walks → question → obfuscation → MiniMax-M2 solves with Tavily web search, ~12 tool calls per trajectory.
Terminal: 84,864 samples (68,924 synthetic + Cascade-Math 8,125 + Cascade-Code 7,815), DeepSeek-V3.2 in Dockerized execution loops under the Terminus 2 scaffold.
Multilingual: line-by-line translation into 6 languages with Qwen2.5-14B-Instruct, format repaired by Qwen3-4B-Thinking.
SQL: 96.5k text-to-SQL across 3 dialects, 60 sectors, ~700 topics, 90 concept buckets, distractor tables injected, validated by dialect parsers and five LLM critics.
Conversational tool use: six-stage fully synthetic pipeline (domain → policy+tools → scenario → 16 simulated interactions → LM judge on outcome and process → drop all-success/all-fail scenarios) → 279,116 conversations across 838 domains (Nano had 15,588 across 5), using Qwen3-235B Thinking/Instruct, Qwen3-32B, DeepSeek-R1-0528, DeepSeek-V3.2, gpt-oss-120b.
General tool use: tool sets from ToolEyes, API-Bank, UltraTools, AutoTools, xLAM, Glaive, Toucan-1.5M; a User-LLM / Assistant-LLM / Tool-LLM three-role simulation seeded with Nemotron-Personas, judged per turn and per trajectory with rule checks, scaled with DeepSeek-V3.2 and GLM-4.7 → 1.5M trajectories.
Algorithm: Token-level global average loss over the packed batch — every output token weighted equally.
Shared-weight MTP head trained alongside with a 0.3-scaled auxiliary loss.
Schedule: 256k-token sequence packing, global batch 64, constant lr 1e-5, 30k warmup samples.

3SFT stage 285% blend + long-context

Data: 85% of the stage-1 blend plus 256k- and 512k-token long-context samples.
Algorithm: Per-conversation normalized loss, averaged equally across conversations, so long outputs stop dominating.
Reason for the split: single-stage SFT caused a marked degradation on long-input / short-output tasks.
Note: Reasoning control lives here.
Off mode: traces stripped from a random 3% of samples.
Low-effort mode (new): 2% of SFT samples generated by GPT-OSS-120B in its low-effort mode over math, STEM QA and IF.
Inference-time budget control: a 350-step semi-on-policy SFT that collects the model's own rollouts and truncates 12% of traces to random budgets.

4RLVR21 envs · 37 datasets · 3 rounds

Environments: 21 environments over 37 datasets, mostly open-sourced in NeMo Gym: competition math with and without a Python tool; a new formal-proof verification environment; competition code plus single-step SWE patch generation; harder curated STEM; a multi-challenge instruction-following set with rubric rewards; two safety environments (over-refusal reduction, and jailbreak robustness where a PAIR-style iterative attacker mines high-success prompts against an early SFT-only checkpoint); the Nano long-context environment; conversational and terminal tool use; Reasoning Gym.
Filtering: Drop prompts the SFT model answers correctly every time, then sort the rest by difficulty (curriculum).
Low-effort prompts start at 2% of RL prompts (math, STEM, coding held out from SFT) and are reduced to 1% (math, STEM), with reward adjusted for token count.
Finding: Training all environments jointly gives stable gains; single-environment RL causes severe regressions on everything else.
Three rounds: 25, 30 and 26 environment types, the later two with low-effort mode.
Algorithm: Asynchronous GRPO.
Training and inference on separate GPUs; inference workers fill a rollout buffer; new weights are pushed as soon as available, so one trajectory can contain tokens from several model versions; the KV cache is not recomputed after an update; policy lag capped at one step.
Train/inference logprob mismatch handled by masking the importance-sampling ratio.
256 prompts × 16 responses per step, batch 4096 = one gradient update per rollout, max generation 49k rising to 64k.
A principle-following GenRM is used inside this stage as well as in the later RLHF stage.
Note: PivotRL for agentic domains (programming, search, terminal, conversational tools): turn-level RL that reuses offline SFT expert trajectories, updates only on “pivot” turns where the policy is uncertain, and rewards similarity to the expert action rather than an exact match — chosen because SFT was cheap but degraded out-of-domain and end-to-end RL was too expensive for every domain.

5SWE-RL20B tok · end-to-end

Environments: Each rollout launches an Apptainer container with the target repository, runs an OpenHands agent loop to produce a patch, and scores it against ground-truth tests for a binary reward.
OpenCode and Codex agent classes are implemented inside OpenHands so the same harness trains against Claude Code– and Codex-CLI–style tool formats.
Schedule: Run as its own stage because SWE rollouts are slow and long-context and would bottleneck a mixed batch; batching and rollout settings are tuned for long-horizon trajectories.
20B tokens.

6RLHF18B tok · principle GenRM

Rewards / verifiers: A principle-following generative reward model initialized from Qwen3-235B-A22B-Thinking-2507 and trained on HelpSteer3, the commercially usable subset of lmarena-140k, and newer human preference data.
Principles steer behaviour on identity and safety topics.
Schedule: Applied throughout multi-environment RL and again as a separate final stage for instruction following, robustness and interaction quality.
18B tokens.

7MTP healingheads only

Note: Retrain the MTP heads on the RLVR prompts with all other weights frozen, restoring speculative-decoding acceptance after RL moved the backbone.

Diff vs. OLMo 3

Four RL stages against OLMo's one; preference done with a GenRM inside RL rather than a DPO stage.
Roughly a dozen named synthetic-data pipelines with a committee of open teachers, versus Dolci's published mix from two teachers.
The key claim: 21 environments trained jointly is stable, single-environment RL regresses everything else.
PivotRL is a deliberate SFT/RL hybrid for agentic domains — offline expert trajectories, on-policy updates at uncertain turns.

NVIDIA · May 2025 ·source · arXiv · alphaXiv

Llama-Nemotron #

ThesisEfficient reasoning: derive the architecture from Llama 3 by NAS, distill 33M samples of reasoning from DeepSeek-R1 with an on/off toggle, and spend RL only on the largest model where it can surpass the teacher.

Architecture

Nano 8B · Super 49B · Ultra 253B
from Llama 3.1-8B / 3.3-70B / 3.1-405B via Puzzle NAS: block-wise local distillation + mixed-integer assembly; FFN Fusion on Ultra
recovery: knowledge distillation + continued pretraining (Ultra +88B tokens)

Openness

weights · yespretrain data · nopost-train data · yescode · yescheckpoints · partialrecipe · yes

Stages

NAS + recoveryPuzzle · KD · CPT
SFT33.0M samples · R1-distilled
Reasoning RLUltra only · GRPO · science
IF RLRLOO · <120 steps
RLHFonline RPO / GRPO

Pipeline

Stage by stage

1NAS + recoveryPuzzle · KD · CPT

Note: Puzzle searches per-block replacements (removed attention, narrower FFNs) under a throughput target and assembles them with mixed-integer programming; the result is healed by block-wise local distillation, then knowledge distillation and continued pretraining.
This replaces pretraining from scratch.

2SFT33.0M samples · R1-distilled

Data: 33,011,757 samples.
Math 22.1M (66.8%): 2.2M reasoning-on, 19.8M reasoning-off.
Code 10.1M (30.6%): 0.99M on, 9.1M off.
Science 709k (all reasoning-on).
Chat 40k, instruction following 56k, safety 31k.
Released as Llama-Nemotron-Post-Training-Dataset.
Synthetic: Math: Art of Problem Solving forums scraped (Middle School excluded); Qwen2.5-32B-Instruct extracts problems, classifies out proofs / multiple-choice / yes-no / invalid, extracts final answers; LLM-based decontamination.
Code: 28,904 competitive problems from TACO, APPS, CodeContests, CodeForces (<0.3% contamination after checks with Llama-3.3-70B and Qwen2.5-32B judges).
Science: StackOverflow QA plus synthetic MCQs — topics from Nemotron-4-340B, questions from Qwen2.5 at several difficulty levels, OpenMathInstruct-2-style variations.
General: synthetic open/closed QA, extraction and brainstorming prompts plus permissive real prompts; 20k ShareGPT/WildChat prompts improved by a Feedback → Edit → Select model trio.
Distilled from: Reasoning-on: DeepSeek-R1, 16 solutions per math problem, multiple per code problem (temp 0.6, top-p 0.95 via SGLang, <think> tags requested), multiple science traces.
Reasoning-off: Qwen2.5-Math-7B-Instruct (64 per math problem), Llama-3.1-Nemotron-70B-Instruct for general prompts, Llama-3.3-70B-Instruct elsewhere.
General responses rejection-sampled with the Llama-3.1-Nemotron-70B reward model.
Filtering: Math solutions kept only if the answer matches, judged by Qwen2.5-32B; the plurality answer is the ground truth when none can be extracted.
Code: reasoning trace must be present, code must sit outside the think block, Tree-sitter syntax check → ~488k Python samples.
Science: majority vote when no gold answer.
Ablation: code data from 25k to 736k samples kept improving; hard CodeContests problems gave the biggest jumps.
Schedule: Nano: three stages — reasoning-only (code/math/science) for 4 epochs at lr 1e-4 to prevent repetitive completions, then mixed on/off data, then a small chat/IF/tool blend; batch 256, 32k packing.
Super: full set, 1 epoch, lr 5e-6, 16k.
Ultra: 24k packing, warmup to 1e-5 then cosine to 1e-6, restarted with fresh optimizer state after gradient explosions.
Higher learning rates were needed for long traces because of length-dependent token averaging.

3Reasoning RLUltra only · GRPO · science

Filtering: Generate 8 responses per question with LN-Super, compute pass rate, discard prompts at \(\ge 0.75\).
Curriculum: a Gaussian target distribution over pass rates that slides from easy to hard across sequential batches (ablation: stabler and higher GPQA than random order).
Rewards / verifiers: Accuracy: Llama-3.3-70B-Instruct judges whether the answer matches a ground truth that may be a number, sentence or paragraph.
Format: reasoning must sit between <think> tags in on mode and be absent in off mode.
Algorithm: GRPO, 72 prompts × 16 responses at temperature 1 / top-p 1, global batch 576, two gradient updates per rollout, trained to convergence on GPQA-Diamond.
NeMo-Aligner with vLLM generation and Megatron training co-located on 72 nodes × 8 H100: TP8 with sequence parallel, CP2, PP18, DP2.
FP8 generation gave 1.8× (32 tok/s/GPU/prompt).
~140k H100 hours.
Finding: Distillation caps the student at the teacher; Ultra's SFT model approached R1 but could not pass it.
RL let it, setting the open-model GPQA state of the art at the time.
Preliminary runs showed RL underperforming distillation for smaller models, so Nano and Super get none.

4IF RLRLOO · <120 steps

Rewards / verifiers: Synthetic prompts with one to ten detailed constraints, scored by an IFEval-style verifier.
RLOO, batch 128, fewer than 120 steps, for Super and Ultra; improved reasoning benchmarks as well as IF.

5RLHFonline RPO / GRPO

Algorithm: Super: two iterations of online RPO maximizing Llama-3.1-Nemotron-70B-Reward on HelpSteer2 prompts — lr 4e-7, KL \(\beta 1\times 10^{-5}\), reward scale 3.0, batch 64, 500 steps each.
Ultra: GRPO instead, 8 samples per prompt, 30 steps, lr 3e-7, batch 288.
Finding: Arena Hard 69.1 → 88.1 for the 49B Super, and every other benchmark except IFEval improved even though neither the prompts nor the reward model target math, code or science.

Diff vs. OLMo 3

The architecture is derived, not designed — NAS from Llama 3 stands in for pretraining.
RL only at 253B; OLMo runs RLVR at 7B and 32B. The report's own ablation says RL loses to distillation at small sizes.
On/off toggle via paired SFT data and a system prompt, versus OLMo's two separately trained paths.
Preference is online RL with a learned reward model (RPO / GRPO), not offline DPO.

Moonshot · Jul 2026 ·source · arXiv · alphaXiv

Kimi K3 #

ThesisOpen frontier intelligence: train nine domain × effort RL experts on white-box agentic environments at million-token context, then merge them back into one model by on-policy distillation.

Architecture

2.8T total · 104B active · 896 experts, top-16 (“Stable LatentMoE”)
hybrid: Kimi Delta Attention (linear, channel-wise forget gates) + Gated MLA; attention residuals to all earlier layers
1M context, no explicit positional encoding · native multimodal (MoonViT-V2) · MTP layer

Openness

weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · norecipe · partial

Stages

Pretraintokens undisclosed
SFTcold start · sizes undisclosed
RL · 9 experts3 domains × 3 efforts
MOPDmerge 9 experts → 1
DeploymentEAGLE-3 draft · QAT

Pipeline

Stage by stage

1Pretraintokens undisclosed

Note: Token count is not stated.
Four-stage long-context curriculum from 8k to 1M; the report claims 2.5× scaling efficiency over Kimi K2 from hyperparameter work.

2SFTcold start · sizes undisclosed

Synthetic: Agentic trajectories synthesized by domain-specialized models from earlier Kimi releases, passed through multi-stage verification and human-in-the-loop annotation, serialized in an XTML chat template.
Sources and volumes are not given.
Note: Quantization-aware training starts at SFT and runs through RL: MoE expert weights in MXFP4, activations MXFP8, everything else in higher precision; rollout and training share the scheme, removing the train/inference mismatch.

3RL · 9 experts3 domains × 3 efforts

Environments: Three domains, each a single RL expert per effort level: general (experience, vision, reasoning, faithfulness, search, knowledge work), general agents (long-horizon assistant, deep research, paragraph writing), coding agents (SWE, coding experience, GPU kernels, web dev).
A unified white-box harness represents tools, system prompts, context management, skills, memories and subagents as composable modules, so training can instantiate Kimi Code, Claude Code, Codex, OpenClaw, Hermes or invented harnesses and rotate them across task groups.
Synthetic: Knowledge-graph-guided task synthesis: agents recursively expand a concept DAG from coarse seeds by web search, deduplicating against existing nodes; sampled nodes plus ancestor context become web queries; retrieved real materials are turned into tasks of a chosen type (coding, knowledge, vision).
Verifiable agentic problems include multi-step web research, professional workflows (investment banking, data analysis, legal) run for dozens to hundreds of steps in sandboxes, and visual reasoning where the model crops, zooms and computes with a Python tool.
Kernel tasks from repositories such as Flash Linear Attention span CUDA, Triton, CuTe DSL, Gluon, ThunderKittens and TileLang in BF16/FP8/FP4.
Personal-assistant tasks run in mock Gmail, Notion, Slack and Canvas over multiple simulated days with dozens of interdependent events — a rollout can reach thousands of tool calls and millions of tokens; initial workspaces are built by agents searching the web.
Rewards / verifiers: Verifiable tasks: direct correctness.
Kernels: zero reward above a numerical error threshold, 0.5 for matching an expert implementation rising toward 1 at the hardware roofline, with a hacking detector for CUDA-graph replay, input caching and precision reduction.
Non-verifiable: an Agentic Generative Reward Model with a mandatory protocol — read the output, write a rubric, score each candidate, record scores — run as tournament-style binary comparisons; a candidate exceeding \(\sigma\) × the cold-start verbosity automatically loses.
Assistant events are judged per event by deterministic rules or LLM evaluators.
Algorithm: Partial rollouts: sample K completions for N prompts, pause generation once a fraction \(\lambda\) of the N×K trajectories finish, resume the rest next iteration; one long trajectory naturally spans iterations, and a per-token regularizer keeps updates local so stale data stays stable.
Optimization follows Kimi K2.5.
Effort control: each problem gets a budget \(b_{0}\) from the cold-start model and a −1 reward if output exceeds \(\tau \cdot b_{0}\) (thinking tokens for general tasks, all output including tool arguments for agentic); \(\tau\) is annealed max → high → low per domain under human guidance.
Scale: Co-located RL to keep a 1M-context run within a few hundred GPUs.
External CPU-DRAM KV-cache pool with write-back, training states offloaded to NVMe between phases, an auto-throttling request scheduler, and reference-model weights streamed through the policy's gradient buffers.
AgentENV Firecracker microVMs: incremental checkpoints at 133 ms and resumes at 49 ms; sandboxes paused while waiting on inference (up to 98% of their lifetime), forked for side-effect-free judging, snapshotted for recovery; tens of thousands live at once.
Finding: Scaling RL FLOPs raises average tool-call steps and scores together across coding, tool use, web dev, search, professional workflows, office deliverables, chart reading and visual puzzles.

4MOPDmerge 9 experts → 1

Algorithm: Multi-Teacher On-Policy Distillation.
For a sampled domain and effort level, the matching expert is the teacher; the per-token reward is \(\operatorname{clip}(\log \pi _{\text{teacher}} / \pi _{\text{student}}, \pm R_{\text{max}})\) with a stop-gradient on the teacher, plugged into the same RL machinery (so partial rollouts still apply).
Finer-grained top-k distillation objectives showed no advantage.

5DeploymentEAGLE-3 draft · QAT

Note: The pretrained MTP layer is fine-tuned into an EAGLE-3-style draft with the target frozen, unrolled seven steps, fusing features from the 1st, 4th and final attention-residual blocks, and trained on the acceptance-rate loss directly rather than a KL surrogate.

Diff vs. OLMo 3

Expert-then-merge: nine RL experts consolidated by on-policy distillation, versus OLMo's single RL run.
Preference comes from a tournament GenRM inside RL; there is no DPO stage.
Task synthesis from an agent-built knowledge graph is the nearest any model report gets to self-play.
Releases weights and kernels, not data — the opposite openness profile to OLMo. SFT is a black box.

Alibaba · Mar 2026 ·source · arXiv · alphaXiv

Qwen3-Coder-Next #

ThesisScale the agentic training, not the model: an 80B-A3B coding agent reaches SWE-bench Verified 70.6 by learning from executable environments at every stage — ~800K synthesized verifiable tasks feed mid-training and RL, four experts climb from one SFT checkpoint, and expert distillation folds them back into one deployable model.

Architecture

80B total · 3B active MoE on the Qwen3-Next base (hybrid attention); base and instruct weights released, Apache 2.0
context extended 32k → 262,144 during mid-training for repository-level data and multi-turn trajectories
fill-in-the-middle kept, in chat and search-and-replace formats, with a proxy that converts patches to autocomplete output

Openness

weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · base + instructrecipe · partial

Stages

BaseQwen3-Next · 80B-A3B
Mid-trainingtrillions of tokens · 262k ctx
SFTverified trajectories · judge-ranked
Expert climbs4 experts · 2 by RL
Expert distillation4 experts → 1

Pipeline

Stage by stage

1BaseQwen3-Next · 80B-A3B

Note: The pretrained Qwen3-Next base: hybrid attention with MoE, 80B total parameters and 3B activated per token. Nothing else about pretraining is in the report; the Qwen3.8-Next architecture paper (Sept 2026) is the family's pretraining document.

2Mid-trainingtrillions of tokens · 262k ctx

Data: Principle: natural data first, “the minimum amount of synthetic data required” — heavy synthetic mixes over-specialise and flatten response diversity.
GitHub expanded from 92 to 370 languages with more pull requests, repositories and code-review data; file-level first, then repository-level code (~600B tokens), which proved more impactful than file-level data alone. Corpus through 30 Sep 2025.
Text–code grounding from Common Crawl and math / programming / education domains, rewritten into clean Markdown by Qwen3-Coder-480B-A35B-Instruct: EvalPlus 54.4 → 63.1 and MultiPL-E 36.0 → 48.4 in a mid-training ablation.
GitHub PRs converted into software-engineering tasks: issue text, code context rebuilt by reverting the patch plus retrieved files (noise on purpose), edits in both search-and-replace and git-diff formats.
Synthetic: Single-turn QA: Common Crawl seeds, Coder-480B writes self-contained question–answer pairs of escalating depth and may abstain on poor documents; Wikipedia-style rewrites were dropped after they hallucinated references, so rewriting is restricted to content-preserving transforms.
Multi-turn agentic trajectories on the synthesized tasks below, generated by six scaffolds — SWE-agent, Mini-SWE-agent, OpenHands, Claude Code, Qwen-Code, Terminus — with Coder-480B as the teacher; rule-filtered for missing termination, task failure and malformed tool calls.
A small instruction-following slice for early monitoring; fill-in-the-middle from Stack-V2 in chat-FIM and search-and-replace formats (the latter wins at equal scale).
Environments: About 800K verifiable software-engineering task instances across nine-plus languages, shared with RL, from two pipelines.
PR mining: each issue-linked PR becomes buggy state / fix / test patch; an environment-building agent writes the Docker image and verifier, automated detection removes non-functional verifiers, a QA agent drops ambiguous tasks and misaligned tests.
Bug injection into SWE-Smith, SWE-Flow, SWE-Rebench and Multi-SWE-RL repositories by model rewriting, semantic perturbation and rule-based transforms; a bug is kept only if it fails existing tests and is fixed by reverting the patch; issue text is generated and bug-triggering tests hidden.
Run on MegaFlow, an Argo-workflow stack on Alibaba Cloud Kubernetes: rollout, evaluation and post-processing per task, agent and environment containers co-located in one pod.
Schedule: Trillions of tokens; context extended from 32,768 to 262,144 for repository-level data and long trajectories; next-token plus FIM objectives.
Best-fit packing instead of concatenate-then-split; highly repetitive segments (headers, config blocks) masked from the loss.
Finding: Within a scaffold, SWE-bench performance rises steadily with mid-training tokens (1B → 8B tested). Across scaffolds transfer is weak: OpenHands trajectories transfer poorly to SWE-Agent, the reverse moderately — framework specialisation is a real trade-off.

3SFTverified trajectories · judge-ranked

Data: Three sources: in-house proprietary corpora (alignment quality, safety); agentic trajectories validated step by step through execution; documentation-grounded open-domain QA with candidate answers filtered for functional correctness and security.
Sizes are not given. SFT and RL prompts are fully disjoint to stop leakage between stages.
Filtering: A Mini-SWE-agent-based user simulator executes the assistant's proposed code or commands and reads compiler output, runtime errors and environment state to decide whether the response actually advances the task — closed-loop verification that removes hallucinated or non-functional answers.
Preference as data: For each request, n candidates from the strongest in-house models; every pair is scored by a dedicated pairwise judge trained on a multi-dimensional checklist (accuracy, usefulness, style) to produce an ordinal ranking; SFT trains on the ranked data.
Reported gains: stylistic consistency, clarity, proactive engagement. There is no DPO or RL preference stage — preferences select SFT samples.

4Expert climbs4 experts · 2 by RL

Web Development expert: SFT on execution-valid trajectories. Every sample is rendered in Playwright-controlled Chromium (a Vite server for React); a VLM scores screenshots against a checklist (layout, completeness, UI quality), then DOM-parsed actions are executed and the VLM compares before / after screenshots to check behaviour.
User Experience expert: Targets tool-call format adherence across CLI / IDE scaffolds (Cline, Qoder, OpenCode, …) whose schemas differ; rule-based validation of tool-call format turned out to be the single most effective filter.
Template diversity at fixed data volume raises SWE-bench Verified; template-following accuracy across community environments averages 92.7 vs 79.3 for Kimi-K2-thinking.
Single-turn RL expert: Execution-verifiable tasks beyond competitive programming: library and API use, I/O and data formats, multiple languages, secure coding and vulnerability repair.
Unit tests synthesized by internal models and kept by majority-vote consensus across independently generated solutions; scaling task diversity improved every coding sub-capability.
Software-engineering RL expert: Multi-turn agentic RL on open datasets plus the in-house environments; per-instance pass rates estimated first, then too-easy and noisy instances dropped so training sits on informative failures.
Trajectory-level completion reward, an unfinished-trajectory penalty past the turn cap, and a turn-level tool-format penalty applied to the tokens of invalid calls.
Reward-hacking blocker: with remotes, branches and tags already removed, agents learned to re-add remotes, clone history or curl the fix; any tool call combining a repository link with git / curl / wget is blocked with explicit feedback. Long-horizon behaviour emerged during RL — average agent turns grew from 50 to 130.
Algorithm: Not named anywhere in the report — no GRPO, GSPO or PPO. The only 2026 Qwen document that names one is Qwen3.5-Omni, and there GSPO is the speech Talker's RL algorithm, not the Thinker's.

5Expert distillation4 experts → 1

Note: The Web Development, User Experience, single-turn RL and software-engineering experts are distilled into the SFT model, which “inherits the strengths of individual experts while preserving the strong instruction following capability of the base SFT model”; one model serves all domains with no expert routing.
The method is not specified and on-policy distillation is never mentioned, so read it as SFT on expert outputs — the “Off-Policy Finetune” baseline in the MOPD paper's terms.
Results: SWE-bench Verified 70.6 / 71.1 / 71.3 with SWE-Agent / Mini-SWE-agent / OpenHands, against DeepSeek-V3.2 70.2, GLM-4.7 74.2, MiniMax-M2.1 74.8 and Kimi-K2.5 73.2 at ten to thirty times the active parameters.
SWE-bench Multilingual 62.8, SWE-bench Pro 42.7, Terminal-Bench 2.0 36.2 (Terminus-2 JSON), LiveCodeBench v6 58.9 vs 51.8 for the Qwen3-Next it started from.

Diff vs. OLMo 3

The specialist-then-distil skeleton at 3B active, from the lab that also ships a 2.4T model: the same shape as DeepSeek V4 and Kimi K3, with the consolidation step left unnamed.
No direct-alignment stage at all — preferences are consumed by a checklist judge to select SFT samples, not by DPO or a reward model.
The environments are the product: ~800K verifiable SWE tasks stored as Docker images and a Kubernetes rollout stack, where OLMo's verifiers are functions.
Two of the four experts are trained by SFT, not RL; the scaling claims are about task diversity and mid-training tokens, not RL steps.
Coding-only, so no math or general reasoning climb, and no thinking mode; it stands in here because it is the only 2026 Qwen text model with a published post-training recipe.

What the later Qwen releases disclose

Qwen3.8-Max (Aug 2026, blog only): 2.4T-A95B on the Qwen 3.5 architecture. The single post-training paragraph describes environments scaled along three axes (task: single → multi-task → multi-day; workspace: files → heterogeneous folders; harness: QwenWork / Claude Code / Codex / OpenClaw / Hermes), a “Universal Reward System” unifying execution checks, rubric adjudication over text and rendered output, and agentic inspection, and an online data balancer over task, difficulty, workspace and harness per batch. reasoning_effort low / medium / xhigh; preserve_thinking on by default.
Qwen3.8-Next architecture (Sept 2026): pretraining only — 125B-A6B, Gated DeltaNet + global attention 3:1, Qwen Sparse Attention at continued pretraining, 51B of n-gram embedding tables off-accelerator. One post-training remark: a sparsity variant with unchanged pretraining metrics was dropped because quality “degraded clearly after post-training”.
Qwen3.5-Omni (Apr 2026): the Thinker's recipe is specialist teachers (SFT + RL from the Qwen 3.5 base) whose generated data is distilled into one model, on-policy distillation from the model's own text-conditioned responses into the audio-conditioned setting, then an interaction-aligned RL stage with no algorithm named. GSPO appears only in the Talker's speech pipeline (DPO on human multilingual pairs, then rule-based rewards with GSPO).

ETH · EPFL · CSCS · Sep 2025 / Jul 2026 ·source · arXiv · alphaXiv

Apertus 1.0 → 1.5 #

ThesisCompliance and multilinguality, fully open. 1.0 puts the contribution upstream of post-training and aligns with an offline absolute-reward method; 1.5 adds a thinking mode whose recipe is still unpublished.

Architecture

8B and 70B dense · xIELU activation, QK-norm, GQA · AdEMAMix optimizer
15T tokens, ~40% non-English across 1,800+ languages · 131k byte-level BPE
retroactive robots.txt opt-out · Goldfish loss masks 2% of tokens by 50-token context hash
1.5: continued pretraining +4T (8B) / +2T (70B) multimodal tokens · image + experimental audio · 262k context · Apache 2.0

Openness

weights · yespretrain data · yespost-train data · yescode · yescheckpoints · yesrecipe · partial

Stages

Pretrain15T · compliant
SFT4.18M examples · 8 mixture iterations
QRPOoffline · absolute rewards
1.5 cont. pretrain+4T / +2T · multimodal
1.5 post-trainrecipe not yet published

Pipeline

Stage by stage

1Pretrain15T · compliant

Note: 15T tokens from openly available sources with retroactive robots.txt exclusions, PII and toxicity filtering, and the Goldfish objective to suppress verbatim memorization.

2SFT4.18M examples · 8 mixture iterations

Data: 4,184,087 examples after filtering.
Foundation 400k (OLMo2 WildChat 299k, Personas 29k, SciRiff 30k, TableGPT 25k, CoCoNot 11k, OASST1 7k).
Math 486k (Llama-Nemotron math 200k, Tülu3 Personas math 126k, NuminaMath with tool calls extracted from solutions 63k, OpenMath GSM8K 50k, Llama-Nemotron chat/safety 47k).
Code 378k (Llama-Nemotron code 200k, Glaive function calling 113k, xLAM 60k, APIGen 5k).
Multilingual 1.43M (SmolTalk2 1.27M across 8 languages, EuroBlocks 157k, s1k translated to 42 languages 1k).
Regional 943k (multilingual Wikipedia QA 884k, Romansh 46k across six written varieties, Swiss-German 6k, African languages 7k, Swiss Charter QA 226).
The-Tome financial/web 545k.
Over 149 languages in the mixture.
Distilled from: No new teacher runs.
Reasoning-bearing data arrives second-hand through Llama-Nemotron's released math and code sets (DeepSeek-R1 traces by proxy) and NuminaMath.
Filtering: Licence filtering keeps only redistributable, commercial-use sources (CC-BY, Apache), excludes NC/SA, robots.txt opt-outs (Reddit, StackExchange) and mis-relicensed mixtures.
Ablation on an 8B 10T checkpoint: decontamination is free, but licence filtering costs 5.8% average and halves MMLU-CoT (0.513 → 0.253) — accepted as the price of compliance.
Keyword removal of other labs' identity markers.
Decontamination by 8-gram match then Ratcliff-Obershelp overlap, which catches translated benchmark items that hashing misses.
Schedule: Global batch 512 / 1,024 with lr 5e-6 / 2e-6, linear decay, max sequence 4,096, AdEMAMix \((\beta _{3} 0.99, \alpha 8)\).
TRL + DeepSpeed; chat template adds a developer message for tools.
Pipeline released at github.com/swiss-ai/posttraining.

3QRPOoffline · absolute rewards

Data: Prompts from the OLMo 2 preference mix minus crawl-forbidden and non-permissive subsets (Flan v2, No Robots).
Qwen3-32B labels prompts as ideologically controversial (73% agreement with 800 human labels) → 380,537 standard and 72,698 controversial, the latter extended with PolygloToxicityPrompts (WildChat subset) and PRISM values/controversy conversations.
Synthetic: Completions from Llama 3.1 8B, Llama 3.3 70B, Qwen2.5 72B, Qwen3 14B and Qwen3 32B at temperature 1 — one with the default system prompt and one steered toward truthful / helpful / honest (UltraFeedback-style) — plus a Qwen2.5-72B Swiss-AI-Charter persona and 10 samples from the Apertus-SFT model itself.
Two are kept per prompt: one scoring above every on-policy sample and one below the 20th percentile of them, since quality within a reasonable range barely mattered and the extremes helped.
Controversial prompts use PersonaHub personas instead of the helpfulness prompts.
Rewards / verifiers: Standard topics: Skywork-Reward-V2-Llama-3.1-8B (trained on 26M preference pairs).
Controversial topics: Qwen3-32B as judge with the full Swiss AI Charter in the prompt — 11 articles of 3–9 numbered clauses (response quality, knowledge standards, respect, harm prevention, value conflicts, professional boundaries, collective decision-making, autonomy, long-term orientation, human agency, AI identity) — scoring 1–9 with clause citations.
The Charter was validated by 163 Swiss residents: 97.3% average agreement, lowest 92.6% for professional-licensing boundaries.
Algorithm: Quantile Reward Policy Optimization: optimizes an absolute reward signal offline, keeping DAA-style stability and cost while avoiding DPO's relative-only signal and its tendency to push probability mass toward out-of-distribution samples.
Accepts reward-model scores, human rankings or judge scores.
DPO trained on the same pairs for ablation only.

41.5 cont. pretrain+4T / +2T · multimodal

Note: Continued pretraining of 1.0 on a multimodal mix: +4T tokens for 8B, +2T for 70B; image input, experimental audio; context 262k.

51.5 post-trainrecipe not yet published

Note: The model card says only “our improved post-training recipe … allows developers to enable a thinking mode.” No stage names, algorithm, or trace sources; the research page lists the 1.5 technical report as coming soon.
Whether RL was added is unknown.

Diff vs. OLMo 3

1.0 has no verifiable-reward stage — the only model here without one. 1.5 ships a thinking mode, which usually implies reasoning RL or trace distillation, but nothing published confirms either.
QRPO with absolute rewards instead of DPO; written-down principles as the judge rubric instead of human comparisons.
Matches OLMo on openness, but the openness serves a compliance argument rather than a reproducibility one — and the report quantifies what compliance costs.
Right reference if the question is “how do you build a corpus you're allowed to train on”; wrong one for post-training.

DeepSeek · Apr 2026 · report Jun 2026 ·source · arXiv · alphaXiv

DeepSeek V4 #

ThesisMillion-token context at low cost, and a post-training pipeline where the mixed RL stage is deleted: train one specialist per domain and per reasoning effort, then merge more than ten of them into one model by full-vocabulary on-policy distillation.

Architecture

Pro 1.6T total · 49B active; Flash 285B · 13B active. DeepSeekMoE + MTP retained; Muon optimizer; mHC hyper-connections constrained to the Birkhoff polytope
Hybrid attention: Compressed Sparse Attention (KV compressed along the sequence + DSA top-k) interleaved with Heavily Compressed Attention (dense over a heavier compression); Flash: 43 layers, \(d=4096\), first two layers sliding-window
1M context · 128K vocab · MIT license for weights and code

Openness

weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · norecipe · partial

Stages

Pretrain>32T tokens
SpecialistsSFT + GRPO per domain × effort
OPD merge>10 teachers → 1 student
InfraFP4 QAT · DSec sandboxes

Pipeline

Stage by stage

1Pretrain>32T tokens

Data: More than 32T tokens: math and code remain core, agentic data is added during mid-training, long documents (scientific papers, technical reports) are prioritized, multilingual corpus enlarged.
Web data filtered for batched auto-generated and templated content to avoid model collapse.
Same tokenizer family as V3 with a few new context-construction tokens.

2SpecialistsSFT + GRPO per domain × effort

Algorithm: Each specialist is fine-tuned then trained with GRPO on domain-specific prompts and rewards, following the V3.2 pipeline with hyperparameters kept close to R1.
Domains named on the model card: mathematics, coding, agent, instruction following.
Note: Reasoning effort is a training-time property, not a prompt trick: separate specialists are trained under different RL length penalties and context windows to produce Non-think, Think High and Think Max.
Think Max additionally injects a fixed system instruction (“Absolute maximum with no shortcuts permitted…”).
All three share the <think>…</think> format.
Rewards / verifiers: Easy-to-verify tasks use rules and test cases.
For hard-to-verify tasks there is no scalar reward model at all: rubric-guided RL data is scored by a Generative Reward Model, and RL is applied to the GRM itself — the actor network is the judge, so judging and generating are optimized together.
Claimed to need only a minimal set of human annotations.
Synthetic: New |DSML| XML tool-call schema (fewer escaping failures than JSON).
Interleaved thinking: in tool-calling conversations all reasoning is preserved across user turns (V3.2 flushed it on each new user message); in plain chat it is still discarded.
Quick Instruction: special tokens appended to the input (<|action|>, <|title|>, <|query|>, <|authority|>, <|domain|>, <|read_url|>) run auxiliary chatbot tasks by reusing the existing KV cache instead of a separate small model.

3OPD merge>10 teachers → 1 student

Algorithm: Loss = \(\sum _{i} w_{i} \cdot \mathrm{KL}(\pi _{\theta} \| \pi _{E_i})\) over student-sampled trajectories — reverse KL, so the student selectively matches whichever expert is relevant to the prompt.
More than ten teachers across domains distill into one model.
The report argues the usual shortcut — a per-token log-ratio used as an RL advantage — gives high-variance gradients, and instead computes the KL over the full vocabulary logits.
Finding: Presented as avoiding the degradation seen with weight merging or mixed RL when many capabilities are combined.
This is the same expert-then-merge shape as Kimi K3's MOPD, implemented with full logits rather than a token-level estimate.

4InfraFP4 QAT · DSec sandboxes

Scale: FP4 QAT through post-training: MXFP4 on MoE expert weights and on the CSA indexer's QK path, FP4→FP8 dequantization shown to be lossless so the FP8 training stack is reused unchanged; rollouts run native FP4 so sampling matches deployment.
Teacher scheduling for full-vocabulary OPD: teacher weights offloaded to distributed storage and loaded on demand, only last-layer hidden states cached, logits reconstructed through the prediction head, samples ordered by teacher so one head is resident at a time, exact KL in a TileLang kernel.
Note: Rollouts are preemptible and fault-tolerant via a per-request token-granular write-ahead log plus saved KV cache; the report notes that regenerating interrupted requests from scratch would bias the model toward shorter outputs, since short responses survive interruptions more often.
DSec sandbox platform: three Rust services over the 3FS filesystem, four substrates behind one SDK (pre-warmed function-call pool, Docker containers on EROFS layers, Firecracker microVMs, QEMU full VMs), hundreds of thousands of concurrent sandboxes per cluster, ordered trajectory logs enabling preemption-safe fast-forward and deterministic replay.

Diff vs. OLMo 3

There is no mixed RL stage and no DPO: specialists are trained separately and merged by on-policy distillation, versus OLMo's single SFT → DPO → RLVR line.
The reward model for non-verifiable tasks is the policy itself, trained as a judge — no separate scalar RM.
Reasoning effort is baked in by training separate specialists with different length penalties, not by a system prompt alone.
Weights are MIT, but no data and only inference code — the recipe is documented at the algorithm level, not the dataset level.

Google DeepMind · Apr 2026 · report Jul 2026 ·source · arXiv · alphaXiv

Gemma 4 #

ThesisFrontier-open capability inside consumer memory budgets. The post-training section is two paragraphs: “a similar post-training approach as in Gemma 3”, plus a thinking mode — so the recipe below is Gemma 3's, with the report's own additions marked.

Architecture

E2B · E4B · 12B · 26B-A4B (MoE) · 31B dense; Apache 2.0
12B is encoder-free: 48×48 image patches through a 35M matmul, raw 40 ms audio chunks projected straight into the embedding space; other sizes keep frozen vision/audio encoders
MTP drafter for speculative decoding; QAT checkpoints (mobile int2/int4 mix, Q4_0); 262k SentencePiece vocab; data cutoff Jan 2025

Openness

weights · yespretrain data · nopost-train data · nocode · nocheckpoints · partialrecipe · partial

Stages

Pretrain“similar to Gemma 3”
Distillation SFTfrom a large IT teacher (Gemma 3 recipe)
RLBOND · WARM · WARP (Gemma 3 recipe)

Pipeline

Stage by stage

1Pretrain“similar to Gemma 3”

Data: Web documents, code, images and audio (audio for E2B/E4B/12B) with a January 2025 cutoff; filtered for benchmark contamination, unsafe content and recitation risk.
Token counts are not given.
Trained on TPU pods with ZeRO-3 optimizer sharding, Pathways, GSPMD and MegaScale XLA.

2Distillation SFTfrom a large IT teacher (Gemma 3 recipe)

Distilled from: Per the Gemma 3 report: an improved version of knowledge distillation from a large instruction-tuned teacher, in the lineage of Agarwal et al.
2024 (on-policy distillation) and Hinton-style KD.
The teacher is not named in either report; a Gemini-class model is the natural reading.
Filtering: Gemma 4's own text: drop examples with personal information, unsafe or toxic outputs, mistaken self-identification, duplicates; add subsets encouraging in-context attribution, hedging and refusals, which improve factuality without hurting other metrics.
Note: New in Gemma 4: a thinking mode, where the model emits a reasoning trace before answering; activation is documented in the chat-format table.
How thinking traces were sourced or trained is not stated.

3RLBOND · WARM · WARP (Gemma 3 recipe)

Rewards / verifiers: Per Gemma 3: a mixture of reward functions targeting helpfulness, math, coding, reasoning, instruction following and multilingual ability while minimizing harm — weight-averaged reward models (WARM) trained on human feedback, code execution feedback, and ground-truth rewards for math problems.
Algorithm: Improved versions of BOND (best-of-N distillation into the policy), WARM (weight-averaged reward models) and WARP (weight-averaged rewarded policies).
Preference and verifiable signals are mixed in one RL phase rather than split into DPO-then-RLVR stages.

Diff vs. OLMo 3

Preference and verifiable rewards live in a single RL phase built on weight-averaging (BOND/WARM/WARP), versus OLMo's DPO stage followed by a separate RLVR stage.
The teacher is a proprietary in-house model, like Phi-4's o3-mini — the recipe cannot be reproduced outside Google.
Thinnest post-training disclosure of any 2026 report here; almost everything above is inherited from the Gemma 3 report.
What it does document well is deployment: QAT variants, an MTP drafter, and an encoder-free multimodal path.

Zhipu · Feb 2026 → Jun 2026 ·source · arXiv · alphaXiv

GLM-5 (→ 5.1 / 5.2 / 5.3) #

ThesisFrom vibe coding to agentic engineering: asynchronous multi-task RL over more than ten thousand verifiable environments, with on-policy distillation across stages so that later RL does not erase what earlier stages learned.

Architecture

744B total · 40B active MoE, scaled from GLM-4.5; DeepSeek Sparse Attention added by continued pretraining; INT4 QAT; trained and adapted on Chinese chip infrastructure
5.1: longer-horizon agentic refinement · 5.2: IndexShare sparse attention, better speculative decoding, flexible-effort reasoning · 5.3: mHC hyper-connections, hybrid sparse/linear attention, plus a 320B-A18B Flash
SFT context 202,752 tokens

Openness

weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · norecipe · yes

Stages

PretrainGLM-4.5 base → DSA
SFTchat · reasoning · coding & agent
Reasoning RLGRPO + IcePop · 4 domains
Agentic RLasync · >10k envs
General RLhybrid rewards
Cross-stage OPDfinal stage

Pipeline

Stage by stage

1PretrainGLM-4.5 base → DSA

Note: Scaled from GLM-4.5 with DeepSeek Sparse Attention introduced through continued pretraining, then mid-training; INT4 quantization-aware training.
Token counts are not the report's focus.

2SFTchat · reasoning · coding & agent

Data: Three categories: general chat (QA, writing, role-play, translation, multi-turn, long context); reasoning (math, code, science); coding and agent (frontend/backend engineering, tool calling, coding agents, search agents, general agents).
Agent and coding data scaled up sharply versus GLM-4.5.
Synthetic: Logical reasoning: verifiable problems with rejection-sampled solutions.
Math and science: keep only problems that are hard for GLM-4.7.
Coding and agent: a large number of execution environments produce trajectories, improved by expert RL and rejection sampling.
Erroneous segments inside trajectories are kept but masked from the loss so the model sees error correction without reinforcing the error.
Role-play data filtered on instruction following, expressiveness, creativity, coherence and long-dialogue consistency, automatically and by humans.
Note: Three thinking behaviours are taught here: interleaved thinking before every response and tool call; preserved thinking that keeps all thinking blocks across turns in coding-agent settings; turn-level thinking that can be switched per request.

3Reasoning RLGRPO + IcePop · 4 domains

Algorithm: GRPO with the IcePop mechanism: the training/inference mismatch ratio \(\rho = \pi _{\text{train}}/\pi _{\text{infer}}\) per token is computed and tokens outside \([1/\beta , \beta ]\) are zeroed out \((\beta = 2)\); the KL term is removed; clip \(\varepsilon _{\text{low}} 0.2\), \(\varepsilon _{\text{high}} 0.28\); fully on-policy with group size 32 and batch size 32.
Finding: With sparse attention the indexer's top-k selection is itself a train/inference mismatch risk.
Non-deterministic CUDA/TileLang top-k kernels collapsed RL within a few steps with a sharp entropy drop; the plain deterministic torch.topk fixed it, and the indexer is frozen during RL.
Data: Mixed training over mathematics, science, code and tool-integrated reasoning, roughly balanced.
Math/science from open sets plus vendor-annotated collections, filtered to problems GLM-4.7 rarely solves but stronger teachers (GPT-5.2 xhigh, Gemini 3 Pro Preview) can.
Code from Codeforces, TACO, SYNTHETIC-2-RL and internal pools decomposed into minimal implementations.
TIR reuses the hardest math/science items plus vendor-built STEM questions designed to need tools.
Rewards / verifiers: Binary outcome rewards from domain- and source-specific judge models or evaluation systems.

4Agentic RLasync · >10k envs

Algorithm: Group-wise objective: reward minus the group mean over K traces, optimizing only model-generated tokens.
Fully asynchronous: inference engine streams trajectories, training engine updates once a threshold batch is reached and pushes weights every K updates; the optimizer is reset after each rollout-engine weight update.
Direct double-sided importance sampling: the ratio is taken against the rollout log-probabilities (no stored old-policy), and tokens outside \([1-\varepsilon _{l}, 1+\varepsilon _{h}]\) are masked entirely.
Samples whose oldest rollout version lags the current policy by more than \(\tau\) are dropped; environment-crash failures are excluded and groups padded or dropped.
Scale: Multi-Task Rollout Orchestrator: each task is a microservice with its own rollout and reward logic, trajectories standardized to a message list, per-task sampling ratios adjusted dynamically, over 1k concurrent rollouts.
A Token-in-Token-out gateway records exact token IDs to avoid re-tokenization drift.
DP-aware routing pins each rollout to one data-parallel rank by consistent hashing so multi-turn prefixes hit the KV cache.
Environments: Software engineering: real Issue–PR pairs filtered and categorized, RepoLaunch builds executable environments and extracts Fail-to-Pass / Pass-to-Pass tests — over 10k environments across thousands of repositories in 9 languages.
Terminal: tasks synthesized from seeds into Harbor format (Docker, test scripts) by a construction agent and refined by a rubric agent, >90% image-build success; a second pipeline turns high-quality code web pages into terminal tasks that the constructing agent validates against itself.
Search: a Web Knowledge Graph from over two million pages, multi-hop questions from expanded subgraphs, three-stage filtering (drop if a tool-free model answers in 1 of 8 tries, drop if an early agent solves it quickly, bidirectional answer verification).
Slide generation: an HTML slide environment with three reward levels — static markup rules, runtime DOM geometry from a distributed renderer, perceptual whitespace checks — with renderer fixes for reward hacks such as truncating overflow or padding spacing; 16:9 compliance rose from 40% to 92%.
Note: Search inference uses context management: keep only the most recent \(k=5\) tool observations and reset the whole tool history past 32k tokens; BrowseComp 55.3 → 62.0 from keep-recent alone, 75.9 with the hybrid.

5General RLhybrid rewards

Rewards / verifiers: Three objectives — foundational correctness (instruction following, logic, facts, hallucination, fluency), emotional intelligence, task-specific quality (writing, text processing, QA, role-play, translation) — supervised by a hybrid of rule-based rewards, outcome reward models and generative reward models, chosen to balance precision, efficiency and hack-resistance.
Note: Expert human-written responses are injected as stylistic anchors because purely model-generated optimization drifts toward verbose, formulaic “model-like” text.

6Cross-stage OPDfinal stage

Algorithm: On-policy distillation where the teachers are the final checkpoints of the preceding stages (SFT, Reasoning RL, General RL) and prompts are drawn from those stages' RL sets.
The GRPO advantage is replaced by \(\operatorname{sg}[\log \pi _{\text{teacher}}(y_{t}) / \pi _{\text{train}}(y_{t})]\) per token; group size 1 and batch 1024 since no group is needed to estimate advantages.
Finding: Motivated by cumulative degradation across sequential RL objectives — the same problem DeepSeek V4 solves by merging domain specialists, here solved by distilling from one's own earlier stages.

Diff vs. OLMo 3

Four post-SFT stages (reasoning RL, agentic RL, general RL, cross-stage OPD) against OLMo's one, with a different GRPO variant in each: IcePop masking, direct double-sided IS, hybrid-reward general RL.
Environment construction is the core contribution — >10k SWE environments, synthesized terminal tasks, a web knowledge graph for search, a rendered-slide reward stack — versus OLMo's four verifier types.
On-policy distillation appears twice in 2026 (DeepSeek V4 across specialists, GLM-5 across stages) and nowhere in the 2025 reports.
Recipe disclosure is close to OLMo's on the algorithm side; data and checkpoints are not released.

All ten on one schema

Compare#

Rows are the axes; OLMo 3's column is the reference and is tinted. Read across a row to see what the field disagrees about; read down a column to see one lab's bet.

	OLMo 3	Phi-4-reasoning	Nemotron 3 Super	Llama-Nemotron	Kimi K3	Qwen3-Coder-Next	Apertus 1.0 → 1.5	DeepSeek V4	Gemma 4	GLM-5
Base / architecture	7B and 32B, dense	14B dense (Phi-4 base; chosen over the mid-train checkpoint for its safety post-training)	120B total · 12B active · LatentMoE	Nano 8B · Super 49B · Ultra 253B	2.8T total · 104B active · 896 experts, top-16 (“Stable LatentMoE”)	80B total · 3B active MoE on the Qwen3-Next base (hybrid attention); 262k context; base + instruct released	8B and 70B dense · xIELU activation, QK-norm, GQA · AdEMAMix optimizer	Pro 1.6T total · 49B active; Flash 285B · 13B active. DeepSeekMoE + MTP retained; Muon optimizer; mHC hyper-connections constrained to the Birkhoff polytope	E2B · E4B · 12B · 26B-A4B (MoE) · 31B dense; Apache 2.0	744B total · 40B active MoE, scaled from GLM-4.5; DeepSeek Sparse Attention added by continued pretraining; INT4 QAT; trained and adapted on Chinese chip infrastructure
SFT scale	Dolci Think SFT · 2.27M	1.4M pairs · 8.3B unique tokens	7M samples · 80B tok	33.0M samples · R1-distilled	cold start · sizes undisclosed	not given; three sources, execution-verified by a user-simulator agent and ranked by a checklist judge; SFT and RL prompts disjoint	4.18M examples · 8 mixture iterations	SFT + GRPO per domain × effort	from a large IT teacher (Gemma 3 recipe)	chat · reasoning · coding & agent
Teachers	QwQ-32B + DeepSeek R1 (SFT); Qwen3 32B/0.6B (DPO pairs)	o3-mini (proprietary)	DeepSeek v3.2, Kimi K2, Qwen3-Coder-480B, GPT-OSS-120B, MiniMax M2/M2.5, GLM-4.7, R1	DeepSeek-R1 (on); Qwen2.5-Math-7B, Llama-3.x-70B (off)	prior Kimi experts (SFT); 9 own RL experts (MOPD)	Qwen3-Coder-480B-A35B-Instruct (CC rewriting, QA, six-scaffold trajectories); its own four experts (distillation)	none new; second-hand via Llama-Nemotron sets	>10 own domain specialists (OPD)	large in-house IT teacher (Gemini-class, unnamed)	GLM-4.7 for filtering; GPT-5.2 / Gemini 3 as solvability bar; own stage checkpoints (OPD)
Synthetic pipelines	persona IF, GPT-4.1 test cases, rewritten chat refs; Delta-Learning pairs	seed rewriting into verifiable forms; RAI-guided safety	~12 pipelines: CLI, SWE, long-ctx, finance 366k, CUDA 100k, safety, search, terminal 85k, SQL 96.5k, conv. tools 279k, general tools 1.5M	AoPS / code / science / general pipelines → 33M samples; Feedback-Edit for chat	knowledge-graph task synthesis; mock Gmail/Notion/Slack/Canvas; agent-built workspaces	~800K verifiable SWE tasks (PR mining with agent-built Docker verifiers; bug injection into SWE-Smith / -Flow / -Rebench / Multi-SWE-RL); CC rewriting; grounded QA; agent trajectories; FIM; unit tests by majority vote	preference completions from 5 open models + charter persona; 1.5 undisclosed	rubric-guided GRM data; \|DSML\| tool schema; Quick Instruction tokens	not disclosed beyond filtering rules	>10k SWE envs (RepoLaunch), Harbor terminal tasks, web knowledge graph search QA, slide renderer
Preference stage	DPO · Delta Learning · 200k	none	RLHF · principle GenRM · 18B tok	online RPO (Super) / GRPO (Ultra) · 70B reward model	tournament GenRM inside RL	none as a stage — a pairwise checklist judge ranks candidates to select SFT data	QRPO · Skywork-RM + Qwen3-32B Charter judge	none — GRM is the actor itself, trained jointly	RL with WARM reward models (BOND/WARM/WARP)	General RL: rules + ORM + GRM + human exemplars
Verifiable-reward RL	GRPO (OlmoRL) · 105k prompts · 4 domains, 5 verifiers incl. LM judge	GRPO · ~6.4k math · rule-based length-aware reward	async GRPO · 21 envs / 37 datasets + SWE-RL containers	GRPO · science · Ultra only; RLOO for IF	partial-rollout RL · white-box harnesses, kernels, assistants, visual sandboxes, search	two RL experts: single-turn unit-test RL beyond competitive programming · multi-turn SWE RL with pass-rate filtering, turn/format penalties and a reward-hacking blocker (turns 50 → 130); algorithm unnamed	none in 1.0 · 1.5 unknown	GRPO per specialist, then full-vocab OPD replaces mixed RL	in the same RL phase: code execution + math ground truth	IcePop-GRPO (4 domains) + async agentic RL (double-sided IS)
Reasoning control	separate Think / Instruct / RL-Zero paths	think tokens, always on	off / regular / low-effort + budget	system prompt on/off	effort budget \(\tau\): low/high/max	none in the report (no thinking mode); Qwen3.8-Max adds reasoning_effort low / medium / xhigh	1.5: switchable thinking mode	Non-think / Think High / Think Max, trained as separate specialists	thinking mode (new in 4)	interleaved · preserved · turn-level thinking; 5.2 flexible effort
Thesis	The model flow is the product: every stage, dataset, checkpoint and the RL code are released so the recipe itself can be studied, forked, and benchmarked against.	Curation beats scale: a 14B model approaches DeepSeek-R1 by SFT-ing on “teachable” prompts with o3-mini traces; a 90-step GRPO pass on ~6k math problems adds a further increment.	Throughput at parity: match GPT-OSS-120B and Qwen3.5-122B on benchmarks while serving 2–7× more tokens per GPU — with an industrial synthetic-data operation feeding a four-stage RL pipeline.	Efficient reasoning: derive the architecture from Llama 3 by NAS, distill 33M samples of reasoning from DeepSeek-R1 with an on/off toggle, and spend RL only on the largest model where it can surpass the teacher.	Open frontier intelligence: train nine domain × effort RL experts on white-box agentic environments at million-token context, then merge them back into one model by on-policy distillation.	Scale the agentic training, not the model: ~800K executable tasks feed mid-training and RL, four experts climb from one SFT checkpoint, expert distillation folds them into one 3B-active model.	Compliance and multilinguality, fully open. 1.0 puts the contribution upstream of post-training and aligns with an offline absolute-reward method; 1.5 adds a thinking mode whose recipe is still unpublished.	Million-token context at low cost, and a post-training pipeline where the mixed RL stage is deleted: train one specialist per domain and per reasoning effort, then merge more than ten of them into one model by full-vocabulary on-policy distillation.	Frontier-open capability inside consumer memory budgets. The post-training section is two paragraphs: “a similar post-training approach as in Gemma 3”, plus a thinking mode — so the recipe below is Gemma 3's, with the report's own additions marked.	From vibe coding to agentic engineering: asynchronous multi-task RL over more than ten thousand verifiable environments, with on-policy distillation across stages so that later RL does not erase what earlier stages learned.
Openness	weights · yespretrain data · yespost-train data · yescode · yescheckpoints · yesrecipe · yes	weights · yespretrain data · nopost-train data · nocode · nocheckpoints · norecipe · partial	weights · yespretrain data · partialpost-train data · yescode · partialcheckpoints · yesrecipe · yes	weights · yespretrain data · nopost-train data · yescode · yescheckpoints · partialrecipe · yes	weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · norecipe · partial	weights + base checkpoint, eval code; no data, no training code; recipe partial (RL algorithm and distillation method unnamed)	weights · yespretrain data · yespost-train data · yescode · yescheckpoints · yesrecipe · partial	weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · norecipe · partial	weights · yespretrain data · nopost-train data · nocode · nocheckpoints · partialrecipe · partial	weights · yespretrain data · nopost-train data · nocode · partialcheckpoints · norecipe · yes

What the table makes visible

Preference tuning has fragmented. Offline DPO with machine-made contrasts (OLMo), offline QRPO on absolute rewards (Apertus), a GenRM inside the RL loop (Nemotron, Kimi), online RL against a reward model (Llama-Nemotron), nothing (Phi-4). There is no consensus stage two.
GRPO is the consensus name, not the consensus algorithm. OLMo drops KL and std-normalization and adds truncated importance sampling; Nemotron masks the IS ratio and runs one step stale; Phi-4 keeps KL and entropy terms; Kimi pauses rollouts mid-flight; Llama-Nemotron uses RLOO for one stage and RPO for another.
Difficulty targeting is universal and named differently everywhere. Phi-4's “teachable” seeds, OLMo's 62.5% pass-rate cutoff, Llama-Nemotron's 0.75 cutoff plus Gaussian curriculum, Nemotron's “drop what the SFT model always solves”, Kimi's per-problem budgets. Same object.
Synthetic data has become an industrial function. Nemotron names about a dozen generation pipelines with counts; Llama-Nemotron ships 33M synthetic samples; OLMo publishes exact prompt tables. Apertus is the outlier that reuses rather than generates — by policy.
Expert-then-merge is the 2026 pattern. Kimi K3 (nine effort×domain experts → MOPD), DeepSeek V4 (more than ten specialists → full-vocabulary OPD, replacing the mixed RL stage outright), GLM-5 (cross-stage OPD from its own earlier checkpoints). None of the 2025 reports do this; all three 2026 frontier reports do.
Openness tracks thesis. The two fully-open models are the two whose thesis is about openness. Product models release weights and keep the data.

Sources are the linked reports; “not disclosed” means absent from them. Pipeline boxes are not to scale — stage volumes are in mixed units and are labeled instead. Counts are quoted as printed in each report; where a report's table and prose disagree (Apertus SFT: 4.18M vs “~3.8M”) the table is used.