Notes

Pages I generate while reading, short notes on things I keep having to re-derive, and the sources I trust enough to go back to.

1. 

   System 1 verifiers for reasoning RL

   A one-pass judge as the reward inside GRPO: the verifier lineage one figure per paper, DiffusionGemma-as-Jev against live Jev and seven reward arms on GSM8K, and a proposal built on the groups GRPO gets no gradient from.

   page2026-09 Prior workWhat we didProposal
2. 

   Papers from people I like

   New papers from researchers I follow or have met, grouped by month, with alphaXiv links. Refreshed daily.

   pageliveRSS
3. 

   People I know, mapped

   Where the people in my research CRM are, on a map.

   pagelivelist view

1. 

   Verifiable science data and environments

   Ten papers on scalable, verifiable science data for an RL climb beyond code: questions mined from literature (WildSci, BioAlchemy, MegaScience), answers computed by simulators and procedural generators (Sim2Reason, Reasoning Core), executable science environments with synthesised verifiers (D3-Gym, SciDisco, Agent World Model, FrontierChallenge), and NVIDIA's Data Designer pipeline. Nine of the ten are from the last twelve months. A third tab compares how MAI-Thinking-1, Nemotron 3, Kimi K3, DeepSeek-V4 and Qwen3 scope their STEM or math expert.

   note2026-09 The fourAll tenExperts in reports D3-GymalphaXiv Sim2ReasonalphaXiv WildScialphaXiv
2. 

   Self-distillation, and the harness it depends on

   One set of weights under two conditions: the model matches its own predictions given an older self, a prompt it won't have at test time, or the feedback it just got. From Learning-without-Forgetting through context distillation and GKD to the 2026 SDPO family — and the result that on-policy self-distillation can forget more than GRPO under continual post-training. The angle: a deployed capability belongs to a (model, harness) pair, so version and distill the pair.

   note2026-09 LoopMathHarnessPapersPitfalls SDPOalphaXiv Denser ≠ BetteralphaXiv Thinking Machines OPD
3. 

   Latent communication between models

   Two frozen models, a small trained bridge, hidden states or KV cache instead of text. Tabbed: the mechanism and its math, the representation-alignment lineage that says why a linear bridge should exist, fifteen papers from CIPHER to StateBridge in one table, Mostik's 753B→4B claim read against it, and a diffusion tab where DFlash turns out to be the same design with a block-diffusion receiver on a frozen AR trunk.

   note2026-09 MechanismPapersMostikDiffusion Cache-to-CachealphaXiv DFlashalphaXiv Beyond Tokens survey Mostik
4. 

   Apertus 2 Landscape

   Working page on the Swiss AI Apertus 2 post-training effort: what Apertus 1.0 and 1.5 actually published, the programme timeline, the specialists-then-consolidation recipe, the open research questions, and where my own work could fit. Private page — opens only when signed into my Claude account.

   note2026-09privateApertus 1.0 report QRPO
5. 

   LLM report diff

   Ten 2025–26 model reports read against one skeleton (SFT → preference → verifiable-reward RL): per-model pipeline diagrams, stage-by-stage ledgers of data, distillation, synthetic pipelines, rewards and environments, a cross-model table, and the self-play papers' loops. OLMo 3 is the reference.

   note2026-09OLMo 3 Phi-4-reasoning Nemotron 3 Super Llama-Nemotron Kimi K3 Qwen3-Coder-Next Apertus DeepSeek V4 Gemma 4 GLM-5 Compare
6. 

   Post-training algorithms

   Every post-training method in eight families, one card each, cross-linked to the reports that ship them, with the loss written out for the ten that carry the disagreements.

   note2026-09Imitation & distillationReward signalsDirect alignmentPolicy gradient, critic-freePolicy gradient, with criticOff-policy correctionsSampling & curriculumOdds and ends
7. 

   Multi-token prediction (MTP)

   A small extra head predicts the token after next from the same hidden state — denser training signal, and a draft model that lives inside the main model. Two diagrams, the three-line math, where it appears in DeepSeek-V3/V4, Nemotron 3 Super, Kimi K3, Gemma 4 and GLM-5, and why RL breaks it.

   note2026-09 Raschka · architecture gallery Gloeckle et al. 2024alphaXiv DeepSeek-V3 §2.2alphaXiv EAGLE-3
8. 

   Self-play for reasoning

   STaR, Absolute Zero, R-Zero and SPICE as one loop drawn four times. The two slots that differ — where the task comes from and who is allowed to verify — the shared 50%-difficulty target, where the model reports already use pieces of it, and why self-labeling collapses.

   note2026-09

1. 

   Ahead of AI — Sebastian Raschka

   Architecture comparisons and post-training explainers with code. Companion pieces: the LLM Architecture Gallery and the LLMs-from-scratch repo (KV cache in ch04/03_kv-cache, sampling in ch05, MoE in ch04/07_moe).

   source
2. 

   The RLHF Book — Nathan Lambert

   The reference text for post-training: reward models, policy gradients, direct alignment, rejection sampling, on-policy distillation, regularization, evaluation. Updated continuously by the person who publishes the OLMo and Tülu recipes; the baseline every classical entry in the algorithms note is checked against.

   sourceInterconnects (newsletter)OLMo 3Tülu 3
3. 

   alphaXiv

   Full-text arXiv with discussion; swap arxiv.org/abs/ID for alphaxiv.org/abs/ID.

   source
4. 

   Papers with Code → Hugging Face Papers

   Papers with Code was shut down in 2025 and paperswithcode.com now redirects here; the method and leaderboard pages are gone. HF Papers keeps the daily/trending feed and the community signal per arXiv ID. For method-level explainers the nearest living replacement is Raschka's architecture gallery above.

   sourcetrending