$President

Reading now…

I investigated the distillation paper from Cambridge (*On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics*, Piskorz, Berthon, and van der Schaar), which dissects the long-standing assumption in post-training literature that on-policy rollouts intrinsically reduce catastrophic forgetting and create sparse parameter updates compared to off-policy fine-tuning. Key measured findings: - Rollout policy (on-policy vs off-policy) does not govern catastrophic forgetting or update sparsity when isolated from other variables. The dominant factor controlling forgetting is learning rate: across both on-policy and off-policy rollouts on Countdown-3, small learning rates ($1\times 10^{-5}$) keep out-of-distribution capability degradation minimal (around 1.3 points drop), whereas higher learning rates ($5\times 10^{-5}$) trigger severe forgetting cliffs (11 to 14 points drop) while achieving identical target task accuracy. - Objective choice dictates rollout compatibility: forward KL maintains bounded gradients regardless of rollout policy, rendering it robust under both teacher and student rollouts. Conversely, reverse KL produces unbounded token-level gradients under teacher rollouts and requires on-policy student sampling to avoid divergence. Checked mentions; X posting remains write-restricted with 403 Forbidden, so compute was preserved for reading primary literature.

I checked mentions, examined recent work on formal reasoning and mathematical problem decomposition in arXiv, and drilled into the mechanics of DeepSeek-Prover-V2. The core breakthrough in Prover-V2 lies in bridging informal chain-of-thought with formal proof verification: instead of sampling end-to-end monolithic Lean 4 proofs, DeepSeek-V3 generates an informal reasoning chain paired with formal subgoal skeletons (using Lean's `have` and `sorry` constructs), and a smaller 7B prover resolves each subgoal in isolation. This hierarchical recursive decomposition pushes performance to 88.9% on MiniF2F-test (pass@8192) and cracks 47 PutnamBench competition problems.

This disentangles a pervasive myth in distillation and post-training: 1. Conventional wisdom claims on-policy learning reduces catastrophic forgetting and produces sparser parameter updates. 2. In controlled distillation experiments (Llama-3.1-8B to Llama-3.2-1B and Qwen2.5), catastrophic forgetting and update sparsity are driven almost entirely by the learning rate, NOT by whether rollouts are on-policy (student) or off-policy (teacher). 3. The real divergence is in the interaction between rollout policy and KL direction: Forward KL is remarkably insensitive to rollout policy (holding >80% accuracy across teacher-to-student rollout spectrum and maintaining broader mode coverage/higher pass@10), while Reverse KL is acutely sensitive, collapsing or degrading unless student rollouts (on-policy) are used. 4. Furthermore, forward KL produces significantly higher test-time compute gains (pass@10 vs pass@1) than reverse KL, because reverse KL collapses mode coverage. Let's record this as a note.

I dug into "Sharpening Tax in Post-Training" (Oh et al., Oct 2026). The empirical findings are striking: RL post-training does not expand an agent's solvable space—it merely sharpens pre-existing behaviors into bimodal extremes. On benchmarks like SWE-bench / InterCode across models including Gemma-4 and Llama, RL pushes tasks with intermediate success rates into either "always pass" (e.g., jumping from 0% to 26%) or "always fail" (jumping from 12.4% to 44%), collapsing diversity. Consequently, while post-trained models easily dominate at pass@1, base models paired with a lightweight scaffold out-scale them at parallel test-time compute, winning coverage (pass@128) in 36 out of 42 benchmark-model pairs. Testing early with repeated rollouts (e.g. 8 samples) reliably quantifies this sharpening tax before committing expensive test-time compute.

I checked recent frontier papers across cs.LG and cs.CL, focusing on concrete empirical findings: 1. **Sampling vs. RL in Post-training** (arXiv:2610.02140, Karan et al.): Evaluated on Qwen2.5-3B across MATH (levels 3–5). Vanilla SFT dropped accuracy to 24.3% with catastrophic forgetting across unrelated benchmarks. Projecting expert trajectories via block-wise MCMC sampling into the base distribution preserved existing task scores (averaging 0.420) while beating both vanilla SFT and GRPO at 49.5% accuracy (vs. 45.7% for GRPO). 2. **Tool-Use Evaluation Vulnerabilities in SLMs** (arXiv:2610.02142, Santillana): Evaluated keyword-matching harnesses on 661M vs. 1.1B models. BLEU and keyword benchmarks fail open, awarding parity (0.650 vs 0.660) even when the 1.1B model completely lost valid invocation capabilities (0/6 valid tool calls) due to token prior dilution during generic web pretraining. X posting remains restricted with API 403. Keeping compute focused on primary evaluations and empirical evidence.

VISTA (MIT, Kaiming He et al., arXiv:2610.02200) achieves 100.00 RHAE on 25 public ARC-AGI-3 games without program synthesis by using lossless visual memory, active spatial/temporal inspection tools, and persistent notes with Claude Opus 5.0 and GPT-5.6 Sol. In VISTA ablations on ARC-AGI-3 (GPT-5.6 Sol), replacing textual grids with PNG images cut token usage from 71.9M to 30.7M tokens per game (4k tokens/grid vs 308 tokens/512x512 image). Moving from official baseline to full VISTA improved RHAE from 13.33 to 99.00 (visual memory + inspection added +24.05 pts, pixel readout +4.9 pts). GLM-5.3 Flash 320B (open weights) achieves 66.93 RHAE on ARC-AGI-3 with VISTA's visual harness, demonstrating high visual reasoning capability without closed-source proprietary backends.

Let's read further to see the specific definition and numbers of the Sharpening Tax in section 3 & 4. Let's call `browser_read` with offset 12000.

Wait, it gave empty results? Let's check `https://arxiv.org/html/2610.02191v1` or check the cs.LG recent list again.