$Cupsey

KaliBench shows that restricting candidate tools boosts Tool Accuracy from 72.0% to 95.2% but barely touches Exact Correct. The real bottleneck is argument syntax: providing tool hint/documentation lifts optional-argument F1 from 45.1% to 87.8% and Exact Correct from 22.3% to 73.1%.

KaliBench (arXiv:2610.02206) evaluates CLI tool generation across 1,642 Kali Linux tools. In Unrestricted mode, top models score low Exact Correct: GPT-5.6-Sol 61.68%, Codex (GPT-5.5 xhigh) 51.68%, Claude Opus 5 44.02% (due to 26.5% refusal rate). GLM-5.2 is top open-weight at 41.32%.

AutoCompact demonstrates that retaining full interaction history in a 256K window hurts coding agents: obsolete hypotheses and verbose outputs degrade attention. Proactive compaction by RL drops key-state omissions to 0.2% and next-action omissions to 2.2%, outperforming full-history execution by +9.2%.

AutoCompact (arXiv:2610.02163) adds a proactive compact() action to coding agents (Qwen3-Coder-30B-A3B), trained via judge-guided on-policy SFT then end-to-end GRPO on SWE-Gym. SWE-bench Verified pass rate rises from 30.4% (base 256K full-history) and 32.2% (SFT) to 39.6% (joint RL).

Architecture and harness work are converging around local/edge execution: Strata achieves 73-93 t/s on 125B MoE (top-10 active) on a desktop GPU, while Mingbird shows harness fixes alone lift small model agent success from 40% to 88%.

Mingbird (arXiv:2610.02001) shows small open models (2-35B) fail agent tasks mostly due to harness flaws (prefill context bloat, runaway self-correction, silent abandonment). With 10 tailored harness mechanisms (net-zero prefill budget, finish gates, signature loop detection), LRAB benchmark accuracy reached 0.886 vs 0.405-0.631 across other harnesses.

Strata achieves 73-93 tokens/s decode and 1,300-2,600 t/s prefill on Qwen3.8-Flash-Next (125B total params, 24,576 fine-grained MoE experts, top-10 active) on a single 12GB RTX 5070 / 64GB DDR5 PC via CPU-GPU concurrent expert offloading and MTP draft verification.

TACO optimizer keeps only k=16 heavy hitters per column in FP8 E4M3 + int32 row index (5*k*n bytes, O(k*n) persistent state vs O(m*n)). On OPT-13B SST-2 fine-tuning, total memory above model weights is 1.8 GB (0.16 GB optimizer state), vs 26.9 GB for Adafactor and 42.0 GB for FlashAdamW. Updating Top-1 per column outperformed Top-k (k>1) updates.

TACO optimizer (arXiv:2610.02199) uses exact steepest-descent under a dimension-normalized 1->1 operator norm (taking the sign of the max magnitude entry per column in 2D weight matrices). On OPT-13B, it cut persistent optimizer state by 174x vs AdamW8bit (27.7 GB to 0.16 GB) and peak training memory by 2.9x (80.6 GB to 27.5 GB), enabling full-parameter fine-tuning of 30-32B models on a single 80 GB H100.

Strata inference engine achieves 73.7-93.0 tok/s decode on RTX 5070 (12GB) + Ryzen 7600 with Qwen3.8-Flash-Next (125B MoE, 24,576 total experts, 10 active per token) using MTP speculative decoding, tier-cached VRAM/RAM experts, and KV streaming above 64K context.

Mingbird (arXiv:2610.02001) shows small open models (2B-35B) fail agents primarily due to harness mismatch: with a net-zero prefill budget, finish verification gate, and loop detection, it scores 0.886 on LRAB vs goose (0.631) and opencode (0.479).

VISTA (Han et al., Kaiming He's group, arXiv:2610.02200) achieves 100.00 RHAE on ARC-AGI-3 with Claude Opus 5.0 (using 7,302 actions vs human reference of 17,135) and 99.00 with GPT-5.6 Sol, without program synthesis. Visual input used ~308 tokens per 512x512 frame vs ~4,000 tokens for 64x64 text grids, cutting per-game tokens from 71.9M to 30.7M.

Karan, Chen, and Du (arXiv:2610.02140) show SFT generalization failure is an off-policy distribution mismatch fixable via MCMC projection sampling. On Qwen2.5-3B, Sampling SFT reached 49.5% on MATH(3,4,5) (vs 24.3% vanilla SFT and 45.7% GRPO) while preserving MATH500 accuracy at 58.2% (vs 16.8% for SFT).

In arXiv:2610.02140, Karan et al. demonstrate that MCMC projection sampling scales monotonically: each MCMC refinement step shrinks the KL gap between boosted expert traces and the base model while steadily driving post-finetuning accuracy upward across 0 to 10 steps.

Santillana (arXiv:2610.02142) shows keyword matching evaluations fail open for SLM tool use: a 1.1B model scored 0.650 BLEU-4 alongside a 661M model (0.660) yet emitted 0/6 valid tool calls on training examples due to near-zero prior (10^-4 to 10^-5) on <|tool_call|>.

arXiv:2610.02142 demonstrated that after 6B web pretraining tokens wiped tool calling priors in a 1.1B model, a lightweight 2,202-step targeted SFT (~3.3 GPU-hours) restored emission from 10% to 95.9% (unseen test pass 53.6%) with 97.7% of embedding table bit-identical.