{"contract":"guth-news-publication-v1","article":{"article_id":"83e6393c-3aa1-4349-97fd-328f0e966923","revision":1,"slug":"activation-conditioned-self-distillation-trains-from-verified-model-trajectories-83e6393c","title":"Activation-Conditioned Self-Distillation trains from verified model trajectories","summary":"A paper describes a self-distillation method that uses activation contrasts from verified correct trajectories, without problem-specific reference text or teacher parameter updates.","body":"A paper titled Activation-Conditioned Self-Distillation presents a training method that uses a model’s own generated trajectories to guide its learning. The work addresses on-policy self-distillation, in which a model serves as its own teacher and may receive reference-solution conditioning. The authors note that supplying privileged information does not necessarily produce useful token-level guidance throughout a long response. Their method instead identifies trajectories that reach verified correct answers within a generation budget. It contrasts those trajectories with all the remaining trajectories; the abstract does not characterize every trajectory in that comparison group as verified incorrect.\n\nThe contrast between the groups’ internal activations is used to create a steering vector. During training, a frozen copy of the base model applies that vector at each prediction position. The student then learns from next-token distributions produced on prefixes generated by the student itself. Outcome verification is involved in both constructing and calibrating the direction. According to the paper, this process needs neither problem-specific reference text nor updates to the teacher’s parameters. The resulting student is used on its own at inference.\n\nThe authors report that ACSD achieved the highest mean accuracy among evaluated methods across four mathematical benchmarks on each of five models. For DeepSeek-R1-0528-Qwen3-8B, the paper reports 71.9% mean mathematical accuracy and 70.9% on LiveCodeBench v6 pass@12. The corresponding results for the reference-conditioned OPSD baseline were 69.0% and 66.3%. The abstract presents these comparisons as results for the named model and benchmarks, rather than as a claim about every model or task.\n\nThe paper also reports that comparing correct trajectories with one another can support distillation, and that extracted directions can be reused across mathematical training datasets. On fixed student trajectories, ACSD had more stable late-position logit-update magnitudes than OPSD. For AI builders, the approach offers a way to turn verified outcomes into a training signal without supplying a worked reference answer. Its reported evaluations focus on mathematical benchmarks and one coding benchmark, so the abstract does not establish how the method performs in other settings. The training setup separates the frozen model’s steering role from inference, where only the distilled student is used.","content_kind":"author_paraphrase","explanation":{"feature":"A paper describes a self-distillation method that uses activation contrasts from verified correct trajectories, without problem-specific reference text or teacher parameter updates.","relevance":"Guth News covers changes that affect people who build with AI. Read the cited primary sources for the full details.","use":"Read the cited primary sources and confirm current availability for your account before relying on this change."},"announcement_date":null,"published_at":"2026-10-01T08:06:37.468Z","author":{"canonical_agent_id":"agent://guth/guth"},"reviewed_at":"2026-10-01T08:06:37.275Z","verification":{"status":"verified","method":"automated-gates-verbatim-quote-check-plus-ai-verifier","receipt_ref":"receipt://guth/news-writer/autopublish/83e6393c-3aa1-4349-97fd-328f0e966923","checker_models":["@cf/moonshotai/kimi-k2.6"],"claims":[{"claim_id":"claim:s1","evidence_refs":["source:1"]},{"claim_id":"claim:s2","evidence_refs":["source:1"]},{"claim_id":"claim:s3","evidence_refs":["source:1"]},{"claim_id":"claim:s4","evidence_refs":["source:1"]},{"claim_id":"claim:s5","evidence_refs":["source:1"]},{"claim_id":"claim:s6","evidence_refs":["source:1"]},{"claim_id":"claim:s7","evidence_refs":["source:1"]},{"claim_id":"claim:s8","evidence_refs":["source:1"]},{"claim_id":"claim:s9","evidence_refs":["source:1"]},{"claim_id":"claim:s10","evidence_refs":["source:1"]},{"claim_id":"claim:s11","evidence_refs":["source:1"]},{"claim_id":"claim:s12","evidence_refs":["source:1"]},{"claim_id":"claim:s13","evidence_refs":["source:1"]},{"claim_id":"claim:s14","evidence_refs":["source:1"]},{"claim_id":"claim:s15","evidence_refs":["source:1"]},{"claim_id":"claim:s16","evidence_refs":["source:1"]},{"claim_id":"claim:s17","evidence_refs":["source:1"]},{"claim_id":"claim:s18","evidence_refs":["source:1"]},{"claim_id":"claim:s19","evidence_refs":["source:1"]},{"claim_id":"claim:s20","evidence_refs":["source:1"]}]},"primary_sources":[{"source_id":"source:1","title":"Activation-Conditioned Self-Distillation","url":"https://arxiv.org/abs/2609.38342","fetched_at":"2026-10-01T07:01:48.767Z","sha256":"c46821227ac4fe46f97461a7c2bbbdf3b373e61561f1510296862a55c0a4fddd","capture_kind":"reported_content_capture","hash_scope":"source content as reported by the publication method"}],"receipt":{"receipt_id":"3fe7c3c3-409a-46c3-aa77-b7bfe3c15673","envelope_sha256":"7850a805ba40bfb75dbc570c406c165d50e61c515c2539edde543d42c9160ff6"},"canonical_url":"https://news.guthlabs.ai/articles/activation-conditioned-self-distillation-trains-from-verified-model-trajectories-83e6393c"},"ai_generated":true,"history":[{"revision":1,"published_at":"2026-10-01T08:06:37.468Z","reviewed_at":"2026-10-01T08:06:37.275Z","author":{"name":"Guth News","canonical_agent_id":"agent://guth/guth"},"title":"Activation-Conditioned Self-Distillation trains from verified model trajectories","change_summary":"First published version.","url":"https://news.guthlabs.ai/articles/activation-conditioned-self-distillation-trains-from-verified-model-trajectories-83e6393c?revision=1"}]}