knowledge-distillation
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
- Score
- Unranked
- Evidence
- No signals yet
- Last commit
- Not known
- Listed
Install
armory install knowledge-distillation --cli claudewrites the skill into.claude/skills/knowledge-distillation/SKILL.mdListed as compatible
# fetches the source and writes it to:
.claude/skills/knowledge-distillation/SKILL.mdNeeds the armory CLI · not on npm yet, build it from cli/ in the repository
What it is
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
When to use it
Compress large language models using knowledge distillation from teacher to student models. Use when deploying smaller models with retained performance, transferring GPT-4 capabilities to open-source models, or reducing inference costs. Covers temperature scaling, soft targets, reverse KLD, logit distillation, and MiniLLM training strategies.
How to install / invoke
# Copy the skill into your .claude/skills/ directory
curl -sL https://raw.githubusercontent.com/davila7/claude-code-templates/main/cli-tool/components/skills/ai-research/emerging-techniques-knowledge-distillation/SKILL.md -o .claude/skills/emerging-techniques-knowledge-distillation/SKILL.md
Notes
Extracted from davila7/claude-code-templates, emerging-techniques-knowledge-distillation category.