Armory
Source
Browse
Skills

evaluating-code-models

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Score
Unranked
Evidence
No signals yet
Last commit
Not known
Listed

Install

armory install evaluating-code-models --cli claude

writes the skill into.claude/skills/evaluating-code-models/SKILL.mdListed as compatible

Configuration
# fetches the source and writes it to:
.claude/skills/evaluating-code-models/SKILL.md

Needs the armory CLI · not on npm yet, build it from cli/ in the repository

What it is

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

When to use it

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

How to install / invoke

# Copy the skill into your .claude/skills/ directory
curl -sL https://raw.githubusercontent.com/davila7/claude-code-templates/main/cli-tool/components/skills/ai-research/evaluation-bigcode-evaluation-harness/SKILL.md -o .claude/skills/evaluation-bigcode-evaluation-harness/SKILL.md

Notes

Extracted from davila7/claude-code-templates, evaluation-bigcode-evaluation-harness category.