Armory
Source
Browse
Skills

evaluating-llms-harness

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Score
Unranked
Evidence
No signals yet
Last commit
Not known
Listed

Install

armory install evaluating-llms-harness --cli claude

writes the skill into.claude/skills/evaluating-llms-harness/SKILL.mdListed as compatible

Configuration
# fetches the source and writes it to:
.claude/skills/evaluating-llms-harness/SKILL.md

Needs the armory CLI · not on npm yet, build it from cli/ in the repository

What it is

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

When to use it

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

How to install / invoke

# Copy the skill into your .claude/skills/ directory
curl -sL https://raw.githubusercontent.com/davila7/claude-code-templates/main/cli-tool/components/skills/ai-research/evaluation-lm-evaluation-harness/SKILL.md -o .claude/skills/evaluation-lm-evaluation-harness/SKILL.md

Notes

Extracted from davila7/claude-code-templates, evaluation-lm-evaluation-harness category.